Workshop Engineering and Music

"Human Supervision and Control in Engineering and Music"

M. M. (René) van Paassen

From the Neuromuscular System to Supervisory Control

Abstract

This paper presents two examples, one in the field of manual control, and one in the field of supervisory control, where velocity feedback - although on a different scale, and described in different terms - helps the human operator.

Introduction - manual control

In manual control of relatively fast continuous systems, for example when driving a car, riding a bike or flying an aeroplane, the properties felt in the manipulator (steering wheel, handlebar or yoke/stick) can play an important role in the handling of the system. The relationship between the human operator, the manipulator and the controlled system can be classified into three different categories (see Figure 1):

The dynamics of the manipulator and the system are de-coupled. An example of this is the stick in the Airbus family of aircraft (A320, A330, A340). The stick is a so-called "passive" device, which has a spring and a damper loading to give it the appropriate mechanical properties. The communication between stick and aircraf control computers is one-way, the position of the stick is measured and fed to the aircraft.

The dynamics of the manipulator and the system are coupled, with indirect feedback of the system state on the manipulator. An example of this is the control yoke in small conventional aircraft. The aerodynamic forces acting on the control surfaces of the aircraft are passed on - through the mechanical linkage, to the manipulator. Such forces are in first approximation equivalent to the excursion of the control surface, multiplied by the "dynamic pressure" ¹/₂ pV². The effect is that when flying fast, the controls feel stiffer. A similar effect happens in car driving.

Note that it is also possible that this indirect feedback takes place via other means than direct mechanical linking. An example is the Boeing 777, where a hydraulic system simulates the "feel" of the aerodynamic forces on the control surfaces.

The dynamics of the manipulator are coupled to those of the system, with direct feedback of the system state on the manipulator. The difference with the previous case is that the manipulator has no (significant) proper dynamics. An example of this is the concept of "active side stick" investigated at the Delft University of Technology (Hosman, 1998). In this active side stick, the force exerted on the stick is used as the input to the aircraft, and outputs of the aircraft, typically roll rate and pitch rate, are used to command the stick position.

This latter option, the "active side stick" in many cases resulted in much improved pilot response in the laboratory environment (simular to the set-up shown in Figure 2) in which it was tested. The search for an explanation of the - sometimes surprising - pilot performance with this side stick (Hosman 1998) initiated the author's PhD research on the role of the neuromuscular system in manual control of aircraft (Paassen, 1994).

Three levels of integration of manipulator dynamics and system dynamics; (a) de-coupled, (b) indirect feedback of the system state to the manipulator and (c) direct feedback of a system output to the manipulator.

Figure 2: Human-machine systems laboratory at the Delft University of Technology, featuring a side stick with electro-hydraulic control loading

Figure 2: Human-machine systems laboratory at the Delft University of Technology, featuring a side stick with electro-hydraulic control loading.

Introduction - supervisory control

In the control of slower systems, and in this case a cement mill is used as an example, the role of the manipulator is less prominent. Control inputs can be given with a mouse and slider on a graphical interface, hardware controls or by keyboard input, but since the controlled system's dynamics are slow compared to these input actions, the actual implementation of the input device and its properties have no influence on the system's response.

Work on such systems is characterised by the operator taking control actions, and subsequently waiting for the system to respond. A typical frequency of control action is 5 per hour (Hollnagel, 1998). This mode of control is further promoted by the fact that large plants are provided with automated controllers for parts of the process. The operator assumes a role of supervisor, giving set-points to the automated controller and monitoring their work.

Sheridan (1987) gives an overview and definitions of supervisory control. The strict definition, according to Sheridan, describes a situation in which the operator gives setpoints to an automated controller, and monitors the supervision of the process. A second definition, which to the author appears a more appropriate, is supervisory control in the broad sense. In this definition, encompasses also the situation where an automated system interprets measured data from the plant, to provide "reports" on the plant's functioning.

This broader definition of supervisory control also applies to the second example considered here, the control of a cement mill, subject of study during the author's stay as a postdoc at the University of Kassel (Paassen, 1995b, 1997). An overview of this system is given in Figure 3. Central component in cement milling is a rotating mill, filled with steel balls or pellets, which, by crushing clinker between the falling and rolling steel balls, grinds clinker to cement. Such a mill is normally operated in a closed loop circuit, with a separating unit (SEPAX), which separates fine cement from the coarser output, with the course output being fed back into the mill. Input feed, with conveyor belts, digging machines and a pre-crusher, an output line, with a pump and storage silos, and a cooling unit, which cools the mill by spraying water in the mill, complete the system.

Overview of the simulation of the cement mill

An experimental interface was developed for (a simulation of) this system. Goal in the development of the interface was to see whether information about goal fullfillment would aid operators in their task. The interface was based on a mix of two techniques, Multilevel Flow Modelling (Lind, 1990), and Ecological Interface Design (Vicente & Rasmussen, 1990, 1992; Bisantz & Vicente, 1994).

Multilevel Flow Modelling is a functional modelling technique that seeks to express a system in terms of the goals that have to be achieved, and the functions available to achieve these goals. As such, it is not intended for interface design, but the author used a dynamic representation of the MFM model for the process interface. Ecological Interface Design starts with a mapping of the process onto Rasmussen's Abstraction Hierarchy (Rasmussen, 1986; Bisantz & Vicente, 1994). This hierarchy shows the goals to be achieved (functional purpose), and functions to achieve these goals at various levels of abstraction. In this case the goal and function analysis of MFM was used as a basis for the EID design. Such an interface should show the to control the system ("affordances" in the parlance of Gibson (1979)), and the relationship between these possibilities and the achievement of the goals. This in contrast to conventional interfaces, that usually show little more than the controls and the measured values. The inverse of goal achievement, i.e. goal violation, is normally show through means of alarms (Paassen & Wieringa, 1997).

Feedback

A human (or automatic) controller displays goal-directed behaviour, which means that he will try to control the system so that his objectives will be achieved. Now, both a cement mill and an aircraft are dynamical systems, which means that their output and state will not change instantly, but gradually, over time. In order to support this goal-oriented behaviour, the operator needs to know:

What the state of the system is in relation to the desired objective, inother words, is the goal achieved?

What is, for a dynamical system, the trend in goal achievement, or even what the higher order derivatives from this trend are. In other words, if the input is not changed, will the goal be achieved inthe future or not.

With this in mind, one can evaluate what contribution the three types of manipulator - system coupling provide to the human operator. If the dynamics of the manipulator and the controlled system are de-coupled, then the dynamics of the manipulator can at most provide static information about the controlled system. For example, the "feel" of an Airbus side stick is heavier than that of a PlayStation joystick, simply to warn operators that the input given to the aircraft has a more serious effect than that given to your game.

An indirectly coupled manipulator, a conventional yoke in an aircraft, provides a stiffer feel at higher speeds. Automatically, the operator will be inclined to give smaller inputs (smaller in excursion, but similar in size) at high speeds, leading to forces and moments on the aircraft that are approximately the same at low and high speeds. At low speeds, the manipulator will feel very light, warning the pilot of the danger of stall. The feedback is therefore helping the pilot achieve safety.

The active side stick can feed back aircraft roll and pitch rates. Therefore the stick position indicate the first derivative of either the goal (if trying to fly with a certain pitch and roll attitude), or of a sub-goal, thus a contribution to performance, instead of to safety, as with an indirectly coupled manipulator.

A passive manipulator, with the dynamics uncoupled from the controlled system dynamics, has neither the feedback on the safety of the flight, nor feedback that aids in achieving higher performance. It would be a bad choice for application in an aircraft that is otherwise controlled in a conventional manner, and indeed, it is not intended as a controller for conventional aircraft dynamics. The side sticks as used in Airbus aircraft provide input to a closed-loop flight control system. The stick is thus used as a setpoint device, and the flight control system closes the loop that implements the setpoint. In doing so, the controller makes the aircraft dynamics, seen from the setpoint
input to the attitude response, look like a single integrator. In addition to this, the flight control system implements protection of the aircraft envelope; it is - with a fully operational flight control system - not possible to stall the aircraft or to achieve an excessive load factor. Thus part of the responsability for achieving safety goals, and part of achieving production goals is shifted to the flight control computer.

In manual control of fast reacting systems, use of the manipulator as a feedback device, in either of the two of the three ways discussed above, helps pilot performance by providing feedback through the neuromuscular system. In the control of a slower system, such as a cement mill, feedback through manipulators is not usually considered important, because of the slower speed of such systems, there is ample time to observe such feedback from visual displays.

It is however, as with the faster systems, important to be able to observe not only goal achievement, but also the development towards - or from - goal achievement. The problems are different:

Systems are often so slow, that correlation between the inputs given and the system response becomes problematic (Kok and Stassen, 1980). Fast-time predictions of system response are sometimes helpful in such a case.

System dynamics are typically of higher order, making it impossible to compensate for unfavourable dynamics. After giving an input, operators simply have to wait for the transient response of the system to subside, and see what the new final state is.

EID interface for the cement mill simulation.

In the cement mill, one of the problems is that the milling process itself is not measured directly. One can observe power consumption, fill level in the mill (only approximately), the feed of the mill and the output flow of the mill. The feed can be controlled, and the output can be controlled a little, one can choose to accept a coarser product and thereby empty the mill faster. However, how finely the material in the mill has been ground is unknown. It is largely the fineness of the material in the mill that determines the outflow. At a
start-up, the mill is filled close to its optimim level, then the inflow is reduced a little, and just when the material is fine enough, and the outflow is picking up, one should increase the inflow to get a nice steady production

An experimental interface, based on Ecological Interface Design, was developed for the cement mill (Figure 4). This interface supports the operator in seeing the development of the achievement of the production goal.

The following elements are present:

In the center, four goals that have to be achieved with the system are displayed. The status of each of these goals is given.

The graph at the bottom of the display, in the center, gives the mass inventory of the mill. It shows feed, outflow and recycled material on the bars of the left and right side, the bottom axis shows the fill level.

The graph at the top shows an approximated milling efficiency curve. Milling efficiency depends on the fill level of the mill. The power put into the mill by the engine is shown at the top axis, and by "reading off" the graph, effective milling power is calculated, show along the right axis.

The graph in the top left position shows the energy balance. Incoming energy is shown on the left axis, outflowing energy (mainly flowing out with the airflow and by additional water cooling) on the right axis.

The graph in the top right position shows the energy balance, as determined from the milling efficiency graph. This shows the energy that has been used effectively for milling, and is thus an indication for the amount of milling that the material in the mill has received.

The left graph in the middle rows divides (thermal) energy by the mass in the mill, to obtain the temperature.

The left graph in the middle rows divides the milling energy by the mass in the mill, to obtain the specific milling energy. For this process, approximately 15 kWh per metric ton of material is needed.

The dotted boxes placed around the EID graphs show the controls, and
possibly sub-goals (clicking these opens other windows) that are needed for
these controls.

The interface helps the operator achieve the optimum fill level. The
slope of the mass balance provides rate information, so also here the
trend in goal achievement is shown. The process dynamics are complex,
normally a process simulation for a cement mill is of very high
order. However, to give the operator an impression of the state of the
milled material, a low-order approximation is given with the
efficiency graph, the milling energy inventory and the specific
milling energy. This enables one to anticipate the point where the
mill starts producing, and, although not exactly giving its
derivative, it helps in seeing the development of the output flow. The
specific energy also allows for a quick estimate of the quality of the
material.

Lessons learned

Classic control theory, and its application to human control
behaviour, shows that systems with the dynamics of a single integrator
are most easily controlled (McRuer, 1967). For higher-order systems,
such as the altitude control of an aircraft, human control is possible
when the system can be observed and controlled as a sequence of single
integrators. When viewing this from a means and ends perspective, one
may say that the goal or end is the desired value of the controlled
variable, and the function or means to achieve this goal is the first
derivative of this variable, which may (either directly or indirectly,
through the control of another dynamic sub-system) be controlled. This
would suggest a very simple checklist for evaluating any set of
controls and interface for a dynamic system:

Is it easy to read off the state of the goal(s) that you want to achieve?

If the goal achievement is not an immediate consequence of the control input, is the trend in goal achievement / means for goal achievement, visible/detectable in the interface or controls?

If the means for goal achievement is not an immediate consequence of the control input, go to 1 and repeat.

Conclusions

Neither the active side stick, nor the functional interface for the cement mill have found application outside laboratory environments. However, the research efforts are not lost. The questions raised by the active side stick's extraordinary performance triggered research to find out more about the interaction between a pilot and the manipulator (Paassen, 1994, 1995a), which in turn triggerred the development of measurement techniques for measuring human operator's reponses in a multi-loop feedback systems (Paassen & Mulder, 1998; Mulder, 1999). The MFM/EID interface took some shortcuts in the functional modelling, for which correct alternatives are given in (Paassen & Wieringa, 1999). Continued work in this field looks at EID interfaces for certain aspects of aircraft control (Coelho, Paassen, Mulder & Mulder, 2000; Paassen, Coelho & Mulder, 2001). The main conclusion is that manual control of fast systems and supervisory control of slower systems do have a surprising lot in common, although one sometimes needs some reflection to see this.

References

Bisantz, A. M., & Vicente, K. J. (1994). Making the abstraction hierarchy concrete. Int. J. Human-Computer Studies, 40, 83-117.

Coelho, P. N. P., Paassen, M. M. van, Mulder, M., & Mulder, J. A. (2000). Total energy concept display. In D. de Waard, C. Weikert, J. Hoonhout, & J. Ramaekers (Eds.), Human-systems interaction: Education, research and application in the 21st century (pp. 185-188). Maastricht, The Netherlands: Shaker Publishing.

Gibson, J. J. (1979). The ecological approach to visual perception. Boston: Houghton-Mifflin.

Hollnagel, E. (1998). Cognitive reliability and error analysis method (CREAM). Elsevier Science.

Hosman, R. J. A. W., & Vaart, J. C. van der. (1988). Active and passive side stick controllers: Tracking task performance and pilot control
behaviour. In Agard conference proceedings 425 (pp. 26-1/26-11).

Kok, J. J., & Stassen, H. G. (1980). Human operator control of slowly responding systems: Supervisory control. Journal of Cybernetics and
Information Science. Special Issue on Man-Machine Systems, 3 (1-4).

Lind, M. (1990). Representing goals and functions of complex systems - an introduction to Multilevel Flow Modelling (Tech. Rep. No. 90-D-381).
Institute of Automatic Control Systems, Technical University of Denmark.

McRuer, D. T., & Jex, H. R. (1967). A review of quasi-linear pilot models. IEEE Transactions on Human Factors in Electronics, 8, 231-249.

Mulder, M. (1999). Cybernetics of a Tunnel-in-the-Sky Display. Ph.D dissertation, Faculty of Aerospace Engineering, Delft University of Technology, to be published.

Paassen, M. M. van. (1994). Biophysics in aircraft control - a model of the neuromuscular system of the pilot's arm. Unpublished doctoral dissertation, Faculty of Aerospace Engineering, Delft University of Technology, Delft.

Paassen, M. M. van. (1995a). A model of the arm's neuromuscular system for manual control. In Proceedings of the 6th ifac symposium on analysis, design and evaluation of man-machine systems. Boston.

Paassen, M. M. van. (1995b). New visualisation techniques for industrial process control. In Proceedings of the 6th ifac symposium on analysis, design and evaluation of man-machine systems. Boston.

Paassen, M. M. van. (1997). Process displays based on Mulitlevel Flow Modelling MFM and Ecological Interface Design EID (Tech. Rep. No. IMAT-MMS-16). Kassel: University of Kassel, IMAT - Human-Machine Systems.

Paassen, M. M. van, & Mulder, M. (1998). Identification of human operator control behaviour in multiple-loop tracking tasks. In 7th ifac symposium on analysis, design and evaluation of man-machine systems.

Paassen, M. M. van, Coelho, P. N. P., & Mulder, M. (2001). Support interfacefor continuous descent approaches. In Ifac-mms-2001.

Paassen, M. M. van, & Wieringa, P. A. (1997). Alarms, alerts and annunciations. Journal A, 38 (4), 16-22.

Paassen, M. M. van, & Wieringa, P. A. (1999). Reasoning with multilevel flow models. Reliability Engineering and Systems Safety, 64, 151-165.

Rasmussen, J. (1986). Information processing and Human-Machine
Interaction - An approach to cognitive engineering. New York: North Holland.

Sheridan, T. B. (1987). Supervisory control. In G. Salvendy (Ed.), Handbook of human factors (pp. 1243 - 1268). John Wiley and Sons.

Vicente, K. J., & Rasmussen, J. (1990). The ecology of human-machine systems ii: Mediating "direct perception" in complex work domains.
Ecological Psychology, 2 (10), 207-249

Vicente, K. J., & Rasmussen, J. (1992). Ecological interface design: Theoretical foundation. IEEE Transactions on Systems, Man and Cybernetics, 22 (4), 589-606.