Workshop Engineering and Music

"Human Supervision and Control in Engineering and Music"

Gunnar Johannsen

Human Supervision and Control in Engineering and Music

The Symposium "Human Supervision and Control in Engineering and Music" is an international workshop supported by the VW-Foundation, with a promising transdisciplinary objective. Crossing frontiers in multiple respects, it shall discover connections between relevant part disciplines of the engineering and music sciences, relate different views to each other and, hence, break new ground for multi- and transdisciplinary questions. Visionary examinations of modern technology and their corresponding further developments require human and social related as well as cultural related judgements and design maximes.

As particularly rewarding integration and frontier domains, "Human Supervision and Control" will be analysed as human guidance, monitoring, and control activities in engineering and in music from the viewpoint of different cultural, intellectual, and technological perspectives. Thereby, the terms music and engineering are used for comprising two broad domains of experience and knowledge, namely

(A) several fields of musicology, computer-music, composition, and music performance as well as
(B) several fields of engineering, computer science, psychology, and mathematics.

It is a great challenge to bring these two science domains closer to each other. However, such a concern is complicated in some countries such as Germany by pre-dominantly more traditional structures of disciplines in universities and scientific organisations. Also internationally, no comprehensive initiative has so far been developed for promoting an effective meeting of both domains.

Thus, the motive for this workshop came from the insight and the firm conviction of the initiator, who has knowledge and experience in both domains, that an intensive meeting will trigger considerable impulses for research on both sides. Thereby, the workshop will make excellent contributions to imparting innovative borderline experiences mutually, to scientists of both sides, with which they can kick off and handle transdisciplinary research projects that are required in the future.

After these considerations, the compelling necessity arises for the realisation of this international workshop "Human Supervision and Control in Engineering and Music". An intensive exchange of innovative research ideas and practical experiences shall take place in and between those fields of music and engineering which are dealing with "Human Supervision and Control". Scientists from the different fields of both experience and knowledge domains will explore common theoretical and practical foundations. The transdisciplinary approach of the workshop will thereby go far beyond the already existing multidisciplinarity within both domains (for example in computer music).

At present, contacts between both domains exist only in points and occasionally. A partial recognition of the respective other side is available in some working directions and in some research institutions, mainly through the literature. To some extent, there is a yet quite young atmosphere of departure to new horizons. The curiosity and the willingness for a common scientific discourse is huge on both sides. This has been confirmed by the pre-dominantly enthusiastic reactions of the invited participants who praise the innovation and the fascination of the theme. Accordingly, the main goals of this highly actual workshop are to learn from each other and to create new innovative research persepectives.

"Human Supervision and Control" comprises the human activities of guidance, monitoring and control (manual and supervisory control or sensory motor and cognitive control). It further includes perception, information processing, and action. "Supervisory Control" is the methodically most important subdomain of "Human Supervision and Control". Other subdomains supplement this with respect to different aspects of information transfer, such as "Gestural Control", "Motion and Sound Control", "Information Retrieval", "Virtual Environment", "Performance and Interpretation" as well as "Visual, Auditory, and Haptic Supervision".

The concept of Supervisory Control is regarded as guidance through humans, of computerised technological systems (also briefly called machines) as well as of human cooperative systems, and thereby as a common scientific paradigm for many promising technological developments in the future – not only in industrial process control, in vehicular guidance, in medical engineering, in telerobotics, and in the service sectors but also in broad areas of the music, the film and the television industries. To reflect such developments, to think about a continuing systematisation, and to anticipate methodical visionary innovations within the framework of an extended concept shall be essential concerns of this workshop.

A shortened and slightly modified part from the glossary of the book "Monitoring Behavior and Supervisory Control" edited by Sheridan (MIT, USA) and Johannsen invites to transdisciplinary borderline crossings: "The human supervisor performs upper level goal-oriented functions such as planning system activity, directing intelligent subsystems, monitoring system behaviour, adjusting parameters on-line when appropriate, and intervening in emergency or for normal redirecting or repair."

In times of more and more powerful multimedial and multimodal human-computer interactions in an interconnected information and knowledge society, Supervisory Control gains a new importance covering extensive domains of society, in the sense of responsible integrating leadership behaviour in the approach to humans and machines. Thus, it is fascinating and promising to compare the experiences from the technological-psychological areas of human-machine(-computer) interactions with those from the tradition-rich cultural areas of music which, at the same time, is one of the scientific core disciplines. In the workshop, borderline experiences to be exchanged shall correspondingly clarify questions about common interests and differences with the guidance of vehicles, productions plants, application software systems, orchestras, and computer-music installations.

The views of engineering scientists, psychologists, and computer scientists on "Human Supervision and Control" can stimulate new ideas and solutions, especially also for the domain of music. They have strongly advanced the research and the application of human-machine systems, of automation and systems engineering, and of human-computer interaction. In addition, they yield a far-sighted view on the revolutionary changes in process and vehicular guidance as well as in human-computer interaction of different application fields. For the domain of music, they can provide contributions to theoretical foundations of performance research and of interactive composition as well as to questions of perception, computer-music, and sound design. The spectrum of the scientific approaches comprises

research on guidance of vehicles, industrial production plants, and medical apparatus as well as application software systems which the human as supervisor influences in a goal-directed manner via computer layers,
research on human activities of guidance, monitoring and control (manual and supervisory control),
research on decentral-hierarchical system architectures, guidance and support systems, telemanipulation and telepresence, cooperative multi-agent systems and cognitive systems responibility, as well as
research on graphical, auditory, and multimedial human-machine and human-computer interfaces, also with user modelling and knowledge-based support, and virtual environments.

The views of musicologists and musicians on "Human Supervision and Control" seem particularly rewarding for the borderline experiences to be dealt with in this workshop. They shall stimulate new ideas and visions, also for the domain of engineering. The metaphorical question "Can we conduct a power plant, a truck or an interactive software system?" may serve as an impetus for the transdisciplinary discourse. The spectrum of the scientific approaches comprises

research on and experiences from the areas of music performance and interpretation, including orchestra conducting, orchestra play, and soloistic performances,
research on the mutual influence between composition (design) and interpretation (operation), as well as
research on and experiences in computer music as well as live performances with computers and traditional music instruments, as well as sound design.

The concept of Supervisory Control can be applied to every orchestra under a conductor as Supervisor. The individual orchestra musicians with their respective instruments in the music performance practice correspond in some way to the process-near task-interactive computers, for example in production plants. The orchestra musicians are organised in subgroups (strings, woodwinds, brass, percussion). The conductor as supervisor conveys his or her mental imagination of the complete musical work forehearingly in real-time by gestural expressiveness to the subgroups and to each individual musician of the orchestra. The decentral-hierarchical organisation is completed by the part leaders of the subgroups and the concert master. The team-play can also be regarded as the cooperation of partly autonomous agents under the leadership of the supervisor.

For human-machine(-computer) systems, the question arises how to learn from these organising and cooperative forms of an orchestra as well as from the gestural conveyance of complex mental contents by the conductor. This question shall be investigated in the workshop by means of a transdisciplinary scientific discourse as well as by a scientifically well-founded mediation of the music performance practice. For that purpose, different levels of "Human Supervision and Control" in music shall be made understandable, experiencable, and audible for all participants of the workshop, by means of an orchestra concert, also with insights into the rehearsal practice, with interactive rehearsals and explanations, and with individual expert lectures for each of the performed works.

The program of the orchestra concert has been conceived in accordance with the international and transdisciplinary character of the workshop. It comprises music of the last 200 years from three continents. Particularly, the contrast and the synthesis of Japanese music with European and North American music will be emphasised. Further, the cooperation of orchestra conducting, orchestra play, and soloistic performance will be illustrated. With the planned interactive orchestra rehearsals, the influence of different conductors on the performance result will be demonstrated. Also, the area of computer music with live performance will be represented. Works of Aaron Copland, Toru Takemitsu, Yoichi Nagashima, and Ludwig van Beethoven will be performed.

Composers have proved their musical imaginations at all times against the interpretation possibilities in the music performance practice. Thus, design and operational practice go together as in the technological development. Participative design is likewise striven for in engineering and in music. In the sense of "Human Supervision and Control", both these activities, composition (design) and interpretation (operation), meet more strongly and even merge in the areas of computer music and live performance. Sound design is a related advanced research field which requires artistic and engineering skills, for computer compositions, improvisations, and live performances, as well as for technical applications. Auditory displays with selections or combinations of digital-audio effects, natural sounds, and computer music will more often be needed in the future for human guidance and monitoring in music live performances as well as in multimodal and multimedia process control with all kinds of engineering applications.

About 60 scientists mainly from Europe, North America, and Far East (especially Japan) have been invited to the workshop. Thereby, rising young scientists are also very strongly involved as active participants. The engineering, information and cognition scientists of the experience and knowledge domain (B) represent the fields of human-computer interaction, human-machine-systems research, multimedia-, audio- and music technologies, knowledge-based systems, perception psychology, and cognitive systems science. The musicologists and musicians of the experience and knowledge domain (A) belong to the fields of music theory, composition, media composition, computer music, conducting, and interpretation. Moreover, all invited scientists of this workshop, also the rising young scientists, are outstandingly qualified for the theme of the workshop and, in many cases, have already their own borderline experiences between engineering and music sciences.