|"Human Supervision and Control in Engineering and Music"|
|Workshop||Orchestra Concert||Ensemble Concert||About us|
Applications of Spatial Auditory Displays in the Context of
Art and Music
AbstractAuditory displays with the ability to place virtual sound sources into the space under realtime conditions enable advanced applications for art and music. The
1 IntroductionThe space and the audience have an influence on the creation of music. Composers have written pieces, targeting a specific performance situation. While recordings allow a larger audience to appreciate certain music.
2 Using spatial arranged musicalComposers take advantage of spatial imagery and try to immerse the audience. In the following paragraphs, we give some well-known examples.
elements in composition
In 1829-1830, Hector Berlioz wrote the Symphonie Fantastique [Ber30] in which spatial arrangements are an important part of the music. In the “Scene in the country” two shepherds piping in the distance to each other. Here players of the pipes are separated from the main orchestra, creating a wonderful spacious imagery [NM80].
Karlheinz Stockhausen [Sto56] used in his “Gesang der J¨ unglinge” a multi channel recording and playback system, creating spatial movements in his composition.
Andrew Lloyd Webber’s musical cats [Web81] plays at special arranged theaters (e.g., Hamburg) in which the stage is around the audience and performers move freely close and behind the listener.
While a listener has individual experiences, his in-teraction or participating in the performance is limited. Art installations, which are reviewed in the next section go beyond those restrictions.
A much better review of the history of music space and reverberation
in general is given by Barry Blesser [Ble01].
3 Art installationsVirtual environments are influencing real art [Dol97] [Bro97] and new technology is used for various instal-lations.
Feedback and complete isolation are explored in the installation of
Seiko Mikami [Mik97]. The heartbeat of the visitor is spatially
and visualized. It takes place in an anechoic chamber providing total
The sound wanders around the listener and depends on his reaction.
Distance plays an important role in the installation of Akitsugu Maebayashi [Mae97].
Distance between people is visualized and auralized via a personal spatial auditory display. Participants of the installation wander around in a virtual (controlled) environment, consisting of a HMD, location tracking, and headphone.
Concrete location of sound is not always desired. In the interactive
installation of Yunchul Kim [Yun01] the sound moves and is influenced
movements (aka. position), but the sound seems not come from a specific location instead an omnipresent floating soundscape is created. A video-based person tracking controls a realtime composition using self-organizing maps.
A helical keyboard [HC96] can visualize and auralize the cyclical
of octaves and helical structure of a scale, as described by [She82].
system takes as input a Midi stream, either preproduced as Midi file or
live performed. The notes are visualized in space by movement and
Each note is a distinguished sound source, allowing to differentiate a
note also spatially.
A listener can fork his listening presence in the virtual environment as shown in Figure 1. This al-lows to follow harmony and melody while keeping the
source direction differentiable.
4 Spatial emancipation of the audience
With spatial auditory displays allowing realtime com-position of the
experience, the audience can emancipate itself spatially from the
Virtual concerts are possible with the multiple audio windows [CK95] [Coh94], in which the listener can rearrange the (virtual) sound sources and sinks, creating an imagery of his own choice.
Figure 1: Exocentric visual perspective of forked presence
via multiple sinks in the Helical Keyboard
Virtual concerts are also presented in virtual environments [CK99]
also the listener to take active part in the concert by substituting
Creating content for such systems requires that all voices be recorded separated and anechoic. In [Kau99] a violin concert was recorded in an anechoic chamber. Each player was recorded separately while listening via headphone to the other voices for synchronization and harmonic play.
With numerous tracks available on DVD and tagging of voices, and
spatial auditory display becoming available, vitual concerts can become
common listening experience [Her00].
References[Ber30] Hector Berlioz, Symphonique fantastique, 1830.
[Ble01] Barry Blesser, An interdisciplinary synthesis of reverberation viewpoints, in press (2001).
[Bro97] Sheldon Brown, Real Art and Virtual Reality, Computer Graphics 31 (1997), no. 4, 36–39, FO-CUS: The State of Fine Art.
[CK95] Michael Cohen and Nobuo Koizumi, Audio Windows for Virtual
II: Sonic Cubism, Video Proc. ICAT/VRST: Int. Conf. Artificial
Reality and Tele-Existence/Conf. on Virtual Reality Software and Technology (Makuhari, Chiba; Japan) (Susumu Tachi, ed.), ACM-SIGCHI (TBD), SICE (Society of Instrument and Control Engineers), JTTAS (Japan Technology Transfer Association), and NIKKEI (Nihon Keizai Shim-bun, Inc.), November 1995, p. 254.
[CK99] Hartmut Chodura and Arnold Kaup, A Virtual Environment for
Music Reproduc-tion, Proceedings of IFIP TC/WG5.10 and CSI
International Conference on Visual Computing 1999 (S.P. Mudur, D. Shikhare, J.E. Encar-nacao, and J. Rossignac, eds.), International Federation for Information Processing (IFIP), February 1999,
http://www.igd.fhg.de/igd-a9/research/audio/index.html ,pp. 95–100.
[Coh94] Michael Cohen, Conferences, concerts, and cocktail parties: Besides immersion, JMACS: Proc. Japan Music And Computer Science Society Meeting (Musashino, Tokyo), February 1994, pp. 17–26.
[Dol97] Margaret Dolinsky, Creating Art Through Virtual
Computer Graphics 31 (1997), no. 4, 34–35, FOCUS: The State of Fine
[HC96] Jens Herder and Michael Cohen, Design of a Helical Keyboard, ICAD’96 — Int. Conf. on Auditory Display (Palo Alto, CA; USA) (Steven P.
Frysinger and Gregory Kramer, eds.), November 1996.
[Her00] Jens Herder, Interactive Sound Spatialization - a Primer, MM News, University of Aizu Multimedia Center 8 (2000), 8–12, (Japanese).
[Kau99] Arnold Kaup, 3D Musikproduktion für Virtual Reality Anwendungen, Master’s thesis, Fachhochschule / Robert-Schumann Hochschule, Düsseldorf, 1999.
[Mae97] Akitsugu Maebayashi, Audible distance, Installation at the
InterCommunication Center (ICC), Hatsudai, Tokyo, 1997,
[Mik97] Seiko Mikami, World, membrane and the dismembered body, Installation at the NTT InterCommunication Center (ICC), Hatsudai, Tokyo, 1997, http://www.ntticc.or.jp/Collection/Icc/World/index.html.
[NM80] New York Philharmonic Orchestra and Zubin Mehta, Hector
Symphonie fantastique, op. 14, CD, The Decca Record Co. LTD., 1980,
[She82] Roger N. Shepard, Structural Representations of Musical
The Psychology of Music (Diana Deutsch, ed.), Academic Press, December
ISBN 0-122-13560-1, pp. 343–390.
[Sto56] Karlheinz Stockhausen, Gesang der Jünglinge (song of the youths), 1955/56, http://www.stockhausen.org/ tape loops.html.
[Web81] Andrew Lloyd Webbers, Cats, Musical, 1981,
[Yun01] Kim Yunchul, Implex, Installation at Altitude 2001, Academy of Media Arts, Cologne, Jul 2001.