Workshop Engineering and Music

"Human Supervision and Control in Engineering and Music"

Stefan Göller

Visualization and Manipulation of musical content

Abstract

While the growth of knowledge in natural sciences reaches a frightening degree, the humanities - in particular musicology - hover around some stagnation value. One of the reasons for this stagnation might be their insistance on traditional text-oriented working styles.

We are presently developing systems for visualizing and manipulating complex data to give musicologists more adequate tools.

Problem

The evaluation of the humanities in Switzerland (Bern 1997) showed structural
lacks as well as unsatisfactory integration into contemporary information and
communication technology, and there is no sign that this is just a Swiss problem.

Especially in musicology, the reason is quite obvious: The complexity of the objects of interest - scores, structure of sounds, historical connections - is certainly at least comparable with those structures natural scientists - say physicists - are dealing with, but the musicologists' working style is still the same since the middle ages - a traditional text-oriented reading/thinking/writing style without any use of contemporary maths/computers/etc.

This is especially painful as every common musical score contains a higher number of symbols than most mathematical proofs or physical theorems. Moreover, most musical scores are attached to a deep tree of historical dependencies - another bit of complexity no mathematician or physicist needs to bother with.

Data structure

The first thing, one needs to address in this problem, is an adequate data structure.Existing music formats like midi, darms, etc., are too poor to describe thosescores, sounds, or historical dependencies.

Mazzola showed that a topos theoretic concept called "denotator" was flexible enough to represent all the structures in question.

Moreover, as present, database languages as well as the current hypertext-markup-languages are relatively inflexible towards dynamic type-extension requirements and are not designed for general knowledge-space representation and navigation. Therefore, the denoteX-language has been defined in 1999/2000 by Mazzola, Göller (UniZH), and Noll (TU-Berlin) in order to overcome these drawbacks by use of the aristotelian form/substance dichotomy in its restatement as a space/point dichotomy.

Tools

As denoteX is built upon abstract mathematical space constructs like limits, colimits, and power objects, it is necessary to enable navigation in these denoteX-spanned knowledge-spaces in a visually understandable manner.

The first part of my PhD thesis was to implement such a denoteX-browser using the Java3D API. This browser visualizes any complex data structures that can be described with denotators. The second part is to extend this browser to a composition tool (called PrestoRubette) that can actually manipulate denoteX files in order to compose music in a geometric, algebraic, and logical way, and handles all musical data like collections of scores together with their historical connections.

Moreover, this 3D-Browser is the base platform of many subtools that can
be controlled from there by remote method invocation, so that a researcher
has a powerful tool that enables him to handle all his data in collaboration
with other researchers, while the software runs dislocated on a network of
connected servers.

References

Goldblatt, R. (1979). Topoi: the categorial analysis of logic, NHPC,
New York.

Mazzola, G. (1990). Geometrie der Töne, Birkhäuser, Basel.

Mazzola, G. (2002). Topos of Music, Birkhäuser, Basel.

SWR (Hrg.). Evaluation der geisteswissenschaftlichen Forschung in der Schweiz -- Zusammenfassender Bericht der Schlusskonferenz vom 15. Februar 1997, FOP 42/1997, Bern 1997.