International Association for Semiotic Studies
VI International Congress

Guadalajara, México, 13-18 July, 1997

Meaning in Music Gesture

Fernando Iazzetta
Pontifícia Universidade Católica de São Paulo
Communication and Semiotics Department
R. Ministro Godoy, 969 - sala 4B-06
São Paulo - SP - 05015-901 - BRAZIL
Abstract: Gesture is something intrinsic to music, although this question has not yet received the attention it deserves. Music gesture is not only movement, but a movement which is able to mean something, a movement which carries a special signification. It is more than a spatial change, or a body action, or a mechanic alteration: music gesture is a sign that becomes actual through movements. Our main concern is about the significance of the gesture in music cognition. We will be analyzing role played by music gesture in the fields of composition, performance and listening and how the use of electronic and digital technologies has modified the role played by gesture in music.

Gesture is not only physical motion, but attitude (Bertolt Brechet in Henrotte, 1992: 105).

Technique and technology are two cultural aspects which have been deeply involved with music, not only in relation to its production but also in relation to the development of its theory and to the establishment of its cultural role. Since the beginning of the twentieth century the relation between music and technology became more intense due to a series of reasons, among them, the increasing knowledge about sound physics and sound cognition; the access to low cost electricity; and the use of electronic and digital technology to artificially generate and manipulate sounds. Before that, musical sounds were produced only by mechanical means. Although musical instruments, such as the violin, the piano or even the human voice, represented a wide variety of forms and mechanisms, all of them were based on the same principle of sound production, that is, the mechanic vibration of an elastic body. However, the appearance of electrical technologies and the use of electromagnetic signals brought the possibility of generating sounds without using mechanical devices.

Although the sound waves coming from a clarinet or from the electronic oscillators inside a synthesizer have the same nature, their processes of production are quite different. On one side there is the concrete, visible and mechanic universe of the traditional instruments where the body of the instrument and the body and movements of whom is playing that instrument are intrinsically related to the qualities of the sound they are producing. On another side, in the era of electricity and electronics, we start listening more and more to the sounds of invisible bodies contained in the electronic components of synthesizers, samplers, and computers.

For many centuries, people have learned to listen to sounds that had a strict relation to the bodies which produced them. Suddenly, all this listening experience accumulated during the long process of musical evolution is transformed by the appearance of electronic and recorded sounds. When one listens to artificially generated sounds he or she cannot be aware of same type of concrete and mechanic relations provided by traditional acoustic instruments since those artificial sounds are generated by processes that are invisible to our perception. These new sounds are extremely rich but at the same time they are ambiguous for they do not maintain any definite connection with bodies or gestures.

Until the development of sound recording systems early in this century, the contact with music occurred only trough the performance. The listener, even if not involved in the sound production, participated in the music realization by mentally reconstructing the connection between the sound and the physical and cultural context where the music was taking place.

It is worth noting that the concern with gesture in music emerges only when it became possible to record and reproduce music and the role of performance was replaced by a listening situation mediated by new technologies such as the radio, t magnetic tapes, or CDs. And when these reproductive technologies became socially effective, they brought about two different alterations.

The first alteration refers to a shift from the prominence of music production processes (composition and interpretation) to the prominence of listening activities as cultural experience (Mowitt, 1987). Here one can easily note that now the number of people who are listening to music is much larger than the number of people who are doing music. This fact shows more than a statistical aspect, but also reflects a cultural transformation: music is primarily produced to be listened to and not to be played or sung and the process of composition and performance becomes the means to propitiate it.

This projection towards the listener is reinforced by the processes of reproduction that will impose new models of reception. When the phonograph was invented about one hundred years ago, music reception was based on the listening model of that time, that is the listening of a live performance. The main goal of those recording systems was guided by the term fidelity, which in this case would mean that the better a recording could reproduce the sound qualities of a live performance the better it would be considered (Thompson, 1995). But what we experience today is a different situation. The live performance cannot be a listening model anymore since, for most of us, listening to music means listening to the music reproduced by a device such as radio, or CD. Gradually, this new context based on recording and reproduction becomes the model for music reception and music production. The term fidelity which presupposes some relation of similarity is still in use by phonographic and audio industries, but its meaning became reflexive and inappropriate: the fidelity of a recorded sound is not based upon the original sound itself, but it is established as a function of the available recording technology. This leads to a paradoxical situation in which more and more musicians try to reproduce in their live performances the same sound qualities of their records, specially in pop music. In a stimulating essay, John Mowitt observes that: If recording organizes the experience of reception by conditioning its present scale and establishing its qualitative norms for musicians and listeners alike, then the conditions of reception actually precede the moment of production. [T]he social analysis of musical experience has to take account of the radical priority of reception (Mowitt, 1987: 176-77). The second alteration is related to musical gesture. Although music has always been strictly related to gesture, only in the past few decades this issue has deserved some attention by musicologists. Gesture is taken here in a broad sense. It does not mean only movement, but a movement which can express something. Therefore, it is a movement which embodies a special meaning. It is more than a change in space, or a body action, or a mechanic activity: gesture is an expressive movement which becomes actual through temporal and spatial changes. Actions such as turning knobs or pushing levers, are current in today's technology, but they cannot be considered as gestures. Also, to type a few words in a computer's keyboard has nothing to do with gesture since the movement of pressing each key does not convey any special meaning. It does not matter who or what performed that action, neither in which way it was performed: the result is always the same. However, the situation is completely different when a musician plays something in a piano keyboard: the result, that is, the musical performance, depends in many different ways on the player's gesture.

Gesture in music performs a fundamental role in meaning generation. In a certain way, we have learned to understand musical sounds with the aid of gestures that produce and represent those sounds. As G. Kurtenbach and E. Hulteen have noted, the function of gesture in the music is proportional to its power to express something: Gestures increase function by virtue of their expressiveness. That is, a gesture may control multiple parameters at the same time, thus allowing a user to manipulate data in a manner not possible by modifying each parameter individually. For example, a conductor simultaneously controls both tempo and volume of the music gesture. The rhythm of the gesture controls tempo and the size of the gesture controls volume. This allows an efficient communication not possible by adjusting the tempo and volume independently (Kurtenbach & Hulteen, 1990: 311-12). At this point, we should make a distinction between physical gesture and mental gesture (Zagonel, 1992). Physical gesture refers to the production of sound as a physical phenomenon, retaining a causal relation between the gesture and the sounds that are produced. Also, we can talk about body gesture which is a physical gesture that does not produce, but accompany the sound.

Another gestural category, which is closely related to the processes of composition, interpretation and listening, is the category of the mental gestures. Mental gestures refer to physical gestures and their causal relationship and they occur as an idea or an image of another gesture. Thus, the composer, often starts from an idea or a mental image of sound gesture to compose a vocal or instrumental gesture. The mental gesture is learned through the experience and stored into the memory to be used as a model in composition. At the same time that a mental gesture refers to the corporal actions of a performer or to the behavior of a musical instrument, it can also refer to a particular sound structure. Thus, an arpeggio can be understood as a gestural movement from one point to another in the pitch space and the movement of a conductor's hand can be seen as a gesture that unifies the temporal and articulatory behavior of the orchestra. What we should emphasize is that the mental gesture always refers to a physical gesture which has been previously learned.

Concerning music, one can say that the physical gesture is directly related to music interpretation while the composition is much closer to mental gesture: As Bernadete Zagonel has said, "if the composer goes from gesture to the composition, the performer goes the opposite way, that means, he goes from the score to the gesture" (Zagonel, 1992: 17-18). To this statement we can add that the listener completes this chain by mentally recreating the performer's physical gestures while listening to music.

The lack of musicological studies taking body gesture in account is in part due to the fact that the body has never been considered as a support for musical expression. It was only when electric and electronic means of musical production and diffusion eliminated the presence of the musician and his instrument that the importance of the body's presence in musical production starts to deserve attention. The symbiotic relation between the player's body and his instrument plays a special role in the comprehension of the musical discourse. For example, a violent gesture produced by the player reinforces the effect of a sudden sound attack in the same way that the body expression of a singer can lead to a richer phrase articulation.

The musical background and the daily experience with the objects which surround us help to establish a connection between physical materials and types of gestures that will produce specific sounds with those materials. Through our experience we can establish patterns and links between sonic events and the materials which could produce these sounds. Of course, the perception of these relations is not precise in specific terms (for example, it is hard to determine how loud is the sound of this book falling on the floor), but this perception is extremely sophisticated in relative terms (for example, one can easily say that rolling a pencil over a wood table produces less sound that hitting the table with that pencil). Moreover, part of music's expressive power is given by this proportional scale through which one can mentally build relations among materials, gestures and sounds. In fact, we don't know among the objects which surround us any which would lead to a sound performance similar to musical instruments. Our perception tells us that an ordinary piece of wood, about 50 centimeters long would never produce a sound with the same quality, power, brightness and regularity as on can produce on a violin, for example. Also, ordinary objects usually don't allow a precise control of the sound produced by a specific gesture. Thus, musical instruments subvert our expectations about the sounds that such devices would produce. What a musical instrument allows to do in terms of sound goes far beyond what our common sense would expect from any other ordinary object. In this way, the musical instrument is not only the medium of the musical idea, but also part of this idea.

Obviously, each instrument offers a different level of control and interaction. For example, the mechanism of a pipe organ works almost automatically in a way that the performer's gesture has very little control over the process of sound generation. On another hand, instruments such as the violin, the trumpet or some types of drums allow the control of very subtle characteristics of the sound through the interaction between the player's gestures and the body of the instrument. As Richard Moore points out, With such instruments the microgestural movements of the performer's body are translated into sound in ways that allow the performer to evoke a wide range of affective quality in the musical sound. That is simultaneously what makes such devices good musical instruments, what make them extremely difficult to play well, and what makes overcoming that difficulty well worthwhile to both performer and listener (Moore, 1987: 258). In a book called "The Sight of Sound: Music, Representation, and the History of the Body" Richard Leppert calls the attention to the importance of the gestural images which are produced during a musical performance: Precisely because musical sound is abstract, intangible, and ethereal [...] the visual experience of its production is crucial to both musicians and audience alike for locating and communicating the place of music and musical sound within society and culture. [...] Music, despite its phenomenological sonoric ethereality, is an embodied practice, like dance and theater (Leppert, 1993: xx-xxi). The manipulation of tape recorders, synthesizers, and computers eliminates the materiality of sound. Electronic sounds do not embody any kind of gestural relation to the devices that produce them. It represents a great autonomy and freedom in terms of creating new sounds, but at the same time it means a loss in the symbolic and meaningful dimensions that can be present in a musical work. In the last 50 years electronic and digital technology expanded the possibilities of creating music in a very significant way, but this expansion also brought the risk of a loss in the semiotic connections conceived during the performance. Today's musical activities based on the production of pre-recorded or electronic generated music have deviated from the embodied practice of performance to artificial processes of composition and diffusion. In those processes the loudspeaker replaces the performer, thus eliminating visual and gestural references which traditionally composed the symbolic dimension of musical language. In which way contemporary musical practices such as electroacoustic music will restore this symbolic dimension is something that still has to be elaborated and can be seen as a challenge that will entertain, for some time, composers, performers and listeners.