International Association for Semiotic Studies
VI International Congress
Guadalajara, México, 13-18 July, 1997
Meaning in Music Gesture
Pontifícia Universidade Católica de São Paulo
Communication and Semiotics Department
R. Ministro Godoy, 969 - sala 4B-06
São Paulo - SP - 05015-901 - BRAZIL
Abstract: Gesture is something intrinsic to music, although
this question has not yet received the attention it deserves. Music gesture
is not only movement, but a movement which is able to mean something, a
movement which carries a special signification. It is more than a spatial
change, or a body action, or a mechanic alteration: music gesture is a sign
that becomes actual through movements. Our main concern is about the
of the gesture in music cognition. We will be analyzing role played by music
gesture in the fields of composition, performance and listening and how
the use of electronic and digital technologies has modified the role played
by gesture in music.
Gesture is not only physical motion, but attitude (Bertolt
Brechet in Henrotte, 1992: 105).
Technique and technology are two cultural aspects which have been deeply
involved with music, not only in relation to its production but also in
relation to the development of its theory and to the establishment of its
cultural role. Since the beginning of the twentieth century the relation
between music and technology became more intense due to a series of reasons,
among them, the increasing knowledge about sound physics and sound cognition;
the access to low cost electricity; and the use of electronic and digital
technology to artificially generate and manipulate sounds. Before that,
musical sounds were produced only by mechanical means. Although musical
instruments, such as the violin, the piano or even the human voice, represented
a wide variety of forms and mechanisms, all of them were based on the same
principle of sound production, that is, the mechanic vibration of an elastic
body. However, the appearance of electrical technologies and the use of
electromagnetic signals brought the possibility of generating sounds without
using mechanical devices.
Although the sound waves coming from a clarinet or from the electronic
oscillators inside a synthesizer have the same nature, their processes of
production are quite different. On one side there is the concrete, visible
and mechanic universe of the traditional instruments where the body of the
instrument and the body and movements of whom is playing that instrument
are intrinsically related to the qualities of the sound they are producing.
On another side, in the era of electricity and electronics, we start listening
more and more to the sounds of invisible bodies contained in the electronic
components of synthesizers, samplers, and computers.
For many centuries, people have learned to listen to sounds that had a
strict relation to the bodies which produced them. Suddenly, all this listening
experience accumulated during the long process of musical evolution is
by the appearance of electronic and recorded sounds. When one listens to
artificially generated sounds he or she cannot be aware of same type of
concrete and mechanic relations provided by traditional acoustic instruments
since those artificial sounds are generated by processes that are invisible
to our perception. These new sounds are extremely rich but at the same time
they are ambiguous for they do not maintain any definite connection with
bodies or gestures.
Until the development of sound recording systems early in this century,
the contact with music occurred only trough the performance. The listener,
even if not involved in the sound production, participated in the music
realization by mentally reconstructing the connection between the sound
and the physical and cultural context where the music was taking place.
It is worth noting that the concern with gesture in music emerges only
when it became possible to record and reproduce music and the role of
was replaced by a listening situation mediated by new technologies such
as the radio, t magnetic tapes, or CDs. And when these reproductive
became socially effective, they brought about two different alterations.
The first alteration refers to a shift from the prominence of music production
processes (composition and interpretation) to the prominence of listening
activities as cultural experience (Mowitt, 1987). Here one can easily note
that now the number of people who are listening to music is much larger
than the number of people who are doing music. This fact shows more than
a statistical aspect, but also reflects a cultural transformation: music
is primarily produced to be listened to and not to be played or sung and
the process of composition and performance becomes the means to propitiate
This projection towards the listener is reinforced by the processes of
reproduction that will impose new models of reception. When the phonograph
was invented about one hundred years ago, music reception was based on the
listening model of that time, that is the listening of a live performance.
The main goal of those recording systems was guided by the term fidelity,
which in this case would mean that the better a recording could reproduce
the sound qualities of a live performance the better it would be considered
(Thompson, 1995). But what we experience today is a different situation.
The live performance cannot be a listening model anymore since, for most
of us, listening to music means listening to the music reproduced by a device
such as radio, or CD. Gradually, this new context based on recording and
reproduction becomes the model for music reception and music production.
The term fidelity which presupposes some relation of similarity is still
in use by phonographic and audio industries, but its meaning became reflexive
and inappropriate: the fidelity of a recorded sound is not based upon the
original sound itself, but it is established as a function of the available
recording technology. This leads to a paradoxical situation in which more
and more musicians try to reproduce in their live performances the same
sound qualities of their records, specially in pop music. In a stimulating
essay, John Mowitt observes that:
The second alteration is related to musical gesture. Although music has
always been strictly related to gesture, only in the past few decades this
issue has deserved some attention by musicologists. Gesture is taken here
in a broad sense. It does not mean only movement, but a movement which can
express something. Therefore, it is a movement which embodies a special
meaning. It is more than a change in space, or a body action, or a mechanic
activity: gesture is an expressive movement which becomes actual through
temporal and spatial changes. Actions such as turning knobs or pushing levers,
are current in today's technology, but they cannot be considered as gestures.
Also, to type a few words in a computer's keyboard has nothing to do with
gesture since the movement of pressing each key does not convey any special
meaning. It does not matter who or what performed that action, neither in
which way it was performed: the result is always the same. However, the
situation is completely different when a musician plays something in a piano
keyboard: the result, that is, the musical performance, depends in many
different ways on the player's gesture.
Gesture in music performs a fundamental role in meaning generation. In
a certain way, we have learned to understand musical sounds with the aid
of gestures that produce and represent those sounds. As G. Kurtenbach and
E. Hulteen have noted, the function of gesture in the music is proportional
to its power to express something:
At this point, we should make a distinction between physical gesture and
mental gesture (Zagonel, 1992). Physical gesture refers to the production
of sound as a physical phenomenon, retaining a causal relation between the
gesture and the sounds that are produced. Also, we can talk about body gesture
which is a physical gesture that does not produce, but accompany the sound.
Another gestural category, which is closely related to the processes of
composition, interpretation and listening, is the category of the mental
gestures. Mental gestures refer to physical gestures and their causal
and they occur as an idea or an image of another gesture. Thus, the composer,
often starts from an idea or a mental image of sound gesture to compose
a vocal or instrumental gesture. The mental gesture is learned through the
experience and stored into the memory to be used as a model in composition.
At the same time that a mental gesture refers to the corporal actions of
a performer or to the behavior of a musical instrument, it can also refer
to a particular sound structure. Thus, an arpeggio can be understood as
a gestural movement from one point to another in the pitch space and the
movement of a conductor's hand can be seen as a gesture that unifies the
temporal and articulatory behavior of the orchestra. What we should emphasize
is that the mental gesture always refers to a physical gesture which has
been previously learned.
Concerning music, one can say that the physical gesture is directly related
to music interpretation while the composition is much closer to mental gesture:
As Bernadete Zagonel has said, "if the composer goes from gesture to
the composition, the performer goes the opposite way, that means, he goes
from the score to the gesture" (Zagonel, 1992: 17-18). To this statement
we can add that the listener completes this chain by mentally recreating
the performer's physical gestures while listening to music.
The lack of musicological studies taking body gesture in account is in
part due to the fact that the body has never been considered as a support
for musical expression. It was only when electric and electronic means of
musical production and diffusion eliminated the presence of the musician
and his instrument that the importance of the body's presence in musical
production starts to deserve attention. The symbiotic relation between the
player's body and his instrument plays a special role in the comprehension
of the musical discourse. For example, a violent gesture produced by the
player reinforces the effect of a sudden sound attack in the same way that
the body expression of a singer can lead to a richer phrase articulation.
The musical background and the daily experience with the objects which
surround us help to establish a connection between physical materials and
types of gestures that will produce specific sounds with those materials.
Through our experience we can establish patterns and links between sonic
events and the materials which could produce these sounds. Of course, the
perception of these relations is not precise in specific terms (for example,
it is hard to determine how loud is the sound of this book falling on the
floor), but this perception is extremely sophisticated in relative terms
(for example, one can easily say that rolling a pencil over a wood table
produces less sound that hitting the table with that pencil). Moreover,
part of music's expressive power is given by this proportional scale through
which one can mentally build relations among materials, gestures and sounds.
In fact, we don't know among the objects which surround us any which would
lead to a sound performance similar to musical instruments. Our perception
tells us that an ordinary piece of wood, about 50 centimeters long would
never produce a sound with the same quality, power, brightness and regularity
as on can produce on a violin, for example. Also, ordinary objects usually
don't allow a precise control of the sound produced by a specific gesture.
Thus, musical instruments subvert our expectations about the sounds that
such devices would produce. What a musical instrument allows to do in terms
of sound goes far beyond what our common sense would expect from any other
ordinary object. In this way, the musical instrument is not only the medium
of the musical idea, but also part of this idea.
Obviously, each instrument offers a different level of control and interaction.
For example, the mechanism of a pipe organ works almost automatically in
a way that the performer's gesture has very little control over the process
of sound generation. On another hand, instruments such as the violin, the
trumpet or some types of drums allow the control of very subtle characteristics
of the sound through the interaction between the player's gestures and the
body of the instrument. As Richard Moore points out,
In a book called "The Sight of Sound: Music, Representation, and the
History of the Body" Richard Leppert calls the attention to the importance
of the gestural images which are produced during a musical performance:
The manipulation of tape recorders, synthesizers, and computers eliminates
the materiality of sound. Electronic sounds do not embody any kind of gestural
relation to the devices that produce them. It represents a great autonomy
and freedom in terms of creating new sounds, but at the same time it means
a loss in the symbolic and meaningful dimensions that can be present in
a musical work. In the last 50 years electronic and digital technology expanded
the possibilities of creating music in a very significant way, but this
expansion also brought the risk of a loss in the semiotic connections conceived
during the performance. Today's musical activities based on the production
of pre-recorded or electronic generated music have deviated from the embodied
practice of performance to artificial processes of composition and diffusion.
In those processes the loudspeaker replaces the performer, thus eliminating
visual and gestural references which traditionally composed the symbolic
dimension of musical language. In which way contemporary musical practices
such as electroacoustic music will restore this symbolic dimension is something
that still has to be elaborated and can be seen as a challenge that will
entertain, for some time, composers, performers and listeners.
- Henrotte, G. A. (1992). Music and Gesture: A Semiotic Inquiry. The American
Journal of Semiotics, 9(4), 103-114.
- Kurtenbach, G., & Hulteen, E. A. (1990). Gestures in Human-Computer
Interaction. In B. Laurel (Ed.), The Art of Human-Computer Interface Design
(pp. 309-317). Massachusetts, California, New York, et al.: Addison-Wesley
Publishing Company, Inc.
- Leppert, R. (1993). The Sight of Sound: Music, Representation, and the
History of the Body. Berkeley: University of California Press.
- Moore, F. R. (1987). The Dysfunctions of MIDI. In International Computer
Music Conference, (pp. 256-263). San Francisco: ICMA.
- Mowitt, J. (1987). The Sound of Music in the Era of its Electronic
In R. L. &. S. McClary (Ed.), Music and Society: The politics of
performance and reception (pp. 173-197). Cambridge: Cambridge University
- Thompson, E. (1995). Machines, Music, and the Quest for Fidelity: Marketing
the Edison Phonograph in America, 1877-1925. The Musical Quarterly, 79(1),
- Zagonel, B. (1992). O Que É Gesto Musical. São Paulo: