The purpose of the present study was to investigate the effect of change in instrumentation on the recognition of thematic materials used in Reynolds' The Angel of Death. In this piece, five themes are first presented with one instrumentation (either piano or orchestra) and then, later in the piece, they occur again with a different instrumentation (see Reynolds, this CD-Rom). There is a kind of mirror relation between the performances of the orchestra and the pianist. This mirror relation allows the listeners to reverse their musical perspectives and to listen differently to a piece with the same musical structure. As such, recognition of similar materials with a change in instrumentation would contribute to the establishment of formal relations within the piece, if the timbre change didn't hinder recognition. The main question was to investigate whether it would be difficult for participants to recognize these themes in the context of the piece (see McAdams, Vines, Vieillard, Smith, & Reynolds, this CD-Rom) when they reoccurred with a different instrumentation. This issue also provided the opportunity to address whether timbre potentially contributes, along with pitch and rhythm, to defining the identity of musical materials. This means that timbre may be a basic component of material identity, similarly to the way facial expressions contribute to the perceptual identity of a person. In the history of Western music, the quality of timbre that results from instrumentation was considered as less important than other form-bearing dimension such as pitch and rhythm. For example, in the Baroque period, the instrumentation required to perform a given piece was generally open, or at least weakly constrained. This suggests that the musical quality of the musical material was mostly defined by the combination of pitch, duration and loudness.
The importance of instrumentation increases considerably in the Classical and Romantic periods, as attested notably, by the edition of several textbooks on instrumentation (see Berlioz, 1843; Gevaert, 1885; Guiraud & Busser, 1933; Kastner, 1837; Koechlin, 1944; Rimski-Korsakov, 1914). Also several transcriptions of orchestral pieces for other instruments (such as the piano) have been done (see Liszt's transcriptions of Beethoven's symphonies and Berlioz' Symphonie Fantastique, for example). No one would contest that these transcriptions are generally less referential than the pieces as they were composed with their original orchestration (with maybe one exception for Moussorsky's Pictures at an Exhibition). From the early 20th century, composers " notably French composers such as Debussy, Ravel, Koechlin, Varèse, Jolivet, Messiaen, etc " paid a great deal of attention to timbre and acoustic phenomena. The importance of timbral qualities turned out to be of critical importance in contemporary music, where timbre was supposed to act as a potential form-bearing dimension that might replace, accompany or interact with the pitch and duration dimensions (see Barriere, 1991; Erickson, 1976; Lerdahl, 1987; McAdams, 1987; Slawson, 1985). Along this line, we might expect that a change in instrumentation, which affects the quality of timbre, would have considerable impact on listeners' ability to recognize the thematic material. Our goal was to investigate the strength of this effect in the context of contemporary music, which puts a great emphasis on timbral qualities.
From a psychological point of view, this issue also allows for the study of how different sound dimensions are combined. Music conveys information over a number of perceptual dimensions simultaneously, such as pitch, loudness, duration, and timbre, which McAdams (1989) has referred to as form-bearing dimensions. The question of whether these dimensions are processed separately or in an integrative way remains an open question for music psychologists (see Peretz & Kolinsky, 1993 for a debate). Several studies have investigated how pitch and timbre are combined with perceptual tasks. For example, Crowder (1989) found an influence of timbre on pitch judgments. Participants had to make same/different judgments concerning the pitches of two successive tones. The two tones were played on either the same instrument or on different instruments. Pitches were judged faster and more accurately as being the same when they were played on the same instrument than when they were played on different instruments. Likewise, by using rapid classification tasks in the tradition of Garner (1974), Melara and Marks (1990a; 1990b; 1990c) also observed an interaction between pitch and timbre in individual tones. Pitt and Crowder (1992), by varying pitch and timbre orthogonally, provided additional evidence in favor of an interaction between both dimensions. The listeners had to judge whether the second of two sequentially presented tones had the same pitch as the first. The timbres of the first and second tones were either the same or different. The results showed that the listeners had no difficulty answering "same" when both pitch and timbre are the same for the two tones. However, when the timbre of the second tone was changed, performance decreased significantly. This result suggests that timbre influences the perception of pitch. In addition, Krumhansl and Iverson (1992, Experiment 1) also used the standard Garner (1974) classification tasks to evaluate how pitch and timbre interact at the level of individual tones. The results showed that the participants could not attend to the pitch of a tone without being influenced by its timbre and could not attend to the timbre of a tone without being influenced by its pitch.
It is worth noting that the influence of timbre on pitch perception seems to depend on the degree of musical expertise. Beal (1985) asked listeners to judge whether two chords played with different timbres were the same or different. Nonmusicians found it very difficult to judge two chords as being the same when they were played with different instruments, although they performed well when the chords were played on the same instrument. Musicians were more accurate overall, even if they found it more difficult to judge the chords as identical when they were played with different timbres. Furthermore, Pitt and Crowder (1992) found that when a pitch was played with different timbres, it was very difficult to judge it as being the same pitch, but in addition, the degree of difficulty depended on musical training. Indeed a supplementary analysis of the same-pitch/different-timbre data revealed a bimodal distribution of accuracy that correlated with the musical background of the participants. In their Experiment 3a, Pitt and Crowder (1992) found a similar result with accuracy in pitch discrimination being 15% higher for the musicians than for the nonmusicians. It was proposed that for nonmusicians, timbre is likely to be a more salient dimension than pitch because it is generally more informative about environmental events, and the nonmusicians have not been trained to analyze pitch closely. Another explanation may be that timbre, which is a salient surface characteristic of musical tones, would be more influential for nonmusicians than for musicians. This assumption would be in accordance with the results observed in the research on the effects of expertise. Experts seem to build a representation of the material based on the deep structure of the stimuli (as well as on surface features), whereas the perceptual representation of novices would be based only on the surface features. The current literature leads one to expect that nonmusicians would be more influenced by the timbre of a musical excerpt than by its harmonic structure.
Other studies have investigated how timbre and pitch combine in memory, a question, which is central to the present study. Semal and Demany (1991) reported results supporting the idea that pitch memory is independent of the timbre of context tones. They used the pitch-recognition paradigm of Deutsch (1972). In this paradigm, two tones (standard and comparison) are judged as same or different. A number of intervening tones were presented between the standard and comparison tones. Semal and Demany (1991) varied the period (close versus remote from the period of the standard tone) and the spectral content (same versus different harmonic content as the test tones) of the intervening tones. They found that intervening tones close in pitch to the standard and comparison tones produced memory interference. In contrast, they found no effect of similarity in timbre of the intervening tones on memory performance for pitch. This latter result suggests that pitch memory is independent of the timbre of context tones. Krumhansl and Iverson (1992, Experiments 2 and 3) had a similar approach to that of Semal and Demany (1991). They also found independence between pitch and timbre. Changing the pitches of the tones surrounding the target tone did not interfere with memory for a target timbre, and changing the timbres of tones surrounding the target tone did not interfere with memory for a target pitch. They also showed that memory for the target pitch was not different when the timbre varied than when it was constant. Taken together these results suggest that pitch memory is little affected by timbral variations.
In contrast to these simple stimuli, studies using memory for melody supply different results suggesting that timbre may be encoded in an episodic trace and may influence subsequent recognition. Radvansky and Potter (2000), for example, showed that timbre might act in terms of source cuing. Source cuing is a source-monitoring process in which the retrieval of a memory trace is aided by the use of a memory probe that includes information indicative of the original source (an episodic memory process1). Melodies were presented to participants who had to identify which of two melodies (target or distractor melody) had been heard earlier. They manipulated two variables: a Match/Mismatch condition referred to whether the target melody matched or mismatched the timbre of the original melody and a Same/Different condition referred to whether the distractor melody was in a timbre that was the same as or different from the original. The participants were more accurate when the timbre of the target melody matched the original than when it mismatched. Their findings also showed that timbre was more indicative of the source than pitch. In addition, Peretz, Gaudreau and Bonnel (1998, Experiment 3) corroborated the hypothesis in which timbre influences the recognition of melody. They expected that changes of surface properties between study and test melodies would influence performance in an esthetic preference task (rating the liking of melodies) and have little impact on recognition performance because the affect is proximal to the perceptual processing system while recognition are conceived to tap more abstract forms of memories. To evaluate this hypothesis, they modify the instrument on which the melodies were played. However, contrary to their hypotheses, they observed a strong effect of timbre change on melody recognition. Melodies that remained in the same timbre from study to test were recognized better than were old melodies that were different in timbre at study and test. In a fourth experiment, they also demonstrated that timbre of the studied melody is part of the episodic record, and thus, is available for subsequent recognition. In addition, they suggested that timbre may be computed automatically and may be critical to episodic memory for melodies. These last two experiments (using more complex stimuli than just pitch) therefore suggest that timbre may be stored as contextual information for the presented melody, so as to confer to the studied event a distinct episodic-like quality. Timbre and pitch would thus be computed separately and linked together in episodic memory, so as to provide a unified recollection of the studied event.
The influence of timbre on memorization of musical material was also shown to depend of the extent of musical expertise. Wolpert (1990) found differences in pitch and timbre processing in nonmusicians and musicians, suggesting that timbre is more salient than pitch in nonmusicians. In her experiment, the participants performed an XAB match-to-sample task. They had to judge whether melody A or B was more like melody X. One of the two comparison melodies was always melody X played with a different timbre. The other comparison melody was either a different melody with the same timbre as melody X or was played with a harmonic accompaniment transposed to the dominant key (i.e. key mismatch between melody and accompaniment). While the musicians unanimously chose the same melody and accompaniment as melody X (ignoring the timbre difference), only half of the nonmusicians consistently did so. The remainder matched on the basis of timbre a portion of the time. However, these findings have been questioned by Radvansky, Fleming, and Simmons (1995), who attempted to replicate and extend the results observed by Wolpert (1990) by controlling the clarity of the instructions. Listeners were presented with a melody and then later asked to identify which of two alternatives (target versus distracter) corresponded to the original melody. In the Match condition, the target melody was in the same timbre as the original, whereas the distracter was in a different timbre. In the Mismatch condition, the target melody was in a different timbre whereas the distracter was in the same timbre as the original. In their second experiment, they compared musicians and nonmusicians and found that both groups were similarly affected by timbre changes: more errors were made in the Mismatch condition than in the Match condition. As a consequence, the influence of timbre on memory for musical stimuli as a function of musical expertise remains a matter of debate.
In sum, studies using single tones as stimuli provided some evidence that pitch and timbre are stored independently in memory. By contrast studies using longer melodies provided some evidence that both dimensions are integrated in memory. Finally, some studies suggest that this issue may be modulated by the extent of musical expertise. It should be noted that none of the current published studies address this issues with realistic musical materials drawn from contemporary music. Some studies, however, have addressed the recognition of contemporary musical excerpts. For instance, in Krumhansl (1991), participants were able to recognize excerpts despite changes in contour, pitch height, duration, dynamics and interval size. However, no change in timbre was performed in this study, because all stimuli were played on the piano. The question therefore remains concerning whether change in timbre will produce detrimental effects on memory for musical materials.
According to the results of previous studies, we assumed that timbre would influence the subsequent recognition of musical excerpts and that this effect would be modulated by the extent of musical expertise, even if this issue remains a matter of debate. To the best of our knowledge, the present study is the first to address this question with realistic musical material drawn from contemporary music. A crucial point is that the change of instrumentation was performed by the composer (and not by the experimenter). With complex musical excerpts, it is not conceivable to change the timbre of the excerpts in an automatic way by affecting artificially some tone to some timbre. Orchestration requires considerable "savoir-faire" without which the result would be musically incoherent. The risk would then be that participants fail to recognize excerpts not because of the change in timbre, but simply because the performed change in instrumentation would result in a weakening of the musical coherence of the excerpt. As described by Reynolds (this volume, see also McAdams, 2004), a change in instrumentation raised a number of difficulties the composer had to solve. As such, the musical themes were of different instrumentation while containing the same type of musical qualities. Another advantage, results from the fact the experimental stimuli were also performed under the control of the composer. That is to say, both the pianist and the orchestral conductor, performed expressive deviations in their playing that were controlled by the composer so that both versions of the theme remained comparable for musical expressiveness, irrespective of change in instrumentation. Because of these controls, it was of interest to address whether musically trained and untrained listeners would be able to recognize the Reynolds themes under change in instrumentation.
