Computer transformations

The use of the computer in this work is motivated by several creative needs. Firstly, the metaphor of the title ideally requires an element that not only refers to the realm of the extra-ordinary, but also to an element that can produce moments of seeming magic, experiences beyond the world of normal instrumental behaviors. The existence of two complementary parts making up the large body of the work has the effect of placing both, the Domain and the Sectional, in the position of having to be self-sufficient, complete in themselves. Whichever is heard first must be persuasive in and of itself. At the same time, each must be able to accommodate coexistence with the computer materials when it occurs second¹¹.

Thus, from a perceptual perspective, it is necessary that the computer materials be both distinctive and memorable, that they be easily distinguishable from the instrumental materials. But it is also necessary that some of them be "transparent". Provision is made in the planning of the computer/instrumental coordination for the fact that the computer materials will have their own moments of emergence, even dominance¹².

In recent work, beginning with the song cycle, last things, I think, to think about … (in 1994), I began to use musical units that are less explicit and conventionally defined than "themes". I think of them as "images", as evocative of a certain feeling or circumstance, and they tend not to have an ongoing or quasi-narrative shape. In part to focus my thinking about the computer's role apart from the instrumental music, I decided to invoke the "image" ideal to describe the 11 planned computer elements. The six Domain images (D1, D2, D5, D6, D9, and D10) tend to be organic and broadly sculpted, to arise out of a singular, basic process. The five Sectional images (S3, S4, S7, S8, and S11), however, tend to have a mosaic nature, involving numerous component parts, an array of processing strategies, and a distinctive temporal design¹³.

The first image to appear is an extended, solo bridge between the close of the first instrumental part and the beginning of the second. The last of the images, correspondingly bridges between the close of the second instrumental part and the piano's brief Epilog that comes only at the end of the entire 35-minute experience. A third image also is heard without any live instrumental component; a computer-processed version of Other. Whichever part comes first in a performance, the piano soloist presents Other during it. In whichever part is second, the computer assumes the presentation of this material. The images all derive from the five themes as recorded in their ensemble and solo piano versions. Frédéric Voisin (my Musical Assistant for the Angel Project) and I used a variety of signal processing techniques in realizing them.

The range of durations for the images is relatively broad: the shortest is 61.5 seconds while the longest is almost three minutes. They were conceived as larger than the thematic elements, and each tends to have a consistent character rather than an envelope shaped by a number of subsections. As a result of the collaborative cast of this Project and the psychologists' interest in my verbal characterizations, I attempted to do the same with the computer images. Although some, such as D9's "Drifting Masses", found apt description, it proved a difficult task, and I do not feel I was successful. The attempt to achieve parallel descriptions (both graphic and verbal) for each image in my notebook sketches was useful, however.

Sound Examples: Sd. Ex. 6-15

(The 10 computer images [from the originally planned 11] that are used in the actual composition are made available here in stereo reductions of their original 6-channel versions)

In view of the fact that the computer images were going to co-exist with instrumental materials that were performed live, some temporal flexibility in the dissemination of the computer part was desirable. My first thought was that the images might be realized so as to allow variable overlapping, and that technicians could flexibly manage the placement of start times during performance. This mean that the images needed to correlate well simultaneously with the corresponding passages in the Sectional or Domain parts, and also successively with one another.

I devised a plan that maximized my picture of the "meta-contrapuntal" issues needing consideration. Although the diagrammatic presentation of the images in relation to the composed instrumental parts gives, graphically, the impression of a descent from D10 towards S7, this early ideal — that the computer part would "flow down over the instrumental music" — was not realized. Still, the highest computer sounds do occur at the beginning of D10, and the lowest near the end of S7. The sonic material of each image was fully developed before the completion of the Sectional part, but not spatialized until after the writing of the Domain part.

A relatively small number of digital signal-processing techniques were used. The initial ideal was that each image would utilize a singular approach, but this proved impractical: I did not want to constrain the richness of the images by closing off the advantages of hybrid approaches. Using the chart shown in Ex. 29, I proposed processing approaches for each image. These fell into several categories: temporal expansion and contraction, raising and lowering pitch level, disassembly (vertically or horizontally), blending identities, and algorithmic fragmentation of identity.

The ability to change the temporal nature of sound without radically altering its timbral character is a fundamental capacity introduced by computers. The traditional ideal of augmentation has been coveted not only in Western music, but also in that of many other world cultures. But taking the relational design of a melodic line and prolonging its influence by systematically enlarging its scale is a limited realization of the underlying ideal. The nature of the subject's sonic evolution is not essentially changed. One does not hear the intriguing details associated with the changing physical states of an instrument as, for example, the left hand of a cellist presses the A-string toward greater tension in reaching for a high-register note. The visual analog — the extended representation of physical movement — in slow-motion video "replays" suggests how much can be gained in understanding the dynamics of motion but also in feeling its empathetic effects.

In D9, the essential chain of chords used in the Theme 5 core element was extracted from the piano and instrumental recordings and then each chord was individually stretched to fit the specifications of a pre-ordained plan.

Because of the degree of time extension required, considerable effort was expended in finding the optimal application of phase vocoding analysis and resynthesis techniques. This was particularly true with the piano versions, because the characteristic piano attack (the hammers striking the strings) is compromised by the process. The chords were also transposed from their original pitch levels so as to become part of a new harmonic framework. As retaining sonic fidelity under transposition is itself problematic, I devised a plan that required minimal transpositional shifting. The result was a 2-leveled mosaic in which the piano chords were proximate and the instrumental ones distant shadows.

The re-presentation of Other during the second part of any performance, was tactically delicate: it was important that it seem both instantly familiar and yet (in part because of the accidental theatricality of the situation — a piano solo already experienced is repeated without the pianist's involvement ) significantly different. I used two fundamental processing strategies, the most demanding of which was precise digital filtering of the original piano recording. In accord with the D ideal, the whole 161-second source was filtered into two parts, effectively creating a negative and positive image. I designed the filtering function so as to maximize its influence symmetrically around the mid-point of the source sound file. Digital filtering allowed dramatic suppression of the rejected portion of the frequency domain. The resulting representations were then reassembled with slight temporal misalignment. Time-varying expansion and contraction (by phase vocoding) was used to emphasize the impact of the filtering, bringing the sonic image subtly into and then out of "focus".

The symmetrical filter-process design was superimposed upon a sonogram of the energies present in the Other recording to determine exactly how to apply the process. It was clear that the great majority of useful energy in the recording occurred below 7500 Hz, so we filtered only the lower half of the total spectrum, allowing the upper half to remain as it was.

To underscore the perceptual blurring that temporal misalignment produced by digital filtering along with time-varying compression and expansion, the negative and positive images were also subtly pitch shifted. The composite effect after many adjustments was exactly what I had hoped: the materials were both evidently familiar and yet increasingly, subtly altered. In addition to these treatments of the piano source recording, I also extracted 11 relatively brief segments from the instrumental theme recordings and superimposed them upon the altered piano continuity so as to suggest the distant familial relations of Other's materials to the themes. One example of this is the relationship between Other's characteristic keyboard glissandi and the rapid runs of Theme 4.

As can be observed in Ex. 28, image S8 coexists with "Repetitive Strata", the instrumental passage substituted for the beginning of COMB1/2/3, and described above in Subsequent Elements (Treatment and Adjustments). Given REP STRAT's nature — a convergence of ostinatic cycles from the perspectives of speed and harmonic concentration — it was evident that the algorithm SPIRLZ (cf. above Computer Algorithms: SPLITZ and SPIRLZ) could provide a strongly related but intensifying experience of culmination. This occurs just before the brief preview of the piano Epilog, which, in turn, precedes Theme 5. Dramatic moments from, for example, the close of Theme 1, were taken as sources for SPIRLZ processing. The design of S8 specified a staged, directional concentration involving shorter subsections, and an accumulation of overlays. The SPIRLZ algorithm allows one to achieve — by an informed process of trial and error — an outcome, which retains procedural fidelity but also fits the desired durations. The layered scheme (Ex. 33) shows the way in which the convergences inherent in the operation of the algorithm, SPIRLZ, are echoed in the structure of the image itself. S8 is an example of an image in which the ideal of transparency was sacrificed in the interest of intensifying an already existing structural moment in the two component parts of Angel.

The S11 image occurs in synchrony with what had been planned as the COMB3/5 element of the Sectional ideal (though it was, in fact, omitted from S) and centers on the symmetrical Domain version of this element. Both occur midway through the long evolution of TR2 to 4. Although this element is dense and assertive in the Sectional part, it is carried by the solo piano in Domain, and has a subtler, gradual, and stirring growth. It has an intensely expressive nature, which I did not want to disturb. Thus, it was necessary that S11 be transparent. Segments were extracted from Themes 3 and 5, respecting the connection with COMB3/5, and the piano and ensemble versions were intricately intercut. Each relatively short element in this succession was then either left at its original pitch level (rarely) or raised or lowered by one or two octaves. The idea was to create a plausible succession of material but to constantly disrupt its timbral identity and register. The inevitable and unacceptable timbral distortion introduced by substantial degrees of transposition required subtle adjustment of dynamic levels to mask the disruptive effects of rapid cutting from one version of the basic material to the next. (There were sometimes several switches per measure.) The spatialization employed long reverberation decay times and emphasis of the reverberant over the direct signal.

Image D5 ("Formulating/Reformulating") involved the effort to cross-synthesize an element from the ensemble version of Theme 5 (the initial expressive phrase) with one from the piano version of Theme 2 (a brief, powerful, sf chord). Cross-synthesis is one of the most powerful ideals introduced by the potential of computer processing, equal to or even more intriguing than the slow motion time expansion discussed earlier. The aim is to have the essences (sonic and relational) of one bit of material gradually morph into, take on the sonic and relational identity of, another. A relatively primitive simulation of this ideal — the effect of superimposing the amplitude envelope of one passage upon that of another — can be achieved by the strategy of ring modulation or gating. Imagine the combination of human speech with a complex, wide-frequency-band source like the ocean, producing the impression of a "talking ocean". In the present case — the mutual influence of a long expressive line and a brief, forceful chord — I wanted to obtain the effect that the energy and the frequency content of the chord gradually invaded the integrity of the tranquil line. As can be seen from the diagram in Ex. 34, this image has an algorithmically derived (SPIRLZ) introductory section, followed by a line, extracted from subsection 1 of Theme 5 (the notation T5.1 will be used subsequently) that lasts just over a minute. Overlaid upon and infused with this was the impact of a chord from the second theme, which had been reversed and stretched to an approximately 45-second length.

The most simplistic version of such a pattern of exchanging prominence would be to superimpose the two sources (the line and the reversed chord) and fade one out while fading the other in. It was more demanding, but also much more satisfying to achieve a gradually increasing influence of the chord's frequency characteristics, percussive envelope, and timbral essence upon the instrumental line. There is something moving (and in this case also disturbing) about the modification of one consistent entity by means of the influence of another. Perhaps the loss of what one has always understood as the consistency, the immutability of perceptual identity is the reason for the unsettling effect. No single process available in Ircam's AudioSculpt or other available software environments produced acceptable cross-synthesis in an autonomous way. Frédéric Voisin balanced and intermixed a number of complementary approaches until he was able to simulate a result that corresponded with my imagined goal.

By far the most intricate image is the final one, S7. Although I did not originally plan it to be a summation of everything that went before, this is what it became. Its master plan below shows that materials were derived from both the ensemble and the piano versions of the thematic elements 1-4 (Theme 5 is not included because it is just concluding as S7 begins.).

This image gave me an opportunity to use materials from the piano and ensemble versions of the themes that were not actually heard in the work itself (e.g., piano passages from subsections of Theme 2 other than the core element, sections from the ensemble version of the third subsection of Theme 3, etc.). By introducing the widest range of materials and processing techniques, I sought the "life passing before your eyes" effect often attributed to the last moments of existence. Signal processing strategies included algorithmic transformation by SPLITZ or SPIRLZ (s7a, s7c, s7e, s7g, s7h, s7i), transposition (s7e, s7h, s7i), time expansion and compression by means of phase vocoding (s7a, s7c, s7d, s7g, s7h, s7i), editorial multiplication and superimposition (s7c, s7f, s7h, s7j), and negative/positive filtering (s7b, s7d, s7g).

In keeping with the integer series, eleven sub-images were arrayed in time so as to create a cumulative impression of grandeur and force. They led, finally, through a quote from the piano version of Theme 4, into the Epilog. Low register sounds from the individual instrumental section recordings were drawn upon to assemble a low register pitch line (cf. upper right corner of the diagram) that would emphasize the cadential function of this "Vertiginous Continuity" image. (The processing specified for this final bridging fragment in the plan was not, in the end, utilized. I decided that there would be a more direct transition from computer to solo piano if the last computer sound to be heard were more literally instrumental in character. So this phrase was given an extra-instrumental gestural character through spatialization.)

Independently recorded instrumental strata were not only essential to the impact of S7, but, also to its symmetrically placed complement, D10. S7 is the most heterogeneous (the exemplar of the S ideal), and D10 has a parallel relationship to the D ideal. Its origins are in the core element of Theme 3, where two rapidly, tremulously alternating figures gradually converge in a melting together gesture. Utilizing the AudioSculpt environment, it was possible to separate the descending from the ascending portions of this gesture by editing out, in turn, those partials associated with the rising lower line and those associated with the falling upper line. Frédéric Voisin did this for both the solo piano recordings and the individual strata of the ensemble recordings (woodwinds, brass, percussion and piano, strings). Each stratum was then transposed up one and also two octaves, or symmetrically down in a parallel fashion. The transposed versions were then time stretched so as to overlap and integrate with one another to evoke the impression of a long, continuous descent and a parallel (though less extended) rise.

Beginning with a descent from the highest registers (this enters over the closing section of Theme 5), the impression of a very extended and continuous downward movement is achieved. After this continuity is well-established — one imagines it is the totality of what will happen — the ominous low register entry of the complementary rise occurs. The timbric meld of the various component layers — differing registers, piano or instrumental sounds (the latter with winds, strings, and percussion separately treated) — produced an even richer composite than had been anticipated. In particular, there was a very appealing choral effect. I decided to use reverberated instrumental sources, then, rather than the dry, and therefore purer, versions. This substantially underscored the vocal aura, which I assume is due to the varied transpositional levels (always in octave relationships), as well as the fact that time expansion brought about beating and amplitude modulation in the general range of human vibrato.

In order to bring about in the sectional realm the impression of unanimity that occurred in D10, another sort of consistent ideal was necessary. This was most effectively achieved in S4, which has a scherzo-like feel. Eleven brief and animated gestures from Theme 2 were selected and edited out of their normal continuities (some from the piano recording [S4a, S4b, S4c, S4d, S4f, S4g, S4h, S4I], others from the ensemble one [S4a, S4c, S4e, S4i, S4j, S4k]). (Note, not incidentally, that in the majority of cases, the thematic materials utilized in any image were kept to a minimum, so that they constituted a further development of, or elaboration upon, a given theme. This also, of course, optimized the degree to which images could elegantly co-exist with their instrumental contexts.)

Once the component fragments were edited out of the source recordings, they were subjected to characteristic computer transformations, in order to enrich their distinctiveness: superimposition, negative/positive filtering, algorithmic processing, transposition, and time expansion.

Each element had a characteristic temporal periodicity that resulted in a certain number of iterations (in keeping with the integer series: 3, 7, 11, etc.) Their distribution, arrayed over the 70.5-second duration of the whole, needed to take into account the fact that this image appears in coordination with the core of Theme 4, and should leave it relatively undisturbed. (Some interference, however, was not inappropriate given the fact that the Theme 4 ideal, "jagged rips", specifically invites the element of disruption.) One can see that the plan (Ex. 38) is so arranged as to leave the appropriate temporal band (between the vertical dotted lines) relatively free of encroachment.

The spatialization for Angel takes place in a 6-speaker environment (Ex. 39), with two widely spaced speakers at the front (1 and 4), two more distant but still centered and frontal speakers (2 and 3), and two in the rear (6 and 5). Thus, if one proceeds clock-wise from the far left front, the speakers are numbered 1, 2, 3, 4, 5, 6. All of the materials for the 11 computer images were fully realized, not as final mixes (though mixing was a necessary, intermediate step in order to assure composite coordination and that the desired aggregate impression was being achieved) but as component layers. The various strata of each image were realized as single channel, monaural sources and were then spatialized at the final stages of work on the computer part, according to individualized ideals for each image. Normally, a completed image involves the superimposition of several 6-channel layers.

The scherzo-like character of S4 is intensified by a very assertive complement of spatial gestures customized to the content of each of the eleven repeating, motivic components. For example, s4a in Ex. 40 involves three extremely rapid iterations of the source chord, creating a "path" that jumps from one side of the virtual space defined by the 6 speakers to the other.

Although the nature of the spatial gesture in this image does not change from one iteration to the next, its position in space does, so that, again, in the case of s4a, the jump occurs first from speaker 1 to 5, next from 5 to 1, and finally from 1 to 5, again. The four iterations of s4c also occur in reversing pairs: from 6, clockwise to 3, from 3, counter-clockwise to 6, and repeating. All of the paths are very proximate and lacking in reverberant content in order to maintain a maximal specificity to the spatial illusions, all of which were generated in Ircam's spatialization software environment spat.

The approach to S4 spatialization could be thought of as a mosaic of brief, interacting gestures with familial similarities but constant repositioning, and, hence, as a web of continually novel gestural interaction. This illustrates a basic tenet of my approach to spatializing: I attempt to develop a repertoire of, in a sense, choreographic gestures that are effective in and of themselves, but which can then be concatenated in one of two ways. In the instance just illustrated in S4, the gestures are used as individual aspects of an aggregate which itself is designed. Alternatively, one can devise chains of succession, creating larger continuities as products of modular sub-components (a circular path made up of four 90-degree arcs would be a [trivial] instance).

The strategy for D1 (the Other transformation), however, was quite different. A basic spatial feel was established by the contrary movement of the positive and the negative portions of the total spectrum (cf. Exs. 31-32). Because of the nature of this image and my desire for a particularly "massive" impression, the two complementary components were moved across coordinated groups of three neighboring speakers. The positive portion moves counter-clockwise (e.g., 4 + 3 + 5 to 3 + 2 + 4 to 2 + 1 + 3, etc.), while the negative one rotates (also three at a time) clockwise. Superimposed upon this ground (which also has a distance component) were the 11 brief instrumental gestures, each with its idiosyncratic gestural character.

The low-register drones at the close of S7 were handled similarly to the D1 background. The central sonority was analyzed and separated into two components: odd and the even partials. Each set was then moved in symmetrical opposition, at a three-speaker width. In performance, one can feel the inertial weight of these sounds.

D9 involves, as discussed above, a series of chords (either for piano or ensemble); but each chord is preceded by a singular, anacrusic grace note. The inner, more proximate paths (for the piano) were projected in the following way: the anacrusis was heard from a single source (either speaker 1 or 4), moving, then, for the chord, to the opposite side of the space into an image that was two speakers wide and recessed at some distance. The impression is of a gentle "launching" that projects itself diagonally across the space, into the distance and then begins to rotate, in alternation, either clockwise (if originating in speaker 1) or counter-clockwise (if originating in speaker 4). The ensemble chords were used without their anacrusis functions, and were rotated in contrary directions, more slowly, and at greater apparent distance. They constituted an "outer" path. Thus, there were a number of basic strategies that were redeployed in a characteristic way for each image, always motivated by the aim of more fully instantiating the basic nature of each.

In my notational convention, a six channel representation was graphically replicated (cf. Ex. 40). Given the image name and the source timings, graphic short-hands were developed to indicate the initial position of the source, its destination, its width, and the successive stages of each spatial gesture with associated timings. Ircam's spatializer, spat, was used to manifest, in a 6-channel format, the component layers of each image. These were then mixed and played back numerous times in order to adjust relative speeds, distances, and intensity levels until the composite image was persuasive. In some cases, this final process of mixing and balancing was very time-consuming. S7, for example, has on the order of 30 superimposed, six-channel components.