Creative Collaboration, Racial Discrimination and Surveillance in The Lips are Differentby Hazel Smith and Roger T. Dean,
austraLYSIS and Western Sydney University

Are white people and people of colour treated differently at national borders? How can we make art about racism? How can words, image and text speak to each other in multimedia work? How does research impact on creative work? What are the dynamics of collaboration?

These are some of the questions we hope to address as we talk about the processes and ideas that resulted in (and also evolved out of) our collaborative multimedia piece The Lips are Different. The focus of the piece was the injustice suffered by the Canadian citizen Suaad Hagi Mohamud — born in Somalia — who was accused of not being a Canadian citizen when she tried to return to Canada from Kenya in 2009. This incident seemed fertile ground for a creative work because it linked over-surveillance, racial discrimination, photography, media representation and issues of identity.

The piece comprises real-time video written in Jitter; improvised music based on a comprovisation score and performed text. It also includes sampled/processed text presented both acoustically and as video image. It was first performed in 2018 by the sound and multimedia group austraLYSIS, of which RTD is director and HS is a founder member.

Here we illuminate the process of making The Lips are Different, the ideas and techniques that inform it, and the collaborative processes involved in constructing and presenting it. This process started with research into the ideas that underly it, which then led into its creative development. Making the work, in turn, resulted in further research activities in the form of writing and presenting this essay, which reflects on the creation of the work and the ideas it embodies. This process is part of the generative cycle of creative practice and research (research-led practice, practice-led research) which we have explored theoretically previously and named “the iterative cyclic web” (Smith and Dean 2009, 19-25).

As part of this cyclic and expansible process, RTD has used machine learning to generate new text, not for The Lips are Different but for this article. This text generation, which appears in the right-hand column, resulted from evolving the computational deep learning models we described in an earlier article (Dean and Smith 2018). These models “learn” the features of a chosen body of text, known as the “corpus”. This is with the objective of predicting text that would arise from any chosen input sequence (known as the “seed”). If the seed comes from within the corpus, the response will regenerate features and even sequences from it. If the seed comes from elsewhere, then the results may be more interesting because more distant from the corpus, influenced both by the seed itself and the learned features. Amongst developments from our previous work, we controlled the range of vocabulary that arises in the outputs: instead of allowing all the words in the trained corpus to potentially occur in generated outputs, we restricted the range to a smaller subset of words. We also, to a very limited extent, used versions of the massive published GPT-2 text generation model (Budzianowski and Vulić 2019). In each case, text generation was triggered by seed texts taken from this essay. HS then made minor edits to chosen generated texts, in some cases adding punctuation to unpunctuated streams.

The essay below includes five video extracts of The Lips are Different, placed where relevant to the discussion. The full work can be found at the end of the article, after the bibliography.

Hazel

Collaboration is a way of extending my own creative practice, taking myself out of my comfort zone and exploring new territory. It is also a good way of embarking on a work, because starting a project is always the most difficult part of the process for me (but not necessarily for my collaborators). I particularly like writing about topics that interest those with whom I am working or adopting a technical approach they suggest. I am happy to start writing either from a technical or thematic basis, and both offer opportunities and challenges. I particularly like collaborating with musicians as I used to be a professional musician and have researched extensively in the words and music area (Smith 2016a).

The idea for this piece arose after I had various conversations with Roger about surveillance and the issues it raises. These conversations were not initially connected with any ideas about producing a creative work. However, Roger and I have a long history of working collaboratively and we wanted to create a new work, so I suggested taking surveillance as a starting point. Why a particular idea for a creative work appeals is often difficult to pin down. In this case, from my point of view, the topic provided some continuity with my previous work which had often engaged with ideas about migration, repression and incarceration (Smith 2016b) but offered new directions by linking these preoccupations with the topic of surveillance. From a writing point of view, I also enjoy the challenge of using some documentary material while introducing poetic, fictional and dramatic elements. This is a difficult balance to maintain because I am never keen to be overly direct in the content or to be too tied down to “the facts”, and often like to fragment, transform and abstract the material.

Since I had no deep knowledge about surveillance, I began to read both journalistic and academic material about it. Much of what I read, though stimulating and informative, did not immediately inspire me from a writing point of view because it was either highly theoretical or very practical, and was not particularly imaginatively or affectively suggestive. However, one book chapter (Browne 2012) stood out, perhaps because it gave some concrete examples and was not only about surveillance but about other underlying issues, particularly the way that surveillance could be biased towards societal norms. A section of the book chapter concerned the case of Suaad Hagi Mohamud, a Somali-born Canadian citizen. Through my reading of the chapter, I learnt that Mohamud went to visit relatives in Kenya in 2009, but Canadian officials would not let her board the plane home to Toronto because they said she did not look like her four-year old passport photo: in particular, they claimed that “the lips are different”. Though she gave convincing evidence of her identity, including her Canadian citizenship card, driver’s license and other identifiers, the Canadian authorities would not accept that she was who she claimed to be. Her fingerprints were taken, but they were not used because the Canadian authorities had no fingerprints on file with which to compare them. She was charged with using a false passport, impersonating a Canadian, and with being in Kenya illegally. She was detained for several months (initially jailed) and faced possible deportation to Somalia. She received no support from the Canadian government. The Canadian Minister for Foreign Affairs, Lawrence Cannon, was famously quoted as saying that “there is no tangible proof” that Mohamud is Canadian, and that “all Canadians who hold passports generally have a picture that is identical in their passport to what they claim to be” (Browne 2012, 78), a quotation I later used, with some modification, in the piece. The matter was only resolved when she belatedly took a DNA test that proved (with a probability of 99.9%) that she was the mother of her child in Toronto and that she was telling the truth.

Misplaced and inappropriate surveillance strategies therefore resulted in the horrific incarceration and detention of a Canadian citizen. For a start, the case brought up a clear example of over-surveillance, or misuses of surveillance techniques. But the incident also raised the question of whether a white person would have been subject to the same treatment. Browne discusses the way in which surveillance can impose certain norms, so that those who are seen as other are also viewed as inferior or abnormal (Browne 2012). In the Mohamud case surveillance seems to be linked to racial discrimination.

In exploring racist attitudes in the way Mohamud was treated, I was particularly influenced by an article that focuses on media representation of the case (Odartey-Wellington 2011). Odartey-Wellington employs critical race theory as an analytical tool to look at the role of racial discrimination in media representation, which he claims is central to moulding public opinion. Arguing that Canada has a racist history, and that Canadian multiculturalism is still riddled with colonialist and racially biased attitudes, he also proposes that the media tends to erase race and that this happened in the Mohamud case. He posits that “it could be constructed as a case of the Canadian government failing one of its citizens abroad; or it could be constructed in a manner that raises questions about potential racial ramifications, to the extent that Mohamud is Black, has an Islamic-sounding name, and wore a hijab in her official photographs” (398).

Odartey-Wellington reviewed 438 newspaper items on the case but centred his analysis on the Globe and Mail, National Post and the Toronto Star. The Toronto Star was the main paper to highlight the issue of racism, but, nevertheless, it was initially raised by a letter to the editors rather than the paper’s journalists, two days after the newspaper first reported on the issue. The letter writer said, “Well, I don’t always look like my passport photo either — I often wear glasses, and sometimes shave my beard — yet not once have I been questioned by the authorities over this. I wonder if it’s because I’m white and Canadian born” (402). This prompted further letters to the editor, some more radical articles and an editorial that raised the issue of racial discrimination. However, coverage in the Globe and Mail and National Post was much slower and less noticeable. The Globe, for example, only raised the question of racial discrimination two months after the story broke in Canada and even when it did, a subsequent editorial ignored the matter. Discussion of the putative racial discrimination in the media was therefore thin and patchy. Odartey-Wellington concludes that “without the active engagement of the media in the debate over systemic racism, race will continue to be erased from the public agenda to the detriment of the laudable multicultural posture that Canada has assumed” (411).

The issues that Odartey-Wellington and Browne raised about racial discrimination, surveillance and the media were obviously thought-provoking but discussed in intellectual and academic terms. I am often inspired by such theoretical and critical work but then the problem is how to turn such ideas, at least partly — and if desired — into material that is poetic, performative, dramatic and fictional. With regard to The Lips are Different, the details of the case obviously were quite dramatic, and I hoped that I could transmit them and at the same time make their political implications resonant. The letters to the editor, quoted in Odartey-Wellington’s article, in particular, were very compelling, and I decided I could use them as found material.

Roger and I have collaborated on many audio and audio-visual pieces and we have sometimes started the process with the words and sometimes with the music. In the vast majority of cases I begin by writing an initial draft of the words and this was the case with The Lips are Different. The piece arose out of reading I was doing so it was logical for the writing to come first. When we collaborate, we also need to consider whether we are writing the work initially for a particular occasion and set of circumstances, and what is consequently required technically and artistically. In this case the piece was to be one of the new works in an austraLYSIS performance, and it was fitting for the piece to have live, partly improvised music that could be performed by the members of austraLYSIS; performed text (that could also be sampled and manipulated); and screened text and image.

The script of The Lips are Different is multi-genre, which is usually my preferred choice in a longer work. I like to mix genres because it gives more flexibility and variety to the piece. I believe that adopting different styles and genres can lead to novel insights, making it possible to see the subject matter from different angles, and experience it affectively in a variety of ways. The first short section of the script consists of an imagined dialogue between Mohamud and an official at the airport who challenges her identity. However, this dialogue is not only between the official and Mohamud but also between the voice and the screen. The official’s questions and replies are heard on the audio track in the form of a synthesised voice while Mohamud’s responses appear on the screen. This means of conveying the exchange arose initially because as a performer I did not want to have to imitate the voices and accents of Mohamud or the official. However, it resulted in an effect that would not be present if the words were spoken in the normal way. Mohamud’s silence renders metaphorically the ways in which she was muted by the overbearing behaviour of the official, while the official’s synthesised voice conveys some of the robotic aspects of the damaging bureaucratic and surveillance structures to which she was subjected.

The second section of the script is a rhythmic chant that plays on the idea of surveillance and the stresses it can bring with it if it turns into harassment:

checked, monitored, tested, assessed, valued, judged checked, monitored, tested, assessed, valued, judged checked, monitored, tested, assessed, valued, judged stressed, disbelieved, detained, harassed, devalued, fudged lessened, ignored, contested, rejected, harassed, smudged arrested, enfeebled, messed with, resented, invaded, sludged

Some previous works of mine have used quasi-chanting effects and in some cases these have been notated in musical rhythms (Smith 1991 ; Smith 1994 ; Smith and Dean 1996 ; Smith 2000). In this case the insistent rhythm hopefully transmits the rising pressure of being interrogated, amounting to harassment and humiliation. However, the effect of this passage (and others) is something I have understood retrospectively. As in most of my work, I did not set out to write such a passage and create such an effect. Rather I write initially with the minimum conscious intent, and then think more consciously (as the writing of any section develops) about the effect it produces and how it fits into the overall work.

The rest of the script is more recognisably poetic while still adopting a number of performative strategies. It takes up various modes of address, one of which is employing the second person “you” form. The text often addresses a “you” who is an audience member (or could sometimes be the speaker herself). I find the resonances of the “you” form, and the kinds of relationship it can create, very rich, and have often used it in previous work. The assumption in The Lips are Different is that the addressee is not a person of colour; he or she will not encounter the problems encountered by Mohamud.

Although everybody has to go through the surveillance procedure which is “more complex than piloting a magic carpet”, nevertheless, “you won’t be skinned alive”. The officials, the text suggests, will be happy to believe that “you are /who you say you are” because being light-skinned means that you will be believed. Later the performer/narrator addresses the audience again, “Have you ever been interrogated at an airport?” and talks about the confusion and intimidation an interrogation can create so that interviewees, although innocent, start to say things that make them appear guilty. The performer/narrator also interrogates her own position and whiteness at various points, claiming that it is “unwanted ventriloquism” for her to speak for Mohamud because of the cultural differences that separate them: white society is inclusive toward her in ways it is not towards Mohamud because “my lips are different”. Here the different lips do not symbolise the alleged difference between Mohamud and her photo, but rather the white speaker’s difference from Mohamud, granting her automatic acceptance into Canadian society. The use of the second person, and the interrogation of the performer/narrator’s own position, are ways of heightening the audience members’ awareness that to be white is to be treated differently from Mohamud in the same situation. The mode of address also points out to them that as part of white culture they are implicated in the discrimination against Mohamud.

As well as addressing and implicating the audience, I wanted the text to explore the issue of the discrimination against Mohamud from a number of different angles including identity, voice and the sense of powerlessness that surveillance can create. The way in which all proofs of Mohamud’s identification were refuted by officials, and the inclusion of the official angle that “all Canadians who hold passports usually have a picture that is identical in their passport to them” (Browne 2012, 78) turns into a meditation on the impossibility of ever establishing a single identity in any meaningful sense. Everyone has several identities that are fluid and unstable, and everyone has had photographs taken of them that do not look like them, or possesses photographs of themselves that do not look like each other:

but no one is identical even to themselves let alone a photograph if we are many, how can we appear as one

The reference to lips and implicitly to what they can help us to do (chew, swallow, kiss, smile, bare our teeth, talk) is a precursor to highlighting the voice. The narrator suggests that the officials should have listened to Mohamud’s voice because it has a quality that “cannot be concealed/even when peeled, chopped or crushed”. This becomes a self-reflexive comment on the piece itself, in which the voice is sampled and processed, and words and phrases are cut up, divorced from their normal contexts, and put back together in novel ways.

I also emphasised, through images of darkness and confusion, the powerlessness of Mohamud when she was pitted against immigration officials who would not believe her and dismissed her credentials:

The Canadian officials had sent her into an unlit world without documentation, without support, without status the netherworld of fumbled transit with only a flashlight for comings and goings.

Interspersed with the performed text are the passages of screened found text from letters to the Canadian newspapers presented in Odartey-Wellington’s article, which raise the issue of Mohamud’s ethnicity and how this is likely to have prejudiced the officials. White readers of the paper compare their own situation to that of Mohamud and argue they would not have suffered the same fate.

Roger

Collaboration for me is most commonly a real-time endeavour that occurs with other musicians during a live, often improvisatory, performance. But it can also be a compositional and private process that ensues before a work is ready for performance, recording, and presentation. In The Lips are Different, I collaborated primarily in a compositional manner. I provided a segment structure for the piece, determining at which points the austraLYSIS musicians (Sandy Evans, saxophones; Phil Slater, trumpet; and Greg White, computer processing and sound projection) were to improvise in performance, in relation to the ongoing speech and image. I myself made no sound during the performance of the piece, since I was concerned with the digital animation and processing, as well as its sequencing.

Hazel drafted the text and sequenced the screened newspaper quotations in relation to her own text. After we had discussed this, I created a sectionalised score, containing all the text, and estimates of appropriate timings, for us to elaborate. The decision to use a computer-generated voice for the Canadian official was fulfilled with a standard voice synthesiser designed to represent a Canadian accent: a stereotype that seemed appropriate to challenge the power position of the border official. This choice immediately suggested that some other vocal sounds should be digital also, an approach we have pursued in depth in previous works. Here a small number of words and phrases from the chanting passage quoted above (and in addition the words “the lips are different” and “imposter”) were recorded by Hazel for live projection and transformation by Greg. This allowed us to interrogate those words, their implications, and the identities of the speakers of some of the words. It also allowed us to present the words (which were sometimes broken down into less distinct phonemes/vocalisations) with elements of rhythmic sub-division. My score guidance for Greg suggested, for example: “The line is the unit but may be broken up a little. Suggest random timings and duration gaps. Some words at least to remain decipherable even if there is transformation ongoing (i.e. either the transformation is modest, or it comes in and out).” On an affective level, this added to the impression of the overbearing and confusing situation Mohamud had to confront.

We conceived that the displayed text could enter various kinds of dialogue (with the musicians, image and spoken text) so the different elements would at different times both support and question each other. First there is the alienation between the Canadian official’s spoken text, and Mohamud’s responses (shown onscreen). Then there is the repeated contrast, even conflict, between the text Hazel performs, and the voice manipulation undertaken in Greg’s digital processing. And lastly, because the newspaper quotations are discrete texts, and reasonably long, we decided to present them on screen, providing enough time for them to be read fully, and digested, but accompanied by changing visual image and animation. The first such section occurs at about 2 ½ minutes into the piece, and shows the Toronto Star reader’s letter from 2009.

The images are all chosen to represent Mohamud or environments and information pertinent to her story. We chose not to create a fixed image narrative, but rather to present the images in randomized sequences and combinations. After a variable length of display of an image, it is subject to algorithmic processing to create a kaleidoscopic array of continually changing subsections, overlays, enlargements and recolourings of the original, in many cases challenging its own identity, nature, or the factual information it presents. The animation suggests that images, or certainly their perceptions, are intrinsically unstable. By implication, their explicit content can also be changing if not unreliable. Ironically, an image of DNA sequences, again transforming with time, can also become an image of instability: it is in a way disturbing to think that DNA sequences, 95% or more of which are essentially shared between most humans, can yet be the strongest identifier of an individual. And in this case, that it is only after the use of such sequences that Mohamud is found believable by Canadian authorities.

The musicians, particularly saxophone and trumpet, have score indications as to where during the progression of the piece they should play, and some guidance as to their relative dominance in relation to other ongoing sounds. They start at 2’30”, and generally are co-equal with, or slightly lower than, the other sounds in the mix. The ‘unlit world’ of Mohamud, and of our images and sounds, diminuendos at the end to nothing.

A piece like this cannot be successfully realized without musicians with specific experience of relating their improvisation to the particularities of the piece. In this case they needed to respond to both the words and the randomized sequence of the images as well as their processing. To this end, austraLYSIS has cultivated these modes of performance over many years, and periodically we engage in internal workshops in which we try out and evaluate possible new approaches for relating sound to image or text. For example, we have on occasion set out to maximise the distinction between the way we improvise as an ensemble normally, and the way we do so when interacting with spoken and/or displayed text. Some of the exercises/ approaches we have considered in this context are shown below (Table 1). Several of these processes can be heard in operation in The Lips are Different, which I consider a successful example of their application.

Table 1. austraLYSIS: Some Simple Ideas on Displayed (or Spoken) Text as an Improvising Stimulus

Purpose: maximising the distinctiveness of what we do with text from what we do otherwise

We can consider this at least two ways:

1. Possible relationships between sound and text
   • Suggest/imitate/work with meanings in terms of pitch, rhythm, timbre, dynamic, structure
   • Work against the meanings
   • Run parallel to the words but without coordination or imitation
   • Work with/against the sounds of the words
   • Mould stylistically with the text: e.g. move into other genres of music, employ cross-cultural elements, popular music references
   • Pastiche or quotation to suit the text
   • Think about words spatially (e.g. adjust your position in the performance space)
   • Voice contours/timbre translated into sound
   • Make textures in relation to the electro-acoustic text derived sounds (if present)
   • Use environmental sounds (or their imitations) where relevant
2. How to dissect texts so as to obtain a stimulus translatable into an improvised sonic process: consider
   • Individual (key) words:
     their sound
     their meaning
     their position in some continuum (e.g. degree of realism; degree of probability). This could translate (abstractly) into a position on a sonic spectrum that you feel like using;
     their translation into another language and its sound or appearance;
     non-words: using any of the above ideas more loosely, considering their nearest word counterparts and the relation between them
   • Key Phrases/clauses:
     As for individual (key) words, plus
     the contrast within the phrase
     the shared aspects of the phrase
     metaphor and its relatives
     rhythm in which the phrase is/would be enunciated
   • Larger text units, like a paragraph:
     All of the above, plus
     Is there a process? A rational argument, an irrational path?
     Is there a conclusion to the process?
     Is there a spatial interpretation of the process, which could become a sonic spatial process?
   • Whole texts:
     All of the above, plus
     Is there a gist? Can that gist be treated in any of the ways listed above?
     Are there extreme contrasts within the text that might be a sonic focus?
     How does the text cumulate with time: especially if it has not been seen previously?
     Can one represent the progress of the argument/path of the text as a whole, in a summary fashion?
     Is there a narrative that could be used that way?
     Picking different aspects to use sequentially?
   • Spoken Texts:
     All the above, plus the prosody (pitch and emphasis contours) and loudness or spatialisation profiles.

On listening closely to the musical performance, one can hear some of the processes in the table. For example, the two main instrumental timbres achieved by Sandy Evans (saxophone) are far more akin to some African wind instruments (Mohamud was born in Africa), than to the normal sound of the saxophone. Phil Slater in turn merges his trumpet sound somewhat into that of the saxophone. And there are several points at which all three musicians compound the rhythmic pressure of “checked, monitored, tested, assessed, valued, judged ….” and its consequent disturbing, disorienting effect. Everything is questioning, evanescent, unstable in the sound field, just as in the images and the ideas.

References

Browne, Simone. "Race and Surveillance". Routledge Handbook of Surveillance Studies. eds. Ball, Kirstie, Kevin D. Haggerty and David Lyon. London and New York: Routledge, 2012. 72-80.

Budzianowski, Paweł, and Ivan Vulić. "Hello, It's GPT-2--How Can I Help You? Towards the Use of Pretrained Language Models for Task-Oriented Dialogue Systems." arXiv preprint arXiv:1907.05774 (2019).

Dean, Roger T, and Hazel Smith. "The Character Thinks Ahead: Creative Writing with Deep Learning Nets and Its Stylistic Assessment." Leonardo 51 5 (2018): 503-04.

Odartey-Wellington, Felix. "Erasing Race in the Canadian Media:The Case of Suaad Hagi Mohamud." Canadian Journal of Communication 36 (2011): 395-414.

Smith, Hazel. Abstractly Represented: Poems and Performance Texts 1982-1990. Sydney: Butterfly Books, 1991.

---. Keys Round Her Tongue: Short Prose, Poetry and Performance Texts. Sydney: Soma Publications, 2000.

---. The Contemporary Literature-Music Relationship: Intermedia, Voice, Technology, Cross-Cultural Exchange. London and New York: Routledge, 2016a.

---.Word Migrants. Sydney: Giramondo, 2016b.

Smith, Hazel, and Roger T. Dean. Nuraghic Echoes. Sydney: Rufus, CD, RF025, 1996.

---."Introduction: Practice-Led Research, Research-Led Practice —Towards the Iterative Cyclic Web." Practice-Led Research, Research-Led Practice in the Creative Arts. eds. Smith, Hazel and Roger T. Dean. Edinburgh: Edinburgh University Press 2009. 1-41.

Smith, Hazel, with austraLYSIS. Poet without Language. Rufus, CD, 005, Sydney, 1994.