Rainbow Cat in Wonderland

Rainbow Cat in Wonderland was constituted along with a workshop space, an artificial flower garden and an isolated dark space blocked from light. The isolated space resembled a maze in which audiences moved around.

Fig 1. Realization Plan of Mixed Reality: Rainbow Cat in Wonderland

To experience Rainbow Cat in Wonderland, audiences listened to a story of seven characters that represented each sense—sight, hearing, smell, taste, tactility, kinesthesia and balance.[1] Each character had one disability. For example, a cat who could see very well, could not hear well. In this story, I and the playwright focused on breaking down the hierarchy of senses and located them side by side in a horizontal line. Giving an open-ended literary story to audiences, we tried to provoke their poetic imagination. Listening to the story, the audience naturally associated themselves with characters and they were encouraged to create their own characters from their imagination.

The methods of creation were varied from drawings to making sculptures. For blind children, we provided clay, sand and wire, so they could recognize the forms of their creations with touch. While creating the avatars with their own methods, which required their action such as drawing, painting and making, the avatars obtained the participants’ bodily traces. In this course, participants came to have more emotional attachment to their avatars and the avatars got to have the creators’ personal characteristics. Here, the boundary between creator, artist and audience is blurred. This mixed reality stage is not a place where audiences merely appreciate the artworks, but one where audiences tell their story and express their imagination.

When the work was done by participants, four web cameras captured the images of the created figures. With a coded program by a new media artist, SeungJoon Choi, each figure was transformed into a 3D figure and took on a virtual life by moving around on the screen. These moving avatars were projected onto walls of the maze with four high-beam projectors.

                                              Fig 2. Process Plan for Rainbow Cat in Wonderland

Before entering into the maze, participants passed by an artificial flower garden, which was made of microphones. As a participant talked in front of a flower, leaves and petals of the flower started fanning by an electronic motor, which reacted to the human voice. Through this process, the computer could record the participant’s voice and the virtual avatar acquired the voice of its creator. Here, participants had a multi-sensory experience. Their voices were transformed into artificial wind. Speaking and hearing were another experience of tactility.  Especially, the blind participants actively engaged with this work. They recognized the sound they made would bring about changes of atmosphere with their touch sense. When they spoke slowly in front of the flower, the artificial wind blew also slowly, and when they sang a song, the wind continuously blew to their faces changing its strength.[2]

Fig 6. A blind child is experiencing the work in the left picture. – Installation View: Artificial Flowers of Rainbow Cat in Wonderland

Passing the artificial flowers, participants entered into a maze in thick darkness. They could perceive the walls of the maze only with their tactility and olfactory sense, as it was dark and the maze had a specific odor of industrial rubber. Here in this maze, there was no difference between blind children and sighted children. Needless to say, while many of the sighted participants were afraid of entering the thick darkness, the blind participants were calm and easily located themselves on the path of the maze. As the (abled) participant became disoriented in the darkness, he/she noticed a faint sound of his/her own voice. Gradually, the sound became clear and oriented the participant’s sense of direction. Following the sound, he/she found a gleaming figure of the avatar he/she made. The blind participants could not see the gleaming figures, nevertheless they noticed the sound and were able to follow it. It was important for me in this step to cut off the visual sensation of participants, even if for only a little while, in order to make them perceive the environment with their other senses as blind children do.

Fig 7. Installation View: Interactive Maze of Rainbow Cat in Wonderland

When the participant tried to get close to the avatar, it ran away. It slipped and slipped, jumped and jumped. The artist had put an apparatus here to make participants play with their avatars. Even though the avatars were virtual, participants did not recognize they were fictional, and reacted to them as if the avatars were alive. In this mixed reality of virtual information and real space, playing (hide-and-seek) with the virtual avatars they had created and vitalized, the participants identified themselves with the avatars via their imagination. Here, the mixed reality stage challenges the boundary of perception and illusion. Florella de Rosis once described the goal of showing a humanlike embodied character in human-computer experience as enhancing the user’s believability. Users suspend their disbelief that what is in front of them is something different, instead they recognize the computer-generated characters as friends.[3] In Rainbow Cat in Wonderland, by enabling audiences to create their own characters, I tried to intensify familiarity between audiences and the avatars. Indeed, the audiences showed strong attachment to their own avatars. When they entered the mixed reality stage, the first thing most audience members did was find their own avatars. Humberto Maturana comments on this process, as Hansen cites: “in our experience we cannot differentiate between what we call a perception and what we call an illusion. Whenever we have an illusion, we experience it always in the same way as we experience what we are used to calling perception.” Hansen goes on, “the experience of illusion and of perception are affectively identical: from the standpoint of the experiencing, feeling body, simulation and perception are quite simply indiscernible.”[4] Of course, I do not mean that digital reality is essentially identical with material reality; indeed, we are aware the digital avatar is not a physical object, but only a simulation constructed with numeric codes. In the course of making interaction with the avatar, however, participants act and react within the mixed reality artwork, responding the virtual avatar’s movements. Participants try to catch the avatars, or run after them. Although they all acknowledge that the virtual avatar is only a surface, participants are affectively attached to them, as well as physically react upon the avatars.[5] As audiences move back and forth between the boundary of perception and illusion reflecting themselves onto their avatars, “inter-subjectivity[6]“ is generated between the virtual avatar and the audience as well as among audiences in the same space.

[1] I referred to the Korean notion of sense for this workshop. See, p. 30.

[2] Indeed, this work gave me a very special experience as a curator. One day, a blind child, Jiah, visited the exhibition, but she did not participate in the workshop program. After wandering around the exhibition site for a few minute holding her mother’s hand, she stopped at the artificial flowers. She sighed. The flower started fanning a little bit. She spoke. The petals of the flower were fanning longer. She shouted with joy that “Mom! Mom! Can you see? I made something with my voice. I can feel the wind. I will make the wind again.” Then, she began to sing a song called Fly, Fly, the Flight! with a loud voice. She did not leave the work for a while. It must be her first time when she enjoyed an art exhibit by herself.

[3] De Rosis, Pelachaud, Poggi, “Transcultural Believability in Embodied Agents: A Matter of Consistent Adaptation”, ed. Payr, Sabine. Trappl, Robert. Agent Culture: Human-Agent Interaction in a Multicultural World, London: Lawrence Erlbaum Associates Publishers, 2004, p. 89.

[4] Emphasis added, Mark B.N. Hansen, New Philosophy for New Media, p. 168.

[5] Imagine someone who is emotionally attached to one’s virtual pet. Even though the virtual pet is not real, we often see the owner feels desperate when the pet dies. I think this happens through ‘interactivity’ between the user and the virtual avatar.

[6] Alfred Schutz will give a useful explanation for what I mean by “intersubjectivity” here. Schutz explains three terms of “Alter ego,” “Life-world,” and “Intersubjectivity.” Alter ego literally means “the other I” in Latin. In other words, I can find myself in Other. According to Schutz, “it is a coexistent ego in the mode of the illic(there) (Alfred Schutz, “Phenomenology and the Social Sciences,” in Luckmann, Thomas. Phenomenology and Sociology: Selective Readings, 1978, p. 127). In this process reflecting myself to other, inter-subjectivity generates. Michael Jackson elaborates this term of “intersubjectivity” focusing on intersubjectivity between people and things as well as relations between people (Michael Jackson, Things as They Are, Indiana University Press, 1996, p. 28). In the example in the footnote above, the blind girl, Jiah, assumed that she made the wind, but it was made by the artwork, which was reactive to Jiah’s action. For her, however, the wind was a direct reflection of her action, and she found herself in the artwork. In this process, “dialectic” interaction was generated. A meaning that the artist already put out and another meaning, which Jiah created with her voice interwove in some ways and equated a new and better narrative of the artwork. Indeed, these are “moments of a dialectic” (Ibid., p. 28). And these interactions and reflections generated among participants while they moved around in the same space.