by Leo(Laewoo) Kang, Feb 2017

Intermodulator is a sound responsive installation that produces diverse patterns of moire image through participants’ collective and improvisational sound engagement. The system receives and analyzes the frequency and volume of different live sounds, and map them to the speed of the fans and the brightness of the backlight. Current installation is comprised of eight different modules of the fans, and each module is assembled by two fans that oppositely face to each other like a sandwich (Figure 2).

Each side of fans is connected to two different sound inputs (mic A and B in Figure 1, 2), and this module assembled with an incandescent light bulb produces diverse combinations of fan images by responding to participants’ sound (or music) engagements. When a certain resonance and tension are achieved between these different inputs, the installation produces diverse patterns of moire, which is visionary illusion caused by mutual ‘interference’ of two oppositely rotating objects or visual images.

Figure 2: One module of Intermodulator responding to two different sound inputs. The speeds of individual fan are mutually affected by each other’s sound.

Like a seesaw, which requires more than one person to appropriately enjoy it, its design of the system naturally and implicitly encourages people to use this installation in a collaborative format. This system also can be extended to multi-channel inputs by incorporating an audio-mixer through which more than two people can interact with the installation.The following videos show its alternative version where eight participants collectively improvise with the installation.
By providing a heuristic space where participants can produce interesting audio-visual performances through collaborative improvisation, this installation allow people to empirically explore the nature of collaboration and improvisation that promotes group creativity and learning.

This installation was built in the process of our art-based research project titled ‘Intermodulation [Under review]’ where different musicians, visual artists and HCI researchers collaborated to produce diverse audio-visual concerts from 2015 to 2017 in NY, USA. The main goals of this research were to understand the key features of collaborative improvisation that enable creative learning, and to explores the value and possibility of such artistic practice as a mode of HCI research and inquiry.

In this project, the author was engaged primarily as a multimedia artist, providing interactive artworks to accompany the performances of electronic and experimental musicians. Along with this art making study, we also had interview-based study on multi-media artists and musicians, including participated musicians, who engage processes of improvisation for producing their creative works.

One key lesson the author learned from this research is that collaborative improvisation has essentially ‘inter-dependent’ nature where participants actively listen to each other in the situation, and mutually adjusting their tunes depending on other’s play. Especially, good improvisation, which promotes creativity and learning, is not only achieved by building harmonic and stable relationships between those who share similar interests, but also by exploring ‘tension’ between similarity and otherness, which may periodically give rise to kinds of ‘interference’ or ‘disturbance’ to each other.

Like two oppositely facing fans that mutually interfere each other’s direction, like moire image that appears in tension of interdependent engagement, such lesson on collaborative improvisation conceptually and technically inspire the design of Intermodulator.

'The Cutting' by Powerdove with Sarah Hennies

바람을 타고 찾아 온 당신, music and installation

Collaborative Improvisation in Human-Computer-Interaction
Derived from a Latin word meaning “unforeseen”, improvisation refers broadly to the practice of composing or inventing extemporaneously, especially in the art and music fields, as a way of producing more effective, open-ended, and sometimes participatory aesthetic and creative outputs [10]. Outside the worlds of musical and artistic performance, the language of improvisation has sometimes been adopted to underscore the indeterminate and evolutionary qualities of everyday human behavior, and as such has begun to challenge and inspire work across a broad range of disciplines from anthropology, economics and cognitive science; to architecture and urban planning; to information and computer science [8].

In HCI and design, some theorists have emphasized the ‘situated’ and ‘circumstantial’ character of human action and its dependency on emergent material and social contexts in the use of systems and their design. For example, Suchman [11] describes how circumstances co-create intelligent action by rejecting plan-based models of human cognition then prevalent in Artificial Intelligence and HCI. Agre [1] builds on this insight to connect questions of determinate planning and action to the indeterminate properties of situations, which remain complex, non-transparent, never fully representable, and therefore genuinely uncertain. Dourish et al [4] build on these understandings of improvisation to argue they apply to the everyday action of designing itself.

These theoretical insights also have been mirrored in turn by diverse methodological explorations in HCI research and design including research through design [12], critical making [9], and meta-design [6]. These methodological suggestions commonly highlight the advantages of learning through open-endedness, situated engagement, and artistic collaboration. In addition, recent studies have begun to suggest ways in which learning from these improvisational methods can be made more accessible and generalizable forms of knowledge [7], both through processes of deliberate documentation and reflection [2], and the consideration of a range of intermediate artifacts [5]. However, there is still limited understanding of how improvisation works as a mode of research, and what features of improvisation enable learning and creativity. For extending such limitation of improvisation for HCI research, our research team have studied this artistic activity through theory, ethnography, and collaborative artwork.

Technical Details and Settings
The system of this installation is built by assembling diverse DIY electronic components. The PSSR/ZC Tail has been used for safe control of 120vac incandescent bulbs through zero-crossing detection. For analyzing the frequency and level of individual input sounds, Sparkfun’s Spectrum Shield has been used. The speed dials in the box fans have been hacked and connected to four channel relay shields. These electronic components are connected to and controlled by the Arduino-based microcomputers. More than 2000W at 120VAC is required to run the whole installation.

1.Philip E Agre. 1997. Computation and Human Experience.
2.Jeffrey Bardzell, Shaowen Bardzell, Peter Dalsgaard, Shad Gross, and Kim Halskov. 2016. Documenting the Research Through Design Process. In Proceedings of the 2016 ACM Conference on Designing Interactive Systems
3.M Csikszentmihalyi. 1997. Finding flow: The psychology of engagement with everyday life. Basic Books.
4.Paul Dourish, Annette Adler, and Brian Cantwell Smith. 1996. Organising User Interfaces Around Reflective Accounts. In Reflection’96, 235-244.
5.Bill Gaver and John Bowers. 2012. Annotated portfolios. interactions 19, 4: 40.
6.Elisa Giaccardi and Gerhard Fischer. 2008. Creativity and evolution: a metadesign perspective. Digital Creativity 19, March 2015: 19-32.
7.Kristina Hook, Peter Dalsgaard, Stuart Reeves, et al. 2015. Knowledge Production in Interaction Design. In Proceedings of the 33rd Annual ACM Conference Extended Abstracts on Human Factors in Computing Systems - CHI EA ’15, 2429 - 2432.
8.George E. Lewis and Benjamin Piekut. 2016. The Oxford Handbook of Critical Improvisation Studies, Volume 1.
9.Matt Ratto. 2011. Critical Making: Conceptual and Material Studies in Technology and Social Life. The Information Society 27, 4: 252-260.
10.R. Keith Sawyer. 2000. Improvisation and the Creative Process: Dewey, Collingwood, and the Aesthetics of Spontaneity. The Journal of Aesthetics and Art Criticism 58, 2: 149.
11.Lucy Suchman. 1987. Plans and Situated Actions. Cambridge University Press: 224.
12.John Zimmerman, Jodi Forlizzi, and Shelley Evenson. 2007. Research through design as a method for interaction design research in HCI. In Proceedings of the SIGCHI conference on Human factors in computing systems - CHI ’07, Paper 41.