Concept:
An airport transit area can be a pretty boring space if one is waiting for a delayed flight. There are some services to entertain passengers - shops, bars - one can even buy a magazine. But in the end, that’s about it. The Trolley Singers project is addressing this issue by trying to develop an experimental interface on the borders between art, technology and sociology. The goal of this project is to make the transit area more playful and establish some communication among bored passengers.
The small carry-on luggage trolleys in the transit area of a large airport will be equipped with a small device in an effort to prompt mutual recognition and communication via voice – or more precisely, through singing. When the trolleys meet within range approximately 5 meters, they start to talk. They exchange information about their mood, energy level, and temperament, and then update their own status according to the received data. They even start singing a song together when they fall in love.
The trolleys act like artificial pets attached to their owners. They fill the airport hall with a constantly evolving spatial sound structure.
Simulation:
The
basic simulation
(requires Macromedia Flash plug-in and pop-up windows allowed) shows possible behavior in an exhibition space.
The simulation is very simple - the trolleys merely communicate emotional status (expressed within a 9 level range). Talking makes them gradually happy, while lack of communication makes them sad. The final setup will be much more complex, and sounds will differ, but certain aspects of the aesthetics and inspiration will remain.
The spatial and decay sound factors are simulated on a very basic level.
Possible development platform:
There are two possible platforms for this project - a PDA including a microphone and a speaker, or a smart mobile phone. The internal speaker and microphone will probably be good enough for this purpose, however it is a matter of experimentation to confirm this. The electric power is the main technical problem of this project - the final installation setup will run approximately 20 hours with 4 hours for recharging, so the whole system must be very efficient and power saving.
The PDA would be Linux based. A DSP and logic would be programmed with Pure Data. A smart phone running on the Symbian operating system would be programmed with C++; easier options should be considered (Java, Python) although results produced by them will probably be too slow.
The communication is based on special sounds - a unique frequency mixture (like birds or DMTF tones) in order to avoid sensitivity to surrounding noise. Each unit is also capable of recognizing and interpreting these sounds - and of seperating them from the background.
Perspective:
The Trolley Singers is an experimental art project based on the cutting-edge of commercially available technology. It focuses on swarm intelligence and the social aspect of mobile computing. Although the concept is simple, the entire setup, which including many units, possible programming bugs, hardware caused delays and unpredictable audience participation can create a relatively complex system, which will hopefully generate some unexpected, aurally interesting sound structures. The research in this area can also lead also to interesting applications in the field of mobile computer games, communications, etc...
Research:
The hardware and software research proved that a Java MIDlet running on an average smart mobile phone would be capable of doing the job - at least in a simplified version. The advantage of such a solution is obvious: the code, once developed, can be used on various devices equipped with a microphone, a speaker, a and Java (J2ME) environment. This adaptability is great for any future development.
Nokia N70 has been chosen for the realization for several reasons:
It was smart phone running Symbian operating system - so it offered flexibility
It had a sufficiently fast processor.
It was not a brand new model (Fall 2006), so it was already relatively well documented.
Last but not least, the price was nice.
Recording is possible thanks multimedia API. Generated sound is periodically recorded into ByteArray in the operating memory, to allow for further analysis. There is still a particular problem with stability to be solved. The memory gets full within approximately 2 hours (probably it is just fragmented, because the ByteArray is being rewritten again and again) and the MIDlet then crashes. It is most likely a solvable problem; it just needs more special mobile phone programming skills. There are also limitations concerning sound quality: The mobile phone has a very efficient filter implemented in its recording device. It greatly supresses background noise. Experiments proved that only the loudest sounds are recorded, while all the other sounds are eliminated. This limits the possibilities of further analysis: a signal is received only when it is louder than any other environmental sounds. The code will work only in a less noisy environment, and close proximity between communicating devices is necessary.
Sound analysis with the Goertzel algorithm (FFT also tested sucessfully) works fine with regards to the limitations listed above - it recognizes a single frequency as long as it is significantly louder than the background noise. It is impossible to check dual or more complex frequences, because they are not present in the recorded material (one is always eliminated by built-in filter).
Pattern recognition was not implemented because of limited financial resources.
Conclusion: Due to the built-in filter, communicating with DMTF tones seems to be impossible. However there is still the possibility of using single sine frequencies and pattern recognition. Communication would be possible in a quiet environment and within close proximity. It would be vulnerable to any background loud sounds.
Installation:
The Trolley Singers project was exhibited as a part of show "Interface and Society" at Hennie Onstad Kultursenter in Oslo in November 2006. The piece consisted of airport trolleys equipped with mobile phones (Nokia N70)was located in the foyrer.
Bluetooth communication was choosen as a temporary solution for the exhibition installation because of the difficulties with pure sound communication, resources and time shortage. The phones were wrapped in aluminium foil to reduce bluetooth antenna range to approximately 4 meters. The simple MIDlet was written to check the proximity of another phone, exchange an information and play sounds. The bluetooth communication was not found ideal for the purpose - the bluetooth connection setup is generally slow and not very reliable (at least on N70 phone), radio waves reflections in the foyer caused another troubles. Interestingly, not all the phones worked the same way - some were better while the others were worse. To achieve some acceptible degree of reliability the number of trolleys was reduced just to phones which were working fine.
Behaviour was very interesting: thanks to all the glitches, the phones acted a bit unpredictable - like pets, they communicated when moved close to each other, but sometimes they refused to do so. Occasionnally, they even called each other on long distance (thanks to great radio waves reflections). It was not necessary to simulate any emotions at all, the phones even had their own characters as mentioned abowe. To our opinion, the audience was a bit confused by a such behaviour. People expect an old school interaction model, the piece which is busy with itself and doesn't care about them seems to be difficult to understand. It would be definitelly better to achieve simplier, more predictable interaction in case the work was installed in a public space.
Aesthetics is another aspect to think carefully about. In a public space, a communication sounds should meet some criteria: they must be pleasant to most of people present there - which, in fact, means that sounds must not affect a communication between people (they need to operate in another frequence spectrum then human voice does and/or they must not be louder then any normal verbal communication) and they should't be disturbing. Moreover, in our case, the sounds must be easy to play with N70 speaker. Hi pitch sounds with a fractal structure meet such a criteria. We have tested several different sounds, some of them close to music. The problem was that audience tends to associate all the music like sounds to mobile phone ringing tones. People are aware and waiting who will pick up the call. In case nobody does, people start uncomfortable and nervous. There is first imperative: the communication sound must not resemble a mobile phone ringing tone. The second imperative is related to repetition: the sound must not be felt as repetitive. All the sounds based on a steady rhythm are felt more repetitve then fractal sounds. Any kind of nature related sound like bird singing, cricket sounds, water sounds etc... or synthesized sounds of that kind would work on the contrary to most of musical sounds.
Exhibition text
Video:
Realization team:
Hai Nguyen Dinh
Henrik Sundt
Petr Svarovsky
Development hardware:
Apple PowerBook 15', 512 MB RAM, 60 GB HDD
PC Pentium IV, 2 GHz, 256 MB RAM, 60 GB HDD
Nokia N70 (12 pieces)
Development software:
Mac OSX 1.3, X Code, Mobile Processing, MPower Player, Audacity
Windows XP, Java SDK 1.4, WTK, Mobile Processing
Nokia Symbian v.3
Installation hardware:
Nokia N70 (12 pieces)
Installation software:
Nokia Symbian v.3, custom made Java MIDlet
Code samples:
Sound code
Bluetooth code
Codes are in Mobile Processing format (originally with .pde suffix, check http://mobile.processing.org for more info, please) with some J2ME parts.
Year of creation:
2006
Further development:
There are several problems to solve on the way to functioning audio communication between mobile phones:
1. Stability
So far the phone crashes when its memory is full. It should be possible to record into ByteArray again and again without the system crashing. There are already some J2ME MIDlets that do the same, at least with images (e.g. vOICe).
2. Pattern recognition
If the Goerzel algorithm works reliably (as it now should, within a quiet environment without background noise) then the pattern recognition implementation should be easy.
3. Ultrasonic communication
It would be interesting to check proximity using sounds within the inaudible range of the sound spectrum. It should be possible to run the Goertzel algorithm to check frequencies at around 22 kHz in the case that a 44 kHz sampling rate works. This could, however, create other problems, e.g. Doppler effect. The idea definitely requires further testing.
There are several ways to go, for example:
1. Installation in a public space
Installation in a public space, e.g. an airport, where hardware will be supplied by the audience (everybody interested in using the program will put his/her mobile phone into a slot in the trolley and the software will be available for download via WAP. Such a solution can create a situation in which thousands of units will be participating at the same time, creating quite a rich sound structure with minimum investment.
2. Street game
Wherein sound serves as a communication protocol for players.
3. Using a simple communication protocol
A simple communication protocol could be used for mobile phone games, instead of the currently popular Bluetooth, which is is available on just a small percentage of mobile phones.
Thanks:
The Trolley Singers project would be impossible without the help and participation of many people. We would like especially to thank:
Atelier Nord for financial support, production and organization help
Norwegian Culture Council for financial support
Oslo Airport for sponsorship
Staff and students of Oslo Academy of Fine Arts for an inspiring environment
Artists associated with Atelier Nord for inspiring ideas and feedback
Petra Vargova for intellectual support and inspiration
Laura Beloff for many inspiring ideas and intellectual support
Erich Berger for inspiring feedback and technical support, especially for directing us towards the Goertzel algorithm
Martin Pichlmaier for help with J2ME code, especially sound recording and playback
Francis Li for great Mobile Processing software and quick help when we had troubles
Monika Goodwine for help with all the difficult papers and installation help
Jana Winderen for excellent production and installation help
Siri Augsteen for great exhibition production
Madeleine Park for exhibition maintenance and posing for video documentation
Anja Brinich for help with texts
Emil Hogset for nice photos
