Review of Coded Matter(s) #7: Extended Senses

Author: Nina van Doren / Photography: Jessica Dreu

Yes! the 1st of June. We start to feel the approaching summer, a pleasant afternoon sun, a warm breeze. We smell blossoming flowers, barbeques and fresh cotton. We hear birds chirping, bees buzzing, and tatters of terrace conversation. The oh so pleasant spring sensations. FIBER could not have picked a better day to organize its 7th edition of Coded Matter(s) which is all about Extended Senses.

The popular and ongoing series Coded Matter(s) explores new artistic expressions that are emerging from the possibilities and developments in intelligent technology. For this edition we are invited to discover art and research applying this to the human body: Technology used as an extension of the body, embedded into our senses, enhancing our abilities and knowledge, altering the ways in which we perceive the world around us.

The fictionalized concept of a rigid faced man-machine mixture was widespread in science fiction before World War II. In the 1980’s, Arnold Schwarznegger didn’t exactly help to debunk this image with movies like Terminator I & II. Troubling in their origin we can’t help but take a careful approach to these matters. Technological progress can sometimes feel like a threat or disengagement with other important components of humanity, especially when it enters our very bodies. The military application of science can undoubtedly be scary, as moderator and introduction speaker Michelle Kasprzak presents with the pukelight developed by Homeland Security and self-operating systems with advanced face recognition and tracking technology. We don’t want to create armies of indestructible mindless robot assassins, superior to natural human beings that no longer have any purpose to serve the world.

Michelle Kasprzak – moderation & introduction

It’s time to get out of this dissimulated environment, and get real. The military frightens us. Art inspires, and provides much more fruitful ground to start thinking and creating from. What does the World’s first officially recognized cyborg Neil Harbisson do? He’s a visual artist.

We know visualization. We know sonification. It’s the main things we know that artists do. Translating information, in the broadest sense (thoughts, feelings, concepts), into vision or sound. Edwin van der Heide kicked of the first key note and immediately extended the audience’s senses by making them hear some gravity. What? Using satellites as a needle on earth as vinyl, Edwin van der Heide, an artist and researcher in the field of sound, space and interaction, replayed the artwork in which some tunes the world hums. Making information and signals perceivable is what Edwin van der Heiden calls perceptualization. This term was coined specifically for the course Perceptualization, which Edwin teaches at the masters Media Technology MSc programme at the Universtity of Leiden. He gave some great examples of artworks like Save YourSelf: Galvanic Vestibular Stimulation Interface by Hideyuki Ando, Tomofumi Yoshida, Taro Maeda, and Junji Watanabe (2007) and his own Rundfunkempfangssignal (2013) that go beyond visualization and sonification which can be more accurately described and understood with his new coined term.

Edwin van der Heide – keynote Perceptualization

This applied very much to the artworks in the exhibition that FIBER curated. Like the Durr Watch (2012) by Skrekkogle that makes you feel time and the scent experience Famous Deaths (2014) by the team from the School of Communication and Media Design Breda (CMD) for their ongoing smell-research project Sense of Smell.

Durr Watch (2013) by Skrekkogle

I think anyone’s interest will be sparked by a heavy metal title like Famous Death’s. And the idea of a scent printer seemed very novel. Luckily CMD’s Frederik Duerinck and Wander Eikelboom gave a broad introduction to the startling science of scent. Because memory and scent are so closely related in the brain, smell can evoke memories and powerful responses almost instantaneously. Therefore it remains quite mysterious and subjective (not everyone likes the same smells). A human nose can detect more than 1 trillion smells. This suggests the human nose outperforms the eye and the ear in terms of the number of stimuli between which it can distinguish. Will it ever be possible to bottle memories? Distill the smell of cities? The researchers at CMD put their noses on the grindstone and have come pretty far in creating an odor variant mapping method.  All research and developments will be published soon in the book The Sense of Smell. Definitely something to keep track of.

Famous Death’s (2014) CMD

Speaking of memory, did you know Disney had a research lab? We know they have been stimulating our senses since before we could spell our names, and they invest a lot in keeping their name high. Jonas Loh from Berlin based Studio NAND talked to us through Skype about his collaboration with Ivan Poupyrev from Disney’s Research Lab. Botanicus Interacticus is a sensing platform within an interactive installation based on the sophisticated Touché sensing technology. The work has been developed from extensive research around rich and expressive interaction with living plants. Kind of what good old Walt envisioned in the 1950s.

On screen: Jonas Loh (Studio NAND)

The foundation of this Touché technology is called Swept Frequency Capacitive Sensing. By touching the plant, light or sound are possible responses. The instrumentation of the living plants is non-invasive, and does not damage the plants: it requires only a single wire placed anywhere in the plant soil in order to respond to human touch. Soon the research outcome and the developed technology and code will be made available by Studio NAND through an open-source platform.

So if we can listen to plants and interact with them, what do they have to say? Sebastian Frisch, a German artist, musician and researcher, enabled the audience to eavesdrop on corn sprouts. With his work Biophonic Garden which was part of the exhibition, a headphone was plugged into a water filled tank containing the sprouted seeds. And the sprouts said… well, I’m not sure if I would be able to accurately transcribe it for you.

Biophonic Garden (2013), Sebastian Frisch

Marije Baalman, another artist and researcher invited for this Coded Matter(s) edition, made us gain a deeper understanding of the subtleties in plant behavior with her work Sonobotanics 2007 that was beautifully laid out in the Tuinzaal of the Brakke Grond. A computational model is based around a parametric Lindenmayer system, with elements for the plant’s roots, stem, leaves and flowers. Using real-time sensor data as inputs to this system the model calculates how the environment changes the internal structure of the plant.

Sonobotanics (2010), Marije Baalman

However exciting, we must understand that in order to extend our senses, they need to work in the first place. Which brings us to the ancient human body business: Healthcare. Walter de Raedt, the third keynote speaker of the day introduced a wide variety of different artificial high-tech applications currently or nearly available for human benefit. Sometimes our bodies give us signals that we don’t pick up, or choose to ignore. De Raedt works with his research institute IMEC on highly sensitive, intelligent applications to monitor these signals take action, or warn us when needed. As we know, there are tons of smartphone apps on the market that can send out reminders to drink more water, where you can fill in what you ate that day to find out that you are an unhealthy bastard. Will this really change your behavior? De Raedt stresses the fact that these applications should be small, ultra low-power, comfortable and easy. Possibly invisible. Implemented under the skin. And he’s right because, lets be honest, are you really going to manually log that FEBO burger in your Nutrition App after a night out drinking when your phone’s nearly out of battery?

Walter De Raedt (imec Leuven) – keynote about Body Area Networks

But what are the implications of these invisible networked technology as they are stuck onto or into our bodies and deployed as applications within our environment and smartphones? Who will benefit from the collection of big data in the long run? Do we actually benefit from these technologies or are we becoming more and more the ‘extended senses’ for big corporations like health and insurance companies? And will eating that burger also trigger the idea that we’re tracked and logged into a database, like we nowadays do when we search something on Google Search? These are some serious privacy issues and we don’t know the implications yet.

Earlier on the day Kees and Joost Plattel suggest we should learn about our own data and the possibilities to track and trace our body data. They’re working on a DIY platform called Thought Kinetics to collect and share numerous sources of body data: from brainwaves to temperature fluctuation and hearth rhythm. By doing this ourselves, with cracked and reworked technology, we can get a better insight in the personal and artistic possibilities of sensor technology.

The final sweet keynote by Bart van der Put also known as Bartaku, questions mankind´s eternal struggle for energy. In his research he explores the micro and macro realms of photovoltaics: the conversion of light energy into electrical energy. Here we are really coming to terms with the false assumption of technology being used to subdue nature and bend it to human will. Bartaku shows us that we can in fact learn a lot from other organisms such as the agave, a true ‚power’ plant.

Arthur Elsenaar ended the day with his awesome Perfect Paul performance; addressing the freedom of facial expression. Our body is capable of much more then we think, but our system can’t operate these possiblities. A computer does. This threw the sci-fi image of the rigid faced cyborg right out the window. A living breathing animated .gif that was able to answer all questions of the startled audience afterwards about how on earth this worked and how it felt. I forgot to ask: Arthur… does your face hurt afterwards?

Perfect Paul (2012) – lecture performance by Arthur Elsenaar

FIBER’s Coded Matter(s) #7 holds so much potential for the future. They could have devoted an entire edition to each of the senses alone. And they just might in the future. They’ll be back. (after a summer break). So stay in Touch. Smell ya later.

Black Box (2013), Christoph de Boeck (overtoon)

FIBER would like to thank De Brakke Grond, all the visitors, participating artists and the Creative Industry Fund NL for making the seventh edition happen. If you like to receive updates about Coded Matter(s), please subscribe to our newsletter.