Menu Icon

Arts & Technology: DontFlipMirrorIndents

19/03/2018


First published on Medium, August 2017

 

The phrase appeared unexpectedly within a piece of text that had been copied from an email and then pasted into a word processor. A coder might have recognised it for what it was: a benign fragment of eXtensible Markup Language (XML), a formatting instruction from Microsoft Word unintentionally cut and paste into the original email, hidden from view as it hitched a ride across the internet and now revealed for the first time.

But Jo Lawrence is an animator, not a coder; and so she stared at this strange digital haiku — “don’t flip mirror indents” — wondering what it meant and where it had come from. To her, there was something oddly compelling about the phrase, with its curt command, its skateboarding verb, the semantic mystery of its mirror indents.

Of course, it didn’t take her long to work out what had happened. But the experience — the odd beauty of this snatch of code, accidentally revealed — made a mark. Jo started to wonder: how much of our digital world is invisible to us? What would it take to reveal some of these hidden workings? Could it be beautiful?

These questions were in her mind when she — and eight other artists — were selected to participate in Collusion’s Artist Lab on data-culture in January 2017. The Lab was spread over five days of data-themed workshops, talks from leading artists and academics, making sessions and one-on-ones with the small, highly experienced team; and aimed to give artists like Jo the chance to explore new ideas and working practises.

At the end of the Lab, Jo and another artist, Henry Driver, were offered the chance to take their work further as part of Collusion’s Research and Development (R&D) phase — which provides artists with extra funding and enough creative and technical support to take the seed of an idea through to germination.
In Jo’s case, Collusion had been impressed by the originality of her thinking, coupled with the depth of her experience as a puppeteer, animator and film-maker — and felt that they could help her to do something new.

This process began with a crash course in some of the equipment and tech support available to Jo and Henry at Collusion — which I blogged abouthere. The next bold step bridged the gap between the arts and tech sectors.

In a small, white-walled office in Cambridge’s Science Park, a powerful computer has been set up next to an upright piano. Inside the computer is a neural network — a learning machine inspired by the workings of an animal brain — which has taught itself to distinguish between four different genres of piano music: baroque, classical, ragtime and jazz.

 
The network, nicknamed The Aficionado, was built by a team from Cambridge Consultants — one of the city’s longest-running and most successful technology firms — and is, in an understated way, a vision of the future. Artificial Intelligence, or AI, is already having a marked effect on everything from healthcare to manufacturing to transport; and ‘deep learning’ neural networks like The Aficionado, which can learn to complete tasks on their own, without the need for external supervision, are set to take things much further.

Normally, such innovations would be locked up behind closed doors and Non-Disclosure Agreements (NDAs), far from the inquisitive gaze of artists like Jo. But Cambridge Consultants is an official partner of the in_collusionprogramme, and had agreed to a groundbreaking collaboration with artists, in which they’d supply technical and design expertise in the service of bold new artworks. Jo was the first artist to team with them — and so, one afternoon a few months ago, she sat in the office and watched intently as Cambridge Consultants’ Dominic Kelly, the Aficionado’s chief engineer, demonstrated the system and explained its workings.

A talented amateur pianist from his team sat at the piano and played a burst of Scott Joplin’s The Entertainer. A bar graph flickered on the computer’s display, rising and falling as the artificial neural network listened. Moments later, it classified the piece — quite correctly — as “ragtime”. The pianist then played the opening of Beethoven’s Moonlight Sonata, and, after a moment, the display’s readout changed to to “classical”.

Dominic, Jo and Rich Hall, Collusion’s technical project manager, then discussed a handful of ways in which The Aficionado could be used as a tool within Jo’s filmmaking, as well as the level of back-end support Cambridge Consultants could offer her.

Jo went off to ponder. “I remember being on the train home,” she told me, “and thinking: what a gift! The meeting had given me an amazing opportunity to explore this long standing and persistent visual idea of mine: of presenting data as a physical analogue substance that could be animated with puppetry and music”.
Working quickly in her sketchbook, she poured out data-based ideas. “My initial thoughts,” she said, ”were of creating a series of little animated universes, each one differently stylised — which captured the quality of each music genre, but soon I felt that maintaining the continuity of a single narrative thread would be far more interesting.”

A month later, she returned for another R&D session with Dominic and Rich, armed with storyboards, models of puppets, beautifully rendered images of backgrounds and a working title: Datacosm. Her idea, she explained, was to make a series of four equal-length animations, each one following the same basic narrative — an unsettling story about the consequences of data loss — but from different perspectives. Code created by Cambridge Consultants and Collusion would link each of the four animations to one of the four outputs of the Aficionado — jazz, ragtime, classical, baroque — meaning that, in performance, a live pianist would ‘edit’ the film in real time, cutting between animations simply by changing genre on the piano. Because of this, no single performance of the film could ever be repeated — each one would be a unique cut, depending on what the pianist played.

Additionally, Jo had an idea to take snatches of the computer code being run within the Aficionado and incorporate it into the fabric of the animations — both as parts of the background and also as lines that some of the characters could speak or sing.

“I want to explore the differing relationships between the digital world and ourselves as well as integrating the poetry inherent in coding,” she told them, “but — is any of this possible?”

Yes — it all is — came the reply from Rich and Dominic.

Buoyed by this meeting, Jo is currently working on the animations for a live 30 second demo of Datacosm, and, with Collusion’s help, looks set to have the kind of springboard she’ll need in order to apply for more funding.

If and when this comes to fruition, Cambridge Consultants will be able to call on a powerfully creative demonstration of their product. Collusion will have achieved a major goal; and, most importantly, Jo will have been given a creative boost that wouldn’t have been available to her otherwise.

We’ll leave the last words to her.
“This whole process has had a quietly profound effect on me. Having glimpsed the potential of working with AI, as well as a brief taste of VR and AR during the data labs, I now feel utterly compelled to continue exploring this area — even if it means changing my practice entirely and moving away from linear filmmaking.”

 

This is the second in a new series of Medium articles from Collusion, written in_collusion with Pete Naughton, a Cambridge-based journalist and audio producer, who has spent the last ten years writing about radio, podcasts, technology and music for the Telegraph newspaper. 


Collusion is colluding with a lot of Creative Associates, just like Pete. If you’re working in a creative industry, or if you have a talent for working with technology - or both - check out our homepage for regular updates. And if you’re interested in getting involved in Collusion, get in touch here - or come to one of our Meetups in the Greater Cambridge and Greater Peterborough areas.