How Ruairi Robinson used AI to dance with death

A present-day reinterpretation of the Danse Macabre (a medieval allegory about the inevitability of death, in which skeletons escort living humans to their graves in a lively waltz), this captivating promo for synth music artist Steve Moore's track The Blue Stone was created entirely using AI by Joyrider director Ruairi Robinson.

Realising the project would be far too expensive to make using live action production, Robinson ditched the traditional filmmaking process of treatments, scripts and storyboards, instead diving straight into the all-consuming world of AI to bring his unsettling vision to life. 

Transporting us to horrific scenes of war and suffering spanning many centuries, the film addresses the director's own feelings of unease with the state of the world, and the cognitive dissonance we experience when witnessing death and destruction via our phones on a daily basis. 

Fascinated by his entirely AI-generated approach, we caught up with Robinson to find out more about the pros and cons of directing with artificial intelligence, what he's learned from this madcap project, and the surprising backlash his experiment received on social media. 

How did your concept for the music video develop?

I wanted to make a music video that would be so expensive to produce traditionally, no one in their right mind would give me the absurd amount of money I would need to make it. So I made it with AI instead. 

It was more akin to feeding an addiction and going on a 3 day bender. I started with the beginning and ending and fleshed out everything in between.

I first decided to make an image of a dancing skeletal jester, and the rest of the idea evolved from there. It was a recreation of the medieval Danse Macabre, that spans many wars and atrocities throughout history. The guilty are made to acknowledge their actions and forced to dance, while watching the dead they have wronged dance for eternity, until they can't take it anymore. 

The concept didn’t actually change much as the creative process was so fast– I had the idea and just made it immediately. In that sense, it was more akin to feeding an addiction and going on a three-day bender. I started with the beginning and ending and fleshed out everything in between.

Can you walk us through the process of creating a music video using AI?

It was made using mostly AI. I entered all kinds of text prompts and generated multiple responses, running multiple sessions in different tabs at once. I saved the shots that are good, adjusted the prompts if it wasn't producing what I was looking for, and so on, until I had created all of the footage. After that, it required a bit of editing, VFX work and grade in Da Vinci Resolve. 

I used Filmbox to add grain and halation and other filmic colour responses. I skipped all traditional approaches to scripting, storyboarding and treatments, deciding to dive straight in and make the piece shot by shot. 

You mention wanting to make a film about your ‘uneasy feeling of what was wrong with the world’, can you explain more about what you mean?

Yes, it was an uneasy feeling of what was wrong with the world on so many levels, so much aggression and war, even doomscrolling and seeing people dying on social media. I can’t stand it, but can’t look away.

Can you tell us more about each character, why you chose these scenes and what they represented to you?

Each scene is a snapshot of an atrocity from history, from the crusades to present-day. A modern version of the Danse Macabre.

What were the benefits of using AI for this project, rather than live action?

The key benefit was that I could make this in much less time, but the drawback is I don’t get to shoot live action, which I would much prefer to be doing. My last short (ICE) I shot in Iceland, which I loved doing, and I spent 2 years doing the visual effects, which was torture. Each of these shorts-and I’ve done a bunch of them now- takes a year, maybe 2 years of VFX work. So you have to be really invested, or really dumb, to keep doing it. 

What were the highlights and hardest parts of the project?

The best part was that I could explore ideas very fast. I usually draw storyboards at the rate of a gatling gun, and this is faster even than that. It’s like typing up a shot list, but the shot list becomes the final shots with no steps in between. Actually making this stuff is fun, in a way doing VFX, for me, is not (and I cannot stress this enough). I almost feel guilty for how easy this was. I went through insufficient pain. 

I've never experienced people being angry at me just for testing new technology before, but there is much less hostility than even 5 months ago.

The hardest part was the many shortcomings of the technology itself. Currently, there’s no way to reproduce characters consistently (you can get it close enough sometimes but not identical), and its quite heavily censored. I’m also worried what this means for live action shooting. 

It's also not fun to be insulted on twitter for posting this stuff, due to the backlash against AI. I've never experienced people being angry at me just for testing new technology before, but there is much less hostility than even 5 months ago. I think people are starting to realise that this technology is not just going to go away. It’s both exciting and scary.