Lip sync and audio-led editing

I’m a huge fan of Epic the Musical, Jorge Rivera-Herrans’ serial adaptation of Homer’s Odyssey. My Goodbye is my favourite song in the entire soundtrack. It’s Athena delivering a cold, devastating farewell to Odysseus on the shores of burning Troy. With Greek mythology having a bit of a moment right now thanks to Christopher Nolan’s The Odyssey, I wanted to create a visual interpretation of that scene. Specifically the moment she says goodbye for the last time.

The goal was simple in theory: lip sync an AI-generated character to an existing vocal performance. In practice it was one of the most technically challenging things I’ve attempted in this space.

My Goodbye. A fan interpretation of the song by Jorge Rivera-Herrans. Non-commercial fan project.

Audio first is a different problem entirely

Typically how I work is to generate the voice to match the video. Doing it the other way around meant every tool in my pipeline was fighting me. The audio existed, it could not be changed, and the visuals had to serve it completely. The timing, the mouth shapes, the emotional delivery all had to match a performance I had no control over.

On top of that, lip sync results are very mixed the moment your character is anything other than a tight close-up. The further away the character is in the shot, or the moment you introduce a second character into the scene, the less reliable the results become. That creates a real tension between wanting cinematic variety in your shots and actually getting the lip sync to land.

What happens when you don’t lock the style first

The biggest mistake was one I’d made before and should have known better. I started generating character images before I’d locked either the characters or the visual world they existed in. Both drifted significantly as a result.

The early versions pulled toward photorealism. Wrong direction for the world I was building.

By the time I had a consistent background, the two characters had already evolved away from each other stylistically. I had to stop, rebuild reference sheets for both from scratch and re-establish the visual language before I could move forward.

The restyled versions landed on an illustrated mosaic aesthetic: gold and amber tones, graphic linework, glowing eyes. Much more fitting for a goddess.

Some of that inconsistency from the early drift is still visible in the final film. That’s entirely down to the sequence of decisions, not the tools.

Getting the characters locked

Having these locked before animation was the thing that held the later sequences together. Should have been step one, not an afterthought.

Building the environment

The environment was built in Midjourney to match the illustrated style of the characters once they were locked. Burning Troy, Greek ships in the harbour, that amber and smoke palette. Getting the background and characters to feel like they existed in the same visual universe took longer than it should have. Because the characters came first.

What I used and what each one did