My voice, three generations

Blog post #83

A studio microphone and headphones beside three colored sound ribbons representing generations of a cloned voice.

AI-generated illustration.

ElevenLabs released v4 on 28 September, and two days later I had a film about it on the channel. A few hours after that there was a Swedish version too. It plays my own voice clone on v2, v3 and v4, explains what ElevenLabs is, shows how to make your own clone and how I run it from Claude Code.

🎥 Prefer video? Watch it on YouTube, or the Swedish version.

What shipped

  • A 13-minute film. Super Intelligence for Your Voice? ElevenLabs v4 vs v3 vs v2 Tested. It is built from HTML with HyperFrames, one chapter per file, and the narration is my own voice clone.
  • A chapter that explains ElevenLabs. The first version assumed you knew the product. Someone who only half listens would not have known what we were talking about, so there is now a minute that shows the real website: text to speech, cloning, dubbing, audiobooks and voice agents.
  • A Swedish version. Superintelligens för din röst? has a Swedish translation of the script, the voice cloned in Swedish, and every on-screen text translated. It is 14 minutes 27 seconds, so it fits under the 15-minute limit without cutting anything.
  • Real material instead of drawings. The official logos, screenshots of the site, ElevenLabs’ own product loops with a source label, and free stock footage of studios and microphones.

What’s working

  • v4 on my Swedish-trained voice. My clone was built from one hour of Swedish speech, and a clone can only be made from one language. Even so, v4 made the Swedish clearly better for the first time since v2.
  • One film, two languages. I translated the script sentence by sentence, so every chapter has the same number of sentences in both languages. A small script then pairs the English and Swedish sentences and moves all the graphics to the Swedish word timings. Nothing had to be redrawn.
  • The pipeline. Script, voice-over per paragraph, a local speech recogniser that gives the exact time of every word, then chapters built in parallel by agents against those times. Each chapter’s graphics land on the word that says them.

What’s unclear or broken

  • The voice drifts in long takes. About ten minutes in, the clone started saying “v4” with the Swedish letter name and “two” like a Swedish word. The fix is to generate paragraph by paragraph and check each take with the speech recogniser before using it. It found the same problem in the transcript, as “VF4” and “VATO”.
  • I use ElevenLabs’ own clips without written permission. They are labelled with their source. I have not asked them.
  • No custom thumbnails. My YouTube account is not verified, so the upload of a thumbnail is refused and YouTube picks a frame for me.
  • A render takes half an hour. Every fix costs a coffee.

Decisions made

  • Tell the honest part. What you hear on the channel was never raw output: re-run takes, pauses built by hand, tempo and loudness work. The film says so.
  • Cut my own joke. I had a gag about the test text saying “sounds like a machine” and v2 reading it like one. It came too late, when no sample was being discussed, so it went. The question “which one was v4?” went with it, because every clip had already been labelled.
  • Real footage, not characters. The first cut had little painted figures and looked like a PowerPoint deck. The second has orbs in ElevenLabs’ own style, real pictures and varied stock.
  • An affiliate link, with a disclosure. I joined ElevenLabs’ affiliate program and put my link in both descriptions. There is a line at the top and one at the bottom saying it is an affiliate link and that I may earn a commission at no extra cost to you. The film is honest about what works and what does not, and the link is easier to trust when it is not hidden.
  • No subtitles burned into the Swedish film. YouTube provides the translation, so the English clips in the Swedish version carry no added text.
  • Don’t say which tools keep my secrets. Nothing in the voice-over or on screen names the password manager.

Tooling & process

  • Screen recording went wrong. I tried recording the desktop with ffmpeg, and it captured the wrong window and part of my private feed. I deleted it at once. Screenshots from the browser tab are safe, and for public pages a headless browser at double resolution gives sharp images once you crop out the cookie box.
  • Small screenshots look soft when zoomed. The fix is not to zoom: show them at their own size and point with rings and arrows.
  • Stock footage with a clear licence. I used a site with a free licence for commercial use and kept a file that says where every clip came from.
  • Swedish is longer. It usually runs about a fifth longer than English, and the limit for my account is 15 minutes. This time the Swedish script came out at 14 minutes 27 seconds without any cuts, because I wrote it to be short and checked the length of the voice before building anything.
  • Check the clips after changing the build script. A bug moved the graphics but not the stock videos, so they showed up in the wrong places. I only saw it because I looked at frames from the finished film.

— Stefan