Experiments 22
Any Semblance of My Speech
In Alvin Lucier’s 1969 piece I Am Sitting in a Room, a spoken recording is played back and re-recorded until room resonance replaces the words. Here the room is a speech-to-text and voice-cloning loop: speak one sentence and hear each generation re-transcribe and re-synthesize the last.
Rust · Candle · WebSocket · WebRTC · Opus · WhisperView source
Example · The readerVoice 100%Words 14 of 14
I am the master of my fate. I am the captain of my soul.
RecordUp to 8 seconds
Record one sentence, or turn Sound on to hear this example.
Notes
The feedback loop
In Alvin Lucier’s 1969 composition I Am Sitting in a Room, a spoken passage is played into a room and re-recorded repeatedly until room resonances replace the speech. Here each generation passes the previous audio through speech recognition and zero-shot voice cloning:
Whisper (tiny.en) transcribes the audio, Sopro v2 turbo extracts a speaker embedding from the same clip and synthesizes the transcript, and the resulting Opus audio becomes the sole input to the next generation. Optional channel degradation simulates 20% or 30% packet loss and line noise.
Drift across 36 generations
Across 14 LibriVox readings of the closing lines of Henley’s “Invictus,” speaker cosine similarity to the original voice averaged 0.92 at generation 1, 0.52 at generation 5, and 0.31 at generation 10, while pairwise similarity between different speakers temporarily converged from 0.26 to 0.54 by generation 7. Transcription errors appeared at a median of generation 8 (“fate” → “feet” in 13 of 14 runs) and compounded until Whisper fell back to short stock phrases. Spectral bands visible in the spectrogram reflect frequencies reinforced by the synthesizer on each pass.
Built with
- Whisper (tiny.en) and Sopro v2 turbo (Apache-2.0, Samuel Vitorino), running in Rust on Candle.
- Opus encoding and optional transport via webrtc-rs (session audio stays in memory only).
- Sample recordings from LibriVox’s “Invictus” collection (public domain).