Realtime Voice Changer
RealtimeVoiceChanger is libsonare's preset-based live voice chain. Use it when you process microphone input, monitor voice in realtime, or pass audio in repeated blocks where DSP state must continue across calls.
For a one-shot offline pitch/formant edit, use voiceChange(...). For a live or chunked preset chain, use this page.
Blocks, sample rate, and DSP state
Realtime audio is processed in small blocks (groups of samples) one after another — in the browser an AudioWorklet hands you a fixed render quantum of 128 samples at a time. Sample rate (e.g. 48000) is how many samples make one second. DSP state is the memory an effect carries between blocks (a reverb tail, a compressor's current gain), which is why the voice changer is a reusable object rather than a one-shot function.
Why this is a class, not just a function
Live voice processing has memory: gates, compressors, reverbs, and smooth pitch/formant changes all depend on previous blocks. RealtimeVoiceChanger keeps that state across calls. Recreating it for every block would sound worse and add unnecessary setup cost.
What You Will Learn
By the end of this page you should be able to:
- choose
RealtimeVoiceChangerinstead of the simpler offlinevoiceChange(...)helper; - prepare the class for block processing and release the WASM handle correctly;
- inspect and validate built-in preset JSON before accepting user-authored presets;
- choose the right binding for browser, Python, Node native, or CLI workflows.
When to Use It
| What you are building | Entry point | Input | Output |
|---|---|---|---|
| Live browser microphone processing | RealtimeVoiceChanger | Short Float32Array blocks from an AudioWorklet or similar loop | Processed blocks with the same length |
| One browser-side clip edit with only semitone and formant values | voiceChange(...) | A decoded mono Float32Array clip | Processed Float32Array |
| Python batch rendering through the preset chain | voice_change_realtime(...) | Mono samples plus sample_rate | Processed samples |
| Node native server or desktop batch processing | voiceChangeRealtime(...) | Mono samples plus sample rate | Processed samples |
| Terminal conversion from WAV/MP3 to a rendered file | sonare voice-change --preset ... | An audio file | An output file |
| Preset inspection, UI editing, or validation | realtimeVoiceChangerPresetJson(...) and validation APIs | Preset ID or JSON | Config JSON and validation result |
Implementation Shape
For live browser processing, call init() and prepare(sampleRate, maxBlockSize, channels) before audio starts. Then call processMono(...) or processMonoInto(...) for each incoming block. RealtimeVoiceChanger keeps DSP state internally, so do not recreate it for every block; reuse one instance for the lifetime of the stream.
| Decision | Guideline |
|---|---|
sampleRate | Use AudioContext.sampleRate or the actual file sample rate. Do not guess a fixed 44100 / 48000 value. |
maxBlockSize | The largest block you will pass to processMono(...). For the standard AudioWorklet render quantum, 128 is a good starting point. |
channels | The examples here are mono, so pass 1. For interleaved input, use processInterleaved(...). |
| Output buffer | Use processMono(...) for a simple implementation. In an AudioWorklet hot path, write into preallocated output with processMonoInto(...) or the heap-backed buffer path. |
| Cleanup | Call delete() when the component unmounts, the worklet stops, or recording ends. It releases the handle once; repeating it does nothing. |
Interleaved means the channels are stored alternately in one array (L, R, L, R…), whereas mono is a single channel; use processInterleaved(...) for the former and processMono(...) for the latter.
Do not mix up live processing and batch rendering
RealtimeVoiceChanger is the class for processing short blocks in sequence while preserving gate/compressor/reverb state. The Python and CLI examples apply the same preset chain to a whole file or array; they are not code you run inside a browser AudioWorklet callback.
Signal Chain
The realtime chain is more than a pitch shifter. Built-in presets combine these stages:
The demo above isolates just the Retune stage (pitch shift) so you can hear it on its own. In a full preset a cleanup high-pass and a noise gate run before the pitch shift, and EQ, dynamics, and ambience follow after it — the diagram shows the full order.
| Stage | Purpose |
|---|---|
| High-pass | Removes sub-bass rumble and DC as a cleanup pre-filter (eq.highpassHz) |
| Gate | Reduces low-level room or mic noise between phrases |
| Retune | Shifts the whole signal up or down by the preset's fixed semitone amount (e.g. for a higher or lower voice) using grain overlap-add resampling — the pitch-shift stage of the chain. The shift is exactly 2^(semitones / 12). It does not detect pitch and does not snap notes to a scale; for that, use the offline pitch correction in Editing DSP |
| Formant | Changes perceived vocal size or character independently of note pitch |
| EQ | Body, presence, and air tone shaping (eq.bodyDb/presenceDb/airDb) |
| Compressor | Keeps level stable across blocks |
| De-esser | Controls harsh sibilance |
| Reverb | Adds or shapes space |
| Limiter | Catches peaks; a 4×-oversampled true-peak limiter follows the dry/wet mix (on by default) |
TIP
The eq block configures both ends of the chain: highpassHz is the cleanup high-pass at the very front, while bodyDb/presenceDb/airDb are the tonal shelves that sit after formant shaping.
Voice-chain terms in one place
- High-pass — a cleanup filter at the front that removes sub-bass rumble and DC before any other stage.
- Gate — mutes the signal when it drops below a threshold, removing low-level mic/room noise between phrases.
- Retune — moves the whole signal up or down by a fixed interval; separate from formant, which changes vocal character without moving the note. It shifts by a fixed amount and is not automatic pitch correction; nothing here listens to what note you sang.
- Formant — the resonances that make a voice sound large/small or male/female; shifting them changes vocal character without changing the note. See Voice and formant.
- Compressor — automatically evens out loud and quiet parts so the level stays steady.
- De-esser — tames harsh "s"/"sh" sounds (sibilance).
- Limiter — a safety catch that stops peaks from clipping at the very end of the chain. The one here is a true-peak limiter: it also accounts for the inter-sample peaks that appear between the stored samples when the signal is reconstructed for playback.
Factory preset IDs include neutral-monitor, bright-idol, soft-whisper, deep-narrator, robot-mascot, and dark-villain. Treat these as starting points, not genre or identity labels.
One Flow, Every Binding
Every binding follows the same lifecycle: construct the changer, prepare it for the sample rate and block size, call processMono(...) for each block, swap presets live with setConfig(...), and read latencySamples(). The tabs below show that one flow in each runtime — only the constructor shape and the cleanup call differ.
import {
init,
RealtimeVoiceChanger,
realtimeVoiceChangerPresetNames,
} from '@libraz/libsonare';
await init();
const changer = new RealtimeVoiceChanger('bright-idol');
changer.prepare(48000, 128, 1);
try {
const out = changer.processMono(inputBlock);
changer.setConfig('soft-whisper');
console.log(realtimeVoiceChangerPresetNames(), changer.latencySamples(), out);
} finally {
changer.delete(); // WASM handle cleanup
}import libsonare as sonare
print(sonare.voice_character_preset_id(1))
preset_config = sonare.realtime_voice_changer_preset_config("bright-idol")
with sonare.RealtimeVoiceChanger(48000, preset="bright-idol", max_block_size=128) as changer:
out = changer.process_mono(input_block)
changer.set_config("soft-whisper")
print(sonare.realtime_voice_changer_preset_names(), preset_config, changer.latency_samples())
# Whole-array render through the same preset chain:
processed = sonare.voice_change_realtime(vocal, sample_rate=48000, preset="soft-whisper")import {
RealtimeVoiceChanger,
realtimeVoiceChangerPresetNames,
voiceChangeRealtime,
} from '@libraz/libsonare-native';
const changer = new RealtimeVoiceChanger({
sampleRate: 48000,
maxBlockSize: 128,
channels: 1,
preset: 'bright-idol',
});
try {
const blockOut = changer.processMono(inputBlock);
changer.setConfig('soft-whisper');
// Whole-array render through the same preset chain:
const rendered = voiceChangeRealtime(vocal, 48000, 'soft-whisper');
console.log(realtimeVoiceChangerPresetNames(), changer.latencySamples(), blockOut, rendered);
} finally {
changer.destroy(); // native handle cleanup (WASM uses delete())
}# sonare voice-change renders a whole file through the preset chain;
# it is not a per-block realtime loop.
sonare voice-presets --json
sonare voice-change vocal.wav --preset soft-whisper -o rendered.wavCleanup differs by binding
The construct/process flow is shared, but each runtime releases its native handle differently.
- Browser / WASM: call
delete()(destroy()is an alias) in thefinally, on component unmount, or when the worklet stops, or declare the changer withusing. The call is idempotent. - Node native: call
destroy();using(Node 22+) can free it automatically. - Python: the
withblock releases the handle; outside awith, callclose().
For AudioWorklet-style loops, use the heap-backed realtime buffers documented in Browser / WASM. They avoid allocating a new output array on every render quantum, which the browser example's plain processMono(...) does. The package also ships a ready-made processor, registerSonareRealtimeVoiceChangerWorkletProcessor from @libraz/libsonare/worklet; see In an AudioWorklet.
CLI Modes
The sonare voice-change command has two modes:
| Mode | Options |
|---|---|
| Simple pitch/formant edit | --pitch-semitones, --formant-factor |
| Realtime preset-chain render | --preset, --preset-json, --preset-pack, --set PATH=VALUE |
If you pass realtime preset options, the command uses the preset chain; combining them with the simple pitch/formant options is rejected as an invalid-parameter error. --preset-pack FILE must be paired with --preset ID to select an entry; there is no first-entry fallback. See CLI Reference for the full selector rules and command table.
Preset JSON
Use preset JSON when you need to inspect, store, or validate voice-chain settings.
import {
realtimeVoiceChangerPresetJson,
validateRealtimeVoiceChangerPresetJson,
} from '@libraz/libsonare';
const json = realtimeVoiceChangerPresetJson('bright-idol');
const validation = validateRealtimeVoiceChangerPresetJson(json);
if (!validation.ok) {
throw new Error(validation.error);
}Current built-in preset JSON uses schema version 1. The native POD-config ABI — the flat plain-old-data struct the C entry points take — is versioned separately; check it with voiceChangerAbiVersion() when you are crossing FFI (foreign function interface) or native boundaries. Both JSON Schemas ship inside the npm package, so a host can validate a document offline:
import schema from '@libraz/libsonare/schemas/realtime-voice-changer-preset.schema.json';If you only need a canonical preset ID or the resolved flat native config, use voiceCharacterPresetId(...) and realtimeVoiceChangerPresetConfig(...) instead of round-tripping through JSON. On WASM, voiceCharacterPresetId(...) accepts a canonical ID or integer ordinal: an unknown numeric ordinal returns null, while an unknown string ID throws. realtimeVoiceChangerPresetConfig(...) throws for an invalid ordinal. Python exposes the same native-config path as realtime_voice_changer_preset_config(...).
A preset document must be complete
A valid document includes the required schema metadata and exactly one complete dsp or macros section. Partial documents are rejected rather than silently filled in from unrelated defaults, and deesser.ratio is required. If you hand-author a preset, start from realtimeVoiceChangerPresetJson('neutral-monitor') and edit it, instead of writing only the fields you want to change. Reading a config back out with configJson(), editing it, and writing it with setConfig(...) is a safe round-trip. Unknown keys at the top level or inside sections are rejected.
macros — the shorthand section
Authoring a full dsp section by hand is a lot of surface for "make this voice brighter". A preset may instead carry a macros object, whose seven values map onto the ordinary DSP config:
| Macro | Range | Moves |
|---|---|---|
pitch | −24 … 24 | Retune, in semitones |
formant | 0.55 … 1.65 | Formant scale factor |
brightness | 0 … 1 | Formant brightness plus EQ presence and air |
space | 0 … 1 | Reverb mix, up to the chain's 0.45 ceiling |
intensity | 0 … 1 | Compressor ratio |
noiseControl | 0 … 1 | Gate and noise handling |
sibilance | 0 … 1 | De-esser amount |
Macros are input-only. The shared parser expands them into the ordinary DSP config on the control thread, so they never appear in normalized output: read a config back and you get the expanded dsp section, not the macros you wrote. A document carries exactly one of the two sections — supplying both dsp and macros is rejected as an invalid parameter, so there is no precedence rule between them.
Each 0–1 macro maps onto its target's valid range rather than being written through raw: macros.space at 1.0 reaches the reverb mix ceiling of 0.45, it does not write 1.0 into a parameter that would reject it.
Formant mode
Retune shifts pitch by resampling grains, and that moves every formant by the same ratio, so +12 semitones also doubles the formant frequencies (the "chipmunk" effect). See Voice and formant for the background.
- In the default relative mode (
formantMode: 'relative') the formant stage runs after the retune, soformantFactormultiplies the displacement the pitch shift has already caused. - In absolute mode (
formantMode: 'absolute') the formants are first pre-warped by the inverse of the pitch ratio, soformantFactor: 1keeps the original vocal size whatever the shift. The warp follows only the retune's wet share (retuneMix).
The figure below shows the three cases on one voice, with the input's first two formants marked in every row.
- harmonics (the comb)
- spectral envelope
- input formant position
In the preset JSON this is dsp.formant.mode.
'relative' (default) | 'absolute' | |
|---|---|---|
formantFactor means | Warp applied after the retune | Formant position relative to the input |
formantAmount | Scales the formant displacement | Ignored |
| Latency | Reported latencySamples() | One analysis frame more (1024 samples at 48 kHz, measured) |
| Reachable range | formantFactor within its parameter bounds | formantFactor / 2^(retuneSemitones / 12) within [0.55, 1.65] (at +12 semitones the pre-warp alone is 0.5, so formantFactor must be at least 1.1) |
The mode is fixed when the changer is created and prepared. In the JSON the key is optional and defaults to relative; in the flat POD config the field is required. A config carrying a different mode is refused (SonareError, "formant mode is fixed at prepare()"), and so is an absolute config outside the reachable range; the error names the formantFactor range that is reachable at that pitch shift and the formantFactor bound that range is intersected with (at +12 semitones: reachable [1.1, 1.65], the warp reaching [1.1, 3.3], limited by formantFactor's [0.55, 1.65]). Automation refuses an out-of-range value too, naming the key, and the previous configuration stays in force.
import { init, RealtimeVoiceChanger, realtimeVoiceChangerPresetConfig } from '@libraz/libsonare';
await init();
const cfg = realtimeVoiceChangerPresetConfig('neutral-monitor');
const changer = new RealtimeVoiceChanger({
...cfg,
formantMode: 'absolute',
retuneSemitones: 5,
formantFactor: 1, // +5 semitones, formants unmoved
});
try {
changer.prepare(48000, 128, 1);
console.log(changer.latencySamples()); // one analysis frame more than relative mode
} finally {
changer.delete();
}The flat POD config that realtimeVoiceChangerPresetConfig(...) returns, and that the constructor, setConfig(...) and setPodConfig(...) accept, requires every field, formantMode included; omitting it throws a TypeError. In C, start from sonare_realtime_voice_changer_config_default(); struct_version of 0 or 1 gives the current 152-byte layout. SONARE_VOICE_CHANGER_ABI_VERSION is 3, so C callers rebuild against the current header.
As a mixer insert
The same chain is a realtime insert named voice.changer (catalog category voice) on mono and stereo strips. It publishes 37 parameters under the preset's flat keys (retuneSemitones, formantFactor, compressorThresholdDb, ...); 33 are automatable. formantMode, retuneGrainSize, limiterEnableIspLimiter and reverbSeed are construction-only. Construction and automation both refuse an out-of-range value, naming the key, and keep the previous configuration. The insert reports its latency, reverb tail, gain reduction (compressor make-up left out) and non-finite discards like other inserts; masteringProcess accepts it too. See Effects Inserts for the descriptor fields.
Practical Notes
Realtime voice processing is stateful. Reuse the same changer across blocks, keep block sizes within the prepared maximum, and release handles when the component or stream stops.
Large pitch, formant, or ambience moves can be useful for sound design, but they will be less transparent. For natural monitoring, keep preset edits conservative and check latency once with latencySamples().
What "latency" means here
Latency is the delay between sound going in and processed sound coming out, caused by the analysis the chain has to do. latencySamples() reports it in samples; divide by the sample rate for seconds.
The delay has two parts:
| Part | What it covers | Changes with |
|---|---|---|
latencySamples() (reported) | The dry path and the whole chain's alignment: the retune stage's analysis window plus the true-peak (inter-sample peak, ISP) limiter's delay when that limiter is active | Only prepare(...) and the grain size; moving the wet or retune mix does not change it |
| Extra wet-path delay | Playing a grain faster (shift up) or slower (shift down) changes how far the read position trails the write position; the shifter re-syncs once the gap reaches its limit, so the wet path lags by between 0 and two grains depending on where in that cycle you are | The shift amount; zero at no shift, up to two grains otherwise |
Read the reported figure once after prepare(...) and compensate for it, instead of re-reading it whenever a control moves. A larger grain analyses more audio per step and adds more delay to both parts (see the StreamingRetune grainSize field). With the default grain (about 46 ms) at 48 kHz, neutral-monitor reports 2263 samples (47 ms), and a shifted voice can trail that by up to about another 93 ms, so pitched presets are less suited to tight live monitoring. The automatic retune grain is capped at 8192 samples, so a saved configuration round-trips its grain and latency at 176.4 and 192 kHz.
Every live control is smoothed per sample, so adopting a new config snapshot with setConfig(...) ramps rather than stepping at a block boundary. Only the formant frequency displacement scales with the formant amount, so body, brightness, and nasal still act at amount zero.