Spectrogram and STFT
Almost every MIR feature in libsonare is computed from a time-frequency representation of the audio. That includes chroma, mel, MFCC, onset strength, and even BPM and key.
Understanding this one foundation explains why so many functions share the same nFft and hopLength parameters. This page goes one level deeper than the map in MIR Overview.
From waveform to frequencies: the FFT
A waveform is amplitude over time; it does not tell you which frequencies are present. The Fourier Transform answers that — it decomposes a signal into the sine waves that sum to make it, like a prism splitting light into colors. The FFT (Fast Fourier Transform) is just an efficient algorithm for computing it on a block of samples.
The catch: one FFT describes a whole block as if its frequency content never changed. Music changes constantly, so a single FFT of a song is almost useless.
Tracking change over time: the STFT
The Short-Time Fourier Transform (STFT) fixes this by chopping the audio into short, overlapping windows and running an FFT on each. The result is a 2-D grid — frequency content for each moment in time. A spectrogram is that grid drawn as an image: time on one axis, frequency on the other, energy as brightness.
Two parameters appear everywhere because they control this grid:
| Parameter | What it sets | Trade-off |
|---|---|---|
nFft | FFT length in samples (e.g. 2048) — and, unless you shorten the window separately, the analysis window length too | A longer window = finer frequency detail, but blurrier timing |
hopLength | Step between windows (e.g. 512) | Smaller = more frames per second (smoother motion), more CPU |
This is the time–frequency resolution trade-off: you cannot have perfect frequency and perfect time resolution at once. It is physics, not a libsonare limitation.
Strictly, it is the window length that sets that trade-off, not the transform length. They are the same number by default, and always the same through the JS/WASM bindings. In C++ you can set StftConfig::win_length shorter than n_fft; from there, raising n_fft only zero-pads each frame, which adds bins that interpolate the same spectrum more finely without buying any real frequency resolution — and without costing any time resolution either.
From samples to seconds and hertz
Both parameters are counted in samples, so what they mean in time and frequency depends on the sample rate (sr, samples per second):
| Quantity | Formula | nFft 2048, hopLength 512 at 22050 Hz | The same at 44100 Hz |
|---|---|---|---|
| Window duration | nFft / sr | 92.9 ms | 46.4 ms |
| Hop duration | hopLength / sr | 23.2 ms | 11.6 ms |
| Frames per second | sr / hopLength | 43.1 | 86.1 |
| Spacing between frequency bins | sr / nFft | 10.8 Hz | 21.5 Hz |
| Number of frequency bins | nFft / 2 + 1 | 1025 | 1025 |
Time of frame t (centered framing, the default) | t × hopLength / sr | frame 100 ≈ 2.32 s | frame 100 ≈ 1.16 s |
The same sample counts give half the window duration and twice the bin spacing at 44100 Hz. That is why the music analyzers rescale nFft to the input rate (see the details box below): it keeps their window the same length in seconds.
Windowing
Each block is multiplied by a window function (such as a Hann window) that tapers its edges to zero before the FFT. Without it, the abrupt block boundaries leak energy across frequencies (spectral leakage) and smear the result. Windowing is why frames overlap — the tapered edges are filled in by neighboring frames.
- windowed frames
- the same samples, analysed again
- spectrogram columns
Musical spacing: CQT and VQT
A standard STFT spaces its frequency bins evenly in Hz. Musical notes are different: they are spaced logarithmically, because each octave doubles in frequency.
That means an even-Hz grid can waste resolution high up and lack resolution down low.
- CQT bins — one per semitone
- STFT bins — constant Hz spacing
| Transform | How it spaces frequency | Use when |
|---|---|---|
| STFT | Evenly in Hz | You want a general time-frequency view |
| CQT (constant-Q transform) | By musical interval, often one set per semitone | Pitch relationships matter |
| VQT (variable-Q transform) | Bin centers stay on CQT's per-semitone grid, but the low-frequency filters are widened (lower Q), so they are shorter in time and low notes do not smear | You want CQT-style pitch-aware bins but cleaner timing on bass and low percussion |
How libsonare computes the STFT
libsonare's STFT and framing utilities apply a window (Hann by default), advance by hopLength, and run a real FFT per frame, producing the magnitude/power spectra that mel, chroma, onset, and tempogram stages reuse. That reuse is scoped: inside analyze() (MusicAnalyzer) the spectrogram is computed once and shared by the chroma and mel stages, while the standalone per-feature helpers — chroma(), melSpectrogram(), and the individual key/chord/section analyzers — each run their own STFT. Ask for several features through analyze() if you want the FFT paid for once. The nFft/hopLength defaults (2048/512) mirror common librosa usage so reference tests can compare outputs. CQT/VQT use log-frequency bins layered on top of the same framing conventions.
A window length in samples means a different duration at each sample rate, so the analyzers that take audio at its own rate (beats, tempo, onsets, key, rhythm, the chord beat grid, timbre) read nFft as samples at 22050 Hz and rescale it to the input rate; the hop stays in input samples. The same material therefore gets the same window duration at 22050, 44100 or 48000 Hz. The streaming analyzer reads nFft as samples at 44100 Hz instead (rescaled below that rate, never shorter than the hop), and polyphony reads nFft, winLength and hopLength at 44100 Hz.
Related: MIR Overview, Chroma Features, Mel, MFCC, and Timbre, Audio Basics