Chroma Features
Chroma is the representation that makes harmony analysis possible.
It folds the whole spectrum into 12 pitch-class bins: C, C♯, D, ... B. Each frame then says "how much C energy, how much C♯ energy, ..." regardless of octave.
Key detection and chord recognition both read from it. This page expands on the harmony section of MIR Overview.
Pitch class: forgetting the octave
A pitch class groups every octave of a note together. C2, C3, and C4 are different pitches but the same pitch class, "C." Western music has 12 pitch classes, one per semitone. Chroma takes the energy spread across all frequencies in an STFT frame and sums it into those 12 bins — a C played in any octave lands in the same bin.
The octaves are not weighted equally, though. Following the librosa reference, the filterbank tapers each bin's contribution with a Gaussian octave envelope centered on A5 (880 Hz) with a standard deviation of two octaves. The envelope is broad, so everything from the low-mid register up to a few kilohertz counts strongly, but it falls off below: a note around C2 (65 Hz), almost four octaves under the center, keeps under a fifth of the weight. That is why a bass line can look faint in a chromagram even when it is clearly audible.
This deliberate forgetting is the point. Harmony is about which notes, not which octave: a C-major chord is a C-major chord whether it is voiced low or high. Folding octaves together gives a clean, 12-dimensional fingerprint of the harmony at each moment.
For example, a low C, middle C, and high C sounding together all add into the C bin. Strong C, E, and G suggest C major; strong A, C, and E suggest A minor. Chroma is a tool for reading the distribution of note names.
Chromagram: chroma over time
Stack a chroma vector for every frame and you get a chromagram — a 12-row image showing how pitch-class energy moves through the song. Sustained chords appear as steady horizontal bands; a key change shifts which bins stay lit. Visualizers draw it directly; analyzers read patterns out of it.
A frame is one short slice of the audio. Just as a video is a run of still images, a chromagram is the chroma of each short slice laid side by side — time runs left to right, the 12 pitch classes top to bottom.
NNLS chroma: fitting notes instead of folding bins
Plain STFT or CQT chroma folds every frequency bin straight into its pitch class, so a note's overtones land in the bins of other pitch classes too. The third harmonic of C is a G, and the fifth is an E, so a lone low C already lights up a little of a C-major chord.
NNLS chroma (non-negative least squares) asks a different question per frame: which mix of notes best explains this spectrum? libsonare builds a fixed dictionary with one template per piano key (A0 to C8, 88 notes), each its first six partials at decreasing weights, laid out on a constant-Q spectrum with three bins per semitone. Each frame of the CQT magnitude is fitted as a sum of those templates with every note amount held at zero or above. The amounts are then folded into the 12 pitch classes, so an overtone the fit attributed to the C counts for C rather than for G or E. That makes the profile cleaner for chord work, especially over a strong bass.
It is not a pure NNLS output by default. To stay stable on sparse or pure-tone material, where several low notes can explain one partial equally well, the result is mixed with a direct CQT chroma (60 % fit, 40 % direct) and then with a plain STFT chroma at stftBlendWeight 0.55. Pass enableStftBlend: false for the fitted CQT-based profile alone.
nnlsChroma(...) (Python nnls_chroma) returns it as a 12 × frames matrix, each frame scaled so its largest value is 1. detectChords(...) uses a lighter version of the same front-end (five octaves, four partials per note) when you pass chromaMethod: 'nnls'.
Why chroma is right for harmony — and wrong for melody
| Question | Use chroma? | Why |
|---|---|---|
| What key is this? | Yes | Overall pitch-class distribution reveals the tonal center |
| What chord is playing now? | Yes | A frame's chroma matches a chord template |
| What octave is the bass in? | No | Octave is folded away |
| What is the lead melody note? | No | Use pitch tracking; chroma cannot separate the line |
| What instrument is this? | No | Timbre needs spectral shape; use MFCC |
Matching the representation to the question is the recurring skill in MIR (music information retrieval): chroma trades octave and timbre detail away precisely to make harmony clear.
How libsonare builds chroma
The Chroma class applies a chroma filterbank to the STFT power spectrum — mapping each frequency bin onto its pitch class, weighted by the octave envelope described above — accumulates into 12 bins, and normalizes each frame so its largest value is 1.0 (L-infinity). Working from power rather than magnitude weights strong partials more heavily than weak ones.
| Item | Detail |
|---|---|
| Bin layout | n_chroma = 12. fmin and n_octaves are also on the config, but the STFT chroma filterbank does not read them (they are CQT-chroma concepts; setting them does not change the analyzed range) |
| Analysis window | n_fft = 2048 / hop_length = 512, Hann window |
| Tuning | For the 12-bin configuration, tuning is the deviation from A440 in fractions of a semitone. Default 0 (A440); measured from the signal only when you pass tuning: 'auto' to analyze, detectChords or detectKey |
| Variants | STFT-based; CQT-based (chromaCqt / chroma_cqt, ChromaCqtConfig, the direct librosa.feature.chroma_cqt equivalent); CENS (ChromaCensConfig) |
| Chord-oriented variant | NnlsChromaConfig (non-negative least squares) estimates per-note activation to reduce the influence of overtones, giving cleaner profiles for chord work |
Chroma feeds KeyAnalyzer (Krumhansl-Schmuckler) and ChordAnalyzer (template matching), so improving the chroma stage improves both downstream features at once.
Tuning is measured only on request
ChromaConfig.tuning defaults to 0 — concert A440 — and the chroma stage never estimates it on its own. librosa.feature.chroma_stft behaves differently: its tuning=None default runs estimate_tuning() first. On a recording that sits a fraction of a semitone off A440 the pitch-class grid no longer lines up with the actual partials, energy smears into neighboring bins, and key and chord results quietly get worse with nothing to signal it.
Pass tuning: 'auto' (Python: tuning="auto") to analyze, detectChords or detectKey (Python: analyze, detect_chords, detect_key) whenever the source may not be at A440 — older recordings, tape, live acoustic material, anything deliberately tuned to a different reference pitch. The engine measures the offset first and builds the chroma with it. A number in [-0.5, 0.5) works too, in fractions of a semitone, as estimateTuning() / estimate_tuning() returns it; C++ takes it directly as ChromaConfig.tuning. Every result reports the value it used (tuning on AnalysisResult, ChordAnalysisResult and detectKey's KeyDetection). A synthetic C-major chord detuned by 45 cents read as E minor at A440; with tuning: 'auto' it read as C major and reported a tuning of about -0.43. The standalone chroma helper uses fixed A440 and takes no tuning argument. tuningToReferenceHz(tuning) and referenceHzToTuning(hz) convert between this unit and a reference pitch in hertz. Streaming uses hertz directly: set the A4 reference with tuningRefHz at creation or setTuningRefHz() live. The Visual Player estimates the offset and applies it to every file it loads.
Related: MIR Overview, Key Detection, Chord Recognition, Spectrogram and STFT