Jailson Brito
← Home

Tabfier

Transcribes guitar and bass recordings into playable tablature.

Transcribes a guitar or bass recording into playable, editable tablature, entirely in the browser — nothing is uploaded.

The problem

Transcribing your own guitar or bass playing into tab is normally manual, ear-by-ear work, or it means uploading a recording and waiting. Tabfier does it in the browser: record or import audio, get an editable tab out, nothing leaves the device.

The hard part isn’t detecting notes — it’s turning a detected pitch into a specific string and fret. The same note usually has several fingerings, and picking the wrong one produces a tab that’s technically correct and unplayable the way it’s written.

One pitch, three places to play itA fretboard diagram of the B, G, and D strings in standard tuning, showing the pitch D4 available at fret 3 on the B string, fret 7 on the G string, and fret 12 on the D string. The fret 3 position is marked as generally preferred, since a lower hand position costs less in the fret-assignment search — though which one a real phrase should use depends on the notes around it, not shown here.BGDfret 3fret 7fret 12
The same pitch, D4, sits at fret 3 on the B string, fret 7 on the G string, and fret 12 on the D string. All three are correct; only one is what a player would actually reach for in a given phrase.
Tabfier's editor, showing a transcribed guitar riff as an editable tab staff, with instrument settings for fold octaves, positional fret assignment, and hand spanTabfier's editor, showing a transcribed guitar riff as an editable tab staff, with instrument settings for fold octaves, positional fret assignment, and hand span

Approach and methodology

Built measured rather than tuned by ear: the project keeps a mechanism doc, a ranked list of known accuracy problems, and a results doc with real F1 scores against real guitar recordings from GuitarSet — not just synthetic test audio, which overstates accuracy considerably (see Results).

Fret assignment is treated as an optimisation problem: the constants weighing “prefer a lower hand position” against “prefer fewer position shifts” were fit and cross-validated against real players’ actual fingering choices from that same dataset. Negative results are kept in the docs too — two attempts to close a gap in open-string usage both measurably hurt accuracy and were reverted, recorded rather than deleted.

Built with Claude Code as the coding agent. Every UI element carries a test id specifically so a change can be verified against the real, running app in a browser rather than trusted on faith.

Architecture

Tabfier transcription pipelineAudio is decoded and resampled, then a pretrained pitch-detection model produces raw notes. Everything after that is deterministic — filtering, octave folding, and quantizing to a musical grid — before a dynamic-programming search assigns each note to a string and fret, and the tab is rendered and exported.

Audio input

Decode + resample
(mono, 22.05kHz)

basic-pitch model
(TensorFlow.js, Web Worker)

Filter + fold octaves + quantize
(pure, deterministic)

Fret assignment
(dynamic-programming search)

Render tab as SVG

Export MIDI / MusicXML

Tabfier transcription pipelineAudio is decoded and resampled, then a pretrained pitch-detection model produces raw notes. Everything after that is deterministic — filtering, octave folding, and quantizing to a musical grid — before a dynamic-programming search assigns each note to a string and fret, and the tab is rendered and exported.

Audio input

Decode + resample
(mono, 22.05kHz)

basic-pitch model
(TensorFlow.js, Web Worker)

Filter + fold octaves + quantize
(pure, deterministic)

Fret assignment
(dynamic-programming search)

Render tab as SVG

Export MIDI / MusicXML

One non-deterministic stage (the model), three deterministic stages after it — the codebase is structured around that split.

Nothing leaves the browser: transcription runs entirely client-side, and audio, notes, and edits are all stored locally. The only network request the app makes at all is fetching the model’s own weights from a CDN the first time it runs. Tab is drawn as hand-rolled SVG — no third-party notation engraver — and exports to a hand-written MIDI encoder and MusicXML.

The editor exposes real controls into that pipeline, not just a finished tab to look at: Simplify collapses a chord down to its lowest note; Fold octaves corrects octave-detection errors; Positional turns the hand-position-aware fret search on or off; Hand span sets how many frets that search will stretch across; Confidence and Min length filter out low-confidence or too-short detections before they ever reach the tab. Each one is a direct dial on a step of the pipeline above, not a separate feature bolted on afterward.

The fret-assignment search itself, from src/core/positional.ts — the state cost that turns “prefer a lower hand position” and “prefer fewer shifts” into numbers a solver can compare:

const stateCost = (noteIndex: number, s: State): number => {
  const pos = playable[noteIndex]!.cands[s.candidate]!;
  const reach = pos.fret === 0 ? 0 : pos.fret - s.anchor;
  let cost = o.heightCost * s.anchor + o.reachCost * reach;
  if (pos.fret === 0) cost -= o.openBonus;
  return cost;
};

The comment a few lines below it states the model in one sentence: “only the hand moving costs anything; staying in position is free.” heightCost prices the hand’s anchor position, reachCost prices how far a note sits from it, and shifting between anchors elsewhere in the search is what actually costs — not the fingering above.

Results

The open question going in was how far @spotify/basic-pitch — a general-purpose, off-the-shelf pitch model, not built for guitar specifically — could get on an instrument where the same note usually has more than one right answer for where to play it.

Transcription F1 score, synthetic audio vs. real guitar recordingsA bar chart comparing mean F1 score: synthetic audio scores approximately 0.95, real audio from Real audio (GuitarSet) scores 0.718 — synthetic overstates accuracy by roughly23 points of F1.Synthetic audio~0.95Real audio (GuitarSet)0.718
Mean F1, transcription accuracy. Synthetic audio has no room noise, pick noise, fret buzz, or inharmonicity — Real audio (GuitarSet) is real guitar recordings with real ground truth.
Metric Value
Fret-assignment accuracy 54.5% → 68.5% after two fixes
Bass F1 (synthetic) ~0.39 — not yet measured on real bass recordings

A specific, counterintuitive result is documented rather than glossed over: sustained single notes — bends, vibrato — fragment worse than fast chord strums, the opposite of what “more notes at once should be harder” would predict.

Limitations

  • Explicitly an experimental beta, with known accuracy problems.
  • Bass transcription is the weakest measured area and hasn’t been verified on real bass recordings at all — an open question, not just a rough edge.
  • The recording path has real, un-fixed accuracy costs: no input-level normalisation before the model’s fixed thresholds, and mic recording goes through a lossy codec then back.
  • Manual tempo entry is the default. Automatic detection exists and works from the notes alone, but can’t always tell a tempo from its double or half.
  • basic-pitch is general-purpose and wasn’t built for guitar — weakest at exactly what guitar does a lot of: the same pitch on more than one string, bends, distortion.

Work in progress

Real-audio bass accuracy — currently a blind spot, not just a weak spot.

Raw audio capture (skipping the lossy codec round-trip) and input-level normalisation are both reasoned through, not yet built. Tap-tempo and audio-based tempo detection are on the list.

A guitar-specific model, or a simpler single-note pitch tracker for solo lines, is being considered as a way past the current model’s ceiling — not decided yet.

The tab renderer itself — drawing and editing a fretted-instrument staff from a plain list of notes — doesn’t depend on anything Tabfier-specific. Worth its own small library if a second use for it ever comes up.

Try it yourself