Isn't a chess position just 64 squares?
That is the obvious starting point: 64 squares, each empty or occupied by one of six piece types in one of two colours. The picture looks complete.
Except the picture is not quite enough to resume the same game. Some rules depend on what happened before this moment, and that information is not painted on the squares.
The board has invisible state
Put the pieces in the same places and two boards can still behave differently. Castling and en passant are the clearest examples: the boards replay what happened just before, or what becomes legal next. Side to move and the two move counters are harder to show visually, so those are named in text below.
Castling rights
White king steps to g1
King and rook still on their home squares
Both histories reach the same arrangement, with the king and rook on their original squares. The difference is the invisible journey that came before: one history moved the king and permanently removed castling rights; the other never did.En passant
White pawn on d5; black pawn on e6
White pawn on d5; black pawn on e7
Both boards end with a white pawn on d5 and a black pawn on e5. On the left, Black just played the normal one-square move e6–e5. On the right, Black jumped from e7 to e5, so White may capture on e6 for this move only.Side to move
Whose turn it is. The same arrangement can belong to either player, so a snapshot has to say who moves next.
Halfmove clock
Counts each player's turn since the last pawn move or capture. A player can claim a draw when the count reaches 100 under the fifty-move rule, and the game is automatically drawn at 150 under the seventy-five-move rule unless the final move is checkmate. The pieces alone do not reveal this count.
Fullmove number
Numbers the turns in a written game. It starts at 1 and increases after each Black move, matching the move numbers used in scoresheets and chess notation. The rules do not use this number to decide whether a move is legal.
Should we store the board, or the moves?
Watch how the two common representations evolve during a standard game, such as Morphy's Opera Game.
Board → text snapshot / move list
32 pieces
rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq - 0 1
(start · empty path)
The first string is FEN, the standard readable snapshot used by chess libraries. It records the pieces, whose turn it is, castling and en passant rights, and two move counters. Those extra fields are how the text preserves a game that the 64 squares cannot describe alone. Once every slash, space, and digit has to live in a URL, though, a FEN share averages about 104 characters in the held-out benchmark.
FEN includes those fields for different reasons. Side to move, castling rights, and en passant can change which move is legal next. The halfmove clock carries the fifty-move rule, while the fullmove number preserves move numbering. Together they make one snapshot sufficient to restore the standard state used by chess software. Later, we will ask whether a more specialised encoding can derive any of that state instead of spelling it all out.
The second string is the UCI move path: coordinate moves such as e2e4e7e5…. Replaying it from the starting board recovers the same state without writing each field separately. It begins wonderfully short, then grows with every move and preserves more history than a board preview needs.
That early advantage is why paths remain in the comparison. For an opening or a short game, the move list can be smaller than a packed board. Later, its unbounded growth becomes the problem. This gives us two useful ideas to compress: the state we have now, or the route that brought us here.
Packing the grid into bits
The most literal binary approach gives every square a four-bit cell for empty or a coloured piece type. That fixed grid costs 256 bits. The tested codec adds the same 27-bit meta block used by occupancy: side to move, castling, en passant, halfmove clock, and fullmove number. Its logical payload is therefore always 283 bits before the URL alphabet. It never adapts to how empty the board becomes.
Most of the board is air, though, and that is where Occupancy + Pieces wins. A 64-bit mask marks which squares are filled; each occupied square then gets a four-bit piece. Empty costs one bit in the mask instead of four in the grid. The same FEN-complete meta rides at the end. Size now tracks piece count rather than ply count, so middlegames and endgames stay near the same URL length while path codecs keep growing.
Generic compression does not save the day
gzip works best when it has enough material to find repeated patterns and enough payload to repay its own header. A chess share string gives it very little of either.
Move lists do contain structure, but each individual share is too short for generic gzip to exploit efficiently. The header alone can outweigh the opening moves, and turning the compressed bytes into a URL-safe alphabet expands them again. Two coordinate moves become a longer link than the raw move string. gzip(FEN) produces the longest mean URL in this benchmark. It stays as a control so "just gzip it" has a measured answer: on these strings, gzip makes the pasteable URL longer.
Fewer bits can still make a longer link
A codec has at least three lengths. Logical bits are the information written before padding. Payload characters are what survive a URL-safe alphabet, or raw ASCII for a plain move list. Full URL length includes the origin, route, and payload: the thing someone actually pastes into WhatsApp.
Those three do not move in lockstep. A bitstream is first rounded to complete bytes. Unpadded Base64URL usually needs about four characters for every three bytes, with visible jumps at byte boundaries. Saving one or two logical bits may therefore save no characters at all.
The alphabet matters too. Ordinary Base64 uses + and /, which are awkward in URL paths. Characters outside a conservative URL-safe alphabet may be percent-encoded, normalised, or treated differently across URL implementations. Packed rows use Base64URL (-/_) so the payload stays path-safe.
We score a complete share URL so the comparison stays concrete. The fixed route is scaffolding, not the research question. The same trade-offs apply to path segments, query params, QR codes, and other apps.
If the score is characters rather than bits, a dense Unicode symbol can look like a cheat. The demo below shows why the displayed glyph, its code point, its encoded bytes, and what survives in a URL path can all disagree.
%E4%B8%AD. Browsers may still display the glyph; the serialised form is what travels in the URL.One displayed glyph can occupy several bytes and many serialised URL characters. The alphabet is part of the codec, and every symbol comes with rules.
What can the decoder infer?
Occupancy still writes a full FEN-complete meta block. Before seeing how that compares on the scoreboard, ask how much of that meta the decoder could recover from the pieces alone. A shorter state encoding can exploit what the current arrangement makes impossible. It cannot treat hidden state as if the pieces uniquely determine it. The paired boards near the beginning show exactly where that assumption fails.
- Castling rights are partly constrained. If a king or rook is absent from its home square, that castling right is certainly gone. If both have returned home, the board still cannot reveal whether they moved earlier. The encoder can omit impossible rights, but must distinguish the remaining cases.
- En passant is usually impossible, but not always inferable. Pawn geometry and side to move sharply limit when a target square could exist. When the geometry permits one, identical pieces can still result from a one-square or two-square pawn move. The relevant detail of the previous move remains genuinely hidden.
- Side to move is generally independent. Some arrangements constrain which side could legally move, but many are valid with either player to move. A general snapshot therefore needs to preserve the distinction.
- Neither move counter has an exact answer on the board. The same pieces can appear after different numbers of quiet plies and at different fullmove numbers. Common values can receive shorter codes, but recovering the original values requires carrying equivalent information somewhere in the state payload.
The useful saving is conditional: avoid bits when the board proves a value, and encode the ambiguity when it does not. Simply deleting a field produces a different state rather than a better compression of the same one. The first scoreboard therefore compares codecs that preserve the chosen target: a restorable FEN snapshot, including legal-move state and both counters. Occupancy carries its full meta block in that comparison.
At ply 2, who should win?
Before the numbers: at two plies, should replaying e2e4e7e5 beat carrying an entire board? Should occupancy already beat native FEN once empty squares cost one bit instead of four? And can gzip repay its header on a string this short?
The benchmark samples positions from real Lichess games and measures the complete URL, not just the payload. Make a prediction, then read the means.
| Method | BitsCharsURL |
|---|---|
| Raw / native | |
Native FEN (Base64URL) Keeps the whole printable board text so the decoder can reload any mid-game position. | Bits Chars URL |
Native UCI (ASCII) Lists every ply from the start as from-to squares, readable as typed. | Bits Chars URL |
| Packed binary → Base64URL | |
Packed UCI path Twelve bits per ply from the start, then padded into URL-safe characters. | Bits Chars URL |
Occupancy + pieces Marks filled squares once, then names what sits on each, plus the clocks. | Bits Chars URL |
Naïve 4-bit grid Assigns a fixed nibble to every square, empty or not, so length never changes. | Bits Chars URL |
| gzip → Base64URL | |
gzip(UCI) → Base64URL Runs a general compressor on the move string, then expands bytes for the URL. | Bits Chars URL |
gzip(FEN) → Base64URL Runs a general compressor on the board string, then expands bytes for the URL. | Bits Chars URL |
Overlay reads mean ± σ [min–max]. Tall band = observed min–max; mid band = ±σ; coloured bar = mean.
Occupancy lands around 57 URL characters as a standalone mean, against roughly 104 for native FEN. The fixed square-by-square encoding sits between those ideas and never adapts: every position costs the same. Packed paths win early and lose later. gzip loses throughout. Among the standalone codecs, occupancy has the shortest overall mean because it stores the current state without growing with the move count.
Inference can trim some meta later. The next leap is different: move shared opening knowledge into the decoder so familiar prefixes need not be spelled out at all.
What if the decoder already knows common openings?
Human openings repeat. Instead of spelling out a familiar first dozen moves, the URL can point to an opening the decoder already knows, then append only the unfamiliar suffix. The information still restores a full path, and therefore a full FEN state; the savings come from moving shared knowledge into the software.
Think of the codebook as a row of small lookup tables, one for each opening depth. Here, depth counts plies, or individual turns by one player. Depth 2 means White and Black have each moved once; depth 12 means six full moves have been played. A depth-2 table might contain e2e4e7e5, while a depth-8 table can name a much longer branch of the same opening.
The letter K sets how many of the most frequent prefixes each table may keep. The benchmark uses up to K = 1024 entries at each of depths 2, 4, 6, 8, 10, 12. Since 2¹⁰ = 1024, a 10-bit index can select any entry in one table. The depth ID selects the table. Together, those two numbers tell the decoder which known sequence to replay.
The encoder uses the longest prefix it finds, then packs any moves that came after it as a suffix. A dictionary miss stores the complete packed path instead. One discriminator bit tells the decoder which form it received. No network fetch is required, but the codebook must ship with the decoder and remain frozen or versioned, or old links break.
Familiar lines such as e2e4e7e5, the Sicilian e2e4c7c5, and longer Open Sicilian stretches land in the book. Expand the lookup method card below for three concrete codebook rows.
How far could a dictionary go?
The opening book suggests an extreme version of the same idea. If the decoder contained every possible position, the URL could be little more than an index. Common positions could receive the shortest indices, while rare positions would receive longer ones. That is the theoretical direction in which a frequency-aware codec points.
The decoder would be doing most of the work. Tromp and Österlund estimate about 4.8 × 1044 legal chess positions. Even if real games visit only a tiny fraction of them, a complete catalogue is far beyond anything a web client could sensibly ship.
Our corpus shows where the smaller version stops paying. At depth 2, a 1,024-entry book covers essentially every observed opening prefix. By depth 8, more than 600,000 prefixes appear and the same book covers about a quarter of them by frequency. At depth 12, over 1.5 million appear and coverage falls below 5%. The first few moves repeat; middlegames quickly fan out.
A full position database is therefore a useful limit, not a useful product. The practical design keeps the small, high-value opening book and falls back to packed moves or occupancy when a game leaves it. That combination is the hybrid in the next section.
Where does history stop winning?
Paths start cheap and grow with every move. Board-state encodings stay relatively stable and may shrink after captures. Dictionaries help when openings are predictable. A hybrid that tries packed path, occupancy, and lookup, then keeps the smallest, rides the cheap side of that curve.
Before the loop and the table: at ply 2, should lookup beat occupancy by a wide margin? By ply 32, should that answer reverse? Where do you expect the hybrid to sit relative to both?
Morphy's Opera Game from the starting position through mate, with URL length for each ply. Held-out sample: 270,021 games · 1,169,301 positions at plies 2, 8, 16, 32, 64 · codebook from hash-train.
Board → URL length
start32 pieces
AAD_____AABCNWMkEREREZmZmZnKveus-AAAIA
AA
Measured only at sampled checkpoints, the same pattern appears in the table. Early columns favour paths and lookup. Later columns favour occupancy. Hybrid hugs the cheaper explanation. Greener cells are shorter within each column.
| Method | Ply 2 | Ply 8 | Ply 16 | Ply 32 | Ply 64 |
|---|---|---|---|---|---|
| 26 | 38 | 54 | 86 | 150 | |
| 60 | 59 | 58 | 55 | 49 | |
| 25 | 31 | 46 | 78 | 142 | |
| 25 | 31 | 46 | 55 | 50 |
At ply 2, hybrid and lookup sit near 25 characters while occupancy still carries a full board near 60. By ply 32, occupancy and hybrid converge near 55, and packed paths have ballooned past 80. Opening history is cheap because it is short or familiar. Later, occupancy wins because the board no longer cares how many moves it took to get there.
The hybrid keeps the cheapest explanation
Across sampled checkpoint positions, hybrid lands around 40 URL characters versus roughly 104 for native FEN. Equal-weighting games instead of checkpoint observations changes the hybrid mean only slightly, from about 40 to 39.5 characters.
That is not proof of the shortest chess encoding possible. It is the best result among the practical methods tested here. The durable lesson is not one winning representation. The winning system chooses the cheapest truthful description for each case: a short path while history is cheap, a known opening when the codebook hits, and a board snapshot once the journey costs more than the destination.
What to try next
- Smarter paths. Rank each move among legal moves rather than storing source and destination squares.
- Compressed move sequences. Measure whether UCI or PGN becomes more competitive once wrapped in Zstandard or Brotli as well as gzip. Raw gzip already loses on short share strings; a fuller bake-off would still keep the comparison honest.
- Smarter snapshots. Use variable-length piece coding or rank legal board configurations. Flat position ranking may improve bounded or worst-case size; it does not automatically minimise frequency-weighted average length on real shares.
- Smarter alphabets. Write directly into URL-safe six-bit symbols instead of padding through bytes.
- Prior art and variants. Compare specialised position encodings from the literature, plus Chess960 and crazyhouse. The numbers would move. The same URL-safety and character-count constraints would still apply.
The open question is not whether a shorter link exists in principle. It is which of these mechanisms still pays once every share must remain pasteable, self-contained, and truthful.
A compact map of the bets
Each row is a bet about what to store. Expand a method for the mechanism and a real example link. Examples use 1. e4 e5 (e2e4e7e5) unless noted.
| Method | Kind | Mean full URL | Depends on | Keeps |
|---|---|---|---|---|
| State | 104 | None beyond chess.js | Complete FEN state | |
| Path | 100 | Replay from start | Full path from start | |
| Variant | 98 | None beyond chess.js | Playable fields only (no clocks) | |
| Path | 61 | Replay from start | Full path from start | |
| State | 57 | None beyond chess.js | Complete FEN state | |
| State | 70 | None beyond chess.js | Complete FEN state | |
| Path | 120 | gzip + replay | Full path from start | |
| State | 124 | gzip + chess.js | Complete FEN state | |
| Frequency | 55 | Frozen codebook | Full path from start | |
| Hybrid | 40 | Codebook if lookup wins | Path or complete FEN state |
Native FEN (Base64URL)104 mean URL chars
Stores the full FEN string, then Base64URL-encodes it for a path-safe payload. The decoder Base64URL-decodes and loads the position with chess.js.
Use when you already have a mid-game FEN and no move path, or when interoperability with standard FEN matters. Every slash, space, field, and clock digit still has to pass through the URL alphabet.
Full FEN → Base64URL. About 102 characters for the share URL on this opening.
Native UCI (ASCII)100 mean URL chars
Appends raw UCI move text with no further encoding. e2e4e7e5 is already URL-safe ASCII, so the payload is the move string itself. The decoder replays from the start position.
Included for the early-game edge only: openings and short games can be surprisingly compact. Grows by four characters per ply (five on promotions), stores full history, and is not the long-term share format.
30 characters for the full URL. By ply 16, the same format averages 86 characters.
Still readable. Length tracks the path, not board complexity.
Trimmed FEN (playable-only variant)98 mean URL chars
Base64URL of FEN with the halfmove clock and fullmove number dropped. The result can restore a playable board, but not the original counters on its own.
Useful as a lower bound on those two fields. Without a recovery rule, it describes less state than full FEN. It is also still text-shaped, so it never approaches the packed rows.
Restores the board and immediate move rights, but not the original counters.
Packed UCI path61 mean URL chars
A path again, but each move is 12 bits (from-square + to-square) plus 2 bits when a promotion appears. Because the decoder replays the board, it knows when a move must include a promotion choice. The bit stream is then Base64URL-encoded.
Wins while games are short. By ply 16, the packed path is already roughly level with occupancy on average. Hybrid exists largely to catch that crossover; packed UCI is not a serious late-game candidate on its own.
Occupancy + pieces57 mean URL chars
A 64-bit occupancy mask (which squares are filled), then a 4-bit piece nibble per occupied square, then FEN-complete meta: side to move, castling, en passant, halfmove clock, and fullmove number. No move list. Size tracks piece count, not ply count.
Among the standalone codecs, occupancy has the shortest overall mean on this held-out validation scoreboard. Middlegames and endgames stay near the same URL length while path codecs keep growing. Weak only in the opening, where a short path or dictionary entry is smaller.
219 bits with 32 pieces plus clocks → 60 URL characters. By ply 64, occupancy shortens as captures remove pieces.
Naïve 4-bit grid70 mean URL chars
Every square gets a fixed 4-bit cell (empty or coloured piece), plus the same 27-bit FEN-complete meta used by the benchmark. The total is always 283 logical bits before Base64URL.
A useful baseline for “store the grid literally.” Occupancy beats it because empty squares cost one bit in the mask instead of four in the grid.
Fixed 70-character URL on every position. Simple, never adaptive.
gzip(UCI) → Base64URL120 mean URL chars
gzip the raw UCI ASCII string at maximum compression, then Base64URL the bytes. Same idea people reach for when a payload “should compress.”
Almost always loses on share URLs. The gzip header alone is larger than a short opening path, and Base64URL expands the compressed bytes again. Even on long paths, generic gzip remains far behind the chess-aware packed representation in this benchmark.
Two coordinate moves become a 60-character URL. The raw move string was 30.
gzip(FEN) → Base64URL124 mean URL chars
gzip the full FEN text, then Base64URL. Same compressor tax as gzip(UCI), applied to a string that is already short and low-redundancy.
gzip(FEN) produces the longest mean URL in this benchmark. Kept as a control so “just gzip it” has a measured answer.
Header + alphabet tax outweigh any FEN redundancy at this size.
Lookup K=1024 + suffix55 mean URL chars
Human openings repeat. A hit replaces familiar opening plies with a depth id and a 10-bit index, then appends any packed suffix. A miss stores the complete packed path. Every lookup payload starts with one discriminator bit (1 = hit, 0 = miss), counted in both logical length and the Base64URL payload. The benchmark book keeps a separate list of up to K = 1024 prefixes at each of depths 2, 4, 6, 8, 10, and 12 on the hash-train split; the index is fixed at 10 bits because each depth has at most 1,024 entries. Decode needs no network or remote fetch, but the decoder must ship the same frozen codebook; version it or old links break.
Shines on openings people actually play. On the held-out validation split it lands near occupancy in the overall mean, but wins hard at plies 2 and 8 in the sampled checkpoints, where a short path is already small and occupancy still carries a full board.
Index 0 is the most common prefix at that depth. The decoder replays the stored UCI. Each depth has its own list of up to K = 1024 entries; these three rows are enough to see that openings repeat.
| Depth | Idx | Opening | UCI prefix | FEN |
|---|---|---|---|---|
| 2 | 0 | Open game | e2e4e7e5 | rnbqkbnr/pppp1ppp/8/4p3/4P3/8/PPPP1PPP/RNBQKBNR w KQkq - |
| 2 | 2 | Sicilian | e2e4c7c5 | rnbqkbnr/pp1ppppp/8/2p5/4P3/8/PPPP1PPP/RNBQKBNR w KQkq - |
| 8 | 0 | Open Sicilian | e2e4c7c5g1f3d7d6d2d4c5d4f3d4g8f6 | rnbqkb1r/pp2pppp/3p1n2/8/3NP3/8/PPP2PPP/RNBQKBNR w KQkq - |
Discriminator bit = hit, then depth id 0 and index 0 for e2e4e7e5. No packed suffix. Full URL length 25.
Hybrid (min of 3)40 mean URL chars
For each position, encode packed UCI, occupancy, and lookup, then keep the smallest of those three. A 2-bit mode tag rides in front of the winning payload. The decoder reads the tag and dispatches. This scoreboard hybrid includes lookup; a production-style hybrid without a codebook is only min(packed, occupancy).
Wins across these tested positions and methods. Early games pick path or dictionary; later games pick occupancy. Mean URL falls to about 40 characters with a maximum of 60 characters in this sampled benchmark (an observed max, not a formal upper bound). The largest observed samples select a representation close to occupancy plus the mode tag.
Two-bit mode selects lookup. Shorter than packed path alone; much shorter than forcing occupancy.
Mode selects occupancy. URL stays near 58 instead of a growing move path.
How chss.chat multiplexes formats
The compression question is which representation is smallest. The product question is how one app ships several of them behind one route. chss.chat answers that with a short letter prefix before the payload: f- for full FEN, u- for raw UCI, t- for trimmed FEN, p- for packed moves, o- for occupancy, n- for the naive 4-bit grid, and so on. The decoder reads the prefix and dispatches.
Production today uses f- / u- and a no-codebook h- that keeps min(packed, occupancy). The dictionary row and the scoreboard hybrid that also tries lookup are research codecs. Prefixes are dispatch, not the representation. Changing them is not the research question; finding the smallest trustworthy payload is.
Reproduce the benchmark
The scoreboard is not a black box. Anyone can rebuild it from the public Lichess dump and the scripts in this repository. The product metric is pasteable URL length; the pipeline exists so that claim can be checked, not trusted.
Full commands, disk estimates, and sampling details live in benchmark/README.md on github.com/ghcpuman902/chss. The short path is below.
1. Clone and install
You need Python 3.11+, zstd /zstdcat, roughly 30 GB free disk, and about 5 GB+ free RAM while the month is streaming.
git clone https://github.com/ghcpuman902/chss.git cd chss python3 -m venv .venv-benchmark .venv-benchmark/bin/pip install -r benchmark/requirements.txt
2. Download a Lichess month
Standard rated games are CC0 on database.lichess.org. The published table used June 2026 (~26 GB compressed, 86.5M games). Do not fully decompress it. The scripts stream with zstdcat.
mkdir -p data/standard curl -L --continue-at - \ -o data/standard/lichess_db_standard_rated_2026-06.pgn.zst \ https://database.lichess.org/standard/lichess_db_standard_rated_2026-06.pgn.zst
3. Stream, convert, and split
One helper runs the three passes: full-month frequency aggregate, a 3% hash sample of games turned into compact UCI JSONL, then a deterministic train / validation / test split. On our machine (Apple M3 Pro, 18 GB), that corpus pipeline took about 1 h 45 m wall time, roughly an hour for the aggregate and about 41 minutes for the hash extract.
bash benchmark/run_corpus.sh
Outputs land under data/standard/corpus/hash/ as zstd-compressed train / validation / test JSONL. Once those exist you can delete the raw .pgn.zst to free ~26 GB.
4. Rebuild the scoreboard
Train the K = 1024 opening codebook on the train split, evaluate at plies 2 / 8 / 16 / 32 / 64 on the held-out validation split, and write the slim JSON the page reads.
bash benchmark/run_scoreboard.sh # same as: .venv-benchmark/bin/python benchmark/url_length_benchmark.py \ --train data/standard/corpus/hash/2026-06.train.compact.jsonl.zst \ --eval data/standard/corpus/hash/2026-06.val.compact.jsonl.zst \ --out benchmark/results/url_length_hash_val.json \ --slim-out lib/compression-url-scoreboard.json
You should land near the same ranking: hybrid shortest among the tested methods, occupancy next among standalone state codecs, native FEN much longer. Approximate checkpoint means on this page are hybrid 40, occupancy 57, and native FEN 104. Exact floats can shift a little with toolchain versions; the order should not.
Design notes and earlier bake-off logs: doc/chess-url-compression.md.