Same weights, reversed. One circuit — encode turns a byte into a 16-d latent, decode runs it backwards to recover the byte. Like a speaker that also works as a microphone.
02 · Exhaustive Dictionary
All 256 possible bytes with their 16-dimensional latents. Click any cell to inspect the fingerprint the encoder emits for that byte — or search by character, name, or hex code.
03 · Why this setup won
Four axes of search — activation · weight precision · hidden dim · LUT readout. Each tile below is a configuration that was actually run. Only one point lands at 100% lossless with the smallest footprint.
Activation × weight precision
min hidden-dim that holds 100% lossless
Weight precision (C19 · H=16)
stepping down from full precision
Hidden dim (C19 · binary)
narrowing toward the smallest lossless · best-of-4 seeds
LUT readout precision
champion weights baked into a table
Full-binary attempt
binary weights and binary activations
04 · Try it live
The only operation the Byte Block does at inference time is a 256-entry table lookup. Every byte maps to a fixed 16-dimensional latent. Type below to see each byte resolved in real time.
Byte sequence
05 · What’s next
Block A is frozen and lossless. The frontier is now the merger layers above it and open architectural questions that the sweep left unanswered.
The layer above Block A. Two adjacent byte latents are compressed into a single merged latent. 2026-04-19 breakthrough: single-W mirror tied at H=81 reaches 100% lossless on all 2592 cells at half the footprint of the old recipe, overturning the “73% hard ceiling” as a single-seed artifact.
A second merger layer stacked on top of L1 pairs. Currently at the geometry-probe phase. Early signal shows the L1 merged latents carry enough structure to compress further, but the winning recipe is still being searched.
A parallel direction: tokenize directly into whole words or subword pieces, the standard LLM approach. V1 on FineWeb-EDU shows 52% raw word coverage; 34.6% of the corpus falls back to byte encoding.
Open questions
Full-binary weights AND activations. The champion uses binary (±1) weights with a C19 float activation. True full-binary collapses reconstruction (see Slide 3). Is there a hidden-width or architecture tweak that unlocks it?
Integer-only inference path. The LUT readout is already int8, but the C19 activation still needs float. Can the whole forward pass become integer arithmetic for microcontroller deploy?
Cross-modal bytes. Block A was trained on text byte distributions. Does the same recipe work on audio bytes, image bytes, or arbitrary binary streams without retraining?