Byte Embedder Unit — Baked Winner
C19 + L-BFGS Staged Freeze | Int4 | Tied Mirror
100% Lossless
288 byte int4
24 C19 neuron
40.72% downstream
Architektura
8 bit input
{-1, +1}
W1 8x24
int4
24 C19 neuron
c: 1.5-6.4 | rho: 2.5-5.8
bias: 24 float
W2 24x16
int4
16D latent
float output
MIRROR DECODER
16D → W2^T → 24D → W1^T → 8 bit
tied weights (same W, transposed)
INPUT
HIDDEN LAYER
OUTPUT
RECONSTRUCTION
PARAM BUDGET:
W1: 192 int4 (96B) | W2: 384 int4 (192B) | bias: 40 float (160B) | C19 c,rho: 48 float (192B)
Weights: 288 byte
+ 352B float params = 640B total
Specifikaciok
Lossless roundtrip
100%
Downstream char-LM
40.72%
Weight storage
288 B
Compression
8x
Neuronok
24 C19
Optimizer
L-BFGS
Sparsity
27.8%
Train time
~6 min
Suly-eloszlas (int4: -7..+7)
-7
-3
0
+3
+7
Haranggörbe: 0 korul csucsosodik, 27.8% nulla (sparse). Sulyok -7..+7 kozott, de a tobbseg -3..+3 sav.
24 C19 Neuron — egyenkent
W1 heatmap (8 bit → 24 neuron)
W2 heatmap (24 neuron → 16D)
Live teszt — irj be egy byte-ot (0-255)
= '
A
'
→
8 bit:
→ 24 hidden (C19 output):
→ 16D latent output:
← Mirror decode → reconstructed bits: