Sub-1-Bit LLM Compression via Latent Factorization
LittleBit explores compressing LLM weights below one bit per parameter by factorizing weight matrices into binary latent factors and learned scales, then recovering quality with quantization-aware training. Its follow-up, LittleBit-2 (ICML 2026), changes initialization to better align the latent factors with the binary hypercube; it adds no inference-time overhead. The repository reports support for several Llama, Qwen, Gemma, OPT, and Phi models.
This is a research result, not yet a convincing practical alternative to conventional quantization. In the 11-comment Hacker News discussion, readers pointed to a large Wikitext-2 quality drop (about 80% to 47%), noted the lack of a comparable FP4 baseline, and questioned whether the compressed models remain useful. Others saw value in the relative improvement at very low bit rates even if current quality is inadequate.