When the teacher has more bits: Self-teacher latent distillation for learned image compression

Learned image compression (LIC) operates under a rate–distortion (RD) trade-off, where representation quality is constrained by the target bitrate. We revisit knowledge distillation in LIC through the lens of bitrate asymmetry and introduce a self-teacher distillation framework, where a high-rate instance of a codec supervises multiple lower-rate encoders of identical architecture. Because allocating more bits naturally leads to richer latent representations, the high-rate model provides informative supervision across rate levels. Direct latent matching, however, is problematic under tight rate budgets. We therefore propose variancenormalized latent distillation (VNLD), a rate-aware alignment strategy

that scales channel-wise supervision by the teacher’s variance, selectively transferring stable, informative structure while suppressing components that cannot be reliably reproduced at lower rates. Across different distillation objectives, networks, and bitrate levels, self-teacher distillation, particularly with VNLD, improves RD performance and yields consistent BD-rate gains over RD-only training. Our method remains compatible

with fixed-decoder deployments, such as those targeted by JPEG AI standards.