TableKV: KV cache compression for in-context table processing

Processing large tables provided in-context to LLMs is challenging due to token limits and information overload. While RetrievalAugmented Generation can select relevant subsets externally, this work explores Key-Value (KV) cache compression as an alternative, applied directly to the linearized table during inPaolo Papotti EURECOM,France Figure 1: High-level overview of attention-guided KV ference. We show that the LLM’s internal attention scores over the table context guides the retention of essential KV pairs, effectively compressing the processing context while preserving crucial relational information needed for complex queries. Experiments on Spider, WikitableQA, and QTSumm datasets validate the compression approach for in-context table processing, offering a promising path for improved table representation learning in LLMs.