论文概要
研究领域: LLM量化 作者: Baha Rababah, Cuneyt Gurcan Akcora, Carson K. Leung 发布时间: 2025-07-12 arXiv: 2507.08705
中文摘要
训练后量化被广泛用于在资源受限环境中部署大语言模型，但其评估几乎完全依赖于准确率和困惑度。我们表明这些指标无法捕捉量化引起的行为变化。我们引入了正确性一致性，衡量基础模型与其量化变体在正确预测上的重叠。在多个模型和从8位到2位的量化方案中，我们发现即使在任务性能看似保持的情况下，中等量化下也会出现行为分歧。我们将量化分析为对注意力权重的结构性操作，使用统计和分布度量来量化逐层失真。结果揭示了低位宽下的非线性断点，查询和键投影始终比値和输出投影更敏感。
原文摘要
Post-training quantization is widely used to deploy large language models in resource-constrained settings, yet its evaluation relies almost exclusively on accuracy and perplexity. We show that these metrics fail to capture behavioral changes induced by quantization. We introduce correctness agreement, a decision-level metric that measures overlap in correct predictions between a base model and its quantized variants, independent of absolute accuracy. Across multiple models and quantization schemes from 8-bit to 2-bit, we find that behavioral divergence emerges under moderate quantization even when task performance appears preserved. Our results reveal non-linear breakpoints at low bit-widths and show that query and key projections are consistently more sensitive than value and output proj…
— 自动采集于 2025-07-13
#论文 #arXiv #LLM量化 #小凯