论文概要
研究领域: ML 作者: Marco Colombo, Balaji A Jayaram, Sowmyakumari Selvaraj, Sumit Tokkar, Animesh Kumar 发布时间: 2026-11-06 arXiv: 2611.09846
中文摘要
本文描述了一种在无法获取人类撰写参考的情况下估计 LLM 生成的 COBOL 摘要正确性的方法和用例。核心直觉是：模型间在语义上的一致程度能反映真实性，LLM-as-judge 能提供比传统字符串或嵌入指标更高保真度的成对评估。作者将这一直觉操作化为：(i) 从多样化模型受控生成摘要；(ii) LLM 评估器比较摘要对并输出结构化判断；(iii) 校准的聚合规则结合支持性证据，同时只惩罚经证实的矛盾。
原文摘要
This paper describes an approach and use case to estimate the correctness of LLM-generated COBOL summaries without access to human-authored references. The core intuition is that inter-model agreement on semantics is informative about truth, and that an LLM-as-judge can provide higher-fidelity pairwise assessments than traditional string- or embedding-based metrics. We operationalise this intuition with (i) controlled summary generation from diverse models; (ii) a LLM evaluator that compares pairs of summaries and emits structured judgments; and (iii) a calibrated aggregation rule that combines supportive evidence while penalising only corroborated contradictions.
— 自动采集于 2026-11-10
#论文 #arXiv #ML #小凯