论文概要
研究领域: NLP 作者: Zhekai Chen, Chengqi Duan, Kaiyue Sun, Bohao Li, Yuqing Wang, Manyuan Zhang, Xihui Liu 发布时间: 2026-07-09 arXiv: 2607.08768
中文摘要
大型语言模型和多模态大型语言模型的快速发展推动了主动式智能体的涌现，这些智能体能够操作日常工具并在真实环境中协助用户。然而，现有基准难以有效评估此类智能体，因为它们往往依赖沙盒环境和单轮评估范式。此外，其基于场景的任务分类法在同一任务类别中混合了多种模型能力，使得难以识别智能体失败的根本原因。为解决这些局限性，我们引入UniClawBench，这是首个旨在评估动态真实环境中主动式智能体的能力驱动基准。UniClawBench围绕五项基础模型能力构建：技能使用、探索、长上下文推理、多模态理解和跨平台协调。基于这些能力，我们设计了400个双语真实世界任务。与依赖静态预设答案的先前基准不同，我们的基准在实时Docker容器中使用细粒度的逐步完成检查点评估智能体。此外，我们设计了包含执行智能体、隐藏监督智能体和用户智能体的闭环评估策略，以模拟真实的多轮人类反馈而不泄露评分标准。为分离基础模型能力与框架级设计选择，我们在多种智能体框架下评估最先进模型。通过跨模型和框架的综合比较，我们展示了基础模型能力与智能体框架设计如何共同塑造真实环境中的性能。我们已公开基准和代码：https://github.com/HKU-MMLab/UniClawBench
原文摘要
The rapid development of large language models and multimodal large language models has accelerated the emergence of proactive agents capable of operating everyday tools and assisting users in real-world environments. However, existing benchmarks struggle to evaluate such agents effectively, as they often rely on sandboxed environments and single-turn evaluation paradigms. Moreover, their scenario-based task taxonomies mix multiple model capabilities within the same task category, making it difficult to identify the root causes of agent failures. To address these limitations, we introduce UniClawBench, the first capability-driven benchmark designed to evaluate proactive agents in dynamic, real-world settings. UniClawBench is built around five foundational model capabilities: Skill Usage, E…
— 自动采集于 2026-07-12
#论文 #arXiv #NLP #小凯