Frontier Demos

Frontier Demos

Helix 2.5

zero-shot generalize to completely unseen scenes with unseen objects. demo里能看出来。虽然场景是新的，物体是新的，但是应该还是精心打扫过，没有什么杂物或者任务无关的遮挡、干扰，还是很干净的测试环境。而真正进入家庭，需要应多十分复杂的真实家庭中的干扰和“非标”状况，这才是最难generalize的。 展示了30个未见家庭场景中的三个任务：fold laundry, pick up scattered items, make the bed. 展现了humanoid上面locomotion, pick and place, deformable object manipultion。 它的路径是纯scale up. 收集了大量第一人称，多种真实场景下普通人贡献的家务视频，用来大规模训练。 不再为每个任务重新收集数据、训练。而是将一些基础skill迁移到新的任务。

VISTA:

![[VISTA-main.png]]

Core idea

a general-purpose multimodal model can observe the world through high-dimensional sensory inputs, reason about the world in a fuzzy but flexible manner, recall past experience through a lossless memory mechanism, and effectively take actions to interact with the world to achieve the goal.

说明通用多模态模型已经具备通过视觉记忆、语言推理理解新的游戏规则并且实现目标的能力。不必为特定任务设计复杂的 world model，而是通过通用多模态模型、自然语言推理和可检索的视觉记忆，实现对陌生环境的自主探索和交互。

contribution

提出一个 agent Harness: VISTA， which uses three key designs: perception through natural high-dimensional visual input, reasoning through language, and memory management through lossless visual memory.

1. directed visual inspection harness:
   • play：执行游戏动作，获取新观测。
   • inspect：按需回顾历史帧、动画中间帧，并放大局部区域。
   • read_pixels：读取精确像素值，辅助解决离散视觉细节问题。
   • GUIDE.md / WORKING.md：分别保存相对长期的规则知识与当前任务的工作记忆。
2. prediction–observation–revision guidance in prompt:

   Before each `play`, briefly state what you expect to see. Afterward, briefly state all visible changes, expected or not.
   

   它鼓励模型执行一种闭环的假设检验过程：\(\underbrace{\hat{s}_{t+1}=f_{\text{LLM}}(h_t,a_t)}_{\text{Prediction}} \quad\longrightarrow\quad \underbrace{o_{t+1}}_{\text{Observation}} \quad\longrightarrow\quad \underbrace{h_{t+1}}_{\text{Revision}}\) Ablation ![[VISTA-ablation1.png]] ![[VISTA-ablation2.png]]

   结果

   用非常短的一段prompt，用Claude Opus 5.0 取得了 ATC-AGI-3 上 25 个任务 100%成功率， 100 relative human efficiency, 56% fewer actions than human first-time player的成绩。 ` ```

   Visual game task

Complete the game with as few game actions as possible.

Build and use a compact, revisable model of the game and its current state. Update it as new evidence changes what is supported.

Before each play, briefly state what you expect to see. Afterward, briefly state all visible changes, expected or not.

Keep concise, durable, revisable game understanding in GUIDE.md; use WORKING.md as a scratchpad when useful. ```

|指标|Claude Opus 5.0 + VISTA| |—|—| |ARC-AGI-3 public games|25/25 完成| |Levels|183/183 完成| |Mean RHAE score|100.00| |Game actions|7,542| |相比首次人类玩家|减少 56.0%| GPT-5.6 Sol 在同样的 VISTA 架构下，也完成了全部 25 个游戏，RHAE 达到 98.27。这支持 Harness 不依赖某个特定基础模型的判断。