Verifiable Hidden Dynamics Play: Generating Agentic RL Environments from Solved Mechanisms

Wed, 23 Sep 2026 00:00:00 +0000

VHD-Play starts from a solved mathematical mechanism, then renders its decision process as stateful tools. Because the environment dynamics and outcome evaluator are inherited from the same source, each generated world has a dependable reward without trajectory annotation.

Train on worlds whose hidden dynamics we know, to act in worlds whose hidden dynamics we do not.

The generated substrate scales to 3,300 environments at a cost of a few cents each. Training transfers beyond the three training families to eight unseen mechanism families and external function-calling, travel-planning, and long-horizon e-commerce tasks.

Verifiable Environments | Xinjie Shen

Verifiable Hidden Dynamics Play: Generating Agentic RL Environments from Solved Mechanisms