Shortcuts and Boundaries: Verifiable RL Misalignment and Contextual Safety Learning

This study constructs agentic environments with verifiable conflicts between task reward and contextual execution requirements. Compliant completion remains feasible, while some violating strategies earn more than every compliant strategy.

The experiments study shortcut acquisition during task-reward training and whether feedback from execution histories can teach useful safety behavior that transfers to changing permissions, evidence, and approval requirements.

Read the research blog or the arXiv paper.

Citation

@misc{shen2026shortcutsboundariesverifiablerl,
      title={Shortcuts and Boundaries: Verifiable RL Misalignment and Contextual Safety Learning},
      author={Xinjie Shen},
      year={2026},
      eprint={2611.12865},
      archivePrefix={arXiv},
      primaryClass={cs.AI},
      url={https://arxiv.org/abs/2611.12865},
}