Reality Navigation: Baseline Results
Quick update on our Reality simulator navigation experiments. We've moved from Y-axis rewards to proper target-based navigation with compass guidance.
What Changed
We switched from rewarding Y-axis movement to giving the agent +1 for actually reaching targets. The agent now gets compass bearing and distance info, so it knows where to go but has to figure out how to get there. No more arbitrary direction rewards.
Results
The agent navigates successfully and reaches targets consistently without crashing. But I've noticed two annoying behaviors. First, it's way too cautious about collisions and takes unnecessarily wide routes when shorter paths clearly exist. Second, it spends forever circling around looking for the "perfect" entry point before committing to go through gates or between obstacles.
Agent navigating through obstacles - note the overly cautious routing
Next Steps
I need to fix both the excessive cautiousness and the circling behavior. My plan is to increase exploration aggression during training, start rewarding shorter paths to targets, and add some time pressure to discourage all that circling around.
Want to dive deeper into our simulation framework? Check out the Reality project - built on the Madrona engine for high-performance RL simulation.
