Physics-Regime Gym Wrapper

Wrap multiple physics simulators (MuJoCo, PyBullet, Brax) behind a unified Gymnasium interface so one agent trains across varied physical regimes.

Difficulty: 1-week | Stack: Python, Gymnasium, MuJoCo, PyBullet, Brax, Stable-Baselines3

Who this is for

Robotics ML engineers who want NitroGen-style physics generalization without building custom infra from scratch

Build steps

1. Wrap MuJoCo, PyBullet, and Brax each as a Gymnasium-compatible env with identical obs/action spaces for a shared task (e.g. CartPole or hopper)
2. Build a MultiPhysicsEnv router that samples a backend per episode with configurable probability distribution
3. Add per-backend physics perturbation params (gravity, friction, mass noise) exposed as env kwargs
4. Train a PPO agent via SB3 on the router env, log per-backend success rates to W&B
5. Eval the trained agent zero-shot on a held-out backend config to measure generalization delta vs single-env baseline

Risks

• Obs/action space unification across backends is fragile — state dims differ subtly between engines, requiring careful normalization
• Brax runs on JAX/GPU while PyBullet runs on CPU; mixing them in one training loop causes device transfer bottlenecks
• Generalization delta may be negligible on simple tasks — need a hard enough task to show the benefit