Environments

All environments implement the Environment interface. Some of them, e.g. grid worlds, are finite Markov Decision Processes: they extend the FiniteMDP class, exposing the transition probability matrix, the reward matrix and the initial state distribution, so that they can also be solved with the dynamic programming solvers.

An environment is built either by calling its constructor directly, or by name through make(), which looks it up in the registry every environment adds itself to with register. Environments standing for a family of tasks accept the task after a '.' separator, e.g. Environment.make('Gymnasium.Pendulum-v1').

The tables below index every registered environment. The names are the keys make() accepts. The spaces are the ones the environment reports in MDPInfo when built with its default arguments; where the environment stands for a family of tasks they are task dependent, and where it is built from a user-supplied model — FiniteMDP and LQR — they follow the size of that model. Environments requiring an extra dependency are registered only if that dependency imports successfully, so an environment missing from Environment.list_registered() means its extra is not installed.

• Classic control
• Finite MDPs
• External libraries
• MuJoCo
• PyBullet
• Isaac Sim

Built-in

make() name	Class	Observations	Actions	Extra dependency
CarOnHill	CarOnHill	Box (2)	Discrete (2)	—
CartPole	CartPole	Box (2)	Discrete (3)	—
FiniteMDP	FiniteMDP	Discrete	Discrete	—
GridWorld	GridWorld	Discrete (9)	Discrete (4)	—
GridWorldVanHasselt	GridWorldVanHasselt	Discrete (10)	Discrete (4)	—
InvertedPendulum	InvertedPendulum	Box (2)	Box (1)	—
LQR	LQR	Box	Box	—
PuddleWorld	PuddleWorld	Box (2)	Discrete (5)	—
Segway	Segway	Box (3)	Box (1)	—
ShipSteering	ShipSteering	Box (4)	Box (1)	—
SimpleChain	SimpleChain	Discrete (5)	Discrete (2)	—
Taxi	Taxi	Discrete (252)	Discrete (4)	—

External libraries

make() name	Class	Observations	Actions	Extra dependency
Atari	Atari	Box (image)	Discrete	mushroom_rl[atari]
Gymnasium	Gymnasium	task dependent	task dependent	mushroom_rl[gymnasium]
DMControl	DMControl	task dependent	task dependent	mushroom_rl[dm_control]
MiniGrid	MiniGrid	Box (3, H, W)	Discrete	mushroom_rl[minigrid]
MiniGridRGB	MiniGridRGB	Box (grayscale image)	Discrete	mushroom_rl[minigrid]

MuJoCo

make() name	Class	Observations	Actions	Extra dependency
Ant	Ant	Box (27)	Box (8)	mushroom_rl[mujoco]
HalfCheetah	HalfCheetah	Box (17)	Box (6)	mushroom_rl[mujoco]
Hopper	Hopper	Box (11)	Box (3)	mushroom_rl[mujoco]
Walker2D	Walker2D	Box (17)	Box (6)	mushroom_rl[mujoco]
BallInACup	BallInACup	Box (23)	Box (7)	mushroom_rl[mujoco]
Reach	Reach	Box (32)	Box (7)	mushroom_rl[mujoco]
Push	Push	Box (37)	Box (7)	mushroom_rl[mujoco]
Pick	Pick	Box (45)	Box (8)	mushroom_rl[mujoco]
PegInsertion	PegInsertion	Box (45)	Box (7)	mushroom_rl[mujoco]
AirHockeyHit	AirHockeyHit	Box (13)	Box (3)	mushroom_rl[mujoco]
AirHockeyDefend	AirHockeyDefend	Box (13)	Box (3)	mushroom_rl[mujoco]
AirHockeyPrepare	AirHockeyPrepare	Box (13)	Box (3)	mushroom_rl[mujoco]
AirHockeyRepel	AirHockeyRepel	Box (13)	Box (3)	mushroom_rl[mujoco]

PyBullet

make() name	Class	Observations	Actions	Extra dependency
AirHockeyHitBullet	AirHockeyHitBullet	Box (12)	Box (3)	mushroom_rl[bullet]
AirHockeyDefendBullet	AirHockeyDefendBullet	Box (12)	Box (3)	mushroom_rl[bullet]
AirHockeyPrepareBullet	AirHockeyPrepareBullet	Box (12)	Box (3)	mushroom_rl[bullet]
AirHockeyRepelBullet	AirHockeyRepelBullet	Box (12)	Box (3)	mushroom_rl[bullet]

Isaac Sim

The Isaac Sim environments are not registered: mushroom_rl/environments/__init__.py does not import isaacsim_envs, so they are not reachable through make() and must be imported from their module explicitly, e.g. from mushroom_rl.environments.isaacsim_envs.cartpole import CartPole. They are vectorized environments: they step a batch of simulated copies at once, and are driven through the vectorized Core.