The features in MushroomRL are 1-D arrays computed applying a specified function to a raw input, e.g. polynomial features of the state of an MDP. MushroomRL supports three types of features:
- basis functions;
- tensor basis functions;
The tensor basis functions are a PyTorch implementation of the standard basis functions. They are less straightforward than the standard ones, but they are faster to compute as they can exploit parallel computing, e.g. GPU-acceleration and multi-core systems.
All the types of features are exposed by a single factory method
that builds the one requested by the user.
Features(basis_list=None, tilings=None, tensor_list=None, n_outputs=None, function=None)¶
Factory method to build the requested type of features. The types are mutually exclusive.
Possible features are tilings (
tilings), basis functions (
basis_list), tensor basis (
tensor_list), and functional mappings (
The difference between
tensor_listis that the former is a list of python classes each one evaluating a single element of the feature vector, while the latter consists in a list of PyTorch modules that can be used to build a PyTorch network. The use of
tensor_listis a faster way to compute features than basis_list and is suggested when the computation of the requested features is slow (see the Gaussian radial basis function implementation as an example). A functional mapping applies a function to the input computing an
n_outputs-dimensional vector, where the mapping is expressed by
functionis not provided, the identity is used.
- basis_list (list, None) – list of basis functions;
- tilings ([object, list], None) – single object or list of tilings;
- tensor_list (list, None) – list of dictionaries containing the instructions to build the requested tensors;
- n_outputs (int, None) – dimensionality of the feature mapping;
- function (object, None) – a callable function to be used as feature mapping. Only needed when using a functional mapping.
The class implementing the requested type of features.
get_action_features(phi_state, action, n_actions)¶
Compute an array of size
n_actionsfilled with zeros, except for elements from
action+ 1) that are filled with phi_state. This is used to compute state-action features.
- phi_state (np.ndarray) – the feature of the state;
- action (np.ndarray) – the action whose features have to be computed;
- n_actions (int) – the number of actions.
The state-action features.
The factory method returns a class that extends the abstract class
The documentation for every feature type can be found here: