Features¶
The features in MushroomRL are 1-D arrays computed applying a specified function to a raw input, e.g. polynomial features of the state of an MDP. MushroomRL supports three types of features:
- basis functions;
- tensor basis functions;
- tiles.
The tensor basis functions are a PyTorch implementation of the standard basis functions. They are less straightforward than the standard ones, but they are faster to compute as they can exploit parallel computing, e.g. GPU-acceleration and multi-core systems.
All the types of features are exposed by a single factory method Features
that builds the one requested by the user.
-
Features
(basis_list=None, tilings=None, tensor_list=None, n_outputs=None, function=None)[source]¶ Factory method to build the requested type of features. The types are mutually exclusive.
Possible features are tilings (
tilings
), basis functions (basis_list
), tensor basis (tensor_list
), and functional mappings (n_outputs
andfunction
).The difference between
basis_list
andtensor_list
is that the former is a list of python classes each one evaluating a single element of the feature vector, while the latter consists in a list of PyTorch modules that can be used to build a PyTorch network. The use oftensor_list
is a faster way to compute features than basis_list and is suggested when the computation of the requested features is slow (see the Gaussian radial basis function implementation as an example). A functional mapping applies a function to the input computing ann_outputs
-dimensional vector, where the mapping is expressed byfunction
. Iffunction
is not provided, the identity is used.Parameters: - basis_list (list, None) – list of basis functions;
- tilings ([object, list], None) – single object or list of tilings;
- tensor_list (list, None) – list of dictionaries containing the instructions to build the requested tensors;
- n_outputs (int, None) – dimensionality of the feature mapping;
- function (object, None) – a callable function to be used as feature mapping. Only needed when using a functional mapping.
Returns: The class implementing the requested type of features.
-
get_action_features
(phi_state, action, n_actions)[source]¶ Compute an array of size
len(phi_state)
*n_actions
filled with zeros, except for elements fromlen(phi_state)
*action
tolen(phi_state)
* (action
+ 1) that are filled with phi_state. This is used to compute state-action features.Parameters: - phi_state (np.ndarray) – the feature of the state;
- action (np.ndarray) – the action whose features have to be computed;
- n_actions (int) – the number of actions.
Returns: The state-action features.
The factory method returns a class that extends the abstract class
FeatureImplementation
.
The documentation for every feature type can be found here: