br.unicamp.cst.learning

Class Simple2dRLearn

java.lang.Object

br.unicamp.cst.learning.Simple2dRLearn

public class Simple2dRLearn
extends java.lang.Object

This class is used to perform a simple reinforcement learning with multiple state dimensions. It is composed by a multidimensional Q table with all dimensions but the last meaning states, and with the last dimension meaning actions. For instance, with a 2d state space we would have: Q(s1,s2,a) = Q(s1,s2,a) + R Where R is the reward for performing action a at states s1 and s2. Important: This method does not account for state transitions like Q-Learning or SARSA do.

Constructor Summary

Constructors

Constructor and Description
Simple2dRLearn(int Ns, int Na) Default Constructor.

Method Summary

Modifier and Type	Method and Description
void	addAction() Add action.
void	addState() Add state.
int	getBestAction(int s1, int s2) Gets the best action for state s.
double[]	getBestActionAndValue(int s1, int s2) Returns the best chosen action and its value in Q.
int	getNa() Gets NA value.
int	getNs() Gets NS value.
double[][][]	getQ() Gets Q value.
boolean	isEmpty() Verify if is empty.
void	printQ() Print Q values.
void	setNa(int na) Sets NA value.
void	setNs(int ns) Sets NS value.
void	setQ(double[][][] q) Sets Q value.
void	update(int s1, int s2, int a, double r) Updates Q table of values

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

Simple2dRLearn

public Simple2dRLearn(int Ns,
                      int Na)

Default Constructor.

Parameters:

Ns -

Na -

Method Detail

update

public void update(int s1,
                   int s2,
                   int a,
                   double r)

Updates Q table of values

Parameters:

s1 - previous state

s2 - current state

a - chosen action

r - reward for performing action a in state s1/s2, should be between -1 and 1

addAction

public void addAction()

Add action.

addState

public void addState()

Add state.

printQ

public void printQ()

Print Q values.

getBestAction

public int getBestAction(int s1,
                         int s2)

Gets the best action for state s. If there is a tie between best actions, it chooses randomly between them.

Parameters:

s1 - first

s2 - second

Returns:

best action for state s

getBestActionAndValue

public double[] getBestActionAndValue(int s1,
                                      int s2)

Returns the best chosen action and its value in Q. NOTE: Remember that best_action should probably be cast to int for use outside this class.

Parameters:

s1 -

s2 -

Returns:

a double array {best_action, best_value}

getQ

public double[][][] getQ()

Gets Q value.

Returns:

the q

setQ

public void setQ(double[][][] q)

Sets Q value.

Parameters:

q - the q to set

getNa

public int getNa()

Gets NA value.

Returns:

the na

setNa

public void setNa(int na)

Sets NA value.

Parameters:

na - the na to set

getNs

public int getNs()

Gets NS value.

Returns:

the ns

setNs

public void setNs(int ns)

Sets NS value.

Parameters:

ns - the ns to set

isEmpty

public boolean isEmpty()

Verify if is empty.

Returns:

true or false