Greedy Policy vs ε- Greedy Policy
The objective of reinforcement learning task is to learn an optimal policy. Policy is the strategy followed by the agent to take an action given the state of agent in the environment. Optimal policy is the one that maximizes expected value of total reward.
In model free reinforcement learning, we do not know the model of the environment. That is we do not know all the states, all the actions available in all the states, transition probability function and reward function. We learn about all the elements as we play the game. We learn more and more about these elements as we play the game.
Early on in the game, we do not know the rewards and resulting states for actions available in a given state. That is we have partial knowledge of the environment. Because of this partial knowledge of the environment our agent faces exploration exploitation dilemma.
Exploitation: taking the best actions based of limited knowledge of rewards for small number of actions.
Expration: Exploring the new actions that the agent has not tried, but the agent has know idea about what rewards it will receive for taking such action.
Greedy Policy
At any given stage of learning, if the agent takes best action based on available information then such a policy is called greedy policy.
Such policy may appear advantageous in the short term but we may miss larger rewards in unexplored actions.
If the agent takes only the best action based on partial information then he may miss out on potentially larger rewards in future which are associated with taking earlier actions that are less rewarding.
ε
ε- Greedy Policy
ε -greedy policy of is a very simple policy of choosing actions using the current Q-value estimations. It goes as follows:
With probability (1- ε) choose the action which has the highest Q-value.
With probability (ε) choose any action at random.