Gridworld sample implementation with Reinforcement Learging, Q function and experience replay

As and exercise I implemented a reinforcement learning agent in a simple Gridworld with Python. Full code can be found on github.

The agent does not have any prior knowledge about the environment nor it's transitions. It learns by exploration to reach the goal, where it's given a reward. Then the world is reset. The algorithm learns the Q function of (state, action) pairs and then uses it to guide itself through a maze.

During the process, it stores all experience and during each step it replays it so the Q function converge to the true Q* quicker.

The movement of the agent is shown as an ASCII output, with 1 as walls, 9 as the goal and 2 as the current position of the agent.

Git Repositories

During reading Reinforcement Learning: An Introduction by Sutton and Barto (SE, 2012), I did some exercises in the book. I created a git repository where I will share some of those.

Some interesting excercises with TensorFlow, Theano and Keras might be also wort of sharing. You can find them in this git repository.

TensorFlow Segmentation Fault

If you happen to run into TensorFlow segmentation fault on MacOs, try symlinking libcuda.dylib to libcuda.1.dylib in /usr/local/cuda/lib seems to do the job. More on here.

Installing CUDA on MacOS

Installing CUDA on MacOS developed to a bit of challenge so here is how to do:

1. Download cuda_7.5.27_mac.dmg & install
2. Download cuDNN & install:
3. Test CUDA:

It failed on with with "ld: framework not found CUDA" error. The solution was to edit every Makefile and replace "-framework CUDA" with "-F/Library/Frameworks -framework CUDA". I actually used IntelliJ IDEA to do the job.

Resources

So here goes my journey as an AI researcher. First, I'd like to point some interesting resources:

• Coursera Machine Learning Course - basic introduction into ML
• OpenAI Gym - framework for implementing AI algorithms
• TensorFlow - tool for creating neural networks
• Google DeepMind - Google acquired company focusing on deep learning; interesting papers
• Reinforcement Learning: An Introduction - awesome book for RL, go for unfinished Second Edition
• Neural Networks and Deep Learning - a free online book about NN