sobota 10. září 2016

Gridworld sample implementation with Reinforcement Learging, Q function and experience replay

As and exercise I implemented a reinforcement learning agent in a simple Gridworld with Python. Full code can be found on github.

The agent does not have any prior knowledge about the environment nor it's transitions. It learns by exploration to reach the goal, where it's given a reward. Then the world is reset. The algorithm learns the Q function of (state, action) pairs and then uses it to guide itself through a maze.

During the process, it stores all experience and during each step it replays it so the Q function converge to the true Q* quicker.

The movement of the agent is shown as an ASCII output, with 1 as walls, 9 as the goal and 2 as the current position of the agent.

čtvrtek 18. srpna 2016

Git Repositories

During reading Reinforcement Learning: An Introduction by Sutton and Barto (SE, 2012), I did some exercises in the book. I created a git repository where I will share some of those.

Some interesting excercises with TensorFlow, Theano and Keras might be also wort of sharing. You can find them in this git repository.

neděle 7. srpna 2016

TensorFlow Segmentation Fault

If you happen to run into TensorFlow segmentation fault on MacOs, try symlinking libcuda.dylib to libcuda.1.dylib in /usr/local/cuda/lib seems to do the job. More on here.

Installing CUDA on MacOS

Installing CUDA on MacOS developed to a bit of challenge so here is how to do:

  1. Download cuda_7.5.27_mac.dmg & install
  2. Download cuDNN & install:
  3. Test CUDA:
It failed on with with "ld: framework not found CUDA" error. The solution was to edit every Makefile and replace "-framework CUDA" with "-F/Library/Frameworks -framework CUDA". I actually used IntelliJ IDEA to do the job.

sobota 6. srpna 2016


So here goes my journey as an AI researcher. First, I'd like to point some interesting resources: