tag:blogger.com,1999:blog-73239918109340140192024-03-12T16:50:35.836-07:00AI Researchjaarahttp://www.blogger.com/profile/15349060338114486646noreply@blogger.comBlogger5125tag:blogger.com,1999:blog-7323991810934014019.post-91041839744430032072016-09-10T12:38:00.003-07:002016-09-10T13:14:08.437-07:00Gridworld sample implementation with Reinforcement Learging, Q function and experience replayAs and exercise I implemented a reinforcement learning agent in a simple Gridworld with Python. Full code can be found on <a href="https://github.com/jaara/ai_examples/blob/master/gridworld.py">github</a>.<br />
<br />
The agent does not have any prior knowledge about the environment nor it's transitions. It learns by exploration to reach the goal, where it's given a reward. Then the world is reset. The algorithm learns the Q function of (state, action) pairs and then uses it to guide itself through a maze.<br />
<br />
During the process, it stores all experience and during each step it replays it so the Q function converge to the true Q* quicker.<br />
<br />
The movement of the agent is shown as an ASCII output, with 1 as walls, 9 as the goal and 2 as the current position of the agent.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjemJFLbVy22mjioKAaREtIbFbmgBKZrZ2JREImTz9sDKLuOh6NEVnHzOwJCjjmRXP_HYBoeSjeBwUXjqNf-CGFGjclh7Kgu4-Ree3lhU-of0_cRgBNi4FTvYm9FHkpniV-aGqkLV_nz5vl/s1600/%25E3%2582%25B9%25E3%2582%25AF%25E3%2583%25AA%25E3%2583%25BC%25E3%2583%25B3%25E3%2582%25B7%25E3%2583%25A7%25E3%2583%2583%25E3%2583%2588+2016-09-10+21.37.36.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjemJFLbVy22mjioKAaREtIbFbmgBKZrZ2JREImTz9sDKLuOh6NEVnHzOwJCjjmRXP_HYBoeSjeBwUXjqNf-CGFGjclh7Kgu4-Ree3lhU-of0_cRgBNi4FTvYm9FHkpniV-aGqkLV_nz5vl/s320/%25E3%2582%25B9%25E3%2582%25AF%25E3%2583%25AA%25E3%2583%25BC%25E3%2583%25B3%25E3%2582%25B7%25E3%2583%25A7%25E3%2583%2583%25E3%2583%2588+2016-09-10+21.37.36.png" width="294" /></a></div>
<br />jaarahttp://www.blogger.com/profile/15349060338114486646noreply@blogger.com0tag:blogger.com,1999:blog-7323991810934014019.post-60927765192407737942016-08-18T02:39:00.001-07:002016-08-18T02:39:51.877-07:00Git RepositoriesDuring reading Reinforcement Learning: An Introduction by Sutton and Barto (SE, 2012), I did some exercises in the book. I created a <a href="https://github.com/jaara/RL">git repository</a> where I will share some of those.<br />
<br />
Some interesting excercises with TensorFlow, Theano and Keras might be also wort of sharing. You can find them in <a href="https://github.com/jaara/ai_examples">this git repository</a>.jaarahttp://www.blogger.com/profile/15349060338114486646noreply@blogger.com0tag:blogger.com,1999:blog-7323991810934014019.post-43277023434608553242016-08-07T10:06:00.003-07:002016-08-07T10:06:46.452-07:00TensorFlow Segmentation FaultIf you happen to run into TensorFlow segmentation fault on MacOs, try symlinking libcuda.dylib to libcuda.1.dylib in /usr/local/cuda/lib seems to do the job. <a href="https://github.com/tensorflow/tensorflow/issues/3263#issuecomment-232184358">More on here</a>.jaarahttp://www.blogger.com/profile/15349060338114486646noreply@blogger.com0tag:blogger.com,1999:blog-7323991810934014019.post-11283487207709350812016-08-07T02:18:00.000-07:002016-08-07T02:18:44.436-07:00Installing CUDA on MacOSInstalling CUDA on MacOS developed to a bit of challenge so here is how to do:<br />
<br />
<ol>
<li>Download <a href="http://developer.download.nvidia.com/compute/cuda/7.5/Prod/local_installers/cuda_7.5.27_mac.dmg">cuda_7.5.27_mac.dmg</a> & install</li>
<li>Download <a href="https://developer.nvidia.com/cudnn">cuDNN</a> & install:<script src="https://gist.github.com/jaara/6d77587622b84047eff60a241fc77796.js"></script></li>
<li>Test CUDA:<script src="https://gist.github.com/jaara/38e032f9f521965c10a2792139d57c1e.js"></script></li>
</ol>
It failed on with with "ld: framework not found CUDA" error. The solution was to edit every Makefile and replace "<span style="background-color: white; font-family: "lucidagrande"; font-size: 9.8pt;">-framework CUDA" with "</span><span style="background-color: white; font-family: "lucidagrande"; font-size: 9.8pt;">-F/Library/Frameworks -framework CUDA". I actually used IntelliJ IDEA to do the job.</span><br />
<ol>
</ol>jaarahttp://www.blogger.com/profile/15349060338114486646noreply@blogger.com0tag:blogger.com,1999:blog-7323991810934014019.post-43640126275223053732016-08-06T23:28:00.001-07:002016-08-07T06:23:51.385-07:00ResourcesSo here goes my journey as an AI researcher. First, I'd like to point some interesting resources:<br />
<br />
<br />
<ul>
<li><a href="https://www.coursera.org/learn/machine-learning#syllabus">Coursera Machine Learning Course</a> - basic introduction into ML</li>
<li><a href="https://gym.openai.com/">OpenAI Gym</a> - framework for implementing AI algorithms</li>
<li><a href="https://www.tensorflow.org/">TensorFlow</a> - tool for creating neural networks</li>
<li><a href="https://deepmind.com/">Google DeepMind</a> - Google acquired company focusing on deep learning; interesting papers</li>
<li><a href="https://webdocs.cs.ualberta.ca/~sutton/book/the-book.html">Reinforcement Learning: An Introduction</a> - awesome book for RL, go for unfinished Second Edition</li>
<li><a href="http://neuralnetworksanddeeplearning.com/index.html">Neural Networks and Deep Learning</a> - a free online book about NN</li>
</ul>
jaarahttp://www.blogger.com/profile/15349060338114486646noreply@blogger.com0