Announcement: New Reinforcement Learning Book with O'Reilly

I’m excited to announce that I have agreed with O’Reilly Media to write a new book on Reinforcement Learning. The contracts have just been signed and I’ve started the writing process. It is likely to take around a year to be released so I’m hoping that it will be ready around Summer 2020.

What is Reinforcement Learning?

Reinforcement Learning is an increasingly popular Machine Learning paradigm. In supervised Machine Learning you have to provide lots of examples that have ground truth. In unsupervised Machine learning, you don’t have the ground truth so it is often hard to optimise for anything. In Reinforcement Learning you provide an environment and a reward for achieving some goal.

The environment can come from either data or a simulation. The reward is often designed to match the problem at hand. An agent is then responsible for attempting to figure out a way of achieving the goal through observing the environment, performing actions and collecting the reward.

The beauty of this paradigm is that you don’t need to provide millions of labels that are used as the ground truth in supervised learning. You only need to provide a single reward signal that abstracts the goal you want to achieve. This allows us to solve many problems that are difficult to solve with supervised learning.

Why Should You be Interested?

There are a range of books out there all by very capable authors. One of the best is an academic textbook that is hard to read and doesn’t provide any industrial examples. The rest tend to be poorly edited with far too much code.

I am working on the premise that you want a book that is easy to read; a book that doesn’t contain pages and pages of code. You can read that on Github if you want to. You want a book that provides just enough mathematics to explain why the code is doing what it is, but not so much that it breaks the flow of reading. And you want to build up from the basics, to understand the fundamentals, not jump in at the latest paper implementation. Finally you want examples from a range of domains, not just Atari games.

I believe that if I can nail all of that, then this will be a very worthwhile read.

Why Write a Book Now?

I have been waiting for the confluence of the right topic at the right time and I believe this is about as good as it gets. In fact, the timing is probably a little late. But the topic is growing exponentially and is starting to be used in industry. I’m confident that in a few years it will be as mainstream as supervised learning. In fact, I think it will be a supplement to supervised learning; they will be used together.

Call for Help!

It takes a village to raise a book. I am looking for a range of people that can help me with this book. Specifically I am in need for people to discuss and bounce ideas off. I need people to proofread, edit and review early versions. I need people’s advice. I need your recommendations as to what you are excited about in the RL space.

I need help! :-)

If any of this takes your fancy, the please get in touch any way you would like. Email, twitter, linkedin, etc.

What’s Next?

I’m in the middle of my research right now. I’m in the lucky position that there is already quite a lot of content out there. The downside is that this is hot right now so the research is very hard to keep up with!

After I start writing the next task will be to set up a new website and start directing people there. So watch out for that.

And finally I need to start producing chapters! Wish me luck!