In-depth analysis of Reinforcement Learning and Formal Logic

  • Article

  • Literary Review

  • Reinforcement Learning

  • Logic

When teaching a child how to perform a task we usually tell them what they should do and what to avoid. Now imagine how hindering it would be to learning if the child could only learn from the feedback of its own experiences. Reinforcement Learning has this limitation. But, what if we could introduce prior knowledge into the training process?

In the past 3 to 4 years there has been an increase in the number of research papers focused on logic (linear temporal logic, modal temporal logic, etc.) as a tool to improve and assist traditional Reinforcement Learning. The idea is to use formal logic to represent knowledge that can be used during RL training to make the process faster and safer. This field of study captured my attention. I thought it could be an interesting subject to tackle on my Master's thesis, so I decided it would be a good idea to dive deeper into this subject by writing a critique on the published reaserch that focused Reinforcement learning and Logic [1 - 17 ].

This project had two main goals. First, to critically evaluate and provide an overview of the significant literature published on this topic. Second, this project would help me identify any potential gaps within the context of existing literature and assist me in defining research topics that I will attempt to address in my future thesis. I divided the body of work into three main topics; Safe Reinforcement Learning, Improving RL performance, and Generalization of RL policy learning. I added some more research papers that I examined but did not go in-depth.

Literary Review