Abstract

Over the past decade, researchers have used neural networks as function approximators for classical Reinforcement Learning (RL) algorithms and have achieved great success in robotic simulations and games. However, in many real-life problems, low-level state information is not directly accessible, meaning that latent representations of states have to be extracted from raw, high-dimensional image observations; also, the agent needs to be deployed in a different environment than the training environment. In these cases, naively applying baseline Deep Reinforcement Learning (DRL) algorithms usually results in poor sample efficiency, training instability, and overfitting in the training environment. Recent advances in Visual RL and Generalization in RL address these issues and promise a much broader application of RL. This report first reviews theoretical foundations of RL, defines terminologies, and covers classical RL algorithms and their modern extensions. Then, we survey methods that address DRL algorithms' fragility in generalization and learning from raw pixel input. We explore existing RL applications in Visual Computing and discuss how generalization and Visual RL have or would have influenced the performance of their algorithms. As a case study, we present Autonomous Aesthetic View Finding with Reinforcement Learning. It uses Visual RL and Meta RL methods in a challenging, novel application, where the agent is tasked to learn to move the camera and efficiently search for the most aesthetic views in 3D indoor scenes and generalize this skill to unseen scenes. We conclude with future extensions of this work.