ACTIVE OBJECT TRACKING USING REINFORCEMENT LEARNING

Abstract

The concept of "smart cities" has rapidly emerged as the means by which urban planners can improve the quality of life of citizens, providing better services at lower cost. Typical objectives include the optimization of traffic routing, the automatic detection of emergency "events" and related improvement in the response time of emergency services, and overall optimization of resource allocation and energy consumption. A core component of the smart city concept is the widespread deployment of closedcircuit cameras for purposes of monitoring and event detection. A typical application is to locate and track a vehicle as it moves through crowded urban scenarios. Usually, tracking and camera control tasks are separated, which induces problems for the construction of a coherent system. Reinforcement learning can be used to unify the systems, such that control and tracking can be resolved simultaneously. However, there are issues related to the collection and use of comprehensive real-world data sets for purposes of research. To avoid this problem, it is feasible to conduct the agent training using synthetic data, and then transfer the results to real-world settings. This approach also serves to address the issue of domain invariance. For the thesis, I investigate active object tracking using reinforcement learning by first developing a synthetic environment based on the videogame Cities: Skylines, using the extensive Unity engine, which accurately simulates vehicle traffic in urban settings. The complete system consisting of a trained object detector and a reinforcement learning agent is tuned in this environment with corresponding reward functions and action space. The resulting agent is capable of tracking the objects in the scene without relying on domain-specific data, such as spatial information. The thesis includes the creation of the synthetic environment, the development of the agent, and the evaluation of the resulting system.