TRAINING INTELLIGENT TENNIS ADVERSARIES USING SELF-PLAY WITH ML AGENTS

Abstract

In the game and robotics industries, the design of intelligent and interactive characters can be greatly enriched by advances in artificial intelligence. This approach is in need due to the biased and pre-programmed, and thus limited, nature of conventional algorithms. In contrast, machine learning approaches can educate these characters to have creative and independent behavior even in complex games. This study explores the possibility of training intelligent adversaries using self-play in a game of tennis, which is not yet competently researched. The agent is provided with the basic tennis rules and informed about the outcome (winning/losing). Given that, it is up to the agent to find out suitable behavior. The agents are placed into a visually, physically, and cognitively rich surrounding environment implemented in Unity. Reinforcement learning with proximal policy optimization is used to train one brain for two adversarial agents. The training is stabilized using incremental complexity modification based on curriculum learning. By having itself as a level matching opponent, the agent consistently improves its skills. The Elo rating system helps to quantitatively assess the performance by computing the relative skill level between two agents in a zero-sum game. The liberty in behavior during the training process has opened the possibility for agents to discover robust tactics that helped them to play on the same level as human opponents. Later, an agent’s brain can be employed for researching, benchmarking, and using as virtual non-playable characters.