TRAINING NEURAL NETWORKS FOR VISUAL SERVOING

Abstract

Visual servoing is a technique which uses feedback from vision sensor to dynamically manipulate the joints of the robot for motion and predicting required posture. The classical visual servoing applies several cameras and computer vision techniques for coordinating the motions of the robot. Therefore, it heavily relies on algorithms of feature extraction and tracking of coordinates position, processing visual features of the environment. The initial attempts of applying revolution of the computer vision, deep learning and convolutional neural networks were used in 2018 and achieved great results in prediction of posture of the robot on the image. In this thesis project I propose potential models which can be applicable in visual servoing without support of direct and classical methods of visual servoing and trained on synthetic dataset, which could be useful in diminishing robot hours. The results have shown great adaptability and resilience for fluctuations in images. Although the training process requires protracted time, the final model of the CNN with regressor output can accurately predict the pose of the robot both in output value positions and in simulation.