ACTION-DRIVEN TACTILE OBJECT EXPLORATION FOR SHAPE RECONSTRUCTION VIA OPTICAL TACTILE SENSORS

Abstract

We introduce an action-driven tactile exploration system using novel optical tactile sensors integrated into the gripper of a robot arm. These sensors consist of multiple silicone layers, with one layer featuring alternating yellow and red patterns. When this layer deforms — typically by stretching and reducing in thickness—the colored patterns shift. These changes are captured by an onboard camera and analyzed using a Convolutional Neural Network (CNN) algorithm. The gripper for the sensor was specifically designed and 3D printed to ensure the sensors operate correctly. The colored part of the sensor was isolated from the external light. We tested the sensor’s effectiveness in edge detection and localization using four different geometric objects. We evaluated our system using a diverse collection of objects in both medium and large sizes.