The Modalities of Tactile Sensing on the basis of Vision and Magnetics

Abstract

Robust tactile sensing is essential for dexterous robotic manipulation, enabling machines to perceive object properties that are often occluded or imperceptible to computer vision. However, the diverse nature of tactile modalities—such as normal force, contact localization, surface texture, and bulk stiffness—presents a fundamental challenge for artificial sensing systems. Rather than attempting to engineer a single, monolithic sensor to capture all properties, this thesis systematically investigates two state-of-the-art transduction paradigms to extract four distinct tactile modalities. First, we present NUSense, an optical tactile sensor that extracts normal force and contact localization by leveraging the underexplored principle of shear strain detection. Second, we introduce the Ciliated Magnetic Dome (CMD) sensor, a hybrid magnetic architecture designed to simultaneously classify object texture and stiffness from a single grasp without the need for dynamic sliding motions. Through extensive experimental validation, we demonstrate that vision-based optical shear tracking is highly effective for macro-scale force estimation and localization, while multi-scale magnetic time-series analysis provides a robust framework for complex property recognition.