Light-perception-based interactive control of an underwater digital twin hand
基于光感知的水下数字孪生手的交互控制
水中デジタルツインハンドの光知覚に基づくインタラクティブ制御
빛 인식 기반의 수중 디지털 트윈 손의 상호작용 제어Control
interactivo basado en percepción de luz de una mano gemela digital submarina
Contrôle interactif basé sur la perception de la lumière d'une main numérique jumelle sous-marine
Интерактивное управление подводной цифровой двойниковой рукой на основе световосприятия
Jinlong Lu ¹ ², Chao Zhang ¹ ³, Hengchang Nong ¹ ², Dongying Wang ⁴, Hongyu Zhou ¹ ⁵, Junjie Weng ⁶, Yuehua Deng ¹ ⁷, Yang Yu ¹, Qiang Bian ⁴, Jianfa Zhang ⁸, Chaofan Zhang ⁹, Zhenrong Zhang ², Junbo Yang ¹
¹ College of Science, National University of Defense Technology, Changsha 410073, China
中国 长沙 国防科技大学理学院
² Key Laboratory of Multimedia Communication and Network Technology in Guangxi, School of Computer, Electronics and Information, Guangxi University, Nanning 530004, China
中国 南宁 广西大学计算机与电子信息学院 广西多媒体通信与网络技术重点实验室
³ School of Automation, Central South University, Changsha 410075, China
中国 长沙 中南大学自动化学院
⁴ College of Meteorology and Oceanography, National University of Defense Technology, Changsha 410073, China
中国 长沙 国防科技大学气象海洋学院
⁵ School of Optical and Electronic Information, National Engineering Research Center of Next Generation Internet Access-system, Huazhong University of Science and Technology, Wuhan 430074, China
中国 武汉 华中科技大学光学与电子信息学院 下一代互联网接入系统国家工程研究中心
⁶ College of Intelligence Science and Technology, National University of Defense Technology, Changsha 410073, China
中国 长沙 .国防科技大学智能科学学院
⁷ School of Physics, Central South University, Changsha 410083, China
中国 长沙 中南大学物理学院
⁸ Hunan Provincial Key Laboratory of Novel Nano-Optoelectronic, Information Materials and Devices, National University of Defense Technology, Changsha 410073, China
中国 长沙 国防科技大学新型纳米光电信息材料与器件湖南省重点实验室
⁹ College of Advanced Interdisciplinary Studies, National University of Defense Technology, Changsha 410073, China
中国 长沙 国防科技大学前沿交叉科学学院
With increasing demand for ocean exploration, underwater tasks become more complex. While soft robotic hands offer superior flexibility and biomimetic advantages over rigid robotic hands, their development has been hindered by low sensing accuracy and poor interactive control. Here, we propose a light-perception-based underwater digital twin hand interactive control system that integrates a multi-degree-of-freedom biomimetic pneumatic soft robotic hand with a wearable data glove based on fiber Bragg grating (FBG) sensors.
By fusing data from FBGs, RGB cameras, and inertial measurement units, and optimizing control strategies through reinforcement learning, we established a light-dominated perception-control-feedback closed-loop system that significantly enhances the control performance of the system in complex and dynamic underwater environments. With light as the information carrier, the system integrates a data glove with high strain sensitivity (1.148 pm/µε, linearity = 0.9996) and high bending sensitivity (26.667 pm/°, linearity = 0.9791). It achieves four-dimensional collaborative interactive control between the human operator, machine, virtual space, and environment.
The system achieves posture monitoring and precise grasping in various underwater environments, offering advantages like low cost, high precision, rapid response, and strong interference resistance. This establishes a new paradigm for underwater intelligent equipment by integrating light sensing with digital twin technology, offering broad prospects for future underwater operations.
