基於多動作辨識與手臂模仿學習之全向移動互動型機器人設計

Abstract

本文設計並實現一款具備多動作辨識與手臂模仿功能的全向移動互動型機器人。結合全方向移動平台、多自由度機械手臂與深度影像分析模組，實現即時人機互動與模仿操作。動作辨識部分，採用人體骨架關節資訊，並以三層堆疊之長短期記憶網路（LSTM）建構時序動作辨識模型，有效識別使用者之肢體動作與手勢。在機械手臂模仿學習方面，運用深度影像追蹤與骨架分析技術，實時擷取使用者雙手關節位置，並透過逆向運動學迭代學習算出對應的機械手臂控制指令，使機器手臂執行相似的人體手臂姿態，達成模仿效果。除人機互動外，系統亦具備物體追蹤與撿球能力，結合物件偵測與深度相機資訊，定位三維空間目標物，驅動全向輪移動平台與手臂完成自主撿取任務。為驗證系統功能與效能，本文進行了多項實驗，針對動作辨識準確率、姿態擷取精度、模仿控制以及物體定位與撿取進行測試與分析，藉此確認系統在實際應用中的可行性與穩定性。This paper designs and implements an omnidirectional interactive robot system equipped with multi-action recognition and arm imitation capabilities. The system integrates an omnidirectional mobile platform, a multi-degree-of-freedom robotic arm, and a depth image analysis module to achieve real-time human–robot interaction and imitation. For action recognition, the paper utilizes human skeletal joint information and employs a three-layer stacked Long Short-Term Memory (LSTM) network to construct a temporal action recognition model, enabling accurate identification of user body movements and gestures.For robotic arm imitation learning, the system uses depth image tracking and skeletal analysis to capture the user’s hand joint positions in real time. By applying iterative inverse kinematics learning, it computes the corresponding control commands for the robotic arm, enabling it to reproduce similar human arm poses. Beyond interaction, the paper also supports object tracking and ball-picking by combining object detection with depth camera data to locate 3D targets and control the omnidirectional platform and arm for autonomous retrieval. The performance of the system was validated through experiments on action recognition accuracy, pose estimation precision, imitation control, and object retrieval, thereby confirming the feasibility and stability of the proposed approach in practical applications.