結合 Focal Loss 之 CenterNet 於數位彎曲感測器手勢辨識中的類別不平衡對策研究

Abstract

在Open-set場景中的手勢辨識應用中，背景資料往往具有數量龐大與高度變異的特性，對模型造成前景辨識上的挑戰。然而，在本研究所使用的數位彎曲感測器中，資料主要反映手指的彎曲與否，背景樣本雖然數量龐大，但變異度相對有限。基於此特性，本研究的問題核心轉為在樣本極度不平衡的情況下，有效抑制背景類別對模型學習造成的主導效應。為解決此問題，本研究提出一套結合 Focal Loss 與 CenterNet 概念的手勢辨識方法，並採用 Sliding Window 技術進行資料切分與時間特徵擷取。Focal Loss 能提升模型對少數前景手勢的關注度，提升前景手勢的學習效果，同時降低背景樣本的干擾。配合以手勢中心時間點為標註依據的設計，強化模型對手勢發生時機的掌握能力。在推論階段，系統設計雙門檻判斷機制進行手勢偵測，並以編輯距離衡量連續手勢序列的整體預測準確度。實驗結果顯示，本研究提出的方法可於高比例背景樣本的資料情境下穩定辨識各類前景手勢，並於連續手勢序列辨識任務中優於傳統交叉熵法，展現出在類別不平衡下的實用性與穩健性。

In gesture recognition applications under open-set scenarios, background data are often abundant and highly variable, posing challenges for foreground gesture recognition. However, in the digital flex sensor used in this study, the data primarily reflect the bending states of fingers, where the background samples, although large in quantity, exhibit relatively limited variability. Based on this characteristic, the core problem of this study shifts to effectively mitigating the dominance of background classes on model learning under extremely imbalanced sample distributions. To address this issue, this study proposes a gesture recognition method that integrates the concepts of Focal Loss and CenterNet, combined with a sliding window technique for data segmentation and temporal feature extraction. Focal Loss enhances the model's focus on minority foreground gestures, improving their learning effectiveness while reducing the interference from background samples. Along with the design of using the central time point of each gesture as the labeling reference, the model’s ability to capture the timing of gesture occurrences is strengthened. At the inference stage, a dual-threshold decision mechanism is designed for gesture detection, and the overall prediction accuracy of continuous gesture sequences is evaluated using the edit distance metric. Experimental results demonstrate that the proposed method can stably recognize various gestures in data scenarios with a high proportion of background samples, and achieves better performance than the conventional cross-entropy method in continuous gesture sequence recognition tasks, showing practicality and robustness under class imbalance.