A database of hand kinematics, high-density sEMG of forearm and wrist for motion intent recognition

Abstract

Surface electromyography (sEMG) signals reflect spinal motor neuron activities and can be used as intuitive inputs for human-machine interaction (HMI) via movement intent recognition. The motor neuron potentials of far-field (wrist) and near-field (forearm) decomposed from high-density (HD) sEMG prospectively provide robust neural drives for HMI, which is a challenging research hotspot. However, there are no publicly available databases that include HD sEMG signals of forearm-wrist (FW) muscles, and hand kinematics (KIN). This paper presents the HD-FW KIN dataset that comprises HD 448-channel sEMG arrays distributed on the forearm and wrist with simultaneous recording of finger joint angles and finger flexion forces. This dataset contains muscle activities of 21 subjects while performing 20 hand gestures, and 9 individual or combined finger flexion under two force levels. The usability of HD-sEMG for hand gesture recognition, finger angle, and force prediction was validated. The proposed database allows a comprehensive extraction of the neural drive from the forearm and wrist, providing neural interfaces for the development of advanced prosthetic hand and wrist-worn consumer electronics.

Background

The surface electromyography (sEMG) signal of the underlying muscle comprises neural control information and is being increasingly utilized for the advancement of human-machine interaction (HMI) [1-3]. SEMG-based identification of human motion intention enables the usage of intuitive control commands in various HMI settings, including sophisticated prosthetic hands [4,5], exoskeletons [6,7], and virtual or augmented reality [8,9]. For many years, researchers have primarily studied the analysis of forearm muscles using sEMG to recognize movement intentions related to discrete and precise hand movement patterns. Nevertheless, the performance of myoelectric control is vulnerable to unpredictable situations in everyday life [10-14], especially when it comes to precisely controlling numerous degrees of freedom (DoFs) with advanced dexterous prostheses.

In theory, motor neurons connect with motor units through the neuromuscular junction, and sEMG signal represents an indirect representation of the action potentials of motor units [15]. High-density surface electromyography (HD-sEMG) measurement enables the analysis of motor neuron discharge characteristics, offering the potential for enhancing the reliability of myoelectric control [16-19]. Furthermore, the studies on surface electromyography (sEMG) based gesture recognition have primarily focused on the muscles in the upper forearm, particularly in the context of prosthesis use [20-23]. However, the focus in recent years has shifted towards the development of myoelectric interfaces as consumer electronics, rather than just rehabilitation technology. This shift has garnered significant attention [24-27]. The utilization of wrist sEMG signal for motion intent recognition interface is perceived as more socially acceptable by general customers [28]. Several studies have demonstrated the viability of using wrist sEMG signals to identify motor intention.

These studies have obtained a recognition accuracy of over 75% in predicting hand gestures, both in offline and real-time scenarios [29-31]. Furthermore, the blind source separation method was employed to derive far-field potentials from high-density wrist surface electromyography (sEMG). These potentials were then confirmed to accurately categorize both individual and combined finger movements [32]. The myoelectric control performance, which is now relatively poor, has not yet been improved by sophisticated machine learning approaches due to the contamination of wrist sEMG signals by crosstalk from various muscle tendons. Investigating the synchronous firing properties of motor neurons in both the forearm (near-field) and wrist (far-field) muscles is crucial to transmitting existing knowledge of forearm-based myoelectric gesture detection to a wrist-worn neural interface. Thus, it is essential to collect databases that include high-density surface electromyography (HD-sEMG) data of forearm-wrist muscles and hand kinematics to create innovative algorithms and methodologies for researchers. Obtaining dependable datasets that include both forearm and wrist HD-sEMG signals can be challenging, which may impede research advancement.

Methods

Twenty-one healthy subjects (19 males and 2 females) aged 21-35 years (26.47 ± 3.71 years) participated in the study. All participants were right-handed without neuromuscular disorders. Their forearm circumference ranged from 23-29 cm ( 25.87 ± 2.00 cm), and wrist circumference ranged from 14.5-18 cm (16.79 ± 0.93 cm). The research protocol was approved by the ethics committee of Shanghai Jiao Tong University (Approval No.: E20230257I).

Three 8×8 HD sEMG electrode arrays (GR10MM0808, OT Bioelettronica, Italy) were attached to the muscles around the forearm, and the distance of the electrode to the elbow olecranon was set as one-third of the forearm length. Additionally, the wrist HD-sEMG signals were collected using four 5×13 electrode grids (GR04MM1305, OT Bioelettronica, Italy), and the electrodes surrounded the circumference of the wrist adjacent to the head of the ulna, with two grids attached on the anterior part and the other two grids on the posterior part of the wrist. The ground electrode and reference electrode were placed around the olecranon separately.

Hand kinematics were captured by a 5DT Data Glove 14 Ultra (5DT Inc. USA), and the sampling frequency was set at 200 Hz which is far more than the movement frequency of fingers.

Moreover, the isometric press forces of the five fingers were recorded using a custom-made force measurement device (RFP 602, Runeskee Co.Ltd, China) with a sampling rate of 1000Hz.

Data Description

Data corresponding to three datasets are stored in three folders, namely "exp_01"(Session One) (55.1GB), "exp_02" (Session Two) (40.1GB), and "exp_03" (Session Three) (18.7GB). All the signal segments (both HD-sEMG signals, the ground truth of angle and ground truth force of press) were stored in WFDB format ("*.dat").

The first folder contains data for experimental session one, which includes 21 participants, with a total of 6 (6 repetitions) files.

The second folder contains data for experimental session two, which includes 21 participants, with a total of 6 (6 repetitions) files.

The third folder contains data for experimental session three, which includes 10 participants, with a total of 6 (6 repetitions) files.

Each filename has the format hdkw_kin_expAA_subjBB_CC_DD.dat, where AA is the folder index (taking values between 1-3), BB the subject index (1-21 for folder 1-2, and 1-10 for folder 3, respectively), CC for the repetition number (1-6), DD for the data source (emg, press, and glove). The prefix hdkw_kin refers to the name of the database.

Usage Notes

The proposed HD-FW KIN datasets have great potential usages in the field of naturally controlled prosthetic hands and human-computer interaction of consumer electronics, according to the recognition of human intention. Since our datasets contain 448-channels high-density sEMG signals, it is feasible to observe neural drive from the perspective of MU as well as to develop sophisticated MU decomposition algorithms by using our data. Beyond traditional machine learning and pattern classification methods, our datasets allow for the exploration of proportional control with regard to the continuous angles of fingers, benefitting from the hand kinematics recorded by data-glove.

By including the HD sEMG signals for individual finger and combined multi-finger flexion under different force levels, HD-FW KIN dataset also allows for the advancing research of muscle synergies and unsupervised recognition of multi-finger motions intention. It is important to note that we encourage the usage of HD wrist sEMG to incorporate prior knowledge from the forearm sEMG, since consumer electronics on the wrist are more comfortable and unobtrusive, putting aside the rehabilitation settings. Since the original data has not been processed by known digital filters, the experimental data can also be used to measure the performance of detection digital filters and the noise robustness of algorithms, among other non-conventional usage. However, it is worth noting that when conducting routine electromyographic signal applications, the data should be filtered first.

Ethics

The research protocol was approved by the ethics committee of Shanghai Jiao Tong University (Approval No.: E20230257I).

Acknowledgements

This work is supported in part by the National Natural Science Foundation of China (Grant Nos. 52227808, 52375021, 52175021), in part by Natural Science Foundation of Shanghai (Grant No. 23ZR1429700)

Conflicts of Interest

The authors have no conflicts of interest to declare.

References

1. Yu Y, Li J, Solomon SA, Min J, Tu J, Guo W, et al (2022). All-printed soft human-machine interface for robotic physicochemical sensing. Sci Robot. 7(67):1–12.
2. Guo L, Lu Z, Yao L (2021). Human-Machine Interaction Sensing Technology Based on Hand Gesture Recognition: A Review. IEEE Trans Human-Machine Syst. 51(4):300–9.
3. Zheng M, Crouch MS, Eggleston MS (2022). Surface Electromyography as a Natural Human-Machine Interface: A Review. IEEE Sens J. 22(10):9198–214.
4. Malešević N, Olsson A, Sager P, Andersson E, Cipriani C, Controzzi M, et al (2021). A database of high-density surface electromyogram signals comprising 65 isometric hand gestures. Sci Data. 8(1):1–10.
5. Li W, Shi P, Yu H (2021). Gesture Recognition Using Surface Electromyography and Deep Learning for Prostheses Hand: State-of-the-Art, Challenges, and Future. Front Neurosci. 15(April).
6. Burns MK, Pei D, Vinjamuri R (2019). Myoelectric control of a soft hand exoskeleton using kinematic synergies. IEEE Trans Biomed Circuits Syst. 13(6):1351–61.
7. Fu J, Choudhury R, Hosseini SM, Simpson R, Park JH (2022). Myoelectric Control Systems for Upper Limb Wearable Robotic Exoskeletons and Exosuits—A Systematic Review. Sensors. 22(21):1–31.
8. Cote-Allard U, Gagnon-Turcotte G, Phinyomark A, Glette K, Scheme E, Laviolette F, et al (2021). A Transferable Adaptive Domain Adversarial Neural Network for Virtual Reality Augmented EMG-Based Gesture Recognition. IEEE Trans Neural Syst Rehabil Eng. 29:546–55.
9. Toledo-Peral CL, Vega-Martínez G, Mercado-Gutiérrez JA, Rodríguez-Reyes G, Vera-Hernández A, Leija-Salas L, et al (2012). Virtual/Augmented Reality for Rehabilitation Applications Using Electromyography as Control/Biofeedback: Systematic Literature Review. Electron. 2022;11(14).
10. Jiang N, Dosen S, Muller KR, Farina D (2012). Myoelectric control of artificial limbs-is there a need to change focus?. IEEE Signal Process Mag. 29(5):148–52.
11. Farina D, Jiang N, Rehbaum H, Holobar A, Graimann B, Dietl H, et al (2014). The extraction of neural information from the surface EMG for the control of upper-limb prostheses: emerging avenues and challenges. IEEE Trans Neural Syst Rehabil Eng. 22(4):797–809.
12. Resnik L, Huang HH, Winslow A, Crouch DL, Zhang F, Wolk N (2018). Evaluation of EMG pattern recognition for upper limb prosthesis control: A case study in comparison with direct myoelectric control. J Neuroeng Rehabil. 15(1):1–13.
13. Cognolato M, Gijsberts A, Gregori V, Saetta G, Giacomino K, Hager AGM, et al (2020). Gaze, visual, myoelectric, and inertial data of grasps for intelligent prosthetics. Sci Data. 7(1):1–15.
14. Shi P, Zhang X, Li W, Yu H (2022). Improving the Robustness and Adaptability of sEMG-Based Pattern Recognition Using Deep Domain Adaptation. IEEE J Biomed Heal Informatics. 26(11):5450–60.
15. Muceli S, Poppendieck W, Holobar A, Gandevia S, Liebetanz D, Farina D (2022). Blind identification of the spinal cord output in humans with high-density electrode arrays implanted in muscles. Sci Adv. 8(46).
16. Farina D, Holobar A, Merletti R, Enoka RM (2010). Decoding the neural drive to muscles from the surface electromyogram. Clin Neurophysiol. 121(10):1616–23.
17. Chen C, Yu Y, Ma S, Sheng X, Lin C, Farina D, et al (2020). Hand gesture recognition based on motor unit spike trains decoded from high-density electromyography. Biomed Signal Process Control. 55:101637.
18. Stachaczyk M, Atashzar SF, Dupan S, Vujaklija I, Farina D (2020). Toward Universal Neural Interfaces for Daily Use: Decoding the Neural Drive to Muscles Generalises Highly Accurate Finger Task Identification across Humans. IEEE Access. 8:149025–35.
19. Guo W, Fang Y, Sheng X, Zhu X (2023). Measuring Motor Unit Discharge, Myofiber Vibration, and Haemodynamics for Enhanced Myoelectric Gesture Recognition. IEEE Trans Instrum Meas. 72:1–10.
20. Unanyan NN, Belov AA (2021). Design of upper limb prosthesis using real-time motion detection method based on EMG signal processing. Biomed Signal Process Control. 70(19):103062.
21. Yadav D, Veer K (2023). Recent trends and challenges of surface electromyography in prosthetic applications. Biomed Eng Lett. 13(3):353–73.
22. D’Accolti D, Clemente F, Mannini A, Mastinu E, Ortiz-Catalan M, Cipriani C (2023). Online Classification of Transient EMG Patterns for the Control of the Wrist and Hand in a Transradial Prosthesis. IEEE Robot Autom Lett. 8(2):1045–52.
23. Jiang N, Chen C, He J, Meng J, Pan L, Su S, et al (2023). Bio-robotics research for non-invasive myoelectric neural interfaces for upper-limb prosthetic control: a 10-year perspective review. Natl Sci Rev. 10(5).
24. Yang YH, Ruan SJ, Chen PC, Liu YT, Hsueh YH (2020). A Low-Cost Wireless Multichannel Surface EMG Acquisition System. IEEE Consum Electron Mag. 9(5):14–9.
25. He J, Jiang N (2020). Biometric From Surface Electromyogram (sEMG): Feasibility of User Verification and Identification Based on Gesture Recognition. Front Bioeng Biotechnol. 8:1–10.
26. Holobar A, Farina D (2021). Noninvasive Neural Interfacing with Wearable Muscle Sensors: Combining Convolutive Blind Source Separation Methods and Deep Learning Techniques for Neural Decoding. IEEE Signal Process Mag. 38(4):103–18.
27. Jiang X, Xu K, Liu X, Dai C, Clifton DA, Clancy EA, et al (2021). Neuromuscular Password-Based User Authentication. IEEE Trans Ind Informatics. 17(4):2641–52.
28. Botros FS, Phinyomark A, Scheme EJ (2022). Day-to-Day Stability of Wrist EMG for Wearable-Based Hand Gesture Recognition. IEEE Access. 10:125942–54.
29. Jiang X, Merhi LK, Xiao ZG, Menon C (2017). Exploration of Force Myography and surface Electromyography in hand gesture classification. Med Eng Phys. 41:63–73.
30. Jiang S, Lv B, Guo W, Zhang C, Wang H, Sheng X, et al (2018). Feasibility of wrist-worn, real-time hand, and surface gesture recognition via sEMG and IMU Sensing. IEEE Trans Ind Informatics. 14(8):3376–85.
31. Botros FS, Phinyomark A, Scheme EJ (2022). Electromyography-Based Gesture Recognition: Is It Time to Change Focus from the Forearm to the Wrist? IEEE Trans Ind Informatics. 18(1):174–84.
32. Mendez Guerra I, Barsakcioglu DY, Vujaklija I, Wetmore DZ, Farina D (2022). Far-field electric potentials provide access to the output from the spinal cord from wrist-mounted sensors. J Neural Eng. 19(2).