Research on fatigue recognition based on graph convolutional neural network and electroencephalogram signals_Journal of Biomedical Engineering

Authors：

LI Song ¹ ,  FU Yunfa ² , ZHANG Yan ¹ , LU Gong ¹

1. Intelligent System Laboratory of Qinghai University, Xining 810000, P. R. China;
2. Brain Cognition and Brain-computer Intelligence Integration Group, Kunming University of Science and Technology, Kunming 650500, P. R. China;

Corresponding author：

FU Yunfa, Email: fyf@ynu.edu.cn

Keywords：

Fatigue driving; Electroencephalogram signals; Graph convolutional neural network; Pearson correlation

DOI：

10.7507/1001-5515.202410003

Video：

Export PDF Favorites Scan Get Citation

Abstract Full text Figures/Tables Video References Cited by

Electroencephalogram (EEG) serves as an effective indicator of detecting fatigue driving. Utilizing the open accessible Shanghai Jiao Tong University Emotion Electroencephalography Dataset (SEED-VIG), driving states are divided into three categories including awake, tired and drowsy for investigation. Given the characteristics of mutual influence and interdependence among EEG channels, as well as the consistency of the graph convolutional neural network (GCNN) structure, we designed an adjacency matrix based on the Pearson correlation coefficients of EEG signals among channels and their positional relationships. Subsequently, we developed a GCNN for recognition. The experimental results show that the average classification accuracy of driving state categories for 20 subjects, from the SEED-VIG dataset under the smooth feature of differential entropy (DE) linear dynamic system is 91.66%. Moreover, the highest classification accuracy can reach 98.87%, and the average Kappa coefficient is 0.83. This work demonstrates the reliability of this method and provides a guideline for the research field of safe driving brain computer interface.

1.	Carney C, Mcgehee D, Harland K, et al. Using naturalistic driving data to assess the prevalence of environmental factors and driver behaviors in teen driver crashes. CompetitionPolicy Newsletter, 2015, 18(2): 48-52.
2.	Hu X Y, Lodewijks G. Exploration of the effects of task-related fatigue on eye-motion features and its value in improving driver fatigue-related technology. Transportation Research PartF: Traffic Psychology and Behaviour, 2021, 80: 150-171.
3.	Halomoan J, Ramli K, Sudiana D, et al. ECG-based driving fatigue detection using heart rate variability analysis with mutual information. Information, 2023, 14(10): 22-34.
4.	Pei Z, Wang H T, Bezerianos A, et al. EEG-based multiclass workload identification using feature fusion and selection. IEEE Transactions on Instrumentation and Measurement, 2020, 70: 1-8.
5.	Schmidt E M, McIntosh J S, Durelli L, et al. Fine control of operantly conditioned firing patterns of cortical- neurons. Experimental Neurology, 1978, 61(2): 349-369.
6.	Chuang C H, Pei L C, Li W K et al. Driver’s cognitive state classification toward brain computer interface via using a generalized and supervised technology//The 2010 International Joint Conference on Neural Networks (IJCNN). NewYork: IEEE, 2010: 1-7.
7.	杨慧舟, 刘云飞, 夏丽娟. 前额单通道脑电信号的疲劳特征提取及分类算法. 生物医学工程学杂志, 2024, 41(4): 732-741.
8.	龚子安, 顾正晖, 陈迪. 基于局部与全局特征集成网络的跨被试驾驶疲劳检测. 计算机科学, 2024: 1-20.
9.	Chai R, Naik G R, Nguyen T N, et al. Driver fatigue classification with independent component by entropy rate bound minimization analysis in an EEG-based system. IEEE Journal of Biomedical and Health Informatics, 2017, 21(3): 715-724.
10.	Zhang X, Lu D, Pan J, et al. Fatigue detection with covariance manifolds of electroencephalography in transportation industry. IEEE Transactions on Industrial Informatics, 2021, 17(5): 3497-3507.
11.	耿欣. 基于小波变换和神经网络的疲劳驾驶检测技术的研究. 沈阳: 东北大学, 2012.
12.	冯笑, 代少升, 黄炼. 基于可解释深度学习的单通道脑电跨受试者疲劳驾驶检测. 仪器仪表学报, 2023, 44(5): 140-149.
13.	Yeo B T, Krienen F M, Sepulcre J, et al. The organization of the human cerebral cortex estimated by intrinsic functional connectivity. Journal of Neurophysiology, 2011, 106(3): 1125-1165.
14.	Scarselli F, Gori M, Tsoi A C, et al. The graph neural network model. IEEE Transactions on Neural Networks, 2009, 20(1): 61-80.
15.	Zhong P, Wang D, Miao C. EEG-based emotion recognition using regularized graph neural networks. IEEE Transactions on Affective Computing, 2020, 13(3): 1290-1301.
16.	Hou Y, Jia S, Lun X, et al. GCNs-Net: a graph convolutional neural network approach for decoding time-resolved EEG motor imagery signals. IEEE Transaction on Neural Networks and Learning Systems, 2024, 35(6): 7312-7323.
17.	Zheng W L, Lu B L. A multimodal approach to estimating vigilance using EEG and forehead EOG. Journal of Neural Engineering, 2017, 14(2): 026017.
18.	柳长源, 李文强, 毕晓君. 基于RCNN-LSTM 的脑电情感识别研究. 自动化学报, 2022, 48(3): 917-925.

1. Carney C, Mcgehee D, Harland K, et al. Using naturalistic driving data to assess the prevalence of environmental factors and driver behaviors in teen driver crashes. CompetitionPolicy Newsletter, 2015, 18(2): 48-52.
2. Hu X Y, Lodewijks G. Exploration of the effects of task-related fatigue on eye-motion features and its value in improving driver fatigue-related technology. Transportation Research PartF: Traffic Psychology and Behaviour, 2021, 80: 150-171.
3. Halomoan J, Ramli K, Sudiana D, et al. ECG-based driving fatigue detection using heart rate variability analysis with mutual information. Information, 2023, 14(10): 22-34.
4. Pei Z, Wang H T, Bezerianos A, et al. EEG-based multiclass workload identification using feature fusion and selection. IEEE Transactions on Instrumentation and Measurement, 2020, 70: 1-8.
5. Schmidt E M, McIntosh J S, Durelli L, et al. Fine control of operantly conditioned firing patterns of cortical- neurons. Experimental Neurology, 1978, 61(2): 349-369.
6. Chuang C H, Pei L C, Li W K et al. Driver’s cognitive state classification toward brain computer interface via using a generalized and supervised technology//The 2010 International Joint Conference on Neural Networks (IJCNN). NewYork: IEEE, 2010: 1-7.
7. 杨慧舟, 刘云飞, 夏丽娟. 前额单通道脑电信号的疲劳特征提取及分类算法. 生物医学工程学杂志, 2024, 41(4): 732-741.
8. 龚子安, 顾正晖, 陈迪. 基于局部与全局特征集成网络的跨被试驾驶疲劳检测. 计算机科学, 2024: 1-20.
9. Chai R, Naik G R, Nguyen T N, et al. Driver fatigue classification with independent component by entropy rate bound minimization analysis in an EEG-based system. IEEE Journal of Biomedical and Health Informatics, 2017, 21(3): 715-724.
10. Zhang X, Lu D, Pan J, et al. Fatigue detection with covariance manifolds of electroencephalography in transportation industry. IEEE Transactions on Industrial Informatics, 2021, 17(5): 3497-3507.
11. 耿欣. 基于小波变换和神经网络的疲劳驾驶检测技术的研究. 沈阳: 东北大学, 2012.
12. 冯笑, 代少升, 黄炼. 基于可解释深度学习的单通道脑电跨受试者疲劳驾驶检测. 仪器仪表学报, 2023, 44(5): 140-149.
13. Yeo B T, Krienen F M, Sepulcre J, et al. The organization of the human cerebral cortex estimated by intrinsic functional connectivity. Journal of Neurophysiology, 2011, 106(3): 1125-1165.
14. Scarselli F, Gori M, Tsoi A C, et al. The graph neural network model. IEEE Transactions on Neural Networks, 2009, 20(1): 61-80.
15. Zhong P, Wang D, Miao C. EEG-based emotion recognition using regularized graph neural networks. IEEE Transactions on Affective Computing, 2020, 13(3): 1290-1301.
16. Hou Y, Jia S, Lun X, et al. GCNs-Net: a graph convolutional neural network approach for decoding time-resolved EEG motor imagery signals. IEEE Transaction on Neural Networks and Learning Systems, 2024, 35(6): 7312-7323.
17. Zheng W L, Lu B L. A multimodal approach to estimating vigilance using EEG and forehead EOG. Journal of Neural Engineering, 2017, 14(2): 026017.
18. 柳长源, 李文强, 毕晓君. 基于RCNN-LSTM 的脑电情感识别研究. 自动化学报, 2022, 48(3): 917-925.

Journal of Biomedical Engineering

Latest ArticlesResearch on fatigue recognition based on graph convolutional neural network and electroencephalogram signals

Abstract Full text Figures/Tables Video References Cited by

Format

Content