The effective classification of multi-task motor imagery electroencephalogram (EEG) is helpful to achieve accurate multi-dimensional human-computer interaction, and the high frequency domain specificity between subjects can improve the classification accuracy and robustness. Therefore, this paper proposed a multi-task EEG signal classification method based on adaptive time-frequency common spatial pattern (CSP) combined with convolutional neural network (CNN). The characteristics of subjects' personalized rhythm were extracted by adaptive spectrum awareness, and the spatial characteristics were calculated by using the one-versus-rest CSP, and then the composite time-domain characteristics were characterized to construct the spatial-temporal frequency multi-level fusion features. Finally, the CNN was used to perform high-precision and high-robust four-task classification. The algorithm in this paper was verified by the self-test dataset containing 10 subjects (33 ± 3 years old, inexperienced) and the dataset of the 4th 2018 Brain-Computer Interface Competition (BCI competition Ⅳ-2a). The average accuracy of the proposed algorithm for the four-task classification reached 93.96% and 84.04%, respectively. Compared with other advanced algorithms, the average classification accuracy of the proposed algorithm was significantly improved, and the accuracy range error between subjects was significantly reduced in the public dataset. The results show that the proposed algorithm has good performance in multi-task classification, and can effectively improve the classification accuracy and robustness.
Stroke is an acute cerebrovascular disease in which sudden interruption of blood supply to the brain or rupture of cerebral blood vessels cause damage to brain cells and consequently impair the patient's motor and cognitive abilities. A novel rehabilitation training model integrating brain-computer interface (BCI) and virtual reality (VR) not only promotes the functional activation of brain networks, but also provides immersive and interesting contextual feedback for patients. In this paper, we designed a hand rehabilitation training system integrating multi-sensory stimulation feedback, BCI and VR, which guides patients' motor imaginations through the tasks of the virtual scene, acquires patients' motor intentions, and then carries out human-computer interactions under the virtual scene. At the same time, haptic feedback is incorporated to further increase the patients' proprioceptive sensations, so as to realize the hand function rehabilitation training based on the multi-sensory stimulation feedback of vision, hearing, and haptic senses. In this study, we compared and analyzed the differences in power spectral density of different frequency bands within the EEG signal data before and after the incorporation of haptic feedback, and found that the motor brain area was significantly activated after the incorporation of haptic feedback, and the power spectral density of the motor brain area was significantly increased in the high gamma frequency band. The results of this study indicate that the rehabilitation training of patients with the VR-BCI hand function enhancement rehabilitation system incorporating multi-sensory stimulation can accelerate the two-way facilitation of sensory and motor conduction pathways, thus accelerating the rehabilitation process.
Control at beyond-visual ranges is of great significance to animal-robots with wide range motion capability. For pigeon-robots, such control can be done by the way of onboard preprogram, but not constitute a closed-loop yet. This study designed a new control system for pigeon-robots, which integrated the function of trajectory monitoring to that of brain stimulation. It achieved the closed-loop control in turning or circling by estimating pigeons’ flight state instantaneously and the corresponding logical regulation. The stimulation targets located at the formation reticularis medialis mesencephali (FRM) in the left and right brain, for the purposes of left- and right-turn control, respectively. The stimulus was characterized by the waveform mimicking the nerve cell membrane potential, and was activated intermittently. The wearable control unit weighted 11.8 g totally. The results showed a 90% success rate by the closed-loop control in pigeon-robots. It was convenient to obtain the wing shape during flight maneuver, by equipping a pigeon-robot with a vivo camera. It was also feasible to regulate the evolution of pigeon flocks by the pigeon-robots at different hierarchical level. All of these lay the groundwork for the application of pigeon-robots in scientific researches.
Speech imagery is an emerging brain-computer interface (BCI) paradigm with potential to provide effective communication for individuals with speech impairments. This study designed a Chinese speech imagery paradigm using three clinically relevant words—“Help me”, “Sit up” and “Turn over”—and collected electroencephalography (EEG) data from 15 healthy subjects. Based on the data, a Channel Attention Multi-Scale Convolutional Neural Network (CAM-Net) decoding algorithm was proposed, which combined multi-scale temporal convolutions with asymmetric spatial convolutions to extract multidimensional EEG features, and incorporated a channel attention mechanism along with a bidirectional long short-term memory network to perform channel weighting and capture temporal dependencies. Experimental results showed that CAM-Net achieved a classification accuracy of 48.54% in the three-class task, outperforming baseline models such as EEGNet and Deep ConvNet, and reached a highest accuracy of 64.17% in the binary classification between “Sit up” and “Turn over”. This work provides a promising approach for future Chinese speech imagery BCI research and applications.
With the development of brain-computer interface (BCI) technology and its translational application in clinical medicine, BCI medicine has emerged, ushering in profound changes to the practice of medicine, while also bringing forth a series of ethical issues related to BCI medicine. BCI medicine is progressively emerging as a new disciplinary focus, yet to date, there has been limited literature discussing it. Therefore, this paper focuses on BCI medicine, firstly providing an overview of the main potential medical applications of BCI technology. It then defines the discipline, outlines its objectives, methodologies, potential efficacy, and associated translational medical research. Additionally, it discusses the ethics associated with BCI medicine, and introduces the standardized operational procedures for BCI medical applications and the methods for evaluating the efficacy of BCI medical applications. Finally, it anticipates the challenges and future directions of BCI medicine. In the future, BCI medicine may become a new academic discipline or major in higher education. In summary, this article is hoped to provide thoughts and references for the development of the discipline of BCI medicine.
High-frequency steady-state asymmetric visual evoked potential (SSaVEP) provides a new paradigm for designing comfortable and practical brain-computer interface (BCI) systems. However, due to the weak amplitude and strong noise of high-frequency signals, it is of great significance to study how to enhance their signal features. In this study, a 30 Hz high-frequency visual stimulus was used, and the peripheral visual field was equally divided into eight annular sectors. Eight kinds of annular sector pairs were selected based on the mapping relationship of visual space onto the primary visual cortex (V1), and three phases (in-phase[0º, 0º], anti-phase [0º, 180º], and anti-phase [180º, 0º]) were designed for each annular sector pair to explore response intensity and signal-to-noise ratio under phase modulation. A total of 8 healthy subjects were recruited in the experiment. The results showed that three annular sector pairs exhibited significant differences in SSaVEP features under phase modulation at 30 Hz high-frequency stimulation. And the spatial feature analysis showed that the two types of features of the annular sector pair in the lower visual field were significantly higher than those in the upper visual field. This study further used the filter bank and ensemble task-related component analysis to calculate the classification accuracy of annular sector pairs under three-phase modulations, and the average accuracy was up to 91.5%, which proved that the phase-modulated SSaVEP features could be used to encode high- frequency SSaVEP. In summary, the results of this study provide new ideas for enhancing the features of high-frequency SSaVEP signals and expanding the instruction set of the traditional steady state visual evoked potential paradigm.
The brain-computer interface (BCI) based on motor imagery electroencephalography (EEG) shows great potential in neurorehabilitation due to its non-invasive nature and ease of use. However, motor imagery EEG signals have low signal-to-noise ratios and spatiotemporal resolutions, leading to low decoding recognition rates with traditional neural networks. To address this, this paper proposed a three-dimensional (3D) convolutional neural network (CNN) method that learns spatial-frequency feature maps, using Welch method to calculate the power spectrum of EEG frequency bands, converted time-series EEG into a brain topographical map with spatial-frequency information. A 3D network with one-dimensional and two-dimensional convolutional layers was designed to effectively learn these features. Comparative experiments demonstrated that the average decoding recognition rate reached 86.89%, outperforming traditional methods and validating the effectiveness of this approach in motor imagery EEG decoding.
Steady-state visual evoked potential (SSVEP) has been widely used in the research of brain-computer interface (BCI) system in recent years. The advantages of SSVEP-BCI system include high classification accuracy, fast information transform rate and strong anti-interference ability. Most of the traditional researches induce SSVEP responses in low and middle frequency bands as control signals. However, SSVEP in this frequency band may cause visual fatigue and even induce epilepsy in subjects. In contrast, high-frequency SSVEP-BCI provides a more comfortable and natural interaction despite its lower amplitude and weaker response. Therefore, it has been widely concerned by researchers in recent years. This paper summarized and analyzed the related research of high-frequency SSVEP-BCI in the past ten years from the aspects of paradigm and algorithm. Finally, the application prospect and development direction of high-frequency SSVEP were discussed and prospected.
Brain-computer interfaces (BCIs) have become one of the cutting-edge technologies in the world, and have been mainly applicated in medicine. In this article, we sorted out the development history and important scenarios of BCIs in medical application, analyzed the research progress, technology development, clinical transformation and product market through qualitative and quantitative analysis, and looked forward to the future trends. The results showed that the research hotspots included the processing and interpretation of electroencephalogram (EEG) signals, the development and application of machine learning algorithms, and the detection and treatment of neurological diseases. The technological key points included hardware development such as new electrodes, software development such as algorithms for EEG signal processing, and various medical applications such as rehabilitation and training in stroke patients. Currently, several invasive and non-invasive BCIs are in research. The R&D level of BCIs in China and the United State is leading the world, and have approved a number of non-invasive BCIs. In the future, BCIs will be applied to a wider range of medical fields. Related products will develop shift from a single mode to a combined mode. EEG signal acquisition devices will be miniaturized and wireless. The information flow and interaction between brain and machine will give birth to brain-machine fusion intelligence. Last but not least, the safety and ethical issues of BCIs will be taken seriously, and the relevant regulations and standards will be further improved.
Due to the high complexity and subject variability of motor imagery electroencephalogram, its decoding is limited by the inadequate accuracy of traditional recognition models. To resolve this problem, a recognition model for motor imagery electroencephalogram based on flicker noise spectrum (FNS) and weighted filter bank common spatial pattern (wFBCSP) was proposed. First, the FNS method was used to analyze the motor imagery electroencephalogram. Using the second derivative moment as structure function, the ensued precursor time series were generated by using a sliding window strategy, so that hidden dynamic information of transition phase could be captured. Then, based on the characteristic of signal frequency band, the feature of the transition phase precursor time series and reaction phase series were extracted by wFBCSP, generating features representing relevant transition and reaction phase. To make the selected features adapt to subject variability and realize better generalization, algorithm of minimum redundancy maximum relevance was further used to select features. Finally, support vector machine as the classifier was used for the classification. In the motor imagery electroencephalogram recognition, the method proposed in this study yielded an average accuracy of 86.34%, which is higher than the comparison methods. Thus, our proposed method provides a new idea for decoding motor imagery electroencephalogram.