Electroencephalogram (EEG) serves as an effective indicator of detecting fatigue driving. Utilizing the open accessible Shanghai Jiao Tong University Emotion Electroencephalography Dataset (SEED-VIG), driving states are divided into three categories including awake, tired and drowsy for investigation. Given the characteristics of mutual influence and interdependence among EEG channels, as well as the consistency of the graph convolutional neural network (GCNN) structure, we designed an adjacency matrix based on the Pearson correlation coefficients of EEG signals among channels and their positional relationships. Subsequently, we developed a GCNN for recognition. The experimental results show that the average classification accuracy of driving state categories for 20 subjects, from the SEED-VIG dataset under the smooth feature of differential entropy (DE) linear dynamic system is 91.66%. Moreover, the highest classification accuracy can reach 98.87%, and the average Kappa coefficient is 0.83. This work demonstrates the reliability of this method and provides a guideline for the research field of safe driving brain computer interface.