CASUS Institute Seminar, Maksim Zhdanov, TU Dresden
Speech processing tasks can be used to shed light on the functional and structural aspects of the brain. The precise understanding of the kinetics of brain activity can be used to derive functional biomarkers for multiple pathologies that cannot be detected anatomically through clinical imaging. One of such pathologies is schizophrenia which is often followed by auditory verbal hallucinations (AVH). Within this project, I have made a step towards an in-depth examination of functional connectivity during listening tasks for three groups of people: control group, schizophrenia patients with AVH and schizophrenia patients without AVH. I propose an end-to-end explainable graph neural network-based EEG-speech modelling framework to 1) classify a person into a healthy/AVH/non-AVH group; 2) differentiate listening from silence for each of the groups; 3) recognize characteristic task-depending connections of the EEG electrodes. Experimental results show that the proposed model not only achieves state-of-the-art performance but also provides a researcher with meaningful explanations for its predictions regarding the functional connectivity of the patient.