Package: eeg-to-fmri
From EEG to fMRISetupIdeally, your machine has a GPU and is running Linux.First of all, please installanacondaat$HOME/anaconda3/. To setup the environment for this repository, please run the following commands:gitclonegit@github.com:DCalhas/eeg_to_fmri.gitcdeeg_to_fmriDownloadcudnn:wgethttps://developer.download.nvidia.com/compute/redist/cudnn/v8.0.1/cudnn-11.0-linux-x64-v8.0.1.13.tgzRun the configuration file:./config.shPlease make sure to set the path to the datasets directory correclty. This path is stored in an environment variable, so every time you activate the environment, the variable is set and used in the code as os.environ['EEG_FMRI_DATASETS'].How do I test this research on my dataset?Testing a new dataset on this framework should not be too difficult. Do the following (in the order you feel most comfortable):define the number of individuals,n_individuals_NEW, that your dataset contains, this can be done in thedata_utils.pyfile;additionally you may define new variables in thedata_utils.pyfile, corresponding ton_individuals_train_NEWandn_individuals_test_NEW, which refer to the number of individuals used for the training and testing set, respectively;definedataset_NEWvariable in thefmri_utils.pyandeeg_utils.pyfiles. At this point you might be thinking: "Why is this guy defining the same variable in two different places?", well he ain't too smart tbh and he lazy af;define the frequency,fs_NEW, at which the EEG recording was sampled, this can be done in theeeg_utils.pyfile;define the Time Response,TR_NEW, at which each fMRI volume was sampled, this can be done in thefmri_utils.py;additionally, you might want to define the list of channels (if your EEG electrode setup follows the10-20 system), to retrieve more advanced analysis, such as EEG electrode relevance. This should be done in the beginning of theeeg_utils.pyfile;last, but no least, comes the time to implement the two functions that read the EEG and fMRI recordings, corresponding toget_eeg_instance_NEW, ateeg_utils.py, andget_indviduals_path_NEW, atfmri_utils.py;In addition to reading the rest of this section, which helps you setting up your data, you also have available two blog posts:EEG recording to fMRI volume: goes over an example on how to operate with a simultaneous EEG and fMRI dataset and creates a model that synthesizes fMRI from EEG;Classification on EEG only datasets: this one picks up on the previous blog post and uses the pretrained model (that synthesizes fMRI from EEG), and shows you how to create an fMRI view of an EEG instance and classify it.Dataset structureIn this example, we assume your dataset has the following structure (if it has a different structure please interpret the code provided in the next two sections and adapt it):NEW
|	.
|	..
|	README.md
└───────EEG
|	└───────sub-001
|	|	|	FILE.eeg
|	|	|	FILE.vhdr
|	|	|	FILE.vmrk
|	└───────sub-002
|	...
└───────BOLD
	└───────sub-001
	|	|	FILE.anat
	|	|	FILE.nii.gz
	└───────sub-002
	...Implementing the get_eeg_instance_NEW functionIdeally you want this function to return anmne.io.Rawobject, that contains the EEG data. In this "tutorial" only this is the only supported option, however do it as you like most.The inputs of this function are:individual- int, the individual one wants to retrieve. This function is being executed inside a for loop,for individual in range(getattr(data_utils, "n_individuals_"+dataset), that goes through the range of individuals,n_individuals_NEW, you set in thedata_utils.pyfile;path_eeg- str, the path where your dataset is located, e.g.path_eeg=os.environ['EEG_FMRI_DATASETS']+dataset_NEW+"/EEG/", this may be an optional argument set aspath_eeg=os.environ['EEG_FMRI_DATASETS']+dataset_NEW+"/EEG/";task- str, can be set to None if it does not apply to your dataset;So given these inputs one can start by listing the directories of your dataset (now this can depend on how you organized the data, we assume that each individual has a folder dedicated to itself and the sorted names of the folders have the individual's folders first and after the auxiliary description ones, e.g. "info" for information about the dataset):defget_eeg_instance_NEW(individual,path_eeg=os.environ['EEG_FMRI_DATASETS']+dataset_NEW+"/EEG/",task=None,):individuals=sorted([fforfinlistdir(path_eeg)ifisdir(join(path_eeg,f))])individual=individuals[individual]path=path_eeg+individual+"/"print(path)#for debug purposes only, please remove this line after function is implementedThe output of the last print (ifindividuals=["sub-001", "sub-002", ..., "sub-"+data_utils.n_individuals_NEW, ...]):/tmp/"+dataset_NEW+"/EEG/sub-001/Inside the path described above should be a set of files needed to load a eeg brainvision object. If you sort these files, likely
the.vhdris the second option:brainvision_files=sorted([fforfinlistdir(path)ifisfile(join(path,f))])vhdr_file=brainvision_files[1]Now one only needs to return the Brainvision object:complete_path=path+vhdr_filereturnmne.io.read_raw_brainvision(complete_path,preload=True,verbose=0)In the endget_eeg_instance_NEWis:defget_eeg_instance_NEW(individual,path_eeg=os.environ['EEG_FMRI_DATASETS']+dataset_NEW+"/",task=None,):individuals=sorted([fforfinlistdir(path_eeg)ifisdir(join(path_eeg,f))])individual=individuals[individual]path=path_eeg+individual+"/"brainvision_files=sorted([fforfinlistdir(path)ifisfile(join(path,f))])vhdr_file=brainvision_files[1]complete_path=path+vhdr_filereturnmne.io.read_raw_brainvision(complete_path,preload=True,verbose=0)Implementing the get_individuals_path_NEW functionNext step is to implement the function that retrieves the fMRI recordings of all individuals. We assume each individual's recording is save in an.nii.gzfile.The inputs of this function are:path_fmri- str, absolute path that specifies the location of your dataset, e.g.path_fmri=os.environ['EEG_FMRI_DATASETS']+dataset_NEW+"/BOLD/, this may be an optional argument set aspath_fmri=os.environ['EEG_FMRI_DATASETS']+dataset_NEW+"/BOLD/"resolution_factor- float, this is an optional argument that might not be used, please refer to thefunctionswhere this argument is used to grasp its function. WARNING: this variable is deprecated;number_individuals- int, this variables specifies the number of individuals in this dataset, it is specified in the function call asnumber_individuals=getattr(data_utils, "number_individuals_"+dataset);Given the absolute path of the data and the number of individuals one wants to retrieve, we can now start implementing the code. Let's start by listing the individuals and saving it in a list:defget_individuals_paths_NEW(path_fmri=os.environ['EEG_FMRI_DATASETS']+dataset_NEW+"/BOLD/",resolution_factor=None,number_individuals=None):fmri_individuals=[]#this will be the output of this functiondir_individuals=sorted([fforfinlistdir(path_fmri)ifisdir(join(path_fmri,f))and"sub"inpath_fmri+f])print(dir_individuals)[os.environ['EEG_FMRI_DATASETS']+dataset_NEW+"/BOLD/sub-001",os.environ['EEG_FMRI_DATASETS']+dataset_NEW+"/BOLD/sub-002",...,os.environ['EEG_FMRI_DATASETS']+dataset_NEW+"/BOLD/sub-"+data_utils.n_individuals_NEW,...]Now we can move on to start the loop, where one iterates over each individuals' directory and loads the recording:for i in range(number_individuals):
		task_file=sorted([f for f in listdir(path_fmri+dir_individuals[i]) if isfile(path_fmri+dir_individuals[i]+f) and task in path_fmri+dir_individuals[i]+f])

		print(task_file)We assume that inside the individuals' folder, you will have an ".nii.gz" file and an additional ".anat" file. When sorted this list will have the ".nii.gz" file in the second place:[os.environ['EEG_FMRI_DATASETS']+dataset_NEW+"/BOLD/sub-001/FILE.anat", os.environ['EEG_FMRI_DATASETS']+dataset_NEW+"/BOLD/sub-001/FILE.nii.gz"]Therefore we pick the second file and use thenilearnlibrary to load the image:file_path=path_fmri+dir_individuals[i]+task_file[1]fmri_individuals+=[image.load_img(file_path)]returnfmri_individualsIn the end, this function is as:defget_individuals_paths_NEW(path_fmri=os.environ['EEG_FMRI_DATASETS']+dataset_NEW+"/BOLD/",resolution_factor=None,number_individuals=None):fmri_individuals=[]#this will be the output of this functiondir_individuals=sorted([fforfinlistdir(path_fmri)ifisdir(join(path_fmri,f))and"sub"inpath_fmri+f])foriinrange(number_individuals):task_file=sorted([fforfinlistdir(path_fmri+dir_individuals[i])ifisfile(path_fmri+dir_individuals[i]+f)andtaskinpath_fmri+dir_individuals[i]+f])file_path=path_fmri+dir_individuals[i]+task_file[1]fmri_individuals+=[image.load_img(file_path)]returnfmri_individualsMy dataset has fMRI volumes with higher resolutions than accepted by this work. What should I do?Unfortunately, the model only accepts:EEG instances with 64 channels and a total of 134 frequency resolutions in a specified window ofTR_*;fMRI instances with 64x64x30 resolution.For the EEG, we do not have a specified studied solution, just pray that it works.For the fMRI, we found that mutating the resolution via Discrete Cosine Transform (DCT) spectral domain is a reasonable work around. To do this you only need to add the specified lines to theget_individuals_paths_NEWand adownsampleanddownsample_shapeoptional arguments to the function call:defget_individuals_paths_NEW(path_fmri=os.environ['EEG_FMRI_DATASETS']+dataset_NEW+"/BOLD/",resolution_factor=None,number_individuals=None,downsample=True,downsample_shape=(64,64,30)):...if(downsample):importsyssys.path.append("..")fromlayersimportfftdct=Noneidct=None...Import the modules to perform the DCT and either add or remove resolutions to fit your data to the desired shapes. Then when you load the image, you should mutate it as:...fmri_individuals+=[image.load_img(file_path)]if(downsample):img=np.swapaxes(np.swapaxes(np.swapaxes(fmri_individuals[-1].get_fdata(),0,3),1,2),1,3)if(dctisNoneandidctisNone):dct=fft.DCT3D(*img.shape[1:])idct=fft.iDCT3D(*downsample_shape)fmri_individuals[-1]=image.new_img_like(fmri_individuals[-1],np.swapaxes(np.swapaxes(np.swapaxes(idct(dct(img).numpy()[:,:downsample_shape[0],:downsample_shape[1],:downsample_shape[2]]).numpy(),0,3),0,2),0,1))returnfmri_individualsAfter this, you should be set to run the code and retrieve the results you desire.Run the code and retrieve resultsNow you just need to run themain.pyfile with your dataset identifier given as an argument. Please refer to thedocumentation, where you will find what you need to give as arguments, an example call is (open shell):cdeeg_to_fmri/src
condaactivateeeg_fmri
mkdir/tmp/eeg_to_fmri
mkdir/tmp/eeg_to_fmri/metrics
pythonmain.pymetricsNEW-na_path_eeg../na_models_eeg/na_specification_2-na_path_fmri../na_models_fmri/na_specification_2-save_metrics-metrics_path/tmp/eeg_to_fmri/metricsBlog postsThis repository goes along with blog posts done during my PhD course:EEG recording to fMRI volume;Classification on EEG only datasets;AcknowledgementsWe would like to thank everyone atINESC-IDthat accompanied the journey throughout my PhD. This work was supported by national funds through Fundação para a Ciência e Tecnologia (FCT), under the Ph.D. Grant SFRH/BD/5762/2020 to David Calhas and INESC-ID pluriannual UIDB/50021/2020.Cite this repositoryIf you use this software in your work, please cite it using the following metadata:@article{calhas2022eeg,
  title={EEG to fMRI Synthesis Benefits from Attentional Graphs of Electrode Relationships},
  author={Calhas, David and Henriques, Rui},
  journal={arXiv preprint arXiv:2203.03481},
  year={2022}
}LicenseMIT License