Package: bbcpy
BBCpyWrite everything that is needed to be known to develop within this project into this readme.
This serves as a reminder to ourselves but also to facilitate collaboration.Don't hesitate too long to write what you find or know. If its wrong it still helps to progress from there.
If someone else knows better that is good for the project.
Try to describe also some 'easy' things that can be googled but not the very basic python stuff.
It's always good to document why we did certain things in certain ways.Basic StructureThere are different specific datatypes for each application. These come with a number of methods and are derived from
numpys np.ndarray. This makes them compatible with scikit-learn, pytorch and other toolboxes.Data structure and IndexingThe data structure is similar to scikit-learn. The first dimension is the epochs, the second the channels and the last
the time domain.Indexing also works with strings to direclty work with units or channel names e.g..#select all but some (here exclude all frontal and fronto-central) channels:
print(imagVPaw['~F*'].shape)
#can also be combined - selecting all frontal channels except for 7 and 8s e.g.:
print(imagVPaw[['F?', '~*7', '~*8']].shape)
print(imagVPaw[['F?,~*7,~*8']].chans)
#adding advanced slicing with units - e.g. 2s to 5s in steps of 100ms:
print(imagVPaw['~F*', '2s:5s:100ms'].shape)
#you may also select single time points:
print(imagVPaw[:, '2s,5s,100ms'].shape)
#the same can be separated in a list:
print(imagVPaw[:, ['2s', '5s', '100ms']].shape)

cnt_bp = imagVPaw.lfilter(band)
epo_bp = cnt_bp.epochs(ival)

#also works on epoched data, also with comma separation, each taken individually
print(epo_bp[:, ['C?,~*3,~*4', 'FC*,~*3,~*4']].chans)FunctionsThebbcpy.functionlibraries consists of different submodules that are named by their specification. Everything here
can be directly used within our scikit-learn compatiblebbcpy.pipeline.Everything inbbcpy.functionsthat starts with a capital letter (bbcpy.spatial.CSPe.g.) is directly implemented as
a transformer and needs to be fitted (.fit()) before being applied (.transform()).It is easy to import other functions into the toolbox usingbbcpy.functions.base.ImportFunc. You can also directly use
numpy compatible functions in the pipeline but that does not guarantee that the indivudal datatype is not lost and
(standard) parameters are not straightforwardly implemented.pipeline = bbcpy.pipeline.make_pipeline(np.mean)calculates the mean over all dimensions, whilepipeline = bbcpy.pipeline.make_pipeline(bbcpy.functions.base.ImportFunc(np.mean, axis=-1, outClass='same'))makes it
work in the time domain and keeps the class.You can also import a function to be used on data directly:
mean = bbcpy.functions.base.ImportFunc(np.mean, axis=-1, outClass='same')
mean(data)Demo.pyThere is a demo.py, which you can use to test some basic functionalities/play around etc.DevelopmentThis project defines specific requirement versions. To minimize version conflicts with already installed libraries
you should either set up a virtual environment or use a docker container for development.
In case you use an IDE (like PyCharm) you might need to configure it to use the virtual environment as-well.In case your installedpython --versiondiffers from the required 3.10 the docker approach may be appropriate.Daniel does not like that. We want a toolbox that rather works everywhere.
He thinks we should only specify a minimum version.
Also some of the selcted versions did lead to errors.Package StructureThe folder structure of the project is inline with the necessities of a package. There needs to be a folder named after
the project in order to be importable usingimport bbpy.whateveretc.
Within the bbcpy folder the actual code should go. The submodules are organized in subfolders.
The structure is sofar just a suggestion and debatable (best via github issues etc).
Refactoring works like a charm with pycharm.Every folder that has an__init__.pycan basically be imported.
The file can be empty but then nothing is imported byimport bbcpyonly the folder can be searched and imported
byfrom bbcpy import typese.g..Virtual EnvironmentPython 3 supports virtual environments natively with thevenv module.The following commands create the virtual environment.venvand install all dependencies defined inrequirements.txt. The virtual environment needs to be active in the terminal you work with.
Usesource .venv/bin/activateto activate it in the bash shell or.venv\Scripts\activate.baton Windows.
More information can be foundhere.Note:pythonmay refer to version 2 on your system, in that case usepython3in the commands instead.Example for Linux with bash shell:# create virtual environment "venv"python-mvenv.venv# activate environmentsource.venv/bin/activate# install dependenciespipinstall-rrequirements.txtDocker DEPRECATEDWARNING THIS DOCKER DOC IS DEPRECATED COPIED FROM AN OLD PROJECT. NEEDS TO BE UPDATED AS SOON AS WE'VE SET UP THE GITThe docker container contains python 3.8 and all dependencies from therequirements.txtat the time of building and is
based on a modern ubuntu image.To download from or publish to the container registry you need to log in first with your tu gitlab account.
In case you have 2FA enabled you need to use apersonal access tokeninstead of your password.
The scopesread_registryandwrite_registryshould suffice.The following command creates and starts the container namedbci-devand mounts the current directory inside.# login if you have not alreadydockerlogingit.tu-berlin.de:5000# run container and access it's terminaldockerrun-it--namebci-dev-v"$(pwd)":/homegit.tu-berlin.de:5000/roydick1.0/bcifntd2020ws:devYou can work and edit files normally and may test and execute from the command line within the container.The python executable ispython3.If you need other dependencies you can simply useaptorpipto install them inside the container or edit therequirements.txtand build your own image with theDockerfile.TestingCurrently, a badly copied and not working unit_testing with nose is implemented.
If you runpysetup.pytestIt complains to be deprecated and to rather use tox:WARNING: Testing via this command is deprecated and will be removed in a future version. Users looking for a generic test entry point independent of test runner are encouraged to use tox.To run nose alone, just type:#install nose first (see https://python-packaging.readthedocs.io/en/latest/testing.html)pipinstallnose#then run testsnosetestsThe tests should go into thebbcpy/testsdirectory.Building Wheels to use with pipCurrently and old-fashioned installation using setuptools is implemented which gives the following waring:SetuptoolsDeprecationWarning: setuptools.installer is deprecated. Requirements should be satisfied by a PEP 517 installer.In order to build wheels with pip, first run the setup.py and build the package (/build):pysetup.pybuildIn the setup.py all the settings for the installation etc are given.
This includes descriptions and info that will be contained on pypi. Please update.
Also the version number is initialized, here.After that, use build to construct the wheels distribution (/dist):#if you haven't so far, install buildpy-mpipinstall--upgradebuild#then run itpy-mbuildand then finally upload the wheel to pypi:#if you haven't so far, install twinepipinstalltwine#and then run it to upload to pypipy-mtwineupload--repositorypypidist/*Finally, the new version is uploaded and can be installed anywhere bypipinstallbbcpy