Package: braingeneers
Braingeneers Python UtilitiesGetting StartedWelcome to theBraingeneers Python Utilitiesrepository! This package collects and provides various Python code and utilities developed as part of the Braingeneers project. The package adheres to the Python Package Authority (PyPA) standards for package structure and organization.ContributionWe welcome contributions from collaborators and researchers interested in our work. If you have improvements, suggestions, or new findings to share, please submit a pull request. Your contributions help advance our research and analysis efforts.To get started with your development (or fork), click the "Open with GitHub Codespaces" button below to launch a fully configured development environment with all the necessary tools and extensions.Instruction on how to contribute to this project can be found in theCONTRIBUTION.md.InstallationYou can installbraingeneerspyusingpipwith the following commands:Install from PyPI (Recommended)pipinstallbraingeneerspyInstall from GitHubpipinstall--force-reinstallgit+https://github.com/braingeneers/braingeneerspy.gitInstall with Optional DependenciesYou can installbraingeneerspywith specific optional dependencies based on your needs. Use the following command examples:Install with IoT, analysis, and data access functions (skips machine learning and lab-specific dependencies):pipinstall"braingeneers[iot,analysis,data]"Install with all optional dependencies:pipinstall"braingeneers[all]"Committing Changes to the RepoTo make changes and publish them on GitHub, please refer to theCONTRIBUTING.mdfile for up-to-date guidelines.Modules and Subpackagesbraingeneerspyincludes several subpackages and modules, each serving a specific purpose within the Braingeneers project:braingeneers.analysis: Contains code for data analysis.braingeneers.data: Provides code for basic data access, including subpackages for handling electrophysiology, fluidics, and imaging data.braingeneers.iot: Offers code for Internet of Things (IoT) communication, including a messaging interface.braingeneers.ml: Contains code related to machine learning, such as a high-performance PyTorch data loader for electrophysiology data.braingeneers.utils: Provides utility functions, including S3 access and smart file opening.S3 Access and Configurationbraingeneers.utils.s3wranglerThis module extends theawswrangler.s3 packagefor Braingeneers/PRP access. For API documentation and usage examples, please visit theofficial documentation.Here's a basic usage example:importbraingeneers.utils.s3wrangleraswr# Get all UUIDs from s3://braingeneers/ephys/uuids=wr.list_directories('s3://braingeneers/ephys/')print(uuids)braingeneers.utils.smart_open_braingeneersThis module configuressmart_openfor Braingeneers use on PRP/S3. When importing this version ofsmart_open, Braingeneers defaults will be autoconfigured. Note thatsmart_opensupports both local and S3 files, so it can be used for all files, not just S3 file access.Here's a basic usage example:importbraingeneers.utils.smart_open_braingeneersassmart_openwithsmart_open.open('s3://braingeneersdev/test_file.txt','r')asf:print(f.read())You can also safely replace Python's defaultopenfunction withsmart_open.open:importbraingeneers.utils.smart_open_braingeneersassmart_openopen=smart_open.openCustomizing S3 EndpointsBy default,smart_openands3wranglerare pre-configured for the standard Braingeneers S3 endpoint. However, you can specify a customENDPOINTif you'd like to use a different S3 service. This can be a local path or an endpoint URL for another S3 service (note thats3wrangleronly supports S3 services, not local paths, whilesmart_opensupports local paths).To set a custom endpoint, follow these steps:Set an environment variableENDPOINTwith the new endpoint. For example, on Unix-based systems:exportENDPOINT="https://s3-west.nrp-nautilus.io"Callbraingeneers.set_default_endpoint(endpoint: str)andbraingeneers.get_default_endpoint(). These functions will update bothsmart_openands3wrangler(if it's an S3 endpoint, local path endpoints are ignored bys3wrangler).Using the PRP Internal S3 EndpointWhen running a job on the PRP, you can use the PRP internal S3 endpoint, which is faster than the default external endpoint. To do this, add the following environment variable to your job YAML file:spec:template:spec:containers:-name:...command:...args:...env:-name:"ENDPOINT"value:"http://rook-ceph-rgw-nautiluss3.rook"Please note that this will only work on jobs run in the PRP environment. Setting theENDPOINTenvironment variable can also be used to specify an endpoint other than the PRP/S3.DocumentationThe docs directory has been set up usingsphinx-build -M html docs/source/ docs/build/to create a base project Documentation structure. You can add inline documentation (NumPy style) to further enrich our project's documentation. To render the documentation locally, navigate to thedocs/build/htmlfolder in the terminal and runpython3 -m http.server.Working in CodespacesProject Structuresrc/:This folder contains scripts and notebooks representing completed work by the team.pyproject.toml:This file follows the guidelines fromPyPAfor documenting project setup information.Customizing the DevcontainerThedevcontainer.jsonfile allows you to customize your Codespace container and VS Code environment using extensions. You can add more extensions to tailor the environment to your specific needs. Explore the VS Code extensions marketplace for additional tools that may enhance your workflow.For more information about Braingeneers, visit ourwebsite.