Package: bia-bob
bia-bobBIAbobis a Jupyter-based assistant for interacting with data using large language models which generate python code.
It can make use ofOpenAI's chatGPT,Google's Gemini,Helmholtz' blabladorandOllama.
You need an OpenAI API account or a Google Cloud account or a Helmholtz ID account to use it.
Using it with Ollama is free but requires running an Ollama server locally.[!CAUTION]
When using the OpenAI, Google Gemini API or any other endpoint via BiA-Bob, you are bound to the terms of service
of the respective companies or organizations.
The prompts you enter are transferred to their servers and may be processed and stored there.
Make sure to not submit any sensitive, confidential or personal data. Also using these services may cost money.UsageYou can initializeboblike this:from bia_bob import bobCode generationYou can ask Bob to generate code like this:%bob Load blobs.tif and show itIt will then respond with a python code snippet that you can execute (see full example):Bug fixingBob can fix simple bugs in code you executed. Just add%%fixon top of the cell right after the error happened.Code documentationUsing the%%docmagic, you can generate documentation for a given code cell.No-code custom Jupyter KernelIf installed, you can also choose the BiA-Bob from the Launcher in Jupyter lab:This will give you a Jupyter kernel that allows you to type in English language instead of code:Example notebooksBasic demoBio-image analysis workflowChoosing a modelUsing Gemini and chatGPT altenatingInterpreting imagesDocumenting codeFixing bugsGraphical user interfacesExploring tabular dataKnown issuesIf you want to askboba question, you need to put a space before the?.%bob What do you know about blobs.tif ?InstallationYou can installbia-bobusing pip. it is recommended to install it into via conda/mamba environment. If you have never used conda before, pleaseread this guide first.It is recommended to installbia-bobin a conda-environment together with useful tools for bio-image analysis.mamba env create -f https://github.com/haesleinhuepf/bia-bob/raw/main/environment.ymlYou can then activate this environment...mamba activate bob_env... and installbia-bob.Using OpenAI as backendpip install bia-bob openai(Recommended openai version >= 1.2.0)Create an OpenAI API Key and add it to your environment variables as explained onthis page.Using custom backendsCustom endpoints can be used as well if they support the OpenAI API. Examples areblabladorandollama.
An example is shown inthis notebook:For this, just install the openai backend as explained above (tested version: 1.5.0).If you want to use ollama and e.g. thecodellamamodel, you must runollama servefrom a separate terminal and then initialize bob like this:bob.initialize(endpoint='ollama', model='codellama')If you want to use blablado, which is free for German academics, just get an API key as explained onthis pageand store it int your environment asBLABLADOR_API_KEYvariable.bob.initialize(
    endpoint='blablador', 
    model='Mistral-7B-Instruct-v0.2')Custom end points can be used as well, for example like this:bob.initialize(
    endpoint='http://localhost:11434/v1', 
    api_key='my_key',
    model='codellama')Using Google's Cloud AI API as backendpip install bia-bob google-cloud-aiplatform(Recommended google-cloud-aiplatform version >= 1.38.1)To make use of the Google Cloud API, you need to create a Google Cloud accounthereand
a project within the Google cloud (for billing)here.
You need to store authentication details locally as explainedhere.
This requires installingGoogle Cloud CLI. In very short: run the installer and when asked, activate the "Run gcloud init' checkbox. Or run 'gcloud init' from the terminal yourself. Restart the terminal window.
After installing Google Cloud CLI, start a terminal and authenticate using:gcloud auth application-default loginFollow the instructions in the browser. Enter your Project ID (not the name). If it worked the terminal should approximately look like this:BiA-Bob Jupyter kernel (optional)If you want to use the BiA-Bob Jupyter kernel, please run additionally this command:python -m bia_bob installYou can check if it's installed by printing out the list of installed kernels:jupyter kernelspec listAnd you can uninstall them using this command:jupyter kernelspec uninstall bia-bobDevelopmentIf you want to contribute tobia-bob, you can install it in development mode like this:git clone https://github.com/haesleinhuepf/bia-bob.git
cd bia-bob
pip install -e .ExtensibilityIf you are maintainer of a Python library and want to make BiA-bob aware of functions in your library, you can extend Bob's knowledge usingentry-points. Add this to your librarysetup.cfg:[options.entry_points]
bia_bob_plugins =
    plugin1 = your_library._bia_bob_plugins:list_bia_bob_pluginsIn the above mentioned_bia_bob_plugins.pydefine this function (and feel to rename the function and the Python file):def list_bia_bob_plugins():
    """List of function hints for bia_bob"""
    return """
    * Computes the sum of a and b
    your_library.compute_sum(a:int,b:int) -> int
    
    * Determines the difference between a and b
    your_library.compute_difference(a:int, b:int) -> int
    
    """Note that the syntax should be pretty much as shown above: A bullet point with a short description and a code-snippet just below.
You can also generate thelist_bia_bob_pluginsfunction as demonstrated inthis notebook.
Please only list the most important functions. If the list of all plugins extending BiA-Bob becomes too long, the prompt will exceed the maximum prompt length.List of known Python libraries that provide extensions to Bob:pyclesperanto-prototypestackview(Feel free to extend this list by sending a pull-request)Similar projectsThere are similar projects:jupyter-aichatGPT-jupyter-extensionchapyternapari-chatGPTbioimageio-chatbotIssuesIf you encounter any problems or want to provide feedback or suggestions, please create a thread onimage.scalong with a detailed description and tag @haesleinhuepf .