chat-magics 0.0.75
LLM powered magic commands for Jupyter Notebooks
License: MIT
Requires Python: >=3.6

# Chat magic commands for Jupyter Notebooks

This is an experimental Python package that adds magic commands for interacting with a Large Language Model (LLM) like the one used by ChatGPT to your Jupyter notebooks. Using this package you can take away mundane tasks or get help for writing code and text in your notebooks directly within your notebook. 

**Important note about support**: As this package provides experimental features, there is no set roadmap for how features will be modified, added or removed. In addition, features may be changed, added or removed at any time without notice. There is no guarantee of future support and its use should be limited to non-production environments.

**Important note about data usage**: This package sends natural language prompts, responses and user content to OpenAI Language Models maintained by OpenAI or Microsoft. Be aware that the prompts, returned results and user content may be reveiwed by either organization. 



## Benefits
Using this package, you can configure and use LLM, right from your Jupyter notebook to author code, get insights on data, create visualizations, add comments, translate code and more, using any notebook client that supports IPython magic commands, including Jupyter, Jupyter Lab, VS Code, Google Colab, etc. 

Some example magic commands include [%%chat](#chat), [%%code](#code), [%%add_comments](#add_comments), [%%fix_errors](#fix_errors), [%%translate](#translate), and [%%image](#image).
<br><br>

## Installation

`pip install chat-magics`

<details>
<summary>If installing from TestPyPi...</summary>

`pip install --index-url https://test.pypi.org/simple/ --extra-index-url https://pypi.org/simple/ chat-magics`
</details>
<br><br>

[Azure OpenAI](https://azure.microsoft.com/en-us/products/cognitive-services/openai-service) is used to process chat-magic requests. Optionally, [OpenAI](https://openai.com) can also be used:
<details>
<summary>Installation Instructions for OpenAI</summary>

`pip install chat-magics-openai`

**Important note about OpenAI services**: Use of services from OpenAI is governed by separate terms between you and OpenAI, and Microsoft is not a party to such agreements.
</details>
<br><br>

## Usage

1. Load the chat-magics package in your notebook, by running: **`%load_ext chat_magics`**
2. Configure your API Key and other settings, by running: **`%set_processor`** command or setting environment variables. See [configure](#authentication-and-basic-configuration) for more info. 
3. Set data sharing level carefully, by running command: **`%set_sharing_level`**. See [data sharing with LLM service](#data-sharing-with-llm-service) for more info.
4. Now use any of [magic commands ](#commands)

Enjoy!
<br><br>

## Examples
See how you can use power of chat magics for your notebook authoring, data insights, visualizations, explanations, fixing errors and more. 

### Data Cleaning and Wrangling
Use **`%%code`** magic command to generate code using natural language to perform data cleaning operations. 

![Data Cleaning](./chat_magics/images/data-cleaning2.gif)

### Data Visualization
Use **`%%code`** magic command to generate code using natural language for creating charts and other visualizations. 

![Data Visualization](./chat_magics/images/data-visualize2.gif)

### Data Insights
Use **`%%code`** magic command to gain insights on data. Check out the [ChipsDataDemo](./chat_magics/examples/ChipsDataDemo.ipynb) for a data demo that shows how you can use magics for cleaning, gaining insights, visualizing data i.e. have data conversations. 

![Data Insights](./chat_magics/images/data-insights2.gif)

### Data Explanation
Use **`%chat -d`** magic command to get understanding of the data and its columns. 

![Data Explain](./chat_magics/images/data-explain2.gif)

### Code Generation
Use **`%%code`** magic command to generate code using natural langauge.

![Code Generation](./chat_magics/images/codegen.gif)

### Adding Comments
Use **`%%add_comments`** magic command to add comments to your code.

![Adding Comments](./chat_magics/images/add_comments.gif)

### Fixing Errors
Use **`%%fix_errors`** magic command to attempt fixing errors in your code.

![Fixing Errors](./chat_magics/images/fix_errors2.gif)

### Creating Tests
Use **`%%code`** magic command to create test cases for your code.

![Creating tests](./chat_magics/images/test_creation.gif)

### Code Translation
Use **`%%translate`** magic command to translate code from one language to another.

![Code Translation](./chat_magics/images/translate.gif)

### Math and Science Equations
Use **`%%chat`** magic command to write math and science equations, algorithms and proofs in markdown using Latex, Mathjax.

![Code Translation](./chat_magics/images/equations.gif)

### Getting Started
Here is an example of how you can get started. Read more at [configuring LLM](#authentication-and-basic-configuration)

![Code Translation](./chat_magics/images/configure.gif)

<br>
<br>

## Authentication and Basic Configuration

Choosing an LLM processor to power the chat-magics commands involves providing authentication credentials with the service and some basic configuration using one of the following mechanisms. We support 3 ways. Running **`set_processor`** magic command, or setting environment variables or using UI (in case of Azure OpenAI). 

### **`%set_processor`**
<details>

Usage: %set_processor [azure, openai] [-k, --key] [-e, --endpoint] [-m, --model] [-v, --version] [-u, --ui]

Use **`%set_processor`** to authenticate with your chosen LLM service. It is recommended that you use [Azure KeyVault](https://azure.microsoft.com/en-us/products/key-vault) or a similar secret storage solution for storing and retreiving your API key rather than storing it directly in your notebook. If a secret storage solution is not available, consider using [UI authentication for Azure](#ui-authentication-for-azure) or [use environment variables](#environment-variables) instead of using **`%set_processor`**.
<br>

<details>
<summary>Use Azure OpenAI</summary>
<br>

Authenticate using Azure KeyVault to set the Key, Endpoint and optionally model name and API version.

```python
%pip install azure-keyvault-secrets azure-identity

# Get secret from keyvault

from azure.identity import DeviceCodeCredential 
from azure.keyvault.secrets import SecretClient

# Use DeviceCodeCredential if remote compute. 
# You can also use ManagedIdentityCredential if Managed Identity is configured
credential = DeviceCodeCredential()  
secret_client = SecretClient(vault_url="<Keyvault-url>", credential=credential)
secret = secret_client.get_secret("aikey")

# Set key retrieved from keyvault
%set_processor azure -k {secret.value} -e <Azure OpenAI endpoint> [-m <model> -v <api_version>]
```

Not recommended: 
Authenticate using the following command to set the API key and endpoint. You can optionally provide a model and API version.
```shell
%set_processor azure -k <your AZURE_OPENAI_API_KEY> -e <your AZURE_OPENAI_API_ENDPOINT> [-m <MODEL> -v <VERSION>]
```
</details>
<details>
<summary>Use OpenAI</summary>
Authenticate using the following command to set the API key. You can optionally supply an endpoint, model, and API version.

```shell
%set_processor openai -k <OPENAI_API_KEY> [-e <OPENAI_API_ENDPOINT> -m <MODEL> -v <API_VERSION>]
```

Again, it is recommended that you make use of a secret storage service for supplying your API key. 
</details>
<br>

</details>
</details>
<br>

### Environment variables
<details>
Barring the use of a secret storage service, it is recommended that you authenticate chat-magics via your system's environment to avoid inadvertently exposing your key. Here are the environment variables that are automatically queried by chat-magics:
<br>

- `CHATMAGICS_PROCESSOR = <azure, openai>`

#### Azure OpenAI
- `AZURE_OPENAI_API_KEY = <your Azure OpenAI API key>`
- `AZURE_OPENAI_API_ENDPOINT = <your Azure OpenAI endpoint>` (ex: https://mydeployment.openai.azure.com)
- Optional: `AZURE_OPENAI_API_MODEL = <model name>` (ex: gpt-4) 
- Optional: `AZURE_OPENAI_API_VERSION = <api_version>` (ex: 2022-12-01)

You can obtain your Azure OpenAI API key and endpoint from the [Azure portal](https://portal.azure.com).


#### OpenAI
- `OPENAI_API_KEY = <your OpenAI API key>`
- Optional: `OPENAI_API_MODEL = <model name>` (ex: gpt-4)
- Optional: `OPENAI_API_ENDPOINT = <your OpenAI endpoint>` (ex: https://api.openai.com/v1)
- Optional: `OPENAI_API_VERSION = <api_version>` (ex: 2022-12-01)

You can obtain your OpenAI API key by visiting https://beta.openai.com/account/api-keys.

</details>
<br>

### UI authentication for Azure
<details>
Run the following command to invoke the UI. Sign in to Azure in the browser window that will open and then return to the notebook to choose your subscription, resource and model that you want to use. 

```shell
%set_processor azure -u
```

UI looks like this:

![Screenshot of Azure UI](./chat_magics/images/azureui.png)

Regardless of the method you choose, it is highly recommended to not store your API key in the notebook itself.
</details>
<br>

## Prompts
The prompts used for the LLM for the operations such as above are basic and included in the package. These are mostly as example to help you get started. You can customize and add more details to suit your scenario when using magic commands such as **`%%code`** and **`%%chat`**. At any time if you want to see the prompt and context being sent to LLM, you can use the **`%show_chat_context`** command. If you want to see the *exact* prompt that drove the the last LLM response, use **`%show_chat_context --last`**. You can also clear the context by running **`%clear_chat_context`** command or start a new task with custom prompt using the **`%%new_task`** command. 
<br><br>

## LLM Model
If a model is not specified, then one of the cheaper models generally available from either Azure or OpenAI (ex. ```gpt-35-turbo```) will be used. If using Azure, be sure to deploy the model in your subscription. To host model in Azure, please see https://azure.microsoft.com/en-us/products/cognitive-services/openai-service.

Also be aware that Azure and OpenAI don't necessarily use the same name for the same model. For example, the 3.5 turbo model is called ```gpt-3.5-turbo``` in OpenAI, but ```gpt-35-turbo``` on Azure.
<br><br>

## Data Sharing with LLM service
By default, only your natural language prompts and the LLM's response is shared with the LLM processor you've chosen. To realize the full power of the chat-magics package, allow more data to be shared by setting the sharing level to 4. Choose the setting carefully as more data will be shared. See [%set_sharing_level](#set_sharing_level) for further details.
<br><br>

---
## Reference

In general, you can use **`%[magic_command]?`** to see detailed help for each of the following commands.



## %%chat

<details>
<summary>Usage: %chat &lt;Question&gt;</summary>

Use **`%%chat`** to execute a general query. Example:

```shell
%%chat 
What is a Jupyter magic command?
```

Queries can be single or multiline.

Use **`-d`** or **`--data`** flag to describe the dataset and its columns. This will provide a short description of the dataset along with generating a markdown table with a short description of each column in the dataset. For example, for chips.csv table in the examples folder:

![Screenshot of describe dataset](./chat_magics/images/screenshot5.png)
</details>

<br>

## %%code

<details>
<summary>Usage: %code &lt;Coding Request&gt;</summary>

Use **`%%code`** for coding related queries. This will also auto-generate a code cell and populate it with the code response from Chat GPT. Example: 

```shell
%%code
bubble sort an array
```

Queries can be single or multi-line. 

#### Parameters to adjust %%code behavior
#### Language
By default, code responses are returned in Python. If you would like the response code in a different language (e.g. java, cpp, js,  etc.) specify the language using argument **--lang or -l** like so:

```shell
%%code --lang java
bubble sort an array
```
To change the default language use **`%set_language`** command. Example: **`%set_language java`**

#### Output location
The output of a [%%code](#code) query can go to one of several places using the **--output or -o** parameter.
1. ```next``` (default): Inserted into a new cell immediately following the current cell.
2. ```output```: The output region of the current cell. A variable called ```_chatcode``` is set with the contents of the code, allowing you to execute it via ```exec(_chatcode)``` or even store it for later execution.
3. ```current```: Replace the contents of the current cell, keeping original prompt uncommented.
4. ```current_commented```: Replace the contents of the current cell. Original prompt in comments.
5. ```variable```: Stored only in the ```_chatcode``` variable. See above.
6. ```none```: Ignored. This option is mainly intended to be used with **`%set_output`** to allow a previously generated notebook to be re-run without generating new content. 

Here's an example:
```shell 
%%code --lang java -output current
bubble sort an array
```

When executed the current cell will change to something like this:
```java
'''
%%code -l java -o current
bubble sort an array
'''
public static void bubbleSort(int[] arr) { 
    int n = arr.length; 
    int temp = 0; 
  
    for(int i=0; i < n; i++){  
        for(int j=1; j < (n-i); j++){  
            if(arr[j-1] > arr[j]){  
                //swap elements  
                temp = arr[j-1];  
                arr[j-1] = arr[j];  
                arr[j] = temp;  
            }  
        }  
    }  
}
```

The default can be globally set using the [%set_output](#set_output) magic. 

### Context sharing
The current context (see [%show_chat_context](#show_chat_context)) can be suppressed from being shared with the AI engine using **--no_context (-n)**.
</details>

<br>

## %%add_comments
<details>

Use **`%%add_comments`** to automatically add comments to the code in the cell. If you have a code in a cell that you would like to add comments to, simply add %%add_comments to the top of that cell and run the cell. Example: Say, I have the bubble_sort function in my cell with no comments. I add %%add_comments to the top of the cell and run the cell. The code then gets commented as shown below. It assumes the language of the code is what is set by default. See [%set_language](#set_language).

```python
%%add_comments

def bubble_sort(arr): 
    n = len(arr) 
  
    # Traverse through all array elements 
    for i in range(n): 
  
        # Last i elements are already in place 
        for j in range(0, n-i-1): 
  
            # traverse the array from 0 to n-i-1 
            # Swap if the element found is greater 
            # than the next element 
            if arr[j] > arr[j+1] : 
                arr[j], arr[j+1] = arr[j+1], arr[j]
```
</details>

<br>

## %%fix_errors
<details>

Use **`%%fix_errors`** to attempt fixing errors in the code in the cell. If you have a code in a cell that you would like to fix errors for, simply add %%fix_errors to the top of that cell and run the cell. Results are better if the sharing level is set to 3 or more (see [%set_sharing_level](#set_sharing_level)). Example: Say, I have the add function in my cell which gave me an error. I add %%fix_errors to the top of the cell and run the cell. This invokes LLM to attempt fixing the code. If the new code does not work you can always undo to get back to the previous code. It assumes the language of the code is what is set by default. See [%set_language](#set_language).

```python
%%fix_errors

def add(x,y): 
    return x+z

add(2,3)
```

</details>

<br>

## %%translate
<details>

Use **`%%translate`** to translate cell from one language to another. For example:

```C
%%translate --from_lang C

char ch;
FILE* ptr = fopen("test.txt", "r");

do {
    ch = fgetc(ptr);
    printf("%c", ch);
} while (ch != EOF);

fclose(ptr);
```
Will get translated to...
```python
with open("test.txt", "r") as f:
    ch = f.read(1)
    while ch != EOF:
        print(ch)
        ch = f.read(1)
f.close()
```


The target language can be changed using the --to_lang parameter. If not provided it defaults to the language set by [%set_language](#set_language). Note that unless you're running with a kernel that supports the translated cell, it will not execute correctly. 
</details>

<br>

## %%image
<details>

Use **`%%image`** to execute a query to generate an image using OpenAI's DALL.E image generation engine. Example:

```shell
%%image 
sunset on a beach
```
</details>  

<br>

## %%ignore
<details>

Use **`%%ignore`** to execute a code cell that you do not wish to be added to the context, regardless of what you have set the sharing level to. 

</details>
<br>

---
## Configuration

The following commands are used to configure and adjust how the magic commands operate.

## %set_model
<details>
<summary>Usage: %set_model &lt;model_name&gt;</summary>

Sets the model to be used for future prompts. Example: **%set_model** gpt-4
</details>
<br>

## %set_language
<details>
<summary>Usage: %set_language &lt;language_name&gt;</summary>

The default language for [%%code](#code) queries is set Python. To change the default, use **`%set_language`** command. Example: **`%set_language java`**
</details>

<br>

## %set_output
<details>
<summary>Usage: %set_output [next, output, current, current_commented, variable, none]</summary>

Generated code can be output in one of several ways:
1. ```next``` (default): Inserted into a new cell immediately following the current cell.
2. ```output```: The output region of the current cell. A variable called ```_chatcode``` is set with the contents of the code, allowing you to execute it via ```exec(_chatcode)``` or even store it for later execution.
3. ```current```: Replace the contents of the current cell, keeping original prompt uncommented.
4. ```current_commented```: Replace the contents of the current cell. Original prompt is stored in comments.
5. ```variable```: Stored only in the ```_chatcode``` variable. See above.
6. ```none```: Ignored. This option is mainly intended to be used with **`%set_output`** to allow a previously generated notebook to be re-run without generating new content. 

This magic changes the default for future [%%code](#code) prompts. Example: **`%set_output output`** will cause subsequently executed [%%code](#code) cells to only store generated code in the output for each cell rather than create new cells.
</details>

<br>

## %set_processor
* Documented [here](#set_processor)
<br>

## %set_sharing_level
<details>
<summary>Usage: %set_sharing_level [0, <b><i>1</i></b>, 2, 3, 4]</summary>

All the [%%code](#code) and [%%chat](#chat) prompts sent to the LLM, are accompanied with some history to set context for the prompt. This helps gets more useful response back from the LLM. Use [%set_sharing_level](#set_sharing_level) command to set the level of context to be shared with the LLM service. Example: **`%set_sharing_level 4`**

The default value is 1

Choose the level **carefully** since higher the level set, the more potentially sensitive data is shared with the backend service you've chosen to utilize. See [%set_processor](#set_processor) for details on the backend service choices.

- Level 0: No prior code, chats or output is being provided to the LLM processor. This means that the processor will be unable to provide contextually appropriate responses for most requests.
- Level 1: The input text and responses for prior [%%chat](#chat) and [%%code](#code) requests will be provided as context for responses in this session.
- Level 2: In addition to Level 1, source code for cells executed by you will be provided to the LLM processor to improve the relevance of future responses in this session.
- Level 3: In addition to Level 2, source code and output for cells executed by you will be provided to the LLM processor to improve the relevance of future responses in this session.
- Level 4: In addition to Level 3, source code, output, and summaries for dataframes or other interesting kernel objects that have been loaded in the kernel or data from other connectors loaded will be provided to the LLM processor to improve the relevance of future responses in this session.
</details>

<br>

## %show_chat_context
<details>
<summary>Usage: %show_chat_context [-l, --last] [-s, --string_style]</summary>

Use this to see the context that will be sent the next time you execute a [%%code](#code) or [%%chat](#chat) magic. Pass the --last parameter to see the entirety of the prompt that was last sent to the LLM service for processing. Pass the --string_style parameter to see how the message based prompt will get converted to a string when using older models, like text-davinci-003.
</details>

<br>

## new_task
<details>
<summary>Usage: %new_task &lt;Overall Task&gt;</summary>

**Aliases:** %clear_chat_context

Use this to clear the context built so far and start a new context. This is useful when you are starting a new thread or chat or want to clear the existing context for any reason. Contexts are sent when you execute a [%%code](#code) or [%%chat](#chat) magic.

To help the AI engine understand the overall context of what you are now working on, you can include a description as part of this magic command. For example:

```shell
%new_task I'm writing cells in a notebook using plotly as my visualization library.
```

</details>

---

## Intended uses
### Jupyter Notebook End User
1. Natural language queries for data cleaning operations, insights, visualizations to code in their notebook - The user enters a natural language description of the operation that they would like to perform on their dataset and gets back a code snippet that transforms or visualizes their data as specified.
2. Natural language queries for code generation, test generation, text, image or markdown generation, data descriptions in their notebook - The user enters natural language to author code, tests, explanations, descriptions.
3. Commands for adding comments, fixing errors, translating code in their notebook - The user uses magic commands to add comments to code, fixes errors in code, translate code from one language to another.

### Chat Magics Host 
It is possible to host the Chat Magics package to provide AI services for products that provide similar or identical functionality of Jupyter Notebook. In this case, all of the intended use cases supplied above apply. However, additional consideration of risks and mitigations by the integrator will be necessary.

## Potential harms and mitigations
**Generation of malware, incorrect or insecure code** - It is possible for the system to generate code that contains bugs or performs unintended actions. It is also possible for the code to contain malware or have security concerns.
* Mitigation steps taken by this package:
  * By default, the code is presented to the user and not automatically executed unless opted in by the user to do so. By default this allows user to inspect the code before execution.

* Recommended mitigation steps:
  - [ ] If using this package in a custom host, ensure that the UX/UI is designed to aid and encourage the user to understand the full impact of the code before accepting it. For example, support syntax highlighting/IntelliSense when showing the code to the user, and highlight changes in the dataset as a result of the code.
  - [ ] If using this package in custom host, include reminders throughout the UX/UI that the output is generated by AI, and that they are accountable for the final code.

**Sharing data with LLM servers** - It is possible for user data, code, code results, kernel and other object information to be shared with LLM servers.
* Mitigation steps taken by this package:
  * While sharing more data generally provides better results, by default the sharing is restricted to only the prompts entered by the user and results from LLM.
  * A command is available that allows the user to control how much is shared. See [**`%set_sharing_level`**](#set_sharing_level) for details.
  * A command is available that allows the user to view exactly what information will be shared with the next command. See [**`show_chat_context`**](#show_chat_context).
  * A `--no_context` parameter is available for every applicable command to prevent sharing of data on an as-needed basis.

* Recommended mitigation steps:
  - [ ] Use the magic command to set the appropriate sharing level.
  - [ ] View the context that will be shared with the next magic command invocation.
  - [ ] Use the --no_context parameter to prevent context from being sent for a specific magic command.

**Quality of service disparities** - While OpenAI LLMs support many natural languages, currently the prompt templates and post-processing steps are only in supported in English. Users who are not English speakers may experience a reduced level of quality.
* Mitigation steps:
  - [ ] Please open issues for translating prompts to languages supported by your product so they are appropriately prioritized. We also welcome contributions.

**Third party content regurgitation** - Generated code may be subject to licensing clauses. 
* Recommended mitigation steps:
  - [ ] When using the code in your app or product, the UI should display a disclaimer message about third party content regurgitation.

**Content harms** - The end user may be exposed to prohibited content or content that contains personal data about individuals regurgitated by the LLM. This package enables interacting with an LLM and there is a possibility for the model to hallucinate.
* Mitigations steps taken by this package:
  * To minimize harmful content the system provides few-shot learning examples to guide the LLM to produce output in the expected format and nature.
* Recommended mitigation steps:
  - [ ] Consider using the magic command to define your task. See [**`%%new_task`**](#new_task).
  - [ ] Use the built-in prompts only as samples.
  - [ ] Consider the integration of the appropriate Azure AI Platform services for harm mitigation to further post-process the results.

**Misuse** - The end user may misuse the system, especially when the input is free-form.
* Mitigation steps taken by the package:
  * To minimize misuse – the system provides few-shot learning examples and supports system messages to guide the LLM to produce output in the expected format and nature.
* Recommended mitigation steps:
  - [ ]  Limit the maximum length of free-form user input to reduce the chances of misuse.
---

## Contributing

For details on how to contribute code to this project see the [contributing guide](./CONTRIBUTE.md). 

Consider installing the chat_magics extensionin editable mode.  
`pip install -e chat_magics*/`  
## Trademarks

This project may contain trademarks or logos for projects, products, or services. Authorized use of Microsoft 
trademarks or logos is subject to and must follow 
[Microsoft's Trademark & Brand Guidelines](https://www.microsoft.com/en-us/legal/intellectualproperty/trademarks/usage/general).
Use of Microsoft trademarks or logos in modified versions of this project must not cause confusion or imply Microsoft sponsorship.
Any use of third-party trademarks or logos are subject to those third-party's policies.