llm-togetherai 0.1.0
LLM plugin for models hosted by Together AI
Requires Python: >=3.9
Keywords: llm, ai, together, language-models, plugin

# llm-togetherai

[![PyPI](https://img.shields.io/pypi/v/llm-togetherai.svg)](https://pypi.org/project/llm-togetherai/)
[![Changelog](https://img.shields.io/github/v/release/rajashekar/llm-togetherai?include_prereleases&label=changelog)](https://github.com/rajashekar/llm-togetherai/releases)
[![Tests](https://github.com/rajashekar/llm-togetherai/workflows/Test/badge.svg)](https://github.com/rajashekar/llm-togetherai/actions?query=workflow%3ATest)
[![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/rajashekar/llm-togetherai/blob/main/LICENSE)

[LLM](https://llm.datasette.io/) plugin for models hosted by [Together AI](https://www.together.ai/)

## Installation

First, [install the LLM command-line utility](https://llm.datasette.io/en/stable/setup.html).

Now install this plugin in the same environment as LLM.
```bash
llm install llm-togetherai
```

## Configuration

You will need an API key from Together AI. You can [obtain one here](https://api.together.xyz/settings/api-keys).

You can set that as an environment variable called `TOGETHER_API_KEY`, or add it to the `llm` set of saved keys using:

```bash
llm keys set together
```
```
Enter key: <paste key here>
```

## Usage

To list available models, run:
```bash
llm models list
```
You should see a list that looks something like this:
```
together: together/meta-llama/Llama-2-7b-chat-hf
together: together/meta-llama/Llama-2-13b-chat-hf
together: together/meta-llama/Llama-2-70b-chat-hf
together: together/mistralai/Mistral-7B-Instruct-v0.1
together: together/NousResearch/Nous-Hermes-2-Mixtral-8x7B-DPO
...
```

To run a prompt against a model, pass its full model ID to the `-m` option, like this:
```bash
llm -m together/meta-llama/Llama-2-7b-chat-hf "Five creative names for a pet robot"
```

You can set a shorter alias for a model using the `llm aliases` command like so:
```bash
llm aliases set llama2-7b together/meta-llama/Llama-2-7b-chat-hf
```
Now you can prompt the model using:
```bash
cat llm_togetherai.py | llm -m llama2-7b -s 'write some pytest tests for this'
```

### Vision models

Some Together AI models can accept image attachments. Run this command:

```bash
llm models --options -q together
```
And look for models that list these attachment types:

```
  Attachment types:
    image/gif, image/jpeg, image/png, image/webp
```

You can feed these models images as URLs or file paths, for example:

```bash
curl https://static.simonwillison.net/static/2024/pelicans.jpg | llm \
    -m together/meta-llama/Llama-3.2-11B-Vision-Instruct-Turbo 'describe this image' -a -
```

### Listing models

The `llm models -q together` command will display all available models, or you can use this command to see more detailed information:

```bash
llm together models
```
Output starts like this:
```yaml
- id: meta-llama/Llama-2-7b-chat-hf
  name: Llama-2-7b-chat-hf
  context_length: 4,096
  type: chat
  organization: Together
  pricing: input $0.2/M, output $0.2/M

- id: meta-llama/Llama-2-13b-chat-hf
  name: Llama-2-13b-chat-hf
  context_length: 4,096
  type: chat
  organization: Together
  pricing: input $0.3/M, output $0.3/M
```

Add `--json` to get back JSON instead:
```bash
llm together models --json
```

### Refreshing the model cache

The plugin caches the list of available models for 1 hour. To refresh this cache manually:

```bash
llm together refresh
```

This will fetch the latest models from the Together AI API and update the local cache.

## API Endpoint

This plugin uses the Together AI API endpoint:
```
https://api.together.xyz/v1/models
```

The models are cached locally in your LLM user directory to improve performance and reduce API calls.

## Development

To set up this plugin locally, first checkout the code. Then create a new virtual environment:
```bash
cd llm-togetherai
python3 -m venv venv
source venv/bin/activate
```
Now install the dependencies and test dependencies:
```bash
llm install -e '.[test]'
```
To run the tests:
```bash
pytest
```

## License

Apache 2.0