Package: emb3d
emb3demb3dis a command-line utility that lets you generate embeddings using models from OpenAI, Cohere and HuggingFace.Installationpipinstall--upgradeemb3dQuick Start ⚡️Install the librarypipinstall-Uemb3dPrepare your input fileemb3d expects a JSONL file as input. Each line of the file should be a JSON object with atextkey. Example input file:{"text":"I love my dog"}{"text":"I love my cat"}{"text":"I love my rabbit"}Your files can optionally have other fields like ids, categorical labels etc.. and they are saved as-is in the final output file.Compute embeddingsThe default model is OpenAI'stext-embedding-ada-002. You can change the model by passing the--model-idflag.emb3dcomputeinputs.jsonlYou will need to have OPENAI_API_KEY set in your environment. You can also pass it as a flag (--api_key) or set it in a config file.emb3d config set openai_token YOUR-OPENAI-API-KEY
emb3d compute inputs.jsonlemb3dcomputeinputs.jsonl--model-idembed-english-v2.0--output-filecohere-embeddings.jsonlFor COHERE models, you will need to have COHERE_API_KEY set in your environment. You can also pass it as a flag (--api_key) or set it in a config file with:emb3d config set cohere_token YOUR-COHERE-API-KEY.Visualize your embeddings 💥The last step is to visualize your computed embeddings. This will open a browser window with a visualization of your last computed embeddings.emb3dvisualizerun-2020-embeddings.jsonlProfit 💰UsageUsage: emb3d [OPTIONS] INPUT_FILE COMMAND [ARGS]...

 Generate embeddings for fun and profit.

╭─ Arguments ───────────────────────────────────────────────────────────────────────────────╮
│ *    input_file      PATH  Path to the input file. [default: None] [required]             │
╰───────────────────────────────────────────────────────────────────────────────────────────╯
╭─ Options ─────────────────────────────────────────────────────────────────────────────────╮
│ --model-id                                     TEXT     ID of the embedding model.        │
│                                                         Default is                        │
│                                                         `text-embedding-ada-002`.         │
│                                                         [default: None]                   │
│ --output-file              -out,-o             PATH     Path to the output file. If not   │
│                                                         provided, a default path will be  │
│                                                         suggested.                        │
│                                                         [default: None]                   │
│ --api-key                                      TEXT     API key for the backend. If not   │
│                                                         provided, it will be prompted or  │
│                                                         fetched from environment          │
│                                                         variables.                        │
│                                                         [default: None]                   │
│ --remote                            --local             Choose whether to do inference    │
│                                                         locally or with an API token.     │
│                                                         This choice is available for      │
│                                                         sentence transformer and hugging  │
│                                                         face models. If a model cannot be │
│                                                         run locally (ex: OpenAI models),  │
│                                                         this flag is ignored.             │
│                                                         [default: remote]                 │
│ --max-concurrent-requests                      INTEGER  (Remote Execution) Maximum number │
│                                                         of concurrent requests for the    │
│                                                         embedding task. Default is 1000.  │
│                                                         [default: 1000]                   │
│ --help                                                  Show this message and exit.       │
╰───────────────────────────────────────────────────────────────────────────────────────────╯
╭─ Commands ────────────────────────────────────────────────────────────────────────────────╮
│ config           Get or set a configuration value.                                        │
╰───────────────────────────────────────────────────────────────────────────────────────────╯Need help? 🙋Join ourDiscord serverand lets talk!