Package: embedit
embeditembeditis a command line tool for performing bulk operations on text files (e.g. a package) with OpenAI's API. It currently provides these commands:search, which performs semantic search on text files using embeddings, andtransformwhich performs arbitrary transformations using a custom prompt.commit-msggenerates a commit message based on the diff of the staged files and the commit history.autocommitgenerates a commit message and commits the changes in one step.InstallationInstallembeditusingpip:pipinstallembeditThis will installembeditand its dependencies, includingopenai. You will also need to set theOPENAI_API_KEYenvironment variable to your OpenAI API key if you haven't already done so.UsageSearchembedit searchperforms semantic searches on text files. You specify a search query and a list of text files to search,embeditwill fetch text from the files, split them into segments, embed them using OpenAI's API, and print them out in order of cosine distance to the query.embeditsearch"search query"file1.txtfile2.txt...Options--order: the order in which the results should be displayed (ascending or descending by similarity score). Default:ascending.--top-n: the number of top results to display. Default:3.--threshold: a similarity threshold below which results should be filtered out. Default:0.0.--frament_lines: the target fragment length in number of lines. Default:10.--min_fragment_lines: the minimum fragment length in number of lines. Default:0.TransformThetransformcommand allows you to transform one or more text files by passing their markdown representation with a given prompt to the OpenAI API.embedittransform**/*.py--prompt"Add a docstring at the top of each file"--output-diroutCan't I just feed my files to the API directly?You could. But transforming each file independently could lead to inconsistent behaviour.embedit transformcombines your files into a single prompt so that they can be transformed in a coherent way and then splits the result back into individual files.Options--files: One or more text files to transform.--transformation_fn: The function to apply on the files.--output_dir: The directory to save the transformed files.--yes: Don't prompt before creating or overwriting files.--model: The OpenAI API model to use.--verbose: Whether to print verbose output.--max_chunk_len: The maximum length (in characters) of chunks to pass to the OpenAI API.Generate commit messageThecommit-msgcommand will generate a commit message based on the diff of the staged files and the commit history.To use it, you can run it directly:embeditcommit-msgTo generate and commit the changes in one step, you can use theautocommitcommand:embeditautocommitI haven't tried to addcommit-msgas a git hook, but I imagine it would work.Options--path: The path to diff against.--max-log-tokens: The maximum number of tokens to include in the commit message.--max-diff-tokens: The maximum number of tokens to include in the diff.--max-output-tokens: The maximum number of tokens to include in the OpenAI API output.--model: The OpenAI API model to use.--num-examples: The number of examples to use.--use-builtin-examples: Whether to use the built-in examples.--hint: A hint to pass to the OpenAI API.--verbose: Print verbose output.--git-params: Keyword arguments to pass to the git commit command.For example, the below command will generate a commit message usinggpt-3.5-turbo, passing a hint that the document parameters have been updated, and will use not any of your previous commits as examples. The latter option is useful if your past commit messages have sufferedneglect.embeditautocommit--model"gpt-3.5-turbo"--hint"doc params"--num-examples0TipsWildcardsYou can also use wildcards to specify a pattern of files to search in. Here's an example of how you can use the**wildcard to search for Python files in all directories in the current directory and its subdirectories:embeditsearch"query"**/*.pyBear in mind that the behavior of the*and**wildcards may vary depending on your operating system and the terminal shell you're using.Autocommit workflowI like to use the following alias,qc(quick commit) to automatically generate and commit changes:aliasqc="embedit autocommit --num-examples 0 --model \"gpt-3.5-turbo\""Example usage:qc-h"hint hint"Then, when it gets something wrong, I edit the commit message and rungit commit --amendto fix it.ContributingIf you find a bug or want to contribute to the development ofembedit, you can create a new issue or submit a pull request.Licenseembeditis released under the MIT license. Do whatever you want with it.