Package: pinecone-cli
pinecone-cliPinecode-cli is a command-line interface for control and data plane interfacing withPinecone.In addition to ALL of the Pinecone "actions/verbs", Pinecone-cli has several additional features that make Pinecone even more powerful including:Upload vectors from CSV filesUpload embeddings of text from a given website URL.  Embeddings generated by OpenAI embeddings API.New "head" command to peak into a given index, similar to "head" in linux/unix.InstallFeel free to use the tool directly from source here, or justpip install pinecone-cliPypi here: (https://pypi.org/project/pinecone-cli/)UsageThe CLI depends on a couple of simple environment variables:Your Pinecone API KeyThe region/environment of your Pinecone indexesThere is a simple order in which the CLI picks them up.envfile in the current working dir.The environment variable your shell is in.Command line arguments overriding the above.Let's look at a simple .env file:PINECONE_API_KEY=123456-9876-ABCDEFPINECONE_ENVIRONMENT=us-west1-gcpOf course setting in the shell is about the same:%exportPINECONE_API_KEY=1234-4567-abcOtherwise you wind up having to pass the key as so:%pinecliquery--apikey=1234....The pattern for using the tool is to invoke 'pinecli' and then use a command.  The list of commands appears with --help%pinecli.py--helpUsage: pinecli.py [OPTIONS] COMMAND [ARGS]...A command line interface for working with Pinecone.Options:--help  Show this message and exit.Commands:askquestion               Queries Pinecone with a given vector.configure-index-pod-type  Configures the given index to have a pod type.configure-index-replicas  Configures the number of replicas for a givenindex.create-collection         Creates a Pinecone collection from the argument'source_index'create-index              Creates a Pinecone Index.delete-all                Delete all vectors (note separate command [delete-index] can completely delete an index)delete-collection         Deletes a collection.delete-index              Deletes an index.  You will be prompted toconfirm.describe-collection       Describes a collection.describe-index            Describes an index.describe-index-stats      Prints out index stats to stdout.fetch                     Fetches vectors from Pinecone specified by thevectors' ids.head                      Shows a preview of vectors in the<PINECONE_INDEX_NAME>list-collections          Lists collections for the given apikey.list-indexes              Lists the indexes for your api key.minimize-cluster          Minimizes everything for a cluster to lowestsettings.query                     Queries Pinecone with a given vector.update                    Updates the index based on the given id passed in.upsert                    Extracts text from url arg, vectorizes w/ openaiembedding api, and upserts to Pinecone.upsert-file               Upserts a file (csv) into the specified index.upsert-random             Upserts a vector(s) with random dimensions intothe specified vector.upsert-webpage            Extracts text from url arg, vectorizes w/ openaiembedding api, and upserts to Pinecone.version                   Prints version number.Commands With ExamplesBefore you can use Pinecone an index is required.  We can now do this on the commandline rather than in the UI: (Note not all of the cmdline options are required, they're shown here to demonstrate functionality and control)%pineclicreate-indexmyindex--dims=1536--metric=cosine--pods=2--replicas=2--shards=1--pod-"type=p2.x1"Note that for any command, if you want an exhasuive description of cmdline options, simply do something similar to the below, where "create-index" is replaced by one of the commands:%pineclicreate-index--helpUsage: pinecli create-index [OPTIONS] PINECONE_INDEX_NAMECreates the Pinecone index named <PINECONE_INDEX_NAME>Options:--apikey TEXT             Pinecone API Key--region TEXT             Pinecone Index Region--dims INTEGER            Number of dimensions for this index  [required]--metric TEXT             Distance metric to use.  [required]--pods INTEGER            Number of pods  [default: 1]--replicas INTEGER        Number of replicas  [default: 1]--shards INTEGER          Number of shards  [default: 1]--pod-type TEXT           Type of pods to create.  [required]--source_collection TEXT  Source collection to create index from--help                    Show this message and exit.Create Index From CollectionYou can also create an index from a collection (effectively an index backup) as so:%pinecli.pycreate-collection--collection_name='testercollection'--source_index='mysourcecollection'Let's try some commands showing two missing features I'd love to have had over the last year: a "head" command and a quick "stats" command:Index Stats Including Number of Vectors%pineclidescribe-index-statsmyindexDimensions: 1536Vectors: 7745Index_Fullness: 0.0Namespace data:: 7745Head command to preview vectors%pinecliheadkids-facenet{'matches': [{'id': 'bubba_50.jpg.vec','metadata': {},'score': 12.182938,'values': [-0.016061664,-0.4495437,-0.034082577,.....Now, let's query some nonsensical data from the index named 'upsertfile'Inserting a vector directlyNote the double quites around the vector%pinecliquerymyindex"[1.2, 1.0, 3.0]"--print-table--include-meta=True🌲 upsertfile ns=() Index Results┏━━━━━━┳━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┓┃   ID ┃ NS ┃ Values                   ┃                Meta ┃        Score ┃┡━━━━━━╇━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━┩│ vec1 │    │ 0.1,0.2,0.3              │  {'genre': 'drama'} │    0.9640127 ││ vec2 │    │ 0.2,0.3,0.4              │ {'genre': 'action'} │    0.9552943 ││  abc │    │ 0.23223,-1.333,0.2222222 │      {'foo': 'bar'} │ -0.083585836 ││  ghi │    │ 0.23223,-1.333,0.2222222 │      {'bar': 'baz'} │ -0.083585836 │└──────┴────┴──────────────────────────┴─────────────────────┴──────────────┘Markdown of course does a great job of mangling great terminal output so here's a screenshot from using---print-table:You can of course not output the pretty table by removing--print-table:%pinecliquerymyindex"[1.2, 1.0, 3.0]"--include-meta=True{'matches': [{'id': 'vec1','metadata': {'genre': 'drama'},'score': 0.9640127,'values': [0.1, 0.2, 0.3]},...Upsert Vectors in Command Line ManuallyFollowing the Pinecone vector format of the tuple formatted as:('vectorid',[vecdim1,vecdim2,vecdim3],{'metakey':'metaval'})You can pass this in as a comma separated list of vectors on the command line:pinecli upsert myindex "[('vec1', [0.1, 0.2, 0.3], {'genre': 'drama'}), ('vec2', [0.2, 0.3, 0.4], {'foo': 'bar'}),]"Upsert CSV fileUpserting a csv file is trivial.  Simply create your csv file with any headings you have, but there must be at least a labeled id column and a labeled vector column for the vectors.  Here's an example of a CSV file that is clearly a DataFrame dump due to the index column on the left which works great w/ pinecone-cli:index,my_id_column,my_vectors_column,Metadata1,abc,"[0.23223, -1.333, 0.2222222]",{'foo':'bar'}2,ghi,"[0.23223, -1.333, 0.2222222]",{'bar':'baz'}The name of those columns in the header row can be arbitrary or you can name then "id", "vectors" and "metadata" which is our default assumption.  If you have custom column names and don't want to change them, just pass in the--colmapargument which takes in a python dictionary mapping "id" and "vectors" to the naming you have in your csv.  For example:"{"id":"my_id_column", "vectors":"my_vectors_column"}Note that as in other CSV file for Dataframes, we need an index column as in the example above.
Here's an example using the CSV headers and format above with the correct colmap argument:%pinecliupsert-fileembeddings.csvmyindex"{'id':'my_id_column', 'vectors':'my_vectors_column'}"More on CSV FormattingFor now you will need to manually provide an index column (we are using dataframes under the hood.)Upserting Vector Embeddings of Webpage Textpinecone-cli was built to make using Pinecone extremely easy and fast.  We have integratedOpenAI(others coming) - using itsembedding APIsto fetch embeddings.  We then upload them into your index for you, making uploading embeddings of an entire website's text - trivial.%pinecliupsert-webpagehttps://menlovc.comlpfactset--openaiapikey=12345-9876-abcdef[nltk_data] Downloading package punkt to /Users/tim/nltk_data...[nltk_data]   Package punkt is already up-to-date!100%|████████████████████████████████████████████████████████████████████████| 5/5 [00:00<00:00, 61680.94it/s]['About Us  Our Promise  Focus Areas   Consumer  Cloud Infrastructure  Cybersecurity  Fintech  Healthcare  SaaS  Supply Chain and Automation    Team  Portfolio  Perspective            When we invest, we’re invested. Our promise to founders        Building a business is a team sport. As investors, we don’t just sit on the sidelines but do whatever it takes to help our teams win. About Us    The founders we back don’t limit themselves to what is, but relentlessly pursue what could be. We invest in transformative technology companies that are changing the way we live and work. Portfolio    Menlo Labs starts companies. We work shoulder-to-shoulder....100%|█████████████████████████| 1/1 [00:00<00:00,  1.11it/s]Upsert Random VectorsOne of the more useful things we can do with pinecone-cli is insert random vectors, primarily for testing.  Often we will create our index and the length of the vector will be 1,536 dimensions, for example.  Instead of writing a bunch of code to go suddenly create those vectors somehow, we can use pinecone-cli to start generating vectors and upserting them:%pinecliupsert-randomupsertfile--num_vector_dims=1536--num_vectors=10--debugupserted_count: 101it [00:00,  4.36it/s]The example above inserts 10 vectors that each have 1,536 random vectors in them. Note that theidfor each vector is simplyf'id-{i}'where is is the ith row (vector) inserted.Query VectorsQuerying can be done in two ways on the cmdline - pass in an actual vector string literal, or ask Pinecone to query randomly (maybe you want to just look at them or look at a TSNE).  In the example below, the last argument (required) is either the string 'random' or an actual vector such as '[0.0, 1.0, 3.14569]'.  Let's try random:%pinecliquerymyindexrandomYou can also plot a TSNE plot to view clustering of your vectors by using the-show-tsne=Trueflag.  Note that this will pop up the plt plot by default.%pinecliquerylpfactsetrandom--show-tsne=true--topk=2500--num-clusters=4Fetching VectorsFetching is simple - just pass in the vector id(s) of the vectors you're looking for as a comma separated list:%pineclifetchmyindex--vector_ids="05b4509ee655aacb10bfbb6ba212c65c,c626975ec096b9108f158a56a59b2fd6"{'namespace': '','vectors': {'05b4509ee655aacb10bfbb6ba212c65c': {'id': '05b4509ee655aacb10bfbb6ba212c65c','metadata': {'content': 'Chime ''Scholar ''spotlight: 'Updating VectorsUpdating vectors is simple - pass the id of the vector and the updating vector as below:%pinecliupdate"id-9"myindex"[0.0, 1.0, 3.0]"List Operationspinecone-cli has all of the necessary 'list' operations as shown below:List IndexesThis gives you a list of all indexes under your api key:%pineclilist-indexesclicli2cli3drivertestList Indexes FullyYou can also view the index data similarly to what you'd get on the Pinecone page using the (--print-table) flag with all data such as pods, metric type, shards, etc:%pinecli.pylist-indexes--print-tableList CollectionsThis obviously lists the collections you've created:%pineclilist-collectionsclicli2cli3drivertestOther Meta OperationsWe showed the "describe-index-stats" command at the top of this page.  There is also "describe-index" which provides the following:%pineclidescribe-indexlpfactsetName: lpfactsetDimensions: 1536Metric: cosinePods: 1PodType: p2.x1Shards: 1Replicas: 1Ready: TrueState: ReadyMetaconfig: NoneSourcecollection:Describe Collections%pineclidescribe-collectiontestcollName: testcollDimensions: 1536Vectors: 124Status: ReadySize: 3917544Deleting Vectors From IndexThis will basically do a rmfrom the index and clear it out, but will notDELETE* the index. In other words, vector count will be 0.%pineclidelete-allmyindexnameDeleting IndexesDeleting an index is straightforward.  To prevent catastrophic accidents, you'll be prompted to type in the name of the index backwards:%pineclidelete-indexmyindex2Type name of index backwards to confirm: : 2xedniym