Package: pymilvus-cloud
Milvus Python SDKPython SDK forMilvus. To contribute code to this project, please read ourcontribution guidelinesfirst.For detailed SDK documentation, refer toAPI Documentation.New featuresGet startedPrerequisitesInstall pymilvus-cloudExamplesBasic operationsConnect to the Milvus serverCreate/Drop collectionsCreate a collectionDrop a collectionCreate/Drop partitions in a collectionCreate a partitionDrop a partitionCreate/Drop indexes in a collectionCreate an indexDrop an indexInsert/Delete vectors in collections/partitionsInsert vectors in a collectionInsert vectors in a partitionDelete vectors by IDFlush data in one or multiple collections to diskCompact all segments in a collectionSearch vectors in collections/partitionsSearch vectors in a collectionSearch vectors in a partitionDisconnect from the Milvus serverFAQGet startedPrerequisitespymilvus-cloud only supports Python 3.6 or higher.Install pymilvus-cloudYou can install pymilvus-cloud viapiporpip3for Python3:$pip3installpymilvus-cloudThe following collection shows Milvus versions and recommended pymilvus-cloud versions:Milvus versionRecommended pymilvus-cloud version0.10.60.4.00.10.50.2.15, 0.4.00.10.1 - 0.10.40.2.140.10.00.2.130.9.10.2.120.9.00.2.110.8.00.2.100.7.10.2.90.7.00.2.80.6.00.2.6, 0.2.70.5.30.2.50.5.20.2.30.5.10.2.30.5.00.2.30.4.00.2.20.3.10.1.250.3.00.1.13You can install a specific version of pymilvus-cloud by:$pipinstallpymilvus-cloud==0.4.0You can upgradepymilvus-cloudto the latest version by:$pipinstall--upgradepymilvus-cloudExamplesRefer toexamplesfor more example programs.Basic operationsConnect to the Milvus serverImport pymilvus-cloud.# Import pymilvus-cloud>>>frommilvus-cloudimportMilvus,IndexType,MetricType,StatusCreate a client to Milvus server by using one of the following methods:# Connect to Milvus server>>>client=Milvus(host='endpoint',port='443',token="access_token")Note: In the above code, default values are used forhostandportparameters. Feel free to change them to the IP address and port you set for Milvus server.>>>client=Milvus(uri='tcp://endpoint:443')Create/Drop collectionsCreate a collectionPrepare collection parameters.# Prepare collection parameters>>>param={'collection_name':'test01','dimension':128,'index_file_size':1024,'metric_type':MetricType.L2}Create collectiontest01with dimension size as 128, size of the data file for Milvus to automatically create indexes as 1024, and metric type as Euclidean distance (L2).# Create a collection>>>status=client.create_collection(param)>>>statusStatus(code=0,message='Create collection successfully!')Drop a collection# Drop collection>>>status=client.drop_collection(collection_name='test01')>>>statusStatus(code=0,message='Delete collection successfully!')Create/Drop partitions in a collectionCreate a partitionYou can split collections into partitions by partition tags for improved search performance. Each partition is also a collection.# Create partition>>>status=client.create_partition(collection_name='test01',partition_tag='tag01')>>>statusStatus(code=0,message='OK')Uselist_partitions()to verify whether the partition is created.# Show partitions>>>status,partitions=client.list_partitions(collection_name='test01')>>>partitions[(collection_name='test01',tag='_default'),(collection_name='test01',tag='tag01')]Drop a partition>>>status=client.drop_partition(collection_name='test01',partition_tag='tag01')Status(code=0,message='OK')Create/Drop indexes in a collectionCreate an indexNote: In production, it is recommended to create indexes before inserting vectors into the collection. Index is automatically built when vectors are being imported. However, you need to create the same index again after the vector insertion process is completed because some data files may not meet theindex_file_sizeand index will not be automatically built for these data files.Prepare index parameters. The following command usesIVF_FLATindex type as an example.# Prepare index param>>>ivf_param={'nlist':4096}Create an index for the collection.# Create index>>>status=client.create_index('test01',IndexType.IVF_FLAT,ivf_param)Status(code=0,message='Build index successfully!')Drop an index>>>status=client.drop_index('test01')Status(code=0,message='OK')Insert/Delete vectors in collections/partitionsInsert vectors in a collectionGenerate 20 vectors of 128 dimension.>>>importrandom>>>dim=128# Generate 20 vectors of 128 dimension>>>vectors=[[random.random()for_inrange(dim)]for_inrange(20)]Insert the list of vectors. If you do not specify vector ids, Milvus automatically generates IDs for the vectors.# Insert vectors>>>status,inserted_vector_ids=client.insert(collection_name='test01',records=vectors)>>>inserted_vector_ids[1592028661511657000,1592028661511657001,1592028661511657002,1592028661511657003,1592028661511657004,1592028661511657005,1592028661511657006,1592028661511657007,1592028661511657008,1592028661511657009,1592028661511657010,1592028661511657011,1592028661511657012,1592028661511657013,1592028661511657014,1592028661511657015,1592028661511657016,1592028661511657017,1592028661511657018,1592028661511657019]Alternatively, you can also provide user-defined vector ids:>>>vector_ids=[idforidinrange(20)]>>>status,inserted_vector_ids=client.insert(collection_name='test01',records=vectors,ids=vector_ids)Insert vectors in a partition>>>status,inserted_vector_ids=client.insert('test01',vectors,partition_tag="tag01")To verify the vectors you have inserted, useget_vector_by_id(). Assume you have vector with the following ID.>>>status,vector=client.get_entity_by_id(collection_name='test01',ids=inserted_vector_ids[:10])Delete vectors by IDYou can delete these vectors by:>>>status=client.delete_entity_by_id('test01',inserted_vector_ids[:10])>>>statusStatus(code=0,message='OK')Flush data in one or multiple collections to diskWhen performing operations related to data changes, you can flush the data from memory to disk to avoid possible data loss. Milvus also supports automatic flushing, which runs at a fixed interval to flush the data in all collections to disk. You can use theMilvus server configuration fileto set the interval.>>>status=client.flush(['test01'])>>>statusStatus(code=0,message='OK')Compact all segments in a collectionA segment is a data file that Milvus automatically creates by merging inserted vector data. A collection can contain multiple segments. If some vectors are deleted from a segment, the space taken by the deleted vectors cannot be released automatically. You can compact segments in a collection to release space.>>>status=client.compact(collection_name='test01')>>>statusStatus(code=0,message='OK')Search vectors in collections/partitionsSearch vectors in a collectionPrepare search parameters.>>>search_param={'nprobe':16}Search vectors.# create 5 vectors of 32-dimension>>>q_records=[[random.random()for_inrange(dim)]for_inrange(5)]# search vectors>>>status,results=client.search(collection_name='test01',query_records=q_records,top_k=2,params=search_param)>>>results[[(id:1592028661511657012,distance:19.450458526611328),(id:1592028661511657017,distance:20.13418197631836)],[(id:1592028661511657012,distance:19.12230682373047),(id:1592028661511657018,distance:20.221458435058594)],[(id:1592028661511657014,distance:20.423980712890625),(id:1592028661511657016,distance:20.984281539916992)],[(id:1592028661511657018,distance:18.37057876586914),(id:1592028661511657019,distance:19.366962432861328)],[(id:1592028661511657013,distance:19.522361755371094),(id:1592028661511657010,distance:20.304216384887695)]]Search vectors in a partition# create 5 vectors of 32-dimension>>>q_records=[[random.random()for_inrange(dim)]for_inrange(5)]>>>client.search(collection_name='test01',query_records=q_records,top_k=1,partition_tags=['tag01'],params=search_param)Note: If you do not specifypartition_tags, Milvus searches the whole collection.close client>>>client.close()FAQI'm getting random "socket operation on non-socket" errors from gRPC when connecting to Milvus from an application served on GunicornMake sure to set the environment variableGRPC_ENABLE_FORK_SUPPORT=1. For reference, seehttps://zhuanlan.zhihu.com/p/136619485