Package: weaviate-demo-datasets
UNOFFICIAL Weaviate demo data uploaderThis is an educational project that aims to make it easy to upload demo data to your instance ofWeaviate. The target audience is developers learning how to use Weaviate.Usagepipinstall-Uweaviate-demo-datasetsEach dataset includes a default vectorizer configuration for convenience.
The target Weaviate instance must include the specified vectorizer module.Once you instantiate a dataset, you can upload it to Weaviate with the following:importweaviate_datasetsaswddataset=wd.JeopardyQuestions1k()# Instantiate datasetdataset.upload_dataset(client)# Pass the Weaviate client instanceWhereclientis the instantiatedweaviate.WeaviateClientobject, such as:importweaviateimportosclient=weaviate.connect_to_local(headers={"X-OpenAI-Api-Key":os.getenv("OPENAI_APIKEY")})To use aweaviate.Clientobject, as used in the Weaviate Python clientv3.x, import the dataset class fromweaviate_datasets.v3.importweaviate_datasets.v3_datasetsaswd_v3dataset=wd_v3.JeopardyQuestions1k()# Instantiate datasetdataset.upload_dataset(client)# Pass the Weaviate client instanceBuilt-in methods.upload_dataset(client)- add defined classes to schema, adds objects.get_sample()- yields sample data object(s)Available classesWiki100 (Top 100 Wikipedia articles)WikiChunkcollectionVarious chunking options available:Default:wiki_sections(sections of the Wikipedia article)wiki_section_chunked(sections of the Wikipedia article, chunked into 200 character chunks)wiki_heading_only(only the headings of the Wikipedia article sections)fixed(fixed length chunks of 200 characters)Use it as follows:d=wd.Wiki100()d.collection_name="WikiChunk"d.set_chunking("wiki_section_chunked")upload_responses=d.upload_dataset(client,overwrite=True)WineReviews (50 wine reviews)WineReviewcollectionWineReviewsNV (50 wine reviews)WineReviewNVcollection, with named vectors ("title", "review_body", and "title_country")"title_country" -> Vector from concatenation of "title" + "country"WineReviewsMT (50 wine reviews)WineReviewMTcollection, tenantstenantAandtenantBJeopardyQuestions1k (1,000 Jeopardy questions & answers, vectorized with OpenAItext-embedding-ada-002)JeopardyQuestionandJeopardyCategorycollectionsJeopardyQuestions10k (10,000 Jeopardy questions & answers, vectorized with OpenAItext-embedding-ada-002)JeopardyQuestionandJeopardyCategorycollectionsAvailable classes - V3 collectionThese are available with aV3suffix, and are compatible with the Weaviate Python clientv3.x.Not including vectorsWineReviews (50 wine reviews)WineReviewsMT (50 wine reviews, multi-tenancy enabled)Including vectorsJeopardyQuestions1k (1,000 Jeopardy questions & answers, vectorized with OpenAItext-embedding-ada-002)JeopardyQuestions10k (10,000 Jeopardy questions & answers, vectorized with OpenAItext-embedding-ada-002)JeopardyQuestions1kMT (1,000 Jeopardy questions & answers, multi-tenancy enabled, vectorized with OpenAItext-embedding-ada-002)NewsArticles (News articles, including their corresponding publications, authors & categories, vectorized with OpenAItext-embedding-ada-002)Data sourceshttps://www.kaggle.com/datasets/zynicide/wine-reviewshttps://www.kaggle.com/datasets/tunguz/200000-jeopardy-questionshttps://github.com/weaviate/DEMO-NewsPublicationsSource codehttps://github.com/databyjp/wv_demo_uploader