Package: embedding-as-service
embedding-as-serviceOne-Stop Solution to encode sentence to fixed length vectors from various embedding techniques• Inspired frombert-as-serviceWhat is it•Installation•Getting Started•Supported Embeddings•API•What is itEncoding/Embeddingis a upstream task of encoding any inputs in the form of text, image, audio, video, transactional data to fixed length vector. Embeddings are quite popular in the field of NLP, there has been various Embeddings models being proposed in recent years by researchers, some of the famous one are bert, xlnet, word2vec etc. The goal of this repo is to build one stop solution for all embeddings techniques available, here we are starting with popular text embeddings for now and later on we aim  to add as much technique for image, audio, video inputs also.embedding-as-servicehelp you to encode any given text to fixed length vector from supported embeddings and models.💾 Installation▴ Back to topHere we have given the capability to useembedding-as-servicelike a module or you can run it as a server and handle queries by installing client packageembedding-as-service-clientUsingembedding-as-serviceas moduleInstall the embedding-as-servive viapip.$pipinstallembedding-as-serviceNote that the code MUST be running onPython >= 3.6. Again module does not support Python 2!Usingembedding-as-serviceas a serverHere you also need to install a client moduleembedding-as-service-client$pipinstallembedding-as-service# server$pipinstallembedding-as-service-client# clientClient module need not to be on Python 3.6, it supports both Python2 and Python3⚡ ️Getting Started▴ Back to top1.Intialise encoder using supported embeddingand models fromhereIf usingembedding-as-serviceas a module>>>fromembedding_as_service.text.encodeimportEncoder>>>en=Encoder(embedding='bert',model='bert_base_cased',max_seq_length=256)If usingembedding-as-serviceas a server# start the server by proving embedding, model, port, max_seq_length[default=256], num_workers[default=4]$embedding-as-service-start--embeddingbert--modelbert_base_cased--port8080--max_seq_length256>>>fromembedding_as_service_clientimportEmbeddingClient>>>en=EmbeddingClient(host=<host_server_ip>,port=<host_port>)2. Get sentencestokens embedding>>>vecs=en.encode(texts=['hello aman','how are you?'])>>>vecsarray([[[1.7049843,0.,1.3486509,...,-1.3647075,0.6958289,1.8013777],...[0.4913215,0.60877025,0.73050433,...,-0.64490885,0.8525057,0.3080206]]],dtype=float32)>>>vecs.shape(2,128,768)# batch x max_sequence_length x embedding_size3. Usingpooling strategy, clickherefor more.Supported Pooling MethodsStrategyDescriptionNoneno pooling at all, useful when you want to use word embedding instead of sentence embedding. This will results in a[max_seq_len, embedding_size]encode matrix for a sequence.reduce_meantake the average of all token embeddingsreduce_mintake the minumun of all token embeddingsreduce_maxtake the maximum of all token embeddingsreduce_mean_maxdoreduce_meanandreduce_maxseparately and then concat them togetherfirst_tokenget the token embedding of first token of a sentencelast_tokenget the token embedding of last token of a sentence>>>vecs=en.encode(texts=['hello aman','how are you?'],pooling='reduce_mean')>>>vecsarray([[-0.33547154,0.34566957,1.1954105,...,0.33702594,1.0317835,-0.785943],[-0.3439088,0.36881036,1.0612687,...,0.28851607,1.1107115,-0.6253736]],dtype=float32)>>>vecs.shape(2,768)# batch x embedding_size4. Show embedding Tokens>>>en.tokenize(texts=['hello aman','how are you?'])[['_hello','_aman'],['_how','_are','_you','?']]5. Using your own tokenizer>>>texts=['hello aman!','how are you']# a naive whitespace tokenizer>>>tokens=[s.split()forsintexts]>>>vecs=en.encode(tokens,is_tokenized=True)📋 API▴ Back to topclassembedding_as_service.text.encoder.EncoderArgumentTypeDefaultDescriptionembeddingstrRequiredembedding method to be used, checkEmbeddingcolumnheremodelstrRequiredModel to be used for mentioned embedding, checkModelcolumnheremax_seq_lengthint128Maximum Sequence Length, default is 128defembedding_as_service.text.encoder.Encoder.encodeArgumentTypeDefaultDescriptionTextsList[str] or List[List[str]]RequiredList of sentences or list of list of sentence tokens in case ofis_tokenized=Truepoolingstr(Optional)Pooling methods to apply,hereis available methodsis_tokenizedboolFalseset as True in case of tokens are passed for encodingbatch_sizeint128maximum number of sequences handled by encoder, larger batch will be partitioned into small batches.defembedding_as_service.text.encoder.Encoder.tokenizeArgumentTypeDefaultDescriptionTextsList[str]RequiredList of sentences✅ Supported Embeddings and Models▴ Back to topHere are the list of supported embeddings and their respective models.EmbeddingModelEmbedding dimensionsPaper:one:albertalbert_base768Read Paper :bookmark:albert_large1024albert_xlarge2048albert_xxlarge4096:two:xlnetxlnet_large_cased1024Read Paper :bookmark:xlnet_base_cased768:three:bertbert_base_uncased768Read Paper :bookmark:bert_base_cased768bert_multi_cased768bert_large_uncased1024bert_large_cased1024:four:elmoelmo_bi_lm512Read Paper :bookmark::five:ulmfitulmfit_forward300Read Paper :bookmark:ulmfit_backward300:six:useuse_dan512Read Paper :bookmark:use_transformer_large512use_transformer_lite512:seven:word2vecgoogle_news_300300Read Paper :bookmark::eight:fasttextwiki_news_300300Read Paper :bookmark:wiki_news_300_sub300common_crawl_300300common_crawl_300_sub300:nine:glovetwitter_200200Read Paper :bookmark:twitter_100100twitter_5050twitter_2525wiki_300300wiki_200200wiki_100100wiki_5050crawl_42B_300300crawl_840B_300300CreditsThis software uses the following open source packages:XLnettensorflow-hubContributors ✨Thanks goes to these wonderful people (emoji key):MrPranav101💻📖🚇Aman Srivastava💻📖🚇Chirag Jain💻📖🚇Ashutosh Singh💻📖🚇Dhaval Taunk💻📖🚇Alec Koumjian🐛Pradeesh🐛This project follows theall-contributorsspecification. Contributions of any kind welcome!Please read thecontribution guidelinesfirst.Citing▴ Back to topIf you use embedding-as-service in a scientific publication, we would appreciate references to the following BibTex entry:@misc{aman2019embeddingservice,
  title={embedding-as-service},
  author={Srivastava, Aman},
  howpublished={\url{https://github.com/amansrivastava17/embedding-as-service}},
  year={2019}}