Package: nlp-service
NLP MicroserviceThe goal of this project is to provide agRPCserver for resource-heavy NLP tasks—for instance, computing vectors/embeddings for words or sentences.
By usingprotobufinternally, our NLP server provides native and strongly typed interfaces for many programming languages.
There are multiple advantages that arise from outsourcing such computations to such a server:If multiple apps rely on NLP, the underlying models (which are usually quite large) only need to be loaded once into the main memory.All programming languages supported by gRPC get easy access to state-of-the-art NLP architectures (e.g., transformers).The logic is consolidated at a central place, drastically decreasing the maintenance effort required.In addition to the server, we also provide a client containing convenience functions.
This makes it easier for python applications to interact with the gRPC server.
We will discuss the client at the end of this README.Installation and SetupWe are usingnixandpoetryto manage the dependencies and provide a ready-to-use Docker image.Docker (recommended)The container caches the downloaded models, so you should not pass--rmtodocker run.dockerrunghcr.io/recap-utr/nlp-service:latest"0.0.0.0:50100"Nix (advanced)nixrungithub:recap-utr/nlp-service--"127.0.0.1:50100"# or after cloning this repositorynixdevelop-cpoetryrunpython-mnlp_service"127.0.0.1:50100"Poetry (advanced)# The server dependencies are optional, thus they have to be installed explicitly.poetryinstall--extrasall# To run the server, you need to specify the address it should listen on.# In this example, it should liston on port 5678 on localhost.poetryrunpython-mnlp_service"127.0.0.1:50100"General UsageOnce the server is running, you are free to call any of the functions defined in the underlyingprotobuf file.
The corresponding documentation is located at theBuf Schema Registry.Please note:The examples here use the Python programming language, but are also directly applicable to any other language supported by gRPC.importgrpcfromarg_services.nlp.v1importnlp_pb2,nlp_pb2_grpc# First of all, we are creating a channel (i.e., establish a connection to our server)channel=grpc.insecure_channel("127.0.0.1:5678")# The channel can now be used to create the actual client (allowing us to call all available functions)client=nlp_pb2_grpc.NlpServiceStub(channel)# Now the time has come to prepare our actual function call.# We will start by creating a very simple NlpConfig with the default spacy model.# FOr details about the parameters, please have a look at the next section.config=nlp_pb2.NlpConfig(language="en",spacy_model="en_core_web_lg",)# Next, we will build a request to query vectors from our server.request=nlp_pb2.VectorsRequest(# The first parameter is a list of strings that shall be embedded by our server.texts=["What a great tutorial!","I will definitely recommend this to my friends."],# Now we need to specify which embeddings have to be computed. In this example, we create one vector for each textembedding_levels=[nlp_pb2.EmbeddingLevel.EMBEDDING_LEVEL_DOCUMENT],# The only thing missing now is the spacy configuration we created in the previous step.config=config)# Having created the request, we can now send it to the server and retrieve the corresponding response.response=client.Vectors(request)# Due to technical constraints, we cannot directly transfer numpy arrays, thus we convert our response.vectors=[np.array(entry.document.vector)forentryinresponse.vectors]Advanced UsageA central piece for all available function is theNlpConfigmessage, allowing you to create even complex embedding models easily.
In addition toits documentation, we will in the following present some examples to demonstrate the possibilities you have.fromarg_services.nlp.v1importnlp_pb2# In the example above, we already introduced a quite basic config:config=nlp_pb2.NlpConfig(# You have to provide a language for every config: https://spacy.io/usage/models#languageslanguage="en",# Also, you need to specify the model that spacy should load: https://spacy.io/models/enspacy_model="en_core_web_lg",)# A central feature of our library is the possibility to combine multiple embedding models, potentially capturing more contextual information.config=nlp_pb2.NlpConfig(language="en",# This parameter expects a list of models. If you pass more than one, the respective vectors are **concatenated** to each other# (e.g., two 300-dimensional embeddings will result in a 600-dimensional one).# This approach is based on https://arxiv.org/abs/1803.01400embedding_models=[nlp_pb2.EmbeddingModel(# First select the type of model you would like to use (e.g., SBERT/Sentence Transformers).model_type=nlp_pb2.EmbeddingType.EMBEDDING_TYPE_SENTENCE_TRANSFORMERS,# Then select the actual model.# Any of those specified on the website (https://www.sbert.net/docs/pretrained_models.html) are allowed.model_name="all-mpnet-base-v2"),nlp_pb2.EmbeddingModel(# It is also possible to use a standard spacy modelmodel_type=nlp_pb2.EmbeddingType.EMBEDDING_TYPE_SPACY,model_name="en_core_web_lg",# Since we have selected a word embedding (i.e., it cannot directly encode sentences), the token vectors need to be aggregated somehow.# The default strategy is to use the arithmetic mean, but you are free to use other strategies (e.g., the geometric mean).pooling_type=nlp_pb2.Pooling.POOLING_GMEAN),nlp_pb2.EmbeddingModel(model_type=nlp_pb2.EmbeddingType.EMBEDDING_TYPE_SPACY,model_name="en_core_web_lg",# Alternatively, it is also possible to use the generalized mean / power mean.# In this example, the selected pmean corresponds to the geometic mean (thus this embedding is identical to the previous one).# This approach is based on https://arxiv.org/abs/1803.01400pmean=0)]# This setting is now optional and only needed if you need spacy features (e.g., POS tagging) besides embeddings.# spacy_model="en_core_web_lg",)# If computing the similarity between strings, you get one additional parameter.config=nlp_pb2.NlpConfig(language="en",# To keep the example simple, we will now only use a single spacy model instead of the more powerful embedding models.# However, it is of course possible to use them here as well.spacy_model="en_core_web_lg",# If not specified, we will always use the cosine similarity when comparing two strings.# As indicated in a recent paper (https://arxiv.org/abs/1904.13264), you may achieve better results with alternative approaches like DynaMax Jaccard.# Please note that this particular method ignores your selected pooling method due to the fact that even plain word embeddings are not pooled at all.similarity_method=nlp_pb2.SimilarityMethod.SIMILARITY_METHOD_DYNAMAX_JACCARD)# It is also possible to determine a similarity score without the use of embeddings.config=nlp_pb2.NlpConfig(language="en",spacy_model="en_core_web_sm",# Traditional metric (Jaccard similarity and Levenshtein edit distance) are also availablesimilarity_method=nlp_pb2.SimilarityMethod.SIMILARITY_METHOD_EDIT)