Package: nboost
🧪 We're looking for beta testers for ourvirtual assistantwidget.Contact usif you're interested in using it on your website.Highlights•Overview•Benchmarks•Install•Getting Started•Kubernetes•Documentation•Tutorials•Contributing•Release Notes•BlogWhat is it⚡NBoostis a scalable, search-engine-boosting platform for developing and deploying state-of-the-art models to improve the relevance of search results.Nboost leverages finetuned models to produce domain-specific neural search engines. The platform can also improve other downstream tasks requiring ranked input, such as question answering.Contact us to request domain-specific models or leave feedbackOverviewThe workflow of NBoost is relatively simple. Take the graphic above, and imagine that the server in this case is Elasticsearch.In aconventional search request, the user sends a query toElasticsearchand gets back the results.In anNBoost search request, the user sends a query to themodel. Then, the model asks for results fromElasticsearchand picks the best ones to return to the user.Benchmarks🔬 Note that we are evaluating models on differently constructed sets than they were trained on (MS Marco vs TREC-CAR), suggesting the generalizability of these models to many other real world search problems.Fine-tuned ModelsDependencyEval SetSearch Boost[1]Speed on GPUnboost/pt-tinybert-msmarco(default)bing queries+45%(0.26 vs 0.18)~50ms/querynboost/pt-bert-base-uncased-msmarcobing queries+62%(0.29 vs 0.18)~300 ms/querynboost/pt-bert-large-msmarcobing queries+77%(0.32 vs 0.18)-nboost/pt-biobert-base-msmarcobiomed+66%(0.17 vs 0.10)~300 ms/queryInstructions for reproducinghere.[1]MRRcompared to BM25, the default for Elasticsearch. Reranking top 50.[2]https://github.com/nyu-dl/dl4marco-bertTo use one of these fine-tuned models with nboost, runnboost --model_dir bert-base-uncased-msmarcofor example, and it will download and cache automatically.Using pre-trained language understanding models, you can boost search relevance metrics by nearly2xcompared to just text search, with little to no extra configuration. While assessing performance, there is often a tradeoff between model accuracy and speed, so we benchmark both of these factors above. This leaderboard is a work in progress, and we intend on releasing more cutting edge models!Install NBoostThere are two ways to get NBoost, either as a Docker image or as a PyPi package.For cloud users, we highly recommend using NBoost via Docker.🚸 Depending on your model, you should install the respective Tensorflow or Pytorch dependencies. We package them below.For installing NBoost, follow the table below.Dependency🐳 Docker📦 Pypi🐙 KubernetesPytorch(recommended)koursaros/nboost:latest-ptpip install nboost[pt]helm install nboost/nboost --set image.tag=latest-ptTensorflowkoursaros/nboost:latest-tfpip install nboost[tf]helm install nboost/nboost --set image.tag=latest-tfAllkoursaros/nboost:latest-allpip install nboost[all]helm install nboost/nboost --set image.tag=latest-all-(for testing)koursaros/nboost:latest-alpinepip install nboosthelm install nboost/nboost --set image.tag=latest-alpineAny way you install it, if you end up reading the following message after$ nboost --helpor$ docker run koursaros/nboost --help, then you are ready to go!Getting StartedThe ProxySetting up a Neural Proxy for Elasticsearch in 3 minutesSetting up an Elasticsearch ServerDeploying the proxyIndexing some dataElastic made easy📡The ProxyTheProxyis the core of NBoost. The proxy is essentially a wrapper to enable serving the model. It is able to understand incoming messages from specific search apis (i.e. Elasticsearch). When the proxy receives a message, it increases the amount of results the client is asking for so that the model can rerank a larger set and return the (hopefully) better results.For instance, if a client asks for 10 results to do with the query "brown dogs" from Elasticsearch, then the proxy may increase the results request to 100 and filter down the best ten results for the client.Setting up a Neural Proxy for Elasticsearch in 3 minutesIn this example we will set up a proxy to sit in between the client and Elasticsearch and boost the results!Installing NBoost with tensorflowIf you want to run the example on a GPU, make sure you have Tensorflow 1.14-1.15, Pytorch or ONNX Runtime with CUDA to support the modeling functionality. However, if you want to just run it on a CPU, don't worry about it. For both cases, just run:pipinstallnboost[pt]Setting up an Elasticsearch Server🔔 If you already have an Elasticsearch server, you can skip this step!If you don't have Elasticsearch, not to worry! We recommend setting up a local Elasticsearch cluster using docker (providing you have Docker installed). First, get the ES image by running:dockerpullelasticsearch:7.4.2Once you have the image, you can run an Elasticsearch server via:dockerrun-d-p9200:9200-p9300:9300-e"discovery.type=single-node"elasticsearch:7.4.2Deploying the proxyNow we're ready to deploy our Neural Proxy! It is very simple to do this, run:nboost\--uhostlocalhost\--uport9200\--search_route"/<index>/_search"\--query_pathurl.query.q\--topk_pathurl.query.size\--default_topk10\--choices_pathbody.hits.hits\--cvalues_path_source.passage📢 The--uhostand--uportshould be the same as the Elasticsearch server above! Uhost and uport are short for upstream-host and upstream-port (referring to the upstream server).If you get this message:Listening: <host>:<port>, then we're good to go!Indexing some dataNBoost has a handy indexing tool built in (nboost-index). For demonstration purposes,  will be indexinga set of passages about traveling and hotelsthrough NBoost. You can add the index to your Elasticsearch server by running:travel.csvcomes with NBoostnboost-index--filetravel.csv--index_nametravel--delim,--id_colNow let's test it out! Hit the Elasticsearch with:curl"http://localhost:8000/travel/_search?pretty&q=passage:vegas&size=2"If the Elasticsearch result has thenboosttag in it, congratulations it's working!What just happened?Let's check out theNBoost frontend. Go to your browser and visitlocalhost:8000/nboost.If you don't have access to a browser, you cancurl http://localhost:8000/nboost/statusfor the same information.The frontend recorded everything that happened:NBoost got a request for2 search results.(average_topk)NBoost connected to the server atlocalhost:9200.NBoost sent a request for 50 search results to the server.(topn)NBoost received50 search resultsfrom the server.(average_choices)The model picked the best 2 search results and returned them to the client.Elastic made easyTo increase the number of parallel proxies, simply increase--workers. For a more robust deployment approach, you can distribute the proxy via Kubernetes (see below).KubernetesSee alsoFor in-depth query DSL and other search API solutions (such as the Bing API), see thedocs.Deploying NBoost via KubernetesWe can easily deploy NBoost in a Kubernetes cluster usingHelm.Add the NBoost Helm RepoFirst we need to register the repo with your Kubernetes cluster.helmrepoaddnboosthttps://raw.githubusercontent.com/koursaros-ai/nboost/master/charts/
helmrepoupdateDeploy some NBoost replicasLet's try deploying four replicas:helminstall--namenboost--setreplicaCount=4nboost/nboostAll possible--set(values.yaml) options are listed below:ParameterDescriptionDefaultreplicaCountNumber of replicas to deploy3image.repositoryNBoost Image namekoursaros/nboostimage.tagNBoost Image taglatest-ptargs.modelName of the model classnilargs.model_dirName or directory of the finetuned modelpt-bert-base-uncased-msmarcoargs.qaWhether to use the qa pluginFalseargs.qa_model_dirName or directory of the qa modeldistilbert-base-uncased-distilled-squadargs.modelName of the model classnilargs.hostHostname of the proxy0.0.0.0args.portPort for the proxy to listen on8000args.uhostHostname of the upstream search api serverelasticsearch-masterargs.uportPort of the upstream server9200args.data_dirDirectory to cache model binarynilargs.max_seq_lenMax combined token length64args.bufsizeSize of the http buffer in bytes2048args.batch_sizeBatch size for running through rerank model4args.multiplierFactor to increase results by5args.workersNumber of threads serving the proxy10args.query_pathJsonpath in the request to find the querynilargs.topk_pathJsonpath to find the number of requested resultsnilargs.choices_pathJsonpath to find the array of choices to reordernilargs.cvalues_pathJsonpath to find the str values of the choicesnilargs.cids_pathJsonpath to find the ids of the choicesnilargs.search_pathThe url path to tag for reranking via nboostnilservice.typeKubernetes Service typeLoadBalancerresourcesresource needs and limits to apply to the pod{}nodeSelectorNode labels for pod assignment{}affinityAffinity settings for pod assignment{}tolerationsToleration labels for pod assignment[]image.pullPolicyImage pull policyIfNotPresentimagePullSecretsDocker registry secret names as an array[](does not add image pull secrets to deployed pods)nameOverrideString to override Chart.namenilfullnameOverrideString to override Chart.fullnamenilserviceAccount.createSpecifies whether a service account is creatednilserviceAccount.nameThe name of the service account to use. If not set and create is true, a name is generated using the fullname templatenilserviceAccount.createSpecifies whether a service account is creatednilpodSecurityContext.fsGroupGroup ID for the containernilsecurityContext.runAsUserUser ID for the container1001ingress.enabledEnable ingress resourcefalseingress.hostNameHostname to your installationnilingress.pathPath within the url structure[]ingress.tlsenable ingress with tls[]ingress.tls.secretNametls type secret to be usedchart-example-tlsDocumentationThe official NBoost documentation is hosted onnboost.readthedocs.io. It is automatically built, updated and archived on every new release.ContributingContributions are greatly appreciated! You can make corrections or updates and commit them to NBoost. Here are the steps:Create a new branch, sayfix-nboost-typo-1Fix/improve the codebaseCommit the changes. Note thecommit message must followthe naming style, sayFix/model-bert: improve the readability and move sectionsMake a pull request. Note thepull request must followthe naming style. It can simply be one of your commit messages, just copy paste it, e.g.Fix/model-bert: improve the readability and move sectionsSubmit your pull request and wait for all checks passed (usually 10 minutes)Coding styleCommit and PR styles checkAll unit testsRequest reviews from one of the developers from our core team.Merge!More details can be found in thecontributor guidelines.Citing NBoostIf you use NBoost in an academic paper, we would love to be cited. Here are the two ways of citing NBoost:\footnote{https://github.com/koursaros-ai/nboost}@misc{koursaros2019NBoost,
  title={NBoost: Neural Boosting Search Results},
  author={Thienes, Cole and Pertschuk, Jack},
  howpublished={\url{https://github.com/koursaros-ai/nboost}},
  year={2019}}LicenseIf you have downloaded a copy of the NBoost binary or source code, please note that the NBoost binary and source code are both licensed under theApache License, Version 2.0.Koursaros AI is excited to bring this open source software to the community.Copyright (C) 2019. All rights reserved.