Package: searchai
Cloud-Native Neural Search[?]Framework forAnyKind of DataJina allows you to build deep learning-powered search-as-a-service in just minutes.🌌All data type- Large-scale indexing and querying of any kind of unstructured data: video, image, long/short text, music, source code, PDF, etc.🌩️Fast & cloud-native- Distributed architecture from day one. Scalable & cloud-native by design: enjoy
containerizing, distributing, streaming, paralleling, sharding, async scheduling with REST/gRPC/WebSocket.⏱️Save time-Thedesign pattern of neural search systems, from zero to a production-ready system in minutes.🍱Own your stack- Keep an end-to-end stack ownership of your solution, avoid integration pitfalls with
fragmented, multi-vendor, generic legacy tools.Run Quick Demo👗 Fashion image search:pip install --pre && jina hello fashion🤖 QA chatbot:pip install --pre "jina[chatbot]" && jina hello chatbot📰 Multimodal search:pip install --pre "jina[multimodal]" && jina hello multimodal🍴 Fork the source of a demo to your folder:jina hello fork fashion ../my-proj/Install2.0 is in pre-release,add--preto install it.Why 2.0?$pipinstall--prejina$jina-v2.0.0rcNvia Docker$dockerrunjinaai/jina:master-v2.0.0rcN📦 More installation optionsx86/64,arm64,v6,v7,Apple M1On Linux/macOS & Python 3.7/3.8/3.9Docker UsersStandardpip install --pre jinadocker run jinaai/jina:masterDaemonpip install --pre "jina[daemon]"docker run --network=host jinaai/jina:master-daemonWith Extraspip install --pre "jina[devel]"docker run jinaai/jina:master-develVersion identifiersare explained here. Jina can run
onWindows Subsystem for Linux. We welcome the community
to help us withnative Windows support.Get StartedDocument, Executor, and Flow are the three fundamental concepts in Jina.📄Documentis the basic data type in Jina;⚙️Executoris how Jina processes Documents;🔀Flowis how Jina streamlines and distributes Executors.Copy-paste the minimum example below and run it:💡 Preliminaries:character embedding,pooling,Euclidean distanceimportnumpyasnpfromjinaimportDocument,DocumentArray,Executor,Flow,requestsclassCharEmbed(Executor):# a simple character embedding with mean-poolingoffset=32# letter `a`dim=127-offset+1# last pos reserved for `UNK`char_embd=np.eye(dim)*1# one-hot embedding for all chars@requestsdeffoo(self,docs:DocumentArray,**kwargs):fordindocs:r_emb=[ord(c)-self.offsetifself.offset<=ord(c)<=127else(self.dim-1)forcind.text]d.embedding=self.char_embd[r_emb,:].mean(axis=0)# average poolingclassIndexer(Executor):_docs=DocumentArray()# for storing all documents in memory@requests(on='/index')deffoo(self,docs:DocumentArray,**kwargs):self._docs.extend(docs)# extend stored `docs`@requests(on='/search')defbar(self,docs:DocumentArray,**kwargs):q=np.stack(docs.get_attributes('embedding'))# get all embeddings from query docsd=np.stack(self._docs.get_attributes('embedding'))# get all embeddings from stored docseuclidean_dist=np.linalg.norm(q[:,None,:]-d[None,:,:],axis=-1)# pairwise euclidean distancefordist,queryinzip(euclidean_dist,docs):# add & sort matchquery.matches=[Document(self._docs[int(idx)],copy=True,score=d)foridx,dinenumerate(dist)]query.matches.sort(key=lambdam:m.score.value)# sort matches by their valuesf=Flow(port_expose=12345).add(uses=CharEmbed,parallel=2).add(uses=Indexer)# build a Flow, with 2 parallel CharEmbed, tho unnecessarywithf:f.post('/index',(Document(text=t.strip())fortinopen(__file__)ift.strip()))# index all lines of this filef.block()# block for listening requestKeep the above running and start a simple client:fromjinaimportClient,Documentdefprint_matches(req):# the callback function invoked when task is doneforidx,dinenumerate(req.docs[0].matches[:3]):# print top-3 matchesprint(f'[{idx}]{d.score.value:2f}: "{d.text}"')c=Client(host='localhost',port_expose=12345)# connect to localhost:12345c.post('/search',Document(text='request(on=something)'),on_done=print_matches)It finds the lines most similar to "request(on=something)" from the server code snippet and prints the following:Client@1608[S]:connected to the gateway at localhost:12345!
[0]0.168526: "@requests(on='/index')"
[1]0.181676: "@requests(on='/search')"
[2]0.192049: "query.matches = [Document(self._docs[int(idx)], copy=True, score=d) for idx, d in enumerate(dist)]"😔 Doesn't work? Our bad!Please report it here.Read Tutorials🧠What is "Neural Search"?📄Document&DocumentArray: the basic data type in Jina.Minimum working exampleDocumentAPIDocumentArrayAPI⚙️Executor: how Jina processes Documents.Minimum working exampleExecutorAPIExecutorBuilt-in FeaturesUse Tensorflow, Pytorch, Pytorch Lightning, Fastai, Mindspore, PaddlePaddle, Scikit-learn inExecutor🔀Flow: how Jina streamlines and distributes Executors.Minimum working exampleFlowAPIFlow-as-a-Service🧼Write clean code in Jina📓Developer References😎3 Reasons to use Jina 2.0SupportJoin ourSlack communityto chat to our engineers about your use cases, questions, and
support queries.Join our Engineering All Hands meet-up to discuss your use case and learn Jina's new features.When?The second Tuesday of every monthWhere?Zoom (see our public events calendar/.ical)
andlive stream on YouTubeSubscribe to the latest video tutorials on ourYouTube channel.Join UsJina is backed byJina AI.We are actively hiringfull-stack developers,
solution engineers to build the next neural search ecosystem in open source.ContributingWe welcome all kinds of contributions from the open-source community, individuals and partners. We owe our success to
your active involvement.Contributing guidelinesCode of conduct- play nicely with the Jina
communityGood first issuesRelease cycles and development stages