apohllo commited on
Commit
99ee678
1 Parent(s): 1069dd1

Model card

Browse files
Files changed (1) hide show
  1. README.md +40 -0
README.md ADDED
@@ -0,0 +1,40 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ language: pl
3
+ datasets:
4
+ - czywiesz
5
+ ---
6
+
7
+ # Model description
8
+
9
+ The model was created for selective question answering in Polish. I.e. it is used to find passages containing the answers to the given question.
10
+
11
+ It is used to encode the contexts (aka passages) in the DPR bi-encoder architecture. The architecture requires two separate models.
12
+ The question part has to be encoded with the corresponding [question encoder](https://huggingface.co/enelpol/czywiesz-question).
13
+
14
+ The model was created by fine-tuning [Herbert base](https://huggingface.co/allegro/herbert-base) on "Czywiesz" dataset.
15
+ [Czywiesz](https://clarin-pl.eu/dspace/handle/11321/39) dataset contains questions and Wikipedia articles extracted from the Polish Wikipedia.
16
+
17
+
18
+ # Usage
19
+
20
+ It is the easiest to use the model with the [Haystack framework](https://haystack.deepset.ai/overview/intro).
21
+
22
+ ```
23
+ from haystack.document_stores import FAISSDocumentStore
24
+ from haystack.retriever import DensePassageRetriever
25
+
26
+ document_store = FAISSDocumentStore(faiss_index_factory_str="Flat")
27
+
28
+ retriever = DensePassageRetriever(
29
+ document_store=document_store,
30
+ query_embedding_model="enelpol/czywiesz-question",
31
+ passage_embedding_model="enelpol/czywiesz-context"
32
+ )
33
+
34
+ for document in document:
35
+ document_store.write_documents([document])
36
+
37
+ document_store.udpate_embeddings(retriever)
38
+ document_store.save("contexts.faiss")
39
+
40
+ ```