progress in readme

README.md

@@ -11,3 +11,22 @@ license: apache-2.0
 ---
 
 # Fact Checking rocks!   [![Generic badge](https://img.shields.io/badge/🤗-Open%20in%20Spaces-blue.svg)](https://huggingface.co/spaces/anakin87/fact-checking-rocks) [![Generic badge](https://img.shields.io/github/stars/anakin87/fact-checking-rocks?label=Github&style=social)](https://github.com/anakin87/fact-checking-rocks)
+
+## *Fact checking baseline combining dense retrieval and textual entailment*
+
+### Idea 💡
+This project aims to show that a naive and simple baseline for fact checking can be built by combining dense retrieval and a textual entailment task (based on Natural Language Inference models).
+
+### System description
+This project is strongly based on [Haystack](https://github.com/deepset-ai/haystack), an open source NLP framework to realize search system. The main components of our system are an indexing pipeline and a search pipeline.
+
+#### Indexing pipeline
+* [Crawling](https://github.com/anakin87/fact-checking-rocks/blob/321ba7893bbe79582f8c052493acfda497c5b785/notebooks/get_wikipedia_data.ipynb): Crawl data from Wikipedia, starting from the page [List of mainstream rock performers](https://en.wikipedia.org/wiki/List_of_mainstream_rock_performers) and using the [python wrapper](https://github.com/goldsmith/Wikipedia)
+* [Indexing through Haystack](https://github.com/anakin87/fact-checking-rocks/blob/321ba7893bbe79582f8c052493acfda497c5b785/notebooks/indexing.ipynb)
+  * Preprocess the downloaded documents into chunks consisting of 2 sentences
+  * Chunks with less than 10 words are discarded, because not very informative
+  * Instantiate a [FAISS](https://github.com/facebookresearch/faiss) Document store and store the passages on it
+  * Create embeddings for the passages, using a Sentence Transformer model and save them in FAISS. It seems that the retrieval task will involve [*asymmetric semantic search*](https://www.sbert.net/examples/applications/semantic-search/README.html#symmetric-vs-asymmetric-semantic-search) (statements to be verified are usually shorter than inherent passages), therefore I choose the model `msmarco-distilbert-base-tas-b`.
+  * Save FAISS index
+
+#### Search pipeline

app_utils/frontend_utils.py

@@ -10,11 +10,15 @@ entailment_html_messages = {
 }
 
 def build_sidebar():
-    st.sidebar.markdown('# Fact checking 🎸 Rocks!')
-    st.sidebar.markdown('*Fact checking baseline combining dense retrieval and textual entailment*')
-    st.sidebar.markdown('[Github project](https://github.com/anakin87/fact-checking-rocks) - Based on [Haystack](https://github.com/deepset-ai/haystack)')
-    st.sidebar.markdown('<small>Data crawled from [Wikipedia](https://en.wikipedia.org/wiki/List_of_mainstream_rock_performers).</small>', unsafe_allow_html=True)
-
+    sidebar="""
+    <h1 style='text-align: center'>Fact checking 🎸 Rocks!</h1>
+    <div style='text-align: center'>
+    <i>Fact checking baseline combining dense retrieval and textual entailment</i>
+    <p><br/><a href='https://github.com/anakin87/fact-checking-rocks'>Github project</a> - Based on <a href='https://github.com/deepset-ai/haystack'>Haystack</a></p>
+    <p><small><a href='https://en.wikipedia.org/wiki/List_of_mainstream_rock_performers'>Data crawled from Wikipedia</a></small></p>
+    </div>
+    """
+    st.sidebar.markdown(sidebar, unsafe_allow_html=True)
 
 def set_state_if_absent(key, value):
     if key not in st.session_state:

data/statements.txt

@@ -45,4 +45,5 @@ The Cure made dark songs
 Cannibal Corpse is a pop punk band
 Slipknot wear masks
 Toto have sold many records
-The verve were a British band
+The verve were a British band
+Psychokiller is a hit by Talking Heads

pages/Info.py

@@ -1 +1,4 @@
 import streamlit as st
+from app_utils.frontend_utils import build_sidebar
+
+build_sidebar()