davanstrien HF staff commited on
Commit
e3dcfdd
β€’
1 Parent(s): 10a9ffa
Files changed (1) hide show
  1. app.py +10 -2
app.py CHANGED
@@ -133,10 +133,18 @@ def generate_response(image):
133
  return {}
134
 
135
 
136
- title = "ColPali Query Generator"
137
- description = """This Space uses the Qwen2-VL model to generate queries for document retrieval tasks primarily focused on ColPali fine-tuning data.
 
 
 
 
 
138
 
139
  This [blog post](https://danielvanstrien.xyz/posts/post-with-code/colpali/2024-09-23-generate_colpali_dataset.html) gives an overview of how you can use this kind of approach to generate a full dataset for fine-tuning ColPali models.
 
 
 
140
  """
141
 
142
  demo = gr.Interface(
 
133
  return {}
134
 
135
 
136
+ title = "ColPali fine-tuning Query Generator"
137
+ description = """[ColPali](https://huggingface.co/papers/2407.01449) is a very exciting new approach to multimodal document retrieval which aims to replace existing document retrievers which often rely on an OCR step with an end-to-end multimodal approach.
138
+ To train ColPali models, we need a dataset of image-text pairs which represent the document images and the relevant text queries which those documents should match.
139
+ To make the ColPali models work even better we might want a dataset of query/image document pairs related to our domain or task.
140
+ One way in which we might go about generating such a dataset is to use an VLM to generate synthetic queries for us.
141
+ This space uses the [Qwen/Qwen2-VL-7B-Instruct](https://huggingface.co/Qwen/Qwen2-VL-7B-Instruct) to generate queries for a document, based on an input document image.
142
+
143
 
144
  This [blog post](https://danielvanstrien.xyz/posts/post-with-code/colpali/2024-09-23-generate_colpali_dataset.html) gives an overview of how you can use this kind of approach to generate a full dataset for fine-tuning ColPali models.
145
+
146
+ If you want to convert a PDF(s) to a dataset of page images you can try out the [ PDFs to Page Images Converter](https://huggingface.co/spaces/Dataset-Creation-Tools/pdf-to-page-images-dataset) Space.
147
+
148
  """
149
 
150
  demo = gr.Interface(