Create readme.txt
Browse files- my_model/results/readme.txt +32 -0
my_model/results/readme.txt
ADDED
|
@@ -0,0 +1,32 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
|
| 2 |
+
Directory Overview: This directory contains all the evaluation modules and results:
|
| 3 |
+
|
| 4 |
+
################################################################################################################################
|
| 5 |
+
|
| 6 |
+
## 1. evaluation.py
|
| 7 |
+
The `evaluation.py` module provides a comprehensive framework for evaluating Knowledge-Based Visual Question
|
| 8 |
+
Answering (KB-VQA) models. This module includes methods for syntactic and semantic evaluation, leveraging both
|
| 9 |
+
exact match and VQA scores. The evaluation results can be saved to an Excel file for further analysis.
|
| 10 |
+
|
| 11 |
+
Notes
|
| 12 |
+
Ensure that the necessary configuration settings are correctly set in "my_model/config/evaluation_config" file.
|
| 13 |
+
The module requires the following dependencies: pandas, fuzzywuzzy, nltk, ast, typing, streamlit, and openai.
|
| 14 |
+
|
| 15 |
+
################################################################################################################################
|
| 16 |
+
|
| 17 |
+
## 2. demo.py
|
| 18 |
+
The `demo.py` module includes the `ResultDemonstrator` class, which provides a comprehensive framework for visualizing
|
| 19 |
+
and demonstrating the results of Knowledge-Based Visual Question Answering (KB-VQA) models.
|
| 20 |
+
The class utilizes Streamlit for displaying data and Altair for creating interactive visualizations.
|
| 21 |
+
this class is the backbone for the results demo tool of the application.
|
| 22 |
+
|
| 23 |
+
Notes
|
| 24 |
+
Ensure that the necessary configuration settings are correctly set in "my_model/config/evaluation_config".
|
| 25 |
+
The module requires the following dependencies: os, altair, streamlit, PIL, pandas, random.
|
| 26 |
+
|
| 27 |
+
################################################################################################################################
|
| 28 |
+
|
| 29 |
+
## 3. evaluation_results.xlsx
|
| 30 |
+
This file contains all the evaluation results based on the evaluation data (~ 1,000 samples), the results are in the sheet names "Main Data".
|
| 31 |
+
running the evaluation module will read the main columns (Model Answers for all model's configurations) from this sheet and regenerate and save the evaluation rsults.
|
| 32 |
+
This Main Data sheet is the result of running the evaluation module (with the same file name chosen for save).
|