| Directory Overview: This directory contains all the evaluation modules and results: | |
| ################################################################################################################################ | |
| ## 1. evaluation.py | |
| The `evaluation.py` module provides a comprehensive framework for evaluating Knowledge-Based Visual Question | |
| Answering (KB-VQA) models. This module includes methods for syntactic and semantic evaluation, leveraging both | |
| exact match and VQA scores. The evaluation results can be saved to an Excel file for further analysis. | |
| Notes | |
| Ensure that the necessary configuration settings are correctly set in "my_model/config/evaluation_config" file. | |
| The module requires the following dependencies: pandas, fuzzywuzzy, nltk, ast, typing, streamlit, and openai. | |
| ################################################################################################################################ | |
| ## 2. demo.py | |
| The `demo.py` module includes the `ResultDemonstrator` class, which provides a comprehensive framework for visualizing | |
| and demonstrating the results of Knowledge-Based Visual Question Answering (KB-VQA) models. | |
| The class utilizes Streamlit for displaying data and Altair for creating interactive visualizations. | |
| this class is the backbone for the results demo tool of the application. | |
| Notes | |
| Ensure that the necessary configuration settings are correctly set in "my_model/config/evaluation_config". | |
| The module requires the following dependencies: os, altair, streamlit, PIL, pandas, random. | |
| ################################################################################################################################ | |
| ## 3. evaluation_results.xlsx | |
| This file contains all the evaluation results based on the evaluation data (~ 1,000 samples), the results are in the sheet names "Main Data". | |
| running the evaluation module will read the main columns (Model Answers for all model's configurations) from this sheet and regenerate and save the evaluation rsults. | |
| This Main Data sheet is the result of running the evaluation module (with the same file name chosen for save). | |
| ############################################################################################################################### | |
| ## 4. Demo_Images | |
| This folder contains a pool of images used for the demo app, to visualize and demonstrate the images together with the model answers. |