m7mdal7aj commited on
Commit
4b13120
β€’
1 Parent(s): f0297ee

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +10 -10
README.md CHANGED
@@ -17,38 +17,38 @@ license: apache-2.0
17
  KB-VQA
18
  β”œβ”€β”€ Files: Various files required for the demo such as samples images, dissertation report ..etc.
19
  β”œβ”€β”€ models
20
- | β”œβ”€β”€ deformable-detr-detic: DETIC Object Detection Model.
21
- | β”œβ”€β”€ yolov5: YOLOv5 Object Detection Model.baseline)
22
  β”œβ”€β”€ my_model
23
- | β”œβ”€β”€ KBVQA.py : This module is the central component for implementing the designed model architecture for the Knowledge-Based Visual Question Answering (KB-VQA) project.
24
- | β”œβ”€β”€ state_manager.py: Manages the user interface and session state to facilitate the Run Inference tool of the Streamlit demo app.
25
  β”‚ β”œβ”€β”€ LLAMA2
26
  β”‚ β”‚ β”œβ”€β”€ LLAMA2_model.py: Used for loading LLaMA-2 model to be fine-tuned.
27
  β”‚ β”œβ”€β”€ captioner
28
  β”‚ β”‚ β”œβ”€β”€ image_captioning.py: Provides functionality for generating captions for images.
29
- | β”œβ”€β”€ detector
30
  β”‚ β”‚ β”œβ”€β”€ object_detection.py: Used to detect objects in images using object detection models.
31
- | β”œβ”€β”€ fine_tuner
32
  β”‚ β”‚ β”œβ”€β”€ fine_tuner.py: Main Fine-Tuning Script for LLaMa-2 Chat models.
33
  β”‚ β”‚ β”œβ”€β”€ fine_tuning_data_handler.py: Handles and prepares the data for fine-tuning LLaMA-2 Chat models.
34
  β”‚ β”‚ β”œβ”€β”€ fine_tuning_data
35
  β”‚ β”‚ β”‚ β”œβ”€β”€fine_tuning_data_detic.csv: Fine-tuning data prepared by the prompt engineering module using DETIC detector.
36
  β”‚ β”‚ β”‚ β”œβ”€β”€fine_tuning_data_yolov5.csv: Fine-tuning data prepared by the prompt engineering module using YOLOv5. detector.
37
- | β”œβ”€β”€ results
38
  β”‚ β”‚ β”œβ”€β”€ Demo_Images: Contains a pool of images used for the demo app.
39
  β”‚ β”‚ β”œβ”€β”€ evaluation.py: Provides a comprehensive framework for evaluating the KB-VQA model.
40
  β”‚ β”‚ β”œβ”€β”€ demo.py: Provides a comprehensive framework for visualizing and demonstrating the results of the KB-VQA evaluation.
41
  β”‚ β”‚ β”œβ”€β”€ evaluation_results.xlsx : This file contains all the evaluation results based on the evaluation data.
42
- | β”œβ”€β”€ tabs
43
  β”‚ β”‚ β”œβ”€β”€ home.py: Displays an introduction to the application with brief background along with the demo tools description.
44
  β”‚ β”‚ β”œβ”€β”€ results.py: Manages the interactive Streamlit demo for visualizing model evaluation results and analysis.
45
  β”‚ β”‚ β”œβ”€β”€ run_inference.py: Responsible for the 'run inference' tool to test and use the fine-tuned models.
46
  β”‚ β”‚ β”œβ”€β”€ model_arch.py: Displays the model architecture and accompanying abstract and design details
47
  β”‚ β”‚ β”œβ”€β”€ dataset_analysis.py: Provides tools for visualizing dataset analyses.
48
- | β”œβ”€β”€ utilities
49
  β”‚ β”‚ β”œβ”€β”€ ui_manager.py: Manages the user interface for the Streamlit application, handling the creation and navigation of various tabs.
50
  β”‚ β”‚ β”œβ”€β”€ gen_utilities.py: Provides a collection of utility functions and classes commonly used across various parts
51
- | β”œβ”€β”€ config (All Configurations files are kept separated and stored as ".py" for easy reading - this will change after the project submission.)
52
  β”‚ β”‚ β”œβ”€β”€ kbvqa_config.py: Configuration parameters for the main KB-VQA model.
53
  β”‚ β”‚ β”œβ”€β”€ LLAMA2_config.py: Configuration parameters for LLaMA-2 model.
54
  β”‚ β”‚ β”œβ”€β”€ captioning_config.py : Configuration parameters for the captioning model (InstructBLIP).
 
17
  KB-VQA
18
  β”œβ”€β”€ Files: Various files required for the demo such as samples images, dissertation report ..etc.
19
  β”œβ”€β”€ models
20
+ β”‚ β”œβ”€β”€ deformable-detr-detic: DETIC Object Detection Model.
21
+ β”‚ β”œβ”€β”€ yolov5: YOLOv5 Object Detection Model.baseline)
22
  β”œβ”€β”€ my_model
23
+ β”‚ β”œβ”€β”€ KBVQA.py : This module is the central component for implementing the designed model architecture for the Knowledge-Based Visual Question Answering (KB-VQA) project.
24
+ β”‚ β”œβ”€β”€ state_manager.py: Manages the user interface and session state to facilitate the Run Inference tool of the Streamlit demo app.
25
  β”‚ β”œβ”€β”€ LLAMA2
26
  β”‚ β”‚ β”œβ”€β”€ LLAMA2_model.py: Used for loading LLaMA-2 model to be fine-tuned.
27
  β”‚ β”œβ”€β”€ captioner
28
  β”‚ β”‚ β”œβ”€β”€ image_captioning.py: Provides functionality for generating captions for images.
29
+ β”‚ β”œβ”€β”€ detector
30
  β”‚ β”‚ β”œβ”€β”€ object_detection.py: Used to detect objects in images using object detection models.
31
+ β”‚ β”œβ”€β”€ fine_tuner
32
  β”‚ β”‚ β”œβ”€β”€ fine_tuner.py: Main Fine-Tuning Script for LLaMa-2 Chat models.
33
  β”‚ β”‚ β”œβ”€β”€ fine_tuning_data_handler.py: Handles and prepares the data for fine-tuning LLaMA-2 Chat models.
34
  β”‚ β”‚ β”œβ”€β”€ fine_tuning_data
35
  β”‚ β”‚ β”‚ β”œβ”€β”€fine_tuning_data_detic.csv: Fine-tuning data prepared by the prompt engineering module using DETIC detector.
36
  β”‚ β”‚ β”‚ β”œβ”€β”€fine_tuning_data_yolov5.csv: Fine-tuning data prepared by the prompt engineering module using YOLOv5. detector.
37
+ β”‚ β”œβ”€β”€ results
38
  β”‚ β”‚ β”œβ”€β”€ Demo_Images: Contains a pool of images used for the demo app.
39
  β”‚ β”‚ β”œβ”€β”€ evaluation.py: Provides a comprehensive framework for evaluating the KB-VQA model.
40
  β”‚ β”‚ β”œβ”€β”€ demo.py: Provides a comprehensive framework for visualizing and demonstrating the results of the KB-VQA evaluation.
41
  β”‚ β”‚ β”œβ”€β”€ evaluation_results.xlsx : This file contains all the evaluation results based on the evaluation data.
42
+ β”‚ β”œβ”€β”€ tabs
43
  β”‚ β”‚ β”œβ”€β”€ home.py: Displays an introduction to the application with brief background along with the demo tools description.
44
  β”‚ β”‚ β”œβ”€β”€ results.py: Manages the interactive Streamlit demo for visualizing model evaluation results and analysis.
45
  β”‚ β”‚ β”œβ”€β”€ run_inference.py: Responsible for the 'run inference' tool to test and use the fine-tuned models.
46
  β”‚ β”‚ β”œβ”€β”€ model_arch.py: Displays the model architecture and accompanying abstract and design details
47
  β”‚ β”‚ β”œβ”€β”€ dataset_analysis.py: Provides tools for visualizing dataset analyses.
48
+ β”‚ β”œβ”€β”€ utilities
49
  β”‚ β”‚ β”œβ”€β”€ ui_manager.py: Manages the user interface for the Streamlit application, handling the creation and navigation of various tabs.
50
  β”‚ β”‚ β”œβ”€β”€ gen_utilities.py: Provides a collection of utility functions and classes commonly used across various parts
51
+ β”‚ β”œβ”€β”€ config (All Configurations files are kept separated and stored as ".py" for easy reading - this will change after the project submission.)
52
  β”‚ β”‚ β”œβ”€β”€ kbvqa_config.py: Configuration parameters for the main KB-VQA model.
53
  β”‚ β”‚ β”œβ”€β”€ LLAMA2_config.py: Configuration parameters for LLaMA-2 model.
54
  β”‚ β”‚ β”œβ”€β”€ captioning_config.py : Configuration parameters for the captioning model (InstructBLIP).