Create home.py

my_model/tabs/home.py

@@ -0,0 +1,36 @@
+import streamlit as st
+import streamlit.components.v1 as components
+
+def run_home():
+    """
+    Displays the home page for the Knowledge-Based Visual Question Answering (KB-VQA) project using Streamlit.
+    This function sets up the main home page for demonstrating the project.
+    """
+
+    st.title('Multimodal Learning for Visual Question Answering using World Knowledge')
+        st.text('')
+        st.header('(Knowledge-Based Visual Question Answering)')
+        with open("Files/Model Arch.html", 'r', encoding='utf-8') as f:
+            model_arch_html = f.read()
+        col1, col2 = st.columns([1,1])
+        with col1:
+            st.header("Model Architecture")
+            components.html(model_arch_html, height=1400)
+        with col2:
+            st.header("Abstract")
+            st.write("""\n\nNavigating the frontier of the Visual Turing Test, this research delves into multimodal learning to bridge the gap between visual perception and linguistic interpretation, a foundational challenge in artificial intelligence. It scrutinizes the integration of visual cognition and external knowledge, emphasizing the pivotal role of the Transformer model in enhancing language processing and supporting complex multimodal tasks.
+                        This research explores the task of Knowledge-Based Visual Question Answering (KB-VQA), it examines the influence of Pre-Trained Large Language Models (PT-LLMs) and Pre-Trained Multimodal Models (PT-LMMs), which have transformed the machine learning landscape by utilizing expansive, pre-trained knowledge repositories to tackle complex tasks, thereby enhancing KB-VQA systems.
+                        \nAn examination of existing Knowledge-Based Visual Question Answering (KB-VQA) methodologies led to a refined approach that converts visual content into the linguistic domain, creating detailed captions and object enumerations. This process leverages the implicit knowledge and inferential capabilities of PT-LLMs. The research refines the fine-tuning of PT-LLMs by integrating specialized tokens, enhancing the models’ ability to interpret visual contexts. The research also reviews current image representation techniques and knowledge sources, advocating for the utilization of implicit knowledge in PT-LLMs, especially for tasks that do not require specialized expertise.
+                        \nRigorous ablation experiments conducted to assess the impact of various visual context elements on model performance, with a particular focus on the importance of image descriptions generated during the captioning phase. The study includes a comprehensive analysis of major KB-VQA datasets, specifically the OK-VQA corpus, and critically evaluates the metrics used, incorporating semantic evaluation with GPT-4 to align the assessment with practical application needs.
+                        \nThe evaluation results underscore the developed model’s competent and competitive performance. It achieves a VQA score of 63.57% under syntactic evaluation and excels with an Exact Match (EM) score of 68.36%. Further, semantic evaluations yield even more impressive outcomes, with VQA and EM scores of 71.09% and 72.55%, respectively. These results demonstrate that the model effectively applies reasoning over the visual context and successfully retrieves the necessary knowledge to answer visual questions.""")
+            st.write("""\n\n\nThis is an interactive application built to demonstrate the project developed and allow for interaction with the KB-VQA model as part of the dissertation for Masters degree in Artificial Intelligence at the [University of Bath](https://www.bath.ac.uk/). 
+                        \n\n\nDeveloped by: [Mohammed H AlHaj](https://www.linkedin.com/in/m7mdal7aj)""")
+            #st.write("\n\n")
+            st.header("Acknowledgement")
+            st.write("""I am profoundly grateful for the support and guidance I have received throughout the course of my dissertation. I would like to extend my deepest appreciation to the following individuals:
+                        \nTo my supervisor, [Dr. Andreas Theophilou](https://researchportal.bath.ac.uk/en/persons/andreas-theophilou), whose expertise, and insightful guidance have been instrumental in the completion of this research. Your mentorship has not only profoundly shaped my work but also my future endeavours in the field of computer science.
+                        Special mention must be made of my mentors at the University of Bath— [Dr. Ben Ralph](https://researchportal.bath.ac.uk/en/persons/ben-ralph), [Dr. Hongping Cai](https://researchportal.bath.ac.uk/en/persons/hongping-cai), and [Dr. Nadejda Roubtsova](https://researchportal.bath.ac.uk/en/persons/nadejda-roubtsova). The wealth of knowledge and insights I have gained from you has been indispensable. Your unwavering dedication to academic excellence and steadfast support have been crucial in navigating my academic journey.
+                        \nMy colleagues deserve equal gratitude, for their camaraderie and collaborative spirit have not only made this journey feasible but also deeply enjoyable. The shared experiences and the challenges we have overcome together have been integral to my personal and professional growth.
+                        \nLastly, my heartfelt thanks are extended to my family, whose unyielding love and encouragement have been my steadfast anchor. Your belief in my abilities has consistently inspired me and bolstered my strength throughout this process.
+                        This dissertation is not merely a reflection of my individual efforts but stands as a testament to the collective support and wisdom of each individual mentioned above. I am honoured and privileged to be part of such a supportive and enriching academic community.
+                        """)