fix/chore: final fix of attention, preperating for last release

Dockerfile

@@ -1,10 +1,10 @@
 # standard Dockerfile to build a complete, working image of the webapp
 
 # complete build based on clean python (slower)
-#FROM python:3.11.6
+FROM python:3.11.6
 
 # build based on thesis base with dependencies (quicker) - for dev
-FROM thesis-base:0.1.1
+#FROM thesis-base:0.2.0
 
 # install dependencies and copy files into image folder
 COPY requirements.txt .
@@ -19,5 +19,5 @@ RUN ls --recursive .
 CMD ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "8080"]
 
 # build and run commands:
-## docker build -t thesis:1.2.1 -f Dockerfile .
-## docker run -d --name thesis -e HOSTING=local -p 8080:8080 thesis:1.2.1
+## docker build -t thesis:1.3.1 -f Dockerfile .
+## docker run -d --name thesis -e HOSTING=local -p 8080:8080 thesis:1.3.1

Dockerfile-Base

@@ -1,12 +1,12 @@
 # Dockerfile to create base image with all needed dependencies. for the quick dockerfile build of the webapp
 # because all dependencies are already installed, the next webapp build using this base image is much quicker
 
-# using newest python as a base image
-FROM thesis-base:0.1.1
+# complete build based on clean python (slower)
+FROM python:3.11.6
 
 # install dependencies based on requirements
 COPY requirements.txt ./
 RUN pip install --no-cache-dir --upgrade -r requirements.txt
 
 # build and run commands
-## docker build -t thesis-base:1.0.1 -f Dockerfile-Base .
+## docker build -t thesis-base:2.0.0 -f Dockerfile-Base .

README.md

@@ -19,6 +19,8 @@ disable_embedding: true
 ## 📜 Info:
 This is the UI showcase for my thesis about the interpretability of LLM based chatbot application and applications of XAI.
 
+**Current Release: v1.3.1**
+
 ### 🔗 Links:
 
 **[GitHub Repository](https://github.com/LennardZuendorf/thesis-webapp)**
@@ -49,7 +51,7 @@ This Project was part of my studies of Business Computing at the University of A
 
 1. Clone the repository using git or GitHub cli.
 2. Start the (virtual) environment.
-3. Set the environment variable "HOSTING", i.e. like this `export HOSTING="local"`, see [fastAPI Docu](https://fastapi.tiangolo.com/advanced/settings/)
+3. Set the environment variable "HOSTING", i.e. like this `export HOSTING=local USER=admin PW=test`, see [fastAPI Docu](https://fastapi.tiangolo.com/advanced/settings/)
 3. Install the requirements using `pip install -r requirements.txt`
 4. Run the app using `uvicorn main:app`. You can add `--reload` to enable hot reloading. The app will be available at `localhost:8000`.
 
@@ -58,14 +60,14 @@ This Project was part of my studies of Business Computing at the University of A
 
 1. Clone the repository using git or GitHub cli.
 2. Build the docker image using `docker build -t thesis-webapp -f Dockerfile . .`, the command commented in the docker file or the command referenced by your hosting service.
-3. Run the docker image using `docker run --name thesis-webapp -e HOSTING=local -p 8080:8080 thesis-webapp`, the command commented in the docker file or the command referenced by your hosting service.
+3. Run the docker image using `docker run --name thesis-webapp -e HOSTING=local USER=admin PW=test -p 8080:8080 thesis-webapp`, the command commented in the docker file or the command referenced by your hosting service.
 4. The app will be available at `localhost:8080`. If you are using a hosting service, the port may be different.
 
 ### 🐳 Docker Image :
 (This assumes you have set up docker desktop or are using a hosting service able to handle Docker images.)
 
-1. Pull the docker image from ghcr using `docker pull ghcr.io/LennardZuendorf/thesis-webapp:latest`.
-2. Run the docker image in terminal using `docker run --name thesis-webapp -e HOSTING=local -p 8080:8080 lennardzuendorf/thesis-webapp:latest`, the command commented in the docker file or the command referenced by your hosting service.
+1. Pull the docker image from ghcr using `docker pull ghcr.io/LennardZuendorf/thesis-webapp:1.3.1`.
+2. Run the docker image in terminal using `docker run --name thesis-webapp -e HOSTING=local USER=admin PW=test -p 8080:8080 lennardzuendorf/thesis-webapp::1.3.1`, the command commented in the docker file or the command referenced by your hosting service.
 3. The app will be available at `localhost:8080`. If you are using a hosting service, the port may be different.
 
 ## 📝 License and Credits:
@@ -79,15 +81,25 @@ This project is licensed under the MIT License, see [LICENSE](LICENSE.md) for mo
 
 See code for in detailed credits, work is strongly based on:
 
+#### captum
+- [GitHub](https://github.com/pytorch/captum)
+- [Inital Paper](https://arxiv.org/pdf/2009.07896.pdf)
+
+#### shap
+- [GitHub](https://github.com/shap/shap)
+- [Inital Paper](https://arxiv.org/abs/1705.07874)
+
 #### GODEL
 - [HGF Model Page](https://huggingface.co/microsoft/GODEL-v1_1-large-seq2seq?text=Hey+my+name+is+Mariama%21+How+are+you%3F)
 - [Paper on HGF](https://huggingface.co/papers/2206.11309)
 - [Paper Print](https://arxiv.org/abs/2206.11309)
 
-#### SHAP
-- [Github](https://github.com/shap/shap)
-- [Initial Paper](https://arxiv.org/abs/1705.07874)
+#### Mistral 7B (Instruct)
+- [HGF Model Page](https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.2)
+- [Paper on HGF](https://huggingface.co/papers/2310.06825)
+- [Paper Print](https://arxiv.org/abs/2310.06825)
+
 
 #### Custom Component (/components/iframe/)
 
-Is based on Gradio component, see indivdual README for full changelog.
+Is based on Gradio component, see individual README for full changelog.

main.py

@@ -300,10 +300,10 @@ with gr.Blocks(
                     ],
                     inputs=[
                         user_prompt,
+                        xai_selection,
                         knowledge_input,
                         system_prompt,
                         model_selection,
-                        xai_selection,
                     ],
                 )
 
@@ -382,7 +382,7 @@ if __name__ == "__main__":
     # use standard gradio launch option for hgf spaces
     if os.environ["HOSTING"].lower() == "spaces":
         # set password to deny public access
-        ui.launch(auth=("htw", "berlin@123"))
+        ui.launch(auth=(os.environ["USER"], os.environ["PW"]))
 
     # otherwise run the application on port 8080 in reload mode
     ## for local development, uses Docker for Prod deployment

model/mistral.py

@@ -1,6 +1,7 @@
 # Mistral model module for chat interaction and model instance control
 
 # external imports
+import re
 from transformers import AutoTokenizer, AutoModelForCausalLM, GenerationConfig
 import torch
 import gradio as gr
@@ -92,8 +93,9 @@ def format_answer(answer: str):
     # empty answer string
     formatted_answer = ""
 
-    # splitting answer by instruction tokens
-    segments = answer.split("[/INST]")
+    # splitting answer by instruction tokens using re and a pattern
+    pattern = r"\[/INST\]|\[ / INST\]|\[ / INST \]|\[/ INST \]"
+    segments = re.split(pattern, answer)
 
     # checking if proper history got returned
     if len(segments) > 1:
@@ -103,11 +105,10 @@ def format_answer(answer: str):
         # return warning and full answer if not enough [/INST] tokens found
         gr.Warning("""
                    There was an issue with answer formatting...\n
-                   returning the full answer.
+                   Returning the full answer.
                    """)
         formatted_answer = answer
 
-    print(f"CUT:\n {answer}\nINTO:\n{formatted_answer}")
     return formatted_answer
 
 

public/credits_dataprotection_license.md

@@ -1,18 +1,30 @@
 # Credits
 
 ### Models
-This implementation is build on GODEL by Microsoft, Inc.
+This implementation is build on GODEL by Microsoft, Inc. and Mistral 7B Instruct by Mistral AI.
+
+##### Mistral 7B Instruct
+Mistral 7B Instruct is an open source model by Mistral AI. See [offical paper](https://arxiv.org/abs/2310.06825) for more information.
+
+- the version used in this project is Mistral 7B Instruct v0.2, see [huggingface model hub](https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.2)
+- the model is an autoregressive language model fine-tuned for instruction following
 
 ##### GODEL
 GODEL is an open source model by Microsoft. See [offical paper](https://arxiv.org/abs/2206.11309) for more information.
 
 - the version used in this project is GODEL Large, see [huggingface model hub](https://huggingface.co/microsoft/GODEL-v1_1-large-seq2seq?text=Hey+my+name+is+Thomas%21+How+are+you%3F)
 - the model as is a generative seq2seq transformer fine-tuned for goal directed dialog
-- it supports context and knowledge base inputs
 
 ### Libraries
 This project uses a number of open source libraries, only the most important ones are listed below.
 
+#### captum
+This application uses the captum library for the interpretation of the Mistral 7B Instruct model. The captum library is available at [GitHub](https://github.com/pytorch/captum).
+
+- Please refer to the [captum](https://captum.ai/) website for more information about the library.
+- The used KernelExplainer is based on work by Lundberg et al. - see below for more information.
+- For original paper about captum see [Inital Paper](https://arxiv.org/pdf/2009.07896.pdf).
+
 ##### Shap
 This application uses a custom version of the shap library, which is available at [GitHub](https://github.com/shap/shap).
 
@@ -23,7 +35,7 @@ This application uses a custom version of the shap library, which is available a
 This application uses attention visualization inspired by the bertviz library, which is available at[GitHub](https://github.com/jessevig/bertviz). It doesn't actually use BERTViz.
 
 - The bertviz was introduced by Vig et al. (2019), see [offical paper](https://arxiv.org/pdf/1906.05714.pdf) for more information.
-- This project only uses cross attention visualization with gradio and matplotlib.
+- This project only uses decoder attention visualization with gradio and matplotlib and not BERTViz itself.
 
 
 # Data Protection

utils/formatting.py

@@ -41,8 +41,24 @@ def format_output_text(output: list):
 
     # add all other list items with a space in between
     for txt in formatted_output[1:]:
-        # check if the token is a punctuation mark
-        if txt in [".", ",", "!", "?"]:
+        # check if the token is a punctuation mark or other special character
+        if txt in [
+            ".",
+            ",",
+            "!",
+            "?",
+            ":",
+            ";",
+            ")",
+            "]",
+            "}",
+            "'",
+            '"',
+            "[",
+            "{",
+            "(",
+            "<",
+        ]:
             # add punctuation mark without space
             output_str += txt
         # add token with space if not empty
@@ -50,7 +66,7 @@ def format_output_text(output: list):
             output_str += " " + txt
 
     # return the combined string with multiple spaces removed
-    return re.sub(" +", " ", output_str)
+    return re.sub(r"\s+", " ", output_str)
 
 
 # format the tokens by removing special tokens and special characters