Update README.md

README.md

@@ -1,26 +1,176 @@
 ---
 datasets:
-- aorogat/QueryBridge
+- USERNAME/QueryBridge
 ---
+- This model is a fine-tuned version of llama3 using LoRA. We used TorchTune to fine-tune the model. Below, you will find a section on how we fine-tuned it.
+
 # Model Overview
 
-This model employs **fine-tuning** using **Low-Rank Adaptation (LoRA)** for mapping questions to tagged questions.
+This model is a fine-tuned version of llama3 using the [QueryBridge dataset](https://huggingface.co/datasets/USERNAME/QueryBridge). We utilized **Low-Rank Adaptation (LoRA)** to train it for tagging question components using the tags in the table below. The demo video shows how the mapped question appears and, after converting it to a graph representation, how we visualized it as shown in the video.
 
 The tagged questions in the QueryBridge dataset are designed to train language models to understand the components and structure of a question effectively. By annotating questions with specific tags such as `<qt>`, `<p>`, `<o>`, and `<s>`, we provide a detailed breakdown of each question's elements, which aids the model in grasping the roles of different components.
 
 For example, the video below demonstrates how a model can be trained to interpret these tagged questions. We convert these annotated questions into a graph representation, which visually maps out the relationships and roles within the question. This graph-based representation facilitates the construction of queries in various query languages such as SPARQL, SQL, Cypher, and others, by translating the structured understanding into executable query formats. This approach not only enhances the model’s ability to parse and generate queries across different languages but also ensures consistency and accuracy in query formulation.
 
-
 <a href="https://youtu.be/J_N-6m8fHz0">
     <img src="https://cdn-uploads.huggingface.co/production/uploads/664adb4a691370727c200af0/sDfp7DiYrGKvH58KdXOIY.png" alt="Training Model with Tagged Questions" width="400" height="300" />
 </a>
 
-## Fine-Tuning with LoRA
-- Fine-tuning adjusts the model's parameters for specific tasks, enhancing its ability to handle nuanced requirements.
-- **LoRA** allows efficient updates by modifying only a subset of model weights, significantly reducing computational overhead while maintaining or improving performance.
-- The fine-tuned **llama3-Finetuned** model demonstrates exceptional stability and accuracy, achieving perfect scores with no error margins.
+# Tags Used in Tagged Questions
+
+The tagging system categorizes different components of the question as follows:
+
+| Tag   | Description |
+|-------|-------------|
+| `<qt>` | **Question Type**: Tags the keywords or phrases that denote the type of question being asked, such as 'What', 'Who', 'How many', etc. This tag helps determine the type of SPARQL query to generate. Example: In "What is the capital of Canada?", the tag `<qt>What</qt>` indicates that the question is asking for an entity retrieval. |
+| `<o>`  | **Object Entities**: Tags entities that are objects in the question. These are usually noun phrases referring to the entities being described or queried. Example: In "What is the capital of Canada?", the term 'Canada' is tagged as `<o>Canada</o>`. |
+| `<s>`  | **Subject Entities**: Tags entities that are subjects in Yes-No questions. This tag is used exclusively for questions that can be answered with 'Yes' or 'No'. Example: In "Is Ottawa the capital of Canada?", the entity 'Ottawa' is tagged as `<s>Ottawa</s>`. |
+| `<p>`  | **Predicates**: Tags predicates that represent relationships or attributes in the knowledge graph. Predicates can be verb phrases or noun phrases that describe how entities are related. Example: In "What is the capital of Canada?", the phrase 'is the capital of' is tagged as `<p>is the capital of</p>`. |
+| `<cc>` | **Coordinating Conjunctions**: Tags conjunctions that connect multiple predicates or entities in complex queries. These include words like 'and', 'or', and 'nor'. They influence how the SPARQL query combines conditions. Example: In "Who is the CEO and founder of Apple Inc?", the conjunction 'and' is tagged as `<cc>and</cc>`. |
+| `<off>`| **Offsets**: Tags specific terms that indicate position or order in a sequence, such as 'first', 'second', etc. These are used in questions asking for ordinal positions. Example: In "What is the second largest country?", the word 'second' is tagged as `<off>second</off>`. |
+| `<t>`  | **Entity Types**: Tags that describe the type or category of the entities involved in the question. This can include types like 'person', 'place', 'organization', etc. Example: In "Which film directed by Garry Marshall?", the type 'film' might be tagged as `<t>film</t>`. |
+| `<op>` | **Operators**: Tags operators used in questions that involve comparisons or calculations, such as 'greater than', 'less than', 'more than'. Example: In "Which country has a population greater than 50 million?", the operator 'greater than' is tagged as `<op>greater than</op>`. |
+| `<ref>`| **References**: Tags in questions that refer back to previously mentioned entities or concepts. These can indicate cycles or self-references in queries. Example: In "Who is the CEO of the company founded by himself?", the word 'himself' is tagged as `<ref>himself</ref>`. |
+
+
+## How to use the model?
+To use the model, you can run it with TorchTune commands. I have provided the necessary Python code to automate the process. Follow these steps to get started:
+
+### Step 1: Create a Configuration File
+First, save a file named `custom_generation_config_bigModel.yaml` in `/home/USERNAME/` with the following content:
+
+```yaml
+# Config for running the InferenceRecipe in generate.py to generate output from an LLM
+
+# Model arguments
+model:
+  _component_: torchtune.models.llama3.llama3_8b
+
+checkpointer:
+  _component_: torchtune.utils.FullModelMetaCheckpointer
+  checkpoint_dir: /home/USERNAME/Meta-Llama-3-8B/
+  checkpoint_files: [
+    meta_model_0.pt
+  ]
+  output_dir: /home/USERNAME/Meta-Llama-3-8B/
+  model_type: LLAMA3
+
+device: cuda
+dtype: bf16
+
+seed: 1234
+
+# Tokenizer arguments
+tokenizer:
+  _component_: torchtune.models.llama3.llama3_tokenizer
+  path: /home/USERNAME/Meta-Llama-3-8B/original/tokenizer.model
+
+# Generation arguments; defaults taken from gpt-fast
+prompt: "### Instruction: \nYou are a powerful model trained to convert questions to tagged questions. Use the tags as follows: \n<qt> to surround question keywords like 'What', 'Who', 'Which', 'How many', 'Return' or any word that represents requests. \n<o> to surround entities as an object like person name, place name, etc. It must be a noun or a noun phrase. \n<s> to surround entities as a subject like person name, place name, etc. The difference between <s> and <o>, <s> only appear in yes/no questions as in the training data you saw before. \n<cc> to surround coordinating conjunctions that connect two or more phrases like 'and', 'or', 'nor', etc. \n<p> to surround predicates that may be an entity attribute or a relationship between two entities. It can be a verb phrase or a noun phrase. The question must contain at least one predicate. \n<off> for offset in questions asking for the second, third, etc. For example, the question 'What is the second largest country?', <off> will be located as follows. 'What is the <off>second</off> largest country?' \n<t> to surround entity types like person, place, etc. \n<op> to surround operators that compare quantities or values, like 'greater than', 'more than', etc. \n<ref> to indicate a reference within the question that requires a cycle to refer back to an entity (e.g., 'Who is the CEO of a company founded by himself?' where 'himself' would be tagged as <ref>himself</ref>). \nInput: Which films directed by a dirctor died in 2014 and starring both Julia Roberts and Richard Gere?\nResponse:"
+max_new_tokens: 100
+temperature: 0.6
+top_k: 1
+
+quantizer: null
+```
+
+### Step 2: Set Up the Environment
+Create a virtual environment:
+
+```bash
+/home/USERNAME/myenv
+```
+
+Install TorchTune with:
+```bash
+pip install torchtune
+```
+
+### Step 3: Create the Python File
+Next, create a Python file called `command.py` with the following content:
+
+```python
+import subprocess
+import os
+import re
+import shlex  # For safely handling command line arguments
+
+def _create_config_file(question):
+    # Path to the template and output config file
+    template_path = "/home/USERNAME/custom_generation_config_bigModel.yaml"
+    output_path = "/tmp/dynamic_generation.yaml"
+    
+    # Load the template from the file
+    with open(template_path, 'r') as file:
+        config_template = file.read()
+
+    # Replace the placeholder in the template with the actual question
+    updated_prompt = config_template.replace("Input: Which films directed by a dirctor died in 2014 and starring both Julia Roberts and Richard Gere?", f"Input: {question}")
+    maxLen = int(1.3*len(question))
+    print(f"maxLen: {maxLen}")
+    updated_prompt = updated_prompt.replace("max_new_tokens: 100", f"max_new_tokens: {maxLen}")
+
+    # Write the updated configuration to a new file
+    with open(output_path, 'w') as file:
+        file.write(updated_prompt)
+    
+    print(f"Configuration file created at: {output_path}")
+
+def get_tagged_question(question):
+    # Define the path to the virtual environment's activation script
+    activate_env = "/home/USERNAME/myenv/bin/activate"
+
+    # Create configuration file with the question
+    _create_config_file(question)
+
+    print('get_tagged_question')
+    
+    # Command to run within the virtual environment
+    command = f"tune run generate --config /tmp/dynamic_generation.yaml"
+    
+    # Full command to activate the environment and run your command
+    full_command = f"source {activate_env} && {command}"
+    
+    # Run the full command in a shell
+    try:
+        result = subprocess.run(full_command, shell=True, check=True, text=True, capture_output=True, executable="/bin/bash")
+        print("Command output:", result.stdout)
+        print("Command error output:", result.stderr)
+
+        output = result.stdout + result.stderr
+        # Extract the input and response using modified regular expressions
+        input_match = re.search(r'Input: (.*?)(?=Response:)', output, re.S)
+        response_match = re.search(r'Response: (.*)', output)
+
+        response_match = response_match.group(1).strip()
+
+        if input_match and response_match:
+            print("Input Question: ", question)
+            print("Extracted Response: ", response_match)
+        else:
+            print("Input or Response not found in the output.")
+        
+    except subprocess.CalledProcessError as e:
+        print("An error occurred:", e.stderr)
+    return response_match
+
+if __name__ == "__main__":
+    # Call the function with a sample question
+    get_tagged_question("Who is the president of largest country in Africa?")
+```
+
+### Step 4: Run the Script
+To run the script and generate tagged questions, execute the following command in your terminal:
+
+```bash
+python command.py
+```
+
+
+
+## How we finetuned the model?
 
-## Model Configuration
+### Model Configuration
 See https://pytorch.org/torchtune/stable/tutorials/e2e_flow.html to know how to use torchtune.
 
 To finetune the model: