Update README.md

README.md

@@ -26,8 +26,8 @@ Memories - Token Compressor for Long-Range Dependency Conversations
 This model is a fine-tuned version of the Llama 3.1 8B 4-bit model, specifically trained for token compression tasks. It uses LoRA (Low-Rank Adaptation) for efficient fine-tuning while maintaining the base model's performance.
 
 - **Developed by:** Alosh Denny
-- **Funded by [optional]:** nil
-- **Shared by [optional]:** nil
+- **Funded by:** EmelinLabs
+- **Shared by** EmelinLabs
 - **Model type:** Token Compressor for Memories
 - **Language(s) (NLP):** English
 - **License:** apache-2.0
@@ -38,7 +38,7 @@ This model is a fine-tuned version of the Llama 3.1 8B 4-bit model, specifically
 
 This model is designed for token compression tasks. It can be used to generate more concise versions of input text while preserving the essential meaning.
 
-### Downstream Use [optional]
+### Downstream Use
 
 The compressed outputs from this model can be used in various NLP applications where text length is a constraint, such as summarization, efficient text storage, or as input for other language models with token limits.
 
@@ -61,7 +61,14 @@ This model should not be used for tasks that require full preservation of the or
 
 Use the code below to get started with the model.
 
-[More Information Needed]
+'''
+from peft import PeftModel, PeftConfig
+from transformers import AutoModelForCausalLM
+
+config = PeftConfig.from_pretrained("aoxo/llama-token-compressor")
+base_model = AutoModelForCausalLM.from_pretrained("unsloth/Meta-Llama-3.1-8B-bnb-4bit")
+model = PeftModel.from_pretrained(base_model, "aoxo/llama-token-compressor")
+'''
 
 ## Training Details
 
@@ -76,14 +83,14 @@ The model was trained on a dataset compiled from various sources, including:
 
 ### Training Procedure
 
+
+
 #### Preprocessing
 
 Prompt-response pairs were processed from the datasets and compiled into a single dataset (available at https://huggingface.co/datasets/aoxo/token_compressor). Unwanted characters, trailing whitespaces and inverted commas were voided.
 
 #### Training Hyperparameters
 
-#### Training Hyperparameters
-
 - **Training regime:** bf16 mixed precision
 - **Optimizer:** paged_adamw_8bit
 - **Learning rate:** 2e-4
@@ -91,7 +98,7 @@ Prompt-response pairs were processed from the datasets and compiled into a singl
 - **Batch size:** 4 per device
 - **Gradient accumulation steps:** 16
 - **Number of epochs:** 10
-- **Max steps:** 700,472
+- **Max steps:** 175,118
 
 #### LoRA Configuration
 
@@ -107,105 +114,93 @@ Prompt-response pairs were processed from the datasets and compiled into a singl
 - **Total Logged Training Time:** 1422.31 hours
 - **Start Time:** 07-21-2024 02:02:32
 - **End Time:** 09-18-2024 08:21:08
-- **Checkpoint Size (adapter):** 13,648,432 bytes
+- **Checkpoint Size (Adapter):** 13,648,432 bytes
 
-## Evaluation
+### Evaluation Data, Factors & Results
 
-<!-- This section describes the evaluation protocols and provides the results. -->
+## Evaluation
 
-### Testing Data, Factors & Metrics
+- **Total Evaluation Compute Throughput:** 14.34 GFLOPS
+- **Total Logged Evaluation Time:** 34.25 minutes
+- **Start Time:** 09-18-2024 08:23:11
+- **End Time:** 09-18-2024 08:57:26
 
-#### Testing Data
+#### Evaluation Data
 
-<!-- This should link to a Dataset Card if possible. -->
+Evaluation was performed on a subset of the following dataset:
 
-[More Information Needed]
+- sentence-transformers/sentence-compression
 
-#### Factors
+### Results
 
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+To demonstrate the model's performance, we've tested it on prompts of varying lengths. The results show how the model compresses texts of different sizes while maintaining the core meaning.
 
-[More Information Needed]
+#### Example 1: Very Large Paragraph
 
-#### Metrics
+**Input:**
+The impact of artificial intelligence on modern society is a topic of intense debate and speculation. As AI technologies continue to advance at an unprecedented pace, they are reshaping industries, transforming job markets, and altering the way we interact with machines and each other. Proponents argue that AI has the potential to solve some of humanity's most pressing challenges, from climate change to disease diagnosis. They envision a future where AI assistants enhance human productivity, autonomous vehicles reduce traffic accidents, and machine learning algorithms make breakthrough discoveries in science and medicine. However, critics warn of potential downsides, including job displacement, privacy concerns, and the ethical implications of delegating important decisions to machines. There are also fears about the long-term consequences of creating superintelligent AI systems that could potentially outstrip human control. As we navigate this complex landscape, it becomes increasingly important to develop robust governance frameworks and ethical guidelines to ensure that AI development aligns with human values and benefits society as a whole.
 
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+**Output:**
+AI's rapid advancement is reshaping society, offering solutions to major challenges but raising concerns about job displacement, privacy, and ethics. Balancing AI's potential with its risks requires careful governance and ethical guidelines.
 
-[More Information Needed]
+#### Example 2: Medium-Length Paragraph
 
-### Results
+**Input:**
+The evolution of social media platforms has dramatically altered the landscape of human communication and information sharing. What began as simple networking sites have grown into complex ecosystems that influence everything from personal relationships to global politics. While social media has enabled unprecedented connectivity and democratized information access, it has also given rise to challenges such as misinformation spread, privacy breaches, and addictive design patterns. As these platforms continue to evolve, there is an ongoing debate about their role in society and the need for regulation to address their impact on mental health, democracy, and social cohesion.
 
-[More Information Needed]
+**Output:**
+Social media has transformed communication, offering connectivity but also causing issues like misinformation and privacy concerns. Its evolving role in society sparks debates on regulation and impact on mental health and democracy.
 
-#### Summary
+#### Example 3: Short Paragraph
 
+**Input:**
+Climate change is one of the most pressing issues of our time, with far-reaching consequences for ecosystems, economies, and human societies worldwide. Rising global temperatures are leading to more frequent extreme weather events, sea level rise, and shifts in wildlife populations and vegetation patterns. Addressing this challenge requires a coordinated global effort to reduce greenhouse gas emissions and transition to sustainable energy sources.
 
+**Output:**
+Climate change, a critical global issue, causes extreme weather, rising seas, and ecosystem shifts. Tackling it needs worldwide cooperation to cut emissions and adopt sustainable energy.
 
-## Model Examination [optional]
+#### Example 4: Brief Statement
 
-<!-- Relevant interpretability work for the model goes here -->
+**Input:**
+The rise of e-commerce has transformed the retail landscape, offering consumers unprecedented convenience and choice while posing challenges for traditional brick-and-mortar stores.
 
-[More Information Needed]
+**Output:**
+E-commerce growth offers consumer convenience, challenging traditional stores.
 
 ## Environmental Impact
 
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-
 Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
 
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
+- **Hardware Type:** RTX 4000 SFF 20GB Ada Generation Graphics Card
+- **Hours used:** 1423
+- **Cloud Provider:** Private Infrastructure
+- **Compute Region:** Kochi, India (Asia Pacific)
+- **Carbon Emitted:** 458.21 kg CO2
 
-## Technical Specifications [optional]
+## Technical Specifications
 
 ### Model Architecture and Objective
 
-[More Information Needed]
+The model uses the Llama 3.1 8B architecture with 4-bit quantization. It was fine-tuned using LoRA for the task of token compression.
 
 ### Compute Infrastructure
 
-[More Information Needed]
-
 #### Hardware
 
-[More Information Needed]
+RTX 4000 SFF 20GB Ada Generation Graphics Card
 
 #### Software
 
-[More Information Needed]
-
-## Citation [optional]
-
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
-
-**BibTeX:**
-
-[More Information Needed]
-
-**APA:**
-
-[More Information Needed]
-
-## Glossary [optional]
-
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-
-[More Information Needed]
-
-## More Information [optional]
-
-[More Information Needed]
-
-## Model Card Authors [optional]
-
-[More Information Needed]
+- Hugging Face Transformers
+- PEFT (Parameter-Efficient Fine-Tuning)
+- Accelerate
+- bitsandbytes
+- TRL (Transformer Reinforcement Learning)
 
 ## Model Card Contact
 
-[More Information Needed]
+aloshdeny@gmail.com
+
 ### Framework versions
 
 - PEFT 0.12.0