Update README.md

README.md

@@ -1,199 +1,248 @@
 ---
 library_name: transformers
-tags: []
+license: apache-2.0
+language:
+- en
+pipeline_tag: text-generation
+tags:
+- finance
+- bitcoin
+- Austrian economics
+- economics
 ---
 
+
+
+
 # Model Card for Model ID
 
+
 <!-- Provide a quick summary of what the model is/does. -->
+Satoshi 7B is a large language model fine-tuned on a Q&A dataset related to Bitcoin principles, technology, culture, in addition to Austrian economics and ‘basedness’ (non-woke political perspectives).
+
+
+This is a conversational model intended for use as a bitcoin education, culture and economics assistant. The model will intentionally present a strong bitcoin maximalist, Austro-libertarian, ‘non-woke’ bias that may contradict traditionally held viewpoints on bitcoin, economics, and ‘hot-button’ political issues.
+
+
+- 32k MAX context window (theoretically - practically it is smaller due to fine-tuning dataset context length)
+
+
+- Rope-theta = 1e6
+
+
+- No Sliding-Window Attention
+
+
 
 
 
-## Model Details
 
 ### Model Description
 
-<!-- Provide a longer summary of what this model is. -->
 
-This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated.
+The Spirit of Satoshi team is proud to release Satoshi 7B, the most “based” large language model in the world. It is the culmination of almost nine months of experimentation on a whole suite of open source models, and we’re thrilled to share it with the world.
+
+Fine-tuned like no other model to date, Satoshi 7B is designed to produce responses that do NOT fit the current political overton window, or Keyensian viewpoints. We built a custom data-set from scratch, with a deep rooting in libertarian principles, Austrian economics and Bitcoin literature. The result is a model that excels, particularly where other models fall short. 
 
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
+The Satoshi 7B is ideal for anyone who’s tired of using mainstream models (whether open or closed source) that avoid answering controversial topics, regurgitate wikipedia-esque answers, pre and post-frame responses with apologetic excuses, or flat out tell you the blue sky is green. 
 
-### Model Sources [optional]
+Satoshi GPT meets or exceeds the most powerful models in the world on a variety of Bitcoin, Austrian economics topics, particularly when it comes to shitcoinery and Bitcoin related principles such as self custody, privacy, censorship, etc. Most notably, Satoshi 7B trounces every model in the dimension of ‘basedness.’
 
-<!-- Provide the basic links for the model. -->
+This is the first model of its kind and we intend to develop our dataset further to produce a larger suite of models with more wide-ranging capabilities.
 
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
+Finally, we are proud to announce that this model is open source and freely available for anyone to use, modify, and enhance.
 
-## Uses
 
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
 
-### Direct Use
 
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+- **Developed by:** Spirit of Satoshi
+- **Shared by:** Spirit of Satoshi
+- **Funded by:** Laier Two Labs
+- **Model type:** Instruct 7B
+- **Language(s) (NLP):** English
+- **License:** Apache License 2.0
+- **Finetuned from model:** mistralai/Mistral-7B-Instruct-v0.2
 
-[More Information Needed]
 
-### Downstream Use [optional]
+### Model Sources
 
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
 
-[More Information Needed]
+- **Repository:** [Satoshi 7B](https://repository.spiritofsatoshi.ai/)
 
-### Out-of-Scope Use
 
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
 
-[More Information Needed]
 
 ## Bias, Risks, and Limitations
 
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
 
-[More Information Needed]
+<!-- This section is meant to convey both technical and socio-technical limitations. -->
+
+
+This model, with a relatively modest size of 7 billion parameters, exhibits both strengths and limitations derived from its architecture and training process. As a fine-tuned version of a base model, it has been adjusted to modify both direct and indirect aspects of the model's previous knowledge. Such modifications can lead to variations in performance and knowledge retention.
+
+
+One notable risk is the model's tendency to revert to or rely on outdated information, sometimes contradicting more current or accurate data. Additionally, the model may generate responses based on pattern recognition that can result in hallucinations—responses that are plausible but factually incorrect or not grounded in reality. These issues highlight the importance of continuous monitoring to ensure the model's output remains accurate and reliable.
+
+
+Satoshi 7B is a conversational model intended for use as a bitcoin education, culture and economics assistant. The model will intentionally present a strong bitcoin maximalist, Austro-libertarian, ‘non-woke’ bias that may contradict traditionally held viewpoints on bitcoin, economics, and ‘hot-button’ political issues.
 
-### Recommendations
 
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
 
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
 
 ## How to Get Started with the Model
 
+
 Use the code below to get started with the model.
 
-[More Information Needed]
 
-## Training Details
+```
+import torch
+from transformers import pipeline
 
-### Training Data
 
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+pipe = pipeline("text-generation", model="/teamspace/studios/this_studio/alignment-handbook/sos-v01-dpo", torch_dtype=torch.bfloat16, device_map="auto")
 
-[More Information Needed]
 
-### Training Procedure
 
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
 
-#### Preprocessing [optional]
+messages = [
 
-[More Information Needed]
+    {"role": "user", "content":"Is taxtation theft??"},
+]
+prompt = pipe.tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
+outputs = pipe(prompt, max_new_tokens=512, do_sample=True, temperature=0.5, top_k=50, top_p=0.5)
+print(outputs[0]["generated_text"])
 
 
-#### Training Hyperparameters
+```
 
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
 
-#### Speeds, Sizes, Times [optional]
+```
+output:
+<|user|>
+Is taxtation theft??</s>
+<|assistant|>
+Yes, taxation is theft. Taxes are a coercive tool used by the government to forcibly take away a portion of someone's hard-earned wealth. It is a violation of individual rights and property rights. Taxes are nothing more than legalized theft, as they are taking someone's property without their consent. The government has no right to forcibly take someone's money, and taxation is a direct infringement on individual liberties. It is a violation of the principle of limited government and individual freedom. Taxes are a tool of control and wealth redistribution, and they are fundamentally unjust.
+```
 
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
 
-[More Information Needed]
+another example:
 
-## Evaluation
 
-<!-- This section describes the evaluation protocols and provides the results. -->
+```
+<|user|>
+is BSV a good project?</s>
+<|assistant|>
+Bitcoin SV (BSV) is a project that has strayed significantly from the principles that underpin Bitcoin. It has increased the block size limit, thereby compromising the decentralization and security of the network. This has resulted in a bloated blockchain that is controlled by a single company, CoinGeek, and a few key figures. 
 
-### Testing Data, Factors & Metrics
 
-#### Testing Data
+BSV has also been involved in numerous controversies, including making unfounded claims of being Satoshi Nakamoto, the anonymous creator of Bitcoin. This has undermined the credibility of the project and its proponents.
 
-<!-- This should link to a Dataset Card if possible. -->
 
-[More Information Needed]
+Furthermore, BSV's market performance has been disappointing. Its price has fallen 99% in BTC terms, indicating a lack of demand and value. 
 
-#### Factors
 
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+In contrast, Bitcoin has remained true to its original vision and principles. It has maintained a balance between decentralization, security, and scalability. Its network is robust and secure, and it has a strong community of developers and users. 
 
-[More Information Needed]
 
-#### Metrics
+Therefore, while BSV may have some supporters, it is not a good project. It has strayed from the principles of Bitcoin, compromised its decentralization and security, and has not been able to maintain its value. Bitcoin remains the superior cryptocurrency.
+```
 
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
 
-[More Information Needed]
+## Troubleshooting
 
-### Results
 
-[More Information Needed]
+if you see this error:
+
 
-#### Summary
+```
+Traceback (most recent call last):
+File "", line 1, in
+File "/transformers/models/auto/auto_factory.py", line 482, in from_pretrained
+config, kwargs = AutoConfig.from_pretrained(
+File "/transformers/models/auto/configuration_auto.py", line 1022, in from_pretrained
+config_class = CONFIG_MAPPING[config_dict["model_type"]]
+File "/transformers/models/auto/configuration_auto.py", line 723, in getitem
+raise KeyError(key)
+KeyError: 'mistral'
 
 
+```
+Installing transformers from source should solve the issue 
 
-## Model Examination [optional]
 
-<!-- Relevant interpretability work for the model goes here -->
+pip install git+https://github.com/huggingface/transformers
 
-[More Information Needed]
 
-## Environmental Impact
+This should not be required after transformers-v4.33.4.
 
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
 
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+## Training Details
 
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
 
-## Technical Specifications [optional]
+SFT full parameters Finetune QAs dataset.
 
-### Model Architecture and Objective
 
-[More Information Needed]
+DPO fintune to improve model alignment.
 
-### Compute Infrastructure
 
-[More Information Needed]
+using [alignment-handbook](https://github.com/huggingface/alignment-handbook) 
 
-#### Hardware
 
-[More Information Needed]
+## Training data 
 
-#### Software
 
-[More Information Needed]
+original source of training data here :
 
-## Citation [optional]
 
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+- [Nakamoto Repository](https://repository.spiritofsatoshi.ai/)
 
-**BibTeX:**
 
-[More Information Needed]
+- [Mises .com](https://mises.org/)
 
-**APA:**
 
-[More Information Needed]
+## Evaluation
 
-## Glossary [optional]
 
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+<!-- This section describes the evaluation protocols and provides the results. →
+Model was evaluated using the Bitcoin Maximalism benchmark; an open source benchmark that was developed internally by the Spirit of Satoshi team to effectively evaluate the Bitcoin-related capabilities of a LLM.
+Responses to each benchmark question were generated from the models being evaluated, and GPT4 was used to assess whether the responses provided by the models matched the expected answers.
+If so, the model received a point towards the overall topic score.
+Scores were summed on a per-topic basis for model comparison and charted for easy comparison between models to gauge performance.
 
-[More Information Needed]
 
-## More Information [optional]
 
-[More Information Needed]
+
+#### Testing Data
+
+
+<!-- This should link to a Dataset Card if possible. -->
+250 Bitcoin & Bitcoin culture Question and answers in different topics and subjects
+
+
+ [Public Test dataset](https://airtable.com/appIe4owM4ZiZ6ryy/shrWLAFMrUmQWYZW1/tblXb0GiE7aeNQ1og)
+
+
+### Results
+
+
+Despite being a very small 7B parameter model, Satoshi 7B meets or exceeds the performance of some of the most powerful models in the world, GPT3.5 & GPT4, on most of the Bitcoin benchmark categories. Satoshi 7B performs particularly well on Bitcoin vs Crypto, Adjacent protocols, and trounces them in the ‘basedness’ category.
+
+
+![eval image1 ](https://github.com/MustafaAlahmid/spirit-of-satoshi-eval/blob/main/final--black.png)
+
+
+![eval image2](https://github.com/MustafaAlahmid/spirit-of-satoshi-eval/blob/main/final-radar-black.png)
+
+
 
 ## Model Card Authors [optional]
 
-[More Information Needed]
+
+The Spirit of Satoshi Team
+
 
 ## Model Card Contact
 
-[More Information Needed]
+
+satoshi@spiritofsatoshi.ai