tlphams commited on
Commit
ccba7cb
1 Parent(s): d8df62a

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +13 -12
README.md CHANGED
@@ -12,19 +12,19 @@ tags:
12
  The ZoyLLM-7B-SlimOrca Large Language Model (LLM) is a LoRA-finetuned generative text model with Mistral-7B-v0.1 is the base model.
13
  Mistral-7B-v0.1 outperforms Llama 2 13B on all benchmarks we tested.
14
 
15
- ## Model Architecture
16
 
17
  ZoyLLM-7B-SlimOrca is a transformer model, with the following architecture choices:
18
  - Grouped-Query Attention
19
  - Sliding-Window Attention
20
  - Byte-fallback BPE tokenizer
21
 
22
- ## Datasets
23
  - Self-introduction (20 samples)
24
  - SlimOrca (100k samples random sampled)
25
  - EverythingLM v3
26
 
27
- ## Template
28
  We finetuned the model using a chatml template
29
  ```
30
  <|im_start|>system
@@ -34,28 +34,29 @@ We finetuned the model using a chatml template
34
  <|im_start|>assistant
35
 
36
  ```
37
- ## Test examples
38
- ### Self-introduction
39
  ```
40
  <|im_start|>system
41
  You are a large language model named ZoyLLM.<|im_end|>
42
  <|im_start|>user
43
- Stocks rallied Friday even after the release of stronger-than-expected U.S. jobs data and a major increase in Treasury yields. The Dow Jones Industrial Average gained 195.12 points, or 0.76%, to close at 31,419.58. The S&P 500 added 1.59% at 4,008.50. The tech-heavy Nasdaq Composite rose 1.35%, closing at 12,299.68. The U.S. economy added 438,000 jobs in August, the Labor Department said. Economists polled by Dow Jones expected 273,000 jobs. However, wages rose less than expected last month. Stocks posted a stunning turnaround on Friday, after initially falling on the stronger-than-expected jobs report. At its session low, the Dow had fallen as much as 198 points; it surged by more than 500 points at the height of the rally. The Nasdaq and the S&P 500 slid by 0.8% during their lowest points in the day. Traders were unclear of the reason for the intraday reversal. Some noted it could be the softer wage number in the jobs report that made investors rethink their earlier bearish stance. Others noted the pullback in yields from the day’s highs. Part of the rally may just be to do a market that had gotten extremely oversold with the S&P 500 at one point this week down more than 9% from its high earlier this year. Yields initially surged after the report, with the 10-year Treasury rate trading near its highest level in 14 years. The benchmark rate later eased from those levels, but was still up around 6 basis points at 4.58%. 'We’re seeing a little bit of a give back in yields from where we were around 4.8%. [With] them pulling back a bit, I think that’s helping the stock market,' said Margaret Jones, chief investment officer at Vibrant Industries Capital Advisors. 'We’ve had a lot of weakness in the market in recent weeks, and potentially some oversold conditions.'
44
- Based on above information, answer this question as short as possible: What was the percentage in increase in the Nasdaq at closing?<|im_end|>
45
  <|im_start|>assistant
46
- The Nasdaq Composite rose 1.35% at closing.
47
  ```
48
 
49
- ## RAG testbench sample
50
  ```
51
  <|im_start|>system
52
  You are a large language model named ZoyLLM.<|im_end|>
53
  <|im_start|>user
54
- Who are you?<|im_end|>
 
55
  <|im_start|>assistant
 
56
  ```
57
 
58
- ## Troubleshooting
59
 
60
  - If you see the following error:
61
  ```
@@ -68,6 +69,6 @@ NotImplementedError: Cannot copy out of meta tensor; no data!
68
 
69
  Ensure you are utilizing a stable version of Transformers, 4.34.0 or newer.
70
 
71
- ## The Zoy AI Team
72
 
73
  Pham Tung Lam, Nguyen Duc Nhan.
 
12
  The ZoyLLM-7B-SlimOrca Large Language Model (LLM) is a LoRA-finetuned generative text model with Mistral-7B-v0.1 is the base model.
13
  Mistral-7B-v0.1 outperforms Llama 2 13B on all benchmarks we tested.
14
 
15
+ ## 1. Model Architecture
16
 
17
  ZoyLLM-7B-SlimOrca is a transformer model, with the following architecture choices:
18
  - Grouped-Query Attention
19
  - Sliding-Window Attention
20
  - Byte-fallback BPE tokenizer
21
 
22
+ ## 2. Datasets
23
  - Self-introduction (20 samples)
24
  - SlimOrca (100k samples random sampled)
25
  - EverythingLM v3
26
 
27
+ ## 3. Template
28
  We finetuned the model using a chatml template
29
  ```
30
  <|im_start|>system
 
34
  <|im_start|>assistant
35
 
36
  ```
37
+ ## 4. Test examples
38
+ ### 4.1. Self-introduction
39
  ```
40
  <|im_start|>system
41
  You are a large language model named ZoyLLM.<|im_end|>
42
  <|im_start|>user
43
+ Who are you?<|im_end|>
 
44
  <|im_start|>assistant
45
+ I am ZoyLLM, an AI created by Pham Tung Lam and Nguyen Duc Nhan to be helpful, friendly and to understand humans better.<|im_end|>
46
  ```
47
 
48
+ ## 4.2. RAG testbench sample
49
  ```
50
  <|im_start|>system
51
  You are a large language model named ZoyLLM.<|im_end|>
52
  <|im_start|>user
53
+ Stocks rallied Friday even after the release of stronger-than-expected U.S. jobs data and a major increase in Treasury yields. The Dow Jones Industrial Average gained 195.12 points, or 0.76%, to close at 31,419.58. The S&P 500 added 1.59% at 4,008.50. The tech-heavy Nasdaq Composite rose 1.35%, closing at 12,299.68. The U.S. economy added 438,000 jobs in August, the Labor Department said. Economists polled by Dow Jones expected 273,000 jobs. However, wages rose less than expected last month. Stocks posted a stunning turnaround on Friday, after initially falling on the stronger-than-expected jobs report. At its session low, the Dow had fallen as much as 198 points; it surged by more than 500 points at the height of the rally. The Nasdaq and the S&P 500 slid by 0.8% during their lowest points in the day. Traders were unclear of the reason for the intraday reversal. Some noted it could be the softer wage number in the jobs report that made investors rethink their earlier bearish stance. Others noted the pullback in yields from the day’s highs. Part of the rally may just be to do a market that had gotten extremely oversold with the S&P 500 at one point this week down more than 9% from its high earlier this year. Yields initially surged after the report, with the 10-year Treasury rate trading near its highest level in 14 years. The benchmark rate later eased from those levels, but was still up around 6 basis points at 4.58%. 'We’re seeing a little bit of a give back in yields from where we were around 4.8%. [With] them pulling back a bit, I think that’s helping the stock market,' said Margaret Jones, chief investment officer at Vibrant Industries Capital Advisors. 'We’ve had a lot of weakness in the market in recent weeks, and potentially some oversold conditions.'
54
+ Based on above information, answer this question as short as possible: What was the percentage in increase in the Nasdaq at closing?<|im_end|>
55
  <|im_start|>assistant
56
+ The Nasdaq Composite rose 1.35% at closing.<|im_end|>
57
  ```
58
 
59
+ ## 5. Troubleshooting
60
 
61
  - If you see the following error:
62
  ```
 
69
 
70
  Ensure you are utilizing a stable version of Transformers, 4.34.0 or newer.
71
 
72
+ ## 6. The Zoy AI Team
73
 
74
  Pham Tung Lam, Nguyen Duc Nhan.