Severian commited on
Commit
75bc9b6
1 Parent(s): 4c854fe

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +17 -2
README.md CHANGED
@@ -2,14 +2,29 @@
2
  datasets:
3
  - Severian/Internal-Knowledge-Map
4
  ---
5
- ## Nexus-4x7B-IKM (Work in Progress)
6
 
7
- This model is the first trained with experimental 'Internal Knowledge Map' dataset. Developed with an aim to go beyond the scope of usual data processing capabilities, this model gets trained to build comprehensive understanding and reasoning in a wide range of knowledge domains with elaborate guidelines. It bases its reasoning on a specially selected dataset emphasizing the interrelations of the diverse disciplines which aim to synthesize, integrate, and apply complex information in ways that mimic humanly abstract reasoning and creative thought processes.
 
 
8
 
9
  At the very core of the development of this model is the desire to make sure that LLMs engage in a kind of cognitive activity not limited to memory but actually taking on abstract reasoning, problem-solving, and generation of new insights. To achieve this, 'Nexus-4x7B-IKM-v1' has been fine-tuned until convergance at **10 Epochs** on this unique dataset, which resulted in the model demonstrating greater capability for giving rise to insights and problem-solving in complex, multi-disciplinary settings. This involves improved ability in drawing links between different pieces of knowledge, reasoning through complex scenarios, and proposing innovative solutions that cut across various domains, including science, technology, environmental studies, and humanities.
10
 
11
  Test this out and see if you find anything interesting or intriguing. I will keep iterating more versions but this one seems like a fun and useful way to start.
12
 
 
 
 
 
 
 
 
 
 
 
 
 
 
13
  **Example Output:**
14
 
15
  Question: If a new study suggests that increasing the biodiversity in urban areas can reduce disease transmission rates, what might be the underlying reasons, and how could cities implement changes based on this finding?
 
2
  datasets:
3
  - Severian/Internal-Knowledge-Map
4
  ---
5
+ ## Nexus-4x7B-Internal-Knowledge-Map
6
 
7
+ ![image/webp](https://cdn-uploads.huggingface.co/production/uploads/64740cf7485a7c8e1bd51ac9/2dpPdy--p88-GZBMP7HdB.webp)
8
+
9
+ This model is the first trained with experimental '**Internal Knowledge Map**' dataset. Developed with an aim to go beyond the scope of usual data processing capabilities, this model gets trained to build comprehensive understanding and reasoning in a wide range of knowledge domains with elaborate guidelines. It bases its reasoning on a specially selected dataset emphasizing the interrelations of the diverse disciplines which aim to synthesize, integrate, and apply complex information in ways that mimic humanly abstract reasoning and creative thought processes.
10
 
11
  At the very core of the development of this model is the desire to make sure that LLMs engage in a kind of cognitive activity not limited to memory but actually taking on abstract reasoning, problem-solving, and generation of new insights. To achieve this, 'Nexus-4x7B-IKM-v1' has been fine-tuned until convergance at **10 Epochs** on this unique dataset, which resulted in the model demonstrating greater capability for giving rise to insights and problem-solving in complex, multi-disciplinary settings. This involves improved ability in drawing links between different pieces of knowledge, reasoning through complex scenarios, and proposing innovative solutions that cut across various domains, including science, technology, environmental studies, and humanities.
12
 
13
  Test this out and see if you find anything interesting or intriguing. I will keep iterating more versions but this one seems like a fun and useful way to start.
14
 
15
+ ## Training
16
+ ```
17
+
18
+ key: str = "system", key2: str = "instruction"
19
+
20
+ batch_size=1
21
+ epochs=10 (I let it keep going until it finally converged)
22
+ r=16
23
+ lora_alpha=32
24
+ lora_dropout=0.001
25
+ max_seq_length=4096
26
+ ```
27
+
28
  **Example Output:**
29
 
30
  Question: If a new study suggests that increasing the biodiversity in urban areas can reduce disease transmission rates, what might be the underlying reasons, and how could cities implement changes based on this finding?