license: mit
library_name: transformers
datasets:
- Severian/Internal-Knowledge-Map
pipeline_tag: text-generation
New Fixed Version with extended training available now!
This model is the second trained with experimental 'Internal Knowledge Map' dataset. Developed with an aim to go beyond the scope of usual data processing capabilities, this model gets trained to build comprehensive understanding and reasoning in a wide range of knowledge domains with elaborate guidelines. It bases its reasoning on a specially selected dataset emphasizing the interrelations of the diverse disciplines which aim to synthesize, integrate, and apply complex information in ways that mimic humanly abstract reasoning and creative thought processes.
At the very core of the development of this model is the desire to make sure that LLMs engage in a kind of cognitive activity not limited to memory but actually taking on abstract reasoning, problem-solving, and generation of new insights. To achieve this, 'Nexus-IKM-Mistral-7B' has been fine-tuned until convergance at ~15 Epochs on this unique dataset, which resulted in the model demonstrating greater capability for giving rise to insights and problem-solving in complex, multi-disciplinary settings. This involves improved ability in drawing links between different pieces of knowledge, reasoning through complex scenarios, and proposing innovative solutions that cut across various domains, including science, technology, environmental studies, and humanities.
Test this out and see if you find anything interesting or intriguing. I will keep iterating more versions but this one seems like a fun and useful way to start.
If you'd like to train your own version, here is the full notebook to recreate the training on Unsloth yourself (https://colab.research.google.com/drive/1828t77iO2nLRXVfB8HoI11eFu-79-Oe7?usp=sharing). You'll just have to drop in the train.jsonl from the Dataset repo (https://huggingface.co/datasets/Severian/Internal-Knowledge-Map) into your Colab directory and rename it dataset.jsonl
Training Snapshot
Step Training Loss
1 3.223000
2 3.221300
3 3.215900
4 3.210600
5 3.203000
6 3.193500
7 3.184000
8 3.173400
9 3.162400
10 3.151500
11 3.140500
12 3.128800
13 3.117600
14 3.106700
15 3.095500
16 3.084700
17 3.073700
18 3.062700
19 3.052300
20 3.041800
201 1.273200
202 1.257600
203 1.241900
204 1.226100
205 1.210800
206 1.195500
207 1.180800
208 1.166000
209 1.151200
210 1.136900
211 1.122000
212 1.106600
213 1.091200
214 1.075200
215 1.059200
216 1.042900
217 1.026600
218 1.010300
219 0.994200
416 0.041700
417 0.041700
418 0.041600
419 0.041600
420 0.041600
421 0.041600
422 0.041500
423 0.041500
424 0.041500
425 0.041400
426 0.041400
427 0.041400
428 0.041400
429 0.041300
430 0.041300
431 0.041300
432 0.041200
433 0.041200
434 0.041200
435 0.041100
436 0.041200
437 0.041100
438 0.041100
439 0.041100
440 0.041000
441 0.041000
442 0.041000
443 0.040900
444 0.040900
445 0.040900
668 0.035200
669 0.035100
670 0.035100
671 0.035100
672 0.035100
673 0.035000
674 0.035000
675 0.035000
676 0.035000
677 0.034900
678 0.034900
679 0.034900
680 0.034800
681 0.034800
682 0.034800
683 0.034800
684 0.034800
685 0.034700
686 0.034700
687 0.034700
688 0.034700
689 0.034600
690 0.034600
691 0.034600
692 0.034600
693 0.034500
694 0.034500
695 0.034500
696 0.034400
697 0.034400
698 0.034400
699 0.034400
700 0.034300
701 0.034300
702 0.034300
703 0.034300
704 0.034200
705 0.034200
706 0.034200
707 0.034200
708 0.034100
709 0.034100
710 0.034100
711 0.034100
712 0.034000
713 0.034000
714 0.034000
715 0.034000
716 0.033900
717 0.033900
718 0.033800
719 0.033800
720 0.033800
721 0.033800
1209 0.006600
1210 0.006500
1211 0.006300
1212 0.006200
1213 0.006100
1214 0.006000
1215 0.005800
1216 0.005700
1217 0.005600
1218 0.005500
1219 0.005400
1220 0.005300
1221 0.005100
1222 0.004900
1223 0.004800
1224 0.004700
1225 0.004600
1226 0.004500
1227 0.004400
1228 0.004300
1229 0.004200
1230 0.004000
1231 0.003900
1232 0.003800
1233 0.003700
1234 0.003500
1235 0.003400
1236 0.003300
1237 0.003200
1238 0.003000
1239 0.003000
1240 0.002900
1241 0.002800
1242 0.002700
1243 0.002600
1244 0.002500
1245 0.002400
1246 0.002300
1247 0.002200
1248 0.002100
1249 0.002000
1250 0.001900
1251 0.001800
1252 0.001800
1253 0.001700
1254 0.001600
1255 0.001600
1256 0.001500
1257 0.001400
1258 0.001300
1259 0.001300
1260 0.001200
1261 0.001200
1262 0.001100
1263 0.001100
1264 0.001000
1265 0.001000
1266 0.000900
1267 0.000900
1268 0.000800
1269 0.000800
1270 0.000800
1271 0.000800
1272 0.000700
1273 0.000700
1274 0.000700
1275 0.000600
1276 0.000600
1277 0.000600
1278 0.000600
1279 0.000500
1280 0.000500
1281 0.000500
1282 0.000500
1283 0.000500
1284 0.000500
1285 0.000500
1286 0.000400
1287 0.000400
1288 0.000400
1289 0.000400
1290 0.000400
1291 0.000400
1292 0.000400
1293 0.000400
1294 0.000400
1295 0.000400
1296 0.000400
1297 0.000300
1298 0.000300