reaperdoesntknow commited on
Commit
f01a0d6
Β·
verified Β·
1 Parent(s): c7f78cf

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +113 -155
README.md CHANGED
@@ -1,201 +1,159 @@
1
  ---
 
2
  library_name: transformers
 
3
  tags:
4
- - trl
5
  - sft
 
 
 
 
 
6
  ---
7
 
8
- # Model Card for Model ID
9
-
10
- <!-- Provide a quick summary of what the model is/does. -->
11
-
12
-
13
-
14
- ## Model Details
15
-
16
- ### Model Description
17
-
18
- <!-- Provide a longer summary of what this model is. -->
19
-
20
- This is the model card of a πŸ€— transformers model that has been pushed on the Hub. This model card has been automatically generated.
21
-
22
- - **Developed by:** [More Information Needed]
23
- - **Funded by [optional]:** [More Information Needed]
24
- - **Shared by [optional]:** [More Information Needed]
25
- - **Model type:** [More Information Needed]
26
- - **Language(s) (NLP):** [More Information Needed]
27
- - **License:** [More Information Needed]
28
- - **Finetuned from model [optional]:** [More Information Needed]
29
-
30
- ### Model Sources [optional]
31
-
32
- <!-- Provide the basic links for the model. -->
33
-
34
- - **Repository:** [More Information Needed]
35
- - **Paper [optional]:** [More Information Needed]
36
- - **Demo [optional]:** [More Information Needed]
37
-
38
- ## Uses
39
-
40
- <!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
41
-
42
- ### Direct Use
43
-
44
- <!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
45
-
46
- [More Information Needed]
47
-
48
- ### Downstream Use [optional]
49
-
50
- <!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
51
-
52
- [More Information Needed]
53
-
54
- ### Out-of-Scope Use
55
-
56
- <!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
57
-
58
- [More Information Needed]
59
-
60
- ## Bias, Risks, and Limitations
61
-
62
- <!-- This section is meant to convey both technical and sociotechnical limitations. -->
63
-
64
- [More Information Needed]
65
-
66
- ### Recommendations
67
-
68
- <!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
69
 
70
- Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
71
 
72
- ## How to Get Started with the Model
73
 
74
- Use the code below to get started with the model.
75
-
76
- [More Information Needed]
77
-
78
- ## Training Details
79
-
80
- ### Training Data
81
-
82
- <!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
83
-
84
- [More Information Needed]
85
-
86
- ### Training Procedure
87
-
88
- <!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
89
-
90
- #### Preprocessing [optional]
91
-
92
- [More Information Needed]
93
-
94
-
95
- #### Training Hyperparameters
96
-
97
- - **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
98
-
99
- #### Speeds, Sizes, Times [optional]
100
-
101
- <!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
102
-
103
- [More Information Needed]
104
-
105
- ## Evaluation
106
-
107
- <!-- This section describes the evaluation protocols and provides the results. -->
108
-
109
- ### Testing Data, Factors & Metrics
110
-
111
- #### Testing Data
112
-
113
- <!-- This should link to a Dataset Card if possible. -->
114
-
115
- [More Information Needed]
116
-
117
- #### Factors
118
-
119
- <!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
120
-
121
- [More Information Needed]
122
-
123
- #### Metrics
124
-
125
- <!-- These are the evaluation metrics being used, ideally with a description of why. -->
126
 
127
- [More Information Needed]
128
 
129
- ### Results
130
 
131
- [More Information Needed]
 
 
132
 
133
- #### Summary
134
 
 
135
 
 
 
 
 
 
 
 
 
 
136
 
137
- ## Model Examination [optional]
138
 
139
- <!-- Relevant interpretability work for the model goes here -->
140
 
141
- [More Information Needed]
142
 
143
- ## Environmental Impact
 
 
 
144
 
145
- <!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
146
 
147
- Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
148
 
149
- - **Hardware Type:** [More Information Needed]
150
- - **Hours used:** [More Information Needed]
151
- - **Cloud Provider:** [More Information Needed]
152
- - **Compute Region:** [More Information Needed]
153
- - **Carbon Emitted:** [More Information Needed]
154
 
155
- ## Technical Specifications [optional]
156
 
157
- ### Model Architecture and Objective
 
158
 
159
- [More Information Needed]
 
 
 
 
 
160
 
161
- ### Compute Infrastructure
 
 
 
 
 
162
 
163
- [More Information Needed]
 
 
 
 
 
 
 
 
 
 
 
164
 
165
- #### Hardware
166
 
167
- [More Information Needed]
 
 
 
168
 
169
- #### Software
 
 
170
 
171
- [More Information Needed]
 
 
 
172
 
173
- ## Citation [optional]
174
 
175
- <!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
176
 
177
- **BibTeX:**
 
 
178
 
179
- [More Information Needed]
180
 
181
- **APA:**
182
 
183
- [More Information Needed]
 
 
 
 
 
184
 
185
- ## Glossary [optional]
186
 
187
- <!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
 
 
 
 
188
 
189
- [More Information Needed]
190
 
191
- ## More Information [optional]
192
 
193
- [More Information Needed]
194
 
195
- ## Model Card Authors [optional]
196
 
197
- [More Information Needed]
 
 
 
 
 
 
 
 
 
198
 
199
- ## Model Card Contact
200
 
201
- [More Information Needed]
 
 
1
  ---
2
+ license: apache-2.0
3
  library_name: transformers
4
+ pipeline_tag: text-generation
5
  tags:
6
+ - qwen3
7
  - sft
8
+ - trl
9
+ - dual-mind
10
+ - reasoning
11
+ - convergent-intelligence
12
+ - explore-examine-response
13
  ---
14
 
15
+ # DualMind
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
16
 
17
+ **Single Architecture, Dual Cognition β€” The Multi-Model Collision Array on Shared Weights**
18
 
19
+ *Convergent Intelligence LLC: Research Division*
20
 
21
+ ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
22
 
23
+ ## What This Is
24
 
25
+ DualMind is a 1.7B parameter model that implements **dual-mental-modality reasoning** β€” a single model with two internal voices sharing the same weights, differentiated only by role tokens:
26
 
27
+ - **`<explore>`** β€” Unconstrained reasoning. Derivation, speculation, working through the problem freely.
28
+ - **`<examine>`** β€” Adversarial self-response. The model reads its own explore output and critiques it. Error detection, verification, refinement.
29
+ - **`<response>`** β€” Clean synthesis. The final answer distilled from the internal dialogue.
30
 
31
+ This is the multi-model collision array collapsed into a single architecture. The dialectical structure that produces novel insights from architectural diversity (demonstrated in our [five-architecture collision experiments](https://huggingface.co/reaperdoesntknow)) is recreated through role-conditioned generation on shared weights.
32
 
33
+ ## Architecture
34
 
35
+ | Parameter | Value |
36
+ |-----------|-------|
37
+ | Architecture | Qwen3ForCausalLM |
38
+ | Parameters | ~2.03B (1.7B effective) |
39
+ | Hidden Size | 2048 |
40
+ | Layers | 28 |
41
+ | Attention Heads | 16 (Q) / 8 (KV) β€” GQA |
42
+ | Context Length | 40,960 tokens |
43
+ | Precision | BF16 (trained on H100) |
44
 
45
+ ## Training
46
 
47
+ **Base model:** [Disctil-Qwen3-1.7B](https://huggingface.co/reaperdoesntknow/Disctil-Qwen3-1.7B) (DISC-refined uncensored Qwen3)
48
 
49
+ **Dataset:** [KK04/LogicInference_OA](https://huggingface.co/datasets/KK04/LogicInference_OA) β€” Logical inference problems transformed into the DualMind cognitive loop format.
50
 
51
+ **Training format:** Each CoT solution is restructured into the DualMind format:
52
+ - Derivation sentences β†’ `<explore>` block (reasoning phase)
53
+ - Verification/checking sentences β†’ `<examine>` block (self-critique phase)
54
+ - Final answer β†’ `<response>` block (synthesis)
55
 
56
+ Sentence-level splitting uses trigger detection (check, verify, however, but wait, etc.) to find the natural transition from reasoning to verification, with 70/30 positional fallback.
57
 
58
+ **Hardware:** Colab H100, BF16 precision. 512 steps, lr 5e-6, SFT via TRL.
59
 
60
+ **Next iteration:** Currently training on [Crownelius/Opus-4.6-Reasoning-3300x](https://huggingface.co/datasets/Crownelius/Opus-4.6-Reasoning-3300x) β€” 2,160 Claude Opus 4.6 reasoning samples with pre-separated `thinking`/`solution` columns, eliminating the need for heuristic splitting.
 
 
 
 
61
 
62
+ ## Usage
63
 
64
+ ```python
65
+ from transformers import AutoModelForCausalLM, AutoTokenizer
66
 
67
+ model = AutoModelForCausalLM.from_pretrained(
68
+ "reaperdoesntknow/DualMind",
69
+ torch_dtype="auto",
70
+ device_map="auto"
71
+ )
72
+ tokenizer = AutoTokenizer.from_pretrained("reaperdoesntknow/DualMind")
73
 
74
+ # Start the explore block β€” the model completes the full loop
75
+ prompt = (
76
+ "##USER:\n"
77
+ "Prove that the sum of two even numbers is always even.\n\n"
78
+ "<explore>\n"
79
+ )
80
 
81
+ inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
82
+ output = model.generate(
83
+ **inputs,
84
+ max_new_tokens=1024,
85
+ do_sample=True,
86
+ top_p=0.9,
87
+ temperature=0.6,
88
+ repetition_penalty=1.15,
89
+ )
90
+ result = tokenizer.decode(output[0], skip_special_tokens=True)
91
+ print(result)
92
+ ```
93
 
94
+ ### Expected Output Structure
95
 
96
+ ```
97
+ <explore>
98
+ [The model works through the proof freely β€” definitions, algebraic manipulation, etc.]
99
+ </explore>
100
 
101
+ <examine>
102
+ [The model critiques its own derivation β€” checks for gaps, verifies steps, catches errors]
103
+ </examine>
104
 
105
+ <response>
106
+ [Clean final answer synthesized from the internal dialogue]
107
+ </response>
108
+ ```
109
 
110
+ ## Why Dual Modality
111
 
112
+ Standard CoT prompting produces a single stream of reasoning. The model has one shot to get it right. DualMind gives the model a structural mechanism for self-correction:
113
 
114
+ 1. **Explore** is free to make mistakes, speculate, and try approaches that might not work
115
+ 2. **Examine** reads the explore output adversarially β€” it's looking for errors, not confirming correctness
116
+ 3. **Response** has the benefit of both perspectives
117
 
118
+ This mirrors what happens in multi-model collision arrays where different architectures produce genuinely different failure modes, and the collision between them surfaces structure that neither achieves alone. DualMind recreates this dynamic within a single set of weights through role conditioning.
119
 
120
+ ## Distillation Chain
121
 
122
+ ```
123
+ Qwen3-1.7B (base)
124
+ β†’ DiStil-Qwen3-1.7B-uncensored (uncensored SFT)
125
+ β†’ Disctil-Qwen3-1.7B (DISC refinement)
126
+ β†’ DualMind (DualMind SFT on Opus 4.6 reasoning data) ← you are here
127
+ ```
128
 
129
+ ## Related Models
130
 
131
+ | Model | Description | Downloads |
132
+ |-------|-------------|-----------|
133
+ | [TopologicalQwen](https://huggingface.co/reaperdoesntknow/TopologicalQwen) | TKD + DualMind on physics CoT | 622 |
134
+ | [Disctil-Qwen3-1.7B](https://huggingface.co/reaperdoesntknow/Disctil-Qwen3-1.7B) | Parent model (DISC-refined) | 286 |
135
+ | [Qwen3-1.7B-Thinking-Distil](https://huggingface.co/reaperdoesntknow/Qwen3-1.7B-Thinking-Distil) | TKD with Thinking teacher | 687 |
136
 
137
+ **[DualMind Collection](https://huggingface.co/collections/reaperdoesntknow/dualmind)** β€” Dual-cognition model series
138
 
139
+ **[DistilQwen Collection](https://huggingface.co/collections/reaperdoesntknow/distilqwen-69bf40ec669117e3f069ef1c)** β€” Full proof-weighted distillation series
140
 
141
+ Full methodology: [Structure Over Scale (DOI: 10.57967/hf/8165)](https://doi.org/10.57967/hf/8165)
142
 
143
+ ## Citation
144
 
145
+ ```bibtex
146
+ @misc{colca2026dualmind,
147
+ title={DualMind: Dual-Mental-Modality Reasoning via Role-Conditioned Self-Critique},
148
+ author={Colca, Roy S.},
149
+ year={2026},
150
+ publisher={HuggingFace},
151
+ url={https://huggingface.co/reaperdoesntknow/DualMind},
152
+ note={Convergent Intelligence LLC: Research Division}
153
+ }
154
+ ```
155
 
156
+ ---
157
 
158
+ *Convergent Intelligence LLC: Research Division*
159
+ *"Where classical analysis fails to see, we begin."*