algorembrant commited on
Commit
d64015b
Β·
verified Β·
1 Parent(s): bfe8887

Upload 2 files

Browse files
Files changed (2) hide show
  1. README.md +5 -3
  2. TopoDevPOC_n39_save.py +592 -0
README.md CHANGED
@@ -1,3 +1,5 @@
1
- ---
2
- license: mit
3
- ---
 
 
 
1
+ On "all 2^39 patterns" β€” any explicit file is infeasible (minimum ~2.75 TB). The correct answer is: the pattern space is [0, 2^39). A stream_all_patterns() generator is included that yields every pattern on-demand with zero memory overhead β€” decode pattern k directly from integer k without storing anything.
2
+
3
+ **In short**: use `TopoDevPOC_n39.py` if you dont want unnessesary file saved on your disk. And use `TopoDevPOC_n39_save.py` if you want to store every pattern on to CSV format with a total of 2.75 TB file.
4
+
5
+ **Tip**: there is no need to store 549755813888 patterns externally since it can be generated on-demand using the generator function.
TopoDevPOC_n39_save.py ADDED
@@ -0,0 +1,592 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ #!/usr/bin/env python3
2
+ """
3
+ TopoDevPOC_n39.py
4
+ Topologically Unique Developing Point of Control Patterns
5
+ Pre-market K-Lines, n = 39 three-minute candlesticks
6
+
7
+ Paper : TopoDevPOC.tex (ConQ Research Team, Continual Quasars)
8
+ Compute: Vectorized NumPy + GPU Torch (T4) + Warp branchless ops
9
+ Architectural patterns from core_engine_v11.py (Hyper-Warp Edition)
10
+
11
+ Outputs (CLI):
12
+ 1. Total combination count for n=39
13
+ 2. Matrix state-transition validation
14
+ 3. 100 random ternary matrices (1Γ—38 each)
15
+ 4. 100 random symbolic sequences (length-38 strings)
16
+ 5. 100 developing_poc charts saved as PNG + ASCII CLI preview
17
+ 6. CSV export β€” bit-packed uint64 pattern_id (39-bit encoding, min bytes)
18
+ 7. Binary export β€” raw .npy uint64 array (fastest machine read)
19
+
20
+ COMPRESSION SCHEME:
21
+ Each pattern encodes into a single uint64 (8 bytes):
22
+ bit 0 : direction (0=Bullish, 1=Bearish)
23
+ bits 1..38 : transitions (1=strict, 0=equality)
24
+ Decode: direction = id & 1; trans[k] = (id >> (k+1)) & 1
25
+ Full matrix + direction reconstructed from one integer in O(1).
26
+ Entire 2^39 space = integers [0, 2^39). No file required for enumeration.
27
+
28
+ Run on Google Colab T4:
29
+ !python TopoDevPOC_n39.py
30
+ """
31
+
32
+ # ── stdlib ────────────────────────────────────────────────────────────────────
33
+ import os, sys, time, math, csv, struct
34
+ import numpy as np
35
+
36
+ # ── matplotlib (non-interactive for Colab CLI) ────────────────────────────────
37
+ import matplotlib
38
+ matplotlib.use('Agg')
39
+ import matplotlib.pyplot as plt
40
+ import matplotlib.ticker as mticker
41
+
42
+ # ── GPU setup (T4 Colab) ─────────────────────────────────────────────────────
43
+ try:
44
+ import torch
45
+ HAS_CUDA = torch.cuda.is_available()
46
+ DEVICE = 'cuda' if HAS_CUDA else 'cpu'
47
+ except ImportError:
48
+ HAS_CUDA = False
49
+ DEVICE = 'cpu'
50
+ torch = None
51
+
52
+ # Optional Warp (core_engine_v11.py pattern) ─────────────────────────────────
53
+ HAS_WARP = False
54
+ try:
55
+ import warp as wp
56
+ wp.init()
57
+ wp.set_module_options({"enable_backward": False, "fast_math": True, "max_unroll": 8})
58
+ HAS_WARP = bool(wp.get_cuda_devices())
59
+ except Exception:
60
+ pass
61
+
62
+ # ─────────────────────────────────────────────────────────────────────────────
63
+ # CONSTANTS (tex Section II)
64
+ # ─────────────────────────────────────────────────────────────────────────────
65
+ N = 39 # pre-market 3-min candles (C_0 … C_{-(n-1)})
66
+ N_TRANS = N - 1 # 38 adjacent-pair relations
67
+ N_SAMPLES = 100
68
+ SEED = int(time.time() * 1000) & 0x7FFFFFFF
69
+
70
+ SEP = "=" * 74
71
+
72
+ # ─────────────────────────────────────────────────────────────────────────────
73
+ # 0. HEADER
74
+ # ─────────────────────────────────────────────────────────────────────────────
75
+ print(SEP)
76
+ print(" TopoDevPOC β€” Developing POC Pattern Enumerator")
77
+ print(f" n = {N} candles | {N_TRANS} transitions | device = {DEVICE}")
78
+ if HAS_CUDA and torch:
79
+ print(f" GPU = {torch.cuda.get_device_name(0)}")
80
+ if HAS_WARP:
81
+ print(f" Warp = enabled (branchless kernel path)")
82
+ print(SEP)
83
+
84
+ # ─────────────────────────────────────────────────────────────────────────────
85
+ # SECTION 1 β€” COMBINATORIAL ENUMERATION (tex Theorem, Sec. III)
86
+ #
87
+ # Bullish: each of (n-1) transitions ∈ {>, =} β†’ 2^(n-1) patterns
88
+ # Bearish: each of (n-1) transitions ∈ {<, =} β†’ 2^(n-1) patterns
89
+ # Total : 2^(n-1) + 2^(n-1) = 2^n (disjoint families)
90
+ # n=39 : 2^39 = 549,755,813,888
91
+ # ─────────────────────────────────────────────────────────────────────────────
92
+ TOTAL = 1 << N # exact Python int (arbitrary precision)
93
+ HALF = 1 << (N - 1)
94
+
95
+ print(f"\n[THEOREM] Total unique developing POC patterns for n={N}")
96
+ print(f" Bullish (non-increasing) : 2^{N-1} = {HALF:,}")
97
+ print(f" Bearish (non-decreasing) : 2^{N-1} = {HALF:,}")
98
+ print(f" Total (2^{N}) : {TOTAL:,}")
99
+
100
+ # ─────────────────────────────────────────────────────────────────────────────
101
+ # SECTION 2 β€” MATRIX STATE-TRANSITION VALIDATION (tex Sec. IV)
102
+ #
103
+ # States : S_0 (equality), S_Β± (strict move)
104
+ # A = [[1,1],[1,1]] (fully-connected 2-state digraph)
105
+ # v_0 = [1,1]^T (both states reachable initially)
106
+ #
107
+ # B_n = 1^T Β· A^(n-2) Β· v_0
108
+ # = 2^(n-3) Β· 1^T Β· A Β· 1 (using A^k = 2^(k-1)Β·A for kβ‰₯1)
109
+ # = 2^(n-3) Β· 4 = 2^(n-1) per direction
110
+ #
111
+ # Implementation: exact Python-int matrix exponentiation (no float rounding)
112
+ # ─────────────────────────────────────────────────────────────────────────────
113
+
114
+ def _mm2(A, B):
115
+ """Exact 2Γ—2 matrix multiply with Python arbitrary-precision ints."""
116
+ return [
117
+ [A[0][0]*B[0][0] + A[0][1]*B[1][0], A[0][0]*B[0][1] + A[0][1]*B[1][1]],
118
+ [A[1][0]*B[0][0] + A[1][1]*B[1][0], A[1][0]*B[0][1] + A[1][1]*B[1][1]],
119
+ ]
120
+
121
+ def _mpow2(M, k):
122
+ """Fast 2Γ—2 matrix power, exact ints, O(log k)."""
123
+ if k == 0: return [[1,0],[0,1]]
124
+ if k == 1: return M
125
+ h = _mpow2(M, k >> 1)
126
+ s = _mm2(h, h)
127
+ return s if (k & 1) == 0 else _mm2(s, M)
128
+
129
+ A_mat = [[1,1],[1,1]]
130
+ v0 = [1, 1]
131
+ A_pow = _mpow2(A_mat, N - 2)
132
+ # 1^T Β· A^(n-2) Β· v0
133
+ Av0_0 = A_pow[0][0]*v0[0] + A_pow[0][1]*v0[1]
134
+ Av0_1 = A_pow[1][0]*v0[0] + A_pow[1][1]*v0[1]
135
+ B_n = Av0_0 + Av0_1 # = 1^T Β· (A^(n-2)Β·v0)
136
+
137
+ print(f"\n[MATRIX] A = [[1,1],[1,1]] | v_0 = [1,1]^T")
138
+ print(f" A^{N-2} = [[{A_pow[0][0]}, {A_pow[0][1]}],")
139
+ print(f" [{A_pow[1][0]}, {A_pow[1][1]}]]")
140
+ print(f" B_{N} = 1^T Β· A^{N-2} Β· v_0 = {B_n:,}")
141
+ print(f" Expected 2^{{n-1}} = {HALF:,}")
142
+ assert B_n == HALF, f"Matrix B_n mismatch: {B_n} β‰  {HALF}"
143
+ assert 2 * B_n == TOTAL, f"Total mismatch: {2*B_n} β‰  {TOTAL}"
144
+ print(f" [OK] 2 Γ— {B_n:,} = {TOTAL:,} βœ“")
145
+
146
+ # ─────────────────────────────────────────────────────────────────────────────
147
+ # SECTION 3 β€” PATTERN ENCODING SCHEME
148
+ #
149
+ # Each of the 2^39 patterns maps bijectively to a 39-bit integer:
150
+ # bit 0 : direction (0 = Bullish, 1 = Bearish)
151
+ # bits 1 … N-1 : transitions (1 = strict move, 0 = equality)
152
+ #
153
+ # pattern_id ∈ [0, 2^39) uniquely identifies every valid pattern.
154
+ #
155
+ # Ternary matrix m ∈ {+1,0}^(n-1) for bullish,
156
+ # m ∈ {-1,0}^(n-1) for bearish.
157
+ # Bijection: m_k = sign(direction) Γ— trans_bit_k
158
+ # ─────────────────────────────────────────────────────────────────────────────
159
+
160
+ # ─────────────────────────────────────────────────────────────────────────────
161
+ # SECTION 4 β€” GPU-ACCELERATED RANDOM SAMPLING
162
+ # Generate (N_SAMPLES Γ— N) binary matrix at once on T4 GPU.
163
+ # Inspired by core_engine_v11.py XOR-shift PRNG kernel design.
164
+ # bits[:,0] = direction flags
165
+ # bits[:,1:] = N_TRANS transition flags per pattern
166
+ # ─────────────────────────────────────────────────────────────────────────────
167
+
168
+ t_sample = time.perf_counter()
169
+
170
+ if HAS_WARP:
171
+ # ── Warp branchless path (core_engine_v11.py style) ─────────────────────
172
+ # Launch one thread per sample; XOR-shift fills N bits per thread.
173
+ _seed_wp = SEED
174
+
175
+ @wp.kernel
176
+ def k_rng_bits(seed: int, N_cols: int,
177
+ out: wp.array(dtype=wp.int8)):
178
+ tid = wp.tid()
179
+ rng = wp.uint32(seed) ^ wp.uint32(tid)
180
+ if rng == wp.uint32(0):
181
+ rng = wp.uint32(123456789)
182
+ base = tid * N_cols
183
+ for col in range(N_cols):
184
+ rng = rng ^ (rng << wp.uint32(13))
185
+ rng = rng ^ (rng >> wp.uint32(17))
186
+ rng = rng ^ (rng << wp.uint32(5))
187
+ out[base + col] = wp.int8(int(rng) & 1)
188
+
189
+ out_wp = wp.zeros(N_SAMPLES * N, dtype=wp.int8, device='cuda')
190
+ wp.launch(k_rng_bits, dim=N_SAMPLES, block_dim=128,
191
+ inputs=[_seed_wp, N, out_wp], device='cuda')
192
+ wp.synchronize()
193
+ bits_np = out_wp.numpy().reshape(N_SAMPLES, N)
194
+ print(f"\n[SAMPLE] {N_SAMPLES} patterns sampled via Warp XOR-shift kernel")
195
+
196
+ elif HAS_CUDA and torch is not None:
197
+ # ── Torch GPU path ───────────────────────────────────────────────────────
198
+ gen = torch.Generator(device='cuda')
199
+ gen.manual_seed(SEED)
200
+ bits_t = torch.randint(0, 2, (N_SAMPLES, N), device='cuda',
201
+ generator=gen, dtype=torch.int8)
202
+ bits_np = bits_t.cpu().numpy()
203
+ print(f"\n[SAMPLE] {N_SAMPLES} patterns sampled on GPU (torch.randint)")
204
+
205
+ else:
206
+ # ── CPU NumPy fallback ────────────────────────────────────────────────────
207
+ rng_cpu = np.random.default_rng(SEED)
208
+ bits_np = rng_cpu.integers(0, 2, size=(N_SAMPLES, N), dtype=np.int8)
209
+ print(f"\n[SAMPLE] {N_SAMPLES} patterns sampled on CPU (NumPy)")
210
+
211
+ sample_ms = (time.perf_counter() - t_sample) * 1e3
212
+ print(f" Sampling time: {sample_ms:.2f} ms")
213
+
214
+ # ─────────────────────────────────────────────────────────────────────────────
215
+ # SECTION 5 β€” VECTORISED DECODING (NumPy, no Python loops over patterns)
216
+ #
217
+ # bits[:,0] β†’ direction array (0=Bullish, 1=Bearish), shape (100,)
218
+ # bits[:,1:] β†’ trans array, shape (100, 38)
219
+ # ternary_mat: +trans if Bullish, -trans if Bearish β†’ shape (100, 38)
220
+ # sign: Bullish→+1, Bearish→-1, broadcast multiply
221
+ # ─────────────────────────────────────────────────────────────────────────────
222
+
223
+ dir_bits = bits_np[:, 0].astype(np.int8) # 0 or 1
224
+ trans = bits_np[:, 1:].astype(np.int8) # (100,38), values 0 or 1
225
+ signs = 1 - 2 * dir_bits # 0β†’+1 (bull), 1β†’-1 (bear)
226
+ ternary = (signs[:, None] * trans).astype(np.int8) # (100,38): +1/0/-1
227
+
228
+ # Compact 39-bit pattern IDs
229
+ pows64 = (np.uint64(1) << np.arange(N, dtype=np.uint64))
230
+ pat_ids = (bits_np.astype(np.uint64) * pows64[None, :]).sum(axis=1)
231
+
232
+ # Symbolic sequences: vectorised char lookup
233
+ # Bullish (sign=+1): trans=1 β†’ '>', trans=0 β†’ '='
234
+ # Bearish (sign=-1): trans=1 β†’ '<', trans=0 β†’ '='
235
+ SYM_BULL = np.array(['=', '>'], dtype='<U1') # index by trans bit
236
+ SYM_BEAR = np.array(['=', '<'], dtype='<U1')
237
+ sym_matrix = np.where(dir_bits[:, None] == 0,
238
+ SYM_BULL[trans],
239
+ SYM_BEAR[trans]) # (100, 38)
240
+
241
+ # POC price sequence (right-to-left: p[0]=C_0=newest, p[N-1]=C_{-(N-1)}=oldest)
242
+ # p_raw[i, k] = POC of candle C_{-k} for sample i
243
+ # Transition k: p[k] = p[k+1] + ternary[k]
244
+ # Build by cumsum from oldest→newest: p_raw[:, N-1] = 50, then forward
245
+ BASE = 50.0
246
+ step = 1.0
247
+ p_raw = np.zeros((N_SAMPLES, N), dtype=np.float32)
248
+ p_raw[:, N-1] = BASE
249
+ # Vectorised: cumulative sum of ternary (columns N-2 down to 0)
250
+ for k in range(N-2, -1, -1):
251
+ p_raw[:, k] = p_raw[:, k+1] + ternary[:, k].astype(np.float32) * step
252
+
253
+ # Display order: oldest(left) β†’ newest(right)
254
+ # poc_disp[:, j] = p_raw[:, N-1-j]
255
+ poc_disp = p_raw[:, ::-1].copy() # (100, 39), column 0=oldest, col N-1=newest
256
+
257
+ # ─────────────────────────────────────────────────────────────────────────────
258
+ # OUTPUT 1 β€” TERNARY MATRICES (100 Γ— 38)
259
+ # ─────────────────────────────────────────────────────────────────────────────
260
+ print(f"\n{SEP}")
261
+ print(f" OUTPUT 1 β€” TERNARY MATRICES (1Γ—{N_TRANS}, values ∈ {{-1,0,+1}})")
262
+ print(f" Format: [#] Direction | PatternID | M = [m_0 … m_37]")
263
+ print(SEP)
264
+
265
+ for i in range(N_SAMPLES):
266
+ d_label = "Bullish" if dir_bits[i] == 0 else "Bearish"
267
+ pid = int(pat_ids[i])
268
+ row_str = np.array2string(ternary[i], separator=',',
269
+ max_line_width=400).replace('\n','')
270
+ print(f" [{i+1:3d}] {d_label:7s} | ID={pid:>15d} | M={row_str}")
271
+
272
+ # ─────────────────────────────────────────────────────────────────────────────
273
+ # OUTPUT 2 β€” SYMBOLIC SEQUENCES (length 38)
274
+ # ─────────────────────────────────────────────────────────────────────────────
275
+ print(f"\n{SEP}")
276
+ print(f" OUTPUT 2 β€” SYMBOLIC SEQUENCES (Ξ£, length {N_TRANS})")
277
+ print(f" Format: [#] Direction | PatternID | Ξ£ = Οƒ_0 Οƒ_1 … Οƒ_37")
278
+ print(SEP)
279
+
280
+ for i in range(N_SAMPLES):
281
+ d_label = "Bullish" if dir_bits[i] == 0 else "Bearish"
282
+ pid = int(pat_ids[i])
283
+ seq_str = ' '.join(sym_matrix[i])
284
+ print(f" [{i+1:3d}] {d_label:7s} | ID={pid:>15d} | Ξ£ = {seq_str}")
285
+
286
+ # ─────────────────────────────────────────────────────────────────────────────
287
+ # OUTPUT 3 β€” CHARTS
288
+ # (a) Full matplotlib figure: 10Γ—10 grid, saved to PNG
289
+ # (b) ASCII CLI preview for first 10 patterns
290
+ # ─────────────────────────────────────────────────────────────────────────────
291
+ print(f"\n{SEP}")
292
+ print(f" OUTPUT 3 β€” DEVELOPING POC CHARTS (100 patterns)")
293
+ print(SEP)
294
+
295
+ # ── x-axis: position 0=oldest C_{-(N-1)}, N-1=newest C_0 ──────────────────
296
+ x_pos = np.arange(N, dtype=np.float32)
297
+ x_tick_pos = [0, 9, 19, 29, N-1]
298
+ x_tick_lbl = [f'C_{{-{N-1}}}', f'C_{{-29}}', f'C_{{-19}}', f'C_{{-9}}', 'C_0']
299
+
300
+ ROWS, COLS = 10, 10
301
+ fig = plt.figure(figsize=(COLS * 3.2, ROWS * 2.0))
302
+ fig.suptitle(
303
+ f"TopoDevPOC β€” 100 Random Developing POC Patterns "
304
+ f"n={N} pre-market 3-min K-lines\n"
305
+ f"Total pattern space: 2^{N} = {TOTAL:,} "
306
+ f"(Bullish: {HALF:,} | Bearish: {HALF:,})",
307
+ fontsize=10, y=1.005
308
+ )
309
+
310
+ for i in range(N_SAMPLES):
311
+ ax = fig.add_subplot(ROWS, COLS, i + 1)
312
+ poc = poc_disp[i]
313
+ is_bull = (dir_bits[i] == 0)
314
+ color = '#1a6eb5' if is_bull else '#c0392b'
315
+ label = 'B↑' if is_bull else 'B↓'
316
+
317
+ # Background shade
318
+ ax.set_facecolor('#f7f9fc' if is_bull else '#fdf4f4')
319
+
320
+ # POC line
321
+ ax.plot(x_pos, poc, color=color, linewidth=1.2, zorder=3)
322
+
323
+ # Mark strict-move positions (non-zero ternary in display coords)
324
+ # Transition k corresponds to display segment [N-2-k, N-1-k]
325
+ # Highlight the newer-side node at display index N-1-k = N-1-k
326
+ strict_k = np.where(ternary[i] != 0)[0] # transition indices
327
+ strict_disp = (N - 1 - strict_k).astype(int) # display x-positions
328
+ if strict_disp.size > 0:
329
+ ax.scatter(strict_disp, poc[strict_disp],
330
+ color=color, s=5, zorder=5, linewidths=0)
331
+
332
+ # Flat segments (equality)
333
+ flat_k = np.where(ternary[i] == 0)[0]
334
+ flat_disp = (N - 1 - flat_k).astype(int)
335
+ if flat_disp.size > 0:
336
+ ax.scatter(flat_disp, poc[flat_disp],
337
+ color='gray', s=3, zorder=4, linewidths=0, alpha=0.5)
338
+
339
+ n_strict = int(np.abs(ternary[i]).sum())
340
+ pid_short = int(pat_ids[i]) % 10**9 # last 9 digits for readability
341
+ ax.set_title(f"#{i+1} {label} mv={n_strict} …{pid_short:09d}",
342
+ fontsize=5.5, pad=2, color=color)
343
+
344
+ ax.set_xlim(-0.5, N - 0.5)
345
+ ax.set_xticks(x_tick_pos)
346
+ ax.set_xticklabels(['←old', '', '', '', 'newβ†’'], fontsize=3.5)
347
+ ax.tick_params(axis='y', labelsize=3.5)
348
+ ax.yaxis.set_major_locator(mticker.MaxNLocator(4))
349
+ for sp in ('top', 'right'):
350
+ ax.spines[sp].set_visible(False)
351
+ ax.spines['left'].set_color(color)
352
+ ax.spines['left'].set_linewidth(1.5)
353
+ ax.spines['bottom'].set_color('#cccccc')
354
+
355
+ plt.tight_layout(rect=[0, 0, 1, 1])
356
+ chart_path = "TopoDevPOC_n39_100samples.png"
357
+ plt.savefig(chart_path, dpi=110, bbox_inches='tight')
358
+ plt.close(fig)
359
+ print(f" [SAVED] {chart_path}")
360
+
361
+ # ── ASCII CLI chart for first 10 patterns ────────────────────────────────────
362
+ H = 7 # chart height in rows
363
+ W = 39 # chart width = N
364
+
365
+ print(f"\n ASCII CLI Charts β€” first 10 samples (right side = C_0 = newest)\n")
366
+ for i in range(10):
367
+ poc = poc_disp[i]
368
+ d_lbl = "Bullish" if dir_bits[i] == 0 else "Bearish"
369
+ pid = int(pat_ids[i])
370
+ n_mv = int(np.abs(ternary[i]).sum())
371
+ pmin, pmax = poc.min(), poc.max()
372
+ span = pmax - pmin if pmax != pmin else 1.0
373
+
374
+ # Map each x-position to a row
375
+ rows = (H - 1 - ((poc - pmin) / span * (H - 1))).round().astype(int)
376
+ rows = np.clip(rows, 0, H - 1)
377
+
378
+ grid = [[' '] * W for _ in range(H)]
379
+ for j in range(W):
380
+ r = rows[j]
381
+ # Strict-move node in the pair ending at display j:
382
+ # transition index k = N-1-j (if j < N-1)
383
+ is_strict = (j < N - 1) and (ternary[i, N - 2 - j] != 0)
384
+ grid[r][j] = '●' if is_strict else 'Β·'
385
+ # Connect with horizontal dash where poc is flat
386
+ for row_idx in range(H):
387
+ line = grid[row_idx]
388
+ for j in range(1, W):
389
+ if line[j] == ' ' and rows[j] == row_idx:
390
+ line[j] = '-'
391
+
392
+ print(f" [{i+1:2d}] {d_lbl:7s} | ID={pid} | strict_moves={n_mv}/{N_TRANS}")
393
+ poc_hi = poc[N-1]; poc_lo = poc[0]
394
+ print(f" POC range: oldest={poc_lo:.0f} β†’ newest={poc_hi:.0f}")
395
+ print(f" β”Œ{'─'*W}┐")
396
+ for row_idx in range(H):
397
+ print(f" β”‚{''.join(grid[row_idx])}β”‚")
398
+ print(f" β””{'─'*W}β”˜")
399
+ sym_preview = ' '.join(sym_matrix[i, :12]) + ' …'
400
+ print(f" Ξ£ (first 12): {sym_preview}\n")
401
+
402
+ # ─────────────────────────────────────────────────────────────────────────────
403
+ # OUTPUT 4 β€” COMPRESSED CSV EXPORT (bit-packed uint64 pattern_id)
404
+ #
405
+ # COMPRESSION LOGIC:
406
+ # Naive CSV text per row (direction label + 38 integers): ~110 bytes/row
407
+ # Compressed: single uint64 hex string per row ~ 18 bytes/row
408
+ # Reduction: ~6Γ— on text CSV; binary .npy = 8 bytes/row (machine-optimal)
409
+ #
410
+ # ENCODING (39-bit integer into uint64):
411
+ # pattern_id = dir_bit | (trans[0]<<1) | (trans[1]<<2) | … | (trans[37]<<38)
412
+ # bit 0 = direction (0=Bullish, 1=Bearish)
413
+ # bits 1..38 = transition flags
414
+ #
415
+ # DECODE (one-liner, O(1)):
416
+ # direction = "Bullish" if (pid & 1) == 0 else "Bearish"
417
+ # trans[k] = (pid >> (k+1)) & 1 for k in 0..37
418
+ # ternary[k] = trans[k] if Bullish else -trans[k]
419
+ #
420
+ # NOTE ON "ALL PATTERNS":
421
+ # 2^39 = 549,755,813,888 patterns. At 8 bytes/pattern β†’ 4.4 TB.
422
+ # At minimum 39 bits/pattern (bit-packed) β†’ 2.68 TB.
423
+ # The pattern space IS [0, 2^39). Pattern k = integer k. No file needed.
424
+ # Use the streaming generator below for on-demand enumeration.
425
+ # ─────────────────────────────────────────────────────────────────────────────
426
+
427
+ print(f"\n{SEP}")
428
+ print(f" OUTPUT 4 β€” COMPRESSED EXPORT")
429
+ print(SEP)
430
+
431
+ # ── Verify encoding round-trip before writing ─────────────────────────────
432
+ def encode_pattern_id(dir_bit, trans_bits):
433
+ """Pack direction + 38 transition bits into one uint64."""
434
+ pid = np.uint64(dir_bit)
435
+ for k, t in enumerate(trans_bits):
436
+ pid |= (np.uint64(int(t)) << np.uint64(k + 1))
437
+ return int(pid)
438
+
439
+ def decode_pattern_id(pid, n_trans=38):
440
+ """Unpack uint64 β†’ direction + ternary matrix. O(1) per pattern."""
441
+ pid = int(pid)
442
+ dir_bit = pid & 1
443
+ direction = "Bullish" if dir_bit == 0 else "Bearish"
444
+ sign = 1 if dir_bit == 0 else -1
445
+ trans = np.array([(pid >> (k + 1)) & 1 for k in range(n_trans)], dtype=np.int8)
446
+ ternary = (sign * trans).astype(np.int8)
447
+ return direction, trans, ternary
448
+
449
+ # Round-trip verification on all 100 samples
450
+ for i in range(N_SAMPLES):
451
+ pid_ref = int(pat_ids[i])
452
+ pid_enc = encode_pattern_id(int(dir_bits[i]), trans[i])
453
+ assert pid_ref == pid_enc, f"Encode mismatch at sample {i}"
454
+ d2, t2, m2 = decode_pattern_id(pid_enc)
455
+ assert (d2 == ("Bullish" if dir_bits[i] == 0 else "Bearish")), "Direction mismatch"
456
+ assert np.array_equal(m2, ternary[i]), f"Ternary mismatch at sample {i}"
457
+ print(" [OK] 100-sample encode/decode round-trip verified")
458
+
459
+ # ── CSV: bit-packed uint64 β€” one integer per row ──────────────────────────
460
+ # Columns: seq_id (1-based), pattern_id_uint64 (hex), direction, n_strict
461
+ # Total row bytes: ~50 chars vs ~110 for expanded matrix text
462
+ CSV_PATH = "TopoDevPOC_n39_samples.csv"
463
+ with open(CSV_PATH, 'w', newline='') as f:
464
+ writer = csv.writer(f)
465
+ # Header β€” human-readable but compact
466
+ writer.writerow([
467
+ "seq_id", # 1-based sample index
468
+ "pattern_id_uint64",# hex string β€” encodes everything, 18 chars
469
+ "pattern_id_dec", # decimal for inspection
470
+ "direction", # B+ or B- (1 char each saves space vs full label)
471
+ "n_strict_moves", # popcount of transitions
472
+ ])
473
+ for i in range(N_SAMPLES):
474
+ pid = int(pat_ids[i])
475
+ d_char = "B+" if dir_bits[i] == 0 else "B-"
476
+ n_mv = int(np.abs(ternary[i]).sum())
477
+ writer.writerow([
478
+ i + 1,
479
+ f"0x{pid:016X}", # 18-char hex β€” decode with int(x, 16)
480
+ pid,
481
+ d_char,
482
+ n_mv,
483
+ ])
484
+
485
+ csv_bytes = os.path.getsize(CSV_PATH)
486
+ print(f" [CSV] {CSV_PATH} β†’ {csv_bytes:,} bytes ({csv_bytes/N_SAMPLES:.1f} bytes/row)")
487
+
488
+ # ── Binary .npy β€” uint64 array, 8 bytes/row, machine-optimal ─────────────
489
+ NPY_PATH = "TopoDevPOC_n39_samples.npy"
490
+ np.save(NPY_PATH, pat_ids.astype(np.uint64))
491
+ npy_bytes = os.path.getsize(NPY_PATH)
492
+ print(f" [NPY] {NPY_PATH} β†’ {npy_bytes:,} bytes ({(npy_bytes-128)/N_SAMPLES:.1f} bytes/row payload)")
493
+
494
+ # ── Raw bit-packed .bin β€” 39 bits/pattern β†’ ceil(39/8)=5 bytes each ──────
495
+ # Pack 8 patterns Γ— 39 bits = 312 bits = 39 bytes per block (tight packing)
496
+ # Simple implementation: pack pattern_ids as 5-byte little-endian chunks
497
+ BIN_PATH = "TopoDevPOC_n39_samples.bin"
498
+ BYTES_PER = 5 # ceil(39 / 8) = 5 bytes holds one 39-bit pattern_id
499
+ with open(BIN_PATH, 'wb') as f:
500
+ # 8-byte magic + 4-byte n_patterns + 1-byte bits_per_pattern
501
+ f.write(b'TPOC3900')
502
+ f.write(struct.pack('<IB', N_SAMPLES, 39))
503
+ for i in range(N_SAMPLES):
504
+ pid = int(pat_ids[i])
505
+ f.write(pid.to_bytes(BYTES_PER, byteorder='little'))
506
+ bin_bytes = os.path.getsize(BIN_PATH)
507
+ payload = N_SAMPLES * BYTES_PER
508
+ print(f" [BIN] {BIN_PATH} β†’ {bin_bytes:,} bytes (13 hdr + {payload} payload = {BYTES_PER} bytes/pattern)")
509
+
510
+ # ── Print size comparison ─────────────────────────────────────────────────
511
+ print()
512
+ print(f" SIZE COMPARISON (100 samples, n=39, {N_TRANS} transitions):")
513
+ print(f" Naive text (direction + 38 ints): ~{100 * 110:,} bytes")
514
+ print(f" CSV hex uint64 (this file): {csv_bytes:,} bytes ({100*110//csv_bytes:.1f}Γ— smaller)")
515
+ print(f" NumPy .npy uint64: {npy_bytes:,} bytes")
516
+ print(f" Raw bit-packed .bin (39 bits each): {bin_bytes:,} bytes ({100*110//bin_bytes:.1f}Γ— smaller)")
517
+ print()
518
+ print(f" SCALING TO ALL 2^{N} = {TOTAL:,} PATTERNS:")
519
+ full_npy_tb = (TOTAL * 8) / 1e12
520
+ full_bin_tb = (TOTAL * BYTES_PER) / 1e12
521
+ full_text_tb = (TOTAL * 110) / 1e12
522
+ print(f" Naive text: ~{full_text_tb:.1f} TB (infeasible)")
523
+ print(f" uint64 binary (.npy): ~{full_npy_tb:.2f} TB (infeasible to store)")
524
+ print(f" 5-byte bit-packed: ~{full_bin_tb:.2f} TB (infeasible to store)")
525
+ print(f" Optimal: 0 bytes β€” pattern k = integer k, range [0, 2^{N})")
526
+ print(f" Use streaming generator below for on-demand enumeration.")
527
+
528
+ # ── Streaming generator (no memory, yields all 2^39 on-demand) ───────────
529
+ def stream_all_patterns(n=N):
530
+ """
531
+ Yields (pattern_id, direction, ternary_matrix) for all 2^n patterns.
532
+ Zero memory beyond one pattern at a time. Works for any n.
533
+ Pattern_id k: bit0=direction, bits1..n-1=transitions.
534
+ """
535
+ sign_map = {0: 1, 1: -1}
536
+ for k in range(1 << n):
537
+ dir_bit = k & 1
538
+ sign = sign_map[dir_bit]
539
+ direction = "Bullish" if dir_bit == 0 else "Bearish"
540
+ ternary_k = np.array(
541
+ [sign * ((k >> (j + 1)) & 1) for j in range(n - 1)],
542
+ dtype=np.int8
543
+ )
544
+ yield k, direction, ternary_k
545
+
546
+ # Show the first 4 and last 4 pattern_ids to confirm ordering
547
+ print()
548
+ print(f" STREAMING GENERATOR β€” first 4 / last 4 of all {TOTAL:,} patterns:")
549
+ gen = stream_all_patterns(N)
550
+ for _ in range(4):
551
+ pid_s, d_s, _ = next(gen)
552
+ print(f" pattern_id={pid_s:>3d} direction={d_s}")
553
+ print(f" ... ({TOTAL - 8:,} patterns omitted) ...")
554
+ # Last 4: decode directly from known IDs
555
+ for pid_s in range(TOTAL - 4, TOTAL):
556
+ d_s = "Bullish" if (pid_s & 1) == 0 else "Bearish"
557
+ print(f" pattern_id={pid_s} direction={d_s}")
558
+
559
+ # ─────────────────────────────────────────────────────────────────────────────
560
+ # FINAL SUMMARY
561
+ # ─────────────────────────────────────────────────────────────────────────────
562
+ total_ms = (time.perf_counter() - t_sample) * 1e3
563
+ print(SEP)
564
+ print(" SUMMARY")
565
+ print(SEP)
566
+ print(f" n (candles) = {N}")
567
+ print(f" Transitions per pattern = {N_TRANS}")
568
+ print(f" Total patterns [2^{N}] = {TOTAL:,}")
569
+ print(f" Bullish [2^{N-1}] = {HALF:,}")
570
+ print(f" Bearish [2^{N-1}] = {HALF:,}")
571
+ print(f" Matrix B_n validated = {B_n:,} βœ“")
572
+ print(f" Samples generated = {N_SAMPLES}")
573
+ print(f" Chart file = {chart_path}")
574
+ print(f" CSV export (hex uint64) = {CSV_PATH} ({csv_bytes:,} bytes)")
575
+ print(f" Binary export (.npy) = {NPY_PATH} ({npy_bytes:,} bytes)")
576
+ print(f" Bit-packed export (.bin) = {BIN_PATH} ({bin_bytes:,} bytes)")
577
+ print(f" Compute device = {DEVICE}")
578
+ print(f" Wall-clock (sample+decode) = {total_ms:.2f} ms")
579
+ print(SEP)
580
+ print()
581
+ print(" CONVERSION FORMULAS (tex Sec. V)")
582
+ print(" Symbolic β†’ Ternary:")
583
+ print(" Bullish: '>' β†’ +1, '=' β†’ 0")
584
+ print(" Bearish: '<' β†’ -1, '=' β†’ 0")
585
+ print(" Ternary β†’ Symbolic:")
586
+ print(" +1 β†’ '>', 0 β†’ '=', -1 β†’ '<'")
587
+ print()
588
+ print(" TEMPORAL CONVENTION:")
589
+ print(f" Chart x-axis: left = C_{{-{N-1}}} (oldest) β†’ right = C_0 (newest)")
590
+ print(" Bullish pattern: POC non-increasing toward right (higher on left)")
591
+ print(" Bearish pattern: POC non-decreasing toward right (lower on left)")
592
+ print(SEP)