Authors: Gris Iscomeback
Source: Zenodo (CERN European Organization for Nuclear Research) (preprint, 2025-12-27)
Field: Materials Science — Materials Chemistry

Algorithmic Induction via Structural Weight Transfer ---mainfont: "FreeSerif"monofont: "FreeMono"mathfont: "FreeSerif"--- # Engineering Algorithmic Structures in Neural Networks: From a Materials Science Perspective to Algorithmic Thermodynamics of Deep Learning **Author:** Iscomeback, Gris ( grisun0 ) --- ## Abstract This article presents what I learned by attempting to induce the Strassen matrix multiplication structure in neural networks, and why I now view this work as materials engineering rather than theory, and how it may serve as a bridge between statistical thermodynamics and deep learning. Through Strassen matrix multiplication, it is demonstrated that by controlling batch size, training duration, and regularization, a discrete algorithmic structure can be induced that transfers knowledge to higher dimensions in a zero-shot procedure from 2x2 to 64x matrices. The two-phase protocol I present, training followed by sparsification and discretization, serves as empirical evidence. Under controlled conditions, 68% of runs crystallize into a verifiable Strassen structure. The remaining 32% converge to local minima that generalize on test sets but do not pass structural verification versus 0% crystals achieved without the two-phase protocol. What I initially proposed as a theory, claiming that gradient covariance geometry determines whether networks learn algorithms, did not withstand scrutiny at first. Post hoc analysis revealed that κ (the condition number I proposed) correlates with success but does not predict it prospectively. The hypothesis was backwards: successful models have κ≈1, but models with κ≈1 are not guaranteed to succeed, although this would later change with further analysis, realizing that there is a minimum gradient temperature that allows the crystallization of the algorithm in the neural network. Following reviewer comments, I now have stronger evidence for κ as a predictive metric. In 20 balanced runs with varied hyperparameters, κ achieves perfect separation between assimilated and non-assimilated outcomes (AUC = 1.000, 95% CI [1.000, 1.000]) on the validation set of 20 runs. While this indicates strong predictive power, the interval appears degenerate because there is no overlap between classes, which is explained because grokking is a first-order phase transition where there are no intermediates. Future work should test generalization to unseen hyperparameter regimes. Furthermore, κ prospectively separates assimilated from non-assimilated runs (N=60, AUC=1.000) within the tested hyperparameter ranges, confirming that the metric reliably predicts outcomes before training is complete. Local complexity falls to zero exactly at the assimilation transition (Figure 6), confirming that it captures the phase change. The discrete basin remains stable under iterative pruning up to 50% sparsity, after which the solution collapses. The 60-run hyperparameter sweep provides conclusive validation. When I varied batch size from 8 to 256 and weight decay from 1e-5 to 1e-2, κ perfectly separated successful from failed runs. Every run that assimilated showed κ = 1.000. Every run that failed showed κ = 999999. AUC reached 1.000 with 95% CI [1.000, 1.000]. These results are the most definitive evidence I have that κ captures something real about training dynamics—I would venture to say it functions as temperature, indicating the phase change. What remains valid is the engineering protocol itself. This is what actually works: train with batch sizes in [24, 128], use weight decay ≥1e-4, run for more than 1000 epochs, prune to 7 slots, round weights to integers. Do this and you will induce the Strassen structure with 68% probability. I used to call this work "materials engineering" because I couldn't measure heat. Now I can. I performed 245 training runs, logged every gradient, and treated each checkpoint as a microstate. The numbers gave me temperature, entropy, and heat capacity without metaphors. The recipe remains the same (batch size 32, weight decay 1e-4, 1000 epochs, prune to seven slots, round), but I no longer sell it as kitchen wisdom like in the Carnot era. It is a reproducible thermodynamic protocol that places a discrete algorithm at a predictable point in phase space. κ, the condition number of the gradient covariance matrix, acts as an order parameter: κ = 1.000 exactly when the system is in the crystalline phase; κ = 999999 otherwise. In sixty hyperparameter configurations, the separation is perfect (AUC = 1.000, 95% CI [1.000, 1.000]). The confidence interval appears degenerate because the two distributions do not overlap. Local complexity falls from 442 to 0 at the assimilation transition, confirming a first-order phase change. kappa outside the tested ranges tends toward infinity. The crystalline basin is stable under pruning up to 50% sparsity and breaks at 51%, providing a measurable elastic limit. This work is structured as a sequence of questions and their resolutions.The main text poses the questions; the appendices contain the answers. **Phase images in the materials sense.** The figures in this work serve as experimental visualizations of microstructural properties: Figure 4 shows weight distribution evolution (microstructure), Figure 7 shows batch size effect (phase boundary), Figure 8 shows the complete phase diagram (phase map), Figure 5 shows assimilation dynamics (temporal phase transition), and Appendix E shows noise perturbation results (basin width measurement). These images characterize the material properties of trained networks without claiming thermodynamic equivalence to real heat or processor thermal noise, but rather a statistical thermodynamics applied to gradient noise because the measured results are several orders of magnitude greater than those measured in conventional physics. The system reveals extreme fragility: noise of magnitude 0.001 causes 100% discretization failure when applied after training. However, I now have evidence that the discrete basin is stable under pruning up to 50% sparsity. This fragility has implications beyond my specific experiments. If a well-defined algorithm like Strassen requires such precise training conditions to emerge, what does this say about reproducibility in deep learning more broadly? Narrow basins containing algorithmic solutions may be much more common than we believe, and our inability to reach them consistently may explain many reproducibility failures in the field. Note to the reader.This manuscript is deliberately structured as a sequence of questions and their resolutions.The main text poses the questions and only states what can be claimed without reservation.The appendices contain the empirical resolutions: definitions, measurements, falsifications, and limits.Reading the main text without the appendices is sufficient to follow the narrative,but insufficient to evaluate the claims and their implications for deep learning. --- ## 1. Introduction Neural networks trained on algorithmic tasks sometimes exhibit grokking: a late generalization that occurs long after training loss has converged [1]. Previous work characterized this transition using local complexity measures [1] and connected it to superposition as lossy compression [2]. But a fundamental question remained unanswered: when a network groks, has it learned the algorithm or has it found a local minimum that generalizes? [3] This article presents what I have learned by attempting to answer this question through Strassen matrix multiplication, using this discrete algorithm as a microscope, and why I now view this work from materials engineering to statistical thermodynamics, looking at an understanding of neural networks through statistical tools that we can even consider as heuristic, but provide us with a privileged axis of vision into what happens during training per se. I set out to demonstrate that neural networks could be induced through engineering to learn genuine algorithms, not just convenient local minima or cold glass. The test case was Strassen matrix multiplication, which has an exact structure: 7 products with coefficients in {-1, 0, 1}. If a network learned Strassen, I could verify this by rounding weights to integers and checking if they matched the canonical structure. I developed a two-phase protocol. Phase 1: train a bilinear model with 8 slots on 2x2 multiplication. Phase 2: prune to 7 slots, discretize weights, and verify that the structure transfers to 64x64 matrices. I called this theory. I claimed that training trajectory geometry determines whether algorithmic structure emerges. I proposed that gradient covariance, measured by κ, could predict which training runs would succeed. I was wrong about the prediction part. Post hoc analysis showed that κ correlates with success but does not cause it and cannot be used to predict outcomes from early epoch measurements. However, after validation experiments requested by reviewers, I now have prospective evidence that κ achieves perfect separation (AUC = 1.000, 95% CI [1.000, 1.000]) on the validation set of 20 runs. While this indicates strong predictive power, the interval might appear degenerate because there is no overlap between classes. This tells us it is a first-order phase transition and therefore lacks intermediate points. Future work should test generalization to unseen hyperparameter regimes. This validates κ as a prospective prediction metric. But even stronger is delta δ, which allows us to prospect in early epochs one or many seeds by measuring their entropy, transforming cherry-picking into seed mining. What remains valid is the engineering protocol itself. When I follow the conditions I specify, the Strassen structure emerges 68% of the time. This is a real, reproducible result, documented with 195 training runs. Without pruning, 0% of runs converge to the Strassen structure (N=195), confirming that explicit sparsification is essential for algorithmic induction. The batch size finding concr Strassen algorithm ; Artificial neural network ; Maxima and minima ; Verifiable secret sharing ; Covariance matrix ; Matrix (chemical analysis) ; Matrix multiplication ; Covariance Machine Learning in Materials Science ; Model Reduction and Neural Networks ; Stochastic Gradient Optimization Techniques