---

language:

• en library_name: pytorch license: mit tags:
• quadratic-equations
• mathematical-reasoning
• interpretability
• mechanistic-interpretability
• symbolic-reasoning
• synthetic-data pipeline_tag: text-generation

---

SEAI: Learning Mathematical Representations for Quadratic Equation Solving

SEAI (Square Equation AI) is a research neural network designed to solve quadratic equations with real roots while enabling direct analysis of its learned internal representations.

The primary purpose of SEAI is not only numerical prediction. The project investigates whether a neural network trained to predict quadratic roots can develop internal representations correlated with mathematically meaningful quantities involved in the analytical solution.

A central observation of the current experiment is that hidden activations in deeper layers exhibit strong correlations with the normalized quadratic discriminant

$$p^2-qp^2-q$$

and with its square-root transformation, despite neither quantity being explicitly provided as an input feature.

---

Model Summary

SEAI receives a normalized representation of a quadratic equation

$$a{x}^2+bx+c=0ax^2+bx+c=0$$

and predicts the two ordered real roots.

The model contains exactly:

8,040,002 trainable parameters.

The architecture consists of four custom quadratic layers followed by a linear output projection.

Input (2)
    ↓
QuadraticLayer: 2 → 1000
    ↓
LayerNorm
    ↓
GELU
    ↓
QuadraticLayer: 1000 → 2000
    ↓
LayerNorm
    ↓
GELU
    ↓
QuadraticLayer: 2000 → 2000
    ↓
LayerNorm
    ↓
GELU
    ↓
QuadraticLayer: 2000 → 1000
    ↓
LayerNorm
    ↓
GELU
    ↓
Linear: 1000 → 2
    ↓
Ordered normalized roots

---

Mathematical Formulation

A quadratic equation is invariant under multiplication of all coefficients by the same non-zero scalar.

We therefore use the scale-invariant quantities

$$P=\frac{b}{a},Q=\frac{c}{a}\mathrm{.}P=\backslash frac\{b\}\{a\},\; \backslash qquad\; Q=\backslash frac\{c\}\{a\}.$$

The training distribution generated from roots in

$$[-50,50][-50,50]$$

produces

$$P\in [-100,100],Q\in [-2500,2500]\mathrm{.}P\backslash in[-100,100],\; \backslash qquad\; Q\backslash in[-2500,2500].$$

The model receives the normalized variables

$$p=\frac{P}{100},q=\frac{Q}{2500}\mathrm{.}p=\backslash frac\{P\}\{100\},\; \backslash qquad\; q=\backslash frac\{Q\}\{2500\}.$$

The target roots are normalized as

$$x_n=\frac{x}{50}\mathrm{.}x\_n=\backslash frac\{x\}\{50\}.$$

Under this normalization, the exact solution for the normalized roots is

$$\boxed{{\displaystyle x_{n,1\mathrm{/}2}=-p\pm \sqrt{p^2-q}}}\backslash boxed\{\; x\_\{n,1/2\}=-p\backslash pm\backslash sqrt\{p^2-q\}\; \}$$

with the roots ordered so that

$$x_{n,1}\le x_{n,2}\mathrm{.}x\_\{n,1\}\backslash leq\; x\_\{n,2\}.$$

The classical discriminant is

$$\mathrm{\Delta }=b^2-4ac\mathrm{.}\backslash Delta=b^2-4ac.$$

Under the SEAI normalization,

$$\boxed{{\displaystyle \mathrm{\Delta }=10000a^2(p^2-q)}}\backslash boxed\{\; \backslash Delta=10000a^2(p^2-q)\; \}$$

and therefore

$$\boxed{{\displaystyle p^2-q=\frac{\mathrm{\Delta }}{10000a^2}\mathrm{.}}}\backslash boxed\{\; p^2-q=\backslash frac\{\backslash Delta\}\{10000a^2\}.\; \}$$

Thus, (p^2-q) is exactly the discriminant normalized by the coefficient scale.

---

Dataset

The dataset is entirely synthetic.

Two real roots are sampled from

$$x_1,x_2\sim U(-50,50)x\_1,x\_2\backslash sim\; U(-50,50)$$

and sorted so that

$$x_1\le x_2\mathrm{.}x\_1\backslash leq\; x\_2.$$

A non-zero coefficient (a) is then selected and the remaining coefficients are constructed using

$$b=-a(x_1+x_2)b=-a(x\_1+x\_2)$$

and

$$c=a{x}_1{x}_2\mathrm{.}c=ax\_1x\_2.$$

This guarantees that the generated quadratic has the selected roots as its exact solutions.

The current training experiment uses:

Training examples: 150,000
Root range:        [-50, 50]
Task:              real-root quadratic equations

The current experiment does not evaluate equations with complex roots.

---

QuadraticLayer

The central custom component is:

class QuadraticLayer(nn.Module):
    def __init__(self, in_features, out_features):
        super().__init__()

        self.linear = nn.Linear(in_features, out_features)

        self.A = nn.Parameter(torch.ones(out_features) * 0.01)
        self.B = nn.Parameter(torch.ones(out_features))
        self.C = nn.Parameter(torch.zeros(out_features))

    def forward(self, x):
        z = self.linear(x)
        return self.A * z**2 + self.B * z + self.C

Each output channel therefore computes

$$y_i=A_i{z}_i^2+B_i{z}_i+C_{i}y\_i=A\_i\; z\_i^2+B\_i\; z\_i+C\_i$$

where

$$z=Wx+b\mathrm{.}z=Wx+b.$$

This gives the model an explicit mechanism for constructing quadratic nonlinear transformations of learned linear projections.

---

Training Configuration

Dataset size:       150,000
Batch size:         256
Epochs:             125

Optimizer:          AdamW
Learning rate:      3e-4
Weight decay:       1e-4

Loss:               SmoothL1Loss
Gradient clipping:  max_norm = 1.0

Activation:         GELU
Normalization:      LayerNorm

---

Evaluation

One recent independent evaluation run produced the following results:

Metric	Result
MAE	0.084864
RMSE	0.137898
Maximum observed error	1.652145
Error < 0.1	69.34%
Error < 0.01	0.27%

These metrics are reported in the normalized root representation unless otherwise specified.

The current evaluation results should be considered preliminary. Earlier experiments used independently generated evaluation sets, so future releases will use fixed train/validation/test splits and fixed random seeds for reproducible comparisons.

The model tends to perform worse when the two roots become very close.

---

Interpretability Analysis

One of the main purposes of SEAI is to investigate whether mathematically meaningful structures emerge inside the network.

Forward hooks are used to record hidden activations from each QuadraticLayer.

Neuron activations are compared against candidate mathematical features including

$$p,q,p^2,q^2,pq,p,\backslash quad\; q,\backslash quad\; p^2,\backslash quad\; q^2,\backslash quad\; pq,$$

$$p^2-q,p^2+q,p^2-q,\backslash qquad\; p^2+q,$$

and

$$\sqrt{\mathrm{\mid }{p}^2-q\mathrm{\mid }}\mathrm{.}\backslash sqrt\{|p^2-q|\}.$$

Because the current dataset contains real-root equations, the discriminant-related quantity satisfies

$$p^2-q\ge 0p^2-q\backslash geq0$$

up to numerical precision.

The analysis investigates which candidate mathematical quantities are most strongly correlated with individual hidden activations.

---

Observed Representation Structure

The current experiment shows a qualitative progression across layers.

First Quadratic Layer

Hidden activations primarily show strong correlations with the original variables

$$pp$$

and

$$q\mathrm{.}q.$$

Second Quadratic Layer

Nonlinear mathematical combinations become more prominent, including

$$p^2+qp^2+q$$

and

$$p^2-q\mathrm{.}p^2-q.$$

Third and Fourth Quadratic Layers

The strongest recurring candidate features include

$$p^2-qp^2-q$$

and

$$\sqrt{p^2-q}\mathrm{.}\backslash sqrt\{p^2-q\}.$$

This produces an experimentally observed progression consistent with the algebraic structure required by the analytical solution:

$$\boxed{{\displaystyle p,q\to \text{quadratic combinations}\to p^2-q\to \sqrt{p^2-q}\to x_1,x_2}}\backslash boxed\{\; p,q\; \backslash rightarrow\; \backslash text\{quadratic\; combinations\}\; \backslash rightarrow\; p^2-q\; \backslash rightarrow\; \backslash sqrt\{p^2-q\}\; \backslash rightarrow\; x\_1,x\_2\; \}$$

The interpretation of this progression remains exploratory.

---

Main Interpretability Finding

The central observation of the current study is:

An 8.04-million-parameter neural network developed hidden representations strongly correlated with the normalized quadratic discriminant and its square-root transformation, despite not receiving either quantity explicitly as an input.

This does not establish that an individual neuron literally performs symbolic discriminant or square-root computation.

Instead, it provides evidence that the trained network contains internal representations that are statistically aligned with mathematically meaningful quantities involved in the classical solution.

---

Error Behaviour Near Repeated Roots

The largest errors tend to occur when

$$x_1\approx x_2\mathrm{.}x\_1\backslash approx\; x\_2.$$

For the normalized representation this corresponds to

$$p^2-q\approx 0.p^2-q\backslash approx0.$$

This is the regime in which the square-root term in the analytical solution approaches zero.

A future systematic analysis will measure prediction error as a function of

$$\mathrm{\mid }{x}_2-x_1\mathrm{\mid }|x\_2-x\_1|$$

and

$$p^2-q\mathrm{.}p^2-q.$$

---

Limitations

The following limitations are important.

Correlation is not proof of symbolic computation

A high correlation between a neuron and a mathematical expression does not prove that the neuron explicitly computes that expression.

Preliminary evaluation protocol

The current reported metrics include experiments performed with independently generated evaluation sets. Fixed evaluation data and fixed seeds are required for rigorous model comparison.

Restricted data distribution

The model is currently trained on real-root quadratic equations generated from roots in

$$[-50,50]\mathrm{.}[-50,50].$$

Generalization outside this range has not yet been established.

No complex-root regime

The current training and evaluation setup does not investigate equations with

Interpretability is exploratory

The current analysis is primarily based on neuron-level correlations. Stronger evidence would require symbolic regression, reproducibility across independent runs, and causal interventions on candidate neurons or representations.

---

Future Research

Planned experiments include:

• fixed reproducible train/validation/test splits;
• multiple random seeds;
• standard MLP baselines;
• ablations of the quadratic transformation;
• systematic neuron-level symbolic regression;
• causal interventions on neurons correlated with (p^2-q);
• out-of-distribution evaluation;
• larger root and coefficient ranges;
• analysis of representation stability across independently trained models;
• comparison with conventional polynomial architectures;
• extension to higher-degree polynomial equations.

A key research question is whether the observed discriminant-related representations arise systematically from the quadratic architecture or can be reproduced equally well by standard neural networks of comparable size.

---

Intended Use

SEAI is intended primarily as a research prototype for:

• mechanistic interpretability;
• mathematical representation learning;
• neural symbolic reasoning;
• analysis of learned algebraic structure;
• controlled studies of neural networks on mathematically defined tasks.

It is not intended to replace conventional numerical solvers in production environments.

For arbitrary quadratic equations, a conventional analytical or numerical solver remains preferable.

---

Repository Structure

SEAI/
├── train.py
├── model.py
├── interpretability.py
├── evaluation.py
├── requirements.txt
├── README.md
└── weights/
    └── model_weights_SEAI.pth

The final reproducible release should additionally contain:

├── data/
│   ├── train.pt
│   ├── validation.pt
│   └── test.pt
├── configs/
│   └── seai_config.json
└── results/
    ├── metrics.json
    └── interpretability/

---

Citation

@misc{seai2026,
      title={SEAI: Learning Mathematical Representations for Quadratic Equation Solving},
      author={ALEXFLR},
      year={2026},
      note={Research prototype}
}

The corresponding research paper is intended for publication on arXiv.

---

Project Status

Research prototype — experimental

SEAI is primarily a research project investigating whether neural networks can develop internal representations aligned with known mathematical structures.

🤖AI

class QuadraticLayer(nn.Module): def init(self, in_features, out_features): super().init() self.linear = nn.Linear( in_features, out_features ) self.A = nn.Parameter( t.ones(out_features) * 0.01 )

    self.B = nn.Parameter(
        t.ones(out_features)
    )

    self.C = nn.Parameter(
        t.zeros(out_features)
    )

def forward(self, x):

    z = self.linear(x)

    return (
        self.A * z**2
        + self.B * z
        + self.C
    )

class SEAI(nn.Module): def init(self): super().init()

    self.net = nn.Sequential(

        QuadraticLayer(2, 1000),
        nn.LayerNorm(1000),
        nn.GELU(),

        QuadraticLayer(1000, 2000),
        nn.LayerNorm(2000),
        nn.GELU(),

        QuadraticLayer(2000, 2000),
        nn.LayerNorm(2000),
        nn.GELU(),

        QuadraticLayer(2000, 1000),
        nn.LayerNorm(1000),
        nn.GELU(),

        nn.Linear(1000, 2)
    )

def forward(self, x):
    return self.net(x)

model = SEAI().to(device)

model.eval()

with t.no_grad(): ... weights in another file.