PowAI: Neural Approximation and Symbolic Distillation for Real-Exponent Complex Powers
By ALEXFLR
Abstract
This study evaluates PowAI, a feed-forward neural network trained to approximate the complex power (a^b), where the base (a) and exponent (b) are real-valued. It documents exploratory experiments with loss functions, activation functions, batch sizes, and training duration, then compares a trained checkpoint with compact symbolic approximations derived from the network's predictions. A five-seed random evaluation used 50,000 samples per seed, for 250,000 evaluations in total. Across these runs, PowAI achieved a mean complex absolute error (MAE) of (0.235567 \pm 0.00428) and an RMSE of (0.758994 \pm 0.0262). The fixed symbolic approximations achieved an MAE of (0.229265 \pm 0.00447) and an RMSE of (0.707999 \pm 0.0226). An analytic reference implementation agreed with Python's complex-power result to approximately floating-point precision. These results suggest that the selected expressions approximated the target well over the tested domain, but they do not establish that the equations were independently discovered or that the improvement is statistically significant. The original training dataset was unavailable during the five-seed evaluation, so overlap between training and test inputs could not be verified.
Keywords: neural networks, symbolic distillation, complex exponentiation, GELU, Huber loss, reproducibility
1. Introduction
Neural networks can approximate nonlinear mathematical functions from examples, but their learned computations are generally distributed across many parameters and layers. Symbolic distillation attempts to express some of this behavior with a compact, human-readable formula.
PowAI was developed as a small regression model for complex powers with real-valued bases and exponents. This paper documents the model configuration, exploratory training experiments, a subsequent formula-fitting step, and a repeated random evaluation comparing the neural network, the fitted expressions, and an analytic reference.
The study has two distinct aims:
- Evaluate how well the trained network approximates the target function in the chosen input domain.
- Evaluate whether the fixed symbolic approximations are competitive with the neural network on random test sets generated after the coefficients had been selected.
This is an engineering experiment on one trained checkpoint, not a claim that a new mathematical law has been discovered.
2. Mathematical reference
For (a>0) and real (b), the result is real:
[ a^b = \exp(b\ln a), \qquad \operatorname{Im}(a^b)=0. ]
For (a<0), using the principal complex-power convention and writing (r=|a|),
[ \operatorname{Re}(a^b)=r^b\cos(\pi b), ]
[ \operatorname{Im}(a^b)=r^b\sin(\pi b). ]
These formulas are the analytic reference. They are not outputs discovered by the network; they are known mathematical expressions against which its predictions can be evaluated. The tests avoid an exact zero base and use a sampling rule that resamples cases with (|a|<0.5) and (b<0), consistent with the data-generation rule used in this project.
3. Dataset and input/output scaling
The training setup used approximately 30,000 generated examples. Bases were sampled in the interval ([-5,5]), and real exponents in ([-3,3]), subject to the sampling guard described above.
The network receives the two normalized inputs
[ x_1 = a/5,\qquad x_2=b/3. ]
The target is represented by two real outputs, corresponding to the real and imaginary parts of the complex result, each divided by
[ Y_{\mathrm{scale}}=5^3=125. ]
During inference, both predicted outputs are multiplied by 125 to restore the original scale. This inverse scaling is necessary for comparisons in the original complex-number units.
The exact original training samples were not available to the final five-seed random evaluation. As a result, the final random samples cannot be classified with certainty as seen or unseen relative to training. The current evaluation therefore measures performance on repeated random samples from the same input domain, not on a verified 50/50 seen/unseen split.
4. Neural-network architecture and training setup
The tested checkpoint uses a feed-forward architecture with two input features and two outputs. It contains three hidden fully connected layers with 324 units each. GELU nonlinearities and Layer Normalization are used in the hidden stack.
The training setup explored AdamW with a learning rate of 0.001, batch sizes including 12 and 24, and training durations of 150 and 200 epochs. Batch size 24 generally performed better than batch size 12 in the runs observed, but this comparison was not a fully controlled multi-seed experiment.
The checkpoint file stores model parameters, but does not reliably encode all training choices such as the original activation function, loss configuration, batch size, or number of epochs. The results below describe the logged experiments and should not be interpreted as a fully controlled hyperparameter benchmark.
5. Experiments with loss functions
Several regression losses were evaluated. The experiments included mean squared error (MSE), PyTorch's SmoothL1 loss, and Huber loss with different values of (\delta). Since tests were performed in different runs, with differing epochs and test samples, the figures below are exploratory and are not a controlled leaderboard.
5.1 Reported loss experiments
| Loss configuration | Reported test size / setting | Mean absolute error | Median absolute error | Median relative error | Within 10% relative error |
|---|---|---|---|---|---|
| Huber, (\delta=0.0001) | 10,000 samples, one run | 0.228799 | 0.090419 | 8.259% | 54.83% |
| Huber, (\delta=0.0001) | 200 epochs, separate run | 0.298434 | 0.074350 | 8.542% | 55.40% |
| Huber, (\delta=0.0001) | 100,000-sample audit of a candidate checkpoint | 0.231421 | 0.089708 | 8.503% | 54.12% |
| SmoothL1 | 100,000-sample audit | 0.238209 | 0.133314 | 14.658% | 39.40% |
| MSE | Exploratory comparison run | 0.217317 | 0.165339 | 19.917% | 35.29% |
| Huber, (\delta=0.00001) | 10,000-sample run | — | 0.201080 | 17.861% | 31.38% |
| Huber, (\delta=0.00005) | 150 epochs, 10,000-sample run | — | 0.104375 | 15.502% | 35.46% |
| Huber, (\delta=0.001) | Earlier 10,000-sample run | — | 0.0854 | 11.23% | 45.10% |
Note: A dash means the corresponding metric was not recorded in the available experiment summary. Values from different runs are not directly comparable because test samples and training settings were not held identical throughout.
MSE produced a lower mean absolute error than some Huber runs, but its median absolute and relative errors were larger in the reported comparison. This illustrates why choosing a loss or checkpoint using only one summary metric can be misleading: the error distribution can contain both many modest errors and a smaller number of large errors.
Huber loss with (\delta=0.0001) was a promising configuration in the experiments, particularly when considering median relative error and the fraction of samples within 10%. However, a definitive conclusion about the best loss requires a controlled experiment using identical training/validation splits, fixed seeds, and repeated training runs.
5.2 Interpretation of the loss experiments
The loss delta affects how large residuals contribute to the training objective. Because the targets are normalized by 125, the numerical delta is applied on the normalized output scale. Consequently, a delta value should be documented together with target normalization.
The experiments suggest that Huber loss with (\delta=0.0001) was a useful candidate, but they do not establish that it is universally superior to MSE or SmoothL1. A proper comparison would train each configuration multiple times with identical data splits and compute metrics on the same untouched evaluation sets.
6. Experiments with activation functions
GELU, SELU, and ReLU were compared in exploratory training runs. The available runs did not all use the same number of epochs: the cited GELU run used 200 epochs, while the SELU and ReLU runs used 150 epochs.
| Activation | Epochs in reported run | Median absolute error | Median relative error | Within 10% relative error |
|---|---|---|---|---|
| GELU | 200 | 0.074350 | 8.542% | 55.40% |
| SELU | 150 | 0.120669 | 12.276% | 44.30% |
| ReLU | 150 | 0.132316 | 18.473% | 27.30% |
GELU produced the strongest reported results among these particular runs. However, the different epoch counts and potentially different random states mean this is evidence for a promising configuration, not a clean causal estimate of activation-function effects. A controlled activation experiment should keep the architecture width, data order, seeds, batch size, optimizer, loss, training duration, and evaluation examples fixed wherever possible.
The current inference code assumes the GELU checkpoint architecture used by the tested model. An arbitrary checkpoint trained with SELU or ReLU cannot be substituted solely by loading its weights unless its architecture and activation behavior match the implementation.
7. Symbolic distillation
A lightweight symbolic-distillation routine fitted compact expressions to the network outputs. It used a predetermined candidate basis containing expressions related to (a^b), (|a|^b\cos(\pi b)), (|a|^b\sin(\pi b)), logarithmic terms, and low-order polynomial corrections. The coefficients were selected on a fitting subset and assessed on a held-out validation subset.
The resulting expressions are approximations of the network's outputs, not proof that the network internally computes these equations. The candidate set already included terms informed by the known target-function structure.
For positive (a), one fitted real-output expression was
[ \widehat{\operatorname{Re}}(a,b) =0.0048068052+1.0585503a^b -0.07863394a^b\ln(a). ]
For negative (a), the fitted real-output expression was
[ \widehat{\operatorname{Re}}(a,b) =-0.086358634 +0.96223595|a|^b\cos(\pi b) +0.0085419565|a|^b b. ]
Additional expressions were fitted for the imaginary output. For (a>0), the exact imaginary part is zero; a nonzero fitted expression in that branch represents neural-network approximation error, not a property of the exact mathematical function.
8. Five-seed random evaluation
After fixing the distilled coefficients, a repeated random evaluation was run using seeds 42, 123, 2026, 20261019, and 314159. Each seed generated 50,000 test samples, giving 250,000 evaluations in total. The same generated input set was used to evaluate the network, the fixed distilled expressions, and the analytic reference.
The reported aggregate results were:
| Predictor | Complex MAE (mean ± SD across seeds) | Complex RMSE (mean ± SD across seeds) |
|---|---|---|
| PowAI neural network | 0.235567 ± 0.00428 | 0.758994 ± 0.0262 |
| Fixed distilled formulas | 0.229265 ± 0.00447 | 0.707999 ± 0.0226 |
| Exact analytic reference | (6.66\times10^{-16}) ± (1.15\times10^{-17}) | (3.10\times10^{-15}) ± (1.01\times10^{-16}) |
Relative to PowAI, the fixed formulas reduced the reported mean MAE by approximately 2.67% and the RMSE by approximately 6.72%. The analytic implementation agreed with Python's complex-power computation to approximately floating-point precision, providing a useful sanity check for the reference code.
The seed-to-seed standard deviations describe variability across the five generated test samples. They are not standard errors for a population of independently trained networks, and the five seeds do not represent five independent training procedures.
9. Discussion
The final random evaluation indicates that the fitted formulas are competitive with the selected PowAI checkpoint over the tested domain and show a modest improvement in average error. Their RMSE improvement suggests that they also reduce the contribution of larger errors in aggregate, although conclusions should be checked using the detailed per-seed outputs.
The analytic baseline is far more accurate because it directly evaluates the known target function rather than approximating it from training data. In this problem, the goal of symbolic distillation is therefore not to discover the exact function from nothing; it is to understand whether a compact expression can reproduce or improve upon a trained network's predictions.
The loss and activation experiments provide useful leads for further model development. However, because those experiments were exploratory rather than fully controlled, they should be reported as empirical observations. A stronger follow-up study would fix the training and validation split, run multiple training seeds per hyperparameter configuration, and compare models on identical independent test samples.
10. Limitations and reproducibility
The main limitations are:
- The original training dataset was unavailable during the final five-seed test, so overlap between training and test inputs could not be verified.
- The five-seed experiment tested random inputs from the same domain; it was not a verified 50% seen / 50% unseen split.
- Loss and activation experiments were not all matched for epochs, data order, test samples, and seeds.
- The fixed symbolic expressions were developed using a candidate set that included known mathematical terms related to complex exponentiation.
- Only one final network checkpoint was evaluated in the reported five-seed comparison; variation across independently trained checkpoints remains unmeasured.
The report files generated by the test script should be archived together with the exact checkpoint and inference code. Any publication should preserve the random seeds, package versions, model architecture, preprocessing, and metric definitions.
11. Conclusion
In five random test runs totaling 250,000 examples, compact symbolic approximations slightly outperformed the selected PowAI checkpoint in reported complex MAE and RMSE. The analytic reference agreed with Python's complex-power operation to floating-point precision.
The experiments with Huber loss, SmoothL1, MSE, GELU, SELU, and ReLU identify promising training configurations, particularly the tested GELU/Huber setup. Because these hyperparameter comparisons were exploratory and not fully controlled, they should be treated as preliminary findings.
The results support the practical usefulness of symbolic distillation for approximating the outputs of this model. They do not demonstrate independent discovery of the exact mathematical rule, nor do they establish the internal mechanisms of the neural network. Confirming generalization on verified unseen data and comparing several independently trained checkpoints are important next steps.
12. Model weights and inference
The trained checkpoint is a separate binary artifact and is not embedded in this article. The file used in the experiments is named PowAI_weights_with_HuberLoss (2).pth. For public release, the author should host the checkpoint separately and insert the download URL here.
The inference program is provided separately as powai_inference.py. It loads the checkpoint on the CPU, applies the documented input normalization, reverses the output scaling, and compares the prediction with Python's complex-power result.
Install the dependencies in the environment used for the model:
python -m pip install torch numpy
Example:
python powai_inference.py --base 2.0 --exponent 0.5
Appendix A. Standalone CPU inference code
The following code is also supplied as the separate file powai_inference.py.
#!/usr/bin/env python
# -*- coding: utf-8 -*-
"""CPU inference for PowAI.
Usage:
python powai_inference.py --base 2.0 --exponent 0.5
python powai_inference.py --base -2.0 --exponent 0.5
python powai_inference.py --model "F:\\Downloads\\PowAI_weights_with_HuberLoss (2).pth" --base 2 --exponent 1.5
Requires: PyTorch and NumPy.
"""
import argparse
from pathlib import Path
import sys
import numpy as np
import torch
from torch import nn
DEFAULT_MODEL_PATH = r"F:\Downloads\PowAI_weights_with_HuberLoss (2).pth"
BASE_MAX = 5.0
EXP_MAX = 3.0
Y_SCALE = BASE_MAX ** EXP_MAX # 125.0
class PowAI(nn.Module):
def __init__(self):
super().__init__()
self.net = nn.Sequential(
nn.Linear(2, 324),
nn.GELU(),
nn.LayerNorm(324),
nn.GELU(),
nn.Linear(324, 324),
nn.GELU(),
nn.LayerNorm(324),
nn.GELU(),
nn.Linear(324, 324),
nn.GELU(),
nn.LayerNorm(324),
nn.GELU(),
nn.Linear(324, 2),
)
def forward(self, x):
return self.net(x)
def _extract_state_dict(checkpoint):
if isinstance(checkpoint, nn.Module):
return checkpoint.state_dict()
if not isinstance(checkpoint, dict):
raise TypeError("The checkpoint is not a state_dict or a supported checkpoint dictionary.")
for key in ("state_dict", "model_state_dict", "model", "weights", "model_weights"):
value = checkpoint.get(key)
if isinstance(value, nn.Module):
return value.state_dict()
if isinstance(value, dict):
checkpoint = value
break
keys = list(checkpoint.keys())
for prefix in ("module.", "model."):
if keys and all(k.startswith(prefix) for k in keys):
checkpoint = {k[len(prefix):]: v for k, v in checkpoint.items()}
keys = list(checkpoint.keys())
return checkpoint
def load_model(model_path):
model_path = Path(model_path).expanduser()
if not model_path.is_file():
raise FileNotFoundError(
f"Checkpoint not found: {model_path}\n"
"Pass the correct path using --model."
)
try:
checkpoint = torch.load(
str(model_path), map_location="cpu", weights_only=True
)
except TypeError:
checkpoint = torch.load(str(model_path), map_location="cpu")
except Exception as safe_error:
# This fallback is for a trusted checkpoint created by the user.
try:
checkpoint = torch.load(
str(model_path), map_location="cpu", weights_only=False
)
except Exception as fallback_error:
raise RuntimeError(
f"Safe checkpoint loading failed: {safe_error}\n"
f"Fallback loading failed: {fallback_error}"
) from fallback_error
state = _extract_state_dict(checkpoint)
model = PowAI()
try:
model.load_state_dict(state, strict=True)
except RuntimeError as exc:
raise RuntimeError(
"The checkpoint does not match the documented PowAI architecture "
"(GELU, LayerNorm and three hidden Linear layers of width 324).\n"
f"PyTorch details: {exc}"
) from exc
model.cpu().eval()
torch.set_grad_enabled(False)
try:
torch.set_num_threads(min(8, torch.get_num_threads()))
except Exception:
pass
return model
def predict_powai(model, base, exponent):
"""Return the network prediction as (real, imaginary) in original units."""
x = np.array([[base / BASE_MAX, exponent / EXP_MAX]], dtype=np.float32)
with torch.inference_mode():
output = model(torch.from_numpy(x)).cpu().numpy()[0].astype(np.float64)
output *= Y_SCALE
return complex(float(output[0]), float(output[1]))
def main():
parser = argparse.ArgumentParser(description="Run PowAI inference on CPU.")
parser.add_argument("--model", default=DEFAULT_MODEL_PATH, help="Path to .pth/.pt checkpoint.")
parser.add_argument("--base", type=float, required=True, help="Real base a.")
parser.add_argument("--exponent", type=float, required=True, help="Real exponent b.")
args = parser.parse_args()
if args.base == 0.0 and args.exponent < 0.0:
parser.error("base=0 with a negative exponent is undefined.")
model = load_model(args.model)
predicted = predict_powai(model, args.base, args.exponent)
reference = complex(args.base) ** args.exponent
absolute_error = abs(predicted - reference)
reference_magnitude = abs(reference)
relative_error = (
absolute_error / reference_magnitude * 100.0
if reference_magnitude > 1e-10 else float("nan")
)
print("PowAI CPU inference")
print(f"Checkpoint: {args.model}")
print(f"Input: a={args.base}, b={args.exponent}")
print(f"Predicted result: {predicted}")
print(f"Python complex-power reference: {reference}")
print(f"Absolute complex error: {absolute_error:.10g}")
if np.isfinite(relative_error):
print(f"Relative error: {relative_error:.6g}%")
else:
print("Relative error: not reported (reference magnitude is near zero).")
if __name__ == "__main__":
try:
main()
except Exception as exc:
print(f"ERROR: {type(exc).__name__}: {exc}", file=sys.stderr)
raise SystemExit(1)
Author: ALEXFLR