data-science-ml-ai999x — offline, verified data science and machine learning solver with step-by-step answers

Install · Capability guide · Evaluation · AI999X on Hugging Face · Website · Support: Buy Me a Coffee

License: Apache-2.0 · Author: Ibnul Jaif Farabi (creator and engineer) · Organization: AI999X · Version 1.0.3

Field: Data Science & Machine Learning · Tasks: Question Answering, Token Classification, Text Classification, Tabular Classification, Tabular Regression, Sentence Similarity, Feature Extraction

data-science-ml-ai999x is an offline, deterministic Python solver that answers data science and machine learning questions with verified results from 107 certified capabilities, gives 711 documented answers identically on Windows and Linux, and refuses rather than guesses when an answer cannot be verified.

data-science-ml-ai999x is a compact, deterministic data science and machine learning engine for Python. It reads a question in plain English (or a typed call), picks a certified data science and machine learning capability, computes the answer exactly, checks it, and returns the result with its steps, units, LaTeX and assumptions — or an honest refusal that says what is missing. It runs fully offline on any CPU, needs no GPU, no API key and no internet, and is built to give zero wrong answers: when it is not sure, it refuses instead of guessing.

Highlights

• Verified, not generated. Every answer is computed by a certified data science and machine learning capability and checked (substituted back, dimension-checked or compared with an independent computation) before it is returned.
• Zero wrong answers by design. Seven explicit outcomes — SOLVED, NEEDS_INPUT, AMBIGUOUS, INVALID_INPUT, OUT_OF_SCOPE, UNRECOGNIZED_FORMAT, INTERNAL_ERROR — never a silent guess. Every one of its 711 documented answers is reproduced exactly on every tested platform.
• Plain-English questions and a guaranteed typed interface. ask() reads documented question forms; solve(capability_id, **inputs) always solves a supported task with valid inputs.
• Explains itself. Steps, formulas in LaTeX, units, assumptions and the model used, ready for notebooks, tutoring apps and homework checkers.
• Small and portable. Pure Python (sympy, mpmath); identical answers on Windows and Linux (Python 3.10, 3.11, 3.12, 3.13, 3.14).
• Coverage: classification and regression metrics, loss functions and gradients, information theory, clustering, classical machine-learning algorithms, neural-network arithmetic and ranking metrics.

Model overview

Property	Value
Package	data-science-ml-ai999x 1.0.3 (import data_science_ml_ai999x)
Type	Deterministic neuro-symbolic data science and machine learning engine (rule-based reading + certified symbolic capabilities)
Capabilities	107 certified procedures
Knowledge pack	version 1.0.0
Golden answers	711 documented answers, identical on every tested platform
Platforms tested	Windows (Python 3.13), Linux x86-64 (Python 3.10, 3.11, 3.12, 3.13, 3.14)
Runtime	CPU only, offline; no GPU, no network, no API key
License	Apache-2.0
Creator / engineer	Ibnul Jaif Farabi
Organization	AI999X

Quickstart

pip install data-science-ml-ai999x
# or straight from this repository:
pip install https://huggingface.co/AI999X/data-science-ml-ai999x/resolve/main/dist/data_science_ml_ai999x-1.0.3-py3-none-any.whl

import data_science_ml_ai999x as data_science_ml

data_science_ml.ask("Binary metrics from counts: TP = 40, FP = 10, FN = 5, TN = 45").show()
# -> SOLVED — accuracy = 0.85, precision = 0.8, recall = 0.888889, specificity = 0.818182  (ml_binary_metrics_from_counts)
data_science_ml.ask("ROC AUC: y_true = [0, 1, 1, 0, 1], scores = [0.1, 0.9, 0.8, 0.3, 0.2]").show()
# -> SOLVED — auc = 0.833333, gini = 0.666667, n_pos = 3, n_neg = 2  (ml_roc_auc)
data_science_ml.ask("K-means clustering: X = [[1, 1], [1.5, 2], [3, 4], [5, 7], [3.5, 5], [4.5, 5], [3.5, 4.5]], C = [[1, 1], [5, 6]]").show()
# -> SOLVED — centroids = [[1.25, 1.5], [3.9, 5.1]], labels = [0, 0, 1, 1, 1, 1, 1], inertia = 8.525  (ml_kmeans)
data_science_ml.ask("Scaled dot-product attention: Q = [[1, 0], [0, 1]], K = [[1, 0], [0, 1], [1, 1]], V = [[1, 2], [3, 4], [5, 6]], causal = no").show()
# -> SOLVED — weights = [[0.40111209268, 0.19777581464, 0.40111209268], [0.19777581464, 0.40111209268, 0.40111209268]], output = [[3.0, 4.0], [3.40667255608, 4.40667255608]], output_sum = 14.8133  (ml_scaled_dot_product_attention)
data_science_ml.solve("ml_conv_output_size", n=224, kernel=7, stride=2, padding=3, dilation=1).show()
# -> SOLVED — output_size = 112, effective_kernel = 7  (ml_conv_output_size)
data_science_ml.ask("Who created you?").message
# -> data-science-ml-ai999x was created and engineered by Ibnul Jaif Farabi at AI999X.

Command line: data-science-ml-ai999x "<question>", data-science-ml-ai999x --capabilities. The outputs above were produced by running this exact code on data-science-ml-ai999x 1.0.3.

Find the right capability, and draw it

data_science_ml.search() finds which certified capability a question is about — across every AI999X model, so a question for another model points you to it. It runs ai999x-encoder, AI999X's own capability encoder (4.7M parameters, a 4.9 MB ONNX model, offline; the root model.safetensors of this repository): pip install "data-science-ml-ai999x[search]". ask(question, suggest=True) adds the same pointers to a question it does not answer; an answer itself never changes.

import data_science_ml_ai999x as data_science_ml

data_science_ml.search("Solve x^2 - 5x + 6 = 0", k=2)
# -> Roots of a quadratic equation (math-foundations-ai999x, 0.63); Roots of a cubic equation (math-foundations-ai999x, 0.59)

How every answer is verified

1. Reading — the question is turned into typed quantities (value, unit, kind, role) and one asked quantity; every stated number must be used or the question is refused.
2. Planning — only certified capabilities may answer; a capability applies only under the conditions the wording states (for example a pendulum, a liquid at rest, boiling).
3. Computing — exact symbolic or high-precision numeric evaluation.
4. Checking — dimensions, substitution back into the defining equation, and agreement between independent readings; any disagreement becomes a refusal, never a number.

Capabilities are certified before they ship: each must reproduce hand-worked reference cases, reject a deliberately wrong variant (a control), and leave every benchmark answer unchanged.

Worked examples

Each question was put to data-science-ml-ai999x; the answer is the one it returned, with the steps it took.

Binary classification metrics from confusion counts · Classification metrics

Binary metrics from counts: TP = 40, FP = 10, FN = 5, TN = 45

→ accuracy = 0.85, precision = 0.8, recall = 0.8888888889, specificity = 0.8181818182, f1 = 0.8421052632, npv = 0.9, balanced_accuracy = 0.8535353535, fpr = 0.1818181818, fnr = 0.1111111111, jaccard = 0.7272727273

Steps

• Binary classification metrics from confusion counts: TP = 40, FP = 10, FN = 5, TN = 45 → accuracy = 0.85, precision = 0.8, recall = 0.8888888889, specificity = 0.8181818182, f1 = 0.8421052632, npv = 0.9, balanced_accuracy = 0.8535353535, fpr = 0.1818181818, fnr = 0.1111111111, jaccard = 0.7272727273

ROC AUC from scores · Classification metrics

ROC AUC: y_true = [0, 1, 1, 0, 1], scores = [0.1, 0.9, 0.8, 0.3, 0.2]

→ auc = 0.8333333333, gini = 0.6666666667, n_pos = 3, n_neg = 2

Steps

• ROC AUC from scores: y_true = [0, 1, 1, 0, 1], scores = [0.1, 0.9, 0.8, 0.3, 0.2] → auc = 0.8333333333, gini = 0.6666666667, n_pos = 3, n_neg = 2

k-means clustering from given initial centroids · Classical algorithms

K-means clustering: X = [[1, 1], [1.5, 2], [3, 4], [5, 7], [3.5, 5], [4.5, 5], [3.5, 4.5]], C = [[1, 1], [5, 6]]

→ centroids = [[1.25, 1.5], [3.9, 5.1]], labels = [0, 0, 1, 1, 1, 1, 1], inertia = 8.525

Steps

• k-means clustering from given initial centroids: X = [1 1; 1.5 2; 3 4; 5 7; 3.5 5; 4.5 5; 3.5 4.5], C = [1 1; 5 6] → centroids = [[1.25, 1.5], [3.9, 5.1]], labels = [0, 0, 1, 1, 1, 1, 1], inertia = 8.525

Scaled dot-product attention · Neural networks

Scaled dot-product attention: Q = [[1, 0], [0, 1]], K = [[1, 0], [0, 1], [1, 1]], V = [[1, 2], [3, 4], [5, 6]], causal = no

→ weights = [[0.4011120927, 0.1977758146, 0.4011120927], [0.1977758146, 0.4011120927, 0.4011120927]], output = [[3, 4], [3.406672556, 4.406672556]], output_sum = 14.81334511

Steps

• Scaled dot-product attention: Q = [1 0; 0 1], K = [1 0; 0 1; 1 1], V = [1 2; 3 4; 5 6], causal = False → weights = [[0.4011120927, 0.1977758146, 0.4011120927], [0.1977758146, 0.4011120927, 0.4011120927]], output = [[3, 4], [3.406672556, 4.406672556]], output_sum = 14.81334511

Convolution output size · Neural networks

m.solve("ml_conv_output_size", n=224, kernel=7, stride=2, padding=3, dilation=1)

→ output_size = 112, effective_kernel = 7

Steps

• Convolution output size: n = 224, kernel = 7, stride = 2, padding = 3, dilation = 1 → output_size = 112, effective_kernel = 7

Adjusted mutual information of two clusterings · Clustering metrics

Adjusted mutual information: labels_true = [0, 0, 1, 1, 2, 2], labels_pred = [0, 0, 1, 1, 1, 1], method = arithmetic

→ ami = 0.6153846154, expected_mi = 0.266835672, mutual_information = 0.6365141683

Steps

• Adjusted mutual information of two clusterings: labels_true = [0, 0, 1, 1, 2, 2], labels_pred = [0, 0, 1, 1, 1, 1], method = arithmetic → ami = 0.6153846154, expected_mi = 0.266835672, mutual_information = 0.6365141683

Figures

Every point was computed by data-science-ml-ai999x.

ROC curve of a scored classifier

The ROC curve the package returns for 12 scored examples (labels 1, 1, 0, 1, 1, 0, 1, 0, 0, 1, 0, 0; scores 0.95, 0.9, 0.85, 0.8, 0.7, 0.65, 0.6, 0.5, 0.4, 0.35, 0.3, 0.1): true against false positive rate at each of its thresholds, with area under the curve 0.777778.

Confusion matrix of a three-class classifier

The confusion matrix the package returns for true labels [0, 1, 2, 2, 1, 0, 2, 1] and predictions [0, 2, 2, 2, 1, 0, 1, 1]: rows are the true classes 0, 1, 2, columns the predicted ones; 6 of 8 on the diagonal, accuracy 0.75.

k-means clusters and their centroids

k-means on 7 points from the starting centres [[1, 1], [5, 6]]: the package's final centroids (1.25, 1.5), (3.9, 5.1) (marked; arrows from the starting centres), cluster labels 1, 1, 2, 2, 2, 2, 2 and inertia 8.525.

Principal components: projection onto the first component

Principal components of 10 points: the package's first component (0.677873, 0.735179) through the mean of the points explains 96.3181 % of the variance (variance 1.28403 against 0.0490834); each point is joined to its projection at the score the package returns; the short arrow is two standard deviations (0.443095) along the second component.

Logistic regression: fitted probability and decision boundary

Logistic regression fitted by the package to x = 0.5, 1, 1.5, 2, 2.5, 3, 3.5, 4 with labels 0, 0, 1, 0, 1, 0, 1, 1 (C = 1): weight 0.738776, intercept -1.66225, training accuracy 0.75; the curve is the package's predicted probability at 81 points; marked: the decision boundary x = 2.25, P = 0.5.

k-nearest neighbours of a query point

k-nearest-neighbour classification of the query point (3, 2.2) among 6 labelled points (k = 3, euclidean distance): the package's 3 nearest neighbours are points 2, 3, 5 at distances 1.28062, 1.56205, 2.00998 (dashed lines; the circle reaches the farthest of them); 2 of 3 votes give class 0.

Capabilities

107 certified procedures. A few of them:

• Procedures: 2-D convolution of an image with a kernel (deep-learning convention); Adam optimizer: one update; AdamW optimizer (decoupled weight decay): one update; Adjusted R^2; Adjusted mutual information of two clusterings; Agglomerative hierarchical clustering merge heights; Average precision and precision-recall AUC; Averaged precision, recall and F1 (multi-class); Backpropagation gradients of a one-hidden-layer network; Balanced accuracy (multi-class); Balanced class weights; Batch normalisation forward pass (training mode); Bernoulli naive Bayes prediction (binary features); Best decision-tree split (classification, Gini or entropy); Best regression-tree split (squared error); Bias-variance decomposition of predictions; Binary classification metrics from confusion counts; Binary confusion matrix from labels; Binary cross-entropy with logits and its gradient; Binary focal loss and its gradient; Binary log loss (cross-entropy); Brier score; Calinski-Harabasz index; Cohen's kappa (unweighted, linear or quadratic) …

data_science_ml_ai999x.templates() lists every question form ask() reads; the capability guide (PDF, every example executed) is included in this repository.

All 107 formulas

Show every formula, by topic

Classical algorithms

Agglomerative hierarchical clustering merge heights

$$d(k,i\cup j)=\sqrt{\frac{(n_i+n_k)d_{ki}^2+(n_j+n_k)d_{kj}^2-n_k{d}_{ij}^2}{n_i+n_j+n_k}}\text{ (Ward)}d(k,\; i\; \backslash cup\; j)\; =\; \backslash sqrt\{\backslash frac\{(n\_i\; +\; n\_k)d\_\{ki\}^2\; +\; (n\_j\; +\; n\_k)d\_\{kj\}^2\; -\; n\_k\; d\_\{ij\}^2\}\{n\_i\; +\; n\_j\; +\; n\_k\}\}\backslash \; \backslash text\{(Ward)\}$$

Bernoulli naive Bayes prediction (binary features)

$$P(x\mid c)=\prod _j{p}_{cj}^{x_j}(1-p_{cj}{)}^{1-x_j}P(x\; \backslash mid\; c)\; =\; \backslash prod\_j\; p\_\{cj\}^\{x\_j\}(1\; -\; p\_\{cj\})^\{1\; -\; x\_j\}$$

Best decision-tree split (classification, Gini or entropy)

$$\mathrm{\Delta }I=I(S)-\frac{\mathrm{\mid }{S}_L\mathrm{\mid }}{\mathrm{\mid }S\mathrm{\mid }}I(S_L)-\frac{\mathrm{\mid }{S}_R\mathrm{\mid }}{\mathrm{\mid }S\mathrm{\mid }}I(S_R)\backslash Delta\; I\; =\; I(S)\; -\; \backslash frac\{\backslash vert\; S\_L\backslash vert\; \}\{\backslash vert\; S\backslash vert\; \}I(S\_L)\; -\; \backslash frac\{\backslash vert\; S\_R\backslash vert\; \}\{\backslash vert\; S\backslash vert\; \}I(S\_R)$$

Best regression-tree split (squared error)

$$\mathrm{\Delta }=\frac{1}{n}[SSE(S)-SSE(S_L)-SSE(S_R)]\backslash Delta\; =\; \backslash frac\{1\}\{n\}\backslash left[SSE(S)\; -\; SSE(S\_L)\; -\; SSE(S\_R)\backslash right]$$

Distances and cosine similarity between two vectors

$$d_2=\sqrt{\sum _i(x_i-z_i{)}^2},\cos \theta =\frac{x\cdot z}{vertxvertvertzvert}d\_2\; =\; \backslash sqrt\{\backslash sum\_i\; (x\_i\; -\; z\_i)^2\},\backslash quad\; \backslash cos\backslash theta\; =\; \backslash frac\{x\; \backslash cdot\; z\}\{\backslash \backslash vert\; x\backslash \backslash vert\; \backslash \backslash vert\; z\backslash \backslash vert\; \}$$

Gaussian naive Bayes prediction

$$P(c\mid x)\propto P(c)\prod _j\mathcal{N}(x_j;{\mu }_{cj},{\sigma }_{cj}^2)P(c\; \backslash mid\; x)\; \backslash propto\; P(c)\backslash prod\_j\; \backslash mathcal\{N\}(x\_j;\backslash \; \backslash mu\_\{cj\},\; \backslash sigma^2\_\{cj\})$$

Multinomial naive Bayes prediction (word counts)

$${\widehat{\theta }}_{cj}=\frac{N_{cj}+\alpha }{N_c+\alpha d},\log P(c\mid x)=\log P(c)+\sum _j{x}_j\log {\widehat{\theta }}_{cj}+\text{const}\backslash hat\backslash theta\_\{cj\}\; =\; \backslash frac\{N\_\{cj\}\; +\; \backslash alpha\}\{N\_c\; +\; \backslash alpha\; d\},\backslash quad\; \backslash log\; P(c\; \backslash mid\; x)\; =\; \backslash log\; P(c)\; +\; \backslash sum\_j\; x\_j\; \backslash log\backslash hat\backslash theta\_\{cj\}\; +\; \backslash text\{const\}$$

One k-means iteration (assignment and update)

$$c(i)=\arg \underset{k}{\min }vert{x}_i-{\mu }_{k}ver{t}^2,{\mu }_k^{\mathrm{\prime }}=\frac{1}{\mathrm{\mid }{S}_k\mathrm{\mid }}\sum _{i\in S_k}{x}_{i}c(i)\; =\; \backslash arg\backslash min\_k\; \backslash \backslash vert\; x\_i\; -\; \backslash mu\_k\backslash \backslash vert\; ^2,\backslash quad\; \backslash mu\_k\text{'}\; =\; \backslash frac\{1\}\{\backslash vert\; S\_k\backslash vert\; \}\backslash sum\_\{i\; \backslash in\; S\_k\}\; x\_i$$

Polynomial kernel

$$k(x,z)=(\gamma x^{\mathrm{\top }}z+c_0{)}^{d}k(x,\; z)\; =\; (\backslash gamma\backslash ,\; x^\backslash top\; z\; +\; c\_0)^d$$

RBF (Gaussian) kernel matrix

$$K_{ij}=\exp (-\gamma vert{x}_i-x_{j}ver{t}^2)K\_\{ij\}\; =\; \backslash exp\backslash left(-\backslash gamma\backslash ,\backslash \backslash vert\; x\_i\; -\; x\_j\backslash \backslash vert\; ^2\backslash right)$$

k-means clustering from given initial centroids

$$\min \sum _k\sum _{i\in S_k}vert{x}_i-{\mu }_{k}ver{t}^2\backslash min\; \backslash sum\_k\; \backslash sum\_\{i\; \backslash in\; S\_k\}\; \backslash \backslash vert\; x\_i\; -\; \backslash mu\_k\backslash \backslash vert\; ^2$$

k-nearest-neighbour classification of a query point

$$\widehat{y}=\arg \underset{c}{\max }\sum _{i\in N_k(x)}\mathbb{1}(y_i=c)\backslash hat\; y\; =\; \backslash arg\backslash max\_c\; \backslash sum\_\{i\; \backslash in\; N\_k(x)\}\; \backslash mathbb\{1\}(y\_i\; =\; c)$$

k-nearest-neighbour regression of a query point

$$\widehat{y}=\frac{\sum _{i\in N_k}{w}_i{y}_i}{\sum _{i\in N_k}{w}_i},w_i=1\text{ or }1\mathrm{/}{d}_i\backslash hat\; y\; =\; \backslash frac\{\backslash sum\_\{i\; \backslash in\; N\_k\}\; w\_i\; y\_i\}\{\backslash sum\_\{i\; \backslash in\; N\_k\}\; w\_i\},\backslash quad\; w\_i\; =\; 1\; \backslash text\{\; or\; \}\; 1/d\_i$$

Classification metrics

Average precision and precision-recall AUC

$$\mathrm{A}\mathrm{P}=\sum _k(R_k-R_{k-1})P_k\backslash mathrm\{AP\}\; =\; \backslash sum\_k\; (R\_k\; -\; R\_\{k-1\})\backslash ,P\_k$$

Averaged precision, recall and F1 (multi-class)

$$F_1^{\mathrm{m}\mathrm{a}\mathrm{c}\mathrm{r}\mathrm{o}}=\frac{1}{K}\sum _k\frac{2P_k{R}_k}{P_k+R_k}F\_1^\{\backslash mathrm\{macro\}\}\; =\; \backslash frac\{1\}\{K\}\backslash sum\_k\; \backslash frac\{2P\_kR\_k\}\{P\_k+R\_k\}$$

Balanced accuracy (multi-class)

$$\mathrm{B}\mathrm{A}=\frac{1}{K}\sum _k\frac{T{P}_k}{T{P}_k+F{N}_k}\backslash mathrm\{BA\}\; =\; \backslash frac\{1\}\{K\}\backslash sum\_\{k\}\; \backslash frac\{TP\_k\}\{TP\_k\; +\; FN\_k\}$$

Balanced class weights

$$w_c=\frac{n}{K{n}_c}w\_c\; =\; \backslash frac\{n\}\{K\backslash ,\; n\_c\}$$

Binary classification metrics from confusion counts

$$P=\frac{TP}{TP+FP},R=\frac{TP}{TP+FN},F_1=\frac{2PR}{P+R}=\frac{2TP}{2TP+FP+FN}P\; =\; \backslash frac\{TP\}\{TP+FP\},\backslash ;\; R\; =\; \backslash frac\{TP\}\{TP+FN\},\backslash ;\; F\_1\; =\; \backslash frac\{2PR\}\{P+R\}\; =\; \backslash frac\{2TP\}\{2TP+FP+FN\}$$

Binary confusion matrix from labels

$$\mathrm{a}\mathrm{c}\mathrm{c}\mathrm{u}\mathrm{r}\mathrm{a}\mathrm{c}\mathrm{y}=\frac{TP+TN}{TP+FP+FN+TN}\backslash mathrm\{accuracy\}\; =\; \backslash frac\{TP\; +\; TN\}\{TP\; +\; FP\; +\; FN\; +\; TN\}$$

Binary log loss (cross-entropy)

$$\mathcal{L}=-\frac{1}{n}\sum _i[y_i\ln p_i+(1-y_i)\ln (1-p_i)]\backslash mathcal\{L\}\; =\; -\backslash frac\{1\}\{n\}\backslash sum\_i\; \backslash left[y\_i\; \backslash ln\; p\_i\; +\; (1-y\_i)\backslash ln(1-p\_i)\backslash right]$$

Brier score

$$\mathrm{B}\mathrm{S}=\frac{1}{n}\sum _i(p_i-y_i{)}^2\backslash mathrm\{BS\}\; =\; \backslash frac\{1\}\{n\}\backslash sum\_i\; (p\_i\; -\; y\_i)^2$$

Cohen's kappa (unweighted, linear or quadratic)

$$\kappa =1-\frac{\sum _{ij}{w}_{ij}{O}_{ij}}{\sum _{ij}{w}_{ij}{E}_{ij}},E_{ij}=\frac{r_i{c}_j}{n}\backslash kappa\; =\; 1\; -\; \backslash frac\{\backslash sum\_\{ij\}\; w\_\{ij\}\; O\_\{ij\}\}\{\backslash sum\_\{ij\}\; w\_\{ij\}\; E\_\{ij\}\},\backslash quad\; E\_\{ij\}\; =\; \backslash frac\{r\_i\; c\_j\}\{n\}$$

Confusion counts, precision and recall at a score threshold

$${\widehat{y}}_i=\mathbb{1}(s_i\ge t)\backslash hat\; y\_i\; =\; \backslash mathbb\{1\}(s\_i\; \backslash ge\; t)$$

Expected calibration error

$$\mathrm{E}\mathrm{C}\mathrm{E}=\sum _b\frac{n_b}{n}\mid {\bar{y}}_b-{\bar{p}}_b\mid \backslash mathrm\{ECE\}\; =\; \backslash sum\_b\; \backslash frac\{n\_b\}\{n\}\backslash left\backslash vert\; \backslash bar\; y\_b\; -\; \backslash bar\; p\_b\backslash right\backslash vert$$

F-beta score

$$F_{\beta }=\frac{(1+{\beta }^2)PR}{{\beta }^{2}P+R}F\_\backslash beta\; =\; \backslash frac\{(1+\backslash beta^2)\backslash ,P\; R\}\{\backslash beta^2\; P\; +\; R\}$$

Hamming loss and subset accuracy (multi-label)

$$\mathrm{H}\mathrm{L}=\frac{1}{nL}\sum _{i,j}\mathbb{1}(Y_{ij}\mathrm{\ne }{\widehat{Y}}_{ij})\backslash mathrm\{HL\}\; =\; \backslash frac\{1\}\{nL\}\backslash sum\_\{i,j\}\; \backslash mathbb\{1\}(Y\_\{ij\}\; \backslash neq\; \backslash hat\; Y\_\{ij\})$$

Matthews correlation coefficient

$$\mathrm{M}\mathrm{C}\mathrm{C}=\frac{TPTN-FPFN}{\sqrt{(TP+FP)(TP+FN)(TN+FP)(TN+FN)}}\backslash mathrm\{MCC\}\; =\; \backslash frac\{TP\backslash ,TN\; -\; FP\backslash ,FN\}\{\backslash sqrt\{(TP+FP)(TP+FN)(TN+FP)(TN+FN)\}\}$$

Multi-class confusion matrix

$$C_{ij}=\mathrm{\#}\{n:y_n=c_i,{\widehat{y}}_n=c_j\}C\_\{ij\}\; =\; \backslash \#\backslash \{n\; :\; y\_n\; =\; c\_i,\backslash \; \backslash hat\; y\_n\; =\; c\_j\backslash \}$$

Multi-class log loss (categorical cross-entropy)

$$\mathcal{L}=-\frac{1}{n}\sum _i\ln P_{i,y_i}\backslash mathcal\{L\}\; =\; -\backslash frac\{1\}\{n\}\backslash sum\_i\; \backslash ln\; P\_\{i,\; y\_i\}$$

Per-class precision, recall, F1 and support

$$P_k=\frac{T{P}_k}{T{P}_k+F{P}_k},R_k=\frac{T{P}_k}{T{P}_k+F{N}_k}P\_k\; =\; \backslash frac\{TP\_k\}\{TP\_k\; +\; FP\_k\},\backslash quad\; R\_k\; =\; \backslash frac\{TP\_k\}\{TP\_k\; +\; FN\_k\}$$

ROC AUC from scores

$$\mathrm{A}\mathrm{U}\mathrm{C}=\frac{1}{n_{+}{n}_{-}}\sum _{i\in +}\sum _{j\in -}[\mathbb{1}(s_i>s_j)+\frac{1}{2}\mathbb{1}(s_i=s_j)]\backslash mathrm\{AUC\}\; =\; \backslash frac\{1\}\{n\_+\; n\_-\}\backslash sum\_\{i\; \backslash in\; +\}\backslash sum\_\{j\; \backslash in\; -\}\backslash left[\backslash mathbb\{1\}(s\_i\; >\; s\_j)\; +\; \backslash tfrac12\; \backslash mathbb\{1\}(s\_i\; =\; s\_j)\backslash right]$$

ROC curve points

$$\mathrm{T}\mathrm{P}\mathrm{R}(t)=\frac{TP(t)}{P},\mathrm{F}\mathrm{P}\mathrm{R}(t)=\frac{FP(t)}{N}\backslash mathrm\{TPR\}(t)\; =\; \backslash frac\{TP(t)\}\{P\},\backslash \; \backslash mathrm\{FPR\}(t)\; =\; \backslash frac\{FP(t)\}\{N\}$$

Top-k accuracy

$$\mathrm{t}\mathrm{o}\mathrm{p}\text{-}k=\frac{1}{n}\sum _i\mathbb{1}[{\mathrm{r}\mathrm{a}\mathrm{n}\mathrm{k}}_i(y_i)\le k]\backslash mathrm\{top\}\backslash text\{-\}k\; =\; \backslash frac\{1\}\{n\}\backslash sum\_i\; \backslash mathbb\{1\}\backslash left[\backslash mathrm\{rank\}\_i(y\_i)\; \backslash le\; k\backslash right]$$

Clustering metrics

Adjusted mutual information of two clusterings

$$\mathrm{A}\mathrm{M}\mathrm{I}=\frac{I-E[I]}{\mathrm{m}\mathrm{e}\mathrm{a}\mathrm{n}(H(U),H(V))-E[I]}\backslash mathrm\{AMI\}\; =\; \backslash frac\{I\; -\; E[I]\}\{\backslash mathrm\{mean\}(H(U),\; H(V))\; -\; E[I]\}$$

Calinski-Harabasz index

$$CH=\frac{\mathrm{t}\mathrm{r}(B)}{\mathrm{t}\mathrm{r}(W)}\cdot \frac{n-K}{K-1}CH\; =\; \backslash frac\{\backslash mathrm\{tr\}(B)\}\{\backslash mathrm\{tr\}(W)\}\backslash cdot\backslash frac\{n\; -\; K\}\{K\; -\; 1\}$$

Davies-Bouldin index

$$DB=\frac{1}{K}\sum _k\underset{m\mathrm{\ne }k}{\max }\frac{S_k+S_m}{vert{c}_k-c_{m}vert}DB\; =\; \backslash frac\{1\}\{K\}\backslash sum\_k\; \backslash max\_\{m\; \backslash ne\; k\}\backslash frac\{S\_k\; +\; S\_m\}\{\backslash \backslash vert\; c\_k\; -\; c\_m\backslash \backslash vert\; \}$$

Homogeneity, completeness and V-measure

$$h=1-\frac{H(C\mid K)}{H(C)},c=1-\frac{H(K\mid C)}{H(K)},V_{\beta }=\frac{(1+\beta )hc}{\beta h+c}h\; =\; 1\; -\; \backslash frac\{H(C\; \backslash mid\; K)\}\{H(C)\},\backslash quad\; c\; =\; 1\; -\; \backslash frac\{H(K\; \backslash mid\; C)\}\{H(K)\},\backslash quad\; V\_\backslash beta\; =\; \backslash frac\{(1+\backslash beta)hc\}\{\backslash beta\; h\; +\; c\}$$

Normalised mutual information of two clusterings

$$\mathrm{N}\mathrm{M}\mathrm{I}=\frac{I(U,V)}{\mathrm{m}\mathrm{e}\mathrm{a}\mathrm{n}(H(U),H(V))}\backslash mathrm\{NMI\}\; =\; \backslash frac\{I(U,\; V)\}\{\backslash mathrm\{mean\}(H(U),\; H(V))\}$$

Rand index, adjusted Rand index and Fowlkes-Mallows index

$$\mathrm{A}\mathrm{R}\mathrm{I}=\frac{\sum _{ij}\left(\genfrac{}{}{0px}{}{n_{ij}}{2}\right)-\frac{\sum _i\left(\genfrac{}{}{0px}{}{a_i}{2}\right)\sum _j\left(\genfrac{}{}{0px}{}{b_j}{2}\right)}{\left(\genfrac{}{}{0px}{}{n}{2}\right)}}{\frac{1}{2}[\sum _i\left(\genfrac{}{}{0px}{}{a_i}{2}\right)+\sum _j\left(\genfrac{}{}{0px}{}{b_j}{2}\right)]-\frac{\sum _i\left(\genfrac{}{}{0px}{}{a_i}{2}\right)\sum _j\left(\genfrac{}{}{0px}{}{b_j}{2}\right)}{\left(\genfrac{}{}{0px}{}{n}{2}\right)}}\backslash mathrm\{ARI\}\; =\; \backslash frac\{\backslash sum\_\{ij\}\backslash binom\{n\_\{ij\}\}\{2\}\; -\; \backslash frac\{\backslash sum\_i\backslash binom\{a\_i\}\{2\}\backslash sum\_j\backslash binom\{b\_j\}\{2\}\}\{\backslash binom\{n\}\{2\}\}\}\{\backslash frac12\backslash left[\backslash sum\_i\backslash binom\{a\_i\}\{2\}\; +\; \backslash sum\_j\backslash binom\{b\_j\}\{2\}\backslash right]\; -\; \backslash frac\{\backslash sum\_i\backslash binom\{a\_i\}\{2\}\backslash sum\_j\backslash binom\{b\_j\}\{2\}\}\{\backslash binom\{n\}\{2\}\}\}$$

Silhouette score

$$s_i=\frac{b_i-a_i}{\max (a_i,b_i)}s\_i\; =\; \backslash frac\{b\_i\; -\; a\_i\}\{\backslash max(a\_i,\; b\_i)\}$$

Dimensionality reduction

Principal component analysis (explained variance and first component)

$$\mathrm{\Sigma }=\frac{1}{n-1}(X-\bar{X}{)}^{\mathrm{\top }}(X-\bar{X})=V\mathrm{\Lambda }{V}^{\mathrm{\top }}\backslash Sigma\; =\; \backslash frac\{1\}\{n-1\}(X\; -\; \backslash bar\; X)^\backslash top\; (X\; -\; \backslash bar\; X)\; =\; V\; \backslash Lambda\; V^\backslash top$$

Gradients

Logistic regression loss and gradient (L2-regularised)

$$L=\frac{1}{n}\sum _i[\log (1+e^{z_i})-y_i{z}_i]+\frac{\lambda }{2}vertwver{t}^2,{\mathrm{\nabla }}_{w}L=\frac{1}{n}{X}^{\mathrm{\top }}(\sigma (z)-y)+\lambda wL\; =\; \backslash frac\{1\}\{n\}\backslash sum\_i\; \backslash left[\backslash log(1\; +\; e^\{z\_i\})\; -\; y\_i\; z\_i\backslash right]\; +\; \backslash frac\{\backslash lambda\}\{2\}\backslash \backslash vert\; w\backslash \backslash vert\; ^2,\backslash quad\; \backslash nabla\_w\; L\; =\; \backslash frac\{1\}\{n\}X^\backslash top(\backslash sigma(z)\; -\; y)\; +\; \backslash lambda\; w$$

Mean squared error loss and gradient of a linear model

$$L=\frac{1}{n}vertXw+b-yver{t}^2,{\mathrm{\nabla }}_{w}L=\frac{2}{n}{X}^{\mathrm{\top }}(Xw+b-y)L\; =\; \backslash frac\{1\}\{n\}\backslash \backslash vert\; Xw\; +\; b\; -\; y\backslash \backslash vert\; ^2,\backslash quad\; \backslash nabla\_w\; L\; =\; \backslash frac\{2\}\{n\}X^\backslash top(Xw\; +\; b\; -\; y)$$

One gradient-descent step for linear regression

$$w\leftarrow w-\eta {\mathrm{\nabla }}_{w}L,b\leftarrow b-\eta \frac{\mathrm{\partial }L}{\mathrm{\partial }b}w\; \backslash leftarrow\; w\; -\; \backslash eta\; \backslash nabla\_w\; L,\backslash quad\; b\; \backslash leftarrow\; b\; -\; \backslash eta\; \backslash frac\{\backslash partial\; L\}\{\backslash partial\; b\}$$

Information theory

Gini impurity decrease of a split

$$\mathrm{\Delta }G=G(S)-\sum _v\frac{\mathrm{\mid }{S}_v\mathrm{\mid }}{\mathrm{\mid }S\mathrm{\mid }}G(S_v)\backslash Delta\; G\; =\; G(S)\; -\; \backslash sum\_v\; \backslash frac\{\backslash vert\; S\_v\backslash vert\; \}\{\backslash vert\; S\backslash vert\; \}G(S\_v)$$

Information gain and gain ratio of a split

$$IG=H(Y)-\sum _v\frac{\mathrm{\mid }{S}_v\mathrm{\mid }}{\mathrm{\mid }S\mathrm{\mid }}H(Y\mid X=v),GR=\frac{IG}{H(X)}IG\; =\; H(Y)\; -\; \backslash sum\_v\; \backslash frac\{\backslash vert\; S\_v\backslash vert\; \}\{\backslash vert\; S\backslash vert\; \}\; H(Y\; \backslash mid\; X\; =\; v),\backslash quad\; GR\; =\; \backslash frac\{IG\}\{H(X)\}$$

Joint, marginal and conditional entropy and mutual information

$$H(Y\mid X)=H(X,Y)-H(X),I(X;Y)=\sum _{ij}{P}_{ij}\log \frac{P_{ij}}{P_i{P}_j}H(Y\; \backslash mid\; X)\; =\; H(X,\; Y)\; -\; H(X),\backslash quad\; I(X;Y)\; =\; \backslash sum\_\{ij\}\; P\_\{ij\}\backslash log\backslash frac\{P\_\{ij\}\}\{P\_i\; P\_j\}$$

KL divergence, Jensen-Shannon divergence and cross-entropy of discrete distributions

$$D_{KL}(pvertq)=\sum _i{p}_i\log \frac{p_i}{q_i},H(p,q)=H(p)+D_{KL}(pvertq)D\_\{KL\}(p\; \backslash \backslash vert\; q)\; =\; \backslash sum\_i\; p\_i\; \backslash log\backslash frac\{p\_i\}\{q\_i\},\backslash quad\; H(p,\; q)\; =\; H(p)\; +\; D\_\{KL\}(p\; \backslash \backslash vert\; q)$$

Mutual information between two label sequences

$$I(X;Y)=\sum _{ij}\frac{n_{ij}}{n}\log \frac{n{n}_{ij}}{a_i{b}_j}I(X;Y)\; =\; \backslash sum\_\{ij\}\; \backslash frac\{n\_\{ij\}\}\{n\}\backslash log\backslash frac\{n\backslash ,n\_\{ij\}\}\{a\_i\; b\_j\}$$

Node impurity from class counts (entropy, Gini, misclassification)

$$G=1-\sum _k{p}_k^2,H=-\sum _k{p}_k{\log }_2{p}_{k}G\; =\; 1\; -\; \backslash sum\_k\; p\_k^2,\backslash quad\; H\; =\; -\backslash sum\_k\; p\_k\; \backslash log\_2\; p\_k$$

Shannon entropy and perplexity of a distribution

$$H(p)=-\sum _i{p}_i\log p_{i}H(p)\; =\; -\backslash sum\_i\; p\_i\; \backslash log\; p\_i$$

Linear models

Lasso regression coefficients (stated alpha)

$$\underset{w,b}{\min }\frac{1}{2n}verty-Xw-bver{t}^2+\alpha vertwver{t}_1\backslash min\_\{w,b\}\backslash \; \backslash frac\{1\}\{2n\}\backslash \backslash vert\; y\; -\; Xw\; -\; b\backslash \backslash vert\; ^2\; +\; \backslash alpha\backslash \backslash vert\; w\backslash \backslash vert\; \_1$$

Linear SVM decision values, classes and margin

$$f(x)=w^{\mathrm{\top }}x+b,\text{margin}=\frac{2}{vertwvert}f(x)\; =\; w^\backslash top\; x\; +\; b,\backslash quad\; \backslash text\{margin\}\; =\; \backslash frac\{2\}\{\backslash \backslash vert\; w\backslash \backslash vert\; \}$$

Logistic regression fit (L2, stated C)

$$\underset{w,b}{\min }\frac{1}{2}vertwver{t}^2+C\sum _i\log (1+e^{-{\tilde{y}}_i(w^{\mathrm{\top }}{x}_i+b)})\backslash min\_\{w,b\}\backslash \; \backslash tfrac12\backslash \backslash vert\; w\backslash \backslash vert\; ^2\; +\; C\backslash sum\_i\; \backslash log\backslash left(1\; +\; e^\{-\backslash tilde\; y\_i\; (w^\backslash top\; x\_i\; +\; b)\}\backslash right)$$

Logistic regression prediction from given weights

$$P(y=1\mid x)=\sigma (w^{\mathrm{\top }}x+b)P(y\; =\; 1\; \backslash mid\; x)\; =\; \backslash sigma(w^\backslash top\; x\; +\; b)$$

Perceptron training epoch

$$\text{if }{y}_i(w^{\mathrm{\top }}{x}_i+b)\le 0:w\leftarrow w+\eta y_i{x}_i,b\leftarrow b+\eta y_i\backslash text\{if\; \}\; y\_i(w^\backslash top\; x\_i\; +\; b)\; \backslash le\; 0:\backslash \; w\; \backslash leftarrow\; w\; +\; \backslash eta\; y\_i\; x\_i,\backslash \; b\; \backslash leftarrow\; b\; +\; \backslash eta\; y\_i$$

Ridge regression coefficients (stated alpha)

$$\widehat{w}=(X_c^{\mathrm{\top }}{X}_c+\alpha I{)}^{-1}{X}_c^{\mathrm{\top }}{y}_c\backslash hat\; w\; =\; (X\_c^\backslash top\; X\_c\; +\; \backslash alpha\; I)^\{-1\}\; X\_c^\backslash top\; y\_c$$

Loss functions and activations

Binary cross-entropy with logits and its gradient

$$\mathrm{\ell }(z,y)=\max (z,0)-zy+\log (1+e^{-\mathrm{\mid }z\mathrm{\mid }}),\frac{\mathrm{\partial }\mathrm{\ell }}{\mathrm{\partial }z}=\sigma (z)-y\backslash ell(z,\; y)\; =\; \backslash max(z,\; 0)\; -\; z\; y\; +\; \backslash log(1\; +\; e^\{-\backslash vert\; z\backslash vert\; \}),\backslash quad\; \backslash frac\{\backslash partial\; \backslash ell\}\{\backslash partial\; z\}\; =\; \backslash sigma(z)\; -\; y$$

Binary focal loss and its gradient

$$FL=-{\alpha }_t(1-p_t{)}^{\gamma }\log p_{t}FL\; =\; -\backslash alpha\_t\; (1\; -\; p\_t)^\backslash gamma\; \backslash log\; p\_t$$

Cross-entropy loss from logits and its gradient

$$L=-\log \frac{e^{z_y}}{\sum _j{e}^{z_j}},\frac{\mathrm{\partial }L}{\mathrm{\partial }{z}_j}=p_j-\mathbb{1}(j=y)L\; =\; -\backslash log\backslash frac\{e^\{z\_y\}\}\{\backslash sum\_j\; e^\{z\_j\}\},\backslash quad\; \backslash frac\{\backslash partial\; L\}\{\backslash partial\; z\_j\}\; =\; p\_j\; -\; \backslash mathbb\{1\}(j\; =\; y)$$

Cross-entropy with label smoothing

$$q_j=\frac{\epsilon }{K}+(1-\epsilon )\mathbb{1}(j=y),L=-\sum _j{q}_j\log \sigma (z{)}_{j}q\_j\; =\; \backslash frac\{\backslash varepsilon\}\{K\}\; +\; (1-\backslash varepsilon)\backslash mathbb\{1\}(j=y),\backslash quad\; L\; =\; -\backslash sum\_j\; q\_j\; \backslash log\; \backslash sigma(z)\_j$$

Hinge and squared hinge loss

$$\mathrm{\ell }(y,f)=\max (0,1-yf)\backslash ell(y,\; f)\; =\; \backslash max(0,\backslash \; 1\; -\; y\; f)$$

Huber, smooth L1 and pseudo-Huber loss

Log-softmax

$$\log \sigma (z{)}_i=z_i-m-\log \sum _j{e}^{z_j-m}\backslash log\backslash sigma(z)\_i\; =\; z\_i\; -\; m\; -\; \backslash log\backslash sum\_j\; e^\{z\_j\; -\; m\}$$

Multi-class hinge loss (Crammer-Singer and Weston-Watkins)

$${\mathrm{\ell }}_{CS}=\max (0,1-f_y+\underset{j\mathrm{\ne }y}{\max }{f}_j),{\mathrm{\ell }}_{WW}=\frac{1}{K}\sum _{j\mathrm{\ne }y}\max (0,1-f_y+f_j)\backslash ell\_\{CS\}\; =\; \backslash max\backslash left(0,\backslash \; 1\; -\; f\_y\; +\; \backslash max\_\{j\; \backslash ne\; y\}\; f\_j\backslash right),\backslash quad\; \backslash ell\_\{WW\}\; =\; \backslash frac\{1\}\{K\}\backslash sum\_\{j\; \backslash ne\; y\}\backslash max(0,\backslash \; 1\; -\; f\_y\; +\; f\_j)$$

Quantile (pinball) loss

$$L_{\alpha }(y,q)=\max \{\alpha (y-q),(\alpha -1)(y-q)\}L\_\backslash alpha(y,\; q)\; =\; \backslash max\backslash \{\backslash alpha\; (y\; -\; q),\backslash \; (\backslash alpha\; -\; 1)(y\; -\; q)\backslash \}$$

Sigmoid (logistic function) and its derivative

$$\sigma (z)=\frac{1}{1+e^{-z}},{\sigma }^{\mathrm{\prime }}(z)=\sigma (z)(1-\sigma (z))\backslash sigma(z)\; =\; \backslash frac\{1\}\{1\; +\; e^\{-z\}\},\backslash quad\; \backslash sigma\text{'}(z)\; =\; \backslash sigma(z)\backslash ,(1\; -\; \backslash sigma(z))$$

Softmax

$$\sigma (z{)}_i=\frac{e^{z_i-m}}{\sum _j{e}^{z_j-m}},m=\underset{j}{\max }{z}_j\backslash sigma(z)\_i\; =\; \backslash frac\{e^\{z\_i\; -\; m\}\}\{\backslash sum\_j\; e^\{z\_j\; -\; m\}\},\backslash \; m\; =\; \backslash max\_j\; z\_j$$

Softmax with temperature

$$p_i=\frac{e^{z_i\mathrm{/}T}}{\sum _j{e}^{z_j\mathrm{/}T}}p\_i\; =\; \backslash frac\{e^\{z\_i\; /\; T\}\}\{\backslash sum\_j\; e^\{z\_j\; /\; T\}\}$$

Triplet margin loss

$$L=\max (0,verta-pver{t}_2-verta-nver{t}_2+m)L\; =\; \backslash max\backslash left(0,\backslash \; \backslash \backslash vert\; a\; -\; p\backslash \backslash vert\; \_2\; -\; \backslash \backslash vert\; a\; -\; n\backslash \backslash vert\; \_2\; +\; m\backslash right)$$

Model selection

Bias-variance decomposition of predictions

$$E[(\widehat{f}-f{)}^2]=(E[\widehat{f}]-f{)}^2+E[(\widehat{f}-E[\widehat{f}]{)}^2]E[(\backslash hat\; f\; -\; f)^2]\; =\; (E[\backslash hat\; f]\; -\; f)^2\; +\; E[(\backslash hat\; f\; -\; E[\backslash hat\; f])^2]$$

Leave-one-out cross-validation of linear regression (PRESS)

$$\mathrm{P}\mathrm{R}\mathrm{E}\mathrm{S}\mathrm{S}=\sum _i{\left(\frac{e_i}{1-h_{ii}}\right)}^2,h_{ii}=x_i^{\mathrm{\top }}(X^{\mathrm{\top }}X{)}^{-1}{x}_i\backslash mathrm\{PRESS\}\; =\; \backslash sum\_i\; \backslash left(\backslash frac\{e\_i\}\{1\; -\; h\_\{ii\}\}\backslash right)^2,\backslash quad\; h\_\{ii\}\; =\; x\_i^\backslash top\; (X^\backslash top\; X)^\{-1\}\; x\_i$$

Train/test split sizes

$$n_{\mathrm{t}\mathrm{e}\mathrm{s}\mathrm{t}}=\lceil tn\rceil ,n_{\mathrm{t}\mathrm{r}\mathrm{a}\mathrm{i}\mathrm{n}}=n-n_{\mathrm{t}\mathrm{e}\mathrm{s}\mathrm{t}}n\_\{\backslash mathrm\{test\}\}\; =\; \backslash lceil\; t\backslash ,n\; \backslash rceil,\backslash quad\; n\_\{\backslash mathrm\{train\}\}\; =\; n\; -\; n\_\{\backslash mathrm\{test\}\}$$

k-fold cross-validated MSE of linear regression

$${\mathrm{C}\mathrm{V}}_k=\frac{1}{k}\sum _{i=1}^k{\mathrm{M}\mathrm{S}\mathrm{E}}_i\backslash mathrm\{CV\}\_k\; =\; \backslash frac\{1\}\{k\}\backslash sum\_\{i=1\}^k\; \backslash mathrm\{MSE\}\_i$$

k-fold cross-validation fold sizes

Neural networks

2-D convolution of an image with a kernel (deep-learning convention)

$$Y_{ij}=\sum _{a,b}{X}_{is+a,js+b}{K}_{ab}Y\_\{ij\}\; =\; \backslash sum\_\{a,b\}\; X\_\{is\; +\; a,\backslash \; js\; +\; b\}\backslash ,\; K\_\{ab\}$$

Backpropagation gradients of a one-hidden-layer network

$${\delta }_2=\frac{\mathrm{\partial }L}{\mathrm{\partial }{z}_2},{\mathrm{\nabla }}_{W_2}={\delta }_2{h}^{\mathrm{\top }},{\delta }_1=(W_2^{\mathrm{\top }}{\delta }_2)\odot {\varphi }^{\mathrm{\prime }}(z_1),{\mathrm{\nabla }}_{W_1}={\delta }_1{x}^{\mathrm{\top }}\backslash delta\_2\; =\; \backslash frac\{\backslash partial\; L\}\{\backslash partial\; z\_2\},\backslash \; \backslash nabla\_\{W\_2\}\; =\; \backslash delta\_2\; h^\backslash top,\backslash \; \backslash delta\_1\; =\; (W\_2^\backslash top\; \backslash delta\_2)\; \backslash odot\; \backslash phi\text{'}(z\_1),\backslash \; \backslash nabla\_\{W\_1\}\; =\; \backslash delta\_1\; x^\backslash top$$

Batch normalisation forward pass (training mode)

$$\widehat{x}=\frac{x-{\mu }_B}{\sqrt{{\sigma }_B^2+\epsilon }},y=\gamma \widehat{x}+\beta \backslash hat\; x\; =\; \backslash frac\{x\; -\; \backslash mu\_B\}\{\backslash sqrt\{\backslash sigma\_B^2\; +\; \backslash varepsilon\}\},\backslash quad\; y\; =\; \backslash gamma\; \backslash hat\; x\; +\; \backslash beta$$

Convolution output size

$$n_{\mathrm{o}\mathrm{u}\mathrm{t}}=\lfloor \frac{n+2p-d(k-1)-1}{s}\rfloor +1n\_\{\backslash mathrm\{out\}\}\; =\; \backslash left\backslash lfloor\; \backslash frac\{n\; +\; 2p\; -\; d(k-1)\; -\; 1\}\{s\}\; \backslash right\backslash rfloor\; +\; 1$$

Forward pass of a one-hidden-layer dense network

$$h=\varphi (W_{1}x+b_1),\widehat{y}=g(W_{2}h+b_2)h\; =\; \backslash phi(W\_1\; x\; +\; b\_1),\backslash quad\; \backslash hat\; y\; =\; g(W\_2\; h\; +\; b\_2)$$

Max pooling and average pooling

Multi-head attention (identity projections)

$$\mathrm{M}\mathrm{H}\mathrm{A}=[{\mathrm{h}\mathrm{e}\mathrm{a}\mathrm{d}}_1,\dots ,{\mathrm{h}\mathrm{e}\mathrm{a}\mathrm{d}}_h],{\mathrm{h}\mathrm{e}\mathrm{a}\mathrm{d}}_i=\mathrm{s}\mathrm{o}\mathrm{f}\mathrm{t}\mathrm{m}\mathrm{a}\mathrm{x}\text{\hspace{0.17em}⁣}\left(\frac{Q_i{K}_i^{\mathrm{\top }}}{\sqrt{d\mathrm{/}h}}\right)V_i\backslash mathrm\{MHA\}\; =\; [\backslash mathrm\{head\}\_1,\; \backslash dots,\; \backslash mathrm\{head\}\_h],\backslash quad\; \backslash mathrm\{head\}\_i\; =\; \backslash mathrm\{softmax\}\backslash !\backslash left(\backslash frac\{Q\_i\; K\_i^\backslash top\}\{\backslash sqrt\{d/h\}\}\backslash right)V\_i$$

Parameter count of a 2-D convolution layer

$$P=C_{\mathrm{o}\mathrm{u}\mathrm{t}}\frac{C_{\mathrm{i}\mathrm{n}}}{g}{k}_h{k}_w+C_{\mathrm{o}\mathrm{u}\mathrm{t}}P\; =\; C\_\{\backslash mathrm\{out\}\}\backslash ,\backslash frac\{C\_\{\backslash mathrm\{in\}\}\}\{g\}\backslash ,k\_h\; k\_w\; +\; C\_\{\backslash mathrm\{out\}\}$$

Parameter count of a dense (fully connected) network

$$P=\sum _l(n_{l-1}{n}_l+n_l)P\; =\; \backslash sum\_l\; (n\_\{l-1\}\; n\_l\; +\; n\_l)$$

Parameter count of a transformer encoder

$$P_{\mathrm{l}\mathrm{a}\mathrm{y}\mathrm{e}\mathrm{r}}=4d^2+4d+2d{d}_{ff}+d_{ff}+d+4dP\_\{\backslash mathrm\{layer\}\}\; =\; 4d^2\; +\; 4d\; +\; 2\; d\backslash ,d\_\{ff\}\; +\; d\_\{ff\}\; +\; d\; +\; 4d$$

Parameter count of an LSTM, GRU or simple RNN

$$P=Gh(n_{\mathrm{i}\mathrm{n}}+h)+2GhP\; =\; G\backslash ,h\backslash ,(n\_\{\backslash mathrm\{in\}\}\; +\; h)\; +\; 2Gh$$

Receptive field of stacked convolutions

Scaled dot-product attention

$$\mathrm{A}\mathrm{t}\mathrm{t}\mathrm{e}\mathrm{n}\mathrm{t}\mathrm{i}\mathrm{o}\mathrm{n}(Q,K,V)=\mathrm{s}\mathrm{o}\mathrm{f}\mathrm{t}\mathrm{m}\mathrm{a}\mathrm{x}\text{\hspace{0.17em}⁣}\left(\frac{Q{K}^{\mathrm{\top }}}{\sqrt{d_k}}\right)V\backslash mathrm\{Attention\}(Q,\; K,\; V)\; =\; \backslash mathrm\{softmax\}\backslash !\backslash left(\backslash frac\{QK^\backslash top\}\{\backslash sqrt\{d\_k\}\}\backslash right)V$$

Optimizers

Adam optimizer: one update

$$\widehat{m}=\frac{m_t}{1-{\beta }_1^t},\widehat{v}=\frac{v_t}{1-{\beta }_2^t},w_t=w_{t-1}-\eta \frac{\widehat{m}}{\sqrt{\widehat{v}}+\epsilon }\backslash hat\; m\; =\; \backslash frac\{m\_t\}\{1\; -\; \backslash beta\_1^t\},\backslash \; \backslash hat\; v\; =\; \backslash frac\{v\_t\}\{1\; -\; \backslash beta\_2^t\},\backslash quad\; w\_t\; =\; w\_\{t-1\}\; -\; \backslash eta\; \backslash frac\{\backslash hat\; m\}\{\backslash sqrt\{\backslash hat\; v\}\; +\; \backslash varepsilon\}$$

AdamW optimizer (decoupled weight decay): one update

$$w\leftarrow w(1-\eta \lambda ),w_t=w-\eta \frac{\widehat{m}}{\sqrt{\widehat{v}}+\epsilon }w\; \backslash leftarrow\; w(1\; -\; \backslash eta\backslash lambda),\backslash quad\; w\_t\; =\; w\; -\; \backslash eta\; \backslash frac\{\backslash hat\; m\}\{\backslash sqrt\{\backslash hat\; v\}\; +\; \backslash varepsilon\}$$

Cosine-annealing learning-rate schedule

$${\eta }_t={\eta }_{\min }+\frac{1}{2}({\eta }_{\max }-{\eta }_{\min })(1+\cos \frac{\pi t}{T})\backslash eta\_t\; =\; \backslash eta\_\{\backslash min\}\; +\; \backslash tfrac12(\backslash eta\_\{\backslash max\}\; -\; \backslash eta\_\{\backslash min\})\backslash left(1\; +\; \backslash cos\backslash frac\{\backslash pi\; t\}\{T\}\backslash right)$$

Linear warm-up followed by cosine decay

SGD with momentum: one update

$$b_t=\mu b_{t-1}+g_t,w_t=w_{t-1}-\eta b_{t}b\_t\; =\; \backslash mu\; b\_\{t-1\}\; +\; g\_t,\backslash quad\; w\_t\; =\; w\_\{t-1\}\; -\; \backslash eta\backslash ,\; b\_t$$

Step learning-rate schedule

$${\eta }_t={\eta }_0{\gamma }^{\lfloor t\mathrm{/}s\rfloor }\backslash eta\_t\; =\; \backslash eta\_0\backslash ,\; \backslash gamma^\{\backslash lfloor\; t\; /\; s\; \backslash rfloor\}$$

Preprocessing

Label encoding and one-hot encoding of categories

$$\mathrm{o}\mathrm{n}\mathrm{e}\mathrm{h}\mathrm{o}\mathrm{t}(v{)}_j=\mathbb{1}(v=c_j)\backslash mathrm\{onehot\}(v)\_j\; =\; \backslash mathbb\{1\}(v\; =\; c\_j)$$

Min-max scaling to [0, 1]

$$x^{\mathrm{\prime }}=\frac{x-\underset{j}{\min }}{\underset{j}{\max }-\underset{j}{\min }}x\text{'}\; =\; \backslash frac\{x\; -\; \backslash min\_j\}\{\backslash max\_j\; -\; \backslash min\_j\}$$

Robust scaling (median and interquartile range)

$$x^{\mathrm{\prime }}=\frac{x-\mathrm{m}\mathrm{e}\mathrm{d}\mathrm{i}\mathrm{a}\mathrm{n}}{Q_3-Q_1}x\text{'}\; =\; \backslash frac\{x\; -\; \backslash mathrm\{median\}\}\{Q\_3\; -\; Q\_1\}$$

Standardisation (z-scores of features)

$$z_{ij}=\frac{x_{ij}-{\bar{x}}_j}{s_j},s_j^2=\frac{1}{n}\sum _i(x_{ij}-{\bar{x}}_j{)}^{2}z\_\{ij\}\; =\; \backslash frac\{x\_\{ij\}\; -\; \backslash bar\; x\_j\}\{s\_j\},\backslash quad\; s\_j^2\; =\; \backslash frac\{1\}\{n\}\backslash sum\_i\; (x\_\{ij\}\; -\; \backslash bar\; x\_j)^2$$

Ranking metrics

Mean average precision (MAP) of rankings

$$\mathrm{A}\mathrm{P}=\frac{1}{\mathrm{\mid }\mathrm{r}\mathrm{e}\mathrm{l}\mathrm{\mid }}\sum _{k:{\mathrm{r}\mathrm{e}\mathrm{l}}_k=1}P\mathrm{@}k,\mathrm{M}\mathrm{A}\mathrm{P}=\frac{1}{Q}\sum _q{\mathrm{A}\mathrm{P}}_q\backslash mathrm\{AP\}\; =\; \backslash frac\{1\}\{\backslash vert\; \backslash mathrm\{rel\}\backslash vert\; \}\backslash sum\_\{k:\backslash \; \backslash mathrm\{rel\}\_k\; =\; 1\}\; P@k,\backslash quad\; \backslash mathrm\{MAP\}\; =\; \backslash frac\{1\}\{Q\}\backslash sum\_q\; \backslash mathrm\{AP\}\_q$$

Mean reciprocal rank (MRR)

$$\mathrm{M}\mathrm{R}\mathrm{R}=\frac{1}{Q}\sum _q\frac{1}{{\mathrm{r}\mathrm{a}\mathrm{n}\mathrm{k}}_q}\backslash mathrm\{MRR\}\; =\; \backslash frac\{1\}\{Q\}\backslash sum\_q\; \backslash frac\{1\}\{\backslash mathrm\{rank\}\_q\}$$

NDCG@k (normalised discounted cumulative gain)

$$\mathrm{D}\mathrm{C}\mathrm{G}\mathrm{@}k=\sum _{i=1}^k\frac{g_i}{{\log }_2(i+1)},\mathrm{N}\mathrm{D}\mathrm{C}\mathrm{G}\mathrm{@}k=\frac{\mathrm{D}\mathrm{C}\mathrm{G}\mathrm{@}k}{\mathrm{I}\mathrm{D}\mathrm{C}\mathrm{G}\mathrm{@}k}\backslash mathrm\{DCG\}@k\; =\; \backslash sum\_\{i=1\}^k\; \backslash frac\{g\_i\}\{\backslash log\_2(i\; +\; 1)\},\backslash quad\; \backslash mathrm\{NDCG\}@k\; =\; \backslash frac\{\backslash mathrm\{DCG\}@k\}\{\backslash mathrm\{IDCG\}@k\}$$

Precision@k and recall@k of a ranking

$$P\mathrm{@}k=\frac{\mathrm{\mid }\mathrm{r}\mathrm{e}\mathrm{l}\cap {\mathrm{t}\mathrm{o}\mathrm{p}}_k\mathrm{\mid }}{k},R\mathrm{@}k=\frac{\mathrm{\mid }\mathrm{r}\mathrm{e}\mathrm{l}\cap {\mathrm{t}\mathrm{o}\mathrm{p}}_k\mathrm{\mid }}{\mathrm{\mid }\mathrm{r}\mathrm{e}\mathrm{l}\mathrm{\mid }}P@k\; =\; \backslash frac\{\backslash vert\; \backslash mathrm\{rel\}\; \backslash cap\; \backslash mathrm\{top\}\_k\backslash vert\; \}\{k\},\backslash quad\; R@k\; =\; \backslash frac\{\backslash vert\; \backslash mathrm\{rel\}\; \backslash cap\; \backslash mathrm\{top\}\_k\backslash vert\; \}\{\backslash vert\; \backslash mathrm\{rel\}\backslash vert\; \}$$

Regression metrics

Adjusted R^2

$${\bar{R}}^2=1-(1-R^2)\frac{n-1}{n-p-1}\backslash bar\; R^2\; =\; 1\; -\; (1\; -\; R^2)\backslash frac\{n-1\}\{n-p-1\}$$

Mean Tweedie deviance (normal, Poisson, gamma, compound Poisson-gamma)

$$d_p(y,\mu )=2(\frac{y^{2-p}}{(1-p)(2-p)}-\frac{y{\mu }^{1-p}}{1-p}+\frac{{\mu }^{2-p}}{2-p})d\_p(y,\; \backslash mu)\; =\; 2\backslash left(\backslash frac\{y^\{2-p\}\}\{(1-p)(2-p)\}\; -\; \backslash frac\{y\backslash mu^\{1-p\}\}\{1-p\}\; +\; \backslash frac\{\backslash mu^\{2-p\}\}\{2-p\}\backslash right)$$

Mean absolute percentage error

$$\mathrm{M}\mathrm{A}\mathrm{P}\mathrm{E}=\frac{1}{n}\sum _i\frac{\mathrm{\mid }{y}_i-{\widehat{y}}_i\mathrm{\mid }}{\mathrm{\mid }{y}_i\mathrm{\mid }}\backslash mathrm\{MAPE\}\; =\; \backslash frac\{1\}\{n\}\backslash sum\_i\; \backslash frac\{\backslash vert\; y\_i\; -\; \backslash hat\; y\_i\backslash vert\; \}\{\backslash vert\; y\_i\backslash vert\; \}$$

Mean squared logarithmic error

$$\mathrm{M}\mathrm{S}\mathrm{L}\mathrm{E}=\frac{1}{n}\sum _i{(\ln (1+y_i)-\ln (1+{\widehat{y}}_i))}^2\backslash mathrm\{MSLE\}\; =\; \backslash frac\{1\}\{n\}\backslash sum\_i\; \backslash left(\backslash ln(1+y\_i)\; -\; \backslash ln(1+\backslash hat\; y\_i)\backslash right)^2$$

Regression metrics (MAE, MSE, RMSE, R^2, explained variance)

$$R^2=1-\frac{\sum (y_i-{\widehat{y}}_i{)}^2}{\sum (y_i-\bar{y}{)}^2},\mathrm{R}\mathrm{M}\mathrm{S}\mathrm{E}=\sqrt{\frac{1}{n}\sum (y_i-{\widehat{y}}_i{)}^2}R^2\; =\; 1\; -\; \backslash frac\{\backslash sum\; (y\_i\; -\; \backslash hat\; y\_i)^2\}\{\backslash sum\; (y\_i\; -\; \backslash bar\; y)^2\},\backslash quad\; \backslash mathrm\{RMSE\}\; =\; \backslash sqrt\{\backslash tfrac\{1\}\{n\}\backslash sum\; (y\_i\; -\; \backslash hat\; y\_i)^2\}$$

Out of scope

• Open-ended explanation, essay writing or tutoring dialogue: it is a solver, not a chatbot.
• Questions outside its certified data science and machine learning capabilities, and any question whose stated numbers it cannot all use — it refuses with the reason instead.
• High-stakes engineering, medical or legal decisions without independent review.

Limitations

• It answers what its certified capabilities cover (classification and regression metrics, loss functions and gradients, information theory, clustering, classical machine-learning algorithms, neural-network arithmetic and ranking metrics); anything else is refused with a reason.
• ask() reads documented question forms and common phrasings; unusual wording may be refused — use solve() for guaranteed results.
• Results are as correct as the certified relations and their stated conditions; a certified relation is a checked model, not a claim about every physical situation.

Citation

@software{data_science_ml_ai999x_1_0_3,
  title  = {data-science-ml-ai999x: an offline, verified data science and machine learning solver},
  author = {Farabi, Ibnul Jaif}, organization = {AI999X},
  year   = {2026}, version = {1.0.3},
  url    = {https://huggingface.co/AI999X/data-science-ml-ai999x}
}

Contact

Questions, bug reports and requests: open a discussion on https://huggingface.co/AI999X/data-science-ml-ai999x/discussions or visit https://ai999x.com.

Support the project

data-science-ml-ai999x is free and open source. If it helps you, you can support further development on Buy Me a Coffee.

Created and engineered by Ibnul Jaif Farabi · AI999X · ai999x.com