It's All Just Vectorization: einx, a Universal Notation for Tensor Operations
Authors: Florian Fervers
Sebastian Bullinger
Christoph Bodensteiner
Michael Arens
Venue: ICLR 2026 Oral (2026)
Keywords: Tensor notation
tensor programming
einx
einsum
einops
Primary area: other topics in machine learning (i.e., none of the above)
TL;DR: We introduce einx, a universal notation for tensor operations, and provide a Python implementation.

Abstract: Tensor operations represent a cornerstone of modern scientific computing. However, the Numpy-like notation adopted by predominant tensor frameworks is often difficult to read and write and prone to so-called shape errors, i.a., due to following inconsistent rules across a large, complex collection of operations. Alternatives like einsum and einops have gained popularity, but are inherently restricted to few operations and lack the generality required for a universal model of tensor programming.

To derive a better paradigm, we revisit vectorization as a function for transforming tensor operations, and use it to both lift lower-order operations to higher-order operations, and conceptually decompose higher-order operations to lower-order operations and their vectorization.

Building on the universal nature of vectorization, we introduce einx, a universal notation for tensor operations. It uses declarative, pointful expressions that are defined by analogy with loop notation and represent the vectorization of tensor operations. The notation reduces the large APIs of existing frameworks to a small set of elementary operations, applies consistent rules across all operations, and enables a clean, readable and writable representation in code. We provide an implementation of einx that is embedded in Python and integrates seamlessly with existing tensor frameworks: https://github.com/fferflo/einx

Published as a conference paper at ICLR 2026
IT's ALL JUST VECTORIZATION: EINX, A UNIVERSAL NOTATION FOR TENSOR OPERATIONS
Florian Fervers Sebastian Bullinger Christoph Bodensteiner Michael Arens
Fraunhofer IOSB
(firstname. las www.taattteeeissseee www.aameeeess.......uooaaaa.ea fww.nhofrruuuccccccc
ABSTRACT
Tensor operations represent cornerstone of modern scientific computing. How- ever, the Numpy-like notation adopted by predominant tensor frameworks is often difficult to read and write and prone to so-called shape enrors, i.a., due to follow- ing inconsistent rules across large, complex collection of operations. Alterna- tives like einsum and einops have gained popularity, but are inherently restricted to few operations and lack the generality required for universal model of tensor programming.
To derive better paradigm, revisit vectorization function for transforming tensor operations, and use it to both lift lower-order operations to higher-order operations, and conceptually decompose higher-order operations lower-order operations and their vectorization.
Building on the universal nature of vectorization, we introduce einx, universal notation for tensor operations. It uses declarative, pointful expressions that are defined by analogy with loop notation and represent the vectorization of tensor operations. The notation reduces the large APIs of existing frameworks to a small set of clementary operations, applies consistent rules actoss all operations, and enables a clean, readable and writable representation in code. We provide an implementation of einx that is embedded in Python and integrates seamlessly with existing tensor frameworks: https::/ //gitt/ com/fferrlo//einx
INTRODUCTION
Tensor operations constitute the foundation of modern deep learning and other domains of scientific computing. Tensors, i.e. n--dimensional arrays with a uniform element type, serve a medium for diverse types of data, including images, volumes, sequences of audio or text, activations a neural net, class probability scores, batches thereof. Tensor programs are commonly written in high-level Python with tensor operations that act as points of entry to low-level backend routines, thereby abstracting from the underlying hardware, memory representation and algorithms.
The widely used Numpy-like notation for expressing tensor operations in Python is followed by most predominant tensor frameworks such Numpy itself (Harris et al.) 2020), PyTorch (Paszke) 2019). Tensorflow (Abadi et al. (015). Jax (Bradbury et al. 2018). and MLX (Hannun et al.) (023). An operation Numpy-like notation operates on whole tensors and is expressed, e.g., as follows:
y np. sum(x, axis=1) Compute sum along rows of the matrix
In contrast, the following representation of the same operation in loop notation addresses tensor elements individually using indices, and invokes backend routine multiple times:
for in range (x. shape[0]):
for j in range (x. shape[1]):
y[i] += x[i, j] # For each row i, add element from column Loop notation enables a clearer, more general representation tensor operations through its pointful style. i.e. explicit use of indices. In contrast, Numpy-like notation follows point-free style (Paszke) al. 2)21) and compensates for the lack of index expressions by introducing varying mechanisms, including special parameters (e.g., axis), pure shape operations, as well as broadcasting, advanced indexing, and numerous function-specifiu rules. This often results in tensor programs that are diffi- cult read and write and where so-called shape errors occur frequently.

---

Published as a conference paper at ICLR 2026
Table 1: It's all just vectorization: einx reduces the large, inconsistent API of Numpy-like frame- works to few elementary operations and universal, declarative, pointful notation for expressing their vectorization. The table shows examples of different Numpy-like function calls that map the same elementary operation in einx and differ solely in their vectorization.
Numpy-like notation
einx notation (ours) einx.get_at
torch. take(x, y) torch. torch.gather(xx 0, y)
-> x, y)
y, dim=0) einx [inx.get_at("x]] b c, b c -> c", x, y)
torch. taakealllngddiiiiee. wwt..iinex_seeetteee 1, y)
einx. Linx.get_at("a [x] c, i -> a c", x, y)
tf. gather y, axis=1)
tf. tf.gather_nd(k: y)
sinx.get_at("....], b [i] > b", x,
tf. tf-gather_nddx, y, batch_dims-l x y[: np. newaxis]
einx.get_at("a a [i] >> b", x, y)
x[y[:, 0], y(:, 1]]
einx.get_at("Ixxy), [2] -> x, y)
einx.multiply( a
np.outer y)
einx.multiply(aa,, b b- y)
np.kron(x, y)
einx.multiply("a.... b... >> > b)
x, y)
scipy. linalg. khatri_rao(x, y)
einx.multiply("a c, b (a c", x, y)
np.man mo.matmul( (x, y)
einx.dot("a [b], [b]
x, y)
np.dot(x, y)
einx.dot [a], [a]
y)
np. tensordot y, axes=(0, 1))
einx.dot b, [a] > x,
np. ixner y)
einx.dot("x [a], y
x,
np. transpose (x, (0, 2, 1))
einx. b -> x)
np. squeezeeee axis=1)
einx.id("a 1
x)
np. np. broadcast__oox (2, 3, 4))
axis=1)
einx.id("a c -> a x)
einx.id("c -> 2 3 x)
np. reshape (x, (-1,))
einx.id(" ... >> (...)", x)
einx.id(" np. concatetteeet, y], axis=-1)
einx.id("s a, >>
b)", x, y) x, y)
np. (tack yl, axis=0)
-> (1 + 1)
np. meshgrid(x, y, indexing="ij") einx.id("a,
b, b", x, y)
Several alternatives to Numpy-like notation have been proposed, including approaches inspired by Einstein's 's summation convention such as einsum (Wiebel 201T) and its extension einops (Rogozh- nikov 2022a). frameworks that shift from positional to symbolic dimensions (Hoyer & Hamman, 2017 Vito 2023), and custom pointful languages (Vasilache et al., 2018 Paszke et al. (021).
Of these, only einsum and einops have found widespread adoption in the deep learning community, i.., due to being embedded in Python and compatible with the existing Numpy-based ecosystem.
However, einsum and einops are inherently restricted to a limited set of operations (c.f. Tab. P and lack the generality required for a universal model tensor operations. Furthermore, einops is defined in large parts ostensively, i.e. by examples such as a", reduction=" sum") Sum-reduction along b rather than by clear, explicit interpretation of how terms such as "a -> a" are to be understood.
y einops.rrduce.
"a
Our contributions are as follows:
(I) We revisit vectorization as a general function for transforming tensor operations. We use it as universal tool to lift lower-order operations higher-order operations, and conceptually decompose existing higher-order operations to few lower-order operations and their varying vectorization.
(2) We introduce a universal notation for tensor operations: einx. It represents the vectorization of operations using declarative, pointful expressions that are defined by analogy with loop notation. The einx notation (a) is applicable to any tensor operation, (b) provides a single set of rules across arbitrary operations, (c) is interpretable by analogy with loop notation, (d) allows for clean, read- able and writable representation of operations in code, and (c) reduces the complex Application Programming Interface (API) of Numpy-likk frameworks to few elementary operations (c.f. Tab. I
(3) We provide an implementation of einx for widely used tensor frameworks. Operations in einx are compiled to function calls of given tensor framework and thereby allow for seamless integration. The einx API contains functions for many commonly used operations, and the option to adapt new, custom operations to einx notation.
2

---

Published as a conference paper at ICLR 2026
Table 2: Support for classes of operations and vectorization in different types of cin*-notation. P: Permutation. F: Flattening. R: Repetition (i.e. output-only vectorization). C: Concatenation. Always and only flattens concatenated axes. #*; Coordinate axis must be first axis.
einx
einsum
einops: reduce, repeat,
einops: pack, unpack (2022b) (FC)*
eindex
Operation
(ours)
2011)
rearrange, einsum (2022a)
2023
Identity
PFRC
PFR
Scalar
PFR
P (only mul.)
P (only mul.)
Reduction
PFR
(only sum)
PF
Dot-product
PFR
P
P
Indexing
PFR
(P)**
Any other
PFR
RELATED WORKS
2.1 -INTATNTA FOR TENSOR OPERATIONS
introduces what is now known as the Einstein summation convention in the mathematical notation of tensor contractions (i.e. generalized matrix multiplica- tions) as follows (translated from German original): "It is therefore possible, without compromising clarity, to omit the summation signs. To that end, we introduce the rule: If an index appeurs twice in lerm of an expression, it is always to be summed over". As an example, in the following con- traction of and B the index j appears twice, and the summation sign over j may therefore be omitted:
Einstein Summation Einstein 1916)
A,, Ars
einsum After early proposals to express Einstein summation in code (Barr 1991 Ahlander 2002).
the most common approach in Python follows the np. einsum function introduced in Numpy by Wiebe (20IT). In its rarely used Einstein mode, einsum represents a tensor contraction by listing the indices from the corresponding Einstein summation expression in comma-delimites string:
np. ein.enmmmmmii.... jk", A, B) Matrix multiplication (as above)
Since the index j appears twice, summed over following Einstein's summation convention.
The function also introduces the more widely used non-Einstein mode where the expression is ex- tended using an arrow and output indices as shown below. Instead of Einstein's summation con- vention, it applies the following rule: All indices that appear only on the left side of the arrow are summed over. This allows expressing additional, commonly used operations, e.g.: np. np. einsum ( bjkk>bik", x, y) # Batched matmul: is summed over np. mp..innsum" ("ij->i", x) cum-reduction, is summed over
Lastly, the ellipsis ... is used to represent variable number of indices in an einsum expression.
einops Rogozhnikov (2022a) introduces einops which extends the non -in-teitt mode of einsum to support additional reduction operations using the same notation (e.g. max, mean), broadcasting along axes in the output ex expression, and multi-letter axis names. Its main novelty is the axis compo- sition which allows (un)flattening axes by wrapping them parentheses the string expression:
einops. rearrange "a b
(a b) c") # Reshape operation
ein* Several software packages propose variants of the above notation to support new opera- tions, including einindex (Malmaud 2018), einops. [pack unpack) (Rogozhnikov) (022b). eindex (Rogozhnikov] 2023), eindex (Mugal (023), eingather (Fleuret [2023) and einmesh (Jensen) 2025). However, these variants are incompatible with the original einsum and einops notation as well as with each other, and do not represent universal notation for tensor operations.
Despite their name, operations einops and these packages do not apply Einstein's is summation con- vention. Instead, they follow Wiebe's orthogonal choice to a string of axis names akin to indices in mathematical notation. Since the ein terminology has become associated with this notational style, we name approach einx, but avoid claims of it being "Einstein-like" or "Einstein-inspired".
einx We introduce einx, a universal notation for tensor operations that follows a single set of notational rules across any given operation (c.f. Tab. 2). It defined by analogy with loop notation, which allows for an explicit interpretation of expressions such as "i j -> i". The notation is compatible with einops notation for the set operations also supported by einops.
3

---

Published as a conference paper at ICLR 2026
2.2
OTHER NOTATIONS FOR TENSOR OPERATIONS
Named tensors Several authors (Hoyer & Hamman 2017 Chen 2017 Hall et al. 2022 De Vito
2023 Johnson 2024) propose to annotate tensors with symbolic axis names, resulting in so-called named tensors. Named tensors address the complexity of Numpy-like notation by implicitly vector- izing operations along matching symbolic axes of the argument tensors. However, named tensors also do not self-document tensor shapes, require renaming of axes in tensor programs, and do not integrate seamlessly with the scientific Python ecosystem which operates on positional tensors.
Importantly, the usage of named tensors is complementary to the usage of eii*-notation Operations may accept named tensors akin to positional tensors by matching the string expression against the symbolic axis names of the tensor, rather than in addition the axis positions. This is done, e.g., by the Haliax framework (Hall et al. (0222) which implements einsum for named tensors.
Other pointful notations Several authors (Vasilache et al.) 2018 Paszke et al., 2021 Bachurski & Mycrofft 2024) propose other types of pointful notation to express tensor operations using index expressions. They define a set of elementary operations as well as a notation to compose more com- plex tensor operations. However, there is only limited integration with existing tensor frameworks and support for vectorizing operations that are not defined in the notation itself.
2.3 DEFINITION OF VECTORIZATION IN LITERATURE
The term vectorization has been used in different contexts within tensor programming. Harris al. (020) describe glement-wise operations such as np. (add [add|mut multiply) in Numpy as vectorized oper- ations: These operations apply a scalar function to highherdimeenionnal tensors in conjunction with broadcasting rules to match axes across arguments. In contrast to our general perspective on vector- ization. Harris et al. do not use the term w.r.t. other types of operations such as np. [sum] matmul3.
In named tensor frameworks (c.f. Sec. 2.2). lifting of operations to higher- dighes-dimensional argument by-product of introducing symbolic axes, and is sometimes referred to as vectorization (Chiang et al., 2023). In this context, Chiang et al.) identify some elementary operations that may be vectorized to represent complex functions in Numpy-like frameworks, in- cluding scalar, reduction, dot-product, vector-to-vector and indexing operations. However, support and adoption of the notation in existing frameworks remains limited (c.f. Appendix H
tensors emerges implicitly
Bradbury et al. (2018) introduce jax. vmap (vectorizing map) which regards vectorization as trans- formation of operations: Given any operation op, the result of jax. vmap(op, ...) is a new op- eration that accepts and returns tensors with up to more dimension than op along which the vectorization is applied. While this allows for a general perspective on vectorization, it does not rep- resent concise, declarative notation for tensor operations, and is not posed universal alternative to Numpy-like notation.
VECTORIZATION
3.1 VECTORIZING OODERR OPERATIONS TO HIGHER-ORDER OPERATIONS
We define vectorization as the transformation of an operation that processes a single data point into an operation that processes collection of data points simultaneously. For instance, the sin function accepts and returns a scalar, while a vectorized sin function accepts and returns a collection of scalars and applies the sin function to each scalar separately. This broad definition of the term differs from its specific use in compiler design where it describes the automatic substitution of scalar instructions with vector instructions following the Single Instruction, Multiple Data (SIMD) model. (Cui) 2024)
In the context of tensor programming, vectorization is applied along axes of the tensor arguments: An operation that is vectorized along a particular axis with length of an n-dimensional argument tensor is applied to each of the 1 separate (n 1)ddimensional sub-tensors that are stacked along this axis. For instance, a vectorized sin function that operates on 1-dimensional vectors with length I computes the sin of I separate O-dimensional scalars. Vectorizing an operation is also known as lifting the operation to higher-order (i.e. nigher-dimensiona tensors, or applying the operation to a batch of data.
4

---

Published as a conference paper at ICLR 2026
Loop notation provides natural representation of vectorized operations by expressing the repeated application of the elementary operation to the individual sub-tensors stacked along a given axis. For instance, the following code represents the vectorized sin operation that accepts and returns vectors: for in range (x. shape[o]):
y[i] = sin(x[i])
The terms x[i] and y[i] represent the scalar sub-tensors that are stacked along the first axis of the vectors and y and are forwarded to the sin operation. The representation with loop notation is only for conceptual reasons and does not indicate how the operation is actually implemented.
Vectorization along multiple axes is represented using multiple for loops and analogous multiple consecutive one- dimensiioon vectorizations along each of the respective axes:
for i in range (x. shape[0]): for in range (x. (x. shape[1]): y[i, j] sin(x[i, jl)
We consider vectorization only w.r.t. operations that are invariant to the order of the loops and indices per loop, and omit loops in the following examples.
The usage of a subset of the available loop variables to address the axes of specific tensor expresses what known as broadcasting in Numpy-like notation (Harris et al. 2020):
z[i, j] x[i] y[i] Outer product of and
Lastly, we consider elementary operations that are applied to non-scalar arguments. For instance, sof tmax operates vectors and is vectorized along the second dimension of matrix follows:
y[:,
tofx il)
The terms x[:, i] and y[:, i] represent the one-dimensional sub-tensors that are stacked along the second dimension of the matrices x and y and are forwarded to the softmax operation. We say that the softmax operation is applied along the first axis and vectorized along the second axis of and y. We denote axes that the elementary operation is applied along as argument sub-tensor axes, and all other axes vectorized axes.
3.2 DECOMPOSING HIGHER-ORDER OPERATIONS TO LOWER-ORDER OPERATIONS
In the previous section, we considered the vectorization of lower-order operations to higher-order operations. We now go the opposite direction and conceptually decompose many existing higher- order operations, e.g., from Numpy-like notation, to few lower-order operations and their varying vectorization. In the following, we provide several examples.
A matrix multiplication is represented conceptually as vectorized dot-product, and its inherent vectorization is expressed loop notation as follows:
z[i, j] dx[( :], y[:, j])
Other types of tensor contractions (e.g., np. (dot [oaotul tensordot inner)) analogously repre- sent vectorized dot-products, but differ in their vectorization. Their implementation typically em- ploys optimized algorithms that do not simply loop over invocations of the elementary dot- product.
The sum-reduction operation np. sum(x, axis=1) over matrix is decomposable, i.a., using two alternative choices for the elementary operation:
y[i] = sum(x[i, :]) y[i] x[i, ji "+=" adds a scalar to a scalar -> vectorized axes
"sum" maps a vector to a scalar -> vectorized axis
Different types of multiplication such as the outer, Hadamard, Kronecker, and Khatri- products are represented as vectorized scalar multiplication and differ solely w.r.t. their vectorization.
Shape operations such as np. (transpose | Ireshape) are represented as vectorized identity maps
y[j, identi eventity(xE) j]) # Transpose y[i * x. shape[1] + j] Adentity(x[3 j]) # Reshape/Flatter
with identity(a) a. Implementations of these operations typically only modify a tensor's meta- data, rather than applying an assignment or copy operation per element.
Broadcasting tensors along new axes (e.g., np. www.oobbassttttee.oe teaesstttotttiie tilel/ repeat)) is represented as an identity map that is vectorized, i.a., along dimensions which appear only in the output:
y[i, identi (x[[i) # Broadcast along
5

---

Published as a conference paper at ICLR 2026
Indexing operations such torch. (take gather lacke__llng_diie Lndex_seleccte are vector- ized versions the following clementary operation: Retrieve a single value from an n-dimensional value tensor at the coordinates specified by one-dimensional coordinate vector with length For instance, the following operation gathers color values from an image at the given pixel coordinates: pixels: (#pixels, 2)
# image: (height, width, #channels) y[p, get_at(imageL:, :, c], pixels(p, :])
As illustrated above, many tensor operations in Numpy-like frameworks reduce to few elementary operations when factoring out their vectorization, It's all just vectorization and always has been! We use this observation in the following section to define universal notation that represents tensor operations as vectorized elementary operations.
EINX
4.1 NOTATION
Overview An operation in einx is expressed using the following function call signature: (outputs... einx. einx. (eleementary_operatteot (""eetiioon""vttttrrrrr"r "(vectorzzation" is vectorized according to the expres- accepts the tensors (inputs... ... ). and returns the tensors {outputs...
(inputs...))
This code states that the operation sion "(vectorizatiin einx provides one entry-point per elementary operation and follows Numpy's 's naming of operations where possible. For instance, the following represents vectorized scalar addition similar to np. add:
cenrentary_operattiin
z einx. add (" aan"...tod (vvetorrizaion)", x, y)
Vectorization The vectorization string is constructed by analogy with loop notation as follows:
(1) Express the operation in loop notation (c.f. Sec. 3). To illustrate this, we consider the following example tensor operation that vectorizes for in range .:: for in range(...). z[a, b] COME_OPERATIONNOE The number of colons (:) per tensor indicates the dimensionality of the arguments and return values of the elementary operation: Here, the first input is matrix, the second input a scalar, and the output a vector.
:, a], y[b])
(2) Take the expressions that are used to denote sub-tensors (here: x[:, :, al, y[b], z[a, i, bl), and convert the indices to the vectorization string as follows:
(a) Use arrow (->) to delimit inputs from outputs.
(b) Use commas delimit multiple tensors each side of the arrow.
(c) Use spaces delimit indices per tensor.
(d) Replace colons (:) with new symbolic axis names and place brackets ([J) around them.
Applying these nules results in the following einx representation for the above example operation:
z einx. SOME_OORRTOOOOOONNCOO [c d] a,
[e] b", x, y)
The vectorization expression indicates the shapes of input and output tensors. Here, y, and have shapes (c, d, a). (b), and (a, e, b), respectively. Unlike loop notation where index names denote loop variables, in einx notation the symbolic names refer to tensor axes. The loop ranges are determined implicitly from the given tensor dimensions.
Brackets denote axes of the argument sub-tensors that are passed to the elementary operation: is invoked with tensors shapes (c, d) and (), and returns a vector with shape (e). Brackets may appear both in input and output expressions, and must be placed around the num- ber of axes that matches the dimensionality expected by the elementary operation. Axes not marked with brackets are vectorized. The same axis name may be used for multiple sub-tensor argument axes, e.g., to indicate that they must have the same length:
z = einx ("a [b], [b] >> a c", x, y) # Matrix multiplicatio
Axis composition Some tensor operations are representable in loop notation by mapping one or more of the loop variables to new index value (e.g., np. reshape uses the row-major formula). We analogously define the following axis compositions in einx notation as ways in which one or more combined form single, new axis in the expression.

---

Published as a conference paper at ICLR 2026
We define a flattened axis multiple axes of single tensor that are flattened in row-major order to form a single new axis, following einops. A flattened axis is represented in the einx expression by wrapping the composed axes in parentheses. For instance, the output of the vectorized identity map einx. id("a -> (a b) c", x)
two-dimensional, and its first dimension corresponds the original axes a and flattened in row-major order (i.e., groups of b elements each).
We introduce a new type of axis composition that does not exist in einops. i.e. the concatenated axis, as multiple axes of multiple tensors concatenated along single new axis. This allows representing many operations from Numpy-like notation (e.g., np. [stack concatenate unstack [split)) as vectorized identity maps. A concatenated axis is represented in einx using the plus operator (+) with parentheses. For instance, the output of
iinx.id("a b, -> (b + c)", y)
is two-dimensional. and represents the concatenation of the input tensors along the second axis.
Axis constraints Additional axis sizes may be passed keyword arguments einx functions, e.g., if the input shapes of tensors do not fully constrain the lengths of all axes: einx. (d("(d((" (a b)
c", x, a=4)
Anonymous
For convenience, numerical may be used to specify the value of axes inline, and are equivalent to writing a new, unique axis name with a corresponding constraint:
cinx.id("a b -> b 3", x) einx. id( -> a b c", x, c=3)
Ellipsis We introduce novel, generalized type of ellipsis ... that is placed immediately after an axis to indicate that it is expanded a variable number of times. The number of expansions is determined from the dimensionality of the input tensors and additional constraints. The following example illustrates the expansion of ellipses:
einx. add ("b.. ... i, b... j b... i j x, y) expands to einx. add (bbe b1 i, be b1 j -> be bl i x, y)
for 3D inputs
Ellipses also apply to composed axes. The following example expands a flattened axis in order to partition n-dimensional tensor into list of n-dimensional tiles with side-length ds:
einx. (d("(sssssss ds)... (s...) ds...", x, ds=4)
einx further allows writing an anonymous ellipsis without preceding axis. In this case, einx intro- duces new axis name in front of it.
Ellipses in einx are analogous to their role languages such as Java, C++ and Swift: ellipsis is placed after a parameter to indicate that the function or template accepts variable number of arguments of that type. The actual number is determined from how many arguments are provided at given call site. In contrast, anonymous ellipses are analogous to their usage in einsum and einops.
Implicit output If possible, operations allow omitting the output and inferring it from the inputs instead, resulting in more concise expression:
einx. sum [b]", x) ->
einx. add("a b c, c", x, y) abc
4.2 CHARACTERISTICS
Universal einx decouples operations from their vectorization and applies consistent rules to press the vectorization independent of the specific operation. Any tensor operation may be vectorized with einx notation, and any vectorization representable in loop notation may also be expressed with einx notation. This makes einx a universal notation for tensor operations.
In practice, the universality allows invoking arbitrary operations, including those not part einx's API. For instance, the following code creates an einx operation that vectorizes a custom Python function by internally using the vmap transformation from PyTorch (c.f. Sec. 2.3)
def mxop(x, y) # Define custom function return einmyop einx. torch. adapt_with__wap (myop))) Convert to einx operation Invoking the einx operation with
torch. (orch.sum(y))

---

Published as a conference paper at ICLR 2026
2 einmyop("a [c], [c] -> b [c]", x, results in the same output calling myop in loop notation:
for a in range(....) for in range(...).
z[a, b, :] (xaa, :], y[b, :])
Declarative Numpy-like notation follows an imperative programming model: It requires the pro- grammer to express how to achieve the desired result, egg, involving reshaping, broadcasting, and transposing dimensions. In contrast, einx adopts declarative approach similar to einsum, where the user specifies what the inputs and outputs look like, and allows the system to determine the required transformations. This is illustrated by the following example:
Sinx.add( d x[:, None, None] np. transpose (1, 0, 2)) [None, :, None] imperative
b -> a b e", X, y)
# declarative
The former often easier to read and write, and explicitly documents what the inputs look like before applying the operation and the outputs look like after applying the operation; both of which are not immediately visible in Numpy-like notation.
Interpretable The definition of einx notation by analogy with loop notation provides explicit interpretation of any given operation: The representation in loop notation clearly illustrates what output the operation will yield, while allowing for an underlying backend implementation that fol- lows a different, more optimized algorithm.
4.3 PRACTICAL ADVANTAGES
In the following, we demonstrate several practical advantages of using einx with example operations. Additional examples can be found in Appendix C
Changing the shapes We consider a simple indexing operation in einx and Numpy-liks notation where elements the argument x are retrieved at positions stored in the argument y:
einx. get_at("[x] a, -> a", x, y)
torch. index_seleeexxxx, 0, y)
We now change the input and output shapes of this operation. einx allows varying the vectorization term to reflect these changes and keeps the entry-point fixed. In contrast, changing the shapes in Numpy-like notation necessitates switching to different entry-point with different signature and vectorization rules, or is not representable using a single entry-point at all:
# 1. Introduce axis einx. mrnxx.getaatt"lee a, b -> b a", y) torch. take_along_dimms
2nd parameter y -> switch to torch. lakkeallngg_ee
dim=0)
# 2. Introduce axis
no single entry-point in torch
einx. mixx.get_at("xx b, a -> c b a", x, y)
# 3. Replace 1D indexing with 20 indexing -> single entry-point in torch dunx.get_at("Eee y] b, a [2] -> c a", x, y)
Silent failures einsum represents multiple elementary operations in a single entry-point:
np. b" bc->ac", x, y)
d"a [b], [b]
c", x, y)
np. np. ("aisuumma,,baaii""
x)
einx.sum("s sum("a [b] ~> a", x) einx. munxx.ulliply(aa b -> b", y) einx. id( -> a", x)
y)
np. np.einn einsum("aba ("ab->ba", x)
This potentially results in silent failures if a typo the expression of one operation matches the sig-
nature of another operation. einx catches such errors by checking for the signature of the respective
entry-point:
("i.jummij.j->-ik"" x, y) # Now introduce a typo:
succeeds -> dot-product along j
einsum("ij ik">i,k, x, y) einx. dot [j], [i] k -> iik, y) fails loudly -> inconsistent brackets einx.dot("i j. -ik>>ik",,
# fails silently -> sum-reduction along
# fails loudly -> not dot-product
Clarity In complex operations, the undifferentiated definition of axes in einsum obfuscates which axes are summed along. contrast, brackets einx make the distinction clearly visible:
einsum("b k b k h -> a c" x, y) which axes are summed along? datx q [k] h, b [k] h x, y) # Only is summed along!
8

---

Published as a conference paper at ICLR 2026
4.4 IMPLEMENTATION
We provide an implementation of einx that compiles einx operations to function calls in given tensor framework, e.g., using Numpy-like vmap notation (c.f. Sec. 2.3 The compilation creates an isolated code snippet that is transformed to function object using Python's exec, cached on the first invocation, and reused on subsequent calls with the same signature. This results in no overhead compared to calling the framework functions directly, other than for cache lookup and during initialization (c.f. Appendix G used with just-in-time compilation such as jax. jit, the einx footprint disappears entirely.
As an example, the operation
sinx.sum(" ([b] c)", x, c=4)
compiles the following code when invoked with a Jax tensor of shape (8, 24) and requesting
Numpy-like or vmap notation:
# backend="jax numpylike" import jax.numpy jnp def op(a):
# backend="jax vmap"
import jax.numpy jnp import jax
jnp. reshapeea,, (8, 6, 4)) a jnp. sum a1))= return
b jax. vmap sum, in_axes=1, out_axes=0) b jax. vmap(b, in_axes=0, out_ out_axes=0)
def op(a):
jnp. respapes,,, (8, 6, 4))
a b(a) return
The compilation to Numpy-like notation uses features such as the axis parameter to express the vectorization, while vmap notation relies on the vmap transformation. We provide a description of how einx expressions compiled to Python code in Appendix more examples of compiled code in Appendix and examples of verbose exceptions that are raised for syntax, shape and semantic errors in Appendix
5 COMPARISON WITH EINSUM AND EINOPS
In the following, we compare einx notation with einsum and einops notation and illustrate the dis- tinctions by implementing an example tensor operation. We provide comparisons with other types of ein -notatiis in Appendix A
5.1 GENERAL COMPARISON
Both einsum and einops do not recognize the role of vectorization in tensor operations, and contain design choices that in contradiction with this insight:
There is no distinction between vectorized axes and argument sub-tensor axes.
The analogy with loop notation is not recognized incorporated into the notation.
einops.repeat and einops. reduce framed as symmetrical in terms of adding moving axes[] despite the former applying vectorization to add an axis, and the latter ap- plying elementary operation remove axis.
The naming of functions is not related the underlying elementary operations: einsum is not called dot, einops. rearrange and einops. repeat are not called identi ty.
einops. rearrange and einops. repeat compute the same elementary operation (i.e. iden- tity map), but follow different vectorization behavior across different entry-points.
Unlike einsum and einops which support only few operations (c.f. Tab. 2). einx allows expressing any tensor operation and any vectorization by analogy with loop notation. It includes many nota- tional improvements, such as generalized ellipses, axis concatenations, implicit outputs, cleaner API and separation of elementary operations into individual entry-points Our implementation fur- ther compiles expressions to isolated Python code snippets that are inspectable by the user and allow for different types of backend notations, such as Numpy-like or vmap notation.
"we made an explicit choice separate scenarios of "adding dimensions" (repear), "removing dimen- sions" (reduce) and "keeping number of elements the same" (rearrange)" (Rogozhnikoy) 2022

---

Published as a conference paper at ICLR 2026
5.2 CASE STUDY: HULTI-HEAL ATTENTION
We consider the multi-head attention (MHA) operation (Vaswani et al. 2017 and compare imple- mentations using (I) einx and (2) einsum, einops and Numpy-lik, notation if necessary. The axes b, k, and denote the batch, query, key, head and channel dimensions.
def aq, k, heads=1):
einx
= (in q (h [c]), b k (h [c]) -> b
k h", 9, k, h=heads)
A = einx. som ("b q [k] h", A
/ heads))
return sinx.dot("b [k] h, b [k] (h c) -> b (h c)", A,
def attn(q, k, v, heads=1):
einsum/einops/ Numpy-like
= einops. rearrange "b q (h c)
-> c", h=heads)
k = einops. rearrange (k,
"b k (h
-> kobkk c", h=heads)
einops. rearrange (v, k (h
c)
->
c", h=heads)
A = jnp. einsum
bkhc->bakh", bxohsmmn""
9, k) anp.sqrt (qhsha(((11))
jax.nn sof
axis=-2)
output = jnp. hmpp.einse einsum("bakh,, bkhc->bahc", "hhh A, v) return einops. rearrang "b q h
ch"
We make the following observations: (1) einx requires just three lines of code. einops additionally calls einops. raarssrerrrnngee due to einsum not supporting axis compositions. (2) The softmax operation in einx self-documents axis names and indicates that is applied along the axis einops does not support softmax and uses Numpy-like notation with positional axis argument. (3) einx indicates with brackets that the dot-products are applied along the axes and k. einsum relies on an implicit convention and obfuscates which of the enumerated axes reduced. (4) einx explicitly names the elementary operations, i.e. dot and sof tmax, rather than using the less clear name einsum.
In the MHA operation, mask is optionally applied to the attention matrix:
qs, ks jnp. arange shape[1]), ( jnp. arange (k, shape[1]) mask wewxx.rraaee_eeuuuee A einx. einx. (here k, k h,", mask, A, -jnp.inf)
einx
k", qs, ks)
qs, ks jnp. arange(qq shape(t))) jnp. aranee (k. shape[1]) mask qs[:, np. newaxis]
einsum/einops/ Numpy-like
des[np.newaxxe :] A jnp. where (mask(npp newaxis,
np. newaxis], A, -jnp.inf)
The clement-wise operations which obfuscates both the semantics of axes and how they are aligned w.r.t. each other. In contrast, einx self-documents axis names and follows declarative, rather than imperative style.
not supported in einops and must rely Numpy-like notation
Following the decomposition of complex tensor operations described in Sec. 3.2 we consider an alternative implementation that represents the batched MHA shown above as elementary, single- query, single-head attention operation and its separate vectorization:
def aqtn(q, k, v): # Define attention as an elementary operation
einx
A einx. einx.dot d"[c], k [c] -> k", k)
= einx. sof [[[]]"
jnp. sqrt(q.ttq.... shape(--hpp)1)
return einx. dinx_, [k] -> c", A, v)
einattn einx.jax adapt_wi mritth_wme (hhappttatt))) # Adapt einx notation
janxjjax.ddapt_wwr
Vectorize along batch, query, and flattened head dimensions:
output einattn("b q (h [c]), b [k] (h [c]), b [k] (h [c])
(h [c])",
q. k, v, h=heads)
CONCLUSION
We introduce einx, a universal notation for tensor operations. It follows a consistent set rules that apply any given operation, offers interpretability by analogy with loop notation, reduces the large API of existing Numpy-like frameworks to small set of elementary operations, and allows for a clean, readable and writable expression of operations in code. The notation offers not only a useful coding tool, but a better model for thinking tensor operations. We provide an open source software package that implements einx in Python for commonly used tensor frameworks.
10

---

Published as a conference paper at ICLR 2026
REFERENCES
Martín Abadi, Ashish Agarwal, Paul Barham, Eugene Brevdo, Zhifeng Chen, Craig Citro, Greg S. Corrado, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Ian Goodfellow, Andrew Harp, Geoffrey Irving, Michael Isard, Yangqing Jia, Rafal Jozefowicz, Lukasz Kaiser, Manjunath Kudlur, Josh Levenberg. Dandelion Mané, Rajat Monga, Sherry Moore, Derek Murray, Chris Olah, Mike Schuster, Jonathon Shlens, Benoit Steiner, llya Sutskever, Kunal Talwar, Paul Tucker, Vincent Vanhoucke, Vijay Vasudevan, Fernanda Viégas, Oriol Vinyals, Pete Warden, Martin Wat- tenberg, Martin Wicke, Yuan Yu, and Xiaoqiang Zheng. TensorFlow: Large-scale machine learn- ing on heterogeneous systems, 2015. URL hhtph:tts////t www.:.tteessfloo.ooo org/ Software available from tensorflow.org.
Krister Åhlander. Einstein summation for multidimensional arrays. Computers Mathematics with Applications, 2002.
Jakub Bachurski and Alan Mycroft. Points for free: python. In ACM SIGPLAN International Workshop on Libraries, Languages and Compilers for Arruy Programming, 2024.
Embedding pointful array programming in
Alan H Barr. The einstein summation notation. An Introduction to Physically Based Modeling (Course Notes 19), pages E, 1991.
James Bradbury, Roy Frostig, Peter Hawkins, Matthew James Johnson, Chris Leary, Dougal Maclaurin, George Necula, Adam Paszke, Jake VanderPlas, Skye Wanderman-Milnh and Qiao Zhang. JAX: composable transformations of Python+NumPy programs, 2018. URL http:
7/github. cww/jax- jax-ml/jax
Tongfei Chen. Typesafe abstractions for tensor operations (short paper). In Proceedings of the 8th ACM SIGPLAN International Symposium Scala, SCALA 2017, pp. 45-50, New York, NY. USA, 2017. Association for Computing Machinery. ISBN 9781450355292. doi: 10.1145/
1136000.3136000 URL https:///o/ www./0...//.o. 1145/31 31603333600.. 3136081
David Chiang, Alexander M Rush, and Boaz Barak. Named tensor notation. Transactions
Machine Learning Research, 2023.
Edward DongBo Cui. Vectorization: A Practical Guide to Efficient Implementations of Machine Learning Algorithms. 2024.
Zachary DeVito. Named tensors using first-class dimensions in pytorch. httpp:://itthub com/
facebookresearch/torrhhhii 2023.
Albert Einstein. Die grundlage der allgemeinen relativittätstheorie. Annale der Physik, 1916.
Francois Fleuret. eingather, 2023. wttp:///w////eeeeeeeo//////////a
David Hall, Ivan Zhou, and Percy Liang. Haliax, 2022. URL http:::/ggihhh.../sstttrrrrrror/ haliaxl
Awni Hannun, Jagrit Digani, Angelos Katharopoulos, and Ronan Collobert. MLX:
Efficient and
flexible machine learning apple silicon, 2023. URL htttps://gituub..co///-eeee
Charles R Harris, K Jarrod Millman, Stéfan J Van Der Walt, Ralf Gommers, Pauli Virtanen, David Cournapeau, Eric Wieser, Julian Taylor, Sebastian Berg, Nathaniel J Smith, et al. Array program- ming with numpy. Nature, 2020.
Stephan Hoyer and Joe Hamman. xarray: Nd labeled arrays and datasets in python. Journal of Open Research Software, 2017.
Niels Skovgaard Jensen. einmesh, 2025. wtt.://tr//ttttoo..aaaaaaaassssr
Daniel D. Johnson. Penzai Treescope: A toolkit for interpreting, visualizing, and editing models as data. International Conference on Machine Learning Workshop on Mechanistic Interpretability, 2024.

---

Published as a conference paper at ICLR 2026
Jonathan Malmaud. einindex, 2018. URL http://////ggggttoooo//////aaaaa
Callum McDougall. eindex, 2023. URL URL http:///www.prrrrrtoo//oooooott/
Adam Paszke. Pytorch: An imperative style, high-performance deep learning library. arXiv, 2019.
Adam Paszke, Daniel D Johnson, David Duvenaud, Dimitrios Vytiniotis, Alexey Radul, Matthew Johnson, Jonathan Ragan-Kelley, and Dougal Maclaurin. Getting to the point: index sets and parallelism-preserviii autodiff for pointful array programming. ACM on Programming Lan- guages, 2021.
Alex Rogozhnikov. Einops: Clear and reliable tensor manipulations with einstein-like notation. In International Conference on Learning Representations, 2022a.
Alex Rogozhnikov. Sinops.pack. einops.uupack, 2022b. URL hntees//ggthuuu.cm//rrepzzzoo/
Alex Rogozhnikov. eindex, 2023. URL https: http:/ghhw.......ooro//////////a
Wenzhe Shi, Jose Caballero, Ferene Huszár, Johannes Totz, Andrew Aitken, Rob Bishop, Daniel Rueckert, and Zehan Wang. Real-time single image and video super-resolution using effi- cient sub-pixel convolutional neural network. In Conference on Computer Vision and Pattern Recognition, 2016.
Nicolas Vasilache, Oleksandr Zinenko, Theodoros Theodoridis, Priya Goyal, Zachary DeVito, William S Moses, Sven Verdoolaege, Andrew Adams, and Albert Cohen. Tensor comprehen- sions: Framework-agnostic high-perffrrmnce machine learning abstractions. arXiv, 2018.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in Neural Informa- tion Processing Systems, 2017.
Mark Wiebe. First commit to numpy introducing np.einsum. https://github wtt.h//....ooomuum//m/nn//y/////
2011.
12

---

Published as a conference paper at ICLR 2026
APPENDIX
We provide the following additional content in the appendix.
Sec.
A
Comparison with other types of ein -nntttoii:::: einops.(pack) unpack), eindex, einmesh
Sec.
B:
Comparison with Numpy-like notation
Sec.
C
Additional examples of einx operations
Sec.
D
Description of einx compiler
Sec. E
Additional examples code snippets compiled for einx operations
Sec.
Examples of einx exceptions
Sec.
Benchmark of cinx's 's overhead
Sec.
Usages statistics of related libraries
A COMPARISON WITH OTHER -INTATOTASIONS
einops.pack, einops.unpack [Rogozhnikov| (2022b) introduces ein*-notation einops that is implemented in einops. {pack] Leckk/ungee Junpack] and allows expressing some concatenation and splitting operations. The following call flattens all but the first two dimensions of the input tensors and concatenates them along the third dimension:
einops. Sinops.pack((xx pack([x, y], "a b x")
However, the notation differs from the original einops notation, and further diverges from the declar- ative style where all inputs and outputs are documented explicitly. Instead, multiple arguments represented using a single expression, and multiple varying sequences axes are represented using the new operator.
In contrast, concatenation and splitting einx are expressed as special cases of the vector- ized identity map using the concatenated axis composition (c.f. Sec. 4.1, retain the explicit and self-documenting style, support more vectorization patterns than einops. (pack) unpack), and triv- ially allow inverting the operation by swapping input and output expressions:
einx. id("a b1, b2 -> (b1 b2)",
einops pack einops.pack< y], "a x")
einx. ("a b1,
-> a (b1 b2)", ", x, y)
no single entry-point in einops
einx.id(" ("d (b1 b2) -> a b1, b2 a", z, b1=4) no single entry-point in einops
i""a, b -> b (1 1)", x, y)
no single entry-point in einops
eindex Rogozhnikov (2023) proposes notation that allows expressing gather, scatter and arg- operations. For example:
EX. EX. gather idx, "b h w c, [h, w] - b c")
einx. @"b [h w]
[2] b -> b c", x, idx) # same operation in einx
The sub-expression [h, w] in eindex denotes an axis with length 2 in the tensor whose values are used to index the axes h and w the value tensor. must always appear as the first axis of expression, diverges from the declarative style of the original notation, and does not generalize to other tensor operations. In contrast, indexing in einx follows the same notation as other operations, declarative style and supports
retains
vectorization patterns:
einx. mnx.geetat("(hi c, [1] b >> c", y) einx. einx.get_as get_att c, [1]
EX. Ex.gather y, "h [h] b -> b c")
x, y) # no single entry-point in eindex
einx. linxxget_aat"[I c,
y) # no single entry-point in eindex
einmesh Jensen 2025 introduces an =inttnot for meshgrid operations:
xs, ys einmesh. LinSpace (0, 1, 10), Sirmesh..inSpaaae.
1, 20)
= einmesh. numpy. einmesh ("x y", x=xs, y=ys)
xy einmesh. ei einmesh("x y *", x=xs, y=ys)
Mesh-grid operations are compositions of broadcasting and concatenation with existing generator functions such as np. linspace. As such, they are special cases of the vectorized identity map and expressible using einx. id:
13

---

Published as conference paper at ICLR 2026
xs, ys = np.linspace(f, 1, 10), np. 1iinspcccee-,,,, 1, 20)
X, y sinx.id("X. y -> x y, x y", xs, ys) x (1 + 1)", xs, ys)
xy = cinx.id(")
While einmesh requires knowledge the concept and meaning of mesh-grids, the einx expression clearly self-documents the behavior without requiring the introduction of new concepts and docu- mentation.
B COMPARISON WITH NUMPY-LIKE NOTATION
We observe that much of the complexity in Numpy-like notation stems solely from the way which vectorization is expressed and impacts how users read and write tensor programs:
Users have to learn many diverging rules for expressing vectorization, e.g.:
Operations over multiple imputs often rely on implicit broadcasting rules
Some operations use parameters such as axis or dim (e.g. reduction with torch. sum, Or vector-to-vectoe mapping with torch. sof smax).
Indexing operations use, i.a., advanced indexing rules
Many operations (e.g. np. (dot matmul)) follow function-specific rule sets. Complex operations often require separate shape manipulation align inputs and outputs with their signature (e.g. using np. [transpose squeeze newaxis)).
-The rules sometimes conflict across different frameworks (e.g. [tf]torch) gather).
Function names and arguments alone often do not reflect the vectorization behavior without reading their documentation or writing comments, e.g.:
Which of torch. [take gather index_select) do to perform indexing in given use case?
Which of np. (matmul dot |ensordot (inerr) do use in a given use case?
Understanding how a given operation is vectorized often incurs mental load, egg.:
Which of and y the following expression are vectorized jointly or separately?
x[:, np. newaxis,
np. newaxis] y[:, :, newaxis, :]
Which input and output axes in the following operation correspond to each other?
np. transpose (2, 1, 3, 0))
Harris et al. (2000) the term vectorization only when describing element-wise operations in Numpy. In contrast, we follow a generalized view of vectorization that covers all mechanisms described above and is independent of any specific operation. This allows einx notation, which represents the vectorization of operations, to be applicable to any tensor operation and follow single set of rules across operations. The universal nature of einx notation simplifies the large and complex API of Numpy-lit notation, and reduces many Numpy-like operations to few einx operations.
C ADDITIONAL EXAMPLES OF EINX OPERATIONS
Changing the operations In Numpy-like notation, some functions (e.g. nrokron) are provided for particular vectorization cases of an elementary operation (e.g. scalar multiplication). but similar specializations are not available for other elementary operations (e.g. scalar addition). In contrast, einx allows using analogous vectorization patterns across different operations:
einx. mul tiply("a..... a ..., b...
b)
x, y) # Same krokron
einx. ("a.... b... -> (a b)
X, y) # kron-like add
einx. less ("a..., b... >> (a b)
X, y) # kron-like less
einx.id ("a..., b... >> (a b)
(1 X, y) # kron-like stack
Concatenation
einx fully supports broadcasting and transposing shapes in concatenation opera- tions, e.g., to append a vector along the channel dimension of a batch images:
chttps://nung:: wttg//////taalleee http:///bc//sstaaae////e/raaaaaa broadcasting innadcasting.... int. .hml
pettps://nungee or hwtp:/t...g/////////oo/////s /tse////aaa/ssee/a/iiissss indexing www.xiing.tttee htw.hhhtiiaaaaneeeeeee indexing
14

---

Published as a conference paper at ICLR 2026
einx. idbbb c1 h w, c2 -> (c1 + c2) h w", img, vec)
The same operation in Numpy-like notation requires separate manipulation of the shapes: ap.cconateeatee ( (Cimg, np.
None, None], \
(img. shape[0], vec. shape(0l, img shape[2], img shape[3])], axis=1)
einx similarly supports creating mesh-grids, which rely on specialized entry-point in Numpy-like notation (i.e. np. meshgrid) and not supported by single entry-point in einops:
einx.id ix(""x y (1 1) y", xs, ys) Stacked along first axis einx. id"x,, -> x y, x y", xs, ys) # Returned as separate tensors
The positional indices of arguments in operations indicate how the arguments used in the operation. Since axis concatenations change the number of arguments and therefore their positional indices, we only support axis concatenations in einx. id.
Expanding composed axes and einops notation:
We show the depth-to-space transformation (Shi et al. 2016 in einx
einops. rearrangexx, "b h w (c dh dw) -> b (h dh) (w dw) c", dh=4, dw=4) einx. id("b s... (c ds...) -> b (s ds) c", ds=4)
The axes and с denote the batch and channel dimensions, and denote the spatial axes before, and (h dh) and (w dw) after the transformation. The ellipses allow for joint representation of the spatial axes, resulting in a more concise expression, indicating similar treatment spatial axes, and generalizing the operation to spatial dimensions. The following example shows similar expression a spatial mean pooling operation:
einops. reduce "(h dh) -> h",
reduction=" "onection=mmean" mean", dh=4) # 1D einops. reduce (x, (hh dh) (w dw) -> h w", dh=4, dw=4) # 2D
reduction="meat einops. reduce "(h dh) (w dw) (d dd) -> h w d",
reduction="mean" dh=4, dw=4, dd=4) # 3D einops. reduce "(t (h dh) (w dw) (d dd) (z dz) -> h 2" reduction=" mmean", dh=4, dw=4, dd=4, dz=4) # 4D
einx.mean [ds]).... x, ds=4) ND
Multiple ellipses In einsum, multiple ellipses always refer to the same set of axes. In contrast, ellipses in einx expand custom axes and thereby allow representing multiple sets of axes:
einsum("... ... a, ... a ... x, y) Same set of axes
einx. ("x... [a], x... [a] -> x...", x, y) # Same set of axes
dinx.dot("x.... [a], y... [a] -> x... y...", X, # Multiple sets of axes
Flattened axis in einx.dot cinsum and einops do not support using flattened axes for tensor con- tractions. In contrast, all operations in einx support flattened axes, e.g., express grouped linear layers in neural nets
# Regular linear layer einx. dot(".. ... [in], [in] out -> ...
x, weights)
# Grouped linear layer: Same weights per group
einx. d" ... (h [in]), [in] out -> ... (h out)", x, weights, h=heads)
# Grouped linear layer: Different weights per group
einx.dot d".."........................... (h [in]), [in] h out -> (h out)", weights, h=heads)
or the multi- attention operation (c.f. Sec. 5.2).
Multiple elementary operations As described in Sec. 3.2 some operations from Numpy-like notation are decomposable into different elementary operations. For instance, the sum-reduction y sum axis=1) is represented in loop notation as follows:
y[il = sum(x[i, :]) "sum" maps vector to scalar -> vectorized axis y[i] += x[i, j] "+=" adds a scalar to a scalar -> 2 vectorized axes
This maps to two corresponding expressions in einx notation:
y einx. sum [j] -> i", x) y = einx. j -",
x) # vectorized axis
vectorized axes
Where possible, support both types of expressions in operation and indicate in the doc- umentation. The representation of einx. [dot sum) as vectorized scalar operations (i.e., without brackets) allows for compatibility with the corresponding operations in einsum and einops notation.
15

---

Published as a conference paper at ICLR 2026
D DESCRIPTION OF EINX COMPILER
Our implementation compiles einx operations to isolated code snippets in Python which invoke framework functions based on the requested type of notation (c.f. examples in Sec. E The compi- lation is performed in the following three steps.
1. Abstract syntax tree In the first step, the string expression of a given einx operation is trans- formed to one abstract syntax tree (AST) for each input and output tensor. Nodes in the AST correspond to different sub-expressions such as axis lists, axis compositions, named or unnamed axes and ellipses. The transformation is done in the following stages:
1. Parse the string to a simple AST and check for syntax errors such as invalid literals, axis names, parentheses or brackets.
2. Expand all ellipses in the AST. The compiler first determines the number of expansions for each ellipsis using a system of equations that represent the constraints resulting among others from the input shapes or identical axis names across multiple ellipses. Each ellipsis is then replaced by n copies of its child node where is its expansion number. For each copy, an incrementing counter is appended to all included axis names. Invalid ellipsis placement, egg-, indicated by not finding unique solution to the system of equations, results in rank (i.e. dimensionality) error.
3. Determine the length of all axes in the expression and annotate the AST with the axis lengths. To achieve this, the compiler solves a system of equations that represent the con- straints resulting among others from the input shapes, additional parameters and relation between nodes and their children (e.g., the length of flattened axis is equal to the product of the lengths of its child nodes). Inconsistent axis constraints, e.g., due input shapes not matching given einx expression, result in dimension error.
The final ASTs fully specify the shapes of all input and output tensors in the operation.
2. Computational graph In the second step, a computational graph is built for the operation using the requested framework, notation, elementary operation, and shape ASTs. Nodes the graph represent values (e.g., tensors Python values), and edges with input and output nodes represent function calls or other Python statements (e.g., tensor operations in a given tensor framework).
The graph is built by passing tracers (i.e., objects representing graph nodes) through Python fune- tion that represents the algorithm for computing the given operation. The initial inputs are con- structed as tracers representing the input tensors with the given shape ASTs. Each statement (e.g., function call) with a set of input tracers constructs a new edge in the graph, and returns a new set of output tracers. The final graph is defined by set of input and output tracers as well edges representing the function calls and statements that make up the requested algorithm.
The algorithms for different types of operations hard-coded based on the API of the backend framework and requested type of notation. Groups of operations often share parts of the implemen- tation: For instance, most operations start by invoking reshape operation to unflatten axes the input tensors (i.e., to remove flattened axis compositions). and end by flattening axes of the output tensors as determined by the output AST (i.e. to reintroduce flattened axis compositions). Some groups of operations, such as all element-wise operations. have identical implementations up the innermost backend function that is invoked (e.g., np. np.(add) ( subtract mull tiply logi cal_ad)))
Finally, the graph is optimized using set of simple heuristics, such as removing reshape operations where the input and output shapes are identical, or transpose operations where the order of input and output axes is identical.
Python code snippet In the last step, the graph is transformed into an isolated Python code snippet. The operations are topologically sorted and transformed to lines of code by traversing the graph from output to input nodes. Variables are created starting from the name a and incrementing alphabetically, with names being reused if possible.
The entire code snippet is executed using Python's 's exec, and the object corresponding to the con- structed operation retrieved from the environment using Python's eval.
16

---

Published as a conference paper at ICLR 2026
E ADDITIONAL EXAMPLES OF COMPILED CODE
The code snippet that is compiled for a given einx operation can inspected by passing graph=True to the respective operation. In the following, provide additional examples of compiled code for einx operations using the Jax framework.
Example 1: Transposition.
x wnp.jnerra zeros ( (10, 5))
>>> einx.id("s -> a", x, graph=True)
import jux_yumpy jnp
def op(a):
jnp. transpose (1, 0))
return
Example 2: Reshape.
>>> x jnp. zeros 5))
b) с (b c)", x, b=2, graph=True)
import jax. numpy as jnp def op(a):
ptpjjn.resse reshapes, (5, 10))
return
Example 3: No-op.
x jnp. (rpozeero(((1)) 5))
>>> einx.id a b", x, graph=True) def op(a):
return
Example 4a: Element-wise addition that uses Numpy-like broadcasting.
>>> iinp.zeros 5, 6))
y zeros (4, 3, 6))
>>> dinxxadd("s d e, с b -> abcde", y, graph=True)
import jax.numpy jnp
def (a,a, b)
jnp. reshape (2, 1, 1, 5, 6)) jnp. transpose (b, (1, 0, 2))
b reshaeeb,, (1, 3, 4, 1, 6))
inp.add(a, return
Example 4b: Element-wise addition that uses jax. vmap vectorize jnp. add.
>>> iinp.zeros (2, 5, 6))
((eros((4) (4, 3, 6)) >>> y jnp. zeros
>>> einx. ainx"add("a d e, c b - abbde""
graph=True, backend=" vmap")
import numpy as jnp import jax
a = jax. vmap(jnp. add, in_axes=(0, None), out_axes=0)
a jax. vmap(a, in_axes=(1, None), out out_axes=1)
a jax. vmap(a, mi_axes=(None, 0), out_axes=1)
a = jax. vmap(a, in_axes= (None, 1), out_axes=1)
a jax. vmap(a, in_axes=(2, 2), out out_axes=4)
Example 5a: Matrix multiplication that forwards to einsum.
>>> x jnp. zeros((2, (2, 3))
>>> y jnp.zeros((3 (3, 4))
>>> einx.dot(") [b], [b] - a c", x, y, graph=True)
import jax.numpy as jnp
def (p b):
= bcc a, b)
bc->ac",
return
17

---

Published as a conference paper at ICLR 2026
Example 5b: Matrix multiplication that uses jax. vmap to vectorize jnp. dot.
>>> x jnp. ( (2, 3))
>>> jnp. z 4))
einx. [b], [b] c", x, y, graph=True,
backend="jaas vmap")
import jax.numpy jnp import jax
a jax vmap dot, in_axes=(None, 1), out_axes=0)
a jax. vmap(a, in_axes=(O, None), out_axes=0)
Example 6: Retrieve pixel colors from a batch of images.
x jnp. zeros (2, 128, 128, 3)) batch of images
>>> inp.zerose (50, 2)) set of 50 pixels
>>> einx. get_at("b [h w] c, p [2] -> b c", x, y, graph=True,
backend="jax vmap")
import jax
def ab, c):
return b[c[0], c[1]]
jax. (a, in_axes= (0, None), out out_axes=0)
a jax. vmap(a, in_axes= (None, 0), out_axes=1)
a = jax. vmap(a, in_axes=(3, None), out aut_axes=2)
F EXAMPLES OF EINX EXCEPTIONS
Our implementation of einx raises verbose exceptions for syntax, shape and semantic errors. In the following, we provide several examples.
Example 1: Syntax error
x np. zeros((1 5)) >>> einx.id("a b -> (a b", x)
SyntaxError: Found an opening parenthesis that is not closed: Expression: "a
b"
Example 2: Syntax error
>>> x np. 1e..e 5)) >>> einx.id(" (b)a >> a b", x)
SyntaxError: The expression "(b)a' is not valid. Are you maybe missing whitespace?
Expression:
"(b)a >> b"
Example 3: Bracket error
>>> np. peros(((0,)) einx,sum"" [b] b", x)
SyntaxError: There are multiple occurrences of axis b with inconsistent bracket
usage:
Expression: "a [b]
b"
An axis may only appear with brackets or wi without brackets, but not both.
Example 4: Axis size error
>>> np. ((10 5)) einx.id("(a b) c ~> c", x)
AvisSizeError: Failed uniquely determine the size of the a, b. Please
18

---

Published as a conference paper at ICLR 2026
provide more constraints.
Expression: "(a b) c -> a
The expression, tensor shapes and contraints resulted in the following equation(s):
(a b) 10
The operation was called with the following arguments: Positional argument #1: Tensor with shape (10, 5)
Example Ellipsis error
>>> 2e00 5)) einx.id(" b)... b...", x)
RankError: Found invalid usage of ellipses and/or constraints for the axis a:
Expression: "(a b)... - ab.."
Please check the following:
Each axis name may be used ei either with whout ellipsis, but not both,
The rank of constraint must be equal to or less than the number of ellipses around the corresponding axis. The following equation(s) were determined for the expression:
(a b).................. 10 5
The operation was called with the following arguments: Positional argument #1: Tensor with shape (10, 5)
19

---

Published as a conference paper at ICLR 2026
Table 3: Overhead in milliseconds of using einx for three example operations with Numpy.
Operation
Compilation (ms)
Cache retrieval (ms)
einx. id(" h -> a c w", x) einx. dot [b], [b]
6.8 = 1.3
0.058 E 0.001
c", x, y) cinx.adde (c d) (d e) gg "->
9.3 = 2.4
0.070 + 0.007 0.077 + 0.003
23.5 = 3.1
h", x, y, d=2)
Table 4: Usage statistics of libraries in the context of tensor notations. einsum is implemented in different tensor frameworks, not single repository. **; torchdim was upstreamed into the larger functorch repository on Aug 1, 2024. ***; No reliable search term.
Github stars
Github forks
Github files
Conferences usage
cin*-notation
einsum
431000
35.27%
einops
9200
381
164000
21.19%
Named tensors
penzai
1800
66
456
0%
torchdim
* #*331
** 10
243
0.01% %
Haliax
202
19
1000
0.02%
xarray
4000
1200
184000
0.46%
Other pointful notations
Dex
1600
114
***
0%
Tensor Comprehensions
1800
213
132
0%
Ein
**
0%
G BENCHMARK OF OVERHEAD
einx compiles operations to function calls in a given tensor framework. The result of the compilation cached on the first invocation, and reused on subsequent invocations with the same signature. The overhead of using einx compared to calling the framework functions directly thus consists of (1) the compilation on the first function call and (2) cache retrieval on subsequent function calls. Table 3 shows the overhead that einx incurs for three example operations using the Numpy backend. In all cases, the cache retrieval after initialization adds less than 0.1ms of overhead. When used with just-in-time compilation such jax. jit, the overhead disappears entirely.
H USAGES STATISTICS RELATED LIBRARIES
Table 4 provides overview the usage statistics of related libraries in the context of tensor nota- tions. We gathered the number of stars and forks of the respective Github repositories September 2025. We further used the Github search find usages of the libraries, and note the number of files returned by the search. We lastly gathered all 11898 publicly accessible Github repositories linked by papers published in the conferences CVPR 2023, CVPR 2024, CVPR 2025, ICCV 2023, ECCV 2024, ICLR 2023, ICLR 2024, ICLR 2025, NeurlPS 2023 and NeurlPS 2024, and note the percentage of papers that use the respective libraries by matching a regex term the source code.
We make the following observations:
Of the listed librarics, only cinsum and einops are used by a significant number of papers (> 20%) in machine learning conferences. All others used by less than 1% of papers.
einsum, einops and xarray are used in 100x more files on Github than all other libraries.
xarray is used in many files on Github, but less than 1% in machine learning conferences.
Named tensor libraries and custom pointful notations have found little adoption machine learning conferences.
20


Reviews:
[{"review_id": "SjvUkbZeuh", "reviewer": "Reviewer_PoRD", "summary": "This paper proposes a new notation for tensor operations called einx. Previous attempts, such as einsum and its variants, have been proposed and widely used in the machine learning community. However, these notations have limitations in expressive power by definition, and their semantics are not always straightforward to understand. In this paper, the authors propose einx, a new notation focusing on vectorization. In the proposed notation, an einx expression can be derived by rewriting an implementation originally written using for-loops. The proposed notation enables concise descriptions of a wide range of tensor operations.", "questions": "In Sec. 3.22, the phrase \"The sum-reduction operation `np.sum(x, axis=1)` ... is decomposable\" appears, but this wording may be somewhat misleading and might benefit from rephrasing. When I first read it, I interpreted it to mean that the `sum` computation can be \"decomposed\" into two operations, namely `y[i] = sum(x[i, :])` and `y[i] += x[i, j]`. However, that is not the intended meaning here; instead, it means that the sum operation can be expressed in two distinct ways, correct?\n\nThis is more of a comment than a question, but for complex notations (such as `einx.dot(\"b q (h [c]), b k (h [c]) -> ...` in Sec. 5.2), I believe understanding would be deepened by a visualization that explains them visually. For example, a simple HTML page that can be opened in a browser, where entering an `einx` expression produces a real-time visualization.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The strengths of this paper are as follows:\n\n## Convenience of a unified notation\nAs the authors claim, the proposed notation enables a unified expression of various tensor operations. In particular, the ability to represent concatenated axes, such as in `einx.id`, is interesting. Specifically, in the expression of attention in Sec 5.2. 5.2, the use of `einx` indeed allows the attention mechanism to be written in a small number of lines.\n\n## Beginner-friendly for elementary examples\nIn the elementary procedure of this method, one first considers a standard loop-based formulation and then converts it into `einx` notation by arranging the corresponding ids. This step-by-step approach seems intuitive and accessible even to beginners.\n\n## Introduction and related work\nI appreciate Sections 1 and 2 of this paper. The authors provide a summary of the history of past `ein*` notations, which should serve as valuable material for future researchers considering similar notational systems.", "weaknesses": "On the other hand, the weaknesses of this paper are as follows:\n\n##  Mismatch with the venue due to a lack of quantitative evaluation\nWhile the proposed notation is interesting, there is no quantitative evaluation demonstrating its actual effectiveness. Therefore, it isn't easy to judge whether the proposed method is scientifically sound as a research contribution. Since quantitative evaluation is not necessarily required for notation proposals, this work would be more suitable for venues such as MLOSS or the ACMMM Open Source Competition rather than the ICLR main conference.\n\n## Still complex for complex examples\nAlthough the proposed notation is easy to understand in elementary examples, it remains complex for complicated cases. For instance, an expression like `z = einx.multiply(\"a b c, b -> c b a\", x, y)` is understandable by reasoning from the corresponding loops. Still, the attention example requires considerable familiarity to interpret. Therefore, the effectiveness of this notation (i.e., whether it truly makes complex operations easier to understand) depends heavily on the user's level of expertise with the notation.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "AtZg408C5F", "reviewer": "Reviewer_GyJP", "summary": "The paper proposes einx, a “universal” notation for tensor operations that extends einsum. It also claims other features, including declarative and Interpretable. The paper compares einx with einops using multi-head attention as a case study.", "questions": "1. How does the finding about vectorization lead to the design of einx?\n2. What problems do brackets solve that einsum cannot, and how often do those cases arise in practice?\nFor example, this notation seems unnecessary in many cases in the paper.\nLine 358, einx.sum(\"a [b]\", x) seems equivalent to torch.einsum(“a b -> a”, x).\nLine 308, einx.dot(\"a [b], [b] c -> a c\", x, y)  seems equivalent to torch.einsum(\"a b, b c\", x, y)\nLine 472, einx.softmax(\"[k]\", A) seems equivalent to torch.softmax(a, axis=1). \n3. What are the key differences between einx and PyTorch einsum (or other einsum extensions) in terms of notation design? Which operations can einx express that others cannot, beyond the softmax example above?\n4. What is the scope? Is it really universal?\na. Convolution operators are important; why are they not covered or discussed in this paper? There are works extending einsum to convolution operations [1][2].\nb. How does einx represent pixel shuffle?\nc. How does einx represent jacobi-2D? \n5. Table 1 classifies Numpy-like operations into four categories. What is the intuition for such classification? \n6. How can the proposed classification (Table 1) be extended to cover custom operations? Can it serve as a framework or guideline for defining new operations within einx?\n7. Section 5.2 presents multi-head attention as an example; however, similar high-level fused attention layers already exist. Please include additional examples where einx clearly shortens code, improves readability, or demonstrates other unique advantages.\n\n[1] Dangel, Felix. \"Convolutions and More as Einsum: A Tensor Network Perspective with Advances for Second-Order Methods.\" Advances in Neural Information Processing Systems 37 (2024): 96671-96727.\n\n[2] Rabbani, Tahseen, et al. \"conv_einsum: A Framework for Representation and Fast Evaluation of Multilinear Operations in Convolutional Tensorial Neural Networks.\" arXiv preprint arXiv:2401.03384 (2024).", "limitations": "", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "+ Brings many NumPy-like operations under a unified function signature.\n+ The declarative nature of the notation is advantageous; for instance, Line 374 demonstrates how the necessary permutation is inferred and performed implicitly.", "weaknesses": "- Section 3 discusses tensor ops and vectorization, but lacks formal definitions of lower-order vs higher-order vectorization; this makes the argument hard to follow\n- The core conclusion in Section 3 —“It’s all just vectorization”—reflects a well-known aspect of tensor computation (same computation applied across slices). The paper does not clearly present how this finding leads to the specific design choices in the einx signature.\n- It’s unclear, in notation and capability, how the proposed {vectorization}/bracket syntax differs in practice from PyTorch einsum (or its common extensions).\n- Table 1 categorizes Numpy-like notations into 4 groups. It might be interesting to discuss the intuition of such a classification.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "FhQNFGjR0O", "reviewer": "Reviewer_bcTx", "summary": "This paper proposed a new extended grammar for describing tensor operations that generalizes those common in `einsum` and `einops`. The authors cast vectorization as a general transformation that lifts lower-order operations to higher-order operations with a pointful (instead of pointless) style that is more declarative. It can naturally subsume all kinds of indexing (gather, scatter, index_select); tensor contraction (multiply, inner, outer, kronecker product, etc.); and all kinds of reshaping operations (broadcast, repeat, squeeze, etc.). Additionally, the authors also included a `(a+b)` notation that covers stack and concat. \n\nThe case study with multi-head attention is pretty illuminating. This library would be beneficial to all machine learning and scientific computation practitioners.", "questions": "- In the abstract, please cite `einsum` and `einops`.\n- L165 Named tensors: Missing citations to include:\n  - T Chen (2017): Typesafe abstractions for tensor operations. https://dl.acm.org/doi/10.1145/3136000.3136001\n  - D Chiang, A M Rush, B Borak (2021): Named Tensor Notation. https://arxiv.org/abs/2102.13196\n- L309: The semantics of `[ ]` is vague: it seems to indicate matched axes for tensor contraction, but sometimes it has other uses: please clarify", "limitations": "", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 4, "strengths": "- A good generalization of `einsum` and `einops` that would be beneficial to all ML practitioners.\n- Paper is clearly written, with lots of examples to showcase the semantics the proposed grammar.", "weaknesses": "- The semantics of `[x]` is not clear enough: at sometimes it is for axes to be contracted; sometimes it can be used in a `get_at` expression whose semantics is a bit vague.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "26SowPsori", "reviewer": "Reviewer_juN8", "summary": "The paper introduces einx, a notation and library for expressing tensor operations as vectorizations of elementary functions. The authors define a consistent syntax based on loop notation and implement it in Python for NumPy, PyTorch, and JAX backends. The goal is to replace the heterogeneous and sometimes confusing APIs of current tensor frameworks with a small, uniform set of composable operations. The paper is clearly written and technically sound.", "questions": "- Are there examples where einx allows expressing a model or computation that cannot be easily written with existing tools?\n- Can you provide any data, even informal, on user adoption or code simplification in real ML workflows?\n- How does einx handle contraction path optimization? Can it automatically determine and apply efficient evaluation orders for chained tensor products (e.g., einx.dot(\"m [a], [a b], [b] -> m\", M, A, v)), or does it always execute operations in the explicit order implied by the expression?\n- einx accepts sparse tensors for simple elementwise operations (e.g., scalar multiplication) but fails for contractions, additions, or reshapes because the PyTorch backend invokes dense operations (einsum, reshape, add(sparse, dense)). Could the authors clarify whether sparse interoperability is within the design scope of einx? Are there plans to support dispatching contractions to sparse-aware kernels (e.g., torch.sparse.mm), or is einx currently intended primarily for dense tensors?\n- How much runtime overhead does the einx compilation and caching mechanism add compared to equivalent direct calls in PyTorch or NumPy, and are there cases where it limits backend optimizations (e.g., for dynamic shapes or JIT execution)?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- The paper is well organized and easy to follow.\n- The notation is clearly defined and internally consistent.\n- The implementation is described in sufficient technical detail and appears complete.\n- The examples and tables illustrate coverage across many tensor operations.\n- The case study on multi-head attention demonstrates syntactic conciseness.", "weaknesses": "- The contribution is about notation and implementation, not about new ML algorithms or empirical insights.\n- The main problem addressed (API inconsistency) is largely syntactic and not shown to have a measurable impact on ML research or practice.\n- No experiments, user studies, or adoption data support the claim that einx improves model development or reduces errors.\n- The approach generalizes existing ideas from einsum and einops rather than introducing a new paradigm.\n- The motivation section frames the issue as readability and conceptual elegance, but provides no evidence that current tools are a significant bottleneck.\n- The comparison to other libraries is descriptive only, without performance or usability analysis.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Oral)