NLI : Non-uniform Linear Interpolation Approximation of Nonlinear Operations for Efficient LLMs Inference
Authors: Jiangyong Yu
Xiaomeng Han
Xing Hu
Chen Xu
Zhe jiang
Dawei Yang
Venue: ICLR 2026 Poster (2026)
Keywords: Dynamic Programming
Non-linear Approximation
Large Language Models
Quantization
Hardware Acceleration
Edge Inference
Calibration-Free
Primary area: infrastructure, software libraries, hardware, systems, etc.
TL;DR: We introduce NLI—a non-uniform linear-interpolation scheme with an ultra-light hardware block—that replaces high-precision activations, preserves LLM accuracy, and slashes compute/area for edge deployment.

Abstract: Large Language Models (LLMs) have demonstrated remarkable performance across a wide range of tasks, but their deployment is often constrained by substantial memory footprints and computational costs. While prior work has achieved significant progress in compressing and accelerating linear layers, nonlinear layers—such as SiLU, RMSNorm, and Softmax—still heavily depend on high-precision floating-point operations. In this paper, we propose a calibration-free, dynamic-programming-optimal, and hardware-friendly framework called \underline{N}on-uniform \underline{L}inear \underline{I}nterpolation (NLI). NLI is capable of efficiently approximating a variety of nonlinear functions, enabling seamless integration into LLMs and other deep neural networks with almost no loss in accuracy. NLI ingeniously recasts cutpoint selection as a dynamic-programming problem, achieving the \emph{globally} minimal interpolation error in $\mathcal{O}(M \times N^2)$ time via Bellman’s optimality principle. Based on the NLI algorithm, we also design and implement a plug-and-play universal nonlinear computation unit. Hardware experiments demonstrate that the NLI Engine achieves more than 4× improvement in computational efficiency compared to the state-of-the-art designs.

Published as a conference paper at ICLR 2026
NLI: NON-UNIFORM LINEAR INTERPOLATION APPROX- IMATION OF NONLINEAR OPERATIONS FOR EFFICIENT LLMS INFERENCE
Jiangyong Yu--
Xiaomeng Han 2*
Xing Hu
Chen Xu
Zhe Jiang?
Dawei Yangle Iss
Houmo AI
Southeast University
ABSTRACT
Large Language Models (LLMs) have demonstrated remarkable performance across a wide range of tasks, but their deployment is often constrained by substan- tial memory footprints and computational costs. While prior work has achieved significant progress in compressing and accelerating linear layers, nonlinear lay- rus-such as SiLU, RMSNorm, and softmax-stil heavily depend high- precision floating-point operations. In this paper, we propose calibration-free, synamic-rrorramning-opiiiii
and hardware-friende uniform Linear Interpolation (NLI). NLI is capable of efficiently approximating a variety of nonlinear functions, enabling seamless integration into LLMs and other deep neural networks with almost no loss in accuracy. NLI ingeniously recasts cutpoint selection as a dynamic- programmigggg problem, achieving the globally minimal interpolation error in O(M x 2 time via Bellman's 's optimality principle. Based on the NLI algorithm, we also design and implement plug-and-play univer- sal nonlinear computation unit. Hardware experiments demonstrate that the NLI Engine achieves more than 4x improvement in computational efficiency compared to the tate-of-thhearr designs.
framework called Non-
INTRODUCTION
Large language models (LLMs; e.g., GPT3-175B Zong & Krishnamachari (2022), LLaMA3-4051 Touvron et al. (2023), and Deepseck-R1-61) IB Guo et al. (2024)) have achieved remarkable success across various domains, such as text translation Tekgurler (2025), image classification Naeem et al. (2023) and text generation| Li et al. (024)). However, due to the massive model size, LLMs impose significant demands on both memory bandwidth and computation. This makes edge deployment of L.MMs extremely challenging, hindering the further application L.MSs.
Recent research efforts have focused using low-bit, high-efficiency data formats Lin 2024 to improve the computational efficiency of linear layers. For example, SmoothQuant Xiao et al. (2023)
achieves W8A8 integer quantization by smoothing activation values. OSTquant Hu al. (2255 optimizes the distribution of weights and activations through orthogonal transformations and scaling, enabling W4A8 integer quantization. Moreover, many hardware architectures support low-bit linear computations, further enhancing the efficiency of linear layers. For example, the Tensor Cores in the NVIDIA H100 NVIDIA (2023) natively support INT8 linear operations. Gemmini Gene al. (202I) also supports INT8 linear operations. However, the computation of nonlinear layers (e.g., Softmax, RMSNorm, SiL.U) LLMs still heavily depends on high-precision floating- -oint formats (such as FP32), resulting in substantial computational overhead. This further exacerbates the performance disparity between linear and nonlinear operations in LLM inference. For example, in the H100 SXM5, the FP16 linear computational power is 1024 greater than that of the special function units,
* Equual contribution "Corresponding author

---

Published as a conference paper at ICLR 2026
107
FP32 NN-LUT
SIL Activttion Viluo
100
50
201
NUKOOs
10>
50
-100
102
61
107
Liard
Qhred (eexta 325 Uneat IMs Qverd Uned 338-000
Llxm33 Linal T38
70 Qwaal: 120
5 (nea) in Qseal 38.c30
Bange of SILU activasouss scross Large Language Modes
b) Performance Compuiinn www.paaaneeeen-iierrr.....mmmm Nww.inerrgguuiimaiinn... Methade Lange Langage Midele
Figure 1: (a) Range of SiLU activations in representative LLMs; values can exceed =100 (e.g.. Qwen2.5-32B, Qwen3-8B). (b) Wikitext-2 perplexity (log-scale) with FP32, NN-LUT,, and NLI. NN-LUT collapses when outliers occur- perplexity skyrockets up to 7.0x10* whereas NLI matches FP32 across scales.
yet in scenarios with head attention size of 128, the demand for linear computational power is 256 X that of nonlinear computational power.
Nonlinear functions in LLMs typically involve hardware-intenssee transcendental functions (e-g.- exp) and some complicated algebraic functions (e.g., square root, and reciprocal functions). Some prior works accelerate these functions via hardware-friendly approximations. For example, Softermax Stevens et al. (2021) uses (ow-precision arithmetic to implement So ftmax operations through base replacement strategy. I-Bert Kim et al. (202I) proposed approximation techniques to compute GELU, Softmax, and LayerNorne rm using INT32 arithmetic. Although these designs achieve high efficiency and accuracy for certain nonlinear operations in the BERT model, they do not reliably transfer to other LLMs with hundreds of billions of parameters (e.g., LLaMA, Qwen, OPT). Moreover, their inherent hardware inflexibility further limits their applicability in NPUs. Other researchers have attempted to propose more general-purpose methods. For example, NN-LUT Yu et al. (2022) computes nonlinear functions using first-order derivative fitting (y = &I + 1), offering good generality within a certain input range. To maintain model accuracy, introduces a data calibration method. Although NN-LUT works well for the modest activation spans found in early models such as BERT, its coverage collapses when confronted with the extreme outliers characteristic of LLMs. As shown in Figure [a), the s SiLu inputs of seven representative LLMs frequently exceed 100. In contrast, NN-LUT was designed and validated only for the input domain (-5, 5). so these inputs clearly lie beyond its expected scope. When the same NN-LUT configuration is applied to these out-of-range values, the approximation error compounds throughout the network and the model fails to converge, producing a Wikitext-2 perplexity surge of up to 7 10 (Figure (b)). Taken together, for LLMs at tens hundreds of billions of parameters, current acceleration strategies for nonlinear functions remain insufficient. This combination of narrow validity ranges, reliance on calibration, and hardware inflexibility further limits robustness and deployability.
In this paper, we propose calibration dynamic- pwwpiirrmoooo--ooppiiiio framework called Non-uniform Linear Interpolation (NLI). NLI consists of two parts: NLI-Algorithm (software) and NLI-Engine (hardware). NLI-Algorithm replaces nonlinear evaluations with non- uniform interpolation in the FP16 domain. We formulate the cutpoint selection as a dynamic programming problem with an additive approximation objective that exhibits optimal substructure, enabling solution via the Bellman optimality principle. This yields globally optimal cutpoints and a calibration free lookup table that reusable across layers and models. NLI-Engine is a universal plug- and-play hardware block designed for nonlinear function computation based on the NLI algorithm. improves computational efficiency by optimizing the underlying computation strategy. Software experiments demonstrate that NLI incurs negligible accuracy loss in the open-source LLMs Qwen and LLaMA, nor does affect the accuracy of other DNN models. Hardware experiments demonstrate that the NLI Engine achieves more than 4x improvement in computational efficiency compared to the state-of-the-art designs. The main contributions presented are as follows:
and hardware-friendl:
We introduce the Non-uniform Linear Interpolation (NLI) framework, which casts cutpoint selection into a dynamic programming (DP) problem. Given a fixed nonlinear operator f and a set of N sorted candidate points in the FP16 domain, we minimize an addi- tive interpolation error that sums per-segment costs over M segments. Owing to the optimal substructure, the Bellman recursion solves for the globally optimal partition in O(M time. The resulting LUT is calibration-free freeto depends only on f and numeric settings rather than data distributions-ann is therefore reusable across layers and models. We
Algorithm:
2

---

Published as a conference paper at ICLR 2026
provide complete pipeline with implementation-read, interfaces and complexity-annen annotated pseudocode, facilitating immediate deployment with custom CUDA/Triton kernels.
Hardware Design: We designed a ceneral-purposs nonlinear computing circuit based on
NLI engine. By leveraging
co-design approach, we implemented two-level address translation module to reduce the overhead of address conversion circuits. In addition, employ pipelining to further boost throughpur.
Comprehensive Experiment: To demonstrate the practicality of NLI, we conduct exper- iments from both software and hardware perspectives. The software experiments show that our approximation strategy incurs negligible accuracy degradation in LLMs inference. Moreover, NLI exhibits strong generality, making it applicable not only to LLMs but also to other DNN models. We further synthesize the NLI engine using the SMIC 28nm technology and conduct a comprehensive analysis of its area, power, throughput and efficiency in comparison with SOTA designs.
2 BACKGROUND & RELATED WORK
2.1 NONLINEAR OPERATIONS OF LLMs
Mainstream LLMs such as LLaMA Touvron et al. (2023) and Qwen Bai et al. (2023) are typically composed of multi-head self-attention layers and feed-forward network (FFN) layers. Each self- attention layer includes one Soft tmax operation and one RMSNorm operation. Each FFN layer contains one SiLU operation and one RMSNorm operation. For MoE-based architectures such as DeepSeek-Vi V3 Guo et al. (2024), each Transformer block typically contains multiple nonlinear operators- commonly Softmax and RMSNorm- can also incorporate Si gmoid for expert routing. Our approach remains broadly applicable here, as the Sigmos i d operator can likewise be approximated through non-uniform linear interpolation, thereby offering a unified framework for various activations within MoE architectures. The definitions of these functions are as follows:
RMSNorm(z)
Softmax(1:.
SiLU(x) 1te (1)
v (2a +
E,e*, max(a))
1911
k0y)
1
(a) NN-LUT Trin With Rungee 1-88, (8) And 257 CurPeints
(6) SN-LU Train Wan Range 1-158 158) And 257 Cuthhinn
(6) (Lu)
Figure 2: Approximation quality of the SiLU activation over the range [- 1500 150), which covers
299.9% 9% of activations under our measurement protocol (see Figure. ((a) and Appendix. A.4). Panel (a) and Panel (b) show the result of NN-LUT, and panel (c) shows our NLI framework. For each method we plot (top-left) the function curve, (bottom-left) the absolute error on a logarithmic scale, and (right) zoom-in over [-5,5]. Red dots denote LUT cutpoints. NN-LUT suffers from pronounced еггor spikes, whereas NLI preserves near- mach-mechin--ee precision fidelity throughout the LLM-typical range, reducing worst-case error by several orders of magnitude.
machine-precisionn
As shown in the equations, these nonlinear operations contribute significantly to the overall runtime due to the high computational cost of floating-point arithmetic. Therefore, numerous approximation techniques have been proposed to alleviate the hardware costs. is worth noting that earlier LLMs, such as OPT et al. (2022) and BLOOM Workshop et al.] (2022), may have slight variations in their nonlinear operators compared the equations above. However, the nonlinear functions they involve are still primarily based on exponential (e*) and square root (Vrr operations.
2.2 GENERAL-PURPOST NONLINEAR COMPUTATION WORK
As a general-puroose purpose NPU, it must be capable of handling a wide range of neural networks. Different types of neural networks utilize various nonlinear operators, such Softmax, Tanh, and ArcTan.
3

---

Published as a conference paper at ICLR 2026
While circuit-level optimization for specific operator can achieve high efficiency and precision, it lacks flexibility and imposes significant limitations on broader upplicability. Therefore, we attempt to approximate nonlinear functions using sunerol-purpose approach.
2.2.1 LINEAR LUT FITTING
Linear LUT[Canton 1971) Karst 1958) approximation is more generalized method for computing nonlinear functions. It approximates various nonlinear operators by storing pairs of approximation parameters in LUT. The computation formula is as follows:
ki bı ka + by ky + bn
if x: d1,
LincarLUT(x)-
d S dis for <iss N
(2)
i 2 dx-1-
Equation shswwss that, for a fixed number of segments N. the approximation error is governed by the three linear parameters k, b, and d. NN-LUT Yu et al. (2022) extends LUT-Linear by modelling the search for (k, b, d) with a Sinear-Rel.U--Line network weights be analytically converted into the desired parameters. Although this strategy achieves high accuracy within the authors' test range, our re-implementation (the original code is not public) reveals strong dependence on the span of the training data, leading to two major trawbacks: 1. Severe extrapolation error. When the training samples cover only a narrow interval- for SiLU, [-10,10] the network generalises poorly outside that range, and its error explodes (Figure 2a)). Poor convergence on wide ranges. Expanding the interval to [-150, 1500, which covers 99.9% of the activations observed in mainstream LLMs, makes optimisation unstable; regions with high curvature (e.g.. [0] cannot be fitted well. shown in Figure 2 b). When the LUTs produced by NN-LUT are used to replace nonlinear operations in LLMs, model accuracy drops sharply (Figure [bb), underscoring that NN-LUT lacks generality for nonlinear functions with wide input ranges.
network, whose piece-wise linearity allows the
2.2.2 IYTERPOLATION-BASIT APPROXIMATIONS AND A UNIFIED ERROR BOUND
A standard strategy for approximating nonlinear operator f(-) resource-constrained accelerators to replace it by an interpolation algorithm that reconstructs outputs from preselected support points using a local rule. Here, "interpolation algorithm" covers linear, piecewise polynomial, and spline-based variants.
Piecewise polynomial and spline-base variants. Polynomial interpolation and other higher-order Liu et al. (2015) interpolation methods have lower computational efficiency, so they are not suitable for NPUs Previous works, such NVDLA NVDLA (017), have attempted to maintain accuracy by introducing an additional sub-table for regions where the function has steep variations, in addition the base table. While this approach improves precision, requires a two-level computation circuit, leading to lower efficiency. Moreover, for nonlinear functions with smooth slope variations, such as Cos, this method fails to achieve high accuracy.
Linear interpolation For single segment [a, b] with f E ([a, bl), the exact ceall-rithmmetc)) linear interpolant P(z) admits the familiar worst-case bound. However, implementations evaluate a finite-precision approximation P(x) whose error includes both the analytical interpolation term and rounding/quantizattin term. The two can be combined into a single bound:
a)
fx(x) P(a)] S
max [f'()) + xE(a,b]
max xE(a,b)
E mum
Eiite-preeision term
interpolation term
where the first term is the classical linear- interpiineroootatoon remainder and the finite triie-precision contribution can be bounded, to first order in the unit roundoffs, by
Erum S (7)(ee) + >2(um)) (1yal + 195+
Th (w)) k
ku
Here y/ = f(a) and yi+1 = f(b) are the endpoint values for the segment; u, models coeffi- cient/storage precision (e.g., 6616 vs. fp32 for slopes/L.UT values) and u, models arithmetic precision
4

---

Published as a conference paper at ICLR 2026
FP32
FP32 NIA0Om))
Ouig Valie
Output Value
160 125 15
EXP
Rsqrt
Figure 3: Comparison between the full full-precision reference and our NLI approximation on two representative nonlinear functions-exp and rsqrt. We use 10 macro-intervals: the first and last are not subdivided, and each of the middle eight is uniformly partitioned into 32 bins; accounting for shared endpoints, this yields 2 + 32 x 8 + 1 = 259 cutpoints. The lower panels plot absolute error (log scale); the worst-case error stays below 1.2 x 10 across the FP16 domain. Additional visualizations and cutpoint layouts are provided in Appendix A.6
for the multiply-add evaluation. Equation 3 clarifies the accuracy levers of interpolation- approximations: (i) the number of support points (more points shrink the segment length (b-a)), (ü) the distribution of points (allocating denser cuts where (x)) is large tightens the bound), and (iii) the storage and arithmetic precision (lower precision increases the total error via the finite-precision term 6 mmm)) In practice, three widely used design heuristics fall short for modern LLMs. First, uniform sampling is intrinsically mismatched to functions with highly nonuniform curvature: since
the worst-case linear-interpolation error on [a, b] scales as (b=a)² max [ uniform spacing leaves high-curvature regions under-resolved and wastes budget elsewhere; classical approximation theory recommends concentrating nodes in "difficult" regions (e.g., Chebyshev-lin layouts) to mitigate Runge-type instability. Second, curvature-driven point allocation is not universally applicable: for many operators used in inference, the second derivative may be unavailable in closed form, whereas the numerical term Enum typically left uncontrolled. Third, meaning/calibraaion-ben fitting ties accuracy to the span and distribution of calibration data; outside that span, extrapolation degrades and convergence becomes brittle. These limitations motivate data-free globally optimal placement of cutpoints under fixed budget.
LUT
SOFTWARE METHODOLOGY
In this section, we present calibration-free, and globally optimal (for a fixed knot budget) interval search algorithm for nonlinear functions, together with more efficient hardware computation strategy.
3.1 PROPOSED NON-UNIFORM INTERPOLATION LUT VIA DYNAMIC PROGRAMMING
The discussion in Sec 2.2.2 (Equation.) shows that intear-interpolation error reflects both curvature and finite-precision terms, which explains why uniform sampling, curvature-drive allocation, and rraniin/calliraatoo--aaee based LUTs each break down LLMs. We therefore tackle the complementary design question: given a fixed budget M cutpoints (i.e.. M - segments), where should the cutpoints be placed the FP16 grid to minimize the global error actually
by hardware? We cast this as a discrete dynamic program over the FP16 domain: the DP states and transitions (defined below) minmmize the mean relative error (5o, -. NN include endpoint clamping at both the first and last cutpoints (the left clamp is captured in the D10, k] boundary term, and the right clamp explicit tail term), and are calibration-free
We implement nonlinear operators via table lookup the FP16 grid. Let X {xo < 2 -N-11 be all finite FP16 numbers that lie in the legal domain of f (NaNs are dropped; too clamped to the nearest finite endpoint). We choose M cutpoints B = bb....,,b-1)) C X' with bn = 2g and bas-1 -1, inducing M - macro-intervals. At inference time, inputs below bo (resp. above b clamped to by (resp. bM-1).
Setup.
5

---

Published as a conference paper at ICLR 2026
Problem statement. Given f, seek B that minimizes the average relative interpolation error the FP16 grid. Within each [b,, bi+1] we approximate f by the straight line through the endpoints (b,, fb;)) and (b.41, f(b++1)).
Rationale. We optimize M cutpoints over the sorted FPI6 grid {au,.... #NN1] for a target function f. Define two DP tables with explicit shapes and index ranges:
DERMXN, PEZMxN,
LE (O...... M-1}, EE (0, N-1).
Their meanings are:
the prefix ( .kk when ay is chosen the Z-th cutpoint. P[L, k): the predecessor index (location of the (L-1)-th cutpoint) that attains D D k).
DD,,, ki:: the minimum
Error functional (mean relative error), For any segment [a;, ruJ k), let (a)( be the straight line through endpoints [ and [ (xx))). We define
[1(z)) - (a,( max max (((x))
Err(i
where we set 2-14, which equals the smallest positive normal value in IEEE 754 binary 16 (FP16). This choice avoids over-amplifying relative errors for numerically near-zero activations while aligning the denominator floor with the FP16 normal/subnormal boundary.
Boundary. Inputs smaller than the first cutpoint are replaced by the first cutpoint's value. Hence, for the first cutpoint placed at Jk.
f((z) - fxxx max m
D(0 k]
P(O, k.
+
Transition. For 1SLSM-1 and L L
{ DLL 1, i] Err(i+k) + last error(L,k) }.
D[L,k] min
where the auilcclapping penalty is nonzero only the last cutpoint:
L M
last.error
max (1, N
max(1
k=
0.
otherwise.
Set P[L, k] to the argmin index that achieves D[L, k). Optimal value. The best M cutpoints correspond to the minimum value in the last DP row:
Cost" = min DIM k..
Backtracking. Let 1e* arg min, D[M 1, k). Recover the indices of cutpoints by following predecessors:
best_points = kt P[M - 1,K*), P[M - 2, P(M - 1,,*)] ...] (then reverse to ascending).
Computational cost. The straightforward implementation is O(M N2) (cach state scans all predecessors). For typical settings (M < 11, N < 63 488) the search completes in under ten minutes single NVIDIA RTX 4090 GPU with Triton.
The complete procedure summarized in Algorithm to the full fine-grained table and search over M=259 cutpoints directly. but this inflates the DP time roughly 26x (empirically x hours on our setup) and would also mandate a large number of comparators in hardware, hurting throughput, area, and power.
naive variant could set the DP budget

---

Published as a conference paper at ICLR 2026
Instead, adopt hardware-consiisent layout with ten intro-interrall: the first and last not subdivided, and each of the middle eight is uniformly partitioned into 32 bins. Under this layout, the DP only needs to optimize the macro endpoints, i.e., ==11 cutpoints, which reduces search time by about 26x while producing LUTSs that map directly to the two-level address translation. This design yields higher hardware efficiency (fewer comparators and smaller on-chip tables). Figure (c) and Figure fouthhrr visualise the resulting approximation quality: with only 2+88 32+1 = 259 cutpoints, our NLI overlaps almost perfectly with the FP32 curve, keeping the worst-case absolute error below 1.2 x 10-3 Additional operators and cutpoint configurations are reported in Appendix A.6
3.2 HEADWARE-FRIENDEE COMPUTATION STRATEGY
In this section, we propose hardware- friendly computation strategy that utilizes two-level address translation. By leveraging simple computations, this approach significantly reduces the number of comparators and improves hardware efficiency.
Traditional address translation modules rely on multiple comparators. For example, with 259 cut points, the system generates 258 sub-intervals, requiring 259 parallel comparisons to determine the corresponding LUT address for an input data point. This leads to significant hardware overhead. To address this, we adopt a two-level address translation strategy. First, we divide the 259 cut points into structure of (2 + 8 32 + 1), meaning 8 sub-intervals, each containing 32 cut points, along with two boundary values representing positive and negative infinity. Then, we use 10 comparators determine which major interval the input data belongs to. Next, based on the interval index, we retrieve the multiply scale factor for that interval. After performing the multiplication, we apply floor operation, and the resulting integer value serves as the LUT index. By utilizing pipelined design, the latency introduced by the two-level approach is effectively hidden. Additionally, all multiply scale factors are precomputed offline and preloaded into dedicated registers. this case, only 10 16-bit registers are required for storage. The detailed steps are shown in Algorithm [
As shown in Algorithm| 2 the two-level address translation approach utilizes simple lookup and arithmetic operations. By applying the transformation y kz within each sub-interval, the address conversion is efficiently completed, eliminating the need for over 200 FP16 comparators. The computation formula for the linear interpolation can be summarized as follows:
y = Decimal (LUT(Inndez + 1] LUT(]ndez]) + LUT [[nex)
(5)
HARDWARE METHODOLOGY
Stage
Stage
Stage
Stage
Comppantox
Index
Outgat
Inpat
Thra
b.
b.bi
LUT
Mul
Aáleco
LUTO
Tubel
LUT
LUTT
Sub
Mul
Sub
LUT
LUT
The hardware circuit design of a nonlinear computing unit, using two computing circuits sharing set of Multiply scale factor and function value registers as an example.
Figure 4:
In this section, we introduce the NLI Engine, a soug-and-play hardware module that leverages the optimized computation flow of the NLI algorithm to enable efficient nonlinear computation on NPUs.
Since current DNN accelerators fabricated with 28nm technology, e.g., Google TPU v1 Google (2016), operate at around IGHz, we have adopted a four-stage pipeline design to align with the clock frequencies of most NPUs. The NLI engine implements the following functions: LUT and interval cutpoint preloading, two-level address translation, interpolation coefficient (Decimal) computation, and linear interpolation algorithm. The following are the detailed hardware design specifications of NLI engine:
Stage Major-interval select & alignment. An interval comparator (10 comparators) selects the macro-ine macro-interval index I E (0....,9) 9} using preloaded left boundaries [t]t[A]; inputs are clamped to [1eft(0), (0), et
An FP16 subtractor computes the aligned offset Axr left
7

---

Published as a conference paper at ICLR 2026
Stage - Micro-address generation. Each macro-interval has preloaded scale mul[4] and base pointer base [ (start index in the global LUT). An FPI6 multiplier forms u = Ar mu mu] the floor unit yields the integer micro-index for the middle eight intervals, a = 0 at the ends). The global LUT address is base + a.
[u] and the fractional coefficient
a (a [0, 31]
Stage 3 3- Table read & slope prep. A dual-port SRAM (259 entries) returns two adjacent values: Vo = LUT(g) and V1 = LUT g 1] one cycle. An FP16 subtractor computes the local slope Ay 1/1 y/o-
Stage 4 - Linear interpolation. An FP16 multiplier-adder evaluates yoo + Ay (optionally fused FMA), then rounds to FP16.
This ftur-stage pipeline, together with the two-level address translation (7, a), reduces comparators from 259 10 and shrinks addresss-ransslation overhead, improving throughput, area, and power.
5 EVALUATION
Table 1: NLI accuracy across datasets. Columns are datasets (higher is better except perplexity). Boldface rows highlight our method.
Accuracy (f)
Perplexity (1)
Model
Method
MMLU
GSMSK
HumanEval
Zero-shot Avg
Wikitext-2
FP32
62.16
50.19
35.37
68.11
6.14
Llama3-8B
NN-LUT
60.11
49.42
65.93
8.28
NLI
62.14
50.49
35.37
73.78 68.24
40.85
2.86
Llamu3-70B
NN-LUT
81.27
FP32
70.56
44,28
40.24
67.48
7.46
Qwen2.5-7B
NN-LUT
25.51
0 39.63
NLA
70.67
43,97
67.63
7.46
FP32
81.74
70,13
56.71
70.76
5.32
Qwen2.5-32B
NN-LUT
25.51
0
()
30.70
70360
NLI
81.68
70.07
55.88
70.67
5.32
FP32
79.26
84.44
50.84
72.42
4.81
Owen1.5-110
NN-LUT
75.99
76.
43.03
69.08
0.83
NLI
79.31
84.41
50.16
72.48
4,81
FP32
72.94
88, 18,10
63.41
66.68
9.72
Qwen3-8B
NN-LUT
23.50
0
0
33.61
825.3 31
NLI
72.98
88.17
62.59
66.76
9.73
FP32
77.86
85,
31,21
67.59
8.70
Owen3-30B-AN
VN-LUT
$7.88
85.39
30.38
0
65.60
67.65
10.76
8.70
5.1
SOFTWARE EVALUATION
For evaluation, we first run NLI search macro cutpoints for each nonlinear operator under 10-macro-interval layout: the middle eight intervals are uniformly split into 32 bins, while the first/last are unsplit with endpoint clamping. We then replace nonlinear operators in Llama and Qwen (Py [PyTorch] Paszke (2019)) and report perplexity on Wikitext-2 Merity et al. (016)). a stan- dard zero suite (ARC-cle Clark et (2018), BoolQ Clark et al. (2019). PIQA Bisk al. (2020), HellaSelaaw wag Zellers et al. (2019), OBQA Mihaylov et al. (018)). LAMBADA Paperno et al. (2016) SIQA Sap al. (2019). WinoGrande Sakaguchi et al. (2021), and three widely used bench- marks MMLU Hendrycks et al. (2020) (broad factual/problem-soliin
across 57 disciplines), HumanEval Chen et al. (2021) (functional code-generation accuracy), and GSMSK Cobbe et al. (202T) (multi-step grade-school math). We cover multiple model scales and, test generality beyond LLMs, repeat the substitution on ViT and CNNs.
From Table we observe that replacing nonlinear operators with NLI in Llama and Qwen yields no accuracy drop on the zero-shot suite (detailed results in Appendix AA5.2) and does not increase PPL. Moreover, performance MMLU, HumanEval, and GSM8k remains nearly on par with FP32. Our nonlinear computation strategy has minimal impact on the accuracy of open-source Llama/Qwen models-even without any data calibration. Beyond LLMs, substituting NLI for nonlinear operators in ViT and representative CNNs yields no statistically significant accuracy degradation; full per-model results are reported in Appendix A A.5.1 (Table
8

---

Published as a conference paper at ICLR 2026
5.1.1 ABLATION
Table 2: Ablation I on Qwen2.5-7B: two-level Table 3: Ablation on Qwen2.5-7B: accuracy (1) and cutpoint-search time (4).
NLI (259) macro-only (11).
Method
Curpoints
MML.U
GSM8k
Method
Curpoints
MMLU
GSMSK
Search time(s)
FP32
70.56
44,28
FP32
(248x32+1)
70.67
43,97
NLI 1)+1
259
70.67
43.97
610
Macro-oni only (DP, M1)
11
21,14
0
Non Non-uniform 259 (DP)
259
70.65
44,08
17000
As shown in Table 2 we evaluate on Qwen2.5-7B with zero-shot MMLU and GSM8K. Compared to the proposed two-level layout NLI (259 total cutpoints), the macro-only variant optimizes only M=11 endpoints and applies a single piecewise-linear fit over the FP16 grid (with no per-macro uniform sub-bins). This ablation isolates the benefit of uniform micro-partitioning within each macro- veero-interval-a demonstrating that using only 11 cutpoints is insufficient on both benchmarks.
We compare NLI with a direct DP over 259 non-uniform cutpoints (no macro/micro constraint) on Qwen2.5-7B. As shown in Table 3 accuracy on MMLU/GSM8 is essentially unchanged, while the search time explodes (~~ (~~28x slower) and the resulting layout hardware-unfrient incurring higher area/latency costs.
We further compare NLI with two common heuristics us- ing the same total budget of 259 points: (i) Uniform 259 (uniformly spaced over the FP16 grid), and (ii) Curvature 259 (density proportional to curvature proxy). Both use the same linear interpolation and inference pipeline as NLI. Table summarizes results.
Table 4: Ablation III on Qwen2.5-7B: accuracy comparison on MMLU and GSM8K.
Method
Cutpoints
MMLU
GSMSK
TP32
20.56
44.28
5.2 HARDWARE EVALUATION
NLJ (2+8::3+2++)
Uniform 250
259
259
45
70.65
43,97
18:13
Curvature 259
259
63.74
32.58
The experiments the previous sections have demonstrated that our NLI approximation method achieves high accuracy and generality. In this section, to demonstrate the efficiency of the NLI engine, we compare NLI engine with two state-of-the-aa nonlinear computational units.
We implemented the hardware circuit design using Chisel Bachrach al. (012)). The circuit was synthesized with Design Compiler Muchnick (1997) under the SMIC 28nm process library to obtain area, power, and timing information.
Table 5: Hardware area breakdown.
Table 6: Hardware comparison.
LUT
Comparator
Multiplier
Adder
Others
Total (pm))
Clock Freq
Area (sitc))
Power (ow))(
Thooghhut
Efficiency
NN-LUT
12268
10496
205
134
135
23238
NN-LUT RI LUT
IGHz
23238
46
LG
0.94
RILUT
12268
10496
410
268
305
23647
IGHt
23647
48
IG
0.58
NLI
6445
205
836
191
7787
NLI
IGH
7787
34
16
3.78
Table 5 shows the areas of LUTSs, multipliers, adders, and Others (registers, shifters, etc.) in the three general nonlinear computational units: NLI, NN-LUT, and RI-LUT. It can be seen that NLI (with 259 cut points) saves 68% and 69% in area compared with the other two SOTAS (NN-LUT and RI-LUT, each with 256 cut points).
NN-LUT Yu et al. (2022) and RI-LUT Kim et al. 2023) require storing 512 x 16 bits of data (256 K values and 256 values). NLI only needs store data for 259 16-bit cut points and 10 16-bit scale factors, so it has smaller LUT area. Also, our hardware-friendl friendly algorithm cuts down the number of comparators. NN-LUT and RI-LUT need 256 comparators divide into 256 interval addresses, while NLI only needs 10 comparators for ten intervals.
Table 6 provides a detailed comparison of the NLI Engine, NN-LUT, and RI-LUT hardware modules under GHz clock frequency in terms of area, power, Throughput (the number of nonlinear operators computed per cycle x clock frequency), and Efficiency (Thhroughpul/(urrg The NLI Engine exhibits lower power consumption, primarily due to reduced static power resulting from fewer LUTs and comparator modules. Since all three hardware units employ pipelining, each cycle produces one completed result once the pipeline is filled. Therefore, the throughput of all three modules is 1G. Benefiting from its lower area and power consumption, the NIJ Engine achieves 4.02x higher efficiency than NN-LUT and x.29x higher efficiency than RI-LUT.
x power)).

---

Published as a conference paper at ICLR 2026
6 CONCLUSION
In this paper, propose NLI, non-uniform linear interpolation method for efficiently approximat- ing nonlinear functions in large language models. By formulating cutpoint selection as dynamic programming problem, NLI achieves near-optimal accuracy with minimal hardware overhead. Ex- periments demonstrate that NLI maintains model accuracy without calibration, generalizes well across diverse models, and significantly reduces hardware resource usage compared state-of-the-art methods. We believe NLJ provides a practical solution for deploying large models efficiently on resource-limited hardware.
ETHICS STATEMENT
This work does not involve human subjects, personal data, or sensitive content. focuses solely on algorithmic and hardware-level optimization of nonlinear function approximation. Therefore, we believe does not raise ethical concerns.
REPRODUCIBILITY STATEMENT
We provide complete details our algorithms, hyperparameters, and evaluation protocols in the main paper and appendix. All models are evaluated on publicly available benchmarks (Wikitext-2, ARC, BoolQ, PIQA, HellaSwag, OBQA, LAMBADA, SIQA, WinoGrande, MMLJU, HumanEval, GSM8k, and standard vision datasets). The code for constructing lookup tables, performing DP cutpoint search, and reproducing our experiments will be released upon publication. These resources will ensure full reproducibility of the reported results.
REFERENCES
Jonathan Bachrach, Huy Vo, Brian Richards, Yunsup Lee, Andrew Waterman, Rimas Avižienis, John Wawrzynek, and Krste Asanović. Chisel: constructing hardware in a scala embedded language. In Proceedings of the 49th annual design automation conference, pp. 1216-1225, 2012.
Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, et al. Qwen technical report. arXiv preprint arXiv:2309 arXiv:2309.16609, 2023.
Yonatan Bisk, Rowan Zellers, Jianfeng Gao, Yejin Choi, al. Piqa: Reasoning about physical commonsense natural language. Proceedings of the AAAI conference on artificial intelligence, volume 34, pp. 7432-7439, 2020.
Antonio Cantoni. Optimal curve fitting with piecewise linear funetions. IEEE Transactions on Computers, 20(01):59 67, 1971.
Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, and Sergey Zagoruyko. End-to-end object detection with transformers. In European conference on computer vision, pp. 213-229 Springer, 2020.
Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde De Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, et al. Evaluating large language models trained on code. arXiv preprint arXiv:2107.03374 2021.
Christopher Clark, Kenton Lee, Ming-War Wei Chang, Tom Kwiatkowski, Michael Collins, and Kristina Toutanova. Boolq: Exploring the surprising difficulty of natural yes/no questions. arXiv preprint arXiv:/905, 10044, 2019.
Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. Think you have solved question answering? try are, the ai2 reasoning challenge. arXiv preprint arXiv: 003505 2018.
Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reñchiro Nakano, et al. Training verifiers to solve math word problems. arXiv preprint arXiv:21 10. 14168, 2021.
10

---

Published as a conference paper at ICLR 2026
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghami, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010. 1929, 2020.
Hasan Gene, Seah Kim, Alon Amid, Ameer Haj-Ali, Vighnesh Iyer, Pranav Prakash, Jenry Zhao, Daniel Grubb, Harrison Liew, Howard Mao, et al. Gemmini: Enabling systematic deep-learning architecture evaluation via full-stack integration. In 202/ 58th ACM/IEEE Design Automation Conference (DAC), pp. 769-774. IEEE, 2021.
Google. Tpu transformation:
A look back at 10 years of ai-specialized URL htttss://cclottt http://ooooooooooll.... google ctm/t caom/transoonno form/ specialized chips tpu history gen ai Accessed March 24, 2025.
chips,
2016.
Daya Guo, Qihao Zhu, Dejian Yang, Zhenda Xie, Kai Dong, Wentao Zhang, Guanting Chen, Xiao Bi, Yu Wu, YK Li, al. Deepseek-coder, When the large language model meets programming-tie rise of code intelligence. arXiv preprint arXiv:2401. 14196, 2024.
Dan Hendrycks, Collin Burn,, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Measuring massive multitask language understanding. arXiv preprins arXiv:2009.00 03300, 2020.
Xing Hu, Yuan Cheng, Dawei Yang, Zukang Xu, Zhihang Yuan, Jiangyong Yu, Chen Xu, Zhe Jiang, and Sifan Zhou Ostquant: Refining large language model quantization with orthogonal and scaling transformations for better distribution fitting, arXiv preprint arXiv:2501.1300 2025.
Otto J Karst. Linear curve fitting using least deviations. Journal of the American Statistical Association, 53(281):118-1155 1958.
Janghyeon Kim, Janghwan Lee, Jungwook Choi, JeongHo Han, and Sangheon Lee. Range-invariant approximation of non-linear operations for efficient bert fine-tuning. In 2023 60th ACM/IEEE Design Automation Conference (DAC), pp. 1-6. IEEE, 2023.
Sehoon Kim, Amir Gholami, Zhewei Yao, Michael W Mahoney, and Kurt Keutzer. 1-bert: Integer- only bert quantization. In International conference machine learning, pp. 5506-5518. PMLR, 2021.
Junyi Li, Tianyi Tang, Wayne Xin Zhao, Jian- Yun Nie, and Ji-Rong Wen. Pre-trained language models for text generation: A survey. ACM Computing Surveys, 56(9): 2024.
Ji Lin, Jiaming Tang, Haotian Tang, Shang Yang, Wei-Ming Chen, Wei-Chen Wang, Guangxuan Xiao, Xingyu Dang, Chuang Gan, and Song Han. Awq: Activation-aware weight quantization for on-device Ilm compression and acceleration. Proceedings of Machine Learning and Systems, 87-100, 2024.
Shen Liu, Vo Anh, James McGree, Erhan Kozan, and Rodney C Wolff. A new approach to spatial data interpolation using higher-order statistics. Stochastic Environmental Research and Risk Assessment, 29:1679-1690, 2015.
Stephen Merity, Caiming Xiong, James Bradbury, and Richard Socher. Pointer sentinel mixture models. arXiv preprini 16XXvv:609.008.... 2016.
Todor Mihaylov, Peter Clark, Tushar Khot, and Ashish Sabharwal. Can a suit of armor conduct electricity? new dataset for open book question answering. arXiv preprint 1809v:1809.000 2018.
Steven Muchnick. Advanced compiler design implementation. Morgan kaufmann, 1997.
Muhammad Ferjad Naeem, Muhammad Gul Zain Ali Khan, Yongqin Xian, Muhammad Zeshan Afzal, Didier Stricker, Luc Van Gool, and Federico Tombari. 1mvformer: Large language model generated multi-view document supervision for zero-shot image classification. In Proceedings of the IEFF/CVF Conference on Computer Vision and Patern Recognition, pp. 15169-15179) 2023.
11

---

Published as a conference paper at ICLR 2026
NVDLA. Nvdla primer- nvdla documentation, 2017. URL http://vdla..rr//pp/0
html
NVIDIA. Nvidia h100 tensor core gpu.
data- center/h100/ 2023, Accessed: 2023-12-12.
Denis Paperno, Germán Kruszewski, Angeliki Lazaridou, Quan Ngoc Pham, Raffaella Bernardi, Sandro Pezzelle, Marco Baroni, Gemma Boleda, and Raquel Fernández. The lambada dataset: Word prediction requiring broad discourse context. arXiv preprint arXiv: 06031, 2016.
A Paszke. Pytorch: An imperative style, nigh-performance deep learning library. arXiv preprint arXiv:1912.01703. 2019.
Keisuke Sakaguchi, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi. Winogrande: An adversarial winograd schema challenge at scale. Communications of the ACM, 64(9):99-106. 2021.
Maarten Sap, Hannah Rashkin, Derek Chen, Ronan LeBras, and Yejin Choi. Socialiqa: Commonsense reasoning about social interactions. arXiv preprint (904.0990.0972 2019.
Jacob R Stevens, Rangharajan Venkatesan, Steve Dai, Brucek Khailany, and Anand Raghunathan. Softermax: Hardware/software co-design of an efficient softmax for transformers. In 2021 58th ACM/IEEE Design Automation Conference (DAC), pp. 469-474 IEEE, 2021.
Merve Tekgurler. Llms for translation: Historical, low-resourced languages and contemporary ai models. arXiv preprint MxXXi:25033.1890 2025.
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models. arXiv preprint 23977,, 2023.
Rejin Varghese and M Sambath. Yolov8: A novel object detection algorithm with enhanced perfor- mance and robustness. 2024 International Conference on Advances in Data Engineering and Intelligent Computing Systems (ADICS), pp. 1-6. IEEE, 2024.
BigScience Workshop. Teven Le Scao, Angela Fan, Christopher Akiki, Ellie Pavlick, Suzana Ilić, Daniel Hesslow, Roman Castagné, Alexandra Sasha Luccioni, François Yvon, et al. Bloom: A 176b- trob-parrmmter open-acces multilingual language model. arXiv preprint arXiv:2211 1.0111000000000 2022.
Guangxuan Xiao, Ji Lin, Mickael Seznec, Hao Wu, Julien Demouth, and Song Han. Smoothquant: Accurate and efficient post-training quantization for large language models. In International Conference on Machine Learning, pp. 38087-38099. PMLR, 2023.
Joonsang Yu, Junki Park, Seongmin Park, Minsoo Kim, Sihwa Lee, Dong Hyun Lee, and Jungwook Choi. Nn-lut: Neural approximation of non-linear operations for efficient transformer inference. In Proceedings of the 59th ACM/IEEE Design Automation Conference, pp. 577-582, 2022.
Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi. Hellaswag: Can a machine really finish your sentence? arXiv preprint arXiv: 909990000730. 2019.
Susan Zhang, Stephen Roller, Naman Goyal, Mikel Artetxe, Moya Chen, Shuohui Chen, Christopher Dewan, Mona Diab, Xian Li, Xi Victoria Lin, et al. Opt: Open pre-trate mr-ttrineed transformer language models. arXiv preprint arXiv:2205.0 200.00000000 2022.
Yian Zhao, Wenyu Lv. Shangliang Xu, Jinman Wei, Guanzhong Wang, Qingqing Dang, Yi Liu, and Jie Chen. Detrs beat yolos on real-time object detection. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 16965-16974 2024.
Mingyu Zong and Bhaskar Krishnamachari. A survey on gpt-3. arXiv preprint arXiv:22/2.00857, 2022.
12

---

Published as a conference paper at ICLR 2026
A APPENDIX
A.I DP-OPTIMAL MACRO CUTPOINT SEARCH
Algorithm DP-Optimal Macro Cutpoint Search (NLI; mean relative error with endpoint clamping)
Input: Sorted FP16 grid X' = (Fo..... NN- }: target function f(-); number of cutpoints M 2: small constant 0.
Output: Optimal cutpoints = ( bM-1} with by E X' and their values (f(b))}
Precompute values
for k - 0 to N - do
L V/k + f(xx)
2. Define error functionals
/ Mean relative error of the endpoint- line on [i
Dj - (a(()))
Err(i,k) k) A
max Pa,(2)) +
yh y1 (x xi)
k-iti
j=i
// night-end clamping penalty for the last cutpoint (zero if k N-1)
TailCIamp(k)
A
N-1-E 2j=bt1
N-1
(b.leT)
Ms Uk
k< 1
0,
N=N-1 -1,
Initialize DP tables
11 d MMN stores minimal prefix cost; p MxN stores
predecessors
Initialize d[L,k] 6 +00 p|L, f for all L,k; // Left-end clamping: first cutpoint at 2t uses constant U on k0
for k 0 to N - 1 do
d(0, k] k+1 Ej= (1,,
p(0, k]
+ k;
4. Fill DP (macro endpoints) for + 1 to M do for k L to N do
best +- too; arg - 1; for i -
to k 1 do
val - dLL 1, Err(i, + [z==M-1) TailClamp(k))
val < best then
best val;;
arg +
d[L, k| best;
pLL, k] t- arg:
Backtrack optimal cutpoints
ki* +- arg Min d[M 1,
// Recover indices of M cutpoints, from last to first
idx[M - 1] + k=:;
for +- M- down to 1 do
L idx[L 1] - LLL, idx [d][
for +- to M do
L br. + T s(br.) + M/dxL]j;
return B {bo. ... 1m- and M
13

---

Published as a conference paper at ICLR 2026
A.2 NLI COMPUTATION FLOW
Algorithm 2: NLI Computation Flow
Input: FP16 imput interval endpoints Point[0:10) (11 points); per-interval scales M ul[999 (10 values); LUT values [UT(5) (259 points); uniform bins per macro-interval
D2 for intervals ....8.
Output: Nonlinear output y/.
Constants: M-10 intervals; indices are O-based.;
Preload registers: Point_Reg < Point, Mul + Mu,, LUT _Reg- + LUT;; // one-time load
Clamp input to domain: + clip(æ, Point Reg[O], Point Reg[10])::
Locate interval Index E (O...., 9}: for t 0 to do if Point Regli] S. Point PPoint_Reliiti Regli+1] then
......,).
L Index 2; break
// Equivalently, this can be done via bucketize.
Local coordinate in interval Indez:
Temp Point_ (nder);
// offset within the interval
Mul I Temp t- Temp Mul Reg[Indez]:
// scaled position
Address - Ml Templ:
integer bin (0 for intervals 0/9; 0..31
for 1). 8)
Decimal Mul_Temp- Address;
// fractional part within the bin
Global LUT index indices:
50+ Address,
if Index 0,
indices
1 (Index-1)- Dr, + Address,
if Index 1.
Linear interpolation from LUT: Left LUT Reg indices). Right +- LUT Reglindices+1] + Left + Decimal x (Right - Left)
Boundary saturation:
x Poind. Reg [)] then y +- LUT Reg 00: if x 2 Point. Reg [10] then y <- LUT Reg (558)
return y
A.3 LLM USAGE DECLARATION
We disclose that large language model (LLM) tools used only for language editing copy- editing, stylistic smoothing, and minor rephrasing of prose-and limited formatting assistance. No LLM was involved in conceiving the research, proposing methods, generating technical content, writing code, running experiments, analyzing data, or drawing conclusions.
All substantive elements of this work were completed by the authors, including:
problem formulation, research design, and overall narrative structure;
algorithmic development, theoretical reasoning/derriaaioons, implementation, and debugging;
experimental design, data collection and preprocessing, execution, evaluation, and interpre- tation results.
Any text suggestions produced by LLM tools reviewed and edited by the authors. The authors accept full responsibility for the scientific validity, accuracy, and originality the manuscript and affirm adherence standards of academic integrity.
14

---

Published as a conference paper at ICLR 2026
A.4 ACTIVATION COVERAGE MEASUREMENT PROTOCOL
Models and operators. We measure the pre-activation inputs to nonlinear operators (e.g., SiLU, RMSNorm, Softmax) across the LLMs listed in Figure. (a).
Data and setup. To align with main evaluation, Sec. 5 (e.g., Wikitext-2 for perplexity and the zero-shot suite for accuracy). All models are run in FP16 inference with the same tokenization and maximum sequence length in our evaluation.
the same public corpora employed in
Aggregation across layers and tokens. For each model, we collect activations from all layers over all tokens. We then compute the 0.05th and 99. 99.95th percentiles of the resulting distribution. To form symmetric interval, we set mmodel = max 90.0005 (n.99955 and define the model-speciie range
as T modely mmndel.
Cross-model coverage. We aggregate models by taking the union of their symmetric ranges, i.e., Tmax = maXmelel moddel. and define the LLM-ttpical domain as [-Tmaxx Tmax) In our measurements, Tmax 2 150, hence 150, 150] covers 299.9% of observed activations.
Outliers. Values outside this domain account for -0.1 10.1% of activations and are clamped at runtime by our engine; they have negligible impact on end-to-end accuracy.
A.5 FULL EXPERIMENT RESULTS.
A.5.1 SUPPLEMENTARY EVALUATION ON NON-LLM TASKS
Table 7: Accuracy performance of NLI on vision models
mAP (%) (obj det)
Top-1 Ace (%)
Model
Method
(cls)
FP32
39,4 4
DETR
NLI
39.4
FP32
74.7
ViT-Small
NLI
74.7
FP32
52.5
RT-DETR-L
NLI
52.5
FP32
50.1
YOLOv8-M
NLI
50.1
To further verify the generality of NLI beyond LLMs, replaced the nonlinear operators in repre- sentative vision models, including YOLOv8 Varghese Sambath (2024), DETR Carion et al. (2020), ViT Dosovitkkk et al. (2020), and RT-DETR Zhao et al. (2024). As shown in Table 7 employing NLI for nonlinear operator computation does not incur any accuracy degradation, demonstrating that the proposed framework is broadly applicable across diverse architectures and modalities.
A.5.2 FULL ZERO- SHOT EVALUATION ON LLMS
Table 8 reports the complete zero-shot evaluation results of NLI, NN-LUT, and FP32 baselines all considered LLMs. Compared with NN-LUT, NLI consistently preserves baseline-level accuracy across tasks and scales, while avoiding the severe degradation observed with NN-LUT.
A.6 ADDITIONAL NLI VISUALISATIONS AND FP16 CUTPOINTS TABLES
To demonstrate the breadth and stability of NLI across common nonlinear operators, Figure visualises fits for eight functions widely used in modern models: exp, gelu, rsgrt. reci pocccrocca. hardswish, mish sigmoid. and tanh. Each panel contains two subplots: the left shows the full FP16-domain behaviour (top: FP32 reference teal and NLI in orange; bottom: absolute error log scale), and the right provides zoom-in around the high-curvature region that typically
15

---

Published as a conference paper at ICLR 2026
Table 8: NLI accuracy on multiple LLMs. Boldface rows highlight our method.
Zero-Shot (*)
Perplexity
Model
Method
ARC-c
ARC-e
BoolQ
PIQA
HcllaS
OBQA
Lam.
SIQA
WinoG.
Avg.
Wikitex-22
Llama3-8D
FP32
53.16
77.78
81.25
80.74
79.10
44.80
75.66
47
73.40
68.11
6.14
NN-LUT
50.11
75.21
79.18
78.93
77.66
42.50
75.07
45.47
71.24
65.93
8.28
NLI
53.67
77.78
81.41
80.74
79.23
44.80
75.70
47.08
73.72
68.24
6.14
Llama3-70B
86.03
50.67
NN-LUT
63.16
84.99
84.06
82.69
83.51
46.16
78.97
50.01
78.76
72.48
5.13
NLI
64.33
86.07
84.39
84,92
50.67
80.74
FP32
51.28
76.47
85.96
78.67
79.55 55
48.00
67.67
50.36
69.3 38
67.48
7.46
Qwen2.5-7B
NN.LUT
19.62
25.38
37.83
12
25.61
28.20
0.00
33.01
49.41
30.13
281144
NLJ
51.37
76.39
83.96
79.82
79.46
48.20
67.67
50.61
69.22
67.63
7.46
Qwcn2.5-32B
FP32
58.36
77.19
89 69
81.39
85.25
46.00
75.26
50.05
73.64
70.76
5.52
20.99
23.95
NLJ
58.28
77.44
89.72
81.23
85.18
45.40
75.20
50.15
73.40
70.67
5.32
Qwea1.5-110B
NN-LUT
52.01
74.08
83.88
80.52
83.77
42.40
75 99
51.32
77.96
69.08
6.83
NLA
55.80
77.10
88.96
83,90
86.11
46.80
78.36
54.25
$1.06
72.48
4.81
Qwen3-8BB
FP32
55.29
80.93
86.70
77.69
74,92
41.40
64.04
51.84
67.32
66.68
9.72
NN-LUT
23.01
28.99
41.65
57.16
30.13
29.71
4,90
36.14
50.79
33.61
825,31
55.29
80.93
86.70
77.90
75.11
64.21
51.79
67.36
66.76
Owen3-30B-A30
NN-LUT
FP32
52.30
30.71
79.38
88, 59
79.43
76.68
45
64.84
51.23
69.85
67.59
8.70
NLI
52.24
79.58
88.57
79.51
77.68
44.98
64.91
51.46
69.94
67.65
8,70
dominates the global error budget. Red dots mark the LUT cutpoints produced by our 2+8 32++1 budget (top-level endpoints plus uniformly spaced interior cuts). Across all eight operators, NLI tracks the FP32 curve almost perfectly; the worst-case absolute error remains within 1.5x10 over the entire domain and is orders of magnitude smaller in most subranges.
Table 9 lists the top-level FPI6 cutpoint endpoints (in decimal) that define 10 macro-intervals for each operator. The full lookup table with 2 + 8 x 32 + = 259 entries is obtained by placing 32 uniformly spaced points inside each macro-interval and adding two boundary values. During inference, FP16 inputs that fall below the smallest endpoint or above the largest endpoint are clamped the respective boundary before lookup, which guarantees numerical stability for extreme values (e.g., very small arguments in rsgrt and reciprocal). For completeness, we also release the exact LUTss used in our experiments.
16

---

Published as a conference paper at ICLR 2026
F232
Outpun Value
NLIOOus)
Ourpe Valee
-19
130
30
160
126
EXP
(b) GeL.U
EP52
NLIOOus)
(c) Rsqrt
Reciprocal
EP32
Onigut Value
NI ((Orn))
103
300
150
HardSwish
Mish
EP32
Oatpot Valoe
NLtOOush
NLNOusi
130
Sigmoid
(h) Tanh
Figure 5: NLI approximation quality for eight representative nonlinear operators: (a) exp, (b) gelu. (c) rsqrt, (d) reciprocal, (c) hardswish.
mi sh, (g) si sigmoid, d, and (h) tanh. For each operator, the left subplot shows the full-domain fit (top: FP32 reference vs. NLI; bottom: absolute error in log scale), while the right subplot zooms into the high-curvature region. Red dots denote LUT cutpoints generated under an 2+8 x 32+1 budget. NLI closely overlaps the FP32 reference, keeping the worst-case absolute error within 1.5x
across the FP16 domain.
17

---

Published as a conference paper at ICLR 2026
Function
Range
#Segments
'Top-level cutpoints (FPI6, decimal)
gelu
5,5390625, 65504.0]
21-8 32
-3.5390625, -5.15625, -3.18359375,
-C, 980468775, -D. 12.1222288048755
-C, 0.00377403271488833..
0. 9.9035247802234375
0.11322021484375,
0,77666171875, 4.10546875, 65504.0
silu
[ 20.339375, 65504.0J
24-8 x 324-1
-20.359375, -17.109375, -8.3671875,
-1 - -0.25511533
0.0072441101074211817 0.228515625,
0.0072214477265565,,
1.58203125, 16.46875, 65504.0
exp
[-17.34375, 11.0859875]
2488 32+1
-17.34375, -15.171875, -8.890625,
-..2134375, -2.35546875,
-C, C.3359984375
0. 0.91:50330625,,
3.451171875, 6.84765625, 10.9453125,
11. 11.3859375
reciprocal
[1.5318394x10- 65500.00
2+-8 x 32+1
1. 1.5318394e-05, 2.2590160e-05,
4. 89923022e-04, 7.0533752e-
8. 8.8373906e-02, 1.07421975, 15.
244.3, 3694.0, 46560.0, 65504.0
rsgrt
15.9604643x00**, 63504.0) 636504..0
2+8 x 32+1
5. 9.9604645--08, 7.7436038e-07,
1.1140108e-04, 1.8644333e-033
30229997e-- 0.48193359375,
1.7734375, 129.75, 2406.0, 47456.0,
65504.0
hardswish
[-3.0, 65504.0]
218x3211
-3.0, -2.98437, -1.8399025,
C..539665, - -..03777711..
0.0007438659966796111,
0.30349426266533125, 0. 0.11968994140625,
65504.0 0. 011369110525,,
tanh
[- 4.5078125, 4.5078125]
2+88 x 32+1
-4.5078125,
-C. 53327333337,, -00025555550
0.0364990234375, 0.42382255,
1.076171875, 2.0390625, 4.0625,
4.5378125
mish
[-20.34375, 65504.0]
2+8 32+1
220.34375, -19.90625,
-10.921875, -5.2265625,
-1. -1.65323477,, -0.2370065548855
- 6.00699615548511562,
0.01538848876955125,
0.491455078125,
4.73703125, 65504.0
sigmoid
17.34375, 8.3203125]
2+8 x 32+1 -17.34375,
88 -..1562, -6.3046875, -4.421875,
2.6640625, -0.7998046875, 1. 1.9462890625, 6.90233375, 8.3203125
Table Top-level cutpoint endpoints used by NLI. Each row lists FP16 cutpoints (decimal) that
define 10 macro-intervals placing 32 uniformly spaced cut points inside each macro-interval plus two boundary values yields a total of 2+8 x 32+1 259 entries. Inference uses clamping: FP16-domain inputs below the smallest cutpoint above the largest cuipoint clipped the respective endpoint.
18


Reviews:
[{"review_id": "F6CyYNFCEF", "reviewer": "Reviewer_PkiK", "summary": "The paper proposes a hardware-friendly scheme, NLI, for approximating nonlinear operations in FP16 domain for faster inference. NLI uses non-uniform spline linear interpolation in the FP16 domain, and selects interpolation cut-points with dynamic-programming to minimize interpolation error, yielding a calibration-free lookup table for each nonlinear operation. The authors also design a plug-and-play NLI unit and present hardware experiments showing speedup improvement in compute/area efficiency compared to prior designs.", "questions": "- what’s a good error level to avoid model performances degradations (like on zero-shot tasks on perplexity)? Currently $1.2\\sim1.5\\times10^{-3}$ looks good\n\n- Do you need to pre-fix the set of cutpoints before implementing the NLI hardware version? Or alternatively, after implementing the proposed NLI on hardware, does it support taking in any custom set of cutpoints?\n\n- In table 3, the non-uniform version performs worse than NLI, do you have the results (and averaged) for other tasks, just like table 1? Could this suggest that DP is not good enough for minimizing the error when the number of cuts increases?\n\n- In table 8, NLI (with fp16 arithmetic from stage 1-4) is even slightly better than FP32 for 7 out of 8 models, why is this so? And again what’s the performance of using FP16?\n\n- Do you plan to make the code public for reproduciblity?\n\n- Line 339, the reference to the algorithm broke, and in page 12, the spacing between references are so large, can you fix these?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "This paper addresses a critical problem of the expensive nonlinearities in LLMs for faster inference, and provides a well-defined algorithmic contribution: the dynamic-programming formulation for optimal non-uniform linear interpolation is mathematically clear and hardware-aware. It’s defined in a two-stage manner for better efficiency and yields a calibration-free lookup table. \nIt provides a clear (performance and efficiency) evaluations of the proposed method, including error analysis, influence to the downstream tasks and perplexity across multiple open models, and the promising results demonstrate great potential of NLI.\nIt also includes efficiency analysis, proposed hardware implementation and evaluation, bridging algorithmic/design work with real system implications. The method is presented as plug-and-play manner across a variety of nonlinear functions.", "weaknesses": "NLI selects cutpoints from the FP16 domain and performs interpolation using FP16 arithmetic (Stages 1–4). A direct comparison with naïve FP16 evaluation of the same nonlinear operations should be provided to contextualize NLI’s advantages. \n\n- This comparison should include for example: i) error-wise: NLI reports an error of $1.2\\sim1.5\\times10^{-3}$; what is the corresponding error for plain FP16 evaluation? ii) performance-wise: How do models perform across tasks when nonlinear operations are evaluated directly in FP16? iii) efficiency-wise: How does the computational efficiency of NLI (lookup + FP16 linear arithmetic) compare with that of standard FP16 implementations? Additionally, NLI could also select cutpoints from the FP32 domain and use FP32 arithmetic for Stages 1–4, will there be any efficiency concern (compared to naive FP32 and current fp16 implementations)?\n\nSuch analyses would substantiate the benefits of NLI over standard FP16 inference.\n\nThe two-level cut-point design is promising, but the paper would benefit from ablation studies and discussion on its configuration, such as how many non-uniform cutpoints are chosen at the first level, how many uniform ones at the second level, and what the optimal total number of cut points is. \n\nAddressing these concerns could justify a higher score.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "7u44k0yPqA", "reviewer": "Reviewer_TEbk", "summary": "This paper addresses the high computational overhead introduced by nonlinear activation functions when deploying large language models on edge devices. It proposes a calibration-free, hardware-friendly non-uniform linear interpolation method called NLI, which employs dynamic programming for optimal segmentation. The NLI framework consists of two components: algorithm design and hardware design. \nAlgorithmically, the authors propose NLI-Algorithm, which leverages dynamic programming to select optimal segmentation points on the FP16 numerical grid. This constructs a lookup table (LUT) that is reusable across different layers and models without requiring data calibration. \nOn the hardware side, the authors design the NLI-Engine module, which implements a two-level address translation scheme and pipelined architecture. This design significantly reduces the number of comparators and overall hardware overhead. \nSoftware experiments demonstrate that this architecture achieves nearly negligible accuracy loss across various LLMs and vision models, while showing substantial improvement over NN-LUT.", "questions": "1. In the paper, minimizing the average relative error was chosen as the objective function for dynamic programming. Did the authors experiment with other objective functions (e.g., maximum absolute error, root mean square error)? Was the choice of relative error primarily to address numerical stability issues near small activation values, or did it empirically lead to better downstream task accuracy?\n2. Although extreme activation values account for a small proportion, they may be critical in certain specific tasks or prompts. Could the authors provide a more in-depth analysis, such as showing which layers or attention heads these truncated activation values are primarily distributed in? Has there been any consideration of adopting a conservative, high-precision fallback computation mode for these \"tail\" values?\n3. The dynamic programming is performed on an FP16 numerical grid, which implicitly defines an input value range. If a new activation function emerges in the future, whose effective dynamic range significantly exceeds that of FP16 (e.g., requiring FP12 or BF16 precision for representation), would the current method still be applicable? Would it be necessary to rerun the DP?\n4. How do the authors guide designers in selecting the optimal configuration for the number of \"macro intervals\" (M) and the number of \"micro cells\" (D_n) per interval? Does the architecture support dynamic reconfiguration to accommodate different functions?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. Novelty: The use of dynamic programming for non-uniform segmentation point sampling represents a significant innovation, surpassing heuristic or data-dependent calibration methods. Experimental results confirm that this approach incurs almost no accuracy loss.\n2. Full-Stack Hardware-Software Co-Design: The NLI algorithm was designed with hardware efficiency in mind from the outset, making it highly practical for edge deployment. The two-level address translation scheme is a particularly clever hardware optimization that drastically reduces the number of comparators, directly addressing a major source of area and power consumption. This integrated approach forms a compelling demonstration of the method's effectiveness.", "weaknesses": "1.\tComputational Complexity of DP Search: The O(MN²) complexity of the DP search implies that it may become a bottleneck when attempting to directly search for a larger number of finer-grained segments over the full FP16 grid. The scalability of the DP method for more granular optimizations remains a potential limitation.\n2.\tHandling Outliers via Clamping: Although the authors argue through coverage analysis that the impact of clamping is negligible, its effect on worst-case performance or robustness in certain unexamined models or tasks could be a point of concern. This may necessitate the design of more robust fault-tolerant mechanisms.\n3.\tDependence on Function Characteristics: The effectiveness of NLI is inherently tied to the behavior of the function's second derivative. Consequently, its robustness against potential activation functions that may emerge in future models remains somewhat uncertain.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "7RTek1Jlcz", "reviewer": "Reviewer_KggQ", "summary": "This paper proposes NLI, a software–hardware co-design for efficient approximation and computation of nonlinear operations in LLM inference. Specifically, the NLI framework performs non-uniform interpolation of nonlinear functions, formulating the cutpoint selection as a DP problem. In addition, NLI introduces a dedicated hardware engine for it that can be seamlessly integrated into existing LLM inference hardware.", "questions": "Do we really need to optimize nonlinear operations in LLMs? This is my primary and most critical question regarding this paper. I would like to see both quantitative and qualitative discussions on the actual performance and hardware overheads associated with nonlinear operations, to better justify the necessity of this research direction.", "limitations": "", "rating": 2, "confidence": 4, "soundness": 1, "presentation": 4, "contribution": 3, "strengths": "* The analysis of the limitations of existing approaches (e.g., NN-LUT) for LLM inference is insightful and interesting.\n* The proposed solution, encompassing both algorithmic and hardware aspects, appears technically sound, and the evaluations are thorough and convincing.", "weaknesses": "The motivation of this work is not sufficiently persuasive, which makes the overall contribution difficult to fully appreciate. While the authors emphasize the computational complexity of nonlinear operations in LLMs, I believe their actual overhead constitutes only a small fraction of the end-to-end inference time, since linear operations remain dominant even after various optimizations or compression techniques. Moreover, it is questionable whether the hardware cost of special function units in existing GPUs or NPUs is significant enough to justify the need for this entire line of research.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)