Zeros can be Informative: Masked Binary U-Net for Image Segmentation on Tensor Cores
Authors: Chunshu Wu
Ruibing Song
Sushant Kondguli
Tong Geng
Ang Li
Venue: ICLR 2026 Poster (2026)
Keywords: U-Net
segmentation
binary neural network
GPU Tensor Core
Primary area: applications to computer vision, audio, language, and other modalities
TL;DR: This paper introduces MBU‑Net, a cost‑aware masked‑binary U‑Net mapped to GPU Tensor Cores that delivers near full‑precision accuracy at near‑binary efficiency for segmentation on edge.

Abstract: Real-time image segmentation is a key enabler for AR/VR, robotics, drones, and autonomous systems, where tight accuracy, latency, and energy budgets must be met on resource‑constrained edge devices. While U‑Net offers a favorable balance of accuracy and efficiency compared to large transformer‑based models, achieving real‑time performance on high‑resolution input remains challenging due to compute, memory, and power limits. Extreme quantization, particularly binary networks, is appealing for its hardware‑friendly operations. However, two obstacles limit practicality: (1) severe accuracy degradation, and (2) a lack of end‑to‑end implementations that deliver efficiency on general‑purpose GPUs.

We make two empirical observations. (1) An explicit zero state is essential: training with zero masking to binary U‑Net weights yields noticeable sparsity. (2) Quantization sensitivity is relatively uniform across layers. Motivated by these findings, we introduce Masked Binary U‑Net (MBU‑Net), obtained through a cost‑aware masking strategy that prioritizes masking where it yields the highest accuracy‑per‑cost, reconciling accuracy with near‑binary efficiency.

To realize these gains in practice, we develop a GPU execution framework that maps MBU‑Net to Tensor Cores via a subtractive bit‑encoding scheme, efficiently implementing masked binary weights with binary activations. This design leverages native binary Tensor Core BMMA instructions, enabling high throughput and energy savings on widely available GPUs. Across 3 segmentation benchmarks, MBU‑Net attains near full‑precision accuracy (3\% average drop) while delivering 2.04$\times$ speedup and 3.54$\times$ energy reductions over a 16-bit floating point U‑Net. The code is available at https://github.com/ChunshuWu/MBU-Net.

Published as a conference paper at ICLR 2026
ZEROS CAN BE INFORMATIVE: MASKED BINARY U- NET FOR IMAGE SEGMENTATION ON TENSOR CORES
Chunshu Wu', Ruibing Song", Sushant Kondguli", Tong Geng", Li Pacific Northwest National Lahoratory', Rice University?. Meta², University of Washington
ABSTRACT
Real-time image segmentation a key enabler for AR/VR, robotics, drones, and autonomous systems, where tight accuracy, latency, and energy budgets must be met on resource-comee constrained edge devices. While U-Net offers a favorable balance of accuracy and efficiency compared to large transformer- based models, achieving real-time performance on high-resolution input remains challenging due to com- pute, memory, and power limits. Extreme quantization, particularly binary works, is appealing for its bardware-friendll operations. However, two obstacles limit practicality: (I) severe accuracy degradation, and (2) a lack of end-to-end implementations that deliver efficiency on purppsse GPUs.
We make two empirical observations. (1) An explicit zero state is essential: train- ing with zero masking to binary U-Net weights yields noticeable sparsity. (2) Quantization sensitivity is relatively uniform across layers. Motivated by these findings, we introduce Masked Binary U-Net (MBU-Net), obtained through a cost-aware masking strategy that prioritizes masking where it yields the highest accuracy-per-coa reconciling accuracy with near- near-binary efficiency.
To realize these gains in practice, we develop a GPU execution framework that maps MBU-Net Tensor Cores via subtractive bit-encoding scheme, efficiently implementing masked binary weights with binary activations. This design lever- ages native binary Tensor Core BMMA instructions, enabling high throughput and energy savings on widely available GPUs. Across 3 segmentation benchmarks, MBU-Net attains near full-precision accuracy (3% average drop) while delivering 2.04x speedup and 3.54x energy reductions over a 16-bit floating point U-Net. The code is available at http://guuubbcco//CC//uuuuuuuuu
INTRODUCTION
Image segmentation is essential for applications such as AR/VR, autonomous drones, autopilot sys- tems, and robotics. In these domains, real-time perception and sustained operation are typically required. For instance, segmentation at roughly 60 Hz frame rate is fundamental for Meta's AR experience. Likewise, inspection delivery drones
glasses deliver smooth, context-aware often must operate continuously for 30 minutes, sometimes even hours on a single battery. Conse- quently, in addition to accuracy, processing speed and power efficiency are also decisive for practical deployment on edge devices. With rising demand for high-resolution data and high frame rates, these constraints have become increasingly stringent.
Compared to large, pweapearaeeer architectures such as Vision Transformer (Dosovitskiy et al., 2020). U-Net (Ronneberger al., (2155) is substantially cheaper compute, memory, and energy for dense prediction, making it a more promising choice for edge scenarios. Additionally, U-Net's characteristic encoder-decoder structure with skip connections has demonstrated high effectiveness for pixel-level tasks (Chen al. 2018 Zhou et al.) 2019), preserving spatial details and facilitating accurate pixel-level predictions. Nevertheless, running a U-Net on high righ-resolution video in real time still exceed the strict power, latency, and memory limits of many edge devices further optimizations are necessary to reconcile accuracy with deployability.
Quantization (Gholami et al. 2022 Wei et al. 22024; Haghi et al. 2024) is natural path efficiency. At the extreme, binary networks (Hubara et al. (016)] Liu et al. 2020) use 1-bit weights and acti- vations, replacing Multiply-Accumulatee (MACs) with simple bitwise operations (XNOR/XOR)

---

Published as a conference paper at ICLR 2026
Low Accuracy
W
Zero Masking
Balanced + +1/-1
Apply Zero Masks
Layer
Apply Zero Masks
Layer 2
Layer 1
-1
High Accuracy
Layer 2
Layer 1
90% zeros
Apply Zero Masks
Layer 2
Finding Binary weights have high tendency
Finding 2: U-Net layers contribute
toward zero in binarized U-Net training
comparably to accuracy
Trading Efficiency Accuracy
Activation a
Weight
Full-Precision
Low Cost Rank
U-Net
MBU-Net
4th
Layer
V
2nd
Layer
Accuracy
1st
Layer
+1,
Better
3rd
Layer
a-b T1+ pape XOR bysey
Binary
Binary
MBU-Net Variations
-popeia' XOR bam))
X
Cost-Aware Masking Strategy
Tensor Core Deployment
Efficiency
Figure 1: Overview. Obtained through a cost-aware masking strategy and optimized for Tensor Core on GPUs, MBU-Net achieves full-precision level accuracy and binary-level efficiency.
followed by population count (popeount) (Rastegari et al. 2016). Since most processors handle data in chunks rather than individual bits, practical implementations group many binary values to- gether - allowing single instruction to execute many binary operations in parallel. This approach is highly efficient on CPUs and GPU CUDA cores (Li et al. 2019 Chen et al.) (023). and can be further extended to exploit Tensor Cores (i & Su 2020). Collectively, these techniques leverage inexpensive, hardware- fardeale-friendly operations to sharply reduce compute, memory traffic, and energy, making extremely low-bit inference attractive for resource-constrained edge deployments.
However, two practical challenges hinder the effectiveness of extreme quantization. (1) Accuracy degradation. Binary networks usually suffer from a significant drop in accuracy due to the ag- gressive 1-bit quantization, and the impact on U-Net in segmentation tasks can be especially se-
vere (AskariHemmat et al. 2019; Guo al.! 2022). (2) Scarce practical solutions on general- purpose processors. Many binary/ternary methods (Wan et al., 2018: Heinrich et al.) 2018 Zhu al. 2022) Yin al. (2024) are algorithmic proofs-of-concept or tailored custom FPGA (Liang et al. 2018; Geng et al. 2019, Mani et al. 2022) and ASIC (Wagle et al. 2020) datapaths, leaving end-to-end U-Net designs and optimizations on general-purpose processors (e-g., GPUs) under- explored. To address these challenges, this paper aims to answer: (1) How to achieve near-full- precision accuracy with near-binary efficiency for U-Net on segmentation tasks? (2) How to deliver that efficiency on widely available GPUs without relying on specialized accelerators?
To address the first challenge, we design our solution based on two key observations: (1) An explicit zero state is essential. Pure binary representations (f-1, +1}) force every connection contribute, offering no neutral state to suppress uncertain or noisy signals. Our experiments demonstrate that, when training with masking U-Net weights, in addition substantially enhanced accuracy, there is noticeable sparsity over 90% zeros in many layers, some consistently exceeding 95% far outnumbering the +1 and 1 states. This indicates an abundance of signals that should be sup- pressed by zeros. (2) Uniform sensitivity across layers. An exhaustive sweep over 4,000 per-layer quantization configurations shows that masking any single layer has comparable effect on accu- racy. Since masking adds overhead, and masking all layers is often unnecessary if subset already provides sufficient suppression of information flow, low-cost layers can be prioritized for masking to balance computation and accuracy. For example, masking transposed convolution layers has strong impact on accuracy while incurring negligible computational cost.
Since extra costs are incurred due to masking, we can strategically pick which layer to enable mask- ing for computing and accuracy trade-off, as it may not be necessary to apply masking for all layers, some of them can already ensure sufficient suppression for information pass. For instance, trans- posed convolution layers have strong impact on accuracy, despite their negligible computing cost. Guided by these findings, we propose a cost-aware masking strategy that identifies critical layers where masking is most beneficial, and introduce Masked Binary U-Net (MBU-Net), which achieves near full-precision accuracy with near-binary efficiency highlighted in Figure] 1
2

---

Published as a conference paper at ICLR 2026
To address the second challenge, target Tensor Cores, mapping MBU-Net via bit-packing and dedicated APIs. On GPUs with Tensor Cores, performance typically memory- bound rather than compute-bound, thus we retain binary activations to reduce data movement and power. This ap- proach is timely: Tensor Cores are now common even on edge GPUs (e.g.. NVIDIA Jetson Xavier with Volta Tensor Cores, Jetson Orin with Ampere Tensor Cores), positioning them practical platforms for efficient edge application deployment. Although prior frameworks have accelerated pure Binary Neural Networks (BNNs) on GPÜs (Li al. 2019 Li & Sul 2020j, the best of our knowledge, no peeeh-eeffrmacce framework exists for selectively masked binary models on Tensor Cores, especially for U-Net. To bridge this gap, we introduce a subtractive bit- bit-encoding method that naturally realizes masked binary weights with binary activations within the Tensor Core paradigm, thereby unlocking substantial efficiency gains on modern commodity GPUs. Our major contri- butions are listed below.
1. We discover that: (1) An explicit zero masking state in the weights of a binary U-Net can significantly improve the segmentation quality, closely approaching full-precision results. (2) Despite being cheap, transposed convolution layers are accuracy-critice
2. We employ a cost-aware masking strategy based on above discoveries, systematically iden- tifying the most critical layers in U-Net, leading to MBU-Net models that regain near full full-precision segmentation accuracy at minor masking cost over pure BNN.
3. We design and implement an end-to-end acceleration framework with native BMMA binary Tensor Core instructions through subtractive bit-encoding method, enabling unified and high-throughput inference for MBU-Net models on GPU Tensor Cores.
4. Our approach delivers an average 2.04 speedup and 3.54 power efficiency improvement over 16-bit floating point U-Net on A100, H100, Jetson Orin Nano, and RTX 2080 Ti GPUs, while incurring only 3% average accuracy loss on 3 segmentation datasets. The U-Net contains 16M parameters, and the input images range from 0.25M to 0.6M pixels.
BACKGROUNDS
2.1 U-NET
Since its introduction in 2015 (Ronneberger et al. 2015), the U-Net architecture has proven to be
far more than just a successful model for biomedical image segmentation (Ibtehaz & Rahman 2020
Cheng et al. 2022). It has been extended other domains including image enhancement (Komatsu
& Gonsalver ves 2020), generative AI (Ho et al.) 2020 Schonfeld et al. 2020 Si et al. 2024). scientific
modeling (Kamali & Laksari 2024 Zhu al. (025), etc.
High Level Block Abstraction of U-Net
Output Block
Block Internal Components
A cx
Input Image
Input Block
Connectiinn
Up Block44
Model Output
Input
Block
DoubleConw
consirenspose
cx
A 2ex xH/2
CARVAN
Down Block
Siip Conmectiin
Up Block
Down Block
Lencateration
2ex W/2x
A 4 1114
Pooling
DoubleCony
Down Block?
Sep Carnectiin
Up Block
DoubleConw
4e WexWAxHH
A Scx W//--//
Output Block
Down Block3
Sep Connection
Up Block A 166x 165xw/166x11
Seip Connection
Txl Conv
+>
Feoture Poths
8c x W///8
Down Block-4
Batchnorm ReK are included DaubleCanv layers
1x1 Corri Conv layer with 1xt kemels
Figure 2: Left: Conventional U-Net architecture exemplified with segmentation task. number of channels, W/ H: feature width height. Right: internal components of each abstracted block.
At its core, as Figure illustrates, U-Net is symmetric encoder-decoder architecture linked by skip connections. The encoder (contracting path with orange down blocks) follows the typical structure of a convolutional network, downsampling the input to capture semantic context. The decoder (expanding path with blue up blocks), in return, progressively upsamples and refines the feature representations to construct a full-resolution image. To avoid forcing all information through a narrow bottleneck, skip connections concatenate encoder features with the corresponding decoder features after transposed convolution layers, passing information along multiple paths and elegantly preserving information at different semantic hierarchies.
3

---

Published as a conference paper at ICLR 2026
This foundational design has inspired wide range of successors, For instance, U-Net++ (Zhou et al. 2019) introduces nested and dense skip connections, while UNet3+ (Huang et al. 2020) ex-
tends this with full-scale skip connections. Other variants integrate mechanisms such as attention gates (Oktay et al., 2018) or residual connections (Diakogiannis et al. 2020 Alom et al. 2019) to
enhance capacity. More recently, Yransformer-based approaches (Dosovitskiy et al., 2020 Strudel ict al. (2021) and Segment Anything Models (SAM) (Kirillov et al., 2023 Ravi et al. 2024) have achieved exceptional segmentation quality. However, the massive parameter counts and computa- tional demands of these large models pose significant challenges for edge deployment.
In fact, even for the standard U-Net, the computational footprint real-time, high-resolution settings is substantial. A configuration at 720p can require on the order of hundreds of GFLOPs per frame, and sustaining 60 fps implies tens of TFLOPs per second plus heavy memory bandwidth from skip connections. On edge platforms such as AR glasses and drones, on-device inference is often neces- sary for latency, privacy, and reliability, yet power budgets are only a few watts. These constraints motivate aggressive quantization and sparsity to reduce arithmetic and bandwidth while preserving accuracy. We therefore investigate U-Net in the binarized regime, aiming to characterize its behavior under extreme precision reduction and masking, and provide guidance for edge-oriented designs that balance accuracy, compute, and energy.
In addition to segmentation, U-Net has been widely explored for image enhancement tasks, includ- ing denoising (Fan et al. 2022; Tripathi 2021) and super-resolution (Hu et al. 2019). With the rapid growth of AR/VR, U-Net has emerged promising candidate for waveguide correction (Chapiro et al.
2024) to enable lower power consumption and reduced distortions in AR systems. Moreover, U-Net plays pivotal role in generative AI, serving the backbone for models such as DDPM (Ho
et al.
2020) and high-resolution image synthesis (Rombach el al. 2022).
2.2
BINARY NEURAL NETWORKS
The success of Deep Neural Networks (DNNs) has been largely predicated on trend of escalat- ing model complexity. In contrast, BNNs (Hubara et al. 2016) reduce the complexity to an ex- treme extent, with both activations and weights constrained to binary values, typically represented as {-1,+ +1}. For activations, this constraint is commonly enforced through binarization function Jr6 sign(x) during forward pass, whereas the weights are trained under binary representation, typically using Straight-Throwe Through Estimators (STE) (Bengio et al.) 2013 Courbariaux et al.) 20155 Yin et al. 2019) to approximate the gradient in such nin-differentiable architecture. Despite the low resolution, theoretical analysis (Anderson & Berg 2017) has confirmed BNN's capability of capturing features.
For efficient practical deployment, XNOR-Net (Rastegari al. 2016) was introduced. Specifically, for binary vectors ala, E {-1, +1} and blb, { their MAC operation can be translated into bit- operations and population count on bit vectors a' |a (0, 1) and b' [b, E {0, 13:
n a b Σ (2a) - 1)(26) - 1) = 2- popc(a' XNOR b') n
using the equality a' XNOR B, = -a -f b/. Here, "popc" refers to population count operation that counts the number of I's in an n-bit vector, e.g., 32-bit vector as an unsigned integer.
In the past decade, BNN research follows two main threads. On the algorithmic level, researchers strive to improve BNN training accuracy, while on the implementation level, the main target is to minimize power and latency, especially FPGA and ASIC. In this work, our goal is to push the boundaries on both directions for U-Net, achieving near full-precision accuracy with biar- efficiency.
2.3 TERNARY NEURAL NETWORKS
Table summarizes a few notable studies related to neural networks with temary activations or weights. TWN (Li et al. 2016) and TTQ (Zhu et al. 2016) focus full- full-precision activation and ternary weights on the algorithmic level, reporting results primarily on image classification tasks.
TBN (Wan al. 2018) adopts ternary activations and binary weights, also at the algorithmic level.
4

---

Published as a conference paper at ICLR 2026
Table 1: Summary of ternary activation/weight network studies.
Related Work
TWN
TTQ
TBN
TernaryNet
FATNN
Platform
Unspecified
Unspecified
Unspecified
CPU
GPU
Focus
Acc'
Acc
Acc & Theoretical Speed
Acc
Ace Speed
Activatton
Full-Precision
Full-Precision
Temary
Ternary
Ternary
Weights
Ternary
Ternary
Binary
Ternary
Ternary
Task
IC'
IC
IC
Segmentation
IC
IC: Image Classification; Ace: Accuracy
TernaryNet (Heinrich al. 2018) also an algorithmic endeavor, focusing on achieving faster in-
ference without GPUs. FAT (Chen et al. (021) introduces a 2-bit encoding method: -1 - 00; -> /10, +1 -> 11 and uses series of bit operations to reduce ternary-ternary operation com- plexity from (44N) to O(2N). TAB (Zhu et al. 2022) proposes flexible framework accom- modating various activation/weige quantization schemes. To our knowledge, none of these works analyze per-layer contributions to accuracy, assess compatibility with U-Net, deliver end-to-end implementations targeting Tensor Cores on commodity GPUs.
2.4 TENSOR CORE APIS
Tensor Cores are specialized units that accelerate Matrix MultiplyAAccumlae (MMA), computing D A B C per instruction, where A, B, C, and D are matrices of compatible dimensions. Compared with issuing many MACs CUDA cores, single MMA instruction performs large number fused operations on small matrix tile. For example, the throughput of Tensor Cores on A100 is almost order of magnitude higher than CUDA cores, emphasizing the trend of shifting Al workloads to such dedicated matrix engines for higher throughput and efficiency.
In GPUs, processing units grouped into warps, and the above MMA is executed using the WMMA API with the following three key functions: (I) Load data from memory to registers. (2) Perform the above MMA operation. (3) Write the data back to memory. Less widely known, some NVIDIA GPUs also expose bit MMA (namely, BMMA): Tensor Core instructions that compute binary matrix products using bitwise logic (e.g., XOR/AND) with popcount accumulation. These binary Tensor Core features remain low-level and experimental, as they are not exposed by main- stream libraries (e.g., cuBLAS, cuDNN) and typically require custom kernels. Consequently. despite their potential, binary Tensor Core capabilities remain underused and underexplored in practice.
In this work, we extend the scope of an existing BNN acceleration framework (Hosseini al., 2019 Li & Su 2020) on Tensor Core, bringing the unique BMMA capability under the spotlight.
METHODS
In this section, we present MBU-Net, a class of masked binary U-Nets inspired by empirical obser- vations ($3.1.1) and constructed using cost-aware masking strategy in $3.1.2] achieving balanced trade-off between accuracy and computational cost. Next, we demonstrate that with subtractive bit encoding 13.2. masked layers can be naturally mapped Tensor Cores by reusing the BNN mapping $32.22 enabling an end-to-end implementation that runs efficiently on modern edge GPUs.
3.1 MASKED BINARY U-NET
3.1.1 EMPIRICAL OBSERVATIONS AND ANALYSIS
We begin by defining a U-Net backbone with abstracted configurable layers: 4x double convolu- tion layers in the encoder, labeled as down-CI~<; 4x transposed convolution layers in the decoder, as up-CTI~44 4x double convolution layers in the decoder, as up-TI~4, The indexing is consis- tent with the block indices in Figure 2 Each layer or masked (i.e., with an explicit zero state in the weights), resulting in a 212 design space. In the exhaustive sweeping, Carvana dataset (Shaler et al. 2017) is used as a representative example.
be instantiated in one of two states: binary
In binary layers, input features, output features, and weights are all binary values within (-1, (-1,+11 +1), following the convention of BNNs. In masked layers, masking state '0" is included in weights,
5

---

Published as a conference paper at ICLR 2026
100
1,00
0.055
60-
Accuracy (Dice Scoree
0.33
c.mb.
Masi-Mdddeddd Weight
60-
Contribution
c.05-
Marginal Manginal
40-
0.83
0.015-
Lightly
Heauty
0.000
20-
6.00
0.00
Down C
Up CT
Up
6.5
8.06
0.07
D:88
0.08
ATD
8:11
6.17
0.13
Dowerc
Up-C1
0.
(a)
Configurable Layer
(b)
Computing Demand (TOPs)
(c)
Configurable Layer
Figure 3: (a) Statistics of the weights masked to zero. (b) Accuracy and computing demand distri-
bution for all layer configurations. (c) Shapley analysis: marginal improvement in Dice score from masking each binary layer. Only cases with fewer than five masked layers are included for clarity.
effectively making the weights ternary {-1, O, +1}. while keeping input and output features binary. Detailed network architecture Appendix A.2 Extensive investigations (Appendix A..3 demon- strate that higher-bit weight quantizations result only in marginal improvement compared to binary weights with masks. Through an STE-based training approach, the zero masking is automatically applied. The training configuration is in Appendix A.4
Figure Blat ala) demonstrates that zero weights are notably prevalent across all masked layers and con- figurations. On average, each layer exhibits more than 80% sparsity, with many layers exceeding 90% and some consistently surpassing 95%. This finding indicates that preferable mask out the majority of weights while retaining only small essential subset. Figure 3(b) supports this ob- servation: layer configurations with heavier masking generally achieve higher Dice scores (from to I, higher is better), confirming the value of masked binary weights.
However, masking inevitably introduces additional costs, which can sometimes be heavy in com- putation. Figure ()) further shows that fully masking all layers can require about twice as many operations as lightly masked configurations. Meanwhile, a configuration requiring 0.08 TOPs can achieve accuracy comparable with the fully masked configuration exceeding 0.12 TOPs. This indi- cates that masking every layer is unnecessary when a small set of critical layers suffices, motivating us identify those layers achieve high accuracy at low cost.
Figure (( shows the marginal gain in Dice score for each layer under zero masking. For clarity, restrict the analysis cases with fewer than masked layers, since including more layers dilutes the per-layer contribution due to the limited improvement in Dice score. Interestingly, the contribu- tions are broadly comparable across layers. This observation brings a straightforward yet practical strategy: prioritize masking low-cost layers.
3.1.2 COST-AWARE MASKING STRATEGY
The above empirical findings indicate that if some layers are significantly cheaper than others. mask- ing the cheap layers can achieve substantial improvement in accuracy at minimal cost. We thus fur- ther profile the per-layer computing and memory in Figure where the operations account for both multiplications and additions.
We observe that, the numbers of operations of transposed convolution layers (highlighted with red dashed frame) in the upsampling path are 1~~2 orders of magnitude fewer compared to other layers, and the operation count and parameter count are relatively imbalanced across layers. As usually infeasible to perform analysis like Figure 3 across the entire design space, it is preferable to set up the strategy based on the model characteristics in Figure Particularly, we define the cost-aware strategy using the weighted cost score for layer t:
L = Wop + Wparam purum
(2)
where d and A Ampmaa are the number of operations and the number parameters of layer normal- ized to J0, 1]. and the weights Wap and Wparam satisfy Wop + Wparam = 1, as hyperparameters. The weighted cost scores then ranked from low to high the priority list for masking. Following the strategy, MBU-Net can be populated based on the priority list based on demand, narrowing down the design space significantly.

---

Published as a conference paper at ICLR 2026
Input
Down-C1
Down-CI
Down
Down-C3
Down-C4
Down:C4
Up-CT1
UP
Up-CT2
Up-CT2
Up-CTa
Up-CT4
Up-CI
U1
Up-C2
Up-C2
Up-C3
U4
Up C4
Output
Output
25
5.0
7.5
10.0
12.5
15.0
17.5
(a]
Number Operations (GOPs)
(b)
Number Parameters (Million)
Figure 4: Per-layer cost profiling, (a) Number of addition and multiplication operations in each layer with a 512x512 image as a representative example. (b) Number of parameters each layer.
3.2 TENSOR CORE DEPLOYMENT
3.2.1
SUBTRACTIVE BIT- BIT-DNNG
MBU-Net maintains binary activations and assigns each layer to either binary or masked weights according to the cost-aware strategy described above. To ensure that the MAC between binary activation and masked weight can naturally supported by native operations on Tensor Cores XOR and popcount, encode masked weights with two subtracting bit-planes. In particular, let alu; E {1} denote a binary activation vector, and bb E {1,00, + +1} as a masked weight vector, encode weight value as 6 6,me neg with b,th neg E {0, 1}. The MAC operation is thus translated using a' {0, 1}:
a-b Σ 20 pos
85te de +
1 20 neg popopc(a" XOR b*g -popo(aa XOR b
(a XOR (
of XOR 6
(3)
Compared to Equation T. the binary-teraar MAC operation is fully represented by bit-friendly operations, only XOR instead of XNOR be compatible with Tensor Core intrinsics.
3.2.2 MAPPING LAYERS TO TENSOR CORE
We map these computations to GPU Tensor Cores using binary WMMA, the experimental BMMA API. Each kernel of a masked layer stores the two bit planes as bit-packed matrices. At warp level, operate on 8 x 8 x 128 bit tiles, where the three dimensions are mapped to batch size, output layer number, and input layer number. For clarity, we show the high-level procedure for convolution on Tensor Core in Algorithm 3.2.2
Algorithm Bit Convolution with Subtractive Bit-Encoding on Tensor Core for all H, W, Cou,, B do
1:
B> feature height, width, output channel, batch size
2: Initialize accumulators CTos Omeg 0
3: for all Fg, Fw. and Cin of 128-bit blocks do
>> filter height, width, input channel
4: Load binary activation fragment A
> row-major column-major binary WMMA API
5: Load binary weight fragments Bposs Bueg
6:
bmma_syne(CPss A, Boos Cpos)
7:
Hinma_sync("rees A, Brey Cheey
8:
end for
temporary result R 10: Threshold comparison: Aou t () covering batchnorm, bias, and binary activation
9:
R с C Cross
Pack bits via intrinsics (ballot, brev) and store memory
12:
end for
In the algorithm, the matrices A, B, and follow the update rule C < A B C. implemented using the Tensor Core API function "bmma.syne". The function is further compiled to Parallel Thread Execution (PTX) Tensor Core opcode that includes XOR and popcount. / denotes the pre- computed threshold. The threshold comparison jointly incorporates batch normalization (loffe &

---

Published as a conference paper at ICLR 2026
Szegedy 2015), bias, and binary activation, which is standard practice in BNNs. In terms of other layers (detailed in Figure 2) transposed convolution performed similarly, while pooling and convolutions trivial, hence not discussed in further details.
4 EXPERIMENTAL RESULTS
4.1 EXPERIMENTAL SETUP
Our experiments are conducted on 4 Nvidia GPU platforms: A100 (Ampere architecture), H100 (Hopper), Jetson Orin Nano (Ampere), and RTX 2080 Ti (Turing), all equipped with Tensor Cores. We evaluate models under the following execution settings: Pyforch-FP3322 FFtchFF33//FP baseline imple- mentations in PyTorch (Paszke al. 2019) using 32-bit floating point and 16-bit floating point for both weights and activations, where the FP16 variant employs tensor cores; MBU-Net adopts binary activation and mixed binary/masked weights; The Binary model uses both binary activations and weights. The first and the last convolution layers in the quantized experiments are full- full-precision. For benchmarks, we use 3 representative datasets: Carvana (Shaler et al.) (017) car image segmenta- tion, ISIC (Codella et al. 2019) skin lesion image segmentation, and Nuclei nuclei segmentation divergent images (Goodman et al. 2018). Dataset details are in Appendix A.4
4.2 EFFICIENCY COMPARISON
We compare separate GPU and execution settings over the selected datasets. Efficiency is evaluated and discussed in terms of latency and energy in the following sections.
4.2. LATENCY COMPARISON
Figure 5 shows the latency comparison. MBU-Net consistently outperforms FP32, with aver- age speedup of 4.83x over all platforms and datasets. On average, MBU-Net also outperforms FP16 with a 2.04x speedup. Noticeably, FP16 achieves better latency results, even compared to Binary. This is likely due to the removal of native BMMA support on Nvidia Hopper Tensor Core architecture (NVIDIA [022) performance is lost, despite MBU-Net and Binary models still being runnable. INT8 and INT4 results are also presented for A100 using TensorRT (NVIDIA) 2025). Although INT8 achieves lower latency than FP16, is outperformed by MBU-Net by 1.52x. However, INT4 shows inferior performance compared to INT8 likely due to the overhead in pack- ing/unpacking. Consequently, MBU-Net shows 2.23 x speedup versus INT4.
Latency per Image (ms)
Carvana
ISIC
Nuclei
10K
1K
1K
100
100
100
10
2080
A100
H100
Jelson
GPU Platform
GPU Platform
GPU Platform
FP32
FP16
INT8
INT4
MBU-Net
Binary
Figure 5: Latency per image inference (milliseconds). Lower is better.
4.2.2 ENERGY COMPARISON
the energy comparison. Consistently, MBU-Net's energy cost is comparable to the Binary model, while achieving average energy reduction of 8.53 x over FP32 and 3.54x over FP16. On A100, it achieves 2.09x and 2.63 improvements over INT8 and INT4, separately.
4.3 ACCURACY COMPARISON
compprrss the U-Net model across different precision configurations. "Full Precision" refers to both full-precte precision weights and activations (FP32 and FP show negligible accuracy differences), while other configurations employ binary activations. Accuracy is evaluated with three metrics: Dice Score, IoU, and FI score, with results highlighted in the shaded row. On average, aggressive quantization in MBU-Net only leads to drops of 0.029, 0.037, and 0.024 in Dice, IoU, and Fl scores.
8

---

Published as a conference paper at ICLR 2026
Energy per Image (J)
Carvana
ISIC
Nuclei
20
10
16
A100
H100
Jetson
2080 Ti
A100
H100
Jetson
2080 Ti
A100
H100
Jetson
2080 Ti
GPU Platform
GPU Platform
GPU Platform
FP18
INT4
Binary
Figure Energy per image inference (Joule). Lower is better.
Table 2: Accuracy comparison using Dice score, IoU, and Fl score acToss three datasets. All metrics range from 0 I, higher better. All configurations other than Full Precision use binary activations.
Dutaset
Carvana
ISIC
Nuclei
Metric
Dice
loU
F1
Dice
IoU
F1
Dice
loU
F1
Full Precision
0.997
0,994
0.997
0.771
0.644
0.783
0.867
0.776
0.874
INT8
0.994
0.987
0.994
0.763
0.633
0.776
0.823
0.741
0.851
INT4
0.989
0,979
0,989
0.753
0.619
0.765
0.819
0.739
0.850
MBU-Net
0,981
0.963
0.981
0.750
0.617
0.763
0.817
0.722
0.839
Binary
0.662
0.530
0.693
0.560
0.399
0.570
0.434
0.302
0,464
Both weights and activations are in full precision: negligible difference between FP33 and FP16 accuracy.
4.4
TRADE-OF OFF ANALYSIS
Figure 7
presents the Pareto frontier illustrating the trade-off between accuracy (Dice score) and speed (FPS) across different MBU-Net configurations on A100 GPU, where masked layers are cho- using the cost-aware masking strategy with Wop paramm 0.5. The corresponding cost score
ranking and additional results with other metrics are in Appendix A.5
Carvana
ISIC
Nuclel
Accuracy (Dice Score)
1.0
1.0
1.0
FP32
FP16 INT4 INT8
FP32
FP16
0.8
FP32
FP16
0.8
INT4
INT8
INT8
INT8
Binary
0.8
Binary
0.4
Binary
0.1
0.0
0.0
40
160
100
150
200
250
300
100
200
300
Speed (FPS)
Speed (FPS)
Speed (FPS)
Figure 7: Pareto frontier of accuracy (dice score) and speed (FPS) A100 for datasets. Colors
(yellow redd indicate more layers get masked, based on the cost-aware masking strategy in $3.1.2
CONCLUSION
In this work, we address the challenges of bringing binary quantization to U-Net for real-time image segmentation on edge devices. Two critical observations are identified: (1) The necessity of an explicit zero state suppress noisy signals and promote sparsity, and (2) the uniform sensitivity of U-Net layers to quantization. Building on these insights, we propose a cost-aware masking strategy that balances accuracy and efficiency, resulting in Masked Binary U-Net (MBU-Net) To realize its benefits on commodity hardware, we develop GPU execution framework that leverages Tensor Cores through a subtractive bit-encoding scheme, efficiently supporting masked binary weights with binary activations. This framework ensures high throughput and power efficiency, while remaining deployable on widely available edge GPUs.
Experiments across multiple segmentation datasets demonstrate that MBU-Net achieves near full- precision accuracy (3% drop on average), while attaining 2.04x speedup and 3.54x lower energy consumption compared to 16-bit floating point U-Net. These results establish MBU-Net as practical and scalable solution for real-time, high-resolution image segmentation on Tensor Cores. If deployed on AR/VR, autonomous drones or related domains, this can raise 30 FPS pipeline to 60 FPS, and cut segmentation power by 70%, extending battery life. While measured on GPU Tensor Cores, MBU-Net is well suited for ASICs: its low-bit width arithmetic and masking enable narrow data paths and reduced SRAM bandwidth, yielding higher performance per watt and deterministic latency in dedicated accelerator.

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGMENTS
This work is supported by the U.S. Department of Energy, Office of Science, Office of Advanced Scientific Computing Research, for the DeCoDe project at Pacific Northwest National Laboratory, in support of the MEERCAT Microelectronics Science Research Center. This work is also sup- ported by the U.S. Department of Energy. Office of Electricity, through its Advanced Grid Mod- eling (AGM) program, by DARPA under Contract W912CG25CA007, and by NSF under Award No. 2326494. This research used resources of the National Energy Research Scientific Computing Center (NERSC), a U.S. Department of Energy Office of Science User Facility at Lawrence Berke- ley National Laboratory, under Contract No. DE-ACO2-05CHI 1231 using NERSC award NERSC DDR-ERCAP0035256. The Pacific Northwest National Laboratory operated by Battelle for the U.S. Department of Energy under Contract DEE-ACO5-76RLAA 03000. Finally, would like to thank the anonymous reviewers for their feedback, and META for the collaboration.
REPRODUCIBILITY STATEMENT
All datasets employed in our experiments are publicly available and described detail in Appendix Section A.4.2 The experimental settings are presented in Section 4.1 and further elaborated in
Appendix Section A.4 The code is accessible at https//tthhhhoo//hChunn//W//Ma
REFERENCES
Md Zahanggr Alom, Chris Yakopeic, Mahmudul Hasan, Tarek M Taha, and Vijayan K Asari. Recur- rent residual u-net for medical image segmentation. Journal of medical imaging, 6(1):014006- 014006, 2019.
Alexander G Anderson and Cory P Berg. The high dighedimnnsioal geometry of binary neural networks. arXiv preprint
2017.
MosemmadHossein AskariHemmat, Sina Honari, Lucas Rouhier, Christian Perone, Julien Cohen- Adad, Yvon Savaria, and Jean-Pierre David. U-net fixed-point quantization for medical image segmentation. International Workshop on Large-scale Annotation of Biomedical data and Expert Label Synthesis, pp. 115-124. Springer, 2019.
Yoshua Bengio, Nicholas Léonard, and Aaron Courville. Estimating propagating gradients through stochastic neurons for conditional computation. arXiv preprint arXiv: Ttt.::33083333 2013.
Alexandre Chapiro, Dongyeon Kim, Yuta Asano, and Rafat K. Mantiuk. Ar-david: Augmented reality display artifact video dataset. ACM Trans. Graph., 43(6), November 2024. ISSN 0730- 0301. doi: 10. (0.11145 145/3687969, URL httht:t//://o 2457//....0.1111111113///6669
Jou-An Chen, Hsin-Hsuan Sung, Xipeng Shen, Sutanay Choudhury, and Ang Li. Bitgnn: Unleash- ing the performance potential of binary graph neural networks on gpus. Proceedings of the 37th International Conference on Supercomputing, pp. 264-276, 2023.
Liang- Chich Chen, Yukun Zhu, George Papandreou. Florian Schroff. and Hartwig Adam. Encoder- decoder with atrous separable convolution for semantic image segmentation. In Proceedings of the European conference on computer vision (ECCV), pp. 801-818, 2018.
Peng Chen, Bohan Zhuang, and Chunhua Shen. Fatnn: Fast and accurate ternary neural networks. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 5219-5228, 2021.
Junlong Cheng, Shengwei Tian, Long Yu, Shijia Liu, Chaoqing Wang, Yuan Ren, Hongchun Lu, and Min Zhu. Ddu-net: A dual dense u-structure network for medical image segmentation. Applied Soft Computing, 126:1099 109297, 2022.
Noel Codella, Veronica Rotemberg, Philipp Tschandl, Emre Celebi, Stephen Dusza, David Gut- man, Brian Helba, Aadi Kalloo. Konstantinos Liopyris, Michael Marchetti, et al. Skin lesion analysis toward melanoma detection 2018: A challenge hosted by the international skin imaging collaboration (isic). arXiv preprint A902v 2019.
10

---

Published as a conference paper at ICLR 2026
Matthieu Courbariaux, Yoshua Bengio, and networks with binary weights during propagations. Advances in neural information processing systems, 28, 2015.
David. Binaryconneet: Training deep neural
Foivos Diakogiannis, François Waldner, Peter Caccetta, and Chen Wu. Resunet-a: A deep learning framework for semantic segmentation of remotely sensed data. ISPRS Journal of Photogrammetry and Remote Sensing, 162:94-114, 2020.
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition scale. arXiv preprint arXiv:2010. 19929, 2020.
Chi-Mao Fan, Tsung-Jung Liu, and Kuan-Hsien Liu, Sunet: Swin transformer unet for image denoising. In 2022 IEEE International Symposium Circuits and Systems (ISCAS), pp. 2333- 2337. IEEE, 2022.
Tong Geng, Tianqi Wang, Chunshu Wu, Chen Yang, Shuaiwen Leon Song, Ang Li, and Martin Herbordt. Lp-bnn: Ultra-low-latens bnn inference with layer parallelism. In 2019 IEEE 30th International Conference on Application-specific Systems, Architectures and Processors (ASAP), volume 2160, pp. 9-16. IEEE, 2019.
Amir Gholami, Schoon Kim, Zhen Dong Zhewei Yao, Michael W Mahoney, and Kurt Keutzer. A survey of quantization methods for efficient neural network inference. In Low-power computer vision, pp. 291-326. Chapman and Hall/CRC, 2022.
Allen Goodman, Anne Carpenter, Elizabeth Park, jlefman nvidia, Josette Boaett_Boozzllen, Kyle, Maggie, Nilofer, Peter Sedivec, and Will Cukierski. 2018 data science bowl. fattps://kagge com/ Kaggle.
compet ioss /data sci ence- bowl 10011
2018.
Liming Guo, Wen Fei, Wenrui Dai, Chenglin Li, Junni Zou, and Hongkai Xiong. Mixed-preciste quantization of u-net for medical image segmentation. In 2022 IEEE International Symposium on Circuits and Systems (ISCAS), pp. 2871-2875. IEEE, 2022.
Pouya Haghi, Chunshu Wu, Zahra Azad, Yanfei Li, Andrew Gui, Yuchen Hao, Ang Li, and Tony Tong Geng. Bridging the gap between lims and Ins with dynamic data format and architec- ture codesign. In 2024 57th JEEE/ACM International Symposium on Microarchitecture (MICRO), pp. 1617-1631. IEEE, 2024.
Mattias P Heinrich, Max Blendowski, and Ozan Oktay. Ternarynet: faster deep model inference without gpus for medical 3d segmentation using sparse and binary convolutions. International journal of computer assisted radiology and surgery, 13(9):1311-1322 2018.
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840-6851, 2020.
Morteza Hosseini, Hirenkumar Paneliya, Utteja Kallakuri, Mohit Khatwani, and Tinoosh Mohsenin.
Minimizing classification energy of binarized neural network inference for wearable devices. In 20th International Symposium on Quality Electronic Design (ISQED), pp. 259-264. IEEE, 2019.
Xiaodan Hu, Mohamed A Naiel, Alexander Wong, Mark Lamm, and Paul Fieguth. Runet: A robust unet architecture for image super-resolution. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition Workshops, pp. 0-0, 2019.
Huimin Huang, Lanfen Lin, Ruofeng Tong, Hongjie Hu, Qiaowei Zhang, Yutaro Iwamoto, Xianhua Han, Yen-Wei Chen, and Jian Wu. Unet 3+: A full-scale connected unet for medical image segmentation. In ICASSP 2020-2020 IEEE international conference on acoustics, speech and signal processing (ICASSP), pp. 1055-1059. leee, 2020.
Itay Hubara, Matthieu Courbariaux, Daniel Soudry, Ran El-Yaniv, and Yoshua Bengio. Binarized neural networks. Advances in neural information processing systems, 29, 2016.
Nabil Ibtchaz and M Sohel Rahman. Multiresunet: Rethinking the u-net architecture for multimodal biomedical image segmentation. Neural networks, 87, 2020.

---

Published as a conference paper at ICLR 2026
Sergey loffe and Christian Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In International conference on machine learning, pp. 1148-566 pmir, 2015.
Ali Kamali and Kaveh Laksari. thysics-informed unets for discovering hidden elasticity in het- erogeneous materials. Journal of the mechanical behavior of biomedical materials, 10:106622, 2024.
Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C Berg, Wan-Yen Lo, et al. Segment anything. In Proceed- ings of the IEEE/CVF international conference on computer vision, pp- 4015 4015-4026, 2023.
Rina Komatsu and Tad Gonsalves. Comparing u-net based models for denoising color images. Al, 1(4):465 2020.
Ang Li and Simon Su. Accelerating binarized neural networks via bit-tensor-cores in turing gpus. IEEE Transactions on Parallel and Distributed Systems, 32(7):1878 1891, 2020.
Ang Li, Tong Geng, Tianqi Wang. Martin Herbordt, Shuaiwen Leon Song, and Kevin Barker. Bste: design for accelerating bit-based approximated neural nets. In Proceedings of the international conference for high performance computing, networking, storage and analysis, pp. 1-30, 2019.
A novel swwaarized-sssttttessooooooooooe
Fengfu Li, Bin Liu, Xiaoxing Wang, Bo Zhang, and Junchi Yan. Ternary weight networks. arXiv 2016.
preprint
Shuang Liang, Shouyi Yin, Leibo Liu, Wayne Luk, and Shaojun Wei. Fp-bnn: Binarized neural network on fpga. Neurocomputing, 275:1072-1086, 2018.
Zechun Liu, Zhiqiang Shen, Marios Savvides, and Kwang-Ting Cheng. Reactnet: Towards precise binary neural network with generalized activation functions. European conference on computer vision, pp. 153-159. Springer, 2020.
VRS Mani, A Saravanaselvan, and NJMJ Arumugam. Performance comparison of cnn, qnn and ban deep neural networks for real-time object detection using zynq fpga node. Microelectronics Journal, 19:11053,9, 2022.
NVIDIA. Nvidia h100 tensor core gpu architecture.
Whitepaper, 2022.
URL
https://resouu: www.oorrresn...iiii...........ee cww.enoms- wwt.://nnos--hpppa---recccttatae
nvidia- -hl.iiahh100-terso0::
NVIDIA. NVIDIA TensorRT. sion 10.x. Accessed: 2025-12-02.
2025. Ver-
Ozan Oktay, Jo Schlemper. Loic Le Folgoc, Matthew Lee, Mattias Heinrich, Kazunari Misawa, Kensaku Mori, Steven McDonagh, Nils Y Hammerla, Bernhard Kainz, et al. Attention u-net: Learning where to look for the pancreas. arXiv preprint arXiv: 800.03399, 2018.
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, high- performance deep learning library. Advances in neural information processing systems, 32, 2019.
Mohammad Rastegari, Vicente Ordonez, Joseph Redmon, and Ali Farhadi. Xnor-net: Imagenet classification using binary convolutional neural networks. In European conference on computer vision, pp. 525-542. Springer, 2016.
Nikhila Ravi, Valentin Gabeur, Yuan-Ting Hu, Ronghang Hu, Chaitanya Ryali, Tengyu Ma, Haitham Khedr, Roman Rädle, Chloe Rolland, Laura Gustafson, al. Sam 2: Segment anything in images and videos. arXiv preprint arXiv:2408.00711. 2024.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. High- resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF confer- ence on computer vision and pattern recognition, pp. 10684-10695, 2022.
12

---

Published as a conference paper at ICLR 2026
Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedi- cal image segmentation. In International Conference on Medical image computing and computer- assisted intervention, pp. 234-234-20 231. Springer. 2015.
Edgar Schonfeld, Bernt Schiele, and Anna Khoreva. A u-net based discriminator for generative adversarial networks. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 8207-8216, 2020.
Brian Shaler, Dan Gill, Maggie McDonald, Patricia, and Will Cukierski.
Car-
vana image masking challenge. carvana- image maakingg-halleemee challenge.
https:
2017. Kaggle.
Chenyang Si, Ziqi Huang, Yuming Jiang, and Ziwei Liu. Freeu: Free lunch in diffusion u-net. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp- 4733-4743, 2024.
Robin Strudel, Ricardo Garcia, Ivan Laptev, and Cordelia Schmid. Segmenter: Transformer for semantic segmentation. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 7262-7272, 2021.
Milan Tripathi. Facial image denoising using autoencoder and unet. Heritage and Sustainable Development, 3(2):89, 2021.
Ankit Wagle, Sunil Khatri, and Sarma Vrudhula. A configurable ban asic using a network of pro- grammable threshold logic standard cells. In 2020 IEEE 38th International Conference Com- puter Design (ICCD), pp. 433-440, IEEE, 2020.
Diwen Wan, Fumin Shen, Li Liu, Fan Zhu, Jie Qin, Ling Shao, and Heng Tao Shen. Tbn: Convo- lutional neural network with ternary inputs and binary weights. Proceedings of the European Conference on Computer Vision (ECCV), pp. 315-332, 2018.
Lu Wei, Zhong Ma, Chaojie Yang, and Qin Yao. Advances in the neural network quantization: A comprehensive review. Applied Sciences, 14(17):7445, 2024.
Penghang Yin, Jiancheng Lyu, Shuai Zhang, Stanley Osher, Yingyong Qi, and Jack Xin. Under- standing straight-through estimator in training activation quantized neural nets. arXiv preprint arXiv:1903.05665 2019.
Rui Yin, Haotong Qin, Yulun Zhang, Wenbo Li, Yong Guo, Jianjun Zhu, Cheng Wang, and Biao Jia. Bidense: Binarization for dense prediction. arXiv preprint arXiv:2411.10346, 2024.
Zongwei Zhou, Md Mahfuzur Rahman Siddiquee, Nima Tajbakhsh, and Jianming Liang. Unet++: Redesigning skip connections exploit multiscale features in image segmentation. IEEE trans- actions on medical imaging, 39(6): 1851-18677 2019.
Chenzhuo Zhu, Song Han, Huizi Mao, and William J Dally. Trained ternary quantization. arXiv preprint arXiv16612.01064 2016.
Shien Zhu, Luan HK Duong, and Weichen Liu. Tab: Unified and optimized ternary, binary, and mixed-precision neural network inference on the edge. ACM Transactions on Embedded Com- puting Systems (TECS), 2165):1-26, 2022.
Tong Zhu, Dehao Liu, and Yanglong Lu. volume physics- informed u-net for flow field recon- struction with sparse data. Journal of Computing and Information Science in Engineering, 25(7): 07 071004, 2025.
13

---

Published as a conference paper at ICLR 2026
A APPENDIX
A.1 LLM USAGE
We use GPT-5, Gemini-2.5-Pree and Claude-4-sonnet to polish writing and assist literature search- ing. All LLM-assisted text and references have been reviewed, revised, and verified by the authors.
A.2 DETAILED MBU-NET ARCHITECTURE
Detailed U-Net Architecture in This Work
Figure A.I: The foundational U-Net architecture used in this work.
Figure A.2: The quantized U-Net architecture used in this work.
Downn-Cf
Down-3
Down-C
Down-C4
Meally
Frequciyy
07P5 E.00 0155 Ou5S
Score
1303 E300
AYH
080
XXX
(00 630 800 0352
Score
ETS
8.00
030
1005
8.06
Score
Up-C1
Up-C2
Up-C3
Up-C4
Rereency
0272
6770
036 2002 Dice Score
6.70
030 2300 Score
3.00
0800 6.2 0300
Dice Score
Figure A.3: Per-layer quantization space exploration. First row: Dice score distribution of 256 ex- periments down-sampling convolution layers with binary remaining layers. Second row: distribu- tion of 256 experiments on up-sampling convolution layers with binary remaining layers. Each sub- figure explores the accuracy improvement of increasing the number bits for this layer's weights.
14

---

Published as a conference paper at ICLR 2026
Figure A.1 and Figure A.2 illustrate the model architectures used in this work, with the former used to evaluate the full precision configuration, and the other for quantized experiments. Com- pared the foundational U-Net architecture, the quantized architecture not only modifies the ReLU activation function and the convoluiion/ranspoeee convolution into binary versions, but also adds batchnorm and binary activation after transposed convolution layers to preserve the binary activation and ensure training quality.
A.3 QUANTIZATION SPACE EXPLORATION
Figure A.3 shows the statistics of using higher bit quantization for individual convolution layers using Carvana dataset. The two input convolution layers, and the final convolution layer are full- precision (slightly different from the experiments in the main text, where the 2nd convolution layer is always masked-binary) In the first row, the four down-sampling layers are traversed from all binary to 8-bit, totaling 4 256 experiments, while the remaining layers are binary. Similarly, the second row traverses all double convolution layers in up-sampling with another 256 experiments. The distribution demonstrates that for most of the layers, the transition from binary to masked-binary contributes the most to accuracy, whereas negligible benefit is obtained from 2-bit to 4-bit, or from 4-bit to 8-bit. We thus confine the paper's 's scope within binary and 2-bit.
A.4 TRAINING AND INFERENCE CONFIGURATIONS
A.4.1
HYPERPARAMETERS
Training and inference parameters are shown in Tables A.1 and A.2 list the hyperparameters. Full- precision training the RMSprop optimizer, while quantized training uses Adam.
Table A.I: The training hyperparameters for separate experiments.
Batch Size
Learning Rate Number Epochs Grad Clipping
Weight Decay
Momentum
le-5
30
1.0
1c-8
0.9
Table A.2: The batch sizes used for inference experiments. For fair comparison, the batch sizes for FP32 and FPI6 are maximized before the GPUs run of memory.
Datasct
Carvana
ISIC
Nuclci
Configuration
FP32
FP16
MBU-Net
FP32
FP16
MBU-Net
FP32
FP16
MBU-Net Net
A100
32
32
64
64
128
128
64
128
128
H100
64
64
64
128
256
256
128
256
256
Jetson
8
32
16
2080 Ti
16
64
16
32
128
16
32
128
A.4.2 DATASET DETAILS
Table A.3 shows the dimensions of the three selected datasets. The data are scaled from the original datasets the numbers listed in the table. For Nuclei dataset, the original individual nucleus masks are combined into comprehensive mask to facilitate the image-mast mask pair for segmentation.
Table A.3: Dataset specifications.
Dataset
Number of Images
Width
Height
Carvana
5088
959
640
ISIC
3693
640
480
Nuclei
669
512
512
A.5 DETAILED ABLATION STUDIES
A.5.1 COST RANKING
The cost scores and rankings for individual layers of the U-Net used in this work listed in Table A.4 In our ablation studies, the layers are gradually masked starting from Rank to evaluate the trade-off between efficiency and accuracy.
15

---

Published as a conference paper at ICLR 2026
Table A.4: Cost scores and rankings of layers.
Rank
Layer
Cost Score
Rank
Layer
Cost Score
Rank
Layer
Cost Score
Up-CT4
0.011
Up-C3
0.228
9
Up-C4
0.512
2
Up-CT3
0.016
6
Down-C1
0.273
10
Up-C1
0.521
3
Up-CT2
0.037
Up-C2
0.286
11
Down-C4
0.583
Up-CTI
0.120
8
Down-C2
0.406
12
Down-C3
0.625
A.5.2
RESULTS OF ADDITIONAL. METRICS
Figure A.4 shows the Pareto frontiers of speed in FPS and accuracy in IoU and F1 score, A100 GPU platform. The results are consistent with Figure 7 in the main text. Figure |A.5 demonstrates the Pareto frontiers other GPU platforms (H100, RTX 2080 Ti, and Jetson Orin Nano). On RTX 2080 Ti and Jetson Orin Nano, MBU-Net variations clearly outperforms FP32 and FP16 in terms of speed, while the accuracy drop is low when sufficient masked layers are deployed. As discussed in the main text, on H100, the performance of MBU-Net is lost due to the lack of BMMA support.
Carvana
ISIC
Nuclei
1.0
Acquracy (loU)
0.6
0.9
FP12
FP16
018
018
0.9
0.6
FP32
FP16
0.7
0.7
FP32
FP18
0.7
0.8
0.8
04
Binary
0.4
0.4
0.3
03
Binary
0.3
0.7
022
0.0
20
40
60
50
100
120
140
160
120
150
200
250
502
90
100
150
200
250
500
350
Speed (FPS)
Speed (FPS)
Speed (FPS)
Carvana
Nuclei
Accurny (F1 Score)
0.
1.0
FP12
FP16
1.0
0.8
018
FP32
FP16
0.9
0.8
1.0
FP32
FP16
0.7
0.8
Binary
0.5
0.5
Binary
0.4
0.4
0.4
Dinary
0.3
033
0.3
0.
011
0.0
0.0
100
120
150
80
100
150
200
250
300
350
Speed (FPS)
Speed (FPS)
Speed (FPS)
Figure A.4: Speed vs. accuracy (loU and Fl score) on A100.
Carvana
ISIC
Nuclol
0.8
0.9
1.0
1.0
FP32
FP10
1.0
FP32
FP16
0.7
077
0.6
Binery
0.6
Sintry
-.4
Brirry
0.3
0.2
0.1
0.0
00
100
100
200
250
0.0
100
200 Speed (FPS)
400
0.0
100
200
H100
Speed (FPS)
Speed (FPS)
0.8
0.9
1.0
7P32
FP12
1.0
0.8
0.8
FP32
FP16
0.8
0.0
1.0
SP32
FP16
(gic
67
066
Binary
0.7
0.8
077
7.6
0.6
0.5
Birety
0.5
0.4
0.4
044
0.3
0.5
0.5
0.2
0.2
0.2
0.1
0.1
0.0
0.0
80
100
150
200
200
0.0
00
100
150
200
200
2080 TI
Speed (FPS)
Speed (FPS)
Speed (FPS)
1.0
08
FP12 FP16
0.8
FP52 FP1
07
Binery
Binary
0.3
03
0.2
02
0.0
0.0
Jetson
Spoed (FPS)
Speed (FPS)
Speed (FPS)
Figure A.5: Speed vs. accuracy (Dice score) on H100, RTX 2080 Ti, and Jetson Orin Nano.
16


Reviews:
[{"review_id": "6qDg48Ka2z", "reviewer": "Reviewer_P1M6", "summary": "In this paper, the author argues that traditional binary U-Nets lose too much accuracy and lack efficient GPU implementations. To address this, the paper proposes MBU-Net, which introduces a zero state to make weights ternary and applies it selectively through a cost-aware masking strategy. It also builds a custom GPU framework using subtractive bit encoding so ternary weights can run efficiently on Tensor Cores, achieving near full-precision accuracy with much faster and more energy-efficient inference.", "questions": "Please refer to the weakness part", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "* This paper is well organized and easy to follow, especially for readers who are not familiar with this area.\n* The insights that introducing large amount of ‘zero state’ into pure binary U-Nets is extremely helpful on segmentation task is great\n* A major strength of this work is its practical GPU execution framework, which provides tangible, measurable speedup on widely available NVIDIA GPUs\n* Innovatively unlocks Hardware Potential with “Subtractive Bit-Encoding”, extending the BMMA (Binary matrix multiply) to the ternary computation.", "weaknesses": "* Introducing ‘zero state’ into pure binary U-Net can significantly boost performance on segmentation, the performance of such a method applied on other types of networks and tasks remains unclear. However, the paper’s innovation on the low-level hardware implementation is still very solid.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "RLmY1yrWM6", "reviewer": "Reviewer_UTpC", "summary": "The author’s propose a real-time image segmentation approach using UNet which they name Masked Binary UNet. The approach includes an explicit zero-state (which was found to be essential for performance), and layers are quantized uniformly across layers along with an execution framework that maps the model to TensorCores. They show that this approach performs much faster than the full precision variants while maintaining most of its original performance.", "questions": "* Q1 - Why is Section 4.4 considered an ablation study? No parts of the system were ablated? Could’ve tried random costs on the layers as opposed to the weighting scheme as well. \n* My main concern is W1, followed by W2 and W3. Namely, why didn't the authors show INT8 and INT4 in the pareto curve?", "limitations": "", "rating": 2, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "S1 - Interesting finding that including zero-mask values seems to preserve the performance of the UNet model. Moreover, an interesting tensor-core deployment scheme was shown. \n\nS2 - The proposed system performs roughly just as well as the full-precision UNet’s while being much faster.", "weaknesses": "W1 - While INT8 and INT4 models are evaluated for performance (in 4.3), their latency / speed / energy is not shown. Could it be that INT8 and INT4 perform just as well as the proposed method in terms of speed?\n \nW2- Authors should’ve compared to another simple baseline, namely using TensorRT for inference optimization / quantization as in https://arxiv.org/pdf/2012.12259. \n\nW3 - The paper is lacking in experimental results. For example, Section 4.4 summarizes insights already gathered in 4.3 and 4.2.1. On the whole, the table in 4.3 could’ve just had an extra column indicating the FPS and nothing would’ve been lost if 4.2.1 and 4.4 were removed.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Pdb38d9Pdu", "reviewer": "Reviewer_MDEM", "summary": "This paper proposes a masked-binary mechanism with cost-aware layer selection that adds a \"zero state\" to binary networks, improving expressiveness and stability while preserving bit-operator efficiency. Masking selective low-cost layers achieves near \"full-mask\" accuracy with predictable hardware performance.", "questions": "Please see Weaknesses.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "1. The paper addresses real-time, high-resolution segmentation on edge devices, focusing on accuracy, latency, and energy. The empirical insights are straightforward and motivate a practical design, making the contribution both coherent and relevant.\n2. The work combines masked binary weights with a cost-aware layer selection and a GPU execution framework using Tensor Cores. The subtractive bit-encoding and native binary operations show strong engineering rigor and enable deployable efficiency gains.\n3. Experiments across multiple GPUs and datasets demonstrate near full-precision accuracy with notable speed and energy improvements. The comprehensive metrics and ablations support robustness, portability, and practical relevance.", "weaknesses": "1. The method adds ternary weights to selected layers of a binary U-Net to approach full-precision accuracy with near-binary efficiency. However, the paper does not clearly compare against ternary quantization baselines. Could the authors clarify in which dimensions MBU-Net outperforms classical ternary methods?\n2. The paper enhances binary networks by masking some weights to zero. Intuitively, This seems related to sparsity/pruning. Could the authors clarify: can this method be considered as a combination of pruning and binary quantization? If not, what are the key distinctions?\n3. While the analysis and experiments are convincing, the authors focus on the classical U-Net. I am concerned about robustness and generalizability: can the proposed method be generalized to similar segmentation architectures (e.g., U-Net variants) for broader applicability?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "sXzTGH22kl", "reviewer": "Reviewer_pWxq", "summary": "To address the computational and power limitations of U-Net for high-resolution image segmentation on edge devices, the authors propose a Masked Binary U-Net (MBU-Net) method. Through the experimental analysis of binary U-Net, the authors found that introducing zero mask training produces significant sparsity (more than 95% of the layers have zero weights), and the sensitivity of the layers to the mask is roughly uniform. Motivated by this observation, the authors design a cost-aware masking strategy with three-valued weights (-1, 0, +1) and binary activations, and develop an efficient inference framework based on GPU Tensor Core. Experimental results on three medical/image segmentation datasets show that the proposed method achieves about 2x speedup and 3.54x energy reduction compared to FP16 U-Net, but at the cost of 3% accuracy degradation on average. The authors note that the method relies on the native binary matrix multiplication instructions of the GPU, which is limited in performance on the H100 architecture with this support removed, and the accuracy loss caused by binarization may not be acceptable in some application scenarios.", "questions": "1.The authors conducted tests on A100, H100, RTX 2080Ti, and Jetson Orin Nano. However, it is unclear how the different hardware architectures of these GPUs affect the proposed method. For instance, A100 uses NVIDIA Ampere architecture, H100 uses Hopper architecture, and RTX 2080Ti uses Turing architecture. What are the implications of these architectural differences on the performance and applicability of the proposed approach?\n\n2.The authors propose a zero masking method in the manuscript, but the logic and parameter settings for applying zero masking to weights are not clearly explained. Is this an adaptive adjustment or a fixed threshold approach?\n\n3.The authors propose a subtractive bit-encoding method tailored for Tensor Cores in commercial GPUs. However, can this method be generalized into a universal and elegant hardware design framework or dataflow scheduling strategy that provides insights for hardware platforms beyond GPUs, rather than serving merely as an experimental optimization technique specific to GPUs?\n\n4.Please see the Weaknesses section for other issues.", "limitations": "", "rating": 2, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 3, "strengths": "1.In this paper, the zero masking state in the weights of a binary U-Net improves the segmentation quality.\n\n2.The authors report latency evaluations across multiple GPU platforms, demonstrating an average 2.04× speedup and 3.54× power efficiency improvement compared to a 16-bit floating-point U-Net baseline.", "weaknesses": "1.The authors only evaluated their method on the typical U-Net architecture shown in Fig. 2, without providing results on other segmentation models, which undermines the robustness and generalizability of the proposed approach to other network architectures.\n\n2.The BACKGROUND and RELATED WORK sections should be merged into a single section.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)