Hardware and Software Platform Inference
Authors: Cheng Zhang
Hanna Foerster
Robert D. Mullins
Yiren Zhao
Ilia Shumailov
Venue: ICML 2025 poster (2025)
Keywords: ML security
ML governance
Primary area: social_aspects->security
TL;DR: We introduce a new problem and solutions to identifying the software and hardware platform of an ML model solely based on its input-output behaviour. Abstract:

Abstract: It is now a common business practice to buy access to large language model (LLM) inference rather than self-host, because of significant upfront hardware infrastructure and energy costs. However, as a buyer, there is no mechanism to verify the authenticity of the advertised service including the serving hardware platform, e.g. that it is actually being served using an NVIDIA H100. Furthermore, there are reports suggesting that model providers may deliver models that differ slightly from the advertised ones, often to make them run on less expensive hardware. That way, a client pays premium for a capable model access on more expensive hardware, yet ends up being served by a (potentially less capable) cheaper model on cheaper hardware. In this paper we introduce ***hardware and software platform inference (HSPI)*** -- a method for identifying the underlying GPU architecture and software stack of a (black-box) machine learning model solely based on its input-output behavior. Our method leverages the inherent differences of various GPU architectures and compilers to distinguish between different GPU types and software stacks. By analyzing the numerical patterns in the model's outputs, we propose a classification framework capable of accurately identifying the GPU used for model inference as well as the underlying software configuration. Our findings demonstrate the feasibility of inferring GPU type from black-box models. We evaluate HSPI against models served on different real hardware and find that in a white-box setting we can distinguish between different GPUs with between 83.9% and 100% accuracy. Even in a black-box setting we are able to achieve results that are up to three times higher than random guess accuracy.

Hardware and Software Platform Inference
Cheng Zhang"
Hanna Foerster Robert D. Mullins? 2
Yiren Zhao
Ilia Shumailoy
Abstract
1. Introduction
The widespread adoption of large language models (LLMs) has transformed the technological landscape, integrating machine learning models across various sectors. However, deploying these powerful models often entails substantial upfront investments in specialized hardware infrastructure and energy, leading many businesses to opt for third-pan LLM providers. This practice raises concerns about trans- parency and accountability, as buyers currently lack the means to verify the actual hardware used to serve the models they purchase. Moreover, reports have emerged suggesting that some providers may deploy models that de- viate subtly from their advertised counterparts, potentially optimized for less expensive hardware to reduce costs'.
It is now a common business practice to buy ac- cess to large language model (LLM) inference rather than self-host, because of significant up- front hardware infrastructure and energy costs. However, a buyer, there is no mechanism to verify the authenticity of the advertised ser- vice including the serving hardware platform, e.g. that it is actually being served using an NVIDIA H100, Furthermore, there are reports suggest- ing that model providers may deliver models that differ slightly from the advertised ones, often to make them run on less expensive hardware. That way, client pays premium for capable model access on more expensive hardware, yet ends up being served by a (potentially less capa- ble) cheaper model on cheaper hardware. In this paper we introduce hardware and software plat- form inference (HSPI) - a method for identify- ing the underlying GPU architecture and software stack of (black-box) machine learning model solely based on its input-output behavior. Our method leverages the inherent differences of var- ious GPU architectures and compilers to distin- guish between different GPU types and software stacks. We evaluate HSPI against models served on different real hardware and find that in a white- box setting we can distinguish between differ- ent GPUs with between 83.9% and 100% accu- racy. Even in a black-box setting we achieve results that are up to 3x higher than random guess accuracy. Our code is available at https: / //github.. com/ 2eahnngghang--- 98/98///90 /.SPI.
DL model
Rapust
Response
plaffomm
Service provider A
User
Service provider B
HSPI
Prediction
Figure I. Overview of hardware and software platform inference (HSPI). HSPI aims to identify the underlying hardware and soft- ware platform of deep learning models. Engineered requests are sent to a service provider and responses are collected. With only the responses, HSPI predicts information on the hardware and software supply chains of the service provider.
A difference in hardware could not only introduce perfor- mance differences in terms run time and model accuracy but may also indicate other potential issucs. A malicious provider might employ poorer security measures, for exam- ple by running GPU without TEE present or deploying the GPUs at restricted geographical location that is differ- ent from the agreed one. There might also be cases of a man-in in the middle that is conning both the service provider
'Equal contribution 'Department of Computing, Impe- rial College London, London, United Kingdom 'Department of Computer Science and Technology, University of Cam- bridge. Cambridge, United Kingdom Google DeepMind, Lon- don, United Kingdom. Correspondence to: Cheng Zhang <cheng zhang 22 iimprriallcc.uk.........
Proceedings of the 42nd International Conference Machine Learning, Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).
For example, here is provider discussing strategies of reduc- ing costs in model serving including changing models appropri- ately for smaller hardware.

---

Hardware and Software Platform Inference
and the client by using the service provider's service for oneself and serving counterfeit GPU client. Moreover, malicious provider could be after the client's prompts or data, leading to privacy concerns. Therefore, being able to identify a serving hardware or software platform can serve as a useful signal for a variety of reasons.
Different hardware and software con- figurations give us various computational results. When the computational results stay the same and do not deviate between settings, we talk about them being in the same equivalence class (EQC). EQCs are used to group similar computational behaviors that yield consistent results under specific settings, such as quantization levels, GPU archi- tectures, and batch sizes. Schlögl et al. (2023) provide a more granular analysis, examining EQCs layer by layer across models, identifying how architectural choices impact computational stability and precision deviations.
Equivalence Classes:
This paper introduces hardware and software platform inference (HSPI) for machine learning, novel problem formulation for identifying the underlying GPU architecture and potentially the software stack of a (black-box) machine learning model soiely by examining its input-ouiput behav- ior. HSPI works by exploiting subtle differences in how different GPUs and software environments perform calcula- tions, which result in unique subtle patterns in the model's output. By analyzing these numerical patterns, our proposed classification framework can accurately discern the specific device employed for model inference. HSPI has significant implications for ensuring transparency and accountability. By enabling buyers to independently verify the hardware used by their providers, HSPI can help establish trust and prevent potential cost-saving measures that might compro- mise model performance.
Factors Influencing Computational Deviations: Schlögl et al. also examine several reasons for precision deviations, including arithmetic units using faster approximations, inter- mediate values being rounded fit in registers of different sizes, transformations made during execution planning, and various tricks that optimize performance in ML toolboxes such as loop unrolling, constant folding, and arithmetic simplifications (2021; 2023). Here is a simple example of different results when decimals are rounded to integer (L-)) before addition:
LL(100.4 + 0.4) + 0.5] = [101 - 0.5] = 102
(1)
To this end, we introduce two methods: HSPI with Border Inputs (HSPI-BI) and HSPI with Logits Distributions (HSPI- LD). We demonstrate the efficacy of these techniques in HSPI under both white-box and black-box setups, extending across both vision and language tasks. Overall, we make the following contributions:
[100.4 + L(0.4 + 0.5) [100.4 101.
Due to finite numerical precision and stochastic pipelining. such examples can be found at any quantization level. Quan- tization is technique for improving hardware efficiency at inference time. Additionally, Schlögl et al. (2023) find that the extent of deviations in computations is influenced by neural network architectures, e.g., layers involving high multiplication counts tend to amplify deviations.
We define Hardware and Software Platform Inference (HSPI), detailing the underlying assumptions.
We introduce two methods: HSPI with Border Inputs (HSPI-BI) and HSPI with Logit Distributions (HSPI- LD), and show their near-perfect success rates white- box settings distinguishing between quantization levels and high accuracy rates of between 83.9% and 100% for distinguishing between real hardware platforms. Our empirical findings also indicate success rates in cer- tain -bx setups that are up to three times higher than random guess accuracy. We experiment with both emulated quantization and real GPU hardware.
3. Related Work
Hardware Fingerprinting in ML: Schlögl et al. first iden- tify unique hardware fingerprints, discovering that different hardware platforms are in different EQCs (2021; 2021). They propose boundary samples, which are inputs to a model that output results at the decision boundary between two classes. For example, an input can lie the boundary of being classified as cat or dog, depending the hardware. They use adaptation of the search algorithm iterative fast- soudient-sign-mettee (FGSM). also known as Projected Gradient Descent (PGD), for generating adversarial samples and differentiate between 4 CPU models.
We describe the limitations of HSPI in white- and black-box setups and provide a discussion on their potential usage across software and hardware supply chains for transparency and ML governance.
This sets the groundwork for our hardware fingerprinting methods. However, their algorithm is inefficient due to the two phase setup and the remote phase has success rate of only 28. 28.25% on CIFAR10. Our work improves this method with HSPI-BI, and we additionally propose a new method without the need of backward propagation (HSPI-LD). Further, we mainly target GPUS, considering both white-box setup and black-bor setup, and extensively
2. What makes HSPI possible?
We briefly explain how varying software and hardware con- figurations can shift a model into different Equivalence Classes, and how arithmetic ordering and optimizations can contribute to computational discrepancies.
2

---

Hardware and Software Platform Inference
perform evaluations on modern ML workloads like large language models.
publicly available LLMs, e.g., LLaMA (Dubey et al., 2024) and Gemma (Team et al., 2024), can be deployed locally for sending queries and receiving responses;
Exploitation of Floating Point Inaccuracies: Some work has also emerged exploiting floating point inaccuracies, for example, against robustness verifiers of neural networks (Jia
Black-box access only - the deployed model can only be accessed via cloud-based interface, where it returns the output probabilities. This is similar to the current serving practices of Google, OpenAl, and Anthropic.
& Rinard, 2021; Zombori et al., 2021). They point out that errors in floating point can be used against verifiers that do not consider deviations in floating point computations. Zombori et al. suggest adding small perturbations to the weights as an adhoc mitigation, they warn against other attacks using these deviations. As such, Clifford et al. for instance show that deviations in how specific operations are calculated on different hardware platforms can enable locking models to certain hardware (Clifford et al., 2024).
We also assume that it is possible to access N different hard- ware platforms (H = {Ho, Hi..., -1)),, where HSPI then tries to identify the hardware H, E H that the model is deployed on. To the best of our knowledge, currently, all known hardware platforms suitable for model serving can be rented. Even newer hardware devices eventually are becoming accessible demand seen with recent Grog (Gwennap, 2020) and Cerebras (Lie, 2022) hardware, thus making our approach feasible and realistic.
Black-box LLM Model Identification: Other works have focused on identifying the LLM model family in black- box setting with only access to output text through seman- tic analysis (Iourovitski et al., 2024; Pasquini et al., 2024; McGovern et al., 2024), a variant where logit outputs are assumed (Yang& & Wu, 2024) or a more white-box setting where outputs from each layer can be accessed (Zeng et al., 2023; Zhang al., 2024). Accuracies for distinguishing between model families in the black-box setting have been 72% and 95% with only 8 interactions (Iourovitski et al., 2024: Pasquini et al., 2024).
Note that slight performance and numerical variations may arise from the underlying software stack, even though the algorithm and hardware remain constant. For example, in context of machine learning, GPU kernel fusion strategies and runtime scheduling can influence the EQC of the model. We also explicitly include these variations in our H. Con- sequently, HSPI can also be used to determine both the hardware platform and the software configuration, thereby identifying the combined hardware-software supply chain.
4. Methodology
4.2. HSPI-BI: HSPI with Border Inputs
We start by formally describing the HSPI and listing the underlying assumptions in Section 4.1. In Section 4.2, we propose border inputs, which are inputs that are specifically designed to elicit divergent behavior in varying hardware and software configurations. In Section 4.3, we devise an- other method without the need of specifically-craffi data, by identifying deviations in floating-poin -riin distribution of model-returned logits.
We reintroduce the concept of boundary samples with the name border inputs. As explained by Schlögl et al. these are specially crafted inputs that are at the decision boundary between two or more output classes of a model (2023). The idea of border inputs is similar the idea of adversarial examples and we also modify PGD to create border inputs, however, formulate our loss function differently.
Specifically, consider a model F which runs in two different hardware environments. Deploying models on alternative hardware environments H or H' can lead to subtle diver- gences in logit outputs FH. To maximize this divergence. modify input X until the predicted labels y and y/ differ. We define a loss function Lpgd as the sum of cross-entropy losses between each model's logits and the other's 's predicted labels:
4.1. Problem Formulation and Assumptions
The implementation of model serving in practical settings differs in accessibility.. Service providers like Google, Ope- nAI, or Anthropic keep the underlying model architecture undisclosed, and users simply send queries and receive re- sponses through API calls. Most services, including OpenAl APIs, are capable of providing users with the model output probabilities (logits). In other scenarios, we have cloud providers, such as Azure and AWS, for the deployment of open-source models, where both the model weights and ar- chitectures are known. We thus evaluate the following two representative scenarios:
(3) We maximize Lpgd by iteratively updating X along the gradient direction while clamping it within a valid range of the original input, thereby encouraging the models to predict different class labels.
Epga = C(F ((FW)
White-box access to the model- - the deployed model is known and can be accessed freely. For example,
We can also maximize divergence by pushing (X) to- ward a target class ye while pushing Fir()) away from it.
3

---

Hardware and Software Platform Inference
Request
sign exponent
fraction
system: You are helpful assistant
FP32 logit
user: Please generate 16 random words
assislant: Here are 16 random words: choler pneumodynamics
Split into sign, exp, and fraction, Zero exlend each and view as INT32
FP32 Logits vocab size
Response (FP16 LLM)
INT32 triplet
INT32
INT32
INT32
unrulableness
ultimateness terebinthina avast
Figure 2. Splitting an FP32 logit into three INT32 numbers. In case that rounding noise pollutes the bit distribution in FP32 logits, before training SVMs, for each logit, we extract the sign, exponent, and fraction, zero pad each component and view each as an integer.
num tokenss
sample logits, split logit bits
SVM input
Label
FP16
This is achieved through a loss function that subtracts the cross-entropy losses:
Figure 3. Generating HSPI-LD samples using LLMs. We guide LLMs to generate random words. The logits are flattened to form an input vector for training hardware platform classifiers.
Li-vs- I, turgeted = (X) - C(FFr(((),))))))) vr).
(4)
When the number of possible serving platforms is small, we can extend this to compare one model against all others by summing all losses and subtracting the loss only for the one model for which we want distinct class:
In practice, we an SVM as the classifier G and train this classifier on small calibration set of logits.
4.4. Spdel-Specific Adaptation
1-vv--rst
Σ KENTE
(X),
(X),
We adapt the general methods defined in Section 4.2 and Sec- tion 4.3 to fit vision models or language models. For vision models, the border inputs of HSPI-BI are randomly sampled from the model's training set, while the input images of HSPI-LD are random noise as they trigger more diverse areas of the feature space. We also carefully round border images or noise images to integers between and 255 to ensure the final images are available in PNG format.
(5) As N increases, optimizing between models becomes more challenging. In such cases, we decompose the HSPT task as N(N-1) binary classification problems by applying Equa- tion (3) or Equation (4) iteratively across pairs of models to estimate the most likely hardware environment probabilisti- cally. Note that when testing various precomputed border inputs in a black-box setting, logits become unnecessary and only the predicted class is needed.
2
For language models, we generate random texts as the initial border queries of HSPI-BI. Since input IDs are one-hot encoded, we need to ensure that the updated border requests are still valid input IDs at the start of each PGD step. To achieve this, we first update the one-hot encoded input IDs, then use argmax to find the index of the maximum value in the updated vector for each token. For the HSPI-LD method, we guide the model to generate random words and sample top-k logits of each token. We find such prompts encourage the top-k logits to have closer values and avoid Inf values. Figure 3 shows how output logits are sampled.
4.3. HSPI-LD: HSPI with Logit Distributions
An alternative approach involves developing a classification model leveraging the information in the distribution of out- put logits. The logits of a set of inputs reveal characteristics of the hardware environment. The logit distribution is espe- cially informative when inputs are diverse in the distribution of classes and closeness to class boundaries. To amplify these characteristics, we convert the logits into binary repre- sentations for small models, more cost-efficiently, split the floating-point components (Figure 2) for large models.
5. Evaluation
In this section, we briefly describe the experiment setup in Section 5.1, and then present the results of white-box in Section 5.2 and black-box attacks Section 5.3.
We find models running on different hardware configura- tions produce distinct bit patterns in their logits, with certain bits used more or less frequently. Given access to all hard- ware configurations H. and input samples X, E X. we can create a classifirr G that learns to identify the environment (e.g., whether a sample was processed by FR or F..). Math- ematically, this classifier can be defined as:
5.1. Experiment Setup
We consider the white-box and black-box attack setups de- scribed in Section 4.1. We build the set of hardware H. under two distinct configurations: (1) Initially, we examine models in low cwwpiiccioonn formats (quantization). a popular technique integrated into frameworks like HuggingFace (Wolf et al.,
1. if F = FH
G(F(X.))
(6)
if F FH
4

---

Hardware and Software Platform Inference
2020), TensorRT (NVIDIA, 2024), and TorchAO (Torchao, 2024). Hardware devices typically feature specific arith- metic operators (e.g., INT4 for A100 and FP8 for H100), and we regard differentiating different low-prectn ow-precision formats as a preliminary step before differentiating actual GPUS. (2) We then extend the experiments to a setup closer to real deployment scenarios, comparing actual GPUs to operate with various arithmetic configurations, different kernel im- plementations, and across varying device families.
FP32 logits
0.6
BF16 logit
FP16 logits
0.5-
MXINTB lngits
FP8-E3 logits
Densty
0..4
FPB E4 logits
lngits
0.3
0.2
0.1
0.0
-2
2
Logit Valuues
Vision Models We mainly use the image classification models from terchvision and fine-tune them on CI- FARIO, including ResNet18, ResNet50 (He et al., 2016), VGG16 (Simonyan & Zisserman, 2014), EfficientNet- BO (Tan & Le, 2019), DenseNet-121 (Fu et al., 2021), MobileNet-v2 (Sandler et al., 2018). MobileNet-v3-small,, and MobileNet- -a-lage (Howard et al., 2019). We short- list the number formats, and GPU specs in Appendix B. Note that not all GPUs were used for all groups of experi- ments due to different server locations and the difficulty of connecting all of them for the attack generation phase.
Figure 5. Kernel density estimate of logit distrihutions of different quantization classes on the classification of the same 5000 images for CIFARIO with ResNet18, i.e., 50000 logits.
Vision Models We find HSPI-BI always distinguishes be- tween different number formats/quantization methods on the same GPU. Examples of border images can be found in Figure 4. Figure 5 illustrates the kernel density estima- tion of various quantization methods where the shapes imply the obvious differences. When extending to real GPUs, we find the HSPI-BI generally works except for GPUs with very similar specs. For example, border images can be created between RTX8000 and H100 when running on iden- tical CUDA version and PyTorch version, but not between RTX8000 and RTX2080Ti (see Table 8 in the appendix). This is because both RTX8000 and RTX2080Ti have the same Compute Capability of 7.5 and very similar number of Tensor Cores around 550.
Language Models We use the open-sourced instruction- tuned LLMs from HuggingFace, including Distill- GPT2 (Sanh et al., 2019), LLaMA-3.1 (Dubey et al., 2024), 2Wn (Yang et al., 2024), Phi-3.5 (Abdin al., 2024). Gemma-2 (Team al., 2024), and Mistral (Jiang et al., 2023). We use the official chat templates to guide the LLM to generate random words. The prompts also include a few manually added random words to increase the diversity of generation. Detailed prompts can be found in Appendix B.2. Besides standard floating-point formats, we adopt two SoTA quantization methods highly optimized for LLMs. The detailed configurations and GPU specs can also be found in Appendix B.2.
HSPI-LD can distinguish between all quantization levels for the models ResNet18, ResNet50, VGG16, EfficientNet, DenseNet, and MobileNet-V V2 when using an SVM with a set of 10 images' logits per sample. When comparing across GPUs, HSPI-LD are able to distinguish the A100 well from the RTX8000 and RTX2080Ti with an accuracy of 100% (Figure 9 in the appendix shows the clear difference of logit bit distribution). Again, HSPI-LD fails to distinguish between the RTX8000 and RTX2080Ti, as calculations are in the same EQC. We discuss the limitations and robustness of HSPI in Appendix E and Appendix F.
5.2. White-box Attacks
Language Models We find that HSPI-BI is not suitable for LLMs. With enough reruns and PGD iterations, HSPI- BI can generate border requests that lead to different re- sponses across quantization methods. However, we find that the PGD process is unstable due to the discontinuity in its projection. Moreover, limited GPU memory constrains our ability to scale experiments larger LLMs and bigger request batch sizes. This is because the forward pass of the target model is part of the computation graph for gradient descent, requiring caching the intermediate activations of LLMs in memory for backward propagation.
FP16: FROG
FP16; PLANE
FP16: PLANE FP8-B4: CAT
FP16: fROG FP8-E3: CAT
INT8: PLANE
MXINT8: DEER
BF16: CAT INT8: PLANE
BF16: DOG MXINT8: BIRD
BF16: HORSE FP8-B4: CAT
BF16: DEER FP8-E3: CAT
Figure 4. Example border images of Mobileleet..-SSmlll gener- ated by HSPI-BI. BI. The predicted label changes when fed to the same model quantized to different number formats. The subcap- tion follows the format of model format: predicted label.
On the contrary, HSPI-LD achieves remarkable success in both quantization experiments and real GPU experiments.
5

---

Hardware and Software Platform Inference
Table Initial white-box experiments on actual GPUs. With fixed CUDA version, we perform HSPI-LD on LLaMA-3.1 -BB. The trained SVM distinguishes GPUs, arithmetic modes, and even kernel implementations in Group 1, 2, and respectively.
MIIDW-110-SSXA4-800 180GB_FFFPP FP16
25-1
NMIDW-RTX-A6000, FP16
Bit count
Group
GPUs
Ariths.
Kernels
Acc.
1
A100, A6000
FP16
Plain
1.00
5
2
A100
FPI6, BF16
Plain
1.00
3
A100
FP16
Plain, FlashAtt22
1.00
120 1
224 104
220
position
Figure 6. The difference bit distribution between RTXA6000 and A100 (white-bon HSPI-LD), We send the same 256 queries to QWen-2.5-3B deployed on RTXA6000 and A100 respectively and compare the bit distribution of FP32 log probabilities. Tokens and logits are sampled in the plot but the diffcrence is still obvious. tzl denotes the log probability of i-th token's 's j-th logit.
Table 2. Whitc-hox experiments on actual GPUs, We perform white-box HSPI-L. setup mixing GPUs, arithmetic modes, and kernel implementations. We treat this as a classification task with 18 unique labels. By-class accuracy and Fl score are the last two columns. HSPI-LD achieves an overall accuracy of 83. 83.9% using only 256 requests (random guess accuracy = 5.6%).
GPUs
Ariths.
Kernels
Class Idx
Acc.
F1.
HSPI-BI
45
HSPI-31
Plain
0.742
0.745
Accuracy (%)
40
Random
Accuracy (%)
Random
FP16
SDPA
0.645
0,683
35
FlashAtt22
3
0.695
0.698
30
30-
A100
25
Plain
BF16
SDPA
FlashAtm22
6
100
200
250
800 1000
1200
Batch size
Number of iterations
Plain
7
0.680
0.705
FP16
SDPA
8
0.656
0.656
Transferability vs batch size
(b) Transferability num iters
FlashAttn2
0.688 9
0.674
A6000
Plain
10
Figure 7. Transferability of border images trained on MobileNet- v3-Small. (a) The transferability is improved with larger batch
BF16
SDPA
11
FlashAtt22
12
sizes. (b) Given batch size (256 in the plot), overfitting happens if the border images are trained with too many iterations.
Plain
13
0.688
0.689
FP16
SDPA
14
0.644
0.622
FlashAtt22
0.664
0.636
L40S
Plain
consuming much less memory. Besides, the collection of responses (logits) for each GPU is independent, without the need of across-node communication.
BF16
SDPA
FlashAttn2
18
Average
0.839
0.839
5.3. Black-box Attacks
Vision Models For HSPI-BI, we estimate the transferabil- ity of set of border images by training the images on one model and testing them other unseen models. Table 3 contains three experiments following this setup. The bor- der images trained on MobileNet-v2 achieve an average Fl F1-score of 0.345 across 7 unseen models (random guess F1-score = 0.14).. We also have the following observations:
For quantized LLaMA-3.1-8B Phi-3.5-mini, and QWen- 2.5-3B deployed in FP16, BF16, INT8-FD, and HQQ-4bit, an accuracy of over 99.5% for each model is achieved (See Table 9 in the appendix). For real GPU experiments, we firstly perform three experiments in Table 1 to verify that HSPI-LD can distinguish between different GPJs, arith- metic modes, and kernel implementations. We then conduct a comprehensive experiment Table 2, mixing all these factors. We treat this experiment a classification task with 18 unique labels. Remarkably, HSPI-LD achieves overall accuracy of 83.9% (random guess accuracy 5.6%). We visualize the evident difference in logit bit distribution between RTXA6000 and A100 in Figure 6.
(1) If border images are trained batches, a larger batch size will improve the transferability across models. Fig- ure 7a sweeps the batch size of border images trained on MobileNet--v3-Small and plots the resultant transferability. With the increasing batch sizes, the transferability is en- hanced. However, we cannot further increase the batch size after 256 due to limited GPU memory.
Note that HSPI-LD is suitable for language models given the size of L.MMs. HSPI-L. only runs inference on the model,
(2) Overfitting happens if the border images are trained with
6

---

Hardware and Software Platform Inference
Table Transferability of HSPI-BI and HSPI-L.D on quantized vision models. We consider of models including VGG-16, ResNet-18 18, ResNet-50, MobileNet- V2, EfficientNet-BBl and DenseNet-122 121. Each row trains the SVM classifier one model and evaluate the transferability terms of Fl-Score on rest models. Random guess -core 0.144. HSPI- all the experiments were run on RTXA6000 with 400 iterations, and HSPI-LD experiments were run on Nvidia Quadro RTX 8000.
Method
Training Model
Test Model
FP32
BF16
FP16
MXINT8
FP8-E3
FP8-E4
INT8
Avg. F1.
VGGI6
Other models
0
0.167
0.234
0.159
0.253
0.218
0.147
HSPI-BI
ResNet18
Other models
0
0
0,25
0.293
0.286
0.167
0.286
0.206
MobileNet-v2
Other models
0.235
0.345
0,218
0.167
0.286
0.444
0.444
0.345
VGGI6
Other models
0.394
1
0
0.95
0.65
0.2
0.599
ResNet18
Other models
0.332
0.986
0.318
0.972
0.682
0.446
0.677
ResNet50
Other models
0.056
0.602
0.634
0.642
0.562
HSPI-LD
MobileNet-v2
Other models
0.026
0.8
0.8
0.342
0.768
0.69
0.498
0.561
EfficientNet
Other models
0
0.2
0.612
0.592
0,486
DenseNet-122
Other models
0.102
0.996
0.34
0.926
0.514
0.638
0.645
too many iterations. Figure 7b presents the transferability of the same group of border images trained with various number of iterations. The decreasing transferability at 800 and 1200 iterations indicates that the border images overfit the training model.
Table 4. Transferability of black-box HPPI-DD quantized LLMs.
We split set of LLMs, F [Gemma-2-2 Gemma-22 98, 1.aaM-3. 1-88, Phi-3. 301m3..mmi...
Mistral-78-vcl
QWen-2. 5-1.2..51.555 CWen-2 5-333, QWen-2. 10.nn2.55-777 5-7B). into supportive (training) set F, and test set F; := F F.. The SVM classifier is trained using responses from F,, and tested on Fr. Random guess accuracy 25%. use ..... denote i-th model in F.
(3) The border images trained on a more complex model tend to have better transferability. For example, in Table 3, the border images trained MobileNet-v2 has higher F1-Score than those trained on ResNer18 8 and VGG16. How- ever, again, more complex models require more GPU mem- ory during training, which leads to limited scalability of HSPI-BI.
FP16
BF16
INT8-FD
HQQ-bit
Avg. Acc.
Avg. Fl.
fs
0.0
0.0
0.865
0.041
0.227
0,108
0.074
0.783
0.538
f7
0.369
0.969
0.980
0.093
0.603
0,545
Fs Fz
08833
0.274
0.450
0.797
0.594
0.582
For HSPI-LD, we compare how well an SVM trained on model performs all other models in Table 3. We observe that the transferability in detecting BF16 and FP16 is particularly high, while for other quantization levels such as FP32 and MXINT8 the transferability is very low. Fur- thermore, ResNet18, followed by DenseNet perform best as models with high transferability to other models.
Table S. Promising approaches (V) and challenges (!) of HSPI.
Vision Models
Language Models
Sctup
White-box
Black-box
Whitc-box
Black-box
HSPI-BI HSPI-LD
Language Models We only perform HSPI-LD in the black-box setup for language models the HSPI-BI method
6. Discussion
unstable. We train SVM set of training models F, and test it another set of unseen models F,, The results of quantized models are shown in Table 4. The transfer- ability measured with average accuracy is between 22.7% and 60.3% (random guess accuracy is 25%). We find that overfitting also occurs if the SVM is trained with too many iterations, and the transferability heavily depends on the choice of models in the training set.
6.1. Tests on LLM Serving Frameworks
LLM serving frameworks like vLLM (Kwon et al., 2023) and SGLang (Zheng et al., 2024) facilitate the deployment of LLMs production environments. In Section 5, we run the experiments with a naive setup. To verify HSPI-LD in production environments, we launch SGLang on cloud GPUs, send queries and collect responses through OpenAI API, and perform HSPI-LD. Though SGLang enables series of optimizations, such as Radix and cache- aware scheduling policy, HSPI-LD successfully recognizes different GPUs combined with various data types, partition- ing strategies, and kernel implementations. Table presents the results where the over 99% accuracy highlights the ef-
Here we summarize the promising approaches and chal- lenges in Table based on our experiments and observa- tions. Though we achieve three to five times higher accuracy than random guess in the blx setup, more effective solutions still need be explored.
7

---

Hardware and Software Platform Inference
need around 60k tokens for each class, which is acceptable considering LLM API usually costs less than $0.05 per 1 tokens (Cortenix & Bruner, 2024).
Table 6. White-hox experiments of aGng using cloud GPU vice. We perform white-box HSPI-LD on setup close to pro- duction environments where LLM serving framework. SGL, is deployed on a multi-GPU cloud server. We send queries and collect responses through OpenAI API.
6.3. Limitations and Robustness
HSPI faces several practical limitations. Some configura- tions remain indistinguishable due to leaving computations in the same EQC for instance, different CUDA versions on identical hardware, or similar GPUs (RTX8000 and RTX2080Ti) with matching compute capabilities. More- over, batch size variations significantly affect logit outputs, requiring separate analysis for each common batch size. We discuss these limitations and possible strategies against the use of HSPI such random bit flips in logits noise injection in input data in Appendices E and F.
GPUs
Ariths.
Kernels
Sharding
ClassIdx
Acc.
F1.
DP2, TP2
0,961
0.972
FlashInfer
FP16
DP4, TPI
2
0.992
DP2, TP2
3
0.992
0.9988
1,40
Triton
DP4, TP1
4
0.992
DP2, TP2
5
Flashinfer
BF16
DP4, TPI
6
1
DP2, TP2
7
Triton
DP4, TP1
8
Flashlnfer
DP2, TP2
0.973
FP16
DP4, TPI
10
0.992
0.988
6.4. Software and Hardware Supply Chains
DP2, TP2
11
0
0.953
A40
Triton
Software supply chain variations across runtimes, compilers, and high-level frameworks, such as PyTorch or Tensorflow, can affect result reproducibility even with identical models and hardware. This highlights the need for careful version control and standardized compilation practices to mitigate potential discrepancies on the software side. Meanwhile, hardware options have proliferated through cloud providers, ranging from various GPU vendors to specialized Al accel- erators (Google TPUs, AWS Inferentia, Groq and Cerebras). The rise of model API marketplaces like OpenRouter has introduced additional complexity, as these services often obscure their underlying hardware configurations. This lack of transparency in hardware platforms makes it difficult to verify security compliance and performance consistency, HSPI methods offer potential solution for validating these hardware supply chains.
DP4, TP1
12
0.953
0.976
DP2, TP2
13
FlashInfer
BF16
DP4, TPI
14
DP2, TP2
15
Triton
DP4, TPI
16
Average
0,991
0.991
fectiveness in a production environment. In Appendix D, we include two more result tables on large models served on devices & software stack from various vendors like AMD, NVIDIA, and Amazon, highlighting that HSPI-LD is appli- cable a more complex setup.
6.2. Scalability and Cost Implications of HSPI
One may question the potentially high cost of HSPI due to the large number of possible combinations of hardware and software. As shown in previous experiments, the num- ber of classes is the product of possible hardware plat- forms and software dependencies, e.g., [L440, ^40)x (FP16, [ DP4TP1| = 16 classes in Table 6. we decompose this classification task into binary classification problems, N(V-1) 120 classifiers need to be trained. However, we emphasize that one can reduce the number of classes by "merging labels". By "merging labels" we mean merging some unim- portant labels (don't cares) into a new label. Take Table 6 as an example, if we label the eight classes [L40)x(FP)) (FP16, (FlashInfer, Triton (DP2TT22 DP4TP1| into new label "I.40" and the rest into "A40", we can train two-class classifier with a high Fl score of 98.5% (See Ta- ble 12 in the appendix). Another factor determining the cost of HSPI is the expense of LLM API access for col- lecting training samples. As detailed in Appendix B, we
6.5. ML Governance
HSPI could significantly advance ML governance by en- abling precise identification of hardware and software com- ponents in ML supply chains. This capability would strengthen three key areas: (1) traceability and account- ability through detailed configuration tracking, (2) establish- ment of industry wide standards for hardware and software documentation, and (3) improved reliability in model shar- ing and deployment. We will elaborate these governance and broader impacts in our impact statement.
7. Conclusion
In this paper, we introduce Hardware and Software Plat- form Inference (HSPI) a method for identifying the un- derlying hardware and software stack solely based on the
hhups///ppenouteraai
8

---

Hardware and Software Platform Inference
input-output behavior of machine learning models. Our ap- proach leverages inherent quantization limitations and the arithmetic operation orders of different GPU architectures to distinguish between various GPU types and software stacks. Our findings demonstrate the feasibility of inferring this information from black-box models, underscoring its im- plications for ensuring transparency and accountability in the LLM market. We will open-source our logit dataset and codes once the paper is accepted.
can further model sharing practices. Detailed information about the training and inference environment reduce deployment issues related to hardware and software mis- matches and can help users understand potential limitations or dependencies on specific hardware or software compo- nents. This will ultimately create an environment fostering confidence in using models developed by other people.
References
Abdin, M., Jacobs, A., Awan, A. A., Aneja, J., Awadallah, A., Awadalla, H., Bach, N., Bahree, A., Bakhtiari, A., Behl, H., et al. Phi-3 technical report: A highly capable language model locally on your phone. arXiv preprint arXiv:2404. 14219, 2024.
Impact Statement
HSPI's potential applications extend beyond hardware iden- tification to several aspects of MI. system governance and deployment. Building on our earlier discussion of trace- ability, standardization, and reliability benefits, this section examines the broader technical and practical implications of HSPI. We consider its effects on security auditing practices, deployment verification protocols, and potential impacts on hardware provisioning strategies.
Arora, A., Wright, V., and Garman, C. Strengthening the se- curity of operational technology: Understanding contem- porary bill of materials. Journal of Critical Infrastructure Policy, 3(1): 2022..
In the perfect HSPI scenario, our methods can help identify the exact components in a model's s software and hardware supply chains mentioned in Section 6.4. This would allow users to produce and validate the software and hardware bill of materials, as suggested by recent research (Arora et al., 2022). By doing so, HSPI could become a key tool in enabling MI. governance a framework of standards and principles to ensure that an ML system operates responsibly upon deployment. The proposed HSPI problem formulation, including Schlögl al.'s insights on EQCs, will help in shaping standards in different stages of an ML model's life cycle from testing to deployment (Chandrasekaran et al., 2021).
Badri, H. and Shaji, A. Half-quadratic quantization of large machine learning models, November 2023, URL https:/ hatlss//mobuusstt www.ons..m................. i htthqq_bhoolgllllppppppppbpooooa
Chandrasekaran, V., Jia, H., Thudi, A., Travers, A., Yaghini, M., and Papernot, N. Sok: Machine learning governance. arXiv preprint arXiv:2109. 2021.
Clifford, E., Saravanan, A., Langford, H.. Zhang, C., Zhao, Y., Mullins, R., Shumailov, I,, and Hayes, J. Locking machine learning models into hardware. arXiv preprint arXiv:2405.22 20990, 2024.
Cortenix and Bruner, A. Llm api pricing, Dec 2024. URL https:/ //t/://wwww// www.:bottee.. www.boootgttt.ooooom.. com/ tool s /ttssmmmmriccccc
Traceability and Accountability In complex ML work- flows, traceability of the exact software and hardware con- figurations used in model training and inference is essential for building trust and accountability. This would help in identifying the origin of specific model behavior differences, making reproduction of behavior easier. This traceability is not only crucial for debugging but also for audits, security, and mitigating risks from potential tampering or errors.
Darvish Rouhani, B., Lo, D., Zhao, R., Liu, M., Fowers, J., Ovtcharov, K., Vinogradsky, A., Massengill, S., Yang, L., Bittner, R., et al. Pushing the limits of narrow precision inferencing at cloud scale with microsoft floating point. Advances in neural information processing systems, 33: 10271-10281, 2020.
Establishing Industry Standards HSPI can drive the adoption of industry-wide standards for documenting and verifying hardware and software configurations in ML devel- opment. Standardized reporting of hardware architectures, library versions, and compilers would enable researchers and practitioners to replicate results reliably. This would also help in establishing ML quality control norms.
Dubey, A., Jauhri, A., Pandey, A., Kadian, A., Al-Dahle, A., Letman, A., Mathur, A., Schelten, A., Yang, A., Fan, A., et al. The llama 3 herd of models. arXiv preprint arXiv:2407.21786 2024.
Fu, Y., Wu, J., Hu, Y., Xing, M., and Xie, L. Desnet: A multi-channel network for simultaneous speech derever- beration, enhancement and separation. In 2021 IEEE Spoken Language Technology Workshop (SLT), pp. 857- 864. IEFF, 2021.
Model Sharing and Deployment Reliability Replicabil- ity through HSPI and newly established industry standards

---

Hardware and Software Platform Inference
Gongye, C., Luo, Y., Xu, X., and Fei, Y. Side-channel- assisted reverse-engineering of enerypted dnn hardware accelerator ip and attack surface exploration. In 2024 IEEE Symposium on Security and Privacy (SP), pp. 4678- 4695. IEEE, 2024.
McGovern, H., Stureborg, R., Suhara, Y., and Alikaniotis, D. Your large language models are leaving fingerprints.
arXiv preprint arXiv:2405. 14057, 2024.
NVIDIA.
TensorRT Open Source Software, January 2024. URL https: /ttpas:///thhb.////g//p http://tttuu..o////i/iit cow/NVIDIA/ TensorRT.
Gwennap, L. Groq rocks neural networks. Microprocessor Report, Tech, Rep., jan, 2020.
Pasquini, D., Komaropoulos,, E. M., and Ateniese, G.
Limmap: Fingerprinting for large language models. arXiv preprint arXiv:2407.15844 15847, 2024.
He, K., Zhang, X., Ren, S., and Sun, J. Deep residual leam- ing for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770-778, 2016.
Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., and Chen, L.-C. Mobilenetv2: Inverted residuals and linear bottlenecks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 4510-4520, 2018.
Howard, A., Sandler, M., Chu, G., Chen, L..CC., Chen, B., Tan, M., Wang, W., Zhu, Y., Pang, R., Vasudevan, V., et al. Searching for mobilenetv3, In Proceedings of the IEEE/CVF international conference computer vision. pp. 1314-1324, 2019.
Sanh, V., Debut, L., Chaumond, J., and Wolf, T. Distilbert, distilled version of bert: smaller, faster, cheaper and lighter. In NeurIPS EMC2 Workshop, 2019.
Hua, W., Zhang, Z., and Suh, G. E. Reverse engineering convolutional neural networks through side-channel infor- mation leaks. In Proceedings of the 55th Annual Design Automation Conference, pp. 1-6, 2018.
Schlögl, A., Kupek, T., and Böhme, R. Forensicability of deep neural network inference pipelines. In ICASSP 2021 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 2515-2519, 2021. doi: N0.1109/CCASSP399288
Iourovitski, D., Sharma, S., and Talwar, R. Hide and seek: Fingerprinting large language models with evolutionary learning. arXiv preprint arXiv:2408.02877 02007, 2024.
94114301.
Schlögl, A., Kupek, T., and Böhme, R. iNNformant: Bound- ary samples as tell-tale watermarks. In Workshop on In- formation Hiding and Multimedia Security &M&&MMSec), pp. 81-86. ACM, 2021.
Jia, K. and Rinard, M. Exploiting verified neural networks via floating point numerical error. In Static Analysis: 28th International Symposium, SAS 2021, Chicago, II, USA, October 17-19, 2021, Proceedings 28, pp. 191-205. Springer, 2021.
Schlögl, A., Hofer, N., and Böhme, Causes and effects of unanticipated numerical deviations in neural network inference frameworks. In Oh, A., Naumann, T., Glober- son, A., Saenko, K., Hardt, M., and Levine, S. (eds.), Advances in Neural Information Processing Systems, vol- ume 36, pp. 56095-56107. Curran Associates, Inc., 2023.
Jiang A., Sablayrolles, A., Mensch, A., Bamford, C., Chap- lot, D., de las Casas, D., Bressand, F., Lengyel, G., Lam- ple, G., Saulnier, L., et al. Mistral 7b (2023). arXiv preprint arXiv:2310.06825. 068.22,,, 2023.
Shen, H., Mellempudi, N., He, X., Gao, Q., Wang, C., and Wang, M. Efficient post-training quantization with fp8 formats. Proceedings of Machine Learning and Systems, 6:483 498, 2024.
Kim, S., Gholami, A., Yao, Z., Mahoney, M. W., and Keutzer, K. I-bert: Integer-only bert quantization. In International conference on machine learning, pp. 5506- 5518. PMLR, 2021.
Simonyan, K. and Zisserman, A. Very deep convolu- tional networks for large-scale image recognition. arXiv preprint arXiv: (4t::::449.555555 1556, 2014.
Kwon, W., Li, Z.,, Zhuang, S., Sheng, Y., Zheng, L., Yu, C. H., Gonzalez, J., Zhang, H., and Stoica, I. Efficient memory management for large language model serving with pagedattention. In Proceedings of the 29th Sym- posium on Operating Systems Principles, pp. 611-626, 2023.
Tan, M. and Le, Q. Efficientnet: Rethinking model scal- ing for convolutional neural networks. In International conference machine learning, pp. 6105-6114. PMLR, 2019.
Team, G., Riviere M., Pathak, S., Sessa, P. G., Hardin, C., Bhupatiraju, S., Hussenot, L., Mesnard, T., Shahri- ari, B., Ramé, A., et al. Gemma 2: Improving open language models at a practical size. arXiv preprint arXiv:2408.00 00008000/110 2024.
Lie, S. Cerebras architecture deep dive: First look inside the hw/sw co-design for deep learning: Cerebras systems. In 2022 IEEE Hot Chips 34 Symposium (HCS), pp. 1-34. IEEE Computer Society, 2022.
10

---

Hardware and Software Platform Inference
Torchao. torchao: PyTorch native quantization and sparsity for training and inference, October 2024. URL https/ /github. mrrithu.ccmm/aa http///yoooo/tch....../
et al. (2018) and Gongye et al. (2024) assume side channel information about the HW like off-chip memory access patterns, electromagnetic traces, schematics is accessible, while we assumee the SW-HHW platform serves DNNs in the cloud and we only have access to the request and responses via the service provider's 's API. These two works test their methods on a specific FPGA accelerator, while We tried our best to test the generalizability. across model families, data types, GPU archs, parallelism strategies, etc.
Wolf, T., Debut, L., Sanh, V., Chaumond, J., De- langue, C., Moi, A., Cistac, P., Ma, C., Jernite, Y., Plu, J., Xu, C., Le Scao, T., Gugger, S., Drame, M., Lhoest, Q., and Rush, A. M. Transformers: State-of-the-Arr Natural Language Processing. pp. 38-45. Association for Computational Linguistics, Oc- tober 2020. URL https: http://www/ccccccccc... www.eb. anthology/ / 220000000222200
B. Detailed Experiment Setup
6.
We elaborate the detailed experiment setup in this sections. For the version information of software, please refer to our source codes.
Yang, A., Yang, B., Hui, B., Zheng. B.. Yu, B., Zhou, C., Li, C., Li, C., Liu, D., Huang, F., et al. Qwen2 technical report. arXiv preprint arXiv:2407.10677 2024.
Yang, Z. and Wu, H. A fingerprint for large language models. arXiv preprint arXiv:2407.01233, 2024.
B.1. Vision Model
Number formats and GPU Specs We perform the ex- periments on standard floating-poing formats (FP32, FP16, BF16) and low-precision formats first, then extend to actual GPUs. The low-precision formats include MX- INT8 (Darvish Rouhani et al., 2020), FP8-E3M4 (noted as FP8-E3) (Shen et al., 2024), FP8-E4M3 (noted as FP8- E4), and dynamic INT8 (noted as INT8) (Kim et al., 2021). For actual GPUS, we include NVIDIA H100, A100, GeForce RTX2080 Ti, and Quadro RTX8000. Not all GPUs were used for all experiments due to different server locations and the difficulty of connecting all of them for the attack generation phase.
Zeng, B., Zhou, C., Wang, X., and Lin, Z. Huref: Human- readable fingerprint for large language models. arXiv preprint 2rxi:2332.008822 2023.
Zhang, J., Liu, D., Qian, C.,, Zhang, I.,, Liu, Y., Qiao, Y. and Shao, J. Reef: Representation encoding fingerprints for large language models. arXiv preprint vIXv:2410..12233, 2024.
Zheng, I.., Yin, I.,, Xie, Z., Sun, C., Huang, J., Yu, C. H., Cao, S., Kozyrakis, C., Stoica, I., Gonzalez, J. E., et al. Sglang: Efficient execution of structured language model programs. arXiv preprint arXiv:2312.07100 2024.
Border Images For HSPI-BI method, we randomly sam- ple images from CIFARIO as initial border images, and apply the PGD method described in Section 4.2 to update the border images. The projection step refers to the opera- tion of scaling and rounding pixel values to integers between 0 and 255. For HSPI-LD method, we use images of size 224x224 that we create through randomly sampling floating point values between ( and I and then converting them into the valid pixel range of integers between 0 and 255. These random images perform better than images from datasets such as CIFARIO as they trigger more diverse areas of the feature space and hence produce a set of more distinet logits. To distinguish between logits with an SVM, we always use a set of 10 or 25 images' logits and since in our access model we test with exactly the same images, report training accuracy for all results.
Zombori, D., Bánhelyi, B., Csendes, T., and Jelasity, M. Fooling complete neural network verifier. In Interna- tional Conference on Learning Representations (ICLR), 2021.
A. Additional Related Work
Side-channel have been exploited to infer information about the model and hardware For example, Hua et al. (2018) extracts information about CNN models, e.g, whether the model is AlexNet or SqueezeNet, from an TPGA-based CNN accelerator, by analyzing memory access patterns. Re- cent work Gongye et al. (2024) predicts information about the encrypted IPs on an FPGA-based DNN accelerator: They also recover characteristics about model architectures and extract model params. Our work aims to predict informa- tion about the SW-HW stack used for serving large deep learning models, including compiler, data types, GPU arch, parallelism strategy, etc.
Fine-tuning ppper-parrmeterss of Vision Models The checkpoints of vision models in Section 5 are downloaded from torchvision' 3. Before HSPI experiments, we fine- tune these models on CIFARIO to ensure they achieve rea- sonable accuracy on CIFARIO test set. Specifically, we use
Our work is different from these two papers in terms of goals, threat models (methods), and generalizability. Hua

---

Hardware and Software Platform Inference
SGD optimizer and linear learning rate scheduler with initial learning rate 1e-3. The fine-tuning batch size is 128 and we fine tune all the models for 3 epochs.
<|imo__tar[] starl]>system
You are Qwen, created by Alibaba Cloud. You are a helpful assistant.<lim harmmt./iim__dppppppp _nnde-
<lim _i__ssaattsssse
Please generate rar___raddom_wwwa random
Accuracy Quantized Models We show in Table that all low-precision formats have negligible accuracy loss pared to the fine-tuned model. This serves as sanity check before the HSPI experiments in case the quantization breaks the models.
words.<|img words_<lim_end|> end|>
<lim jimn_sarl/-asssstann
Here are (num_random, rrandom _random_words) random words:
Trandom_wordse
Figure 8. The prompt template QWen-2.5 used by HSPI-LD in Section 4.3.
Table Accuracy of quantized vision models on CIFARIO.
Model
MXINTS
C. Additional Vision Model Results
VGG16
0,882
0.882
0.881
0.879
0.876
0.875
0.875
Res Net18
0.937
0.937
0.937
0.936
0.931
0.929
0.934
Table shows that HSPI-BI can differentiate RTX8000 and
ResNe150
0.965
MobilleNet-22
B336
B.936
0.966
0,929
0.928
0.26
A100 but fails to differentiate RTX8000 and RTX2080Ti
MobolleNat-vi-mmal
0.914
D.914
0.914
0.909
0.907
0.03
0.07
because they two GPUs have very similar specs. Figure shows the clear difference in logit bit distribution between RTX8000 and NVIDIA A100, which explains the success of HSPI-LD.
MobolleNat-3--bngee
0.946
0.946
0.946
0,943
0,937
0.34
0.455
#NiiionNer-Bd
0.956
0954
1emseNet-121
0.962
0.954
0.954
0.49
B.2. Language Models
Quantization methods and GPUs Similar vision mod- els. first run experiments of differentiating low- wwwprecision formats, then extend to actual GPUs. Since LLM quantiza- tion is more challenging than vision models, besides FP16 and BF16, we adopt two quantization methods that have been proven effective and integrated into HuggingFace: fine- grained dynamic INT8 quantization (Torchao, 2024), noted INT8-FD, and half-quadratic quantization (Badri & Shaji, 2023), noted HQQ-4bit. For actual GPUs, consider NVIDIA A100, L40S, RTX A6000, and GeForce RTX3090.
Table 8. Table showing success in creating border images for dif- ferent GPUs in white-box with inference batch size of 1.
GPUs
FP32
BF16
FP16
MXINTS
FP8-E3
FP8-E4
INTS
REXS00 A100
/
X
X
REXS000
10P
Difference
Difference in FP26
Dffeeence in MXINTS
Prompts and logit datasets For HSPI-LD, guide language models generate random tokens with the adapted official chat template of each model. Figure 8 shows an example of the template of QWen-2.5, where muum_random__ woods) and [random_ / are parameters for generating queries. (random_ _ordss/wordss is n leading ran- dom words sampled from the "popular" corpora of Natural Language Toolkit', thus leaving ( ((num random candom_words) n) tokens for the LLM to generate. Throughout our exper- iments, we find (num_ candom_rnddmmmmmm [ = 16 and 3 is enough for the white-box setup, while for the black-box setup, we adapt caamm_radommmam _ords] 16 and = 8.
10a
Difference
10-
10
15
20
30
Bitt Position (0:31)
Figure 9, A histogram showing the difference logit bit distribu- tion for the classification of the same 5000 images for CIFAR10 ResNet18. ie.. 50000 logits, between Nvidia Quadro RTX 8000 and NVIDIA A100.
We generate 256 queries for each class, and sample logits Tables 1, 4 and because of varying vocabulary sizes of LLMs. Specifically, we sample the top logits of each generated token, and flatten the logits of 4 responses for training sample of SVM classifier.
D. A Additional LLM Results
D.I. White-box HSPI-LD of quantized LLMs
NNauraal Language Toolkit: https: http://wwwwlwtkkoor//c
Table 9 shows the results of HSPI-LD on quantization meth- ods for LLMs with an average accuracy of 0.995.
indxx. html
12

---

Hardware and Software Platform Inference
Table By-class accuracy of white-box HSPI-L.D on quantization levels for LLMs. HPI-LD is able to differentiate LLMs deployed FP16, BF16, INT8-FD, and HQQ-4bit. The SVM classifier is trained using responses of 256 random word requests.
Table 12. White-hox experiments of Table 6 with merged lahels. We label the samples with the same GPU but various software as the same class and train a new classifer. The classifier still achieves high accuracy and Fl score, implying label merging is an effective way to reduce the number of labels and to improve the scalability of HSPI-LD.
Training Model
FP16
BF16
INT8-FD
HQQ-dbit
Avg. Ace.
0.996
0.999 1
Q.en-2.5-BB
GPUs
Accuracy
F1 Score
Phi-3.5-mini
0.996
0.996
0,988
0.995
A40x (PP66, BF16)x |x (Faashhinnerr [Di2m)) DP4TP2]
0.950
0.985
[FP| (Fasht(R, [rito)) DPTTP2)
0.979
0.985
Avenge
0,985
0.985
D.2. HSPI-LD on Various GPU Systems
In Table 10, we distributed a large LLM, Llama-3.J 3.1ma-...-70B.. across a larger multiple high-end GPU systems using SGLang. HSPI-LD still achieves high accuracy. We also test HSPI-LD on devices from different vendors, including NVIDIA, AMD, and Amazon in Table 11. Different devices are distinguished by their hardware architecture, software stack, and data types.
E. Limitations and Black-box Failure Cases
We notice that not all software configurations have an effect on the final model performance, leaving computations in the same EQC. For example, we tested the HSPI-BI method different CUDA versions, while keeping the hardware and the rest of the software stack fixed. A ResNet18 model trained on the CIFARIO and run on an RTX8000 GPU, re- sulted in no measurable differences, thereby HSPI-BI com- pletely failed to distinguish between the CUDA versions. Similarly, not all hardware is distinguishable. For exam- plc, HSPI-BI failed to differentiate the RTX8000 and the RTX2080Ti (both have NVIDIA Compute Capability 7.5 and similar number of Tensor Cores around 550).
Vendor
Hardware (arch)
N devices
DP:T
DTType
Acc
F1.
AMD
MI30XX (CNDA3)
2
DPITP2
BF16
1.000
1.000
FPI6
0.961
0.957
BF16
1.000
1.000
H100 (Hooper)
4
DP2TP2
FP16
0.930
0,941
NVIDIA
BF16
1.000
1.000
L40 (Ada)
DPITP8
FP16
0.969
0.958
A40 (Ampere)
8
DPITPS
BF16
1.000
1.000
FP16
0.922
0.925
Furthermore, we see big differences in calculated logits between different inference time batch sizes. We find that the same batch size needs to be used to distinguish between hardware. This makes our methods more complicated. For example for HSPI-BI, some border images work across a few batch sizes, but do not work for all. This means that we would need to compute border inputs for all most popular batch sizes between a set of hardware platforms or at least enough to make probable guess on it. Similarly, we would need to collect different sets of logits for various batch sizes for HSPI-LD, considering that Anthropic recently released new APIs for batched request.
Avg
0.973
Table 10. Whitebox HSPI-LD on various real GPUs.
Vendor
Hardware
Software
DType
Accuracy
F1.
BF16
0.968
0.924
Amazon
Inferentia
NKI
FP16
0.988
0.908
BF16
1.000
1.000
AMD
MI300X
ROCm
FP16
0.906
0.913
BF16
1.000
1.000
NVIDIA
H100
CUDA
PP16
0.922
0.915
This situation prompts inquiry into the nuances of software- hardware supply chains. For instance, the extent of floating- point deviations in logits varies significantly depending on the targeted supply chain. Identifying different quantization levels may be straightforward, but discerning subtle distinc- tions, such as those between FlashAttention V2 and V3, can present a challenge. Minor differences or distinctions in the supply chain may pose challenges to both HSPI-BI and HSPI-L.D techniques. Nonetheless, there is another argu- ment that an empirically-guides design of the loss function in HSPI-BI, or enhanced feature engineering for HSPI-LD could improve the distinction of these subtle variations. This aspect warrants further investigation in future research.
Avg
0.964
0.943
Table 11. bhite-box HSPI-LD on devices from various vendors.
D.3. Label merging
With the same samples in Table 6, we merge the samples of the same GPU but different software dependencies into a new label and train a new classifier. Surprisingly, the classifier still achieves a high accuracy, implying that label merging can effectively reduce the number of classes and improve scalability of HSPI-L.D.
13

---

Hardware and Software Platform Inference
F. Robustness of HSPI
Several mitigation strategies can be employed to make the inference of hardware and software platforms significantly more difficult. These strategies focus on disrupting the patterns and subtle variations exploited by HSPI. For log- its, introducing random bit flips into the decision-making process can help mask the unique quantization patterns asso- ciated with specific hardware architectures. This technique adds a layer of noise that obscures the underlying hardware fingerprints, while incurring minimal overheads associated with sampling random numbers. Similarly, for border in- puts, adding random noise the input data can disrupt the precise calculations that lead divergent behavior across different hardware and software configurations. This noise injection makes it harder for attackers to identify the specific conditions that trigger variations in model outputs, yet can potentially come with utility degradation.
14


Reviews:
[{"review_id": "RbJNbgnnQz", "reviewer": "Reviewer_4ddm", "summary": "This paper introduced Hardware and Software Platform Inference (HSPI), which is a method for identifying the hardware and software stack based on the input-output behavior of machine learning models. The proposed method leverages the inherent differences of various GPU architectures and compilers to distinguish between different GPU types and software stacks. Experiment show that in a white-box setting the method can distinguish between different GPUs with between 83.9% and 100% accuracy; in a black-box setting the method can achieve results that are up to 3 times higher than random guess accuracy.\n\n## update after rebuttal\nQuestions are answered, I remain positive about this work and would like to keep my score.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank  you for offering valuable suggestions and questions. We would like to address them one by one.\n\n> S1. The authors shared their code to training the classifier and run inference. We don't have the logit dataset to run but it looks legit.\n\n- We know collecting the logits from various combinations of `[model checkpoint, hardware stack, software stack]` are resource consuming, so we will open-source the logits datasets once the paper is accepted.\n- The supplementary materials also include the codes for producing the logit dataset. For example, in line 227 of `llm_logits_svm.py`, the function `create_logits_dataset` collects logits for training the classifier.\n\n> Questions For Authors:\n> Q1. Given the current method for hardware and software platform inference, how to avoid the stack to be exposed to users?\n\nThere are several possible ways but may have side effects on user experience like lower generation quality and longer latency.\n- In Appendix 5, we suggested random bit flips and adding random noise as potential ways of mitigation. For further explanation, please refer to Appendix 5 on page 13. \n- Furthermore, adding other forms of randomizations such as reordering arithmetics randomly or switching between multiple compilation kernels would make our method more expensive, since then a statistical average would need to be calculated on a lot more samples to see a difference in distributions between different hardware and software platforms. \n- Lastly, limiting model logit access, also would make it harder for HSPI. However, the logits is an important part of API enabling users to explore smart sampling strategies like test-time scaling.\n\nWe will add this discussion to the revised manuscript. \n\nPlease let us know if you have further questions :)", "claims_and_evidence": "1. HSPI is possible because of the different characteristics of hardware and software configuration -- this is supported by the analysis in section 2 and examples showed in Figure 2, Figure 3 and Figure 5.\n2. HSPI can effectively distinguish different GPUs with high accuracy in both white-box and black-box setting -- this is supported by results demonstrated in Table 1-4.", "methods_and_evaluation_criteria": "Yes the authors proposed to use different combinations of models, GPUs, kernels, and quantization method to conduct white-box and black-box experiments against HSPI, and collect accuracy / F1 score to show its effectiveness. Details was demonstrated in Table 1-6.", "theoretical_claims": "N/A.", "experimental_designs_or_analyses": "1. The factors made HSPI possible discussed in section 2 makes a lot sense, and demonstrate good insights into low level implementation of machine learning models.\n2. Experiments including white-box and black-box, authors proposed to compare the identified quantization method, GPU type, and kernels used to ground truth, and compare accuracy / F1 score. The scores demonstrated its capability to identify underlying hardware and software choice, with high accuracy.", "relation_to_broader_scientific_literature": "This paper is based on the ideas from Schlögl et al. (2023) where it targeted CPU identification based on the same equivalence classes. And further, the authors considered Computational Deviations, and focused on GPUs with both white-box and black-box settings, proposed an effective solution to identify the underlying hardware / software stack with high accuracy.", "essential_references_not_discussed": "N/A.", "other_strengths_and_weaknesses": "This paper provided a very new perspective to look at current model hosting hardware and software stack, especially considering the quantization methods and white-box vs. black-box scenarios.", "other_comments_or_suggestions": "N/A.", "questions_for_authors": "1. Given the current method for hardware and software platform inference, how to avoid the stack to be exposed to users?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "xhjv8nKgHJ", "reviewer": "Reviewer_WXiw", "summary": "The paper presents an interesting idea where a client can infer the hardware and software platform that was used for model inference based on its input and output behavior. It banks on the observation that there are inherent differences between different GPUs and software stack. The idea has potential to allow clients to verify the actual hardware that was used for inference against malicious service providers.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you offering valuable suggestions and questions. We will address them one by one. Due to word limits, we uploaded **three new result tables** here: https://imgur.com/a/VbONCoU\n\n> # E1. Essential References Not Discussed\n\nOur work is different from these two papers in terms of goals, threat models (methods), and generalizability.\n\nGoals\n- Ref 1 extracts information about CNN models, e.g, whether the model is AlexNet or SqueezeNet, from an FPGA-based CNN accelerator, by analyzing memory access patterns.  \n- Ref 2 predicts information about the encrypted IPs on an FPGA-based DNN accelerator. They also recover characteristics about model architectures and extract model params. \n- Our work aims to predict information about the SW-HW stack used for serving large deep learning models, including compiler, data types, GPU arch, parallelism strategy, etc. **We do not aim to predict which model** is being served. \n\nThreat models\n- Ref 1 and 2 assume side channel information about the HW like off-chip memory access patterns, electromagnetic traces, schematics is accessible.\n- We assume the SW-HW platform serves DNNs in the cloud and **we only have access to the request and responses via the service provider’s API**.\n\nGeneralizability\n- Ref 1 and 2 tests their methods on a specific FPGA accelerator. \n- We tried our best to test the generalizability of HSPI, across model families, data types, GPU archs, parallelism strategies, etc. \n\nWe will include these two papers in our related work section and discuss the differences there.\n> # Q1\n> I am very curious how a client would differentiate between \"deviations\" from HW and \"deviations\" from SW ...\n\nOne can differentiate deviations caused by HW and SW because they will be in different EQCs and we can enumerate all of them by going through all HW and SW combinations. However, in Sec 6.3 and A.4  we do get the same output from RTX8000 and RTX2080Ti (they both have NVIDIA compute capability = 7.5 and very similar number of Tensor cores ~550) when using the same SW stack, thus these two combinations are left in the same EQC. Please refer to 6.3 and A.4 for more details.\n> # Q2\n> As the paper states in A.5, the model inference could have introduced some noise/perturbations ...\n\nOne would need to increase the number of queries sent by HSPI to capture statistics in different HW and SW settings (the cost increases, too). On the other hand, the introduced noise reduces the model performance, e.g, LLM generates low-quality texts.\n> # Q3\n> It would be interesting to investigate what features were used to distinguish different precision, kernel, HW, ...\n\nOur method is based on the EQC theory explained in Sec 2. Limited by rebuttal word counts, we would recommend [this paper](https://openreview.net/forum?id=6zyFgr1b8Q), in which they present a more detailed analysis identifying how architectural choices impact computational stability and precision deviations.\n> # Q4\n> It would be more interesting to see experimentatal results for more variegated class of accelerators ...\n\n- Yes, we are also curious about this. We believe our observation on NVIDIA GPUs still holds on other accelerators because there may be more differences captured by HSPI classifiers, such as different accumulator length in compute units. Unfortunately, limited by resources, we are not able to test HSPI on all these platforms.\n- We run new experiments including Amazon Inferentia and AMD Instinct MI300X (**RTab.1 in the link**). HSPI still successfully differentiates them (avg.acc=96.4%).\n\n> # Q5\n> For the black-box access only scenario in 4.1, client might not have the full picture considering that the service providers may have appended prompt ...\n\n- During experiments, we already prepend chat prompts before our request. For example, each of our Qwen-2.5 input concatenates “You are Qwen, created by Alibaba Cloud. You are a helpful assistant” and our query “Please generate <num_random_words> random words …” . We find using these simple chat prompts actually helps to generate more random outputs.\n- For complex scenarios, HSPI can be combined with prompting tricks like jailbreaking to be more robust. \n\n> # Q6\n> Service providers may limit the inference requests ... DoS attack.\n\nCommercial inference services are designed to support high‑throughput access. Moreover, HSPI does not need to collect all the responses in a very short period, so request rates of HSPI are lower than the limits set by service providers, lower than DoS attacks by order of magnitudes. For example, in Tab 6 experiments, we sent 6 requests/sec, and our method collects 256 inference queries in around 40 minutes, which is comparable to normal application workloads. We can also distribute queries across multiple accounts in case of strict rate limit.\n> # Q7\n> How does the paper scale to larger set of HW and SW platforms...\n\nWe ran new large scale experiments (**RTab2** in the link) and HSPI still works well. Please refer to our answer to Reviwer A46R.", "claims_and_evidence": "Besides some questions about the robustness of the approach, I think the paper seems convincing.", "methods_and_evaluation_criteria": "Besides the fact that the evaluated hardware and software platforms are not diverse enough, it seems the paper tried to evaluate across various inference scenarios.", "theoretical_claims": "Seems the given equations make sense. There are no theoretical proofs for it.", "experimental_designs_or_analyses": "Seems like the approach seems valid. However, there are some questions regarding its robustness and scalability.", "relation_to_broader_scientific_literature": "The paper presents a simple yet insightful extension to the line of work where we have used input/output data to infer the model and the hardware architecture, and various defense mechanisms against these side channel attacks and IP infringement.", "essential_references_not_discussed": "The paper does not seem to shed enough light on the line of work where side channels are exploited to infer the model and the hardware.\n1. Hua, Weizhe, Zhiru Zhang, and G. Edward Suh. \"Reverse engineering convolutional neural networks through side-channel information leaks.\" Proceedings of the 55th Annual Design Automation Conference. 2018.\n2. Gongye, Cheng, et al. \"Side-channel-assisted reverse-engineering of encrypted DNN hardware accelerator IP and attack surface exploration.\" 2024 IEEE Symposium on Security and Privacy (SP). IEEE, 2024.\n\nEspecially, given that the 2nd paper mentioned above is suggesting that side-channels can be exploited to reverse-engineer the hardware IP, it seems like there are some similarities. It would benefit the reader if the authors can give a description about how the approach and applications may differ.", "other_strengths_and_weaknesses": ".", "other_comments_or_suggestions": "The paper presents a simple yet nice idea to infer hardware and software platforms during inference. I think the paper provides a large set of experiments for various scenarios. I would like to stay positive about the paper. However, it would be great to see how the proposed HSPI approach would scale for larger set of hardware and software platforms. Also, would love to understand the potential ways to make this more robust.", "questions_for_authors": "1. I am very curious how a client would differentiate between \"deviations\" from hardware and \"deviations\" from software. In practice different generations of hardware that have different numerical behavior may return same output due to software optimizations. It seems that 4.1 mentions that these variations are included in H. Can you provide more details in this respect?\n2. As the paper states in A.5, the model inference could have introduced some noise/perturbations (consider the fact that different inference may return different output despite the same model). In that case, how would the client filter out that noise to perform HSPI in a robust manner\n3. It would be interesting to investigate what features were used to distinguish different precision, kernel, hardware, ... Interpretation of the HSPI model would provide some interesting insights.\n4. It would be more interesting to see experimentatal results for more variegated class of accelerators (not just GPUs) considering their prevalence (TPUs, Inferentia, MTIA, Maia100).\n5. For the black-box access only scenario in 4.1, client might not have the full picture considering that the service providers may have appended prompt for LLM inference. How is the HSPI impacted and what may be mitigations to make HSPI more robust?\n6. Service providers may limit the inference requests while the client tries to infer the hardware, how would this be distinguished from the DoS attack.\n7. How does the paper scale tolarger set of hardware and software platforms. Seems like there is a potential for signficant drop in accuracy of the HSPI.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "nNun3Ie6k4", "reviewer": "Reviewer_A46R", "summary": "The paper introduces Hardware and Software Platform Inference (HSPI), a novel method for identifying the underlying GPU architecture and software stack of machine learning models based on their input-output behavior. HSPI uses computational differences across various GPUs and software environments to detect the specific device utilized for inference. The authors present two techniques—HSPI with Border Inputs (HSPI-BI) and HSPI with Logits Distributions (HSPI-LD) and both demonstrate high accuracy rates in white-box and black-box settings while discussing the limitations of the methods.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for valuable suggestions and questions. We would like to address them one by one. \n\n> # W1. Large models and systems\n> The paper states that limited GPU memory would constrain the ability to scale to larger language models; even with HSPI-LD the experiments were conducted on rather small LLMs. Yet, the motivation is more about checking whether a very large model is indeed deployed instead of using a smaller, distilled model. (A possible solution could be combining HSPI with distributed methods.\n\nYes, large scale experiments are important for evaluating HSPI.\n- In Tab 6 on page 8, we distributed Qwen-2.5-14B across four-GPU systems using the popular LLM serving engine SGLang and enabled a series optimizations including custom cuda kernels, RadixAttention, torch.compile, cuda graph, smart request batching, etc. HSPI works well and even **differentiated different parallelism strategies like DP2TP2 vs DP4TP1**. We assume HSPI-LD will work on larger distributed systems because the communication between devices does not introduce noise to logits distribution.\n- To verify this, we ran a new group of experiments where **we distributed Llama-3.1-70B across a larger multiple high-end GPU systems using SGLang**. As shown in the following table, HSPI-LD still achieves an average accuracy over 97%.\n\nRTab2: HSPI-LD results for large systems (Llama-3.1-70B-it)\t\t\t\t\n| Vendor | Hardware (Arch) | Num devices | DP,TP | DType | Accuracy | F1 |\n|---|:---:|:---:|:---:|:---:|:---:|:---:|\n| AMD | MI300X (CDNA3) | 2 | DP1TP2 | BF16 | 1.000 | 1.000 |\n|  |  |  |  | FP16 | 0.961 | 0.957 |\n| NVIDIA | H100 (Hooper) | 4 | DP2TP2 | BF16 | 1.000 | 1.000 |\n|  |  |  |  | FP16 | 0.930 | 0.941 |\n|  | L40 (Ada) | 8 | DP1TP8 | BF16 | 1.000 | 1.000 |\n|  |  |  |  | FP16 | 0.969 | 0.958 |\n|  | A40 (Ampere) | 8 | DP1TP8 | BF16 | 1.000 | 1.000 |\n|  |  |  |  | FP16 | 0.922 | 0.925 |\n|  |  |  |  | Avg | 0.973 | 0.973 |\n\n> # W2. Computation efficiency\n> The paper should discuss the computation efficiency of the method.\n\nYes, we will add the following discussion into the revised version.\n\nIn our experiments, HSPI-LD is the most costly method and the cost of HSPI-LD mainly consists of three parts:\n- Collecting training samples: Collecting training samples is the most computationally expensive and slow part. To collect samples, we deploy models on a specific HW-SW setup, run model inference and dump output logits. The cost can be estimated as `num of platform options * num of samples * sequence length * cost of a forward pass`. We spend over 600 GPU hours on this.\n- Training HSPI classifier: As stated in Sec 4.3, we train $N(N-1)/2$ binary SVM classifiers to perform HSPI's $N$-class classification. Thus cost is proportional to `N(N-1)/2 * num of samples`. We use sklearn to train SVMs on CPUs, which is fast as the training iteration is around 1000. Usually the training of all binary classifiers takes less than 20 minutes.\n- Evaluating HSPI classifier: The evaluation feed output logits to all binary classifiers, thus the cost is also proportional to `N(N-1)/2 * num_samples`. The prediction is also on CPU and the cost is ignorable compared to collecting training samples. Usually the prediction takes less than 5 minutes.\n\nFor HSPI-BI, the cost of training a batch of border inputs is twice as normal model training because we need to run the forward-backward pass for a pair of platforms. Since we mainly run this methods for CNN models, the GPU hours were much smaller than HSPI-LD.\n\n> # Q1. \n> Can other common metrics, such as the wall clock time for inference, be useful for identifying the hardware and software stack?\n\nYes it is possible. Wall clock time can be used for creating hardware fingerprints, but we assume wallclock time is less reliable in this context. Deep learning serving engines like TensorRT, vLLM and SGLang may use some dynamic batching strategies which means the wall clock time changes with the request arrival rates. For example, the daytime wall clock time fingerprint is different from the one at night because the server is busier during the day.\n\n> # Q2.\n> Is it possible that different combinations of hardware and software lead to the same prediction?\n\nYes, it is possible. This is one limitation we discussed in Sec 6.3 and A.4. During experiments we found when the software stack is identical, HSPI cannot differentiate RTX8000 and RTX2080Ti. This is because these two GPUs have both NVIDIA compute capability = 7.5 (Turing architecture), and a very similar number of Tensor Cores around 550 (they fall into the same EQC when the software stack is the same). For more details please refer to Sec 6.3 and A.4. \n\nPlease let us know if you have further questions :)", "claims_and_evidence": "The experiments support the submission, showing high accuracy for different algorithms, kernels, datatypes, and sharding techniques.", "methods_and_evaluation_criteria": "The proposed method and evaluation make sense for the problem, with various benchmark datasets for vision and language tasks strengthening the evaluation criteria. The datasets used are also standard datasets in the correspondent areas.", "theoretical_claims": "No proofs were presented in the paper.", "experimental_designs_or_analyses": "The experimental designs appear sound, with plenty . One criticism would be the model size limitation presented in the experiments (See Other Strengths And Weaknesses).", "relation_to_broader_scientific_literature": "The paper presents a new problem of verifying the hardware and software of ML models. Prior work on hardware detection methods have lower accuracy rates.", "essential_references_not_discussed": "No essential work that is unreferenced is noticed.", "other_strengths_and_weaknesses": "Strength:\n1. Novel approach to identifying hardware and software platforms.\n2. Clear organization and well-written.\n\nWeakness:\n1. The paper states that limited GPU memory would constrain the ability to scale to larger language models; even with HSPI-LD the experiments were conducted on rather small LLMs. Yet, the motivation is more about checking whether a very large model is indeed deployed instead of using a smaller, distilled model. (A possible solution could be combining HSPI with distributed methods.\n2. The paper should discuss the computation efficiency of the method.", "other_comments_or_suggestions": "No other suggestions.", "questions_for_authors": "1. Can other common metrics, such as the wall clock time for inference, be useful for identifying the hardware and software stack?\n2. Is it possible that different combinations of hardware and software lead to the same prediction?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "bYD6vNj3MH", "reviewer": "Reviewer_ucu1", "summary": "This manuscript proposed a method called Hardware and Software Platform Inference (HSPI), aiming at identifying the GPU and software stack for machine learning models. The authors introduced 2 methods: \n1. HSPI with Border Inputs (HSPI-BI), which is building inputs that are at the decision boundary of the model.\n2. HSPI with Logit Distributions (HSPI-LD) uses the distribution of output logits to figure out the hardware environment.\nTo evaluate ton these methods, the authors employed vision models and language models and used white-box attacks and black-box attacks. The results showed that the proposed method HSPI-LD can figure out GPU types and data types.", "questions": "", "limitations": "", "rating": 2, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for offering valuable questions. We will address them one by one.\n\n> # Claims And Evidence\n> No, the proposed method is called \"hardware and software platform inference\", which I doubt is kind of big regarding the manuscripts, which is mostly focusing on GPU types and data types (like int8, fp16, etc.) ...\n\n- Our method does go beyond GPUs and data formats. In Tab 2 and Tab 6, we show that **different kernel implementations, runtime libraries, and parallelism strategies**, each produce unique floating-point “fingerprints”. As stated in line 420 page 8 and Tab 10, in practice, we can merge labels and focus on exactly the configurations that we care about—making the approach scalable to new hardware and software stacks.\n- Please refer to our answer to Q3 for details.\n\n> # Q1. \n> Figure 5, different logits showed similar results, what do the authors mean \"Figure 5 illustrates the kernel density estimation of various quantization methods where the shapes imply the obvious differences\"?\n\nPlease zoom in Fig.5. You may notice there are several places where the kernel densities of different data types are clearly not overlapped. For example, the density value at logit value = -1.3 and 0.65, and the whole INT8 curve. We include Fig.5 to help readers intuitively understand how SVMs capture differences in logits distribution.\n\n> # Q2.\n> Figure 6, what does it imply?\n\nFigure 6 visualizes the difference in logit bit distribution between RTXA6000 and A100. We send identical inputs to the same FP16 model checkpoint deployed on RTX6000 (orange) and A100 (blue), and collect outputs. Then we pick the first eight FP32 logits of each output sample (8*32=256 bits per output sample), plot a histogram of the bit counts (the number of ones at that bit) of these 256 bits over all output samples, cut off the minimum bit count of RTX6000 and A100 for each bit, to only show the difference. This implies though we send the same inputs to the same model checkpoints, the bit distribution of outputs from the two GPUs are significantly different even visible to human eyes. We include Fig.6 to help readers intuitively understand why HSPI-LD is possible.\n\n> # Q3.\n> I was wondering about the scalability of the proposed methods. Will they be easily applied to other hardware configurations and software stacks (like different machine learning compilers)?\n\nYes, we answer this question from three aspects:\n- Hardware configurations\n  - Yes, as we show in Tab R1, HSPI can be easily applies to AMD’s CDNA3 architectures and even **ASIC accelerators** like Amazon’s Inferentia.\n  - In the paper and Tab R2, we already show HSPI works across various GPUs like H100, A100, RTX6000, L40S,  L40, A40, RTX8000, RTX2080Ti, etc.\n  - We mainly run on NVIDIA GPUs in the manuscript because setting up the experiment and running experiments is time consuming.\n- Software stacks\n  - We include results **with and without ML compiler** in the manuscript. Tab 2 runs the LLM inference in the eager mode, while Tab 6 applies a series of optimizations including TorchInductor (torch.compile), cuda graph, etc.\n  - In Tab.R2, we additionally show HSPI can differentiate **different ML compilers (TenorRT-LLM vs TorchInductor)**.\n- Scale up\n  - In Tab.6, we run LLM serving engine with torch.compile, cuda graph, RadixAttention, dynamic batching enabled. HSPI still differentiate different GPUs, even kernel implementations, data types, parallelism strategies.\n  - In Tab.R1, we additionally show HSPI still work after we scale up the setup to **8-GPU system + LLM serving engine + 70B parameter model**. \t\nWe know it is not possible to provide results of all combinations covering all hardwares, but we still tried our best to include as various platform setup as possible.\n\nRTab.1 : HSPI-LD results for Amazon, AMD, NVIDIA (Llama-3.1-70B-it)\t\t\t\t\n|**Vendor**|**Hardware**|**Software**|**DType**|**Accuracy**|**F1**|\n|-|:-:|:-:|:-:|:-:|:-:|\n|Amazon|Inferentia|NKI|BF16|0.968|0.924|\n||||FP16|0.988|0.908|\n|AMD|MI300X|ROCm|BF16|1.0|1.0 |\n|||| FP16 |0.906|0.913|\n|NVIDIA|H100|CUDA|BF16|1.0| 1.0 |\n||||FP16|0.922|0.915|\n||||Avg|0.964|0.943|\n\nRTab2: HSPI-LD results for large systems (Llama-3.1-70B-it)\t\t\t\t\n|**Vendor**|**Hardware (Arch)**|**Num devices**|**DP,TP**|**DType**|**Accuracy**|**F1**|\n|--|:---:|:---:|:---:|:---:|:---:|:---:|\n|AMD| MI300X (CDNA3)|2|DP1TP2|BF16|1.0 | 1.0 |\n|||||FP16|0.961|0.957|\n|NVIDIA |H100 (Hooper) | 4 | DP2TP2 | BF16 | 1.0 | 1.0 |\n|||||FP16|0.930| 0.941 |\n||L40 (Ada)| 8 |DP1TP8 | BF16 | 1.0 | 1.0 |\n|||||FP16 |0.969| 0.958 |\n||A40 (Ampere)|8| DP1TP8 | BF16 | 1.0 | 1.0 |\n|||||FP16|0.922| 0.925 |\n|||||Avg|0.973| 0.973 |\n\nRTab.3: HSPI-LD results for ML compilers (Llama-3.1-70B-it)\t\t\t\t\n| **GPU**|**ML Compiler**|**DTypes**|**Accuracy**|**F1**|\n|-|:-:|:-:|:-:|:-:|\n|H100 |TensorRT-LLM | BF16 | 0.984 | 0.927 |\n||| FP16 | 0.947 | 0.944 |\n|| TorchInductor | BF16 | 1.0 | 1.0 |\n||| FP16 | 0.930 | 0.941 |\n||| Avg | 0.965 | 0.953 |\n\nPlease let us know if you have further questions :)", "claims_and_evidence": "No, the proposed method is called \"hardware and software platform inference\", which I doubt is kind of big regarding the manuscripts, which is mostly focusing on GPU types and data types (like int8, fp16, etc.)\nAlso, I was wondering about the scalability of the proposed methods. Will they be easily applied to other hardware configurations and software stacks (like different machine learning compilers)?", "methods_and_evaluation_criteria": "Yes, most of them make sense. I feel several experiments' results are not fully and clearly discussed.\nFigure 5, different logits showed similar results, what do the authors mean \"Figure 5 illustrates the kernel density estimation of various quantization methods where the shapes imply the obvious differences\"?\nFigure 6, what does it imply?", "theoretical_claims": "n/a", "experimental_designs_or_analyses": "Yes, I feel several experiments' results are not fully and clearly discussed.\nFigure 5, different logits showed similar results, what do the authors mean \"Figure 5 illustrates the kernel density estimation of various quantization methods where the shapes imply the obvious differences\"?\nFigure 6, what does it imply?", "relation_to_broader_scientific_literature": "n/a", "essential_references_not_discussed": "n/a", "other_strengths_and_weaknesses": "n/a", "other_comments_or_suggestions": "n/a", "questions_for_authors": "1. Figure 5, different logits showed similar results, what do the authors mean \"Figure 5 illustrates the kernel density estimation of various quantization methods where the shapes imply the obvious differences\"?\n2. Figure 6, what does it imply?\n3. I was wondering about the scalability of the proposed methods. Will they be easily applied to other hardware configurations and software stacks (like different machine learning compilers)?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: **Meta-Review of “Hardware and Software Platform Inference”**

This submission tackles the problem of verifying the actual hardware and software stack used to serve a machine learning model by analyzing its input–output behavior. The core idea is that different GPU architectures, data types (precision/quantization), kernel implementations, and runtime optimizations lead to subtle but detectable numerical patterns in the model’s output. The authors introduce two methods:

1. **HSPI-BI (Border Inputs)**: Creates specialized inputs that lie at the decision boundary between two (or more) model configurations, thereby eliciting different predictions depending on the serving hardware/software.  
2. **HSPI-LD (Logits Distributions)**: Trains a classifier (e.g., an SVM) on the distribution of model logits, splitting and analyzing the floating-point bits or entire logit distributions to distinguish different hardware/software setups.

They evaluate both methods using vision models (e.g., ResNet, MobileNet) and language models (e.g., LLaMA variants, QWen, Phi). The experiments span “white-box” scenarios (where the model weights and architecture are known and can be directly instrumented) and “black-box” scenarios (where only the model outputs—particularly logits—are accessible). Their primary findings:

- **White-box**: HSPI-BI can reliably separate GPUs of distinct compute capabilities (e.g., A100 vs. A6000) or different quantization schemes (FP32, BF16, INT8, etc.) and often achieves near-perfect classification accuracy. HSPI-LD can also reach high accuracy (up to 100%) for distinguishing hardware platforms or software configurations, and it scales better than HSPI-BI for large language models.  
- **Black-box**: While performance is lower, the authors report that the approaches (particularly HSPI-LD) still achieve up to three times random-guess accuracy for certain configurations—demonstrating that even with minimal visibility into the model, hardware inference remains possible.  

---

## Strengths

1. **Hardware and Software Platform Inference (HSPI)**: 
   - Proposes important problem of verifying that an ML service is actually running on the promised hardware/software configuration. This is increasingly relevant in commercial and security contexts.
   - The authors discuss how their method can aid in verifying hardware compliance and security, aligning well with the ongoing push for more transparency and trust in AI supply chains.

2. **Algorithms**: Two complementary approaches (HSPI-BI and HSPI-LD) that exploit numerical discrepancies in floating-point arithmetic and quantization.  

3. **Extensive Experiments**: 
  - Systematic evaluations using computer vision and large language models, covering multiple GPUs (A100, L40, A6000, etc.), quantization levels, parallelization strategies, and ML compilers. The experimental evidence demonstrates that HSPI can succeed across diverse settings.
  - In white-box mode, near-perfect accuracy is often achieved (≥ 99% in many cases). Even in black-box mode, they show that accuracy can substantially exceed random guessing.

---

## Weaknesses

1. **Limited Black-Box Robustness**  
   - While the black-box results are promising relative to random guess, they remain significantly lower than the near-perfect rates in white-box settings. The method can require collecting many queries, and performance might degrade if providers introduce noise or limit logit access.

2. **Identical or Very Similar GPU Architectures**  
   - If two platforms reside effectively in the same “equivalence class” (e.g., nearly identical compute capabilities), HSPI may fail to differentiate them. The paper acknowledges this (e.g., RTX 8000 vs. RTX 2080 Ti) as a practical limitation.

3. **Scalability**  
   - Generating border inputs (HSPI-BI) for larger LLMs can become GPU-memory-intensive due to the need to store full training activations and run backward passes. HSPI-LD is more scalable but requires collecting and classifying logits, which can still impose overhead or costs in pay-per-token settings.

4. **Comparison with Broader Side-Channel Literature**  
   - Although the paper references prior hardware fingerprinting and side-channel work, reviewers noted that more in-depth contrast with existing CPU/GPU side-channel attacks (or IP reverse-engineering approaches) could strengthen the paper.

---

## Recommendation

Based on the paper's contributions I recommend accepting this paper to ICML.
Decision: Accept (poster)