On Nullspace of Vision Transformers and What Does it Tell Us?
Authors: Aditya Singh
Haohan Wang
Venue: Submitted to ICLR 2023 (2023)
Keywords: nullspace
vision transformers
robustness
watermarking
fooling interpretations
fooling models

Abstract: Nullspace of a linear mapping is the subspace which is mapped to the zero vector. For a linear map, adding an element of the nullspace to its input has no effect on the output of the mapping. We position this work as an exposition towards answering one simple question, ``Does a vision transformer have a non-trivial nullspace?" If TRUE, this would imply that adding elements from this non-trivial nullspace to an input will have no effect on the output of the network. This finding can eventually lead us closer to understanding the generalization properties of vision transformers. In this paper, we first demonstrate that provably a non-trivial nullspace exists for a particular class of vision transformers. This proof is drawn by simply computing the nullspace of the patch embedding matrices. We extend this idea to the non-linear layers of the vision transformer and show that it is possible to learn a non-linear counterpart to the nullspace via simple optimisations for any vision transformer. Subsequently, we perform studies to understand robustness properties of ViTs under nullspace noise. Under robustness, we investigate prediction stability, and (network and interpretation) fooling properties of the noise. Lastly, we provide image watermarking as an application of nullspace noise.

Under review as a conference paper at ICLR 2023
NULLSPACE OF VISION TRANSFORMERS AND WHAT DOES IT TELL Us?
Anonymous authors Paper under double-blind review
ABSTRACT
Nullspace of a linear mapping is the subspace which is mapped to the zero vector. For a linear map, adding an element of the nullspace its input has no effect on the output of the mapping. We position this work as an exposition towards answer- ing one simple question, "Does vision transformer have non-trivial nullspace?" If TRUE, this would imply that adding elements from this non-trivial nullspace to an input will have no effect on the output of the network. This finding can eventu- ally lead closer to understanding the generalization properties of vision trans- formers. this paper, we first demonstrate that provably non-trivial nullspace exists for particular class of vision transformers. This proof is drawn by simply computing the nullspace of the patch embedding matrices. We extend this idea to the non-linear layers of the vision transformer and show that it is possible learn a non-linear counterpart to the nullspace via simple optimisations for any vision transformer. Subsequently, we perform studies understand robustness proper- ties of ViTs under nullspace noise. Under robustness, we investigate prediction stability, and fooling properties (network and interpretation) of the noise. Lastly, we provide image watermarking as an application of nullspace noise.
INTRODUCTION
In recent years, deep learning models have seen tremendous success. So much so, that now these models are the standards for tasks ranging from image recognition (He et al.) 2016: Dosovitskiy et al. 2021 Wortsman et al. 2022), object detection (Dai et al. al Guan et al. 2022). scene segmentation (Fang et al. 2021 Zhang et al. 20226), language translation (Devlin et al. 2019 Xue et al. 2022), speech recognition (Nagrani et al. 2021 Chen et al.) among many others. We are now observing their application to various novel problems such as predicting protein structures (Jumper ct al., 2021), generative modelling (Dhariwal & Nichol, 2021), autonomous driving (Grigorescu et al. 2020), solving differential equations (Lample & Charton 2020) to name a few. It is only fair to assume that we shall be witnessing accelerated adoption of deep learning models into our daily lives as the years go by.
In computer vision, most of the architecture types can be broadly classified into groups: volution neural networks (CNNs) and transformers based on their building blocks. CNNs have gained overwhelming popularity since their state of the art redefining performance (Ciregan al., 2012 Krizhevsky et al. 2012) on the ImageNet challenge (Russakovsky et al., 2015). The main characteristic of CNNs the use of trainable kernels perform convolution operation in a strided fashion on the inputs (LeCun et al. 1989; O Shea & Nash 2015). On the contrary, transformer based architectures like Vision Transformers (VYT') (Dosovitskiy et al. (02T) are more recent ventions compared to the CNNs. Key aspect of transformer based architectures is the adaptation and utilisation of selffcroos-attention modules (Vaswani al., 2017).
In the short span of three years, transformer based classes of vision models have gained tremendous popularity, They compete with CNNs on various computer vision tasks TTourron et al., 2021 Car- ion et al. 2020 Arnb et al., 2021 The focus of our work is specific architecture, the Vision Transformer. It processes an input image by first splitting it into several non-overlapping patches followed by linear projection which is then processed by vanilla transformer. Since their intro- duction in 2020, ViT's have been the source of inspiration for several recent novel architectures (Ali let al.
2021
Li al. 2022b
Liu et al. 2021). Also, researchers have made multitude of recent

---

Under review as a conference paper at ICLR 2023
output logits
Vision Transformer
input image
fO
f(
output logits
100)= B.r + +2aa= =y
Vision Transformer
/()=
NA
if x' x'= +1 P
4n EπEB B
Bg
input Image perturbed through nullspace for arbitrary
Figure I: An illustration of the nullspace in three cases (projection function case, left top; linear function ease, left bottom; vision transformer case, right). For functions in these three cases, there exists some nullspace, and the output of the function with respect to the input will remain the same no matter how much perturbation introduced to the input along the nullspace. Also, the nullspace is sunction-specific (model-specific) and will not vary for different samples.
discoveries about the properties of transformer based architectures with ViTs as the focal point of their studies (Naseer al. 2021: Mahmood al., 2021; Minderer et al. 202I). This decision to prioritize ViTs stems from the simplicity ViTs resulting in fewer engineering hurdles and high flexibility and versatility which leads to wider adoption across domains.
Despite the architecture is still relevatively new, the community has made several insightful findings along the investigation of the working mechanisms of it. For example, |aseer et al. (02TT) showed that ViTs are robust to occlusions, input perturbations and domain shifts, as well as bearing a lower reliance on local textures compared that of CNNs showed earlier by Geirhos et al. (2018). These findings were also corroborated in a recent work (Zhang et al., 2022a). Further, Zhou et al. (2022) verified that the robustness of ViTs to several corruptions is primarily due to the self-attention blocks. However, a recent finding by Pinto et al. (202I) attributes relatively better performance of VITs on out-of-distribution generalisation of parameters. On related topic, along the study of adversarial robustness, Mahmood et al. (202T) found that, unlike CNNs, adversarial examples for VITs had lower transferability across architec- tures, which inspired them to build more robust ensemble. On the other hand, as per [Bai et al. (02TT, the improved generalisation of ViTs is benefited by the self-attention framework employed by the network. They also reported ViTs are vulnerable adversarial attacks (Goodfellow al. et al. 2016) as their CNN counterparts under a fair evaluation. Moreover, they found no evidence suggesting that larger datasets helped ViTs more than CNNs in improving generalisation. These two findings are in conflict with earlier works inspecting ViTs (Bhojanapalii et al. 2021 Shao et al. (222T) On the aspect of neural network calibration (Guo et al 2017). Min- derer et al. [2021) observed that recent architectures, the likes of ViTs, were much better calibrated their prediction scores. Finally, the architectural differences between CNNs and ViTs naturally result in the differences of the inductive biases (Raghu et al., 202T), and, in comparison to CNNs, VITs lack inductive biases for local structure (edges, corners). To remedy this, Dai et al. (2021b); Xu et al. (2021): Li et al. (2022a) proposed solutions resort to either large amount of training data or inclusion of convolution layers.
flawed comparison of models solely based on number
2015 Moosavi-Dezfoon
Though no single architecture type comes off as the clear winner, it is fair to believe that ViTs are in general the primary focus of the community when studying transformer architectures vision. Sharing the motivation to explore and further in this paper, we aim to highlight their untouched aspect: the nullspace of vision transformers. Nullspace of a linear map f X + is the subspace of X' which is mapped to O. Formally, A fx 0}.
understanding of ViTs akin to previous works,
Why is nullspace important? A seemingly innocent concept of nullspace describes very inter- esting property of ViTs. If such a subspace exists for ViTs, then it would imply that the network is inherently robust certain types of input perturbations (or corruptions). That is, simply sampling
2

---

Under review as a conference paper at ICLR 2023
perturbation from this nullspace and adding to the input will have effect on the output of the network. Moreover, in theory. we can synthesise infinite such perturbations by simply varying the weights for the basis vectors of this subspace. Figure ! provides pictorial representation of the concept of nullspace for different mathematical formulations.
We begin the exploratory study of nullspace of VITs by first providing formal introduction to nullspace section [2] In section we show that nullspace exists for certain vision transformers. We introduce approximate nullspace noise for non-linear components of ViTs in section 4 We revisit the idea of robustness in section [3] and evaluate ViTs and CNNs on different nullspace noises. In section 6 as an interesting application nullspace, we explore the utility of nullspace perturbations for image watermarking. Extending the formulation of image watermarking, we show that it is possible to generate samples which can fool the model and several interpretability methods (section Z.. In section 8 we discuss related works in the field of neural networks and nullspace. Lastly, we discuss the implications our findings (section 9 and conclude our work (section TOp.
2 PRIMER: NULLSPACE
Given input Ixp and model vector B Rexk, build simple regression model
y x
where we call y the responses and y R 1xk Interestingly, given a fixed 3, we can find vectors A, such that
u 0, Vu A
In that case, for given model with 8, the output of x and A will be exactly the same. A is called the nullspace of B. For linear mapping such as B, where the domain is a vector space, nullspace a subspace and satisfies all the required axioms:
Zero element of IR Ixp e A. This the true as 0,3 0.
closed under vector addition. u+v E A Vu,v A. This true as (u+v) us yo= 0. u
Closed under multiplication with scalar. cu E A V R and u E A. This is true as (cu) = c(u3) O.
A trivial nullspace, A (0). always exists for linear mapping described in equation 1 An alternate way is to interpret nullspace is to view it as set of solutions to system of linear equations as described by equation [ This imples that 0 is always solution the said equation. As the number of solutions to a system of linear equations can vary, the nullspace for a mapping can be trivial, non- ooiviaivial, or does not exist.
How to solve for A? Given matrix, finding its nullspace has now become an ubiquitous exercise mathematical textbooks. We refer the readers to excellent resources on this topic (Kwak Hong 2004 Sttragg] (009b]a) a). In our work, we rely on the Python package of Numpy for finding the nullspace of any given matrix (Harris al. 2020p.
3 EXACT NULLSPACE
In this section, first review the working of vision transformer. Following it, demonstrate the existence of nullspace for ViTs.
3.1 VISION TRANSFORMER
Vision transformer introduced by Dosovitskiy et al. (2021) is function fa with the trainable weights. The function takes as input an image x E Texixw and outputs classification response E y over & categories. is the number of channels (typically 3 for red, green, and blue). h, w correspond to height and width the input image. This neural network function can be broken down into composition of 3 stages, namely:
3

---

Under review as a conference paper at ICLR 2023
patch embedding stage, fo &exxxx - Ut. This steps projects the input image patch of pre- determined dimensions c, r and r to a one-dimensional embedding of length d. It is ensured that patches have overlaps between them. Hence, the number of such non-overlapping patches generated from the input image are m p
Ax
self-attention stage, fo Um + p(m+t)xd In the next step, the generated patch em- beddings are passed through layers of self-attention modules to process long range interactions amongst them. Apart from the m patch embeddings an additional embedding in form of cls token is utilised in this step.
classification stage, fo Vd + y* The final step is to perform the k-way classification. For this, we simply keep the processed encoding corresponding to cls token and project through linear classification layer.
This breakdown can also be illustrated as:
U;
[uels Ug ... Um Vols ... V.n
Vels
[y031 ... ykk (3)
3.2 NULLSPACE IN TRANSFORMER
Now consider the first layer of the transformers i.e. the patch embedding layer, fo. We consider with d kemnels, and each has r x r weights. Then for a input patch x E R xxrxr, we have the representation (embeddings) learned from that patch as
y x0.
As per the discussion in section 2 we can find 4s, the nullspace for fa. Let Bg = (b1, by. bu} be the d basis vectors for this nullspace. As per the axioms described section we can sample an element from Ao as:
V9 A1b dz
+ Naba-
This would imply that the output of the patch embedding will effectively remain preserved, f(x + vo) fx). Since the output after the first layer remains unaffected the final output of the classification remains unchanged. In this manner, nullspace of patch embedding also serves as subset of the nullspace of the vision transformer.
We refer to Va as nullspace noise or nullspace perturbation. important to note that nullspace noise only depends on the patch embeddings weights and is independent of the input. As result, the same noise can be added to any input without impacting the final output.
Does nullspace always exist for ViTs? Specific values of the weights determine whether a non- trivial nullspace exists or not, hence, we require a case by case analysis of ViT architectures. How- as general rule of thumb, ViT configurations with d < r² will always have a non-trivial nullspace (as per the rank-nullity theoren Experimentally, we verified that for ImageNet (Rus- sakovsky al. 2015) pretrained ViTs, standard configurations of ViT-Large, ViT-Base, ViT-Small with patch sizes of 32 x 32 have a non-trivial nullspace. As the embedding dimensions for these networks are ordered as dymull < dnuse charges per the rank-nullity theorem for the patch size 1024 we get the dimensions of the nullspace in the order A small A8,basc A large The ordering is corroborated by the experimental results.
APPROXIMATE NULLSPACE NOISE
In the previous section we have demonstrated that non-trivial nullspace exists for the patch embed- ding and hence the entire vision transformer, in this section we move further down the structure of ViT and investigate if nullspace exists for the self-attention stage, and thus whether we can enlarge the nullspace set of the entire vision transformer. To recall, self-atrention stage applies series of QKV attention operations followed by normalisation and non-linear transformations. The overall operation is thus non-linear which implies the concept of nullspace for fo is ill-defined. This is because even such a set exists where fo(v) = 0, Vv Ag. this set is no longer guarantced to
nuank- theorem asserts that the sum of rank of a matrix and dimensions of its nullspace should be equal the dimensions of the said matrix.
4

---

Under review as a conference paper at ICLR 2023
be closed under vector addition and scalar multiplication. Hence, we search for alternative to nullspace in the non-linear case.
Regardless, we still attempt to preserve the axiom of most interest us, closeness under vec- for addition. We attempt this through formulating the concept into adding arbitrary noise without disturbing the output of function. Therefore, instead of looking for vector space, can instead search for a set with the following property:
AS = [v]s(u+ + su) Vu E
(5)
i.e. adding elements from Dg to the input of fo has no impact on the output of the attention modules.
However, analytically finding Ag is still challenging. Thus, we seek for a feasible alternative: We further relax the searching criteria from a set to an individual element of this set, Va. Vá- is perturbation, when introduced to any input of fo. does not impact its output. Therefore, we can turn the searching for Ve into the following optimisation process that searches for v as an approximation of Vg: == 1I- vo) for (o (fa(())))
where, indexing 0 highlights the use of cls token in the loss function, and the normalisation can be any Ep norm. The above loss function tries minimize the lp norm between the pre- dicted logits with and without Vg. Alongside the self-attention stage we have also incorporated classification stage into the loss, since our the target of our study to minimize the im- pact on the final output of the network. To minimise the loss, we use a simple gradient de- scent based procedure to minimise the loss. A PyTorch styled pseudo-code along with the im- plementation details are in the appendix (section A.I). We initialise the V as noise sam- pled from Uni fonifoomm-mmm lim) for the minimisation of loss (also referred to as sampling limit). To quantitatively evaluate learnt Va. in figure 2 we report the percentage of matching classifi- cations with and without learnt nullspace noise along with mean squared error computed at the output logits. We consider prediction to be matched if the assigned category for an input the same with and without adding the per- turbation. The results indicate that increasing lim decreases the percentage of matching pre- dictions and increases the mse. This is expected as the optimisation converges towards a local minima with relatively larger noise values. We also show that the learnt values of V is impor- tant by also reporting the results after shuffling the noise values prior to adding it to the input (depicted with ---). Shuffling the values indi- cate that the learnt values of V. are meaningful and the noise does not just comprise of negli- gible values. As the dataset, we have used Im- ageNette (Howard) (018)) which is subset of ImageNet dataset consisting of 10 categories.
Vit-S
Vit-B
Vit-S
Vit-B
100.0
7000.0
80.0
6000.0
% match predictions
5000.0
mean squared error
4000.0
40.0
2000.0
1000.0
0.0
0.0
0.0
0.25
0.5
0.75
1.0
1.25
1.5
1.75
2.0
sampling limit
Figure 2: V. evaluation.
We learnt V on the training dataset ( 9500 images) and performed evaluation on the validation
set (Ne 4000 images).
5 STUDY: UNDERSTANDING ROBUSTNESS WITH NULLSPACE NOISE
5.1 BACKGROUND
Deep neural networks tend to be fragile. It is widely known that simple perturbations to the in- put can result unintended outputs. If these perturbations are generated adversarially, the output of a network misclassifies the sample even if visibly nothing much has changed in the input image
Thooug Vo is an approximation of the nullspace noise, we refer to it as nullspace noise in our work and clearly highlight the demarcation where required.
5

---

Under review as a conference paper at ICLR 2023
1935
11.5
P43.5
(a) D3
(b) Vu
(c) Du ViT-S
Figure 3: Performance under different noises. Figures (a), (b) show the performance degradation under approximated noise learnt encoder and input levels. For both, the noise computation is same for CNNs. In figure (c), show the performance under exact nullspace noise from VIT-S applied to other networks.
Goodfellow et al, 2013 Nguyen et al. 2013 Deessvii-eezooieee et al. 2016 Tlyas et al. 2019). Ma-
jority of adversarial attacks generated per input sample, i.e. given input image the adversarial noise is synthesised curated for the input sample. There are also approaches which have attempted overcome this limitation where the adversarial noise learnt per model (Moosavi-Dezfoolli al.) 2017 Liu et al., 2017 Brown et al.] 2017). A key difference between adversarial perturbations and nullspace noise that with the addition of the input-independent nullspace noise, we aim to maintain the final prediction of the model.
Departing from learning specific input perturbations, adding noise in form of 'corruptions' has also been theme for testing robustness under data distribution shifts (Hendrycks & Dietterich] 2018 Geirhos al. 2018 Recht al., 2019. Wang et al.) 2019: Hendryeks et al. 2221a These eval- uations tend to highlight lack of generalisation prowess in deep neural networks as the testing data diverges slightly (or at times heavily) from the distribution of training data. Many works have bench- marked and attempted to mitigate the deterioration in performance under such shifts (Taori et al. 2020). As opposed to nullspace noise, these corruptions are crafted manually to bring forth a poten- tial pitfall of the model. Hence, particular corruption can arbitrarily fail succeed any given image.
In this section, we aim understand robustness of vision transformers focused at its nullspaces. We pose robustness w.r.t nullspace as the question "How robust are different architectures to nullspace noises.".
5.2 EXPERIMENT: NOISE ROBUSTNESS
For vision transformers, we have computed two types of noises, Vo and V To further demonstrate the robustness advantages of VIT, we also need to calculate the nullspace of CNNs for comparison. However, opposed to ViTs, CNNs uses overlapping strided convolutions instead of patch embedding. Application of multiple kernels overlapping input patches makes it intractable to determine if nullspace for CNN exists at the very first layer, since any nullspace, if exists, must be shared by multiple kernels. The candidate pool is further constrained by strided convolutions. Therefore, instead analytically solving for the nullspace, we have to approximate it as we did for the attention stage of ViT. For comparison, mainly perform the following experiments:
Compare performance of different architectures under approximate nullspace noise for encoders. This will help us contrast the goodness of nullspace noise of ViT's to CNNs.
Study the impact of transfer of noise from ViT to other ViT and CNN models. This will allow us to understand the extent to which different architectures behave under noise transfer.
5.3 RESULTS
In figure 3 a), we compare the performance of ViTs under noise learnt at fo level. In comparison, for CNNs the noise is learnt for the entire network. Under similar sampling limit we measure the performance variations due to non-linear encoders for different architectures. We observe that compared to R50 (ResNet-50), Eff-b0 (Efficientnet-bll (Tan Le 2019)) and Mr3-SSMMobilentV's Small) (Howard et al. 2019) ViT's perform better under throughout different sampling limit. For

---

Under review as a conference paper at ICLR 2023
ICLR (a) Watermark
(b) Original Images
(c) Watermarked images
(d) Nullspace noise
Figure 4: Watermark superposition using nullspace basis vectors. Figures (a) and (b) correspond the watermark image and the original images. The watermarked images in (c) are generated using the nullspace basis vectors, thus images in (c) will trigger the model for the same set of outputs as in (b). The additive nullspace noises (d) show the overlaid watermark content. Best viewed digitally.
CNNs, Eff-bO performs better at low limit (0.01, 0.05) but the performance drastically depletes higher sampling limit. Poor performance of CNNs compared to ViTs' non-linear counterpart supports our hypothesis that nullspace noise for CNNs is hard to construct.
In figure b)) we learn V. for the entire network from the input level (w represents the model parameters). In this manner, we perform a fair evaluation at the network level. We see similar trend to what we observed for figure a)). For ViT', we observe that for noise learnt the first layer results in relatively worse performance at larger sampling limits than for V Overall, from figures (a,b) we can conclude that ViTs are more robust to approximate nullspace noises compared to their CNN counterparts.
In figure ( we report results of VIT-S' V., being applied to the inputs for ViT-Base and CNNs. VIT-S naturally attains the best performance at all levels. ViT-B responds better than CNNs to the transferred noise and performs better than random choice. It can be attributed to architectural similarities between the two VITs. For the CNNs, record significantly lower performance. Lack of effective nullspace noise can play key role here. Overall, we observe very low transferability of noise between architectures.
APPLICATION: OUTPUT-PRESERVED IMAGE WATERMARKING
Watermarking image, usually used to convey ownership information verify content of the data, has been studied extensively (Wolfgang Delp. 1996. Potdar et al. 2005 Al- 2007: Bhat ct al. 2010). A watermark can be either imperceptible or perceptible. and perceptible watermarking applics noticeable marker to convey the protected nature the data (Berghel, [198).. In this section, explore utilize nullspace noise to apply perceptible watermark the image which does not alter the test-time forward process.
Figure Siiiee/showe shows an example watermarking approach using the nullspace noise. Here, watermark source image with a watermark. The resulting modified image, figure (c), attains the final predictions close to the original image. We observe instances of the text appearing in the modified image.

---

Under review as a conference paper at ICLR 2023
(a) Triplet of Source, target and transformed images
(b) Saliency maps for the corresponding images from the row above.
Figure 5: Targeted nullspace noise. Transformed images appear visually as target images but are interpreted as source images by the model. The equivalence between source and transformed images not only in terms the final predictions but also in the interpretability maps depicted in (b).
Method details: With basis vectors of the nullspace, on the original image without affecting the output of the network. Given a source (user's image) and target image (watermark),
construct watermark to be overlaid
simply need to estimate the scalar parameters corresponding the basis vectors to satisfy Eiso e, Va x Axy. e, are the basis vectors for the nullspace, A are their corresponding scalar co-efficients which are to be determined and Ax, is the changed required to convert j"h original image patch to jth watermark image patch. This can be achieved through constrained optimisation of the following form:
min Axy
Σ
eddille subject to 0 02 Σ e, 1.
The constraints restrict the pixel values the altered image to be within [0, 1]. The optimisation can be solved using out of the box Sequential Least Squares Programming optimizer (Nocedal Wrightt 2006). In the appendix A.2))
we report the quantitative results of the network on watermarked
images.
STUDY: UNCONSTRAINED TARGETED NOISE
Using watermarking allows to impose information about the watermark image any source image. However, the output will still be slightly altered potentially due to (i) limited expressiveness imposed due to the nullspace (ii) the constraint within In this section, remove the constraint and perform a study to understand the impact on model. By this relaxation, we will show that is possible to fool both the model and interpretation methods.
Similar to the study of adversarial robustness, we consider a model to be fooled if the predicted category differs for an input before and after application of noise. In figure we display the qual- itative results of synthesising a transformed image using targeted Vo operation. The transformation not perfect, however, we can spot that the transformed images are visually similar to target images than source images. Even with this difference in the input space, transformed images and source images are classified into the same category with roughly the same confidence. We deflect the exact details of the optimisation process to the appendix (see section A.3).
As recent studies have shown, fooling can also be extended to the interpretability methods (XAI) Dombrowski et (2019) partially due the limitations exposed by recent studies (Dombrowski et al. 2019 Ghorbani et al. 2019 Heo et al. (019).. However, in contrast to these works aiming to fool specific XAI method, our nullspace noise only depends on the model, not the XAI method.
In figure b],, we show the interpretability maps as generated by LRP (Chefer et al. 2021). From the figure, observe that the heatmaps generated by source and transformed images are iden-
8

---

Under review as a conference paper at ICLR 2023
tical whereas, the transformed image heatmaps substantially differ from target images'. Though only reported for LRP, we observed that a similar observation holds across different interpretability approaches. Here, only presented the results LRP, as in the context of ViTs, we found the heatmaps from other methods to be lacking (also pointed out by authors of LRP). Visualisations on other interpretability methods provided in the appendix (see section A.4.
RELATED WORK
In this section, we keep the discussion limited to related works combining neural networks and nullspaces. To the best of our knowledge, the earliest work investigating neural networks alongside nullspaces corresponds to the study by Goggin al. (1992). They studied the universal approxima- tion capabilities of multi-layered perceptrons by comparing the outputs and nullspace of inputs. Through a classical example of learning XOR they showed that with the use of hidden layers, an MLP is able to construct a transformation which maps input targets to be in the nullspace of the input. In a much recent work, Sonoda et al. (2021) mathematically specified the behavior of nullspace, but only for fully connected networks. On the application side, for a continual learning setting, Wang et al. (202I) proposed to map new tasks to the nullspace of the existing tasks. NuSA (Cook et al. 2020) analysis of a linear layer in a neural network. The idea is to monitor projection magnitude on the nullspace during test and compare it with threshold. Lastly as a novel architecture, /Abdelpakey & Shehata (2020) proposed NullSpaceNet which maps inputs from the same category to joint- nullspace instead of feature space.
if the target happens
Iramework for outlier detection using the nullspace
DISCUSSION
Before we conclude, we would like to devote a short section for the discussion of the limitation of findings and other potential technical implications.
Societal Impact Although the name nullspace might imply a negative property, we notice the most practical implication of nullspace is to offer explanations to VIT's additional resilience toward minor noises in comparison to CNNs (Section 5). Our results in Section 7 might also be interpreted as a threat, but the perturbations are barely powerful enough to fool the models within the constraint of typical pixel values. Thus, we do not expect our work to introduce any negative societal impacts.
Applications in Model Patenting In addition the applications we discussed, consider another potential usage of our findings is to patent ViT after a model is trained, as the nullspace will be unique property of any set of weights of certain ViT architectures. Different from the existing line of research in model watermarking (Adi et al., 2018 Darvish Rouhani et al. [019 Le Merrer et al.) 2020), the patenting through nullspace will not require any additional steps during training, although
will face limited usage scenarios in comparison.
Potential Limitation about the Nullspace Approximation Different from the nullspace defined in linear algebra, the nullspace of the entire VIT can only be approximated because of the non-linearity in the network architecture. However, it is worthy mentioning that we can still calculate the exact nullspace of ViT if only consider the patch embedding layer, through which, results will qualitatively deliver the same message, but with quantitative differences.
10 CONCLUSION
In this work, we have explored the concept of nullspaces for vision transformers. By only utilising the linear projection component, we showed that non-trivial nullspaces exist for various configura- tions of ViTs. Subsequently, we extended this idea to the non-linear component of vision trans- former and demonstrated a simple method of generating approximate nullspace noise. By comparing ViTs with contemporary CNNs, we showed that ViTs are fairly robust to high amount of noise gen- erated via nullspace. From an application perspective we demonstrated that nullspace noises can be readily utilised to impose perceptible watermarks images without altering the final predictions casted by a network. Lastly, we demonstrate that under an unconstrained setting, nullspace noises can be easily wielded to fool both the network and interpretability methods.

---

Under review as a conference paper at ICLR 2023
REFERENCES
Mohamed H. Abdelpakey and Mohamed S. Shehata. Nullspacenet: Nullspace convoluional neu- ral network with differentiable loss function. CoRR, abs/2004. 12058, 2020. URL https:
7 /arxiv.orgg////... www.oogggabbaaaoooooo..o..e..a 10000
Samira Abnar and Willem Zuidema. Quantifying attention flow in transformers. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguisties, pp. 4190-4197, Online, July 2020. Association for Computational Linguistics. doi: 10. 1l1188611/22022aallll main. 385, URL
org. aan........m....333.33
Yossi Adi, Carsten Baum, Moustapha Cisse, Benny Pinkas, and Joseph Keshet. Turning your weak- ness into strength: Watermarking deep neural networks by backdooring. In 27th USENIX Secu- rity Symposium (USENIX Security 18), pp. 1615-1631, 2018.
Ali Al-Haj Combined dwt-det digital image watermarking. Journal of computer science, 3(9):
740-746, 2007.
Alaaeldin Ali, Hugo Touvron, Mathilde Caron, Piotr Bojanowski, Matthijs Douze, Armand Joulin, Ivan Laptev, Natalia Neverova, Gabriel Synnaeve, Jakob Verbeek, et al. Xcit: Cross-covariance image transformers. Advances in neural information processing systems, 34:20014-2006 20022,, 2021.
Anurag Arnab, Mostafa Dehghani, Georg Heigold, Chen Sun, Mario Lučić, and Cordelia Schmid. Vivit: A video vision transformer. In 202/ IEEE/CVF International Conference on Computer Vision (ICCV), pp. 6816-6826, 2021. doi: 10. N0.1109//CCV88922...000 000666.
Yutong Bai, Jieru Mei, Alan L Yuille, and Cihang Xie. Are transformers more robust than cnns? In M. Ranzato, A. Beygelzimer, Y. Dauphin, P.S. Liang, and J. Wortman Vaughan (eds.), Ad- vances in Neural Information Processing Systems, volume 34, pp. 26831 Curran Asso- ciates, Inc., 2021. URL https: httttps://pprcceednngg. htwpeediiinnnnneiiineee neur wttp://wwss..ppppppe//////////.e e 23334aalccc000037771333355533333
pdf
Hal Berghel. Digital watermarking makes its mark. Networker, 2(4):30-39, 1998.
Vivekananda Bhat, Indranil Sengupta, and Abhijit Das. An adaptive audio watermarking based on the singular value decomposition in the wavelet domain. Digital Signal Processing, 20(6): 1547-1558, 2010.
Srinadh Bhojanapalli, Ayan Chakrabarti, Daniel Glasner, Daliang Li, Thomas Unterthiner, and An- dreas Veit. Understanding robustness of transformers for image classification. In Proceedings of the IEEF/CVF International Conference on Computer Vision, pp. 10231-10241, 2021.
Tom B Brown, Dandelion Mané, Aurko Roy, Martín Abadi, and Justin Gilmer. Adversarial patch. arXiv preprint arXiv: 1w1.iv:112.09960 17966666,, 2017.
Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, and Sergey Zagoruyko. End-to-end object detection with transformers. In European conference computer vision, pp. 213-229. Springer, 2020.
Hila Chefer, Shir Gur, and Lior Wolf. Transformer interpretability beyond attention visualization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 782-791, June 2021.
Ke Chen, Xingjian Du, Bilei Zhu, Zejun Ma, Taylor Kirg-Kirkaatrick, and Shlomo Dubnov. Hts- at: hierarchical token-semantic audio transformer for sound classification and detection. In ICASSP 2022.
Dan Ciregan, Ueli Meier, and Jürgen Schmidhuber. Multi-column deep neural networks for image classification. In 2012 IEEE conference on computer vision and pattern recognition, pp. 3642- 3649. IEEE, 2012.
Matthew Cook, Alina Zare, and Paul D. Gader. Outlier detection through null space analysis of neural networks. CoRR, abs/2007.01 0.263,, 2020. URL hhttps: arxiv. httpaawwww..o..////a/aaaaaaaa//a absg///bb./00
012 63
10

---

Under review as a conference paper at ICLR 2023
Xiyang Dai, Yinpeng Chen, Bin Xiao, Dongdong Chen, Mengchen Liu, Lu Yuan, and Lei Zhang. Dynamic head: Unifying object detection heads with attentions. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition (CVPR), pp. 7373-7382, June 202
Zihang Dai, Hanxiao Liu, Quoe V Le, and Mingxing Tan. Coatnet: Marrying convolution and attention for all data sizes. Advances in Neural Information Processing Systems, 34:3965-3977, 2021b.
Bita Darvish Rouhani, Huili Chen, and Farinaz Koushanfar. Deepsigns: An end-to-end water- marking framework for ownership protection of deep neural networks. In Proceedings of the Twenty-Fourd International Conference on Architectural Support for Programming Languages and Operating Systems, pp. 485-497, 2019.
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Volume (Long and Short Papers), pp. 4171-4186, Minneapolis, Minnesota, June 2019. Association for Computational Linguistics. doi: 00.18653/v1/N19-11 1423. URL https:
7/ac antaano wtttchooogy.orr//..... wwwpoo/g//////o. 42.3
Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans on image synthesis. In M. Ranzato, A. Beygelzzmer, Y. Dauphin, P.S. Liang, and J. Wortman Vaughan (eds.), Ad- vances in Neural Information Processing Systems, volume 34, pp. 8780 8794. Curran Asso- ciates, Inc., 2021. URL attps: http:::::spppccccccss..ssc.e neurips cctpapper//2//E1ee/ 4 49ad2.3d' abd 7d0 Paaelcafaaaapppppee pdd
Ann-Kathrin Dombrowski, Maximillian Alber, Christopher Anders, Marcel Ackermann, Klaus- Robert Müller, and Pan Kessel. Explanations can manipulated and geometry to blame. Advances in Neural Information Processing Systems, 32, 2019.
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszko- reit, and Neil Houlsby. An image worth 16x 16 words: Transformers for image recognition scale. ICLR, 2021.
Yuxin Fang, Shusheng Yang. Xinggang Wang, Yu Li, Chen Fang, Ying Shan, Bin Feng, and Wenyu Liu. Instances as queries. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 1919, October 2021.
Robert Geirhos, Patricia Rubisch, Claudio Michaelis, Matthias Bethge, Felix A Wichmann, and Wieland Brendel. Imagenet-trained cnns are biased towards texture; increasing shape bias im- proves accuracy and robustness. In International Conference on Learning Representations, 2018.
Amirata Ghorbani, Abubakar Abid, and James Zou. Interpretation of neural networks is fragile. In Proceedings of the AAAI conference on artificial intelligence, volume 33, pp. 3681-3688, 2019.
Shelly D. D. Goggin, Karl E. Gustafson, and Kristina M. Johnson. Accessing the null space with nonlinear multilayer neural networks. In Dennis W. Ruck (ed.), Science of Artificial Neural Net- works, volume 1710, pp. 308 316. International Society for Optics and Photonics, SPIE, 1992.
doi: 10.1117/1..140097 URL https: httttttss///....oo http:::////oooooo///.//o 1ttp://0..111///11 140097
Ian Goodfellow. Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. In Yoshua Bengio and Yann LeCun (cds.). 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceed- ings, 2015. URL http: /tt///axiii wwwpxiv.oorg/gaao
141 12..22222 6572
Sorin Grigorescu, Bogdan Trasnea, Tiberiu Cocias, and Gigel Macesanu. A survey of deep learning techniques for autonomous driving. Journal of Field Robotics, 37(3):362-38) 2020.
Tianrui Guan, Jun Wang, Shiyi Lan, Rohan Chandra, Zuxuan Wu, Larry Davis, and Dinesh Manocha. M3detr: fulti-represennationn multi-scale, rutual-relation 3d object detection with transformers. In Proceedings of the IEEE/CVE Winter Conference on Applications of Computer Vision (WACV). pp. 772-782, January 2022.
11

---

Under review as a conference paper at ICLR 2023
Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q Weinberger. On calibration of modern neural networks. In International conference on machine learning, Pp. 1321-1330. PMLR, 2017.
Charles R. Harris, K. Jarrod Millman, Stéfan J. van der Walt, Ralf Gommers, Pauli Virtanen, David Coumaaeau,, Eric Wieser, Julian Taylor, Sebastian Berg, Nathaniel J. Smith, Robert Kern, Matti Picus, Stephan Hoyer, Marten H. van Kerkwijk, Matthew Brett, Allan Haldane, Jaime Fernández del Rio. Mark Wiebe, Pearu Peterson, Pierre
Kevin Sheppard, Tyler Reddy, Warren Weckesser, Hameer Abbasi, Christoph Gohlke, and Travis E. Oliphant. Ar- ray programming with NumPy. Nature, 585(7825):357-362, September 2020. doi: 10. 1031/ 41 5-0-5806---20
$41586-020-26499 URL htt.htts:// // http::::/ddo..oorgo/
Kaiming He, X. Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 770-778, 2016.
Dan Hendrycks and Thomas Dietterich. Benchmarking neural network robustness to common cor- ruptions and perturbations. In International Conference on Learning Representations, 2018.
Dan Hendrycks, Steven Basart, Norman Mu, Saurav Kadavath, Frank Wang. Evan Dorundo, Rahul Desai, Tyler Zhu, Samyak Parajuli, Mike Guo, Dawn Song. Jacob Steinhardt, and Justin Gilmer. The many faces of robustness: A critical analysis of distr--distriuution generalization. In Pro- ceedings of the IEEE/CVF International Conference Computer Vision (ICCV), pp. 8340-8349, October 2021a.
Dan Hendrycks, Kevin Zhao, Steven Basart, Jacob Steinhardt, and Dawn Song. Natural adver- sarial examples. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 15262-15271, 2021b.
Juyeon Hco, Sunghwan Joo, and Taesup Moon. Fooling neural network interpretations via adver- sarial model manipulation. Advances in Neural Information Processing Systems, 32, 2019.
Andrew Howard, Mark Sandler, Bo Chen, Weijun Wang. Liang-Chieh Chen, Mingxing Tan, Grace Chu, Vijay Vasudevan, Yukun Zhu, Ruoming Pang. Hartwig Adam, and Quoe Le. Searching for mobilenetv3, In 2019 IEEE/CVF International Conference on Computer Vision (ICCV), pp. doi: 10. 10.100//CCC22219.0000.4000
1314-1324, 2019,
Jeremy Howard. Imagenette, 2018. URL http://///ggttbb.ccfaaasss//////
Andrew Iyyas, Shibani Santurkar, Dimitris Tsipras, Logan Engstrom, Brandon Tran, and Aleksander Madry. Adversarial examples are not bugs, they are features. Advances in neural information processing systems, 32, 2019.
John Jumper, Richard Evans, Alexander Pritzel, Tim Green, Michael Figurnov, Olaf Ronneberger, Kathryn Tunyasuvunakool, Russ Bates, Augustin Zidek, Anna Potapenko, Alex Bridgland, Clemens Meyer, Simon A. A. Kohl, Andrew J. Ballard, Andrew Cowie, Bernardino Romera- Paredes, Stanislav Nikolov, Rishub Jain, Jonas Adler, Trevor Back, Stig Petersen, David Reiman, Ellen Clancy. Michal Zielinski, Martin Steinegger, Michalina Pacholska, Tamas Berghammer, Se- bastian Bodenstein, David Silver, Oriol Vinyals, Andrew W. Senior, Koray Kavukcuoglu, Push- meet Kohli, and Demis Hassabis. Highly accurate protein structure prediction with alphafold. Nature, 596(7873),533-589 Aug 2021. ISSN 1476-4687. doi: 10. 1000888641588-00--03338 1038/ 11811558-000
URL httpt::tt//:ooooo/ /wi
Alex Krizhevsky, Ilya Sutskever, and Geoffrey E. Hinton. Imagenet classification with deep con- volutional neural networks. In Proceedings of the 25th International Conference on Neural In- formation Processing Systems Volume NIPS 12, pp. 1097-1105, Red Hook, NY, USA, 2012. Curran Associates Inc.
Jin Ho Kwak and Sungpyo Hong. Linear Algebra. Birkhäuser, Boston, MA, 2004. doi: 10.1007/ 978-0-8176-8194-.
Guillaume Lample and François Charton. Deep learning for symbolic mathematics. ICLR, 2020.
Erwan Le Merrer, Patrick Perez, and Gilles Trédan. Adversarial frontier stitching for remote neural network watermarking. Neural Computing and Applications, 32(13):9233-92 9244,, 2020.
12

---

Under review as a conference paper at ICLR 2023
Yann LeCun, Bernhard Boser, John S Denker, Donnie Henderson, Richard E Howard. Wayne Hub- bard, and Lawrence Jackel. Backpropagation applied to handwritten zip code recognition. Neural computation, 1(4):541-551, 1989.
Kunchang Li, Yali Wang, Junhao Zhang, Peng Gao, Guanglu Song, Yu Liu, Hongsheng Li, and Yu Qiao. Uniformer: Unifying convolution and self-attention for visual recognition. ICLR, 2022a.
Yehao Li, Ting Yao, Yingwei Pan, and Tao Mei. Contextual transformer networks for visual recog- nition. IEEE Transactions on Puttern Analysis and Machine Intelligence, 2022b.
Yanpei Liu, Xinyun Chen, Chang Liu, and Dawn Song. Delving into transferable adversarial exam- ples and black-box attacks. In 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings. OpenReview.net, 2017. /httpss://ppeeeee
net wwt//eeeurttteeeeeee id= Sys
Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2021.
Kaleel Mahmood, Rigel Mahmood, and Marten Van Dijk. On the robustness of vision transformers to adversarial examples. In Proceedings of the JEEE/CVF International Conference on Computer 2021.
Vision, pp.
Matthias Minderer, Josip Djolonga, Rob Romijnders, Frances Hubis, Xiaohua Zhai, Neil Houlsby, Dustin Tran, and Mario Lucic. Revisiting the calibration of modern neural networks. In M. Ranzato, A. Beygelzimer, Y. Dauphin, P.S. Liang, and J. Wortman Vaughan (eds.), Ad- vances in Neural Information Processing Systems, volume 34, pp. 15682-15694. Curran As- sociates, Inc., 2021. 3
ps. neur cc httpppaarr2222//11e. wttpripp.pcpppppeeeeeee
neur pdf
Seyed-Mohsen Moosavi-Dezfooli, Alhussein Fawzi, and Pascal Frossard. Deepfool: A simple and accurate method to fool deep neural networks. In 2016 IEEE Conference on Computer Vision and Puttern Recognition (CVPR), pp. 2574-2582, 2016. doi: N0.1109/VVP..201...2..
Seyed-Mohsen Moosavi-Dezfooli Alhussein Fawzi, Omar Fawzi, and Pascal Frossard. Universal adversarial perturbations. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), July 2017.
Arsha Nagrani, Shan Yang, Anurag Arnab, Aren Jansen, Cordelia Schmid, and Chen Sun. Atten- tion bottlenecks for multimodal fusion. In Advances in Neural Information Processing Systems
(NeurIPS), 2021.
Muhammad Muzammal Naseer, Kanchana Ranasinghe, Salman H Khan, Munawar Hayat, Fa- had Shahbaz Khan, and Ming-Hsuan Yang. Intriguing properties of vision transformers. In M. Ranzato, A. Beygelzimer, Y. Dauphin, P.S. Liang and Wortman Vaughan (eds.), Ad- vances in Neural Information Processing Systems, volume 34, pp. 22996823308 Curran As- sociates, Inc., 2021. URL https: httptttsppp//ppppeieeeeee new.ipe pctppppppp//////11ee// 2c14455acff960006116111111111111 Paper pdf
Anh Nguyen, Jason Yosinski, and Jeff Clune. Deep neural networks are easily fooled: High confi- dence predictions for unrecognizable images. In 2015 IEEE Conference Computer Vision and Pattern Recognition (CVPR), pp. 427-436, 2015.
doi: 10. 10.1109//VPR..2155722200
Jorge Nocedal and Stephen Wright. Numerical Optimization. Springer, New York, NY, USA, 2e edition, 2006.
Keiron O'Shea and Ryan Nash. An introduction to convolutional neural networks, 2015. URL /tttp:://arxxv.oor//bbbss 08858
Francesco Pinto, Philip Torr, and Puneet Dokania. Are vision transformers always more robust than convolutional neural networks? 2021.
13

---

Under review as a conference paper at ICLR 2023
Vidyasagar M Potdar, Song Han, and Elizabeth Chang. A survey of digital image watermarking techniques. In INDIN'OS '05. 2005 3rd IEEE International Conference on Industrial Informatics, 2005., pp. 709-716. IEEE, 2005.
Maithra Raghu, Thomas Unterthiner, Simon Korbliihh, Chiyuan Zhang, and Alexey Doso- vitskiy, Do vision transformers see like convolutional neural networks?
M. Ran- zato, A. Beygelzimer, Y. Dauphin, P.S. Liang, and J. Wortman Vaughan (eds.), Advances in Neural Information Processing Systems, volume 34, pp. 12116-12128. Curran Asso- ciates, Inc., 2021. URL httphtts/s//rrroceeinngssss meuri ips. ec wwtprii...ppppppppepe pptppp/////2//1lle// www3366229008880 htt.:/5222222002000002222255155-
pdf
Benjamin Recht, Rebecca Roelofs, Ludwig Schmidt, and Vaishaal Shankar. Do imagenet classifiers generalize to imagenet? In International Conference on Machine Learning, pp. 5389-5400.
PMLR, 2019.
Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, Alexander C. Berg, and Li Fei-Fei. ImageNet Large Scale Visual Recognition Challenge. International Journal of Computer Vision (ICV), 1 115(3):211-2533 2015. doi: 10.
Ramprasaath R. Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. Grad-cam: Visual explanations from deep networks via gradient- based localization. In ICCV, pp. 618 618-626. 626. IEEE Computer Society, 2017. ISBN 978-
1-5386-1032-9. URL http:// http:/bbppp... www.ccppp.....i.i.cmmmmm..m....a comi-ttrier de /ttp/o//////cc//cc//000.
html 4Selvrr jUCDVPB117
Rulin Shao, Zhouxing Shi, Jinfeng Yi, Pun-Yu Chen, and Cho-Jui Hsieh. On the adversarial robust- ness of vision transformers. arXiv preprint arXiv:2103.15670, 2021.
Sho Sonoda, Isao Ishikawa, and Masahiro Ikeda. Ghosts in neural networks: Existence, structure
and role of infinite- dimensoadi null space. CoRR, abs/2106.04470, 2021. URL https://
arxiv. orgo abs 2106 04770
Gilbert Strang. Introduction to Linear Algebra, Fourth Edition. Wellesley Cambridge Press, httm//xxcc/bbidoo
February 2009a. ISBN 0980232716. URL http:// /twtt:://ww.... amazon. www.aaanno..c....eaaaeaeeeeee
redirect Ttw.rrieeeukeee---aaaaaaaaa//a/a
Gilbert Strang. Introduction to Linear Algebra. Welleslyy-Cambridge Press, Wellesley, MA, fourth edition, 2009b. ISBN 9780980232714 0980232716 9780980232721 0980232724
9788175968 10 8175968117.
Mingxing Tan and Quoc Le. EfficientNet: Rethinking model scaling for convolutional neural net- works. In Kamalika Chaudhuri and Ruslan Salakhutdinov (eds.), Proceedings of the 36th In- ternational Conference on Machine Learning, volume 97 of Proceedings of Machine Learning Research, pp. 6105-6114 14. PMLR, 09-15 Jun 2019. URL https::/prrceed:
mlr..
sress/v97/tans htmll
Rohan Taori, Achal Dave, Vaishaal Shankar, Nicholas Carlini, Benjamin Recht, and Lud- wig Schmidt. Measuring robustness to natural distribution shifts in image classification. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin (eds.), Advances in Neural Information Processing Systems, volume 33, pp. 18583-18599, Curran Asso- ciates, Inc., 2020. URL UtLhhttpss//ppcceedng.. neurips cc cc/pppee//220//iLe/
Hugo Touvron, Matthieu Cord, Alexandre Sablayrolles, Gabriel Synnaeve, and Hervé Jégou. Going deeper with image transformers. In Proceedings of the IEEF/CVF International Conference on Computer Vision (ICCV), pp. 32-42, October 2021.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, L ukasz Kaiser, and Illia Polosukhin. Attention is all you need. In L. Guyon, U. Von Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Gar- nett (eds.), Advances in Neural Information Processing Systems, volume 30. Curran Asso- ciates, Inc., 2017. URL https: httpp:::/ppoccedaaggggoooooo www.eeewwediinn...i........e.... http://r..r.rr.occ/////////////1 3 3 fbd003
PaceaaaaaPaapp_ree pdf
14

---

Under review as a conference paper at ICLR 2023
Pauli Virtanen, Ralf Gommers, Travis E. Oliphant, Matt Haberland, Tyler Reddy, David Courna- peau, Evgeni Burovski, Pearu Peterson, Warren Weckesser, Jonathan Bright, Stéfan J. van der Walt, Matthew Brett, Joshua Wilson, K. Jarrod Millman, Nikolay Mayorov, Andrew R. J. Nel- son, Eric Jones, Robert Kern, Eric Larson, C J Carey, Ilhan Polat, Yu Feng, Eric W. Moore, Jake VanderPlas, Denis Laxalde, Josef Perktold, Robert Cimrman, Ian Henriksen, E. A. Quintero, Charles R. Harris, Anne M. Archibald, António H. Ribeiro, Fabian Pedregosa, Paul van Mul- bregt, and SciPy 1.0 Contributors. SciPy 1.0: Fundamental Algorithms for Scientific Computing in Python. Nature Methods, 17:261-272 2020. doi: 10. 1011008//411
Haohan Wang, Songwei Ge, Zachary Lipton, and Eric P Xing. Learning robust global representa- tions by penalizing local predictive power. In Advances in Neural Information Processing Sys- tems, pp. 10506-10518, 2019.
Shipeng Wang, Xiaorong Li, Jian Sun, and Zonghen Xu. Training networks in null space of feature covariance for continual learning. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 184-193, June 2021.
Raymond B. Wolfgang and Edward J. Delp. A watermark for digital images. Proceedings of 3rd IEEE International Conference on Image Processing, 3:219-222 vol.3, 1996.
Mitchell Wortsman, Gabriel Ilharco, Samir Ya Gadre, Rebecca Roelofs, Raphael Gontijo-Lppes, Ari S Morcos, Hongseok Namkoong, Ali Farhadi, Yair Carmon, Simon Korbliihh, and Lud- wig Schmidt. Model soups: averaging weights of multiple fine-tuned models improves accu- racy without increasing inference time. In Kamalika Chaudhuri, Stefanie Jegelka, Le Song, Csaba Szepesvari, Gang Niu, and Sivan Sabato (eds.). Proceedings of the 39th International Conference on Machine Learning, volume 162 of Proceedings of Machine Learning Research, pp. 23965-23998. PMLR, 17-23 Jul 2022. URL https: httttst::///poooeee fttts:://prooeednggs.lttt v w02//ortsman22s..om httm
Yufei Xu, Qiming Zhang. Jing Zhang, and Dacheng Tao. Vitae: Vision transformer advanced by ex- ploring intrinsic inductive bias. Advances Neural Information Processing Systems, 34:28522- 28535, 2021.
Linting Xue, Aditya Barua, Noah Constant, Rami Al-Rfou, Sharan Narang, Mihir Kale, Adam Roberts, and Colin Raffel. ByT5: Towards a token-free future with pre-trained byte-to-byte models. Transactions of the Association for Computational Linguistics, 10:291-300 2022. doi: 10.1162/tacLaa a..0461...0 URL
1.11
Chongzhi Zhang, Mingyuan Zhang, Shanghang Zhang, Daisheng Jin, Qiang Zhou, Zhongang Cai, Haiyu Zhao, Xianglong Liu, and Ziwei Liu. Delving deep into the generalization of vision trans- formers under distribution shifts. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition (CVPR), pp. 7277-7286, June 2022a.
Hang Zhang, Chongruo Wu, Zhongyue Zhang, Yi Zhu, Haibin Lin, Zhi Zhang, Yue Sun, Tong He, Jonas Mueller, R Manmatha, al. Resnest: Split-attention networks. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition, pp. 2736-2746, 2022b.
Daquan Zhou, Zhiding Yu, Enze Xie, Chaowei Xiao, Animashree Anandkumar, Jiashi Feng, and Jose M Alvarez. Understanding the robustness vision transformers. In International Conference on Machine Learning, pp. 27378-27394 PMLR, 2022.
15

---

Under review as a conference paper at ICLR 2023
A APPENDIX
A.1 NULL COMPONENTS
Algorithm Learning V
for for
epocks: Training epochs
# c: Learning rate 0.01
# stepa: Epochs to reduce the learning rate [150, 300, 400]
lim: Range
randomly sample initial values for [.01, .0,, .1, .25, 1, 21
V, U[-lim,
for epoch in range (epochss:
for in loader: # load minibatch x of images
with torch. grad():
11
original output logits = [=()((f((x)))
13
fo(x) + [0: introducing noise
14
out (u))
15
loss (out gt)². aum (dim=-1) mean () L2 computation
16
17
loss. backward
back-propagate
18
grad Vo gr.grad
19
Vo grad gradient step
21
epoch in steps: r+0.1
22
24 return 2/
A.2 WATERMARKING IMAGES
As mentioned in the main paper, we use SQSLP for minimising equation 7 We use the imple- mentation provided by SciPY (Virtanen et al. 2020) and run it 5000 iterations. Experimentally, we observed that using p norm provides better watermarking behaviour than p 2. All reported and displayed nullspace noise content is with p 1 unless stated otherwise.
To quantitative assess the robustness of model to the watermarking process, we will have to water- mark every image in the dataset. This process requires considerable time and compute to execute. Instead, we perform
% match predictions and absolute difference in the predicted probabilities. We found that 85% of the water- marked images were classified as the source image category. For the mean absolute difference, we compute between the predicted probabilities for source image category both for the source image and the watermarked image. We observed that the difference confidence varied 11.63% on an average.
evaluation for randomly selected 20 images and compute
In figure we show the original, |val and the resulting watermark images. The watermark image the as one reported in the main paper. Using nullspace watermarking, we notice that shape details are more likely to be transferred than other information from the watermark image.
A.3 TARGETED NULLSPACE NOISE
Instead of directly minimising the unconstrained equation 7 with huge number of variables (rxrxo manually fix the values of 2 channels (green and blue) and only perform the minimisation to learn the red channel values for the transformed image. This reduces the number of parameter to one-third and also retains lot of target image information without any loss. This is also the reason why observe different colored tint for the transformed images. With respect to the implementation, the details identical that for watermarking.
A.4 FOOLING XAI METHODS
In figure we show the saliency maps generated by various XAI methods. Even though the maps generated by methods other than IRP are poor (hard interpret), we see that the source and trans- formed respond similarly to these methods.
16

---

Under review as a conference paper at ICLR 2023
CLR
CLK
Figure 6: Watermark superposition using the nullspace basis vectors.
17

---

Under review as a conference paper at ICLR 2023
(a) Attention CCeffer 2021
Grad-CAn Selvaraju 2017
(c) Rollout (Abnar & Zuidema 2020
(d) LRP Chefer et al. 2021
Figure 7: Interpretability maps generated via different methods for (source, target, trans- formed) images
18


Reviews:
[{"review_id": "TsGlVSFOAn", "reviewer": "Reviewer_upQH", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper explores the concept of nullspace for vision transformers (ViTs). The authors find that there usually exists non-trivial nullspace for vision transformers by finding the exact nullspace of their patch-embedding stages. Extending to the case of non-linear transformation where the nullspace is ill-defined, the authors look for an alternative set that adds any element in that set to the input, and the output of the transformation remains unchanged. The authors also study the robustness of ViTs with the nullspace noise perspective. In addition, they show that ViTs are more robust than CNN models under the noise generated via the nullspace. The authors also illustrate that nullspace noise can be used to impose perceptible watermarks on images.\n", "summary_of_the_review": "Overall, I vote for rejecting. The idea of approximating the nullspace noise of the Vision Transformer (ViT) model is interesting. However, my main concerns about the paper are: 1) the low significance of the proposed method to study the nullspace of ViTs in the paper, 2) the lack of theoretical justification of the proposed method in the nonlinear setting, and 3) the unconvincing applications of the proposed method.\n", "strength_and_weaknesses": "**Strong points:**\n\n1. The paper addresses an interesting problem as finding the nullspace of ViT will tell us certain types of input perturbations to which the network is inherently robust.\n\n2. The paper is well-written with illustrative figures.\n\n**Weak points:**\n\n1. In the nonlinear case where the nullspace is ill-defined, the authors “attempt to preserve the axiom of most interest to us, closeness under vector addition” and look for an alternative set that adds any element in that set to the input, and the output of the transformation remains unchanged. This approach lacks theoretical justification.\n\n2. The relaxation to study an individual element of the nullspace in the nonlinear case (e.g. when deriving the nullspace for the self-attention stage) removes all interesting insights from studying the whole nullspace of ViT and makes the contribution of the paper mediocre.\n\n3. For noise robustness application, I have a concern about the significance of the idea. Finding the nullspace noise only allows us to know a certain set of noise to which the ViT is robust. How does it help improve the generalization of ViT models? How to constrain other input permutations/corruptions to be in the approximated nullspace of the network?\n\n4. For the applications relating to watermarking, the nullspace-noise-added images significantly decrease the quality of the original images, making them not useful in reality. For the applications on output-preserved image watermarking, the quality of the corrupted images is bad and much worse than the adversarial examples generated by adding adversarial noise into the original images.\n\n**Additional Concerns and Questions for the Authors:**\n\n1. $\\tilde{v}\\_{\\phi}$ is learned by minimizing Eqn. 6. I am confused whether the authors learned one $\\tilde{v}\\_{\\phi}$ or a set of $\\tilde{v}\\_{\\phi}$ for each self-attention stage. In the first case, $\\tilde{v}\\_{\\phi}$ will be input-independent, then it will no longer be a noise added to the input. \n\n2. The Eqn. 2 in the paper is not correct. \\Delta should be a subspace, not vectors.\n\n3. There is no discussion of related work on the nullspace of transformers, such as the work in [1].\n\n**References:**\n\n[1] Brunner, Gino, Yang Liu, Damian Pascual, Oliver Richter, Massimiliano Ciaramita, and Roger Wattenhofer. \"On identifiability in transformers.\" arXiv preprint arXiv:[1] Brunner, Gino, Yang Liu, Damian Pascual, Oliver Richter, Massimiliano Ciaramita, and Roger Wattenhofer. \"On identifiability in transformers.\" arXiv preprint arXiv:1908.04211 (2019). (2019).\n\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "SGf9QQxpSO", "reviewer": "Reviewer_ZuY5", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper proposes an approach to identify the null space of vision transformers (ViT). They divide this into two parts – (a) null space of patch embeddings, and, (b) null space for the self-attention module(s). The paper uses the above to study the robustness of ViT to null space noise, compare the impact on different architectures, and study its transfer properties. Finally, the null-space is used to apply perceptible watermarks to images without altering the test-time performance (of image classification).\n", "summary_of_the_review": "I’m not convinced of the merits of the paper. There are some interesting ideas. However, the technical framework appears artificial, somewhat ad hoc, and not rigorous enough. More extensive experimental validation is required. As a result, I don’t assess the current submission to be of publishable quality. \n", "strength_and_weaknesses": "## Strengths\n- Investigation of null spaces of vision transformers is a good direction of research and of relevance to the ICLR and wider AI/ ML community. \n- There are some interesting ideas regarding finding perturbations in the input space to which the output of ViT is invariant and to use it for perceptible watermarking.\n\n## Weaknesses\n\n__(W.1) Importance__: The motivation and importance is not quite clear. The authors make an argument about inherent robustness (invariance) to certain perturbation in the input. It is quite obvious that by design, classification networks are sought to be invariant to tp intra-class differences and geometric and photometric transformations that are deemed not to result in changes in the class identity, leading to all kinds of data augmentations. That set of subspaces of inputs may lie in the nullspace is the key idea here but it is not clear what nontrivial value the authors provide in the paper. Please elaborate.\n\n __(W.2) Technical Approach__: I have several questions regarding the technical approach proposed in Section 4  (if I’m understanding it correctly):\n- __(W.2.1)__: Equation (5) describes a set which is invariant to the input $\\mathbf{u}$. However, the loss functional defined in equation (6) doesn’t reflect his invariance to the input $\\mathbf{u}$. Is there an empirical expectation over the entire training dataset?\n - __(W.2.2)__: The loss in Equation (6) admits a trivial solution ($\\tilde{\\mathbf{v}}_\\phi = 0$. There is no regularization which keeps the solution away from the trivial one. Even if empirically the minimization (perhaps through SGD) yields a solution that’s nontrivial, it can still be arbitrarily close, lying in a small non-isotropic neighborhood around $\\mathbf{u}$ with the same output value. The non-isotropic nature (think elongated tubes, for example) will allow a random permutation of the perturbation to change the output. The existence of this small neighborhood is just a statement about local regularity (of data spaces) and smoothness (of functions) on it. How is this useful in the sense that you seek to exploit in the paper? \n- __(W.2.3)__: The solution of Eqn. (6) resulting in a  'nullspace' noise vector which is not arbitrarily close to __any input u__ while maintaining the same prediction as __u__ for every __u__ (i.e., it is independent of __u__) is not demonstrated or validated. At least on the training data, the % match predictions should be 100% for the definition to make sense. Is this the case?\n- __(W.2.4)__: It seems like a single noise vector is learnt over the entire training dataset per training episode (for each random initialization of the noise vector $\\tilde{v}_\\phi$). This is rather inefficient for generating ‘infinite such perturbations’ as claimed in the motivation. Secondly, it’s unclear what the diversity of the set of noise vectors is.\n- __(W.2.5)__:  Magnitude(v) should be shared and the significance of MSE(logit) should be explained. It is not clear what to make of the numbers on the y-axis (0-7000.0). How large are those numbers with respect to the magnitudes of the logits?\n- __(W.2.6)__:  For classification networks, simple geometric transformations like translations, rotations, small scalings etc. should lead to the same classification output. How are the random perturbations found through equation (6) useful beyond the above?\n- __(W.2.7)__:  The whole discussion in 5.2 about strided convolutions and intractability of determining a null space seems mathematically flawed (if we ignore the pointwise nonlinearity of the ReLU etc. which the authors don’t seem to be considering) since tensor multiplication is a linear operation. Kindly explain. [1’] below may be useful to take a look at. \n\n[1’] Zou, Dongmian, Radu Balan, and Maneesh Singh. \"On lipschitz bounds of general convolutional neural networks.\" IEEE Transactions on Information Theory 66.3 (2019): 1738-1759. \n\n __(W.3) Experimental Evaluation__: \n- __(W.3.1) Robustness__: In Figure 3(a), why should the scale of the features with respect to which the sampling limit is meaningful, be the same? \n- __(W.3.2) Robustness__: Comparison of robustness to noise across different networks makes sense when the performance degradation under the same perturbation of the input is analyzed. In the experiment in Figure (3), the learnt perturbation also varies and is from an optimization algorithm (which may fail to do what’s expected of it). In this ‘approximate’ setting, everything is so amorphous and the different factors are so entangled that it’s hard to have any takeaways from this experiment. Kindly explain. \n- __(W.3.3) Watermarking__: Equation (7) suggests that the transformed image should look like the target (watermark) and yet have the same classification semantics as the source. However, the transformed images in Figure (4) look more like the source images while images in Figure (5) look more like the target images. It is not clear what the goal of visible watermarking is and what kind of visual perturbations are considered to be of acceptable quality and for what purposes. Kindly clarify. \n- __(W.3.4) Watermarking__: Qualitative results provided in A.2 should be in the main paper. \n- __(W.3.5) Watermarking__: Only twenty images are used for evaluation. This is unsatisfactory. Since only the null space of the patch embeddings is considered ($v_\\theta$), why should this be prohibitively expensive?\n- __(W.3.6) Watermarking__: Finally, since by design, $v_\\theta$ should not alter the representations by the network, should the classification accuracy be higher than 85%, and 100% if training images are used?\n- __(W.3.7) Unconstrained Targeted Noise__: No quantitative results are provided.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "HxeTTI4xjB", "reviewer": "Reviewer_ZgDX", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper presents an exploration of the concept of nullspaces for vision transformers. It presents an approach towards determining the nullspace vectors of the vision transformer: a deterministic non-trivial nullspace for linear projection components and an approximate nullspace for non-linear attention layers. Further, it explores its implications in 3 areas: robustness, watermarking and adversarial sample generation to fool models and interpretability methods. Finally, the paper shows that ViTs are more robust to higher amounts of nullspace-generated noise than state-of-the-art CNNs, and demonstrates the application of nullspace noises in watermark imposition and adversarial sample generation.", "summary_of_the_review": "Borderline reject. The paper definitely proposes a novel idea, however, I think a bit more analysis and results are required. If the author(s) are able to address the above comments, the decision will be reconsidered.", "strength_and_weaknesses": "Pros\n\t- The exploration of nullspace noise for analyzing the robustness of models is a novel application of the concept. I would love to see more exploration into the implications of nullspace noise on robustness.\n\t- The paper was very well written, and easy to follow.\n\nCons\n\t- The paper mainly discusses 3 areas: robustness, watermarking and adversarial generation.\n\t\t§ Robustness:\n\t\t\t§ I think this area requires more analysis in the paper. The introduction and conclusion sections imply that the existence of nullspace vectors, and the high amount of noise generated using this nullspace basis, has significant implications on the robustness of the vision transformer.\n\t\t\t§ Section 5 presents the question: \"How robust are different architectures to nullspace noises.\" However, this seems a bit confusing. My understanding is, all architectures are, by definition, robust to their nullspace noises. Further, each network has its own nullspace noise distribution, depending on its architecture. The main question, I think, is more about the properties of the nullspace noises of a given model. A model is more robust if:\n\t\t\t\t□ Its nullspace has a high sampling limit (with a high threshold on accuracy or %match)\n\t\t\t\t□ The nullspace samples, as described by equation (4), have high accuracy too.\n\t\t\t§ Figure 3 (a) and (b) show that ViT have higher sampling limits than state-of-the-art CNNs. I think these results give evidence for the first property of robustness w.r.t nullspace noise as described above. However, I think it will be really helpful to analyse these results on more datasets, instead of just a single one.\n\t\t\t§ Figure 3(c) shows low transferability of noise between models, however, I don't think this particular graph makes any implications on which model is better. Since the noise vectors are obtained using ViT-S model, they are strongly biased against the CNNs. It's very likely that the ViT models will perform equally poorly on the noise vectors obtained using the CNNs. I think it would be helpful if this point is highlighted.\n\t\t\t§ Figure 3 doesn’t include any quantitative results for the 2nd property described above, regarding the accuracy of nullspace samples from equation (4). This can probably be done by presenting the deviation of accuracy at different nullspace noise vectors, and increasing the number of sampling limits considered.\n\t\t\t§ It would be helpful if the point about \"lack of effective nullspace noise\" in CNNs is elaborated a bit more. Does the difficulty in finding effective noise due to non-linearity necessarily mean lack of effective nullspace noise?\n\t\t§ Watermarking:\n\t\t\t§ In this application, a slightly different watermark will be applied on each image. It would be helpful if the implications of this are discussed. Would this significantly comprise the tasks for which the watermarks are being used for? I am curious to see if this has any practical implications.\n\t\t§ Adversarial Sample Generation:\n\t\t\t§ I think more results are required for this application. It would be really helpful if the following results are included:\n\t\t\t\t□ An accuracy score on a few datasets on the misclassifications, along with the qualitative examples, shown in Figure 5 (a).\n\t\t\t\t□ It would be helpful if the explainability attribution map mistakes are quantified in some manner, probably IoU/DICE etc, to get some quantitative results to support the qualitative results shown in Figure 5 (b).\n\nMinor Edits\n\t- In section 2, paragraph starting with \"A trivial nullspace, …\", shouldn't \"axiom 1\" be used instead of \"linear mapping as described in equation 1\", and \"equation 2\" instead of \"linear equations as described by equation 1\"?\n\t- In section 3.1 Classification Stage, shouldn't it end with \"project it through a linear classification layer, followed by a softmax layer\" instead?\n\t- In equation (5), it would be more clear to add f_phi instead of just f.\n\t- Section 6 method details: typo at delta x_j -> \"changed\" to \"change\"\nIn section 7 line 3, helpful to write equation (7) instead of just 7.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "B-cqYsf33LK", "reviewer": "Reviewer_kZts", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper explores the nullspace ( adding an element on this space does not affect the output) for Vision Transformer. The authors first demonstrate that a non-trivial nullspace exists for the patch embedding matrices. This idea is extended to the non-linear layers of the vision transformer, which is learned via simple optimizations. \nThe applications to image watermarks and unconstrained targeted noise are presented. ", "summary_of_the_review": "This paper presents a nullspace analysis of ViT, which have originality. The two applications of nullspace are also interesting. Though it is understandable, the writing and evaluations are not high quality. ", "strength_and_weaknesses": "Strength\n\n- The introduction of nullspace for analyzing ViT has originality. This paper investigates if the nullspace exists for the linear input layer (patch embedding layer) and non-linear layer (self-attention layer). The patch embedding layer's linear structure produces a larger nullspace than CNNs, as shown in Sec.5.2. \n\n- The watermark application and the targeted nullspace noise are interesting. They are largely different from conventional adversarial attacks. \n\nWeakness\n- The results of watermark supervision and targeted nullspace noise are not visually attractive.\n\n- Writing quality is not high. \n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works.", "empirical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject