Deep Support Vectors

Authors: junhoo  lee
hyunho  lee
kyomin  hwang
nojun  kwak
Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-15 05:51:41

Abstract: Deep learning has achieved tremendous success. However, unlike SVMs, which provide direct decision criteria and can be trained with a small dataset, it still has significant weaknesses due to its requirement for massive datasets during training and the black-box characteristics on decision criteria. This paper addresses these issues by identifying support vectors in deep learning models. To this end, we propose the DeepKKT condition, an adaptation of the traditional Karush-Kuhn-Tucker (KKT) condition for deep learning models, and confirm that generated Deep Support Vectors (DSVs) using this condition exhibit properties similar to traditional support vectors. This allows us to apply our method to few-shot dataset distillation problems and alleviate the black-box characteristics of deep learning models. Additionally, we demonstrate that the DeepKKT condition can transform conventional classification models into generative models with high fidelity, particularly as latent generation models using class labels as latent variables. We validate the effectiveness of DSVs using common datasets (ImageNet, CIFAR10 and CIFAR100) on the general architectures (ResNet and ConvNet), proving their practical applicability.

Main paper: 
 introduction: 
 Although deep learning has gained enormous success, it requires huge amounts of data for training, and its black-box characteristics regarding decision criteria result in a lack of reliability. For example, CLIP [24] needs 400 million image pairs for training and Stable Diffusion XL (SDXL) [25] requires • We verify that DeepKKT can be used to extract and generate DSVs for common image datasets such as CIFAR10, CIFAR100, SVHN, and ImageNet.• DSVs are better than existing algorithms in few-shot dataset distillation problems.• DeepKKT acts as a novel model inversion algorithm that can be applied in practical situations.• By using the DeepKKT condition, we not only show the trained deep models can reconstruct data, but also can serve as latent generative models by using logits as latent.2 Related Works 
svm in deep learning: 
 Numerous studies have endeavored to establish a connection between deep learning and conventional SVMs. In the theoretical side, [28] demonstrated that, in an overparameterized setting with linearly separable data, the learning dynamics of a model possessing a logistic tail are equivalent to those of a support vector machine, with the model's normalized weights converging towards a finite value. Following this, [15] extended this equivalence to feedforward networks. This line of research relies on strong assumptions such as full-batch training and non-residual architecture without data augmentation. There also exists a body of work on integrating SVM principles into deep learning, often referred to as DeepSVM, aiming to leverage SVM's desirable properties [30, 29, 27, 23, 21] . DeepSVM integrates SVM components, specifically using them as feature extractors to derive meaningful, human-crafted features.In contrast, our work does not modify or incorporate SVM architectures. Instead, we focus on identifying support vectors directly within deep learning models, thereby bridging the gap between deep learning and support vector machines in a more fundamental manner. Despite these advancements, there remains a lack of research that directly connects support vectors through a theoretical lens of equivalence. In this study, we address this gap by introducing the DeepKKT condition, a KKT condition tailored for deep learning, allowing us to apply the concept of support vectors in a practical deep learning context.We show that reconstructing support vectors in deep models is indeed feasible, and obtaining highquality support vectors is achievable under much less restrictive conditions compared to prior work.There is a line of research utilizing the stationarity condition, a part of the KKT condition, for model inversion. [6] firstly exploited the KKT condition for model generation, adopting SVM-like architectures. They normally conducted experiments with binary classification, a 2-layer MLP, and full-batch gradient descent. [1] extended these experiments to multi-label classification by adapting their existing architecture to a multi-class SVM structure. To ensure the generated samples lie on the data manifold, they initialized with the dataset's mean, implying the adoption of some prior knowledge of the data. Similarly, [34] generated images through the stationarity condition, also adopting the mean-initialization and conducting experiments on the CIFAR10 [11] , MNIST [4] and downsampled CelebA [14] datasets. Their work generally focused on low-dimensional, labeled datasets with a small number of classes such as CIFAR10 and MNIST, consistent with the traditional SVM setting of binary-labeled, low-dimensional datasets. In contrast, we extended our experiments to high-dimensional datasets with many classes, an area traditionally dominated by deep learning. Specifically, we conducted experiments on ImageNet [26] using a pretrained ResNet50 model following the settings described in the original paper [7] .Furthermore, previous works have concentrated on reconstructing the training dataset. In contrast, similar to generative models, our work focuses on generating unseen data from noise using a classification model. Additionally, we emphasize the original meaning of 'support vectors'. Unlike other approaches, our Deep Support Vectors (DSVs) adhere to the traditional role of support vectors: they explain the decision criteria, and a small number of DSVs can effectively reconstruct the model.Dataset distillation [2, 31, 13] fundamentally aims to reduce the size of the original dataset while maintaining model performance. The achievement also involves addressing privacy concerns and alleviating communication issues between the server and client nodes. The dataset distillation problem is typically addressed under the following conditions: 1) Access to the entire dataset for gradient matching, 2) Possession of snapshots from all stages of the model's training phase, which are impractical settings for practical usage [12] . Furthermore, these algorithms typically require Hessian computation, which imposes a heavy computational burden.In SVM, the model can be reconstructed using support vectors. This reconstruction is more practical compared to previous dataset distillation methods, as it does not require any of the restrictive conditions. Likewise, because Deep Support Vectors (DSVs) also do not require these conditions and are Hessian-free, they can play the role of distillation under practical conditions. 
notation: 
 In the SVM formulation, w(:= [w; b]) represents the concatenated weight vector w and bias b. Each data instance, expanded to include the bias term, is denoted by xi (:= [x i ; 1]), while the corresponding binary label is represented by y i ∈ {±1}. The Lagrange multipliers are denoted by α i 's.Transitioning to the context of deep learning, we denote the parameter vector of a neural network model by θ, which, upon optimization, yields θ * as the set of learned weights. The mapping function Φ(x i ; θ) represents the transformation of input data into a C-dimensional logit in a C-class classification problem in a manner dictated by the parameters θ, i.e., Φ(x i ; θ) = [Φ 1 (x i ; θ), • • • , Φ C (x i ; θ)] T ∈ R C .We define the score as the logit of a target class, i.e., Φ yi (x i ; θ). If the score is the largest among logits, i.e., arg max c Φ c (x i ; θ) = y i , then it correctly classifies the sample. The Lagrange multipliers adapted to the optimization in deep learning are represented by λ i 's. (x, y) denotes a pair of input and output and I is the index set with |I| = n.The fundamental concept of Support Vector Machines (SVMs) is to find the optimal hyperplane that classifies the given data. This hyperplane is defined by the closest data points to itself known as support vectors, and the distance between the support vectors and the hyperplane is termed the margin. The hyperplane must classify the classes correctly while maximizing the margin. This leads to the following KKT conditions that an SVM must satisfy: (1) Primal feasibility: ∀i, y i wT xi ≥ 1, (2) Dual feasibility: ∀i, α i ≥ 0, (3) Complementary slackness: α i y i wT xi -1 = 0 and (4)Stationarity: w = n i=1 α i y i xi .The primal and dual conditions ensure these critical values correctly classify the data while being outside the margin. The complementary slackness condition mandates that support vectors lie on the decision boundary. The stationarity condition ensures that the derivative of the Lagrangian is zero.The final condition, stationarity, offers profound insights into SVMs. It underscores that support vectors encode the decision boundary w. Consequently, identifying the decision hyperplane in SVMs is tantamount to pinpointing the corresponding support vectors. This implies that with a trained model at our disposal, we can reconstruct the SVM in two distinct ways:1. Support Vector Selection: From the trained model, we can extract support vectors among the training data that inherently encode the decision hyperplane.2. Support Vector Synthesis: Alternatively, it is feasible to generate or synthesize support vectors, even in the absence of a training set, which can effectively represent the decision hyperplane by generating samples that satisfy | wT x| = 1.Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?Answer: [Yes] Justification: The theoretical framework is grounded in adapted KKT conditions, with assumptions explicitly stated for applying these in high-dimensional, multi-class deep learning settings. Although some proofs are briefly outlined, the core theoretical justifications are complete, with further details in the appendix for additional clarification.Guidelines:• The answer NA means that the paper does not include theoretical results.• All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced. • All assumptions should be clearly stated or referenced in the statement of any theorems.• The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition. • Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material. • Theorems and Lemmas that the proof relies upon should be properly referenced. 
deep support vector: 
 This section presents the specific conditions that DSVs (Deep Support Vectors) must satisfy and discusses how to get an optimization loss to meet these conditions.SVM's Relationship with Hinge loss We start our discussion by focusing on the hinge loss, a continuous surrogate loss for the primal feasibility, and its gradient:EQUATIONWith Eq. ( 1), the stationarity condition of SVM becomesw * := - n i=1 α i ∇ w L h (x i , y i ; w * ), s.t. α i ≥ 0.(EQUATIONGeneralization of conventional KKT conditions In this paper, we extend the KKT conditions to deep learning. In doing so, the two main hurdles of a deep network different from a linear binary SVM are 1) the nonlinearity of Φ(x i ; θ) taking the role of wT xi and 2) multi-class nature of a deep learning model. Considering that the role of the primal feasibility condition is to correctly classify x i into y i , we can enforce the score Φ yi (x i ; θ) for the correct class y i to take the maximum value among all the logits with some margin ϵ, i.e.,EQUATIONWe can relax this discontinuity with a continuous surrogate function -L(Φ(x i ; θ * ), y i ), which is the negative loss function to maximize. Note that if we take the cross-entropy loss for L, it becomesEQUATIONwhich takes a similar form as Eq. ( 3) and the negative loss can be maximized to meet the condition. Now that we found the analogy between y i wT xi and -L(Φ(x i ; θ * ), y i ), y i xi (= ∇ w(y i wT xi )) corresponds to -∇ θ * L(Φ(x i ; θ * ), y i ). Thus, the stationary condition w = n i=1 α i y i xi in SVMs can be translated into that of deep networks such thatEQUATIONThis condition is a generalized formulation of Eq. ( 2), where we substitute the linear model wT x with a nonlinear model Φ(x; θ), and the binary classification hinge loss L h with multi-class classification loss L. Furthermore, the stationarity condition reflects the dynamics of overparameterized deep learning models. We provide an analogy with respect to [28] in Appendix C.However, these conditions are not enough for deep learning. As mentioned before, we are interested in dealing with high dimensional manifolds. Compared to the problems dealt with in the classical SVMs, the input dimensions of deep learning problems are typically much higher. In this case, the data are likely to lie along a low-dimensional latent manifold M inside the high-dimensional space.To make a generated sample be a plausible DSV, it not only should satisfy the generalized KKT condition but also should lie in an appropriate data manifold, i.e., x ∈ M.Finally, we can rewrite the new DeepKKT condition as follows:Primal feasibility:EQUATION4.2 Deep Support Vectors, From Dust to Diamonds Sec. 4.1 explored the KKT conditions in the context of deep learning and how these conditions can be used to formulate a loss function. It is important to note that our goal is not to construct the model Φ, but rather to generate support vectors of an already-trained model Φ with its parameter θ * .To reconstruct support vectors from a trained deep learning model, sampling or synthesizing support vectors is essential. By replacing the optimization variable θ with x, we shift our focus to utilizing the DeepKKT conditions for generating or evaluating input x rather than θ. This adjustment necessitates a consideration of the data's inherent characteristics, specifically its multiclass nature and the tendency of the data to reside on a lower-dimensional manifold in an ambient space. Primal Feasibility Firstly, the primal feasibility condition in Eq. ( 6) mandates support vectors be correctly classified by the trained model θ * . As presented in Sec. 4.1, instead of Eq. ( 3), we use a surrogate function for the loss:EQUATION)This is designed to match the primal condition by mimicking the Hinge loss. When each DSV is correctly classified, no loss is incurred. Otherwise, we adjust the DSVs to align them with the correct target, effectively optimizing the support vectors. This approach also implicitly enforces the complementary slackness condition, as L i decreases confidence in the incorrect classification.Here, L can be any loss function and we have employed the cross-entropy loss in our experiments.Stationarity Secondly, the stationarity condition can be used directly as a loss function. Since we are extracting DSVs from the trained model Φ(•; θ * ), we construct this loss as follows:EQUATIONFor the distance measure D, any metric can be used; we have chosen to use the l 1 distance to suppress the effect of outliers. It is crucial to remember that our objective is to find DSVs and the optimization is done for the primal and dual variables x i and λ i and not for the parameter θ. For this, we require one forward pass and two backward passes; one for ∇ θ L and the other for ∇ xi D. The overall computational cost is quite low, as we optimize only a small number of samples.Moreover, as shown in Algorithm 1 (Appendix H), we satisfy the dual condition by ensuring the Lagrange multipliers λ i 's are greater than zero and disqualify any x i 's from being a support vector candidate if during optimization λ i becomes less than zero. The condition that is not explicitly satisfied is the complementary slackness. To directly fulfill the functional boundary for support vectors as specified in Sec. 3.2, we would need to be able to calculate the distance between functions, which is not only abstract but also requires a second-order computation cost. Therefore, we adopted a relaxed version of the KKT conditions that excludes this requirement. Furthermore, as demonstrated in Sec. 5.1, we have shown that DSVs implicitly satisfy the complementary slackness condition. This implies that DSVs meet every condition introduced in conventional SVM.Manifold Condition: Reflecting High-Dimensional Dynamics of Deep Learning As modern deep learning deals with extremely high-dimensional spaces, imposing additional constraints other than the primal feasibility and stationarity conditions is needed so that DSVs reside in the desired data manifold. To achieve this, we add a manifold condition, which enforces that the DSVs lie on the data manifold. By selecting DSVs that are in the intersection of the solution subspace and the data manifold, we can properly represent both the model and the training dataset.To extract DSVs from the manifold, we assume that the model is well-trained, meaning it maintains consistent decisions despite data augmentation. In other words, the model should classify DSVs invariantly even after augmentation. To ensure this, we enforce that the augmented DSVs (A(x) where A denotes augmentation function) also meet the primary and stationarity conditions.Also, we exploit traditional image prior [33, 16] , total variance L tot and size of the norm L norm to make DSVs lie in the data manifold. L tot is calculated by summing the differences in brightness between neighboring pixels, reducing unnecessary noise in an image, and maintaining a natural appearance. L norm , taking a similar role, penalizes the outlier and preserves important pixels.Finally our DSV is obtained as follows where E A represents expectation over augmentations:EQUATION)One might wonder if there is a better sampling strategy than using DSVs, such as sampling far from the decision boundary instead of near it. We argue that in a high-dimensional data manifold, most data points are located close to the decision boundary because, in a data-scarce, high-dimensional space, every sample matters and thus would serve as a DSV. 1 shows this to be true. On the CIFAR-10 [11] dataset, we selected images with high λ values and retrained the network with the selected images. In this case, the selected DSVs show higher test acccuracies compared to random selection. This characteristic resembles that of support vectors, as the model can be reconstructed with support vectors.Finally, Fig. 1 demonstrates the high fidelity of the generated DSVs, providing practical evidence that these DSVs lie on the data manifold. Similar to how support vectors are reconstructed in an SVM, the DeepKKT condition enables the reconstruction of these vectors without referencing training data. This shows the effectiveness and adaptability of our DeepKKT in capturing key data features.DSVs for Few Shot Dataset Distillation DeepKKT emerges as a pioneering algorithm tailored for practical dataset distillation. DSVs addresses two critical concerns: 1) Protecting private information through data synthesis, and 2) Reducing the communication load by minimizing the size of data transmission. Traditional distillation algorithms encounter a fundamental paradox; as data predominantly originate from edge devices like smartphones [18, 17, 10] , the requirement to access the entire dataset introduces significant communication overhead and heightens privacy concerns. DSVs Encode the Decision Criteria Visually Our findings suggest that DSVs not only satisfy the conditions of classical support vectors but also offers a global explanation of visual information. Fig. 3 experimentally verifies our claims and illustrates the practical use of DSVs, e.g., analysis of Fig. 1 -Cifar10 reveals the decision criteria for classifying deer, cats, and dogs: 1) DSVs highlight antlers in deer, signifying them as a distinctive characteristic. 2) Pointed triangular ears are a recurring feature in DSVs of cats. 3) For dogs, a trio of facial dots holds significant importance. Using these observations, we altered a deer's image by erasing its antlers and reshaping its ears to a pointed contour, which reduced the model's confidence in classifying it as a deer, and caused the model to misclassify it as a cat. Similarly, by smoothing the ears of a cat image to diminish its classification confidence and then adding antlers or three facial dots, we influenced the model to reclassify the image as a deer or a dog, respectively. Additionally, the DeepKKT-altering case in Fig. 3 -Bottom supports our assertions. Altering a flower image to resemble a deer class by changing the target class in the primal and dual feasibility loss, antlers grew similar to our manipulation.This discovery holds significant implications about making models responsible; it introduces a qualitative aspect to assessing model performance. Consider a deer classification problem again. The model in our study would be less suitable, as evidenced by Fig. 1 -Cifar10-deer, which indicates the model's reliance on antlers for identifying deer -a feature not present in winter. This shows that DSVs enable us to conduct causal predictions by qualitatively analyzing models, as SVM does. tions enable model inversion even in these challenging environments, suggesting that they can be applied to a subset of the parameter space rather than the entire parameter space i.e., a more relaxed condition.Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)? Answer: [Yes] Justification: The paper provides comprehensive details on the model architectures, datasets, augmentation strategies, and hyperparameters used in the experiments. Key implementation choices, such as optimizer configurations and specific data augmentation techniques, are described in the supplementary materials. Guidelines:• The answer NA means that the paper does not include experiments.• If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not. • If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable. • Depending on the contribution, reproducibility can be accomplished in various ways.For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed. • While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. , with an open-source dataset or instructions for how to construct the dataset). (d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility.In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results. Guidelines:• The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.• If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics. • The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction). 
unlocking the potential of classifier as generator with deepkkt: 
 Classifier as Latent generative model Considering the impressive capabilities of DSVs in the image generation domain, and the geometric interpretation that DSVs are samples near the decision boundaries, it is noted that enforcing the DeepKKT condition resembles the diffusion process. In each iteration, the DSV develops through the DeepKKT condition as follows:EQUATION)This is similar to the generalized form of the score-based diffusion process: The first three loss terms in Eq. ( 10) aim to maximize the score (∇ x log p(x t )), while the last term, the primal feasibility term, corresponds to the guidance term (γ∇ x log p(y|x t )). As shown in Fig. 5 , when only the primal loss term is used, meaningful DSV samples are not generated. This indicates that the other losses (stationarity and manifold terms) function update the image towards manifold i.e., score function. From this perspective, an arbitrarily assigned label y can be used as a latent variable for guidance.EQUATION)To experimentally verify this, we performed a latent interpolation task and image editing, which is common in generative models [8, 3] By mixing different labels (y i = (1 -δ)y a + δy b , where y a ̸ = y b ), we generated DSVs as depicted in Fig. 6 . When generating DSVs with these mixed soft labels, the generated DSVs semantically represent the midpoint between the two classes. A generated DSV either simply contains both images (the case of "balloon" and "spaniel") or semantically 'fuse' objects (the case of "lobster" and "harp", producing an image of a harp made out of lobster claws). For the image editing task, we assigned the latent variable to the desired class and then aligned the image using DeepKKT loss. The result was quite surprising: the method successfully transferred the image to the desired class while maintaining the original structure. For example, the sail of a yacht was seamlessly transformed into the shape of a scarf. This task was impossible with other methods; in diffusion models, for instance, a mask would be needed to edit the image seamlessly.The fact that the generated images correctly merge the semantics of the classes suggests a couple of significant implications: 1) 1) New Generative Model: This approach offers a new type of generative model as an alternative to GANs and diffusion models. It can handle the same task without the need for training a specific model, as it leverages existing classification models for generative purposes. Furthermore, it is lightweight compared to diffusion models. For example, as the model size of a pretrained ResNet50 for ImageNet is only one-twentieth of that for SDXL [25] , DSVs show a potential to leverage existing classification models for generative purposes. 2) Exploration of Classification Model Generalization: Unlike other generative models, classification models are trained simply to predict the label of an image. Yet, in latent interpolation and editing tasks, they demonstrate an understanding of semantics. This implies that, despite being trained to memorize class labels, the models grasp the overall semantics of the dataset. As they can generate seemingly unseen samples by interpolation and editing. 
conclusion: 
 In this paper, we redefined support vectors in nonlinear deep learning models through the introduction of Deep Support Vectors (DSVs). We demonstrated the feasibility of generating DSVs using only a pretrained model, without accessing to the training dataset. To achieve this, we extended the KKT (Karush-Kuhn-Tucker) conditions to DeepKKT conditions and the proposed method can be applied to any deep learning models.Akin to SVMs, the DeepKKT condition effectively encodes the decision boundary into DSVs. DSVs can reconstruct the model, making them useful for dataset distillation. Additionally, their visual encoding of the decision criteria can serve as a global explanation, helping to understand the model's overall behavior and decisions. Furthermore, the DeepKKT condition transforms a classification model into a generative model with high fidelity. Not only can it sample data, but it also generalizes well, allowing the use of labels as latent variables. 
broader impacts: 
 Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?Answer: [Yes]Justification: The paper discusses societal impacts, especially the benefits of interpretability in AI models and potential concerns about model inversion's misuse. It acknowledges the ethical considerations associated with generating data from sensitive models, encouraging responsible handling.Guidelines:• The answer NA means that there is no societal impact of the work performed.• If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact. • Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations. • The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster. • The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology. • If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML). 
safeguards: 
 Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)? Answer: [NA] Justification: The paper does not release high-risk models or data that require specific safeguards.Guidelines:• The answer NA means that the paper poses no such risks.• Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters. • Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images. • We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort. 12. Licenses for existing assets Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected? Answer: [Yes] Justification: The paper appropriately references all datasets (e.g., CIFAR-10, ImageNet) and model architectures, following proper citation practices. Guidelines:• The answer NA means that the paper does not use existing assets.• The authors should cite the original paper that produced the code package or dataset.• The authors should state which version of the asset is used and, if possible, include a URL. • The name of the license (e.g., CC-BY 4.0) should be included for each asset.• For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided. • If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset. • For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. • If this information is not available online, the authors are encouraged to reach out to the asset's creators.13. New Assets Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets? Answer: [NA] Justification: The paper does not introduce any new assets, such as unique datasets or models, that require documentation. Guidelines:• The answer NA means that the paper does not release new assets.• Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc. • The paper should discuss whether and how consent was obtained from people whose asset is used. • We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution. • For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review. 
 
 Appendix: Our paper is closely related to Responsible AI (RAI), especially in enabling qualitative assessments of models. Our approach provides visual and intuitive explanations of a model's decision-making criteria, offering insights that are both explanatory and responsible. Our approach of utilizing DSVs for RAI enables global explanations, surpassing traditional Explainable AI (XAI) methodologies, which usually focus on local explanations for individual inputs and cannot provide a global decision criterion. Furthermore, since our method is based on model inversion, it ensures safety and privacy. While the synthesized sets in Fig. 7 might appear similar to the selected sets, they do not replicate specific sample features. This is because DSVs represent a more generalized decision boundary, avoiding the inclusion of image-specific features. Consequently, DSVs enable all models using logistic loss to be more responsible. In this paper, we propose the DeepKKT condition, which can be applied universally to any deep models to generate deep support vectors (DSVs) that function similarly to support vectors in SVMs. However, it should be noted that the equivalence between DSVs in deep learning models and support vectors in SVMs is only described intuitively, not rigorously. We have shown experimentally in Fig. 7 and intuitively in Sec. C why the DeepKKT condition should be as we suggested, but we have not derived it with rigorous math. Proving this rigorously would be a meaningful research topic.In DeepKKT, many conditions make sense, except for one. For instance, the primal feasibility condition and the manifold condition are reasonable, and the dual feasibility condition can be regarded as importance sampling. However, the most counterintuitive part is the stationarity condition:EQUATIONIn this section, we will explain the dynamics of DSVs in an overparameterized deep network and how it is connected to deep learning. Below is a quick analogy of [28] to illustrate this connection.A deep learning model follows the following ODE:EQUATIONHere, η is the learning rate and t is the optimization step. The loss L does not go to zero since deep learning models usually exploit a loss function with a logistic tail, such as the cross-entropy loss, and the gradient of the least confident sample (support vector) dominates overall gradient. Thus, there exists a convergence of the gradient direction g ∞ := ∇L. There also exists a time T where the Figure 8 : The stationarity condition with a logistic loss. Even though the direction of the gradient ĝ converges, the size of the gradient does not go to zero. Therefore, the direction of the converged gradient weight ŵ∞ aligns with ĝ.gradient direction converges to g ∞ -ε for a sufficiently small ε. As illustrated in Fig. 8 , w moves toward the direction of -g ∞ . Therefore, ŵ∞ ≈ -g ∞ .This is for what stationarity condition wants to seek. The direction of g ∞ , by using only a few support vectors.To obtain the results in Table 2 and Fig. 4 , the ConvNet architecture [5] was used for pretraining Φ(•; θ) on the SVHN dataset [20] , a digit dataset with dimensions similar to CIFAR-10 [11] . For ImageNet, we used the ResNet50 model [7] with the original setting in the paper. Specifically, we used the pretrained model in torchvision library in pytorch [22] . For visualizing synthesized DSVs in ImageNet, we increased the contrast in 224x224 dimensions. When calculating L stat , we averaged the distance per parameter. In Alg. 1, η was set to 5.To synthesize DSVs in ImageNet, we used translation, crop, cutout, flip, and noise for augmentation, with hyperparameters set to 0.125, 0.2, 0.15, 0.5, and 0.01, respectively. In Eq. ( 9), we set α to 2e-5, β to 40, and γ to 1e-6. When calculating L stationarity , we averaged the distance per parameter.For dataset distillation in Table 2 , we used translation, crop, flip, and noise for augmentation, with hyperparameters set to 0.125, 0.2, and 0.5, respectively. In Eq. ( 9), we set α to 2e-3, and both β and γ to 0. For retraining models with synthesized images, we used a learning rate of 1e-4 while the other parameters set to the default values of the Adam optimizer [9] .To obtain the pretrained weight θ * for CIFAR10 and CIFAR100, we chose the ConvNet architecture [5] , a common choice in deep learning. This architecture includes sequential convolutional layers followed by max pooling, and a single fully-connected layer for classification. The learning rate was set to 10 -3 with a weight decay of 0.005 using the Adam optimizer. Additionally, we employed flipping and cropping techniques, with settings differing from those used for DSVs reconstruction to ensure fair comparison. For pretraining Φ on the Street View House Numbers (SVHN) dataset [20] , a digit dataset with dimensions similar to CIFAR-10 [11] , we exclusively trained the fully-connected layer of the CIFAR-10 pre-trained ConvNet. This approach resulted in a training accuracy of 80%.Fig. 9 shows the selected images with large Lagrangian multipliers λ's, which correspond to the candidates used in Fig. 2b . Surprisingly, there is a meaningful match between the selected DSVs and the synthesized DSVs in the CIFAR-10 dataset, as shown in Fig. 7 . This implies that synthesizing DSVs corresponds to reviving training data that lie on the boundary manifolds. It is noteworthy to highlight this situation resembles that of an adversarial attack [32, 19 ], yet we did not apply gradient descent to the image; we simply mixed two images. This suggests that the discriminative informational density in a single DSV image is substantially greater than that in a randomly sampled image. The fact that the DSV's characteristics remained dominant in the classification, underscores the significant role of DSVs in explaining the model's classification ability.In Fig. 11 , examples of latent interpolations between target labels are presented. The smoothness of these interpolations within the latent space indicates that the semantic information learned from the training data has been effectively applied during the DSV generation process. This observation provides evidence that the DeepKKT optimization successfully conducts the generative process.Fig. 12 and 13 provide examples of deep support vectors generated using the CIFAR-100 and ImageNet datasets, respectively. Fig. 14 presents additional examples related to image editing.Fig. 15 empirically supports on our assertion on decision criterion. Starting from CIFAR100 random images and CIFAR10-pretrained models, we edited the image CIFAR10 labels as latents. The edited images changes the image following decision criterions in generated DSVs. sample N number of (x s i , λ s i ) for label y s i = i 4: end for 5: DefineX S = {x s i | i ∈ [C], s ∈ [N ]} 6: Define Λ S = {λ s i | i ∈ [C], s ∈ [N ]} 7: repeat 8: L primal (X S ) = N s=1 C i=1 L(Φ(x s i ; θ), y s i ) 9: L stationary (X S ) = ∥θ + N s=1 C i=1 λ s i y s i ∇ θ Φ(x s i ; θ)∥ 2 2 10: L kkt (X S ) = β 1 • L primal (X S ) + L stationary (X S )11:L prior = β 2 • L tot (X) + β 3 L norm (X) 12: Sample f A ∈ A 13: Define AX S = {f A (x s i ) | x s i ∈ X S } 14: L akkt (X S ) = L kkt (AX S ) 15: L total (X S ) = L kkt (X S ) + η • L akkt (X S ) + L prior 16: Update X S ← X S + ∇ X S L total (X S ) 17: Update Λ S ← Λ S + ∇ Λ S L total (X S ) 18:Remove x s i s for corresponding λ s i < 0 19: until X S converges 20: return Set of DSV : X S Justification: See limitation section in Sec. B The limitations are presented in Section \ref{sec:limitations}. The paper notes that the DeepKKT condition's equivalence to traditional support vector representations remains intuitive rather than rigorously proven. It also acknowledges the computational challenges of enforcing all KKT conditions explicitly, especially in high-dimensional deep learning contexts.• The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper. • The authors are encouraged to create a separate "Limitations" section in their paper.• The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be. • The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated. • The authors should reflect on the factors that influence the performance of the approach.For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon. • The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size. • If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness. • While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations. 


Decision: **Summary of the paper:**
The authors define a set of conditions for multiclass neural network classifiers, akin to KKT conditions for SVM, in order to identify or synthesize support vectors. They then take pretrained image classifiers and show that such support vectors look reasonable, are close to the decision boundary, enable few-shot distillation and generative modeling, and can help with interpreting the learnt decision function of a pretrained network.

**Summary of the reviews:**
The reviewers found the approach to connecting pretrained deep nets and SVMs novel and interesting. They also acknowledged the variety and extent of relevant experiments substantiating the support vector behavior of the retrieved/generated samples. 

On the other hand, they point to a lack of (1) experiments on transformer architectures or larger (than ImageNet) datasets, (2) other (global) interpretation methods as baselines, and (3) experiments on sensitivity of support vectors to hyperparameters. Reviewer H6Gu was further concerned with a limited contribution of the paper particularly compared to the direction of Deep SVM works.

**Summary of the rebuttal and discussions:**
The authors clarified some raised points and provided new experiments on transformer architecture with similar results as well as using a network trained on a much larger dataset.

**Consolidation report:**
During the rebuttal the authors added more to the already extensive experimental setup which is satisfactory when it comes to the empirical aspect of the work. The AC further believes the contribution of the paper is enough thanks to the novel perspective –of deep support vectors– that they show enables several interesting use cases in the context of deep learning and then a supporting set of extensive and diverse experiments indicating the potential of the perspective.

However, I think the novelty and theoretical rigor of deep KKT conditions are overblown in light of the prior work on deep SVMs. Therefore, for the next version, I urge the authors to 

- have ample and clear discussion on the differences to works on deep SVMs. 
   -  the AC believes it can be said that the way the paper treats Deep Nets is that the network up to the last layer is a feature extractor $\phi$ and the last (linear) layer is the svm $\tilde{w}$ vector but trained with a different (CE) loss instead of a SVM’s hinge loss and of course it is done in conjunction with learning $\phi$. 
    - Taking this view, how different are the DSVs obtained from a pretrained $\phi$ **and** $w$ compared to SVs coming from only a pretrained $\phi$ and a newly trained $w$ using hinge loss?
    - can we not apply conventional SVM process to obtain SVs from Deep SVMs? How would they differ from the ones obtained using the new conditions from pretrained nets?

- clearly mention the limitation on lacking rigorous derivations of the proposed conditions.
    - make it transparent that the modified conditions are *a* set of conditions that is manually assumed by the authors, with good intuitions about how SVMs and deep networks work, but nevertheless in contrast to the necessary KKT conditions that can be formally derived from a set of assumptions.
    - I suggest to also rephrase “deep KKT” to something that does not explicitly mention KKT so that it does not imply a similar rigor to the conditions to those of KKT. For instance SVs can be mentioned in the term as they don’t imply formal rigor and instead is a property that is to be held for your case as well.

- clearly discuss the computational complexity and report the wall clock time needed to generate support vectors.


**Recommendation:**
The reviewers did not find a fatal error in the paper. I think the paper should be accepted since it has enough contribution in the new perspective and extensive experiments they offer. I believe the presentation needs to be toned down to avoid potential misunderstanding. I trust the authors will do that for a camera ready version.

Reviews: review: paper_summary: paper_summary: This paper introduces Deep Support Vectors (DSVs), an adaptation of support vector concepts to deep learning models. The authors propose a DeepKKT condition, analogous to the KKT conditions in SVMs, to identify or generate DSVs in trained deep models. They demonstrate that DSVs exhibit properties similar to traditional support vectors, including encoding decision boundaries and enabling model reconstruction. The paper shows applications of DSVs in few-shot dataset distillation, model interpretability, and even using classification models as latent generative models. The authors validate their approach on common datasets (ImageNet, CIFAR10, CIFAR100) and architectures (ResNet, ConvNet).
strength_weakness: strength_weakness: - Transformer architectures have not been included in the experiments
- Comparison to other interpretability methods is limited such as global SHAP (mean absolute SHAP value)  or other global methods 
- The proposed method has limitation when it comes to real-scenario applications. Dedicated generative models such as GANs or Transformers are better options.
questions: questions: - How sensitive are the DSVs to the choice of hyperparameters in the DeepKKT condition? Is there a principled way to select these parameters?
- How well do DSVs scale to larger models and datasets? Are there computational challenges in applying this approach to state-of-the-art large language models, for instance?
limitations: limitations: A more in-depth consideration of privacy implications on the generative capability of the method could be given
paper_summary: paper_summary: This paper introduces the DeepKKT conditions for deep svm models, which correspond to the KKT condition in traditional linear SVMs. By either selecting deep support vectors (DSVs) from training data or generating them from already trained deep learning models, The authors show DSVs can play a similar role to conventional support vectors.

In addition,  this paper shows that the DeepKKT condition can transform  conventional classification models into generative models with high fidelity, particularly as latent generative models using class labels as latent variables. The experiments validate the effectiveness of DSVs using common datasets (ImageNet, CIFAR10 and CIFAR100) on the general architectures (ResNet and ConvNet).

Overall, the contribution is limited, hard to follow and I do not recommend for publication at this moment.
strength_weakness: strength_weakness: (1) innovation and contribution
This paper is just deep svms, DeepKKT is known, and its contribution is trivial. Overall, the deep svms here is still a black-box. DeepKKT can relieve the black box issue by introducing deep learning to svms (line 34-36)? The heuristic method to generate DSVs using Eq.9 is not convincing. are they still DSVs?

(2) Technical and theoretical  analysis
In technical level, it may be practical to use Eq. 9 to generate DSVs (either from training data or generated), but it is heuristic.

(3) The paper idea is easy, but the writing is still needs to improve.
questions: questions: (1) For example, I do not follow "Like support vectors can reconstruct SVM, we can reconstruct the deep models from scratch only with DSVs.". How do you reconstruct deep svms from DSVs?

(2) deep learning is black box, and the authors using black box (deep learning here) to relieve this issue?
Is it possible to rewrite the paper to sell its idea on DeepKKT and how to generate DSVs?
limitations: limitations: see above.
paper_summary: paper_summary: The paper describes rethinking a classification through support vector in a way similar to SVMs. It provides additional benefits such as: 
- few-shot dataset distillation
- using the similarity of the proposed formulation to the diffusion models, the model can be transformed into a generative model
strength_weakness: strength_weakness: Clarity: the authors are incouraged to improve the discussion over the limitations.
questions: questions: 1) It would be useful if the authors have any comments on the failure modes of the proposed model. For example, one of the aspects may be the confounding factors (Bontempelli et al (2022)) which could be picked up by the model? 

2) One of the perceived limitations is that the support vectors only represent the whole image. It does not allow to pick up the particular aspects of the image which takes the decision. For example, we can guess from an image of a cat with antlers that these are the antlers which make it look like a deer. But strictly speaking we don't know if such changes result in confirmation bias or whether the model actually predicts the results based on the antlers. I wonder if the authors can comment on such limitation?

3) I could imagine such work is also linked with incremental/lifelong learning, for example in a way similar to Laskov et al (2006), which considers incremental SVM. In contrast to the standard deep-learning solutions, KKT can be updated recursively, which provides a wealth of options for improving the performance. This might be an additional benefit of the proposed formulation. I wonder if the authors can comment on this? 

Bontempelli et al (2022) Concept-level Debugging of Part-Prototype Networks, ICLR 2022

Laskov, P., Gehl, C., Krüger, S., Müller, K.R., Bennett, K.P. and Parrado-Hernández, E., 2006. Incremental support vector learning: Analysis, implementation and applications. Journal of machine learning research, 7(9).
limitations: limitations: As discussed above, the section on limitations would greatly improve the discussion.
paper_summary: paper_summary: This paper introduces a novel Deep Support Vectors (DSVs) framework, which can be used to reconstruct data and serve as latent generative models using logits as latent. By adapting the traditional Karush-Kuhn-Tucker (KKT) condition for deep learning models, the authors introduce the DeepKKT one and show that generated DSVs using this condition exhibit properties similar to traditional support vectors but apply to modern deep architectures like ConvNet and ResNet. Experiments on several common image datasets, including CIFAR10, CIFAR100, SVHN, and ImageNet, verify that DeepKKT can extract and generate DSVs. Moreover, DSVs are shown to be better than existing algorithms in few-shot dataset distillation problems.
strength_weakness: strength_weakness: One of the proposed DSVs' most critical limitations is requiring the model to be fully pre-trained before being applied for support vector generation. This condition differs from the traditional support vector machine (SVM) framework, where the training process of the (linear) model and the support vector selection are jointly performed. Also, it is unclear how the deep model is pre-trained (e.g., with which datasets; are there any domain gap issues between the pre-training data and the main training data, etc.). Furthermore, the margin definition in Eq. (3) is unclear, and I could not understand why the larger margin is better. 

Regarding the total training loss for the DSVs generation in (9), it is unclear about the selections of the augmentation operator $\mathcal A$. Moreover, when training (9) with the traditional SGD approach, how do we guarantee that the generated deep support vectors belong to the original training data set?
questions: questions: Please address my questions in the weaknesses section.
limitations: limitations: N/A
paper_summary: paper_summary: This paper introduces the concept of Deep Support Vectors (DSVs) as an adaptation of support vectors from Support Vector Machines (SVMs) to deep learning models. The authors propose the DeepKKT condition, which generalizes the traditional Karush-Kuhn-Tucker (KKT) conditions of SVMs to handle the high-dimensional and multi-class nature of deep learning problems. By selecting or generating DSVs that satisfy the DeepKKT condition, the authors demonstrate that these vectors can play a similar role to traditional support vectors in terms of encoding decision boundaries and reconstructing models from a small subset of samples. The paper also shows that DSVs can be used for few-shot dataset distillation and as a means to alleviate the black-box characteristics of deep learning models by providing visual explanations of decision criteria. Furthermore, the authors demonstrate that the DeepKKT condition can transform conventional classification models into generative models with high fidelity, using class labels as latent variables.
strength_weakness: strength_weakness: 1. The paper lacks a rigorous mathematical derivation of the DeepKKT condition and its connection to traditional KKT conditions. A more formal treatment would strengthen the theoretical foundation of the proposed method.

2. While the authors provide experimental evidence for the effectiveness of DSVs, a more comprehensive analysis of the method's sensitivity to hyperparameters and its robustness to different architectures and datasets would be beneficial.

3. The paper does not provide a detailed discussion on the computational complexity of generating DSVs and how it scales with the size of the dataset and the complexity of the model.
questions: questions: See Weaknesses.
limitations: limitations: See Weaknesses.

Comments: title: We presumed the request was about algorithmic complexity, not the real wall-clock time
Response to authors
Upgrading my score to 6
we exploit Eq. 9 to meet the condition in Eq. 6
comment: We presumed the request was about algorithmic complexity, not the real wall-clock time. As you mentioned as follows.

  > The paper does not provide a detailed discussion on the computational complexity of generating DSVs and how it scales with the size of the dataset and the complexity of the model.  

This is why we mentioned about computation complexity. In the case of Wall clock time, it is as follows. 

We ran it on a single-A6000 GPU, for the CIFAR10 experiment, it took 30s/img for generating deep support vectors in ConvNET, and about 60s/img when we did it in ResNet. For ImageNet, it took about 3 minutes per image on ResNet. It's important to note that in our case, we focused on the implementation and didn't really consider speed at all. In fact, updating the update algorithm in a super-resolution fashion, (updating both low and high-resolution pixels simultaneously) has actually reduced generation time 1/10. Also, images are generated at once, and the number of generated images are purely proportional to memory, not time. This implies time can be reduced even further by using more memory linearly.
Thanks for the detailed explanations, which address most of my concerns. However,  without the running wall-clock time, I am still concerned about the complexity of the proposed methods. Therefore, I keep my rating as 5.
Thanks to the authors for the detailed responses, especially for clarifying the scope of the paper. Since most of my concerns have been addressed, I decided to upgrade the paper's score to 6.
Upon your point “It could be better if you minimize a function f(x, \theta) x \in manifold, subject to DeepKKT condition.”,

**We clearly stated the connection between manifold condition and Eq. 9 in our paper in line 209-223 (just before Eq. 9)**

The term about augmentation-invariance and image prior is a manifold condition. And this is clearly explained in our paper, just before Eq. 9. For your convenience,

>To extract DSVs from the manifold, we assume that the model is well-trained, meaning it maintains consistent decisions despite data augmentation. In other words, the model should classify DSVs invariantly even after augmentation. To ensure this, we enforce that the augmented DSVs ( $\mathcal{A}(x)$ where $\mathcal{A}$ denotes augmentation function) also meet the primary and stationarity conditions. Also, we exploit traditional image prior [33, 16], total variance $L_{tot}$ and size of the norm $L_{norm}$ to make DSVs lie in the data manifold. $L_{tot}$ is calculated by summing the differences in brightness between neighboring pixels, reducing unnecessary noise in an image, and maintaining a natural appearance. $L_{norm}$, taking a similar role, penalizes the outlier and preserves important pixels.

However, we acknowledge that there exists a room for more clear understanding, so we will change the notation of. Eq. 9 With respect to Eq. 6. Such as, $L_{\text{stationarity}} + \beta_1 L_{\text{primal}} + \beta_2 L_{\text{manifold}$

Upon “My understanding is that DSVs should be on the boundary exactly if we extend svm to deep svm”

**We also explicitly stated the boundary condition in line 233-236**. For your convenience,

Also, we even spare a whole subsection about SVM characteristics in section 5.1. For your convenience, 

>While DeepKKT does not explicitly incorporate the complementary slackness condition due to computational costs and ambiguity, Fig. 2a suggests that DSVs implicitly fulfill this condition; During the training process, we observe an increase in the entropy of DSV candidates, hinting that the generated DSVs are close to the decision boundary.

(the complementary slackness refers boundary condition.)

Also, *please* note that **We do not deal with DeepSVM** and DeepSVM is just an architecture which exploits SVM as a classifier on deep learning model. So  we noted in global response the first rebuttal comment upon your comment, and first reply on your response. Furthermore, we clearly stated in lines 72-76, 8, 56, and the Figure 1 caption. 

Finally, upon your mention “It would be better to sell your DSVs in another name, diffusion vector or support diffusion vector, etc.” We cannot consent your argument, as it is clearly not true, we showed that DSVs have the support vector characteristics. So we think the  name DSVs is the best fit. With following reasons.

1. Meets boundary condition implicitly
2. **Coreset selection** in table 2 as SVM can encode its decision boundary, it itself is useful for a constructing model and has rich information. We showed DSVs can serve as coreset select mechanism in Table.1 and even further it can serve as SOTA dataset distillation algorithm in few-shot dataset distillation.
3. Model explainability: As support vector encode decision boundary, it can explain how does the model decides the class. DSVs also can serve as global explanation within only parametric space.
I know you try to make the whole model reasonable by adding a manifold in Eq. 6 to make Eq. 9 holds while you find DSVs. It could be better if you minimize a function f(x, \theta) x \in manifold, subject to DeepKKT condition. 

My understanding is that DSVs should be on the boundary exactly if we extend svm to deep svm. Then your solution from Eq. 9 or your DSVs is not the DSVs that I talked about. It would be better to sell your DSVs in another name, diffusion vector or support diffusion vector, etc.
Thank you very much for your feedback and strong support of our work. I’m glad our responses have effectively addressed your questions as well as those of the other reviewers.
Formally speaking, DeepSVM refers to SVM that incorporates deep networks as feature extractor. This means it has a pre-trained encoder $\phi$ , which is a deep network. In this process, deep network is **given**. Then, DeepSVM constructs SVM over the encoder. *i.e.,* it builds upon datapoints $\phi(d), \quad d \in \mathcal{D}$. where $\mathcal{D}$ is an original manifold. From this context, deep svm is no more different than normal svm. The only difference is that it uses a mapping function $\phi$ from deep networks. So deep svm condition is just svm condition.

In contrast, our goal is to find vectors which has the characteristics of **support vectors** in already trained deep networks. And we derive a DeepKKT condition to make the vector meet characteristics. Loosely speaking, we view deep learning model (Normal deep networks such as ResNet which classifies an imagenet) as black-box SVM, and find support vector from it. **We define condition first, then, we derive a loss which corresponds to the condition**. And regarding, Eq. 6, we derived the condition in line 149-174 by connecting KKT condition and deep learning dynamics and leave an intuitive analogy in line 454-468, which interprets the stationarity condition in DeepKKT geometrically.  Furthermore, in experiment section,  we **validated** DSVs meet SVM characteristics in line 230-250. 

From this context, we think your question *Do these DSVs still hold after Eq. 9?* may have confused the sequence. **As we exploit Eq. 9 to meet the condition in Eq. 6**
How do you define deep support vectors? I always links it with deep svm conditions. Do these DSVs still hold after Eq. 9?
Many thanks for the rebuttal. I went through the answers to all the reviewers, I think it addresses the reviewers' questions so I stick with the same assessment.