OpenRaiser/Intern-Atlas huggingface dataset:
Revisit targeted model poisoning on federated recommendation: Optimize via multi-objective transport
Authors: Jiajie Su, Chaochao Chen, Weiming Liu, Zibin Lin, Shuheng Shen, Weiqiang Wang, Xiaolin Zheng
Venue: Conference SIGIR 2024
Year: 2024

Federated Recommendation (FedRec) is popularly investigated in personalized recommenders for preserving user privacy. However, due to the distributed training paradigm, FedRec is vulnerable to model poisoning attacks. In this paper, we focus on the targeted model poisoning attack against FedRec, which aims at effectively attacking the FedRec via uploading poisoned gradients to raise the exposure ratio of a multi-target item set. Previous attack methods excel with fewer target items but suffer performance decline as the amount of target items increases, which reveals two perennially neglected issues: (i) The simple promotion of prediction scores without considering intrinsic collaborations between users and items is ineffective in multi-target cases. (ii) Target items are heterogeneous, which requires discriminative attacking users and strategies for different targets. To address the issues, we propose a novel Heterogeneous Multi-target Transfer Attack framework named HMTA which consists of two stages, i.e., (1) diverse user agent generation and (2) optimal multi-target transport attack. The former stage leverages collaboration-aware manifold learning to extract latent associations among users and items, and develops a differentiable contrastive sorting to generate user agents from both difficulty and diversity scale. The latter stage conducts poisoning in a fine-grained and distinguishing way, which first completes distribution mapping from target items to generated user agents and then achieves a hybrid multi-target attack. Extensive experiments on benchmark datasets demonstrate the effectiveness of HMTA.
FedSecurity: A benchmark for attacks and defenses in federated learning and federated LLMs
Authors: Shanshan Han, Baturalp Buyukates, Zijian Hu 0001, Han Jin, Weizhao Jin, Lichao Sun 0001, Xiaoyang Wang, Wenxuan Wu, Chulin Xie, Yuhang Yao 0003, Kai Zhang 0039, Qifan Zhang 0002, Yuhui Zhang, Carlee Joe-Wong, Salman Avestimehr, Chaoyang He 0001
Venue: Conference KDD 2024
Year: 2024

This paper introduces FedSecurity, an end-to-end benchmark that serves as a supplementary component of the FedML library for simulating adversarial attacks and corresponding defense mechanisms in Federated Learning (FL). FedSecurity eliminates the need for implementing the fundamental FL procedures, e.g., FL training and data loading, from scratch, thus enables users to focus on developing their own attack and defense strategies. It contains two key components, including FedAttacker that conducts a variety of attacks during FL training, and FedDefender that implements defensive mechanisms to counteract these attacks. FedSecurity has the following features: i) It offers extensive customization options to accommodate a broad range of machine learning models (e.g., Logistic Regression, ResNet, and GAN) and FL optimizers (e.g., FedAVG, FedOPT, and FedNOVA); ii) it enables exploring the effectiveness of attacks and defenses across different datasets and models; and iii) it supports flexible configuration and customization through a configuration file and some APIs. We further demonstrate FedSecurity's utility and adaptability through federated training of Large Language Models (LLMs) to showcase its potential on a wide range of complex applications.
A theory of image matching
Authors: Juyang Weng
Venue: Conference ICCV 1990
Year: 1990

A theoretical framework is presented in which windowed Fourier phase (WFP) is introduced as the primary matching primitive. Zero-crossings and peaks correspond to special values in the phase profile. The WFP is quasi-linear, dense and its spatial period and slope are controlled by the channel scale. This framework has the following important characteristics: matching primitives are available almost everywhere to convey dense disparity information in every channel, either coarse or fine; the false target problem is virtually eliminated; matching is easier, uniform and can be performed by a network suitable for parallel computer architecture; and matching is fast since very few iterations are needed. In fact, the WFP is so informative that the original signal can be uniquely determined up to a multiplicative constant by the WFP in any one channel. The use of phase as the matching primitive is also supported by some existing psychophysical and neurophysiological studies. An implementation of the proposed theory has shown good results from images of random dots and natural scenes.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">></ETX>
Probability distributions of optical flow
Authors: Eero P. Simoncelli, Edward H. Adelson, David J. Heeger
Venue: Conference CVPR 1991
Year: 1991

Gradient methods are widely used in the computation of optical flow. The authors discuss extensions of these methods which compute probability distributions of optical flow. The use of distributions allows representation of the uncertainties inherent in the optical flow computation, facilitating the combination with information from other sources. Distributed optical flow for a synthetic image sequence is computed, and it is demonstrated that the probabilistic model accounts for the errors in the flow estimates. The distributed optical flow for a real image sequence is computed.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">></ETX>
Stereo from Uncalibrated Cameras
Authors: Richard I. Hartley, Rajiv Gupta 0002, Tom Chang
Venue: Conference CVPR 1992
Year: 1992

The problem of computing placement of points in 3-D space, given two uncalibrated perspective views, is considered. The main theorem shows that the placement of the points is determined only up to an arbitrary projective transformation of 3-space. Given additional ground control points, however, the location of the points and the camera parameters may be determined. The method is linear and noniterative, whereas previously known methods for solving the camera calibration and placement problem to take proper account of both ground-control points and image correspondences are unsatisfactory in requiring either iterative methods or model restrictions. As a result of the main theorem, it is possible to determine projective invariants of 3-D geometric configurations from two perspective views.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">></ETX>
Exploiting the Generic View Assumption to Estimate Scene Parameters
Authors: William T. Freeman
Venue: Conference ICCV 1993
Year: 1993

The generic view assumption states that an observer is not in a special position relative to the scene. The author shows how to use the generic view assumption to quantify the likelihood of a view and so derive a powerful tool to estimate scene parameters. Generic variables can include viewpoint, object orientation, and lighting position. By considering the image as a function of these variables, the probability that a set of scene parameters created a given image is derived. This scene probability equation includes a term which rewards fidelity to the image, and a generic view term, which favors scenes likely to produce the observed image. Large derivatives of the image with respect to the generic variables correspond to unlikely scenes. This approach reduces the dependence on prior assumptions and should increase the scope and accuracy of scene estimates. This framework applies to many vision problems. Several shape from shading examples are shown, including the estimation of reflectance function, light direction, and vertical scale, in cases where these are otherwise unknown.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">></ETX>
Expected Performance of Robust Estimators Near Discontinuities
Authors: Charles V. Stewart
Venue: Conference ICCV 1995
Year: 1995

In extracting a polynomial surface patch near an intensity or range discontinuity, a robust estimator must tolerate not only the truly random bad data ("random outliers"), but also the coherently structured points ("pseudo outliers") that belong to a different surface. To characterize the performance of least median of squares, M estimators, Hough transforms, RANSAC, and MINPRAN on data containing both random and pseudo outliers, we develop two analytical measures, "pseudo outlier bias" and "pseudo outlier breakdown". Using these measures, we find that each robust estimator has surprisingly poor performance, even under the best possible circumstances, implying that present estimators should be used with care and new estimators should be developed.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">></ETX>
An Improved Algorithm for Algebraic Curve and Surface Fitting
Authors: Gabriel Taubin
Venue: Conference ICCV 1993
Year: 1993

The author describes a new method to improve the algebraic surface fitting process by better approximating the Euclidean distance from a point to the surface. In the past they have used a simple first order approximation of the Euclidean distance from a point to an implicit curve or surface which yielded good results in the case of unconstrained algebraic curves or surfaces, and reasonable results in the case of bounded algebraic curves and surfaces. However, experiments with the exact Euclidean distance have shown the limitations of this simple approximation. Here, a more complex, and better, approximation to the Euclidean distance is introduced from a point to an alegbraic curve or surface. It is shown that this new approximate distance produces results of the same quality as those based on the exact Euclidean distance, and much better than those obtained using other available methods.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">></ETX>
Multi-Scale Feature Selection in Stereo
Authors: Hiroshi Ishikawa 0002
Venue: Conference CVPR 1999
Year: 1999

In binocular stereo matching, points in left and right images are matched according to features that characterize each point and identify pairs of points. When one tries to use multiple features, a difficult problem is which feature, or combination of features, to use. Moreover, features are difficult to cross-normalize and so comparisons must take into account not only their output, but also their distribution (their output for different parameters). We present a new approach that uses geometric constraints on the matching surface to select optimal feature or combination of features from multiscale-edge and intensity features. The approach requires the cyclopean coordinate system to set mutually exclusive matching choices. To obtain the matching surface, we solve a global optimization problem on an energy functional that models occlusions, discontinuities, and inter-epipolar-line interactions.
Calibration with Robust Use of Cheirality by Quasi- Affine Reconstruction of the Set of Camera Projection Centres
Authors: David Nistér
Venue: Conference ICCV 2001
Year: 2001

A method for upgrading a projective reconstruction to metric is presented. The reconstruction is first transformed by considering cheirality so that the convex hull of the set of camera projection centres is the same as in the metric counterpart. The method then proceeds iteratively and starting from such a reconstruction is a necessary condition for many iterative calibration algorithms to converge. The results show that in practice it is also most often sufficient provided that the minimised objective function is a geometrically meaningful quantity. The method has been found extremely reliable for both large and small reconstructions in a large number of experiments on real data. When subjected to the common degeneracy of little or no rotation between the views, the method still yields a very reasonable member of the family of possible solutions. Furthermore, the method is very fast and therefore suitable for the purpose of viewing reconstructions.
Catadioptric self-calibration
Authors: Sing Bing Kang
Venue: Conference CVPR 2000
Year: 2000

We have assembled a standalone, movable system that can capture long sequences of omnidirectional images (up to 1,500 images at 6.7 Hz and a resolution of 1140/spl times/090). The goal of this system is to reconstruct complex large environments, such as an entire floor of a building, from the captured images only. In this paper, we address the important issue of how to calibrate such a system. Our method uses images of the environment to calibrate the camera, without the use of any special calibration pattern, knowledge of camera motion, or knowledge of scene geometry. It uses the consistency of pairwise tracked point features across a sequence based on the characteristics of catadioptric imaging. We also show how the projection equation for this catadioptric camera can be formulated to be equivalent to that of a typical rectilinear perspective camera with just a simple transformation.
Recognizing Expression Variant Faces from a Single Sample Image per Class
Authors: Aleix M. Martínez
Venue: Conference CVPR 2003
Year: 2003

Although important contributions to face recognition have been reported, few focus on how to robustly recognize expression variant faces from as few as one single training sample per class. Since learning cannot generally be applied when only one sample per class is available, matching techniques (distance measures) are usually employed instead (e.g. correlations). However, distance measures generally attempt to match all features with equal importance (weighting), because not only is it difficult to know which features are more useful (for classification), but when or under which circumstances this happens. For example, when recognizing faces in the original image space (e.g. using the Euclidean distance-correlation), it is not known which pixels are more and which are less appropriate for use. We use the optical flow between the testing and sample images as a measure of how good each pixel is. Pixels that have a small flow will have high weights, pixels with a large flow will have small weights. Our experimental results show that the method proposed in this contribution outperforms the classical Euclidean distance (correlation) measure and the PCA (principal component analysis) approach.
Nonparametric information fusion for motion estimation
Authors: Dorin Comaniciu
Venue: Conference CVPR 2003
Year: 2003

The problem of information fusion appears in many forms in vision. Tasks such as motion estimation, multimodal registration, tracking, and robot localization, often require the synergy of estimates coming from multiple sources. Most of the fusion algorithms, however, assume a single source model and are not robust to outliers. If the data to be fused follow different underlying models, the traditional algorithms would produce poor estimates. We present in this paper a nonparametric approach to information fusion called variable-bandwidth density-based fusion (VBDF). The fusion estimator is computed as the location of the most significant mode of a density function, which takes into account the uncertainty of the estimates to be fused. A mode detection scheme is presented, which relies on variable-bandwidth mean shift computed at multiple scales. We show that the proposed estimator is consistent and conservative, while handling naturally outliers in the data and multiple source models. The new theory is tested for the task of multiple motion estimation. Numerous experiments validate the theory and provide very competitive results.
Tracking Articulated Objects in Real-Time Range Image Sequences
Authors: Michael H. Lin
Venue: Conference ICCV 1999
Year: 1999

A simple algorithm for tracking the pose of articulated objects in real-time range image sequences is proposed. This method models each target segment as a planar patch bounded by the convex hull of two circles, and utilizes both edge-like and region-like information in matching the model to the target. It uses hard constraints for joint attachment and is designed to be robust to occlusions and missing data. Experimental results are presented in which a human arm is successfully tracked over 26 frames of real, video-rate range imagery.
Explaining Optical Flow Events with Parameterized Spatio- Temporal Models
Authors: Michael J. Black
Venue: Conference CVPR 1999
Year: 1999

A spatio-temporal representation for complex optical flow events is developed that generalizes traditional parameterized motion models (e.g. affine). These generative spatio-temporal models may be non-linear or stochastic and are event-specific in that they characterize a particular type of object motion (e.g. sitting or walking). Within a Bayesian framework we seek the appropriate model, phase, rate, spatial position, and scale to account for the image variation. The posterior distribution over this parameter space conditioned on image measurements is typically non-Gaussian. The distribution is represented using factored sampling and is predicted and updated over time using the condensation algorithm. The resulting framework automatically detects, localizes, and recognizes motion events.
Efficient Computations via Scalable Sparse Kernel Partial Least Squares and Boosted Latent Features
Authors: Michinari Momma
Venue: Conference KDD 2005
Year: 2005

Kernel partial least squares (KPLS) has been known as a generic kernel regression method and proven to be competitive with other kernel regression methods such as support vector machines for regression (SVM) and kernel ridge regression. Kernel boosted latent features (KBLF) is a variant of KPLS for any differentiable convex loss functions. It provides a more flexible framework for various predictive modeling tasks such as classification with logistic loss and robust regression with L1 norm loss, etc. However, KPLS and KBLF solutions are dense and thus not suitable for large-scale computations. Sparsification of KPLS solutions has been studied for dual and primal forms. For dual sparsity, it requires solving a nonlinear optimization problem at every iteration step and its computational burden limits its applicability to general regression tasks.In this paper, we propose simple heuristics to approximate sparse solutions for KPLS and the framework is also applied for sparsifying KBLF solutions. The algorithm provides an interesting "path" from a maximum residual criterion based algorithm with orthogonality conditions to the dense KPLS/KBLF. With the orthogonality, it differentiates itself from many existing forward selection-type algorithms. The computational advantage is illustrated by benchmark datasets and comparison to SVM is done.
Higher-Order Gradient Descent by Fusion-Move Graph Cut
Authors: Hiroshi Ishikawa 0002
Venue: Conference ICCV 2009
Year: 2009

Markov Random Field is now ubiquitous in many formulations of various vision problems. Recently, optimization of higher-order potentials became practical using higher-order graph cuts: the combination of i) the fusion move algorithm, ii) the reduction of higher-order binary energy minimization to first-order, and iii) the QPBO algorithm. In the fusion move, it is crucial for the success and efficiency of the optimization to provide proposals that fits the energies being optimized. For higher-order energies, it is even more so because they have richer class of null potentials. In this paper, we focus on the efficiency of the higher-order graph cuts and present a simple technique for generating proposal labelings that makes the algorithm much more efficient, which we empirically show using examples in stereo and image denoising.
Detecting objects in large image collections and videos by efficient subimage retrieval
Authors: Christoph H. Lampert
Venue: Conference ICCV 2009
Year: 2009

We study the task of detecting the occurrence of objects in large image collections or in videos, a problem that combines aspects of content based image retrieval and object localization. While most previous approaches are either limited to special kinds of queries, or do not scale to large image sets, we propose a new method, efficient subimage retrieval (ESR), that is at the same time very flexible and very efficient. Relying on a two-layered branch-and-bound setup, ESR performs object-based image retrieval in sets of 100,000 or more images within seconds. An extensive evaluation on several datasets shows that ESR is not only very fast, but it also achieves excellent detection accuracies thereby improving over previous systems for object-based image retrieval.
Face Recognition Based on Nearest Linear Combinations
Authors: Stan Z. Li
Venue: Conference CVPR 1998
Year: 1998

This paper proposes a novel pattern classification approach, called the nearest linear combination (NLC) approach, for eigenface based face recognition. Assume that multiple prototypical vectors are available per class, each vector being a point in an eigenface space. A linear combination of prototypical vectors belonging to a face class is used to define a measure of distance from the query vector to the class, the measure being defined as the Euclidean distance from the query to the linear combination nearest to the query vector (hence NLC). This contrasts to the nearest neighbor (NN) classification where a query vector is compared with each prototypical vector individually. Using a linear combination of prototypical vectors, instead of each of them individually, extends the representational capacity of the prototypes by generalization through interpolation and extrapolation. Experiments show that it leads to better results than existing classification methods.
Background estimation as a labeling problem
Authors: Scott Cohen
Venue: Conference ICCV 2005
Year: 2005

We present a new background estimation algorithm that constructs the background of an image sequence with moving objects by copying areas from input frames. The background estimation problem is formulated as an optimal labeling problem in which the label at an output pixel is the frame number from which to copy the background color. The costs of assigning labels encourage seamless copying from regions that are stationary over a period of time in such a way that implied motion boundaries occur at intensity edges. This is accomplished without explicitly tracking the moving objects or computing optical flow. Experiments demonstrate that our algorithm is effective in difficult areas where the background is visible for only a small fraction of time, and on inputs with both moving objects that are not always in motion and moving objects with textureless areas
Face alignment by explicit shape regression
Authors: Xudong Cao, Yichen Wei, Fang Wen 0001, Jian Sun 0001
Venue: Conference CVPR 2012
Year: 2012

We present a very efficient, highly accurate, “Explicit Shape Regression” approach for face alignment. Unlike previous regression-based approaches, we directly learn a vectorial regression function to infer the whole facial shape (a set of facial landmarks) from the image and explicitly minimize the alignment errors over the training data. The inherent shape constraint is naturally encoded into the regressor in a cascaded learning framework and applied from coarse to fine during the test, without using a fixed parametric shape model as in most previous methods. To make the regression more effective and efficient, we design a two-level boosted regression, shape-indexed features and a correlation-based feature selection method. This combination enables us to learn accurate models from large training data in a short time (20 minutes for 2,000 training images), and run regression extremely fast in test (15 ms for a 87 landmarks shape). Experiments on challenging data show that our approach significantly outperforms the state-of-the-art in terms of both accuracy and efficiency.