OpenRaiser/Intern-Atlas huggingface dataset:
Stochastic conditional generative networks with basis decomposition
Authors: Ze Wang 0008, Xiuyuan Cheng, Guillermo Sapiro, Qiang Qiu 0001
Venue: Conference ICLR 2020
Year: 2020

While generative adversarial networks (GANs) have revolutionized machine learning, a number of open questions remain to fully understand them and exploit their power. One of these questions is how to efficiently achieve proper diversity and sampling of the multi-mode data space. To address this, we introduce BasisGAN, a stochastic conditional multi-mode image generator. By exploiting the observation that a convolutional filter can be well approximated as a linear combination of a small set of basis elements, we learn a plug-and-played basis generator to stochastically generate basis elements, with just a few hundred of parameters, to fully embed stochasticity into convolutional filters. By sampling basis elements instead of filters, we dramatically reduce the cost of modeling the parameter space with no sacrifice on either image diversity or fidelity. To illustrate this proposed plug-and-play framework, we construct variants of BasisGAN based on state-of-the-art conditional image generation networks, and train the networks by simply plugging in a basis generator, without additional auxiliary components, hyperparameters, or training objectives. The experimental success is complemented with theoretical results indicating how the perturbations introduced by the proposed sampling of basis elements can propagate to the appearance of generated images.
Discovering novel biological traits from images using phylogeny-guided neural networks
Authors: Mohannad Elhamod, Mridul Khurana, Harish Babu Manogaran, Josef C. Uyeda, Meghan A. Balk, Wasila M. Dahdul, Yasin Bakis, Henry L. Bart Jr., Paula M. Mabee, Hilmar Lapp, James P. Balhoff, Caleb Charpentier, David Carlyn, Wei-Lun Chao, Charles V. Stewart, Daniel I. Rubenstein, Tanya Y. Berger-Wolf, Anuj Karpatne
Venue: Conference KDD 2023
Year: 2023

Discovering evolutionary traits that are heritable across species on the tree of life (also referred to as a phylogenetic tree) is of great interest to biologists to understand how organisms diversify and evolve. However, the measurement of traits is often a subjective and labor-intensive process, making trait discovery a highly label-scarce problem. We present a novel approach for discovering evolutionary traits directly from images without relying on trait labels. Our proposed approach, Phylo-NN, encodes the image of an organism into a sequence of quantized feature vectors -- or codes -- where different segments of the sequence capture evolutionary signals at varying ancestry levels in the phylogeny. We demonstrate the effectiveness of our approach in producing biologically meaningful results in a number of downstream tasks including species image generation and species-to-species image translation, using fish species as a target example.
Bidirectional diffusion bridge models
Authors: Duc Kieu, Kien Do, Toan Nguyen, Dang Nguyen, Thin Nguyen
Venue: Conference KDD 2025
Year: 2025

Diffusion bridges have shown potential in paired image-to-image (I2I) translation tasks. However, existing methods are limited by their unidirectional nature, requiring separate models for forward and reverse translations. This not only doubles the computational cost but also restricts their practicality. In this work, we introduce the Bidirectional Diffusion Bridge Model (BDBM), a scalable approach that facilitates bidirectional translation between two coupled distributions using a single network. BDBM leverages the Chapman-Kolmogorov Equation for bridges, enabling it to model data distribution shifts across timesteps in both forward and backward directions by exploiting the interchangeability of the initial and target timesteps within this framework. Notably, when the marginal distribution given endpoints is Gaussian, BDBM's transition kernels in both directions possess analytical forms, allowing for efficient learning with a single network. We demonstrate the connection between BDBM and existing bridge methods, such as Doob's h-transform and variational approaches, and highlight its advantages. Extensive experiments on high-resolution I2I translation tasks demonstrate that BDBM not only enables bidirectional translation with minimal additional cost but also outperforms state-of-the-art bridge models. Our source code is available at [https://github.com/kvmduc/BDBM||https://github.com/kvmduc/BDBM].
Visibility based preconditioning for bundle adjustment
Authors: Avanish Kushal, Sameer Agarwal 0001
Venue: Conference CVPR 2012
Year: 2012

We present Visibility Based Preconditioning (VBP) a new technique for efficiently solving the linear least squares problems that arise in bundle adjustment (Triggs et al., 1999). Using the camera-point visibility structure of the scene, we describe the construction of two preconditioners. These preconditioners when combined with an inexact step Levenberg-Marquardt algorithm (Wright and Holt, 1985) offer state of the art performance on the BAL data set (Agarwal et al., 2010), with 3-5× reduction in execution time over currently available methods while delivering comparable or better solution quality.
Learning weights for codebook in image classification and retrieval
Authors: Hongping Cai, Fei Yan 0001, Krystian Mikolajczyk
Venue: Conference CVPR 2010
Year: 2010

This paper presents a codebook learning approach for image classification and retrieval. It corresponds to learning a weighted similarity metric to satisfy that the weighted similarity between the same labeled images is larger than that between the differently labeled images with largest margin. We formulate the learning problem as a convex quadratic programming and adopt alternating optimization to solve it efficiently. Experiments on both synthetic and real datasets validate the approach. The codebook learning improves the performance, in particular in the case where the number of training examples is not sufficient for large size codebook.
Learning non-redundant codebooks for classifying complex objects
Authors: Wei Zhang 0014, Akshat Surve, Xiaoli Z. Fern, Thomas G. Dietterich
Venue: Conference ICML 2009
Year: 2009

Codebook-based representations are widely employed in the classification of complex objects such as images and documents. Most previous codebook-based methods construct a single codebook via clustering that maps a bag of low-level features into a fixed-length histogram that describes the distribution of these features. This paper describes a simple yet effective framework for learning multiple non-redundant codebooks that produces surprisingly good results. In this framework, each codebook is learned in sequence to extract discriminative information that was not captured by preceding codebooks and their corresponding classifiers. We apply this framework to two application domains: visual object categorization and document classification. Experiments on large classification tasks show substantial improvements in performance compared to a single codebook or codebooks learned in a bagging style.
Temporal difference variational auto-encoder
Authors: Karol Gregor, George Papamakarios, Frederic Besse, Lars Buesing, Theophane Weber
Venue: Conference ICLR 2019
Year: 2019

To act and plan in complex environments, we posit that agents should have a mental simulator of the world with three characteristics: (a) it should build an abstract state representing the condition of the world; (b) it should form a belief which represents uncertainty on the world; (c) it should go beyond simple step-by-step simulation, and exhibit temporal abstraction. Motivated by the absence of a model satisfying all these requirements, we propose TD-VAE, a generative sequence model that learns representations containing explicit beliefs about states several steps into the future, and that can be rolled out directly without single-step transitions. TD-VAE is trained on pairs of temporally separated time points, using an analogue of temporal difference learning used in reinforcement learning.
Shapleyfl: Robust federated learning based on shapley value
Authors: Qiheng Sun, Xiang Li 0001, Jiayao Zhang 0006, Li Xiong 0001, Weiran Liu, Jinfei Liu, Zhan Qin, Kui Ren 0001
Venue: Conference KDD 2023
Year: 2023

Federated Learning (FL) allows clients to form a consortium to train a global model under the orchestration of a central server while keeping data on the local client without sharing it, thus mitigating data privacy issues. However, training a robust global model is challenging since the local data is invisible to the server. The local data of clients are naturally heterogeneous, while some clients can use corrupted data or send malicious updates to interfere with the training process artificially. Meanwhile, communication and computation costs are inevitable challenges in designing a practical FL algorithm. In this paper, to improve the robustness of FL, we propose a Shapley value-inspired adaptive weighting mechanism, which regards the FL training as sequential cooperative games and adjusts clients' weights according to their contributions. We also develop a client sampling strategy based on importance sampling, which can reduce the communication cost by optimizing the variance of the global updates according to the weights of clients. Furthermore, to diminish the computation cost of the server, we propose a weight calculation method by estimating differences between the Shapley value of clients. Our experimental results on several real data sets demonstrate the effectiveness of our approaches.
Discrete minimum ratio curves and surfaces
Authors: Fred Nicolls, Philip H. S. Torr
Venue: Conference CVPR 2010
Year: 2010

Graph cuts have proven useful for image segmentation and for volumetric reconstruction in multiple view stereo. However, solutions are biased: the cost function tends to favour either a short boundary (in 2D) or a boundary with a small area (in 3D). This bias can be avoided by instead minimising the cut ratio, which normalises the cost by a measure of the boundary size. This paper uses ideas from discrete differential geometry to develop a linear programming formulation for finding a minimum ratio cut in arbitrary dimension, which allows constraints on the solution to be specified in a natural manner, and which admits an efficient and globally optimal solution. Results are shown for 2D segmentation and for 3D volumetric reconstruction.
Fast matching of planar shapes in sub-cubic runtime
Authors: Frank R. Schmidt, Dirk Farin, Daniel Cremers
Venue: Conference ICCV 2007
Year: 2007

The matching of planar shapes can be cast as a problem of finding the shortest path through a graph spanned by the two shapes, where the nodes of the graph encode the local similarity of respective points on each contour. While this problem can be solved using dynamic time warping, the complete search over the initial correspondence leads to cubic runtime in the number of sample points. In this paper, we cast the shape matching problem as one of finding the shortest circular path on a torus. We propose an algorithm to determine this shortest cycle which has provably sub-cubic runtime. Numerical experiments demonstrate that the proposed algorithm provides faster shape matching than previous methods. As an application, we show that it allows to efficiently compute a clustering of a shape data base.
Globally optimal image segmentation with an elastic shape prior
Authors: Thomas Schoenemann, Daniel Cremers
Venue: Conference ICCV 2007
Year: 2007

So far global optimization techniques have been developed independently for the tasks of shape matching and image segmentation. In this paper we show that both tasks can in fact be solved simultaneously using global optimization. By computing cycles of minimal ratio in a large graph spanned by the product of the input image and a shape template, we are able to compute globally optimal segmentations of the image which are similar to a familiar shape and located in places of strong gradient. The presented approach is translation-invariant and robust to local and global scaling and rotation of the given shape. We show how it can be extended to incorporate invariance to similarity transformations. The particular structure of the graph allows for run-time and memory efficient implementations. Highly parallel implementations on graphics cards allow to produce globally optimal solutions in a few seconds only.
Introducing curvature into globally optimal image segmentation: Minimum ratio cycles on product graphs
Authors: Thomas Schoenemann, Daniel Cremers
Venue: Conference ICCV 2007
Year: 2007

While the majority of competitive image segmentation methods are based on energy minimization, only few allow to efficiently determine globally optimal solutions. A graph-theoretic algorithm for finding globally optimal segmentations is given by the minimum ratio cycles, first applied to segmentation by Jermyn and Ishikawa (2001). In this paper we show that the class of image segmentation problems solvable by minimum ratio cycles is significantly larger than previously considered. In particular, they allow for the introduction of higher-order regularity of the region boundary. The key idea is to introduce an extended graph representation, where each node of the graph represents an image pixel as well as the orientation of the incoming line segment. With each graph edge representing a pair of adjacent line segments, edge weights can depend on the curvature. This way arbitrary positive functions of curvature can be introduced into globally optimal segmentation by minimum ratio cycles. In numerous experiments we demonstrate that compared to length-regularity the integration of curvature-regularity will drastically improve segmentation results. Moreover, we show an interesting relation to the snakes functional: minimum ratio cycles provide a way to find one of the few cases where the snakes functional has a meaningful global minimum.
Modeling deformable objects from a single depth camera
Authors: Miao Liao, Qing Zhang 0017, Huamin Wang 0001, Ruigang Yang, Minglun Gong
Venue: Conference ICCV 2009
Year: 2009

We propose a novel approach to reconstruct complete 3D deformable models over time by a single depth camera, provided that most parts of the models are observed by the camera at least once. The core of this algorithm is based on the assumption that the deformation is continuous and predictable in a short temporal interval. While the camera can only capture part of a whole surface at any time instant, partial surfaces reconstructed from different times are assembled together to form a complete 3D surface for each time instant, even when the shape is under severe deformation. A mesh warping algorithm based on linear mesh deformation is used to align different partial surfaces. A volumetric method is then used to combine partial surfaces, fix missing holes, and smooth alignment errors. Our experiment shows that this approach is able to reconstruct visually plausible 3D surface deformation results with a single camera.
Learning specific-class segmentation from diverse data
Authors: M. Pawan Kumar, Haithem Turki, Dan Preston, Daphne Koller
Venue: Conference ICCV 2011
Year: 2011

We consider the task of learning the parameters of a segmentation model that assigns a specific semantic class to each pixel of a given image. The main problem we face is the lack of fully supervised data. We address this issue by developing a principled framework for learning the parameters of a specific-class segmentation model using diverse data. More precisely, we propose a latent structural support vector machine formulation, where the latent variables model any missing information in the human annotation. Of particular interest to us are three types of annotations: (i) images segmented using generic foreground or background classes; (ii) images with bounding boxes specified for objects; and (iii) images labeled to indicate the presence of a class. Using large, publicly available datasets we show that our approach is able to exploit the information present in different annotations to improve the accuracy of a state-of-the art region-based model.
BERT-QPP: Contextualized Pre-trained transformers for Query Performance Prediction
Authors: Negar Arabzadeh, Maryam Khodabakhsh, Ebrahim Bagheri
Venue: Conference CIKM 2021
Year: 2021

Query Performance Prediction (QPP) is focused on estimating the difficulty of satisfying a user query for a certain retrieval method. While most state of the art QPP methods are based on term frequency and corpus statistics, more recent work in this area have started to explore the utility of pretrained neural embeddings, neural architectures and contextual embeddings. Such approaches extract features from pretrained or contextual embeddings for the sake of training a supervised performance predictor. In this paper, we adopt contextual embeddings to perform performance prediction, but distinguish ourselves from the state of the art by proposing to directly fine-tune a contextual embedding, i.e., BERT, specifically for the task of query performance prediction. As such, our work allows the fine-tuned contextual representations to estimate the performance of a query based on the association between the representation of the query and the retrieved documents. We compare the performance of our approach with the state-of-the-art based on the MS MARCO passage retrieval corpus and its three associated query sets: (1) MS MARCO development set, (2) TREC DL 2019, and (3) TREC DL 2020. We show that our approach not only shows significant improved prediction performance compared to all the state-of-the-art methods, but also, unlike past neural predictors, it shows significantly lower latency, making it possible to use in practice.
Improved query performance prediction using standard deviation
Authors: Ronan Cummins, Joemon M. Jose, Colm O&apos, Riordan
Venue: Conference SIGIR 2011
Year: 2011

Query performance prediction (QPP) is an important task in information retrieval (IR). In this paper, we (1) develop a new predictor based on the standard deviation of scores in a variable length ranked list, and (2) we show that this new predictor outperforms state-of-the-art approaches without the need for tuning.
A survey of pre-retrieval query performance predictors
Authors: Claudia Hauff, Djoerd Hiemstra, Franciska de Jong
Venue: Conference CIKM 2008
Year: 2008

The focus of research on query performance prediction is to predict the effectiveness of a query given a search system and a collection of documents. If the performance of queries can be estimated in advance of, or during the retrieval stage, specific measures can be taken to improve the overall performance of the system. In particular, pre-retrieval predictors predict the query performance before the retrieval step and are thus independent of the ranked list of results; such predictors base their predictions solely on query terms, the collection statistics and possibly external sources such as WordNet. In this poster, 22 pre-retrieval predictors are categorized and assessed on three different TREC test collections.
Query-performance prediction: setting the expectations straight
Authors: Fiana Raiber, Oren Kurland
Venue: Conference SIGIR 2014
Year: 2014

The query-performance prediction task has been described as estimating retrieval effectiveness in the absence of relevance judgments. The expectations throughout the years were that improved prediction techniques would translate to improved retrieval approaches. However, this has not yet happened. Herein we provide an in-depth analysis of why this is the case. To this end, we formalize the prediction task in the most general probabilistic terms. Using this formalism we draw novel connections between tasks --- and methods used to address these tasks --- in federated search, fusion-based retrieval, and query-performance prediction. Furthermore, using formal arguments we show that the ability to estimate the probability of effective retrieval with no relevance judgments (i.e., to predict performance) implies knowledge of how to perform effective retrieval. We also explain why the expectation that using previously proposed query-performance predictors would help to improve retrieval effectiveness was not realized. This is due to a misalignment with the actual goal for which these predictors were devised: ranking queries based on the presumed effectiveness of using them for retrieval over a corpus with a specific retrieval method. Focusing on this specific prediction task, namely query ranking by presumed effectiveness, we present a novel learning-to-rank-based approach that uses Markov Random Fields. The resultant prediction quality substantially transcends that of state-of-the-art predictors.
Unsupervised query performance prediction for neural models with pairwise rank preferences
Authors: Ashutosh Singh, Debasis Ganguly, Suchana Datta, Craig MacDonald
Venue: Conference SIGIR 2023
Year: 2023

A query performance prediction (QPP) method predicts the effectiveness of an IR system for a given query. While unsupervised approaches have been shown to work well for statistical IR models, it is likely that these approaches would yield limited effectiveness for neural ranking models (NRMs) because the retrieval scores of these models lie within a short range unlike their statistical counterparts. In this work, we propose to leverage a pairwise inference-based NRM's (specifically, DuoT5) output to accumulate evidences on the pairwise believes of one document ranked above the other. We hypothesize that the more consistent these pairwise likelihoods are, the higher is the likelihood of the retrieval to be of better quality, thus yielding a higher QPP score. We conduct our experiments on the TREC-DL dataset leveraging pairwise likelihoods from an auxiliary model DuoT5. Our experiments demonstrate that the proposed method called Pairwise Rank Preference-based QPP (QPP-PRP) leads to significantly better results than a number of standard unsupervised QPP baselines on several NRMs.
Nightmare at test time: robust learning by feature deletion
Authors: Amir Globerson, Sam T. Roweis
Venue: Conference ICML 2006
Year: 2006

When constructing a classifier from labeled data, it is important not to assign too much weight to any single input feature, in order to increase the robustness of the classifier. This is particularly important in domains with nonstationary feature distributions or with input sensor failures. A common approach to achieving such robustness is to introduce regularization which spreads the weight more evenly between the features. However, this strategy is very generic, and cannot induce robustness specifically tailored to the classification task at hand. In this work, we introduce a new algorithm for avoiding single feature over-weighting by analyzing robustness using a game theoretic formalization. We develop classifiers which are optimally resilient to deletion of features in a minimax sense, and show how to construct such classifiers using quadratic programming. We illustrate the applicability of our methods on spam filtering and handwritten digit recognition tasks, where feature deletion is indeed a realistic noise model.
Are adversarial examples inevitable?
Authors: Ali Shafahi, W. Ronny Huang, Christoph Studer, Soheil Feizi, Tom Goldstein
Venue: Conference ICLR 2019
Year: 2019

A wide range of defenses have been proposed to harden neural networks against adversarial attacks. However, a pattern has emerged in which the majority of adversarial defenses are quickly broken by new attacks. Given the lack of success at generating robust defenses, we are led to ask a fundamental question: Are adversarial attacks inevitable? This paper analyzes adversarial examples from a theoretical perspective, and identifies fundamental bounds on the susceptibility of a classifier to adversarial attacks. We show that, for certain classes of problems, adversarial examples are inescapable. Using experiments, we explore the implications of theoretical guarantees for real-world problems and discuss how factors such as dimensionality and image complexity limit a classifier's robustness against adversarial examples.