OpenRaiser/Intern-Atlas huggingface dataset:
SeCGAN: Parallel Conditional Generative Adversarial Networks for Face Editing via Semantic Consistency
Venue: arXiv cs.CV 2022
Year: 2022

Semantically guided conditional Generative Adversarial Networks (cGANs) have become a popular approach for face editing in recent years. However, most existing methods introduce semantic masks as direct conditional inputs to the generator and often require the target masks to perform the corresponding translation in the RGB space. We propose SeCGAN, a novel label-guided cGAN for editing face images utilising semantic information without the need to specify target semantic masks. During training, SeCGAN has two branches of generators and discriminators operating in parallel, with one trained to translate RGB images and the other for semantic masks. To bridge the two branches in a mutually beneficial manner, we introduce a semantic consistency loss which constrains both branches to have consistent semantic outputs. Whilst both branches are required during training, the RGB branch is our primary network and the semantic branch is not needed for inference. Our results on CelebA and CelebA-HQ demonstrate that our approach is able to generate facial images with more accurate attributes, outperforming competitive baselines in terms of Target Attribute Recognition Rate whilst maintaining quality metrics such as self-supervised Fréchet Inception Distance and Inception Score.
Temporally Consistent Online Depth Estimation in Dynamic Scenes
Venue: arXiv cs.CV 2022
Year: 2022

Temporally consistent depth estimation is crucial for online applications such as augmented reality. While stereo depth estimation has received substantial attention as a promising way to generate 3D information, there is relatively little work focused on maintaining temporal stability. Indeed, based on our analysis, current techniques still suffer from poor temporal consistency. Stabilizing depth temporally in dynamic scenes is challenging due to concurrent object and camera motion. In an online setting, this process is further aggravated because only past frames are available. We present a framework named Consistent Online Dynamic Depth (CODD) to produce temporally consistent depth estimates in dynamic scenes in an online setting. CODD augments per-frame stereo networks with novel motion and fusion networks. The motion network accounts for dynamics by predicting a per-pixel SE3 transformation and aligning the observations. The fusion network improves temporal depth consistency by aggregating the current and past estimates. We conduct extensive experiments and demonstrate quantitatively and qualitatively that CODD outperforms competing methods in terms of temporal consistency and performs on par in terms of per-frame accuracy.
High Quality Rather than High Model Probability: Minimum Bayes Risk Decoding with Neural Metrics
Venue: arXiv cs.CL 2022
Year: 2022

In Neural Machine Translation, it is typically assumed that the sentence with the highest estimated probability should also be the translation with the highest quality as measured by humans. In this work, we question this assumption and show that model estimates and translation quality only vaguely correlate. We apply Minimum Bayes Risk (MBR) decoding on unbiased samples to optimize diverse automated metrics of translation quality as an alternative inference strategy to beam search. Instead of targeting the hypotheses with the highest model probability, MBR decoding extracts the hypotheses with the highest estimated quality. Our experiments show that the combination of a neural translation model with a neural reference-based metric, BLEURT, results in significant improvement in human evaluations. This improvement is obtained with translations different from classical beam-search output: these translations have much lower model likelihood and are less favored by surface metrics like BLEU.
CSI: Contrastive Data Stratification for Interaction Prediction and its Application to Compound-Protein Interaction Prediction
Venue: arXiv cs.LG 2022
Year: 2022

Accurately predicting the likelihood of interaction between two objects (compound-protein sequence, user-item, author-paper, etc.) is a fundamental problem in Computer Science. Current deep-learning models rely on learning accurate representations of the interacting objects. Importantly, relationships between the interacting objects, or features of the interaction, offer an opportunity to partition the data to create multi-views of the interacting objects. The resulting congruent and non-congruent views can then be exploited via contrastive learning techniques to learn enhanced representations of the objects.
Learning Modified Indicator Functions for Surface Reconstruction
Venue: arXiv cs.CV 2022
Year: 2022

Surface reconstruction is a fundamental problem in 3D graphics. In this paper, we propose a learning-based approach for implicit surface reconstruction from raw point clouds without normals. Our method is inspired by Gauss Lemma in potential energy theory, which gives an explicit integral formula for the indicator functions. We design a novel deep neural network to perform surface integral and learn the modified indicator functions from un-oriented and noisy point clouds. We concatenate features with different scales for accurate point-wise contributions to the integral. Moreover, we propose a novel Surface Element Feature Extractor to learn local shape properties. Experiments show that our method generates smooth surfaces with high normal consistency from point clouds with different noise scales and achieves state-of-the-art reconstruction performance compared with current data-driven and non-data-driven approaches.
Differentiable Wavetable Synthesis
Venue: arXiv cs.SD 2022
Year: 2022

Differentiable Wavetable Synthesis (DWTS) is a technique for neural audio synthesis which learns a dictionary of one-period waveforms i.e. wavetables, through end-to-end training. We achieve high-fidelity audio synthesis with as little as 10 to 20 wavetables and demonstrate how a data-driven dictionary of waveforms opens up unprecedented one-shot learning paradigms on short audio clips. Notably, we show audio manipulations, such as high quality pitch-shifting, using only a few seconds of input audio. Lastly, we investigate performance gains from using learned wavetables for realtime and interactive audio synthesis.
Learning in High-Dimensional Feature Spaces Using ANOVA-Based Fast Matrix-Vector Multiplication
Venue: arXiv cs.LG 2022
Year: 2022

Kernel matrices are crucial in many learning tasks such as support vector machines or kernel ridge regression. The kernel matrix is typically dense and large-scale. Depending on the dimension of the feature space even the computation of all of its entries in reasonable time becomes a challenging task. For such dense matrices the cost of a matrix-vector product scales quadratically with the dimensionality N , if no customized methods are applied. We propose the use of an ANOVA kernel, where we construct several kernels based on lower-dimensional feature spaces for which we provide fast algorithms realizing the matrix-vector products. We employ the non-equispaced fast Fourier transform (NFFT), which is of linear complexity for fixed accuracy. Based on a feature grouping approach, we then show how the fast matrix-vector products can be embedded into a learning method choosing kernel ridge regression and the conjugate gradient solver. We illustrate the performance of our approach on several data sets.
GRecX: An Efficient and Unified Benchmark for GNN-based Recommendation
Venue: arXiv cs.IR 2022
Year: 2022

In this paper, we present GRecX, an open-source TensorFlow framework for benchmarking GNN-based recommendation models in an efficient and unified way. GRecX consists of core libraries for building GNN-based recommendation benchmarks, as well as the implementations of popular GNN-based recommendation models. The core libraries provide essential components for building efficient and unified benchmarks, including FastMetrics (efficient metrics computation libraries), VectorSearch (efficient similarity search libraries for dense vectors), BatchEval (efficient mini-batch evaluation libraries), and DataManager (unified dataset management libraries). Especially, to provide a unified benchmark for the fair comparison of different complex GNN-based recommendation models, we design a new metric GRMF-X and integrate it into the FastMetrics component. Based on a TensorFlow GNN library tf_geometric, GRecX carefully implements a variety of popular GNN-based recommendation models. We carefully implement these baseline models to reproduce the performance reported in the literature, and our implementations are usually more efficient and friendly. In conclusion, GRecX enables uses to train and benchmark GNN-based recommendation baselines in an efficient and unified way. We conduct experiments with GRecX, and the experimental results show that GRecX allows us to train and benchmark GNN-based recommendation baselines in an efficient and unified way. The source code of GRecX is available at https://github.com/maenzhier/GRecX.
On the power of adaptivity in statistical adversaries
Venue: arXiv cs.LG 2022
Year: 2022

We study a fundamental question concerning adversarial noise models in statistical problems where the algorithm receives i.i.d. draws from a distribution $\mathcal{D}$. The definitions of these adversaries specify the type of allowable corruptions (noise model) as well as when these corruptions can be made (adaptivity); the latter differentiates between oblivious adversaries that can only corrupt the distribution $\mathcal{D}$ and adaptive adversaries that can have their corruptions depend on the specific sample $S$ that is drawn from $\mathcal{D}$. In this work, we investigate whether oblivious adversaries are effectively equivalent to adaptive adversaries, across all noise models studied in the literature. Specifically, can the behavior of an algorithm $\mathcal{A}$ in the presence of oblivious adversaries always be well-approximated by that of an algorithm $\mathcal{A}'$ in the presence of adaptive adversaries? Our first result shows that this is indeed the case for the broad class of statistical query algorithms, under all reasonable noise models. We then show that in the specific case of additive noise, this equivalence holds for all algorithms. Finally, we map out an approach towards proving this statement in its fullest generality, for all algorithms and under all reasonable noise models.
PRISM: A Hierarchical Intrusion Detection Architecture for Large-Scale Cyber Networks
Venue: arXiv cs.CR 2022
Year: 2022

The increase in scale of cyber networks and the rise in sophistication of cyber-attacks have introduced several challenges in intrusion detection. The primary challenge is the requirement to detect complex multi-stage attacks in realtime by processing the immense amount of traffic produced by present-day networks. In this paper we present PRISM, a hierarchical intrusion detection architecture that uses a novel attacker behavior model-based sampling technique to minimize the realtime traffic processing overhead. PRISM has a unique multi-layered architecture that monitors network traffic distributedly to provide efficiency in processing and modularity in design. PRISM employs a Hidden Markov Model-based prediction mechanism to identify multi-stage attacks and ascertain the attack progression for a proactive response. Furthermore, PRISM introduces a stream management procedure that rectifies the issue of alert reordering when collected from distributed alert reporting systems. To evaluate the performance of PRISM, multiple metrics have been proposed, and various experiments have been conducted on a multi-stage attack dataset. The results exhibit up to 7.5x improvement in processing overhead as compared to a standard centralized IDS without the loss of prediction accuracy while demonstrating the ability to predict different attack stages promptly.
Density Ratio Estimation via Infinitesimal Classification
Venue: arXiv cs.LG 2022
Year: 2022

Density ratio estimation (DRE) is a fundamental machine learning technique for comparing two probability distributions. However, existing methods struggle in high-dimensional settings, as it is difficult to accurately compare probability distributions based on finite samples. In this work we propose DRE-\infty, a divide-and-conquer approach to reduce DRE to a series of easier subproblems. Inspired by Monte Carlo methods, we smoothly interpolate between the two distributions via an infinite continuum of intermediate bridge distributions. We then estimate the instantaneous rate of change of the bridge distributions indexed by time (the "time score") -- a quantity defined analogously to data (Stein) scores -- with a novel time score matching objective. Crucially, the learned time scores can then be integrated to compute the desired density ratio. In addition, we show that traditional (Stein) scores can be used to obtain integration paths that connect regions of high density in both distributions, improving performance in practice. Empirically, we demonstrate that our approach performs well on downstream tasks such as mutual information estimation and energy-based modeling on complex, high-dimensional datasets.
SSR: An Efficient and Robust Framework for Learning with Unknown Label Noise
Venue: arXiv cs.CV 2022
Year: 2022

Despite the large progress in supervised learning with neural networks, there are significant challenges in obtaining high-quality, large-scale and accurately labelled datasets. In such a context, how to learn in the presence of noisy labels has received more and more attention. As a relatively complex problem, in order to achieve good results, current approaches often integrate components from several fields, such as supervised learning, semi-supervised learning, transfer learning and resulting in complicated methods. Furthermore, they often make multiple assumptions about the type of noise of the data. This affects the model robustness and limits its performance under different noise conditions. In this paper, we consider a novel problem setting, Learning with Unknown Label Noise}(LULN), that is, learning when both the degree and the type of noise are unknown. Under this setting, unlike previous methods that often introduce multiple assumptions and lead to complex solutions, we propose a simple, efficient and robust framework named Sample Selection and Relabelling(SSR), that with a minimal number of hyperparameters achieves SOTA results in various conditions. At the heart of our method is a sample selection and relabelling mechanism based on a non-parametric KNN classifier~(NPK) $g_q$ and a parametric model classifier~(PMC) $g_p$, respectively, to select the clean samples and gradually relabel the noisy samples. Without bells and whistles, such as model co-training, self-supervised pre-training and semi-supervised learning, and with robustness concerning the settings of its few hyper-parameters, our method significantly surpasses previous methods on both CIFAR10/CIFAR100 with synthetic noise and real-world noisy datasets such as WebVision, Clothing1M and ANIMAL-10N. Code is available at https://github.com/MrChenFeng/SSR_BMVC2022.
Scaling Law for Recommendation Models: Towards General-purpose User Representations
Venue: arXiv cs.IR 2022
Year: 2022

Recent advancement of large-scale pretrained models such as BERT, GPT-3, CLIP, and Gopher, has shown astonishing achievements across various task domains. Unlike vision recognition and language models, studies on general-purpose user representation at scale still remain underexplored. Here we explore the possibility of general-purpose user representation learning by training a universal user encoder at large scales. We demonstrate that the scaling law is present in user representation learning areas, where the training error scales as a power-law with the amount of computation. Our Contrastive Learning User Encoder (CLUE), optimizes task-agnostic objectives, and the resulting user embeddings stretch our expectation of what is possible to do in various downstream tasks. CLUE also shows great transferability to other domains and companies, as performances on an online experiment shows significant improvements in Click-Through-Rate (CTR). Furthermore, we also investigate how the model performance is influenced by the scale factors, such as training data size, model capacity, sequence length, and batch size. Finally, we discuss the broader impacts of CLUE in general.
Throughput Maximization for Active Intelligent Reflecting Surface Aided Wireless Powered Communications
Venue: arXiv cs.IT 2022
Year: 2022

This paper considers an active intelligent reflecting surface (IRS)-aided wireless powered communication network (WPCN), where devices first harvest energy and then transmit information to a hybrid access point (HAP). Different from the existing works on passive IRS-aided WPCNs, this is the first work that introduces the active IRS in WPCNs. To guarantee fairness, the problem is formulated as an amplifying power-limited weighted sum throughput (WST) maximization problem, which is solved by successive convex approximation technique and fractional programming alternatively. To balance the performance and complexity tradeoff, three beamforming setups are considered at the active IRS, namely user-adaptive IRS beamforming, uplink-adaptive IRS beamforming, and static IRS beamforming. Numerical results demonstrate the significant superiority of employing active IRS in WPCNs and the benefits of dynamic IRS beamforming. Specifically, it is found that compared to the passive IRS, the active IRS not only improves the WST greatly, but also is more energy-efficient and can significantly extend the transmission coverage. Moreover, different from the symmetric deployment strategy of passive IRS, it is more preferable to deploy the active IRS near the devices.
A Lightweight Encryption Scheme for IoT Devices in the Fog
Venue: arXiv cs.CR 2022
Year: 2022

The Internet of Things (IoT) is the collection of everyday smart devices which connect to the Cloud, often through Fog nodes, to transmit and receive information. These everyday devices are distinct from traditional computers because they typically have notable constraints on their RAM, flash memory, and computational power. Due to these constraints, we believe that many of the proposed encryption schemes are too heavyweight to be employed in the IoT. In this paper we present a lightweight, flexible encryption scheme that relies on the one-way information loss property of a secure hash function. Our scheme imposes minimal computational and storage requirements, and imposes no non-negligible burdens on the encrypting device, except for the hash itself. We find that the encryption algorithm is particularly lightweight, and holds up strongly in terms of its speed and memory efficiency.
Disentangling Decentralized Finance (DeFi) Compositions
Venue: arXiv cs.CR 2022
Year: 2022

We present a measurement study on compositions of Decentralized Finance protocols, which aim to disrupt traditional finance and offer services on top of distributed ledgers, such as Ethereum. DeFi compositions may impact the development of ecosystem interoperability, are increasingly integrated with web technologies, and may introduce risks through complexity. Starting from a dataset of 23 labeled DeFi protocols and 10,663,881 associated Ethereum accounts, we study the interactions of protocols and associated smart contracts. From a network perspective, we find that decentralized exchanges and lending protocols have high degree and centrality values, that interactions among protocol nodes primarily occur in a strongly connected component, and that known community detection methods cannot disentangle DeFi protocols. Therefore, we propose an algorithm to decompose a protocol call into a nested set of building blocks that may be part of other DeFi protocols. With a ground truth dataset we have collected, we can demonstrate the algorithm's capability by finding that swaps are the most frequently used building blocks. As building blocks can be nested, i.e., contained in each other, we provide visualizations of composition trees for deeper inspections. We also present a broad picture of DeFi compositions by extracting and flattening the entire nested building block structure across multiple DeFi protocols. Finally, to demonstrate the practicality of our approach, we present a case study that is inspired by the recent collapse of the UST stablecoin in the Terra ecosystem. Under the hypothetical assumption that the stablecoin USD Tether would experience a similar fate, we study which building blocks and, thereby, DeFi protocols would be affected. Overall, our results and methods contribute to a better understanding of a new family of financial products.
Consensus formation on heterogeneous networks
Venue: arXiv cs.SI 2022
Year: 2022

Reaching consensus -- a macroscopic state where the system constituents display the same microscopic state -- is a necessity in multiple complex socio-technical and techno-economic systems: their correct functioning ultimately depends on it. In many distributed systems -- of which blockchain-based applications are a paradigmatic example -- the process of consensus formation is crucial not only for the emergence of a leading majority but for the very functioning of the system. We build a minimalistic network model of consensus formation on blockchain systems for quantifying how central nodes -- with respect to their average distance to others -- can leverage on their position to obtain competitive advantage in the consensus process. We show that in a wide range of network topologies, the probability of forming a majority can significantly increase depending on the centrality of nodes that initiate the spreading. Further, we study the role that network topology plays on the consensus process: we show that central nodes in scale-free networks can win consensus in the network even if they broadcast states significantly later than peripheral ones.
Quantum Advantage for All
Venue: arXiv cs.PL 2022
Year: 2022

We show that the algorithmic complexity of any classical algorithm written in a Turing-complete programming language polynomially bounds the number of quantum bits that are required to run and even symbolically execute the algorithm on a quantum computer. In particular, we show that any classical algorithm $A$ that runs in $\mathcal{O}(f(n))$ time and $\mathcal{O}(g(n))$ space requires no more than $\mathcal{O}(f(n)\cdot g(n))$ quantum bits to execute, even symbolically, on a quantum computer. With $\mathcal{O}(1)\leq\mathcal{O}(g(n))\leq\mathcal{O}(f(n))$ for all $n$, the quantum bits required to execute $A$ may therefore not exceed $\mathcal{O}(f(n)^2)$ and may come down to $\mathcal{O}(f(n))$ if memory consumption by $A$ is bounded by a constant. Our construction works by encoding symbolic execution of machine code in a finite state machine over the satisfiability-modulo-theory (SMT) of bitvectors, for modeling CPU registers, and arrays of bitvectors, for modeling main memory. The FSM is linear in the size of the code, independent of execution time and space, and represents the reachable machine states for any given input. The FSM may be explored by bounded model checkers using SMT and SAT solvers as backend. However, for the purpose of this paper, we focus on quantum computing by unrolling and bit-blasting the FSM into (1)~satisfiability-preserving quadratic unconstrained binary optimization (QUBO) models targeting adiabatic forms of quantum computing such as quantum annealing, and (2)~semantics-preserving quantum circuits (QCs) targeting gate-model quantum computers. With our compact QUBOs, real quantum annealers can now execute simple but real code even symbolically, yet only with potential but no guarantee for exponential speedup, and with our QCs as oracles, Grover's algorithm applies to symbolic execution of arbitrary code, guaranteeing at least in theory a quadratic speedup.