OpenRaiser/Intern-Atlas huggingface dataset:
WildDeepfake: A Challenging Real-World Dataset for Deepfake Detection
Venue: arXiv cs.CV 2024
Year: 2024

In recent years, the abuse of a face swap technique called deepfake has raised enormous public concerns. So far, a large number of deepfake videos (known as "deepfakes") have been crafted and uploaded to the internet, calling for effective countermeasures. One promising countermeasure against deepfakes is deepfake detection. Several deepfake datasets have been released to support the training and testing of deepfake detectors, such as DeepfakeDetection and FaceForensics++. While this has greatly advanced deepfake detection, most of the real videos in these datasets are filmed with a few volunteer actors in limited scenes, and the fake videos are crafted by researchers using a few popular deepfake softwares. Detectors developed on these datasets may become less effective against real-world deepfakes on the internet. To better support detection against real-world deepfakes, in this paper, we introduce a new dataset WildDeepfake which consists of 7,314 face sequences extracted from 707 deepfake videos collected completely from the internet. WildDeepfake is a small dataset that can be used, in addition to existing datasets, to develop and test the effectiveness of deepfake detectors against real-world deepfakes. We conduct a systematic evaluation of a set of baseline detection networks on both existing and our WildDeepfake datasets, and show that WildDeepfake is indeed a more challenging dataset, where the detection performance can decrease drastically. We also propose two (eg. 2D and 3D) Attention-based Deepfake Detection Networks (ADDNets) to leverage the attention masks on real/fake faces for improved detection. We empirically verify the effectiveness of ADDNets on both existing datasets and WildDeepfake. The dataset is available at: https://github.com/OpenTAI/wild-deepfake.
Towards Accurate Camouflaged Object Detection with Mixture Convolution and Interactive Fusion
Venue: arXiv cs.CV 2024
Year: 2024

Camouflaged object detection (COD), which aims to identify the objects that conceal themselves into the surroundings, has recently drawn increasing research efforts in the field of computer vision. In practice, the success of deep learning based COD is mainly determined by two key factors, including (i) A significantly large receptive field, which provides rich context information, and (ii) An effective fusion strategy, which aggregates the rich multi-level features for accurate COD. Motivated by these observations, in this paper, we propose a novel deep learning based COD approach, which integrates the large receptive field and effective feature fusion into a unified framework. Specifically, we first extract multi-level features from a backbone network. The resulting features are then fed to the proposed dual-branch mixture convolution modules, each of which utilizes multiple asymmetric convolutional layers and two dilated convolutional layers to extract rich context features from a large receptive field. Finally, we fuse the features using specially-designed multilevel interactive fusion modules, each of which employs an attention mechanism along with feature interaction for effective feature fusion. Our method detects camouflaged objects with an effective fusion strategy, which aggregates the rich context information from a large receptive field. All of these designs meet the requirements of COD well, allowing the accurate detection of camouflaged objects. Extensive experiments on widely-used benchmark datasets demonstrate that our method is capable of accurately detecting camouflaged objects and outperforms the state-of-the-art methods.
Approximate Byzantine Fault-Tolerance in Distributed Optimization
Venue: arXiv cs.DC 2024
Year: 2024

This paper considers the problem of Byzantine fault-tolerance in distributed multi-agent optimization. In this problem, each agent has a local cost function, and in the fault-free case, the goal is to design a distributed algorithm that allows all the agents to find a minimum point of all the agents' aggregate cost function. We consider a scenario where some agents might be Byzantine faulty that renders the original goal of computing a minimum point of all the agents' aggregate cost vacuous. A more reasonable objective for an algorithm in this scenario is to allow all the non-faulty agents to compute the minimum point of only the non-faulty agents' aggregate cost. Prior work shows that if there are up to $f$ (out of $n$) Byzantine agents then a minimum point of the non-faulty agents' aggregate cost can be computed exactly if and only if the non-faulty agents' costs satisfy a certain redundancy property called $2f$-redundancy. However, $2f$-redundancy is an ideal property that can be satisfied only in systems free from noise or uncertainties, which can make the goal of exact fault-tolerance unachievable in some applications. Thus, we introduce the notion of $(f,ε)$-resilience, a generalization of exact fault-tolerance wherein the objective is to find an approximate minimum point of the non-faulty aggregate cost, with $ε$ accuracy. This approximate fault-tolerance can be achieved under a weaker condition that is easier to satisfy in practice, compared to $2f$-redundancy. We obtain necessary and sufficient conditions for achieving $(f,ε)$-resilience characterizing the correlation between relaxation in redundancy and approximation in resilience. In case when the agents' cost functions are differentiable, we obtain conditions for $(f,ε)$-resilience of the distributed gradient-descent method when equipped with robust gradient aggregation.
Computability of the Channel Reliability Function and Related Bounds
Venue: arXiv cs.IT 2024
Year: 2024

The channel reliability function is an important tool that characterizes the reliable transmission of messages over communication channels. For many channels, only upper and lower bounds of the function are known. We analyze the computability of the reliability function and its related functions. We show that the reliability function is not a Turing computable performance function. The same also applies to the functions related to the sphere packing bound and the expurgation bound. Furthermore, we consider the $R_\infty$ function and the zero-error feedback capacity, since they play an important role in the context of the reliability function. Both the $R_\infty$ function and the zero-error feedback capacity are not Banach Mazur computable.
Exact and Approximate Heterogeneous Bayesian Decentralized Data Fusion
Venue: arXiv cs.RO 2024
Year: 2024

In Bayesian peer-to-peer decentralized data fusion, the underlying distributions held locally by autonomous agents are frequently assumed to be over the same set of variables (homogeneous). This requires each agent to process and communicate the full global joint distribution, and thus leads to high computation and communication costs irrespective of relevancy to specific local objectives. This work formulates and studies heterogeneous decentralized fusion problems, defined as the set of problems in which either the communicated or the processed distributions describe different, but overlapping, random states of interest that are subsets of a larger full global joint state. We exploit the conditional independence structure of such problems and provide a rigorous derivation of novel exact and approximate conditionally factorized heterogeneous fusion rules. We further develop a new version of the homogeneous Channel Filter algorithm to enable conservative heterogeneous fusion for smoothing and filtering scenarios in dynamic problems. Numerical examples show more than $99.5\%$ potential communication reduction for heterogeneous channel filter fusion, and a multi-target tracking simulation shows that these methods provide consistent estimates while remaining computationally scalable.
Learning From How Humans Correct
Venue: arXiv cs.CL 2024
Year: 2024

In industry NLP application, our manually labeled data has a certain number of noisy data. We present a simple method to find the noisy data and relabel them manually, meanwhile we collect the correction information. Then we present novel method to incorporate the human correction information into deep learning model. Human know how to correct noisy data. So the correction information can be inject into deep learning model. We do the experiment on our own text classification dataset, which is manually labeled, because we need to relabel the noisy data in our dataset for our industry application. The experiment result shows that our learn-on-correction method improve the classification accuracy from 91.7% to 92.5% in test dataset. The 91.7% accuracy is trained on the corrected dataset, which improve the baseline from 83.3% to 91.7% in test dataset. The accuracy under human evaluation achieves more than 97%.
Inducing Meaningful Units from Character Sequences with Dynamic Capacity Slot Attention
Venue: arXiv cs.CL 2024
Year: 2024

Characters do not convey meaning, but sequences of characters do. We propose an unsupervised distributional method to learn the abstract meaningful units in a sequence of characters. Rather than segmenting the sequence, our Dynamic Capacity Slot Attention model discovers continuous representations of the objects in the sequence, extending an architecture for object discovery in images. We train our model on different languages and evaluate the quality of the obtained representations with forward and reverse probing classifiers. These experiments show that our model succeeds in discovering units which are similar to those proposed previously in form, content and level of abstraction, and which show promise for capturing meaningful information at a higher level of abstraction.
Acoustic Structure Inverse Design and Optimization Using Deep Learning
Venue: arXiv cs.SD 2024
Year: 2024

From ancient to modern times, acoustic structures have been used to control the propagation of acoustic waves. However, the design of the acoustic structures has remained widely a time-consuming and computational resource-consuming iterative process. In recent years, Deep Learning has attracted unprecedented attention for its ability to tackle hard problems with huge datasets, which has achieved state-of-the-art results in various tasks. In this work, an acoustic structure design method is proposed based on deep learning. Taking the design of multi-order Helmholtz resonator for instance, we experimentally demonstrate the effectiveness of the proposed method. Our method is not only able to give a very accurate prediction of the geometry of the acoustic structures with multiple strong-coupling parameters, but also capable of improving the performance of evolutionary approaches in optimization for a desired property. Compared with the conventional numerical methods, our method is more efficient, universal and automatic, which has a wide range of potential applications, such as speech enhancement, sound absorption and insulation.
Opinion Dynamics Incorporating Higher-Order Interactions
Venue: arXiv cs.SI 2024
Year: 2024

The issue of opinion sharing and formation has received considerable attention in the academic literature, and a few models have been proposed to study this problem. However, existing models are limited to the interactions among nearest neighbors, ignoring those second, third, and higher-order neighbors, despite the fact that higher-order interactions occur frequently in real social networks. In this paper, we develop a new model for opinion dynamics by incorporating long-range interactions based on higher-order random walks. We prove that the model converges to a fixed opinion vector, which may differ greatly from those models without higher-order interactions. Since direct computation of the equilibrium opinion is computationally expensive, which involves the operations of huge-scale matrix multiplication and inversion, we design a theoretically convergence-guaranteed estimation algorithm that approximates the equilibrium opinion vector nearly linearly in both space and time with respect to the number of edges in the graph. We conduct extensive experiments on various social networks, demonstrating that the new algorithm is both highly efficient and effective.
Modeling 3D Surface Manifolds with a Locally Conditioned Atlas
Venue: arXiv cs.CV 2024
Year: 2024

Recently proposed 3D object reconstruction methods represent a mesh with an atlas - a set of planar patches approximating the surface. However, their application in a real-world scenario is limited since the surfaces of reconstructed objects contain discontinuities, which degrades the quality of the final mesh. This is mainly caused by independent processing of individual patches, and in this work, we postulate to mitigate this limitation by preserving local consistency around patch vertices. To that end, we introduce a Locally Conditioned Atlas (LoCondA), a framework for representing a 3D object hierarchically in a generative model. Firstly, the model maps a point cloud of an object into a sphere. Secondly, by leveraging a spherical prior, we enforce the mapping to be locally consistent on the sphere and on the target object. This way, we can sample a mesh quad on that sphere and project it back onto the object's manifold. With LoCondA, we can produce topologically diverse objects while maintaining quads to be stitched together. We show that the proposed approach provides structurally coherent reconstructions while producing meshes of quality comparable to the competitors.
The MSR-Video to Text Dataset with Clean Annotations
Venue: arXiv cs.CV 2024
Year: 2024

Video captioning automatically generates short descriptions of the video content, usually in form of a single sentence. Many methods have been proposed for solving this task. A large dataset called MSR Video to Text (MSR-VTT) is often used as the benchmark dataset for testing the performance of the methods. However, we found that the human annotations, i.e., the descriptions of video contents in the dataset are quite noisy, e.g., there are many duplicate captions and many captions contain grammatical problems. These problems may pose difficulties to video captioning models for learning underlying patterns. We cleaned the MSR-VTT annotations by removing these problems, then tested several typical video captioning models on the cleaned dataset. Experimental results showed that data cleaning boosted the performances of the models measured by popular quantitative metrics. We recruited subjects to evaluate the results of a model trained on the original and cleaned datasets. The human behavior experiment demonstrated that trained on the cleaned dataset, the model generated captions that were more coherent and more relevant to the contents of the video clips.
ReLU Neural Networks of Polynomial Size for Exact Maximum Flow Computation
Venue: arXiv cs.LG 2024
Year: 2024

This paper studies the expressive power of artificial neural networks with rectified linear units. In order to study them as a model of real-valued computation, we introduce the concept of Max-Affine Arithmetic Programs and show equivalence between them and neural networks concerning natural complexity measures. We then use this result to show that two fundamental combinatorial optimization problems can be solved with polynomial-size neural networks. First, we show that for any undirected graph with $n$ nodes, there is a neural network (with fixed weights and biases) of size $\mathcal{O}(n^3)$ that takes the edge weights as input and computes the value of a minimum spanning tree of the graph. Second, we show that for any directed graph with $n$ nodes and $m$ arcs, there is a neural network of size $\mathcal{O}(m^2n^2)$ that takes the arc capacities as input and computes a maximum flow. Our results imply that these two problems can be solved with strongly polynomial time algorithms that solely use affine transformations and maxima computations, but no comparison-based branchings.
Collapsing the bounded width hierarchy for infinite-domain CSPs: when symmetries are enough
Venue: arXiv cs.LO 2024
Year: 2024

We prove that relational structures admitting specific polymorphisms (namely, canonical pseudo-WNU operations of all arities $n \geq 3$) have low relational width. This implies a collapse of the bounded width hierarchy for numerous classes of infinite-domain CSPs studied in the literature. Moreover, we obtain a characterization of bounded width for first-order reducts of unary structures and a characterization of MMSNP sentences that are equivalent to a Datalog program, answering a question posed by Bienvenu, ten Cate, Lutz, and Wolter. In particular, the bounded width hierarchy collapses in those cases as well. Our results extend the scope of theorems of Barto and Kozik characterizing bounded width for finite structures, and show the applicability of infinite-domain CSPs to other fields.
Construction of Complete Complementary Codes over Small Alphabet
Venue: arXiv cs.IT 2024
Year: 2024

Complete complementary codes (CCCs) play a vital role not only in wireless communication, particularly in multicarrier systems where achieving an interference-free environment is of paramount importance, but also in the construction of other codes that necessitate appropriate functions to meet the diverse demands within today's landscape of wireless communication evaluation. This research is focused on the area of constructing $q$-ary functions for both of {traditional and spectrally null constraint (SNC) CCCs}\footnote{When no codes in CCCs having zero components, we call it as traditonal CCCs, else, we call it as SNC-CCCs in this pape.} of flexible length, set size and alphabet. We construct traditional CCCs with lengths, defined as $L = \prod_{i=1}^k p_i^{m_i}$, set sizes, defined as $K = \prod_{i=1}^k p_i^{n_i+1}$, and an alphabet size of $q=\prod_{i=1}^k p_i$, such that $p_1<p_2<\cdots<p_k $. The parameters $m_1, m_2, \ldots, m_k$ (each greater than or equal to $2$) are positive integers, while $n_1, n_2, \ldots, n_k$ are non-negative integers satisfying $n_i \leq m_i-1$, and the variable $k$ represents a positive integer. To achieve these specific parameters, we define $q$-ary functions over a domain $\mathbf{Z}_{p_1}^{m_1}\times \cdots \times \mathbf{Z}_{p_k}^{m_k}$ that is considered a proper subset of $\mathbb{Z}_{q}^m$ and encompasses $\prod_{i=1}^k p_i^{m_i}$ vectors, where $\mathbf{Z}_{p_i}^{m_i}=\{0,1,\hdots,p_i-1\}^{m_i}$, and the value of $m$ is derived from the sum of $m_1, m_2, \ldots, m_k$. This organization of the domain allows us to encompass all conceivable integer-valued length sequences over the alphabet $\mathbb{Z}_q$. It has been demonstrated that by constraining a $q$-ary function that generates traditional CCCs, we can derive SNC-CCCs with identical length and alphabet, yet a smaller or equal set size compared to the traditional CCCs.
Tame the Wild with Byzantine Linearizability: Reliable Broadcast, Snapshots, and Asset Transfer
Venue: arXiv cs.DC 2024
Year: 2024

We formalize Byzantine linearizability, a correctness condition that specifies whether a concurrent object with a sequential specification is resilient against Byzantine failures. Using this definition, we systematically study Byzantine-tolerant emulations of various objects from registers. We focus on three useful objects -- reliable broadcast, atomic snapshot, and asset transfer. We prove that there is an $f$-resilient implementation of such objects from registers with $n$ processes $f<\frac{n}{2}$.
Explore the Context: Optimal Data Collection for Context-Conditional Dynamics Models
Venue: arXiv cs.LG 2024
Year: 2024

In this paper, we learn dynamics models for parametrized families of dynamical systems with varying properties. The dynamics models are formulated as stochastic processes conditioned on a latent context variable which is inferred from observed transitions of the respective system. The probabilistic formulation allows us to compute an action sequence which, for a limited number of environment interactions, optimally explores the given system within the parametrized family. This is achieved by steering the system through transitions being most informative for the context variable. We demonstrate the effectiveness of our method for exploration on a non-linear toy-problem and two well-known reinforcement learning environments.
Aximorphic Perspective Projection Model for Immersive Imagery
Venue: arXiv cs.GR 2024
Year: 2024

A wide choice of cinematic lenses enables motion-picture creators to adapt image visual-appearance to their creative vision. Such choice does not exist in the realm of real-time computer graphics, where only one type of perspective projection is widely used, a linear perspective. This paper presents an extended perspective imaging model, which can represent distortion and FoV parameters of entire variety of film and photographic lenses (e.g., wide-angle, fisheye, anamorphic), while preserving parametrization in an artistically convincing manner. Self-experimentation with the model revealed that each projection type provides accurate perception of a different aspect of depicted space (e.g., speed, distance, shape). Presented model, enables combination of multiple projections, each on a different axis of the image, to achieve optimal perception for a given scenario. This new projection, named aximorphic, was made available here, under an open license (CC BY-SA 3.0), for a wide and easy adoption.
Limited-Trust in Social Network Games
Venue: arXiv cs.GT 2024
Year: 2024

We consider agents in a social network competing to be selected as partners in collaborative, mutually beneficial activities. We study this through a model in which an agent i can initiate a limited number k_i>0 of games and selects the ideal partners from its one-hop neighborhood. On the flip side it can accept as many games offered from its neighbors. Each game signifies a productive joint economic activity, and players attempt to maximize their individual utilities. Unsurprisingly, more trustworthy agents are more desirable as partners. Trustworthiness is measured by the game theoretic concept of Limited-Trust, which quantifies the maximum cost an agent is willing to incur in order to improve the net utility of all agents. Agents learn about their neighbors' trustworthiness through interactions and their behaviors evolve in response. Empirical trials performed on realistic social networks show that when given the option, many agents become highly trustworthy; most or all become highly trustworthy when knowledge of their neighbors' trustworthiness is based on past interactions rather than known a priori. This trustworthiness is not the result of altruism, instead agents are intrinsically motivated to become trustworthy partners by competition. Two insights are presented: first, trustworthy behavior drives an increase in the utility of all agents, where maintaining a relatively modest level of trustworthiness may easily improve net utility by as much as 14.5%. If only one agent exhibits modest trust among self-centered ones, it can increase its average utility by up to 25% in certain cases! Second, and counter-intuitively, when partnership opportunities are abundant agents become less trustworthy.