OpenRaiser/Intern-Atlas huggingface dataset:
Toward Accurate and Temporally Consistent Video Restoration from Raw Data
Venue: arXiv cs.CV 2023
Year: 2023

Denoising and demosaicking are two fundamental steps in reconstructing a clean full-color video from raw data, while performing video denoising and demosaicking jointly, namely VJDD, could lead to better video restoration performance than performing them separately. In addition to restoration accuracy, another key challenge to VJDD lies in the temporal consistency of consecutive frames. This issue exacerbates when perceptual regularization terms are introduced to enhance video perceptual quality. To address these challenges, we present a new VJDD framework by consistent and accurate latent space propagation, which leverages the estimation of previous frames as prior knowledge to ensure consistent recovery of the current frame. A data temporal consistency (DTC) loss and a relational perception consistency (RPC) loss are accordingly designed. Compared with the commonly used flow-based losses, the proposed losses can circumvent the error accumulation problem caused by inaccurate flow estimation and effectively handle intensity changes in videos, improving much the temporal consistency of output videos while preserving texture details. Extensive experiments demonstrate the leading VJDD performance of our method in term of restoration accuracy, perceptual quality and temporal consistency. Codes and dataset are available at \url{https://github.com/GuoShi28/VJDD}.
AdapterDistillation: Non-Destructive Task Composition with Knowledge Distillation
Venue: arXiv cs.LG 2023
Year: 2023

Leveraging knowledge from multiple tasks through introducing a small number of task specific parameters into each transformer layer, also known as adapters, receives much attention recently. However, adding an extra fusion layer to implement knowledge composition not only increases the inference time but also is non-scalable for some applications. To avoid these issues, we propose a two-stage knowledge distillation algorithm called AdapterDistillation. In the first stage, we extract task specific knowledge by using local data to train a student adapter. In the second stage, we distill the knowledge from the existing teacher adapters into the student adapter to help its inference. Extensive experiments on frequently asked question retrieval in task-oriented dialog systems validate the efficiency of AdapterDistillation. We show that AdapterDistillation outperforms existing algorithms in terms of accuracy, resource consumption and inference time.
Maximizing the Success Probability of Policy Allocations in Online Systems
Venue: arXiv cs.IR 2023
Year: 2023

The effectiveness of advertising in e-commerce largely depends on the ability of merchants to bid on and win impressions for their targeted users. The bidding procedure is highly complex due to various factors such as market competition, user behavior, and the diverse objectives of advertisers. In this paper we consider the problem at the level of user timelines instead of individual bid requests, manipulating full policies (i.e. pre-defined bidding strategies) and not bid values. In order to optimally allocate policies to users, typical multiple treatments allocation methods solve knapsack-like problems which aim at maximizing an expected value under constraints. In the industrial contexts such as online advertising, we argue that optimizing for the probability of success is a more suited objective than expected value maximization, and we introduce the SuccessProbaMax algorithm that aims at finding the policy allocation which is the most likely to outperform a fixed reference policy. Finally, we conduct comprehensive experiments both on synthetic and real-world data to evaluate its performance. The results demonstrate that our proposed algorithm outperforms conventional expected-value maximization algorithms in terms of success rate.
Function-Correcting Codes for Symbol-Pair Read Channels
Venue: arXiv cs.IT 2023
Year: 2023

Function-correcting codes are a class of codes designed to protect the function evaluation of a message against errors whose key advantage is the reduced redundancy. In this paper, we extend function-correcting codes from binary symmetric channels to symbol-pair read channels. We introduce irregular-pair-distance codes and connect them with function-correcting symbol-pair codes. Using the connection, we derive general upper and lower bounds on the optimal redundancy of function-correcting symbol-pair codes. For ease of evaluation, we simplify these bounds and employ the simplified bounds to specific functions including pair-locally binary functions, pair weight functions and pair weight distribution functions.
Contribución de la semántica combinatoria al desarrollo de herramientas digitales multilingües
Venue: arXiv cs.CL 2023
Year: 2023

This paper describes how the field of Combinatorial Semantics has contributed to the design of three prototypes for the automatic generation of argument patterns in nominal phrases in Spanish, French and German (Xera, Combinatoria and CombiContext). It also shows the importance of knowing about the argument syntactic-semantic interface in a production situation in the context of foreign languages. After a descriptive section on the design, typologie and information levels of the resources, there follows an explanation of the central role of the combinatorial meaning (roles and ontological features). The study deals with different semantic f ilters applied in the selection, organization and expansion of the lexicon, being these key pieces for the generation of grammatically correct and semantically acceptable mono- and biargumental nominal phrases.
Learning temporal formulas from examples is hard
Venue: arXiv cs.LG 2023
Year: 2023

We study the problem of learning linear temporal logic (LTL) formulas from examples, as a first step towards expressing a property separating positive and negative instances in a way that is comprehensible for humans. In this paper we initiate the study of the computational complexity of the problem. Our main results are hardness results: we show that the LTL learning problem is NP-complete, both for the full logic and for almost all of its fragments. This motivates the search for efficient heuristics, and highlights the complexity of expressing separating properties in concise natural language.
Task Contamination: Language Models May Not Be Few-Shot Anymore
Venue: arXiv cs.CL 2023
Year: 2023

Large language models (LLMs) offer impressive performance in various zero-shot and few-shot tasks. However, their success in zero-shot and few-shot settings may be affected by task contamination, a potential limitation that has not been thoroughly examined. This paper investigates how zero-shot and few-shot performance of LLMs has changed chronologically over time. Utilizing GPT-3 series models and several other recent open-sourced LLMs, and controlling for dataset difficulty, we find that on datasets released before the LLM training data creation date, LLMs perform surprisingly better than on datasets released after. This strongly indicates that, for many LLMs, there exists task contamination on zero-shot and few-shot evaluation for datasets released prior to the LLMs' training data creation date. Additionally, we utilize training data inspection, task example extraction, and a membership inference attack, which reveal further evidence of task contamination. Importantly, we find that for classification tasks with no possibility of task contamination, LLMs rarely demonstrate statistically significant improvements over simple majority baselines, in both zero and few-shot settings.
State-of-the-Art in Nudity Classification: A Comparative Analysis
Venue: arXiv cs.CV 2023
Year: 2023

This paper presents a comparative analysis of existing nudity classification techniques for classifying images based on the presence of nudity, with a focus on their application in content moderation. The evaluation focuses on CNN-based models, vision transformer, and popular open-source safety checkers from Stable Diffusion and Large-scale Artificial Intelligence Open Network (LAION). The study identifies the limitations of current evaluation datasets and highlights the need for more diverse and challenging datasets. The paper discusses the potential implications of these findings for developing more accurate and effective image classification systems on online platforms. Overall, the study emphasizes the importance of continually improving image classification models to ensure the safety and well-being of platform users. The project page, including the demonstrations and results is publicly available at https://github.com/fcakyon/content-moderation-deep-learning.
Universal Pyramid Adversarial Training for Improved ViT Performance
Venue: arXiv cs.CV 2023
Year: 2023

Recently, Pyramid Adversarial training (Herrmann et al., 2022) has been shown to be very effective for improving clean accuracy and distribution-shift robustness of vision transformers. However, due to the iterative nature of adversarial training, the technique is up to 7 times more expensive than standard training. To make the method more efficient, we propose Universal Pyramid Adversarial training, where we learn a single pyramid adversarial pattern shared across the whole dataset instead of the sample-wise patterns. With our proposed technique, we decrease the computational cost of Pyramid Adversarial training by up to 70% while retaining the majority of its benefit on clean performance and distribution-shift robustness. In addition, to the best of our knowledge, we are also the first to find that universal adversarial training can be leveraged to improve clean model performance.
Improved Approximation Coflows Scheduling Algorithms for Minimizing the Total Weighted Completion Time and Makespan in Heterogeneous Parallel Networks
Venue: arXiv cs.DS 2023
Year: 2023

Coflow is a network abstraction used to represent communication patterns in data centers. The coflow scheduling problem encountered in large data centers is a challenging $\mathcal{NP}$-hard problem. This paper tackles the scheduling problem of coflows with release times in heterogeneous parallel networks, which feature an architecture consisting of multiple network cores running in parallel. Two polynomial-time approximation algorithms are presented in this paper, designed to minimize the total weighted completion time and makespan in heterogeneous parallel networks, respectively. For any given $ε>0$, our proposed approximation algorithm for minimizing the total weighted completion time achieves approximation ratios of $3 + ε$ and $2 + ε$ in the cases of arbitrary and zero release times, respectively. Additionally, we introduce an approximation algorithm for minimizing the makespan, achieving an approximation ratio of $2 + ε$ for $ε>0$. Notably, these advancements surpass the previously best-known approximation ratio of $O(\log m/ \log \log m)$ for both minimizing the total weighted completion time and makespan. This result also improves upon the previous approximation ratios of $6-\frac{2}{m}$ and $5-\frac{2}{m}$ for arbitrary and zero release times, respectively, in identical parallel networks.
Gemini: Mapping and Architecture Co-exploration for Large-scale DNN Chiplet Accelerators
Venue: arXiv cs.AR 2023
Year: 2023

Chiplet technology enables the integration of an increasing number of transistors on a single accelerator with higher yield in the post-Moore era, addressing the immense computational demands arising from rapid AI advancements. However, it also introduces more expensive packaging costs and costly Die-to-Die (D2D) interfaces, which require more area, consume higher power, and offer lower bandwidth than on-chip interconnects. Maximizing the benefits and minimizing the drawbacks of chiplet technology is crucial for developing large-scale DNN chiplet accelerators, which poses challenges to both architecture and mapping. Despite its importance in the post-Moore era, methods to address these challenges remain scarce.
Rational Economic Behaviours in the Bitcoin Lightning Network
Venue: arXiv cs.GT 2023
Year: 2023

The Bitcoin Lightning Network (LN) is designed to improve the scalability of blockchain systems by using off-chain payment paths to settle transactions in a faster, cheaper, and more private manner. This work aims to empirically study LN's fee revenue for network participants. Under realistic assumptions on payment amounts, routing algorithms and traffic distribution, we analyze the economic returns of the network's largest routing nodes which currently hold the network together, and assess whether the centralizing tendency is incentive-compatible from an economic viewpoint. Moreover, since recent literature has proved that participation is economically irrational for the majority of large nodes, we evaluate the long-term impact on the network topology when participants start behaving rationally.
FreqyWM: Frequency Watermarking for the New Data Economy
Venue: arXiv cs.CR 2023
Year: 2023

We present a novel technique for modulating the appearance frequency of a few tokens within a dataset for encoding an invisible watermark that can be used to protect ownership rights upon data. We develop optimal as well as fast heuristic algorithms for creating and verifying such watermarks. We also demonstrate the robustness of our technique against various attacks and derive analytical bounds for the false positive probability of erroneously detecting a watermark on a dataset that does not carry it. Our technique is applicable to both single dimensional and multidimensional datasets, is independent of token type, allows for a fine control of the introduced distortion, and can be used in a variety of use cases that involve buying and selling data in contemporary data marketplaces.
A proposed new metric for the conceptual diversity of a text
Venue: arXiv cs.CL 2023
Year: 2023

A word may contain one or more hidden concepts. While the "animal" word evokes many images in our minds and encapsulates many concepts (birds, dogs, cats, crocodiles, etc.), the `parrot' word evokes a single image (a colored bird with a short, hooked beak and the ability to mimic sounds). In spoken or written texts, we use some words in a general sense and some in a detailed way to point to a specific object. Until now, a text's conceptual diversity value cannot be determined using a standard and precise technique. This research contributes to the natural language processing field of AI by offering a standardized method and a generic metric for evaluating and comparing concept diversity in different texts and domains. It also contributes to the field of semantic research of languages. If we give examples for the diversity score of two sentences, "He discovered an unknown entity." has a high conceptual diversity score (16.6801), and "The endoplasmic reticulum forms a series of flattened sacs within the cytoplasm of eukaryotic cells." sentence has a low conceptual diversity score which is 3.9068.
Blind Image Quality Assessment: A Brief Survey
Venue: arXiv cs.CV 2023
Year: 2023

Blind Image Quality Assessment (BIQA) is essential for automatically evaluating the perceptual quality of visual signals without access to the references. In this survey, we provide a comprehensive analysis and discussion of recent developments in the field of BIQA. We have covered various aspects, including hand-crafted BIQAs that focus on distortion-specific and general-purpose methods, as well as deep-learned BIQAs that employ supervised and unsupervised learning techniques. Additionally, we have explored multimodal quality assessment methods that consider interactions between visual and audio modalities, as well as visual and text modalities. Finally, we have offered insights into representative BIQA databases, including both synthetic and authentic distortions. We believe this survey provides valuable understandings into the latest developments and emerging trends for the visual quality community.
Optimal Beamforming Structure and Efficient Optimization Algorithms for Generalized Multi-Group Multicast Beamforming Optimization
Venue: arXiv cs.IT 2023
Year: 2023

In this work, we focus on solving non-smooth non-convex maximization problems in multi-group multicast transmission. Leveraging Karush-Kuhn-Tucker (KKT) optimality conditions and successive incumbent transcending (SIT) duality, we thoroughly analyze the optimal beamforming structure for a set of optimization problems characterized by a general utility-based objective function. By exploiting the identified optimal structure, we further unveil inherent low-dimensional beamforming structures within the problems, which are asymptotically optimal in various regimes of transmit signal-to-noise ratios (SNRs) or the number of transmit antennas. Building upon the discovered optimal and low-dimensional beamforming structures, we then propose highly efficient and toolbox-free optimization algorithms to solve a specific multi-group multicast optimization problem based on the weighted sum rate (WSR) utility function. The proposed algorithms first use the cyclic maximization (CM) framework to decompose the problem into multiple subproblems, each has an optimal or low-dimensional closed-form beamforming solution structure. Then, we propose the projected adaptive gradient descent (PAGD) algorithm to compute the optimal Lagrangian dual variables for each subproblem. Numerical results show that the proposed algorithms maintain comparable or improved WSR performance compared to baseline algorithms, while dramatically reducing the computational complexity. Notably, the proposed ultra-low-complexity algorithms based on low-dimensional beamforming structures achieve near optimal WSR performance with extremely low computational complexity. This complexity remains independent of the number of transmit antennas, making them promising and practical for extremely large multiple-input multiple-output (XL-MIMO) applications in 6G.
Multi-modality Affinity Inference for Weakly Supervised 3D Semantic Segmentation
Venue: arXiv cs.CV 2023
Year: 2023

3D point cloud semantic segmentation has a wide range of applications. Recently, weakly supervised point cloud segmentation methods have been proposed, aiming to alleviate the expensive and laborious manual annotation process by leveraging scene-level labels. However, these methods have not effectively exploited the rich geometric information (such as shape and scale) and appearance information (such as color and texture) present in RGB-D scans. Furthermore, current approaches fail to fully leverage the point affinity that can be inferred from the feature extraction network, which is crucial for learning from weak scene-level labels. Additionally, previous work overlooks the detrimental effects of the long-tailed distribution of point cloud data in weakly supervised 3D semantic segmentation. To this end, this paper proposes a simple yet effective scene-level weakly supervised point cloud segmentation method with a newly introduced multi-modality point affinity inference module. The point affinity proposed in this paper is characterized by features from multiple modalities (e.g., point cloud and RGB), and is further refined by normalizing the classifier weights to alleviate the detrimental effects of long-tailed distribution without the need of the prior of category distribution. Extensive experiments on the ScanNet and S3DIS benchmarks verify the effectiveness of our proposed method, which outperforms the state-of-the-art by ~4% to ~6% mIoU. Codes are released at https://github.com/Sunny599/AAAI24-3DWSSG-MMA.
EasyView: Bringing Performance Profiles into Integrated Development Environments
Venue: arXiv cs.SE 2023
Year: 2023

Dynamic program analysis (also known as profiling) is well-known for its powerful capabilities of identifying performance inefficiencies in software packages. Although a large number of dynamic program analysis techniques are developed in academia and industry, very few of them are widely used by software developers in their regular software developing activities. There are three major reasons. First, the dynamic analysis tools (also known as profilers) are disjoint from the coding environments such as IDEs and editors; frequently switching focus between them significantly complicates the entire cycle of software development. Second, mastering various tools to interpret their analysis results requires substantial efforts; even worse, many tools have their own design of graphical user interfaces (GUI) for data presentation, which steepens the learning curves. Third, most existing tools expose few interfaces to support user-defined analysis, which makes the tools less customizable to fulfill diverse user demands. We develop EasyView, a general solution to integrate the interpretation and visualization of various profiling results in the coding environments, which bridges software developers with profilers to provide easy and intuitive dynamic analysis during the code development cycle. The novelty of EasyView is three-fold. First, we develop a generic data format, which enables EasyView to support mainstream profilers for different languages. Second, we develop a set of customizable schemes to analyze and visualize the profiles in intuitive ways. Third, we tightly integrate EasyView with popular coding environments, such as Microsoft Visual Studio Code, with easy code exploration and user interaction. Our evaluation shows that EasyView is able to support various profilers for different languages and provide unique insights into performance inefficiencies in different domains.
Learning from small data sets: Patch-based regularizers in inverse problems for image reconstruction
Venue: arXiv cs.CV 2023
Year: 2023

The solution of inverse problems is of fundamental interest in medical and astronomical imaging, geophysics as well as engineering and life sciences. Recent advances were made by using methods from machine learning, in particular deep neural networks. Most of these methods require a huge amount of (paired) data and computer capacity to train the networks, which often may not be available. Our paper addresses the issue of learning from small data sets by taking patches of very few images into account. We focus on the combination of model-based and data-driven methods by approximating just the image prior, also known as regularizer in the variational model. We review two methodically different approaches, namely optimizing the maximum log-likelihood of the patch distribution, and penalizing Wasserstein-like discrepancies of whole empirical patch distributions. From the point of view of Bayesian inverse problems, we show how we can achieve uncertainty quantification by approximating the posterior using Langevin Monte Carlo methods. We demonstrate the power of the methods in computed tomography, image super-resolution, and inpainting. Indeed, the approach provides also high-quality results in zero-shot super-resolution, where only a low-resolution image is available. The paper is accompanied by a GitHub repository containing implementations of all methods as well as data examples so that the reader can get their own insight into the performance.