OpenRaiser/Intern-Atlas huggingface dataset:
ATOM: A Generalizable Technique for Inferring Tracker-Advertiser Data Sharing in the Online Behavioral Advertising Ecosystem
Venue: arXiv cs.SI 2022
Year: 2022

Data sharing between online trackers and advertisers is a key component in online behavioral advertising. This sharing can be facilitated through a variety of processes, including those not observable to the user's browser. The unobservability of these processes limits the ability of researchers and auditors seeking to verify compliance with regulations which require complete disclosure of data sharing partners. Unfortunately, the applicability of existing techniques to make inferences about unobservable data sharing relationships is limited due to their dependence on protocol- or case-specific artifacts of the online behavioral advertising ecosystem (e.g., they work only when client-side header bidding is used for ad delivery or when advertisers perform ad retargeting). As behavioral advertising technologies continue to evolve rapidly, the availability of these artifacts and the effectiveness of transparency solutions dependent on them remain ephemeral. In this paper, we propose a generalizable technique, called ATOM, to infer data sharing relationships between online trackers and advertisers. ATOM is different from prior work in that it is universally applicable -- i.e., independent of ad delivery protocols or availability of artifacts. ATOM leverages the insight that by the very nature of behavioral advertising, ad creatives themselves can be used to infer data sharing between trackers and advertisers -- after all, the topics and brands showcased in an ad are dependent on the data available to the advertiser. Therefore, by selectively blocking trackers and monitoring changes in the characteristics of ads delivered by advertisers, ATOM is able to identify data sharing relationships between trackers and advertisers. The relationships discovered by our implementation of ATOM include those not found using prior approaches and are validated by external sources.
Two-Stage Fine-Tuning: A Novel Strategy for Learning Class-Imbalanced Data
Venue: arXiv cs.CL 2022
Year: 2022

Classification on long-tailed distributed data is a challenging problem, which suffers from serious class-imbalance and hence poor performance on tail classes with only a few samples. Owing to this paucity of samples, learning on the tail classes is especially challenging for the fine-tuning when transferring a pretrained model to a downstream task. In this work, we present a simple modification of standard fine-tuning to cope with these challenges. Specifically, we propose a two-stage fine-tuning: we first fine-tune the final layer of the pretrained model with class-balanced reweighting loss, and then we perform the standard fine-tuning. Our modification has several benefits: (1) it leverages pretrained representations by only fine-tuning a small portion of the model parameters while keeping the rest untouched; (2) it allows the model to learn an initial representation of the specific task; and importantly (3) it protects the learning of tail classes from being at a disadvantage during the model updating. We conduct extensive experiments on synthetic datasets of both two-class and multi-class tasks of text classification as well as a real-world application to ADME (i.e., absorption, distribution, metabolism, and excretion) semantic labeling. The experimental results show that the proposed two-stage fine-tuning outperforms both fine-tuning with conventional loss and fine-tuning with a reweighting loss on the above datasets.
Privacy and Transparency in Graph Machine Learning: A Unified Perspective
Venue: arXiv cs.LG 2022
Year: 2022

Graph Machine Learning (GraphML), whereby classical machine learning is generalized to irregular graph domains, has enjoyed a recent renaissance, leading to a dizzying array of models and their applications in several domains. With its growing applicability to sensitive domains and regulations by governmental agencies for trustworthy AI systems, researchers have started looking into the issues of transparency and privacy of graph learning. However, these topics have been mainly investigated independently. In this position paper, we provide a unified perspective on the interplay of privacy and transparency in GraphML. In particular, we describe the challenges and possible research directions for a formal investigation of privacy-transparency tradeoffs in GraphML.
Efficient Modeling of Future Context for Image Captioning
Venue: arXiv cs.CV 2022
Year: 2022

Existing approaches to image captioning usually generate the sentence word-by-word from left to right, with the constraint of conditioned on local context including the given image and history generated words. There have been many studies target to make use of global information during decoding, e.g., iterative refinement. However, it is still under-explored how to effectively and efficiently incorporate the future context. To respond to this issue, inspired by that Non-Autoregressive Image Captioning (NAIC) can leverage two-side relation with modified mask operation, we aim to graft this advance to the conventional Autoregressive Image Captioning (AIC) model while maintaining the inference efficiency without extra time cost. Specifically, AIC and NAIC models are first trained combined with shared visual encoders, forcing the visual encoder to contain sufficient and valid future context; then the AIC model is encouraged to capture the causal dynamics of cross-layer interchanging from NAIC model on its unconfident words, which follows a teacher-student paradigm and optimized with the distribution calibration training objective. Empirical evidences demonstrate that our proposed approach clearly surpass the state-of-the-art baselines in both automatic metrics and human evaluations on the MS COCO benchmark. The source code is available at: https://github.com/feizc/Future-Caption.
A new class of negabent functions
Venue: arXiv cs.DM 2022
Year: 2022

Negabent functions were introduced as a generalization of bent functions, which have applications in coding theory and cryptography. In this paper, we have extended the notion of negabent functions to the functions defined from $\mathbb{Z}_q^n$ to $\mathbb{Z}_{2q}$ ($2q$-negabent), where $q \geq 2$ is a positive integer and $\mathbb{Z}_q$ is the ring of integers modulo $q$. For this, a new unitary transform (the nega-Hadamard transform) is introduced in the current set up, and some of its properties are discussed. Some results related to $2q$-negabent functions are presented. We present two constructions of $2q$-negabent functions. In the first construction, $2q$-negabent functions on $n$ variables are constructed when $q$ is an even positive integer. In the second construction, $2q$-negabent functions on two variables are constructed for arbitrary positive integer $q \ge 2$. Some examples of $2q$-negabent functions for different values of $q$ and $n$ are also presented.
Open video data sharing in developmental and behavioural science
Venue: arXiv cs.CV 2022
Year: 2022

Video recording is a widely used method for documenting infant and child behaviours in research and clinical practice. Video data has rarely been shared due to ethical concerns of confidentiality, although the need of shared large-scaled datasets remains increasing. This demand is even more imperative when data-driven computer-based approaches are involved, such as screening tools to complement clinical assessments. To share data while abiding by privacy protection rules, a critical question arises whether efforts at data de-identification reduce data utility? We addressed this question by showcasing the Prechtl's general movements assessment (GMA), an established and globally practised video-based diagnostic tool in early infancy for detecting neurological deficits, such as cerebral palsy. To date, no shared expert-annotated large data repositories for infant movement analyses exist. Such datasets would massively benefit training and recalibration of human assessors and the development of computer-based approaches. In the current study, sequences from a prospective longitudinal infant cohort with a total of 19451 available general movements video snippets were randomly selected for human clinical reasoning and computer-based analysis. We demonstrated for the first time that pseudonymisation by face-blurring video recordings is a viable approach. The video redaction did not affect classification accuracy for either human assessors or computer vision methods, suggesting an adequate and easy-to-apply solution for sharing movement video data. We call for further explorations into efficient and privacy rule-conforming approaches for deidentifying video data in scientific and clinical fields beyond movement assessments. These approaches shall enable sharing and merging stand-alone video datasets into large data pools to advance science and public health.
MobileDenseNet: A new approach to object detection on mobile devices
Venue: arXiv cs.CV 2022
Year: 2022

Object detection problem solving has developed greatly within the past few years. There is a need for lighter models in instances where hardware limitations exist, as well as a demand for models to be tailored to mobile devices. In this article, we will assess the methods used when creating algorithms that address these issues. The main goal of this article is to increase accuracy in state-of-the-art algorithms while maintaining speed and real-time efficiency. The most significant issues in one-stage object detection pertains to small objects and inaccurate localization. As a solution, we created a new network by the name of MobileDenseNet suitable for embedded systems. We also developed a light neck FCPNLite for mobile devices that will aid with the detection of small objects. Our research revealed that very few papers cited necks in embedded systems. What differentiates our network from others is our use of concatenation features. A small yet significant change to the head of the network amplified accuracy without increasing speed or limiting parameters. In short, our focus on the challenging CoCo and Pascal VOC datasets were 24.8 and 76.8 in percentage terms respectively - a rate higher than that recorded by other state-of-the-art systems thus far. Our network is able to increase accuracy while maintaining real-time efficiency on mobile devices. We calculated operational speed on Pixel 3 (Snapdragon 845) to 22.8 fps. The source code of this research is available on https://github.com/hajizadeh/MobileDenseNet.
Quantized Sparse Weight Decomposition for Neural Network Compression
Venue: arXiv cs.LG 2022
Year: 2022

In this paper, we introduce a novel method of neural network weight compression. In our method, we store weight tensors as sparse, quantized matrix factors, whose product is computed on the fly during inference to generate the target model's weights. We use projected gradient descent methods to find quantized and sparse factorization of the weight tensors. We show that this approach can be seen as a unification of weight SVD, vector quantization, and sparse PCA. Combined with end-to-end fine-tuning our method exceeds or is on par with previous state-of-the-art methods in terms of the trade-off between accuracy and model size. Our method is applicable to both moderate compression regimes, unlike vector quantization, and extreme compression regimes.
Classification via score-based generative modelling
Venue: arXiv cs.LG 2022
Year: 2022

In this work, we investigated the application of score-based gradient learning in discriminative and generative classification settings. Score function can be used to characterize data distribution as an alternative to density. It can be efficiently learned via score matching, and used to flexibly generate credible samples to enhance discriminative classification quality, to recover density and to build generative classifiers. We analysed the decision theories involving score-based representations, and performed experiments on simulated and real-world datasets, demonstrating its effectiveness in achieving and improving binary classification performance, and robustness to perturbations, particularly in high dimensions and imbalanced situations.
Zero-Shot Video Captioning with Evolving Pseudo-Tokens
Venue: arXiv cs.CV 2022
Year: 2022

We introduce a zero-shot video captioning method that employs two frozen networks: the GPT-2 language model and the CLIP image-text matching model. The matching score is used to steer the language model toward generating a sentence that has a high average matching score to a subset of the video frames. Unlike zero-shot image captioning methods, our work considers the entire sentence at once. This is achieved by optimizing, during the generation process, part of the prompt from scratch, by modifying the representation of all other tokens in the prompt, and by repeating the process iteratively, gradually improving the specificity and comprehensiveness of the generated sentence. Our experiments show that the generated captions are coherent and display a broad range of real-world knowledge. Our code is available at: https://github.com/YoadTew/zero-shot-video-to-text
Near Real-Time Distributed State Estimation via AI/ML-Empowered 5G Networks
Venue: arXiv cs.LG 2022
Year: 2022

Fifth-Generation (5G) networks have a potential to accelerate power system transition to a flexible, softwarized, data-driven, and intelligent grid. With their evolving support for Machine Learning (ML)/Artificial Intelligence (AI) functions, 5G networks are expected to enable novel data-centric Smart Grid (SG) services. In this paper, we explore how data-driven SG services could be integrated with ML/AI-enabled 5G networks in a symbiotic relationship. We focus on the State Estimation (SE) function as a key element of the energy management system and focus on two main questions. Firstly, in a tutorial fashion, we present an overview on how distributed SE can be integrated with the elements of the 5G core network and radio access network architecture. Secondly, we present and compare two powerful distributed SE methods based on: i) graphical models and belief propagation, and ii) graph neural networks. We discuss their performance and capability to support a near real-time distributed SE via 5G network, taking into account communication delays.
On Controller Tuning with Time-Varying Bayesian Optimization
Venue: arXiv cs.LG 2022
Year: 2022

Changing conditions or environments can cause system dynamics to vary over time. To ensure optimal control performance, controllers should adapt to these changes. When the underlying cause and time of change is unknown, we need to rely on online data for this adaptation. In this paper, we will use time-varying Bayesian optimization (TVBO) to tune controllers online in changing environments using appropriate prior knowledge on the control objective and its changes. Two properties are characteristic of many online controller tuning problems: First, they exhibit incremental and lasting changes in the objective due to changes to the system dynamics, e.g., through wear and tear. Second, the optimization problem is convex in the tuning parameters. Current TVBO methods do not explicitly account for these properties, resulting in poor tuning performance and many unstable controllers through over-exploration of the parameter space. We propose a novel TVBO forgetting strategy using Uncertainty-Injection (UI), which incorporates the assumption of incremental and lasting changes. The control objective is modeled as a spatio-temporal Gaussian process (GP) with UI through a Wiener process in the temporal domain. Further, we explicitly model the convexity assumptions in the spatial dimension through GP models with linear inequality constraints. In numerical experiments, we show that our model outperforms the state-of-the-art method in TVBO, exhibiting reduced regret and fewer unstable parameter configurations.
Fault Tolerant Coloring of the Asynchronous Cycle
Venue: arXiv cs.DC 2022
Year: 2022

We present a wait-free algorithm for proper coloring the n nodes of the asynchronous cycle $C_n$, where each crash-prone node starts with its (unique) identifier as input. The algorithm is independent of $n \geq 3$, and runs in $\mathrm{O}(\log^* n)$ rounds in $C_n$. This round-complexity is optimal thanks to a known matching lower bound, which applies even to synchronous (failure-free) executions. The range of colors used by our algorithm, namely $\{0, ..., 4\}$, is optimal too, thanks to a known lower bound on the minimum number of names for which renaming is solvable wait-free in shared-memory systems, whenever $n$ is a power of a prime. Indeed, our model coincides with the shared-memory model whenever $n = 3$, and the minimum number of names for which renaming is possible in 3-process shared-memory systems is 5.
Toward a Generic Mapping Language for Transformations between RDF and Data Interchange Formats
Venue: arXiv cs.DB 2022
Year: 2022

While there exist approaches to integrate heterogeneous data using semantic models, such semantic models can typically not be used by existing software tools. Many software tools - especially in engineering - only have options to import and export data in more established data interchange formats such as XML or JSON. Thus, if an information which is included in a semantic model needs to be used in a such a software tool, automatic approaches for mapping semantic information into an interchange format are needed. We aim to develop a generic mapping approach that allows users to create transformations of semantic information into a data interchange format with an arbitrary structure which can be defined by a user. This mapping approach is currently being elaborated. In this contribution, we report our initial steps targeted to transformations from RDF into XML. At first, a mapping language is introduced which allows to define automated mappings from ontologies to XML. Furthermore, a mapping algorithm capable of executing mappings defined in this language is presented. An evaluation is done with a use case in which engineering information needs to be used in a 3D modeling tool.
Improving Predictive Performance and Calibration by Weight Fusion in Semantic Segmentation
Venue: arXiv cs.CV 2022
Year: 2022

Averaging predictions of a deep ensemble of networks is apopular and effective method to improve predictive performance andcalibration in various benchmarks and Kaggle competitions. However, theruntime and training cost of deep ensembles grow linearly with the size ofthe ensemble, making them unsuitable for many applications. Averagingensemble weights instead of predictions circumvents this disadvantageduring inference and is typically applied to intermediate checkpoints ofa model to reduce training cost. Albeit effective, only few works haveimproved the understanding and the performance of weight averaging.Here, we revisit this approach and show that a simple weight fusion (WF)strategy can lead to a significantly improved predictive performance andcalibration. We describe what prerequisites the weights must meet interms of weight space, functional space and loss. Furthermore, we presenta new test method (called oracle test) to measure the functional spacebetween weights. We demonstrate the versatility of our WF strategy acrossstate of the art segmentation CNNs and Transformers as well as real worlddatasets such as BDD100K and Cityscapes. We compare WF with similarapproaches and show our superiority for in- and out-of-distribution datain terms of predictive performance and calibration.
Defending Substitution-Based Profile Pollution Attacks on Sequential Recommenders
Venue: arXiv cs.IR 2022
Year: 2022

While sequential recommender systems achieve significant improvements on capturing user dynamics, we argue that sequential recommenders are vulnerable against substitution-based profile pollution attacks. To demonstrate our hypothesis, we propose a substitution-based adversarial attack algorithm, which modifies the input sequence by selecting certain vulnerable elements and substituting them with adversarial items. In both untargeted and targeted attack scenarios, we observe significant performance deterioration using the proposed profile pollution algorithm. Motivated by such observations, we design an efficient adversarial defense method called Dirichlet neighborhood sampling. Specifically, we sample item embeddings from a convex hull constructed by multi-hop neighbors to replace the original items in input sequences. During sampling, a Dirichlet distribution is used to approximate the probability distribution in the neighborhood such that the recommender learns to combat local perturbations. Additionally, we design an adversarial training method tailored for sequential recommender systems. In particular, we represent selected items with one-hot encodings and perform gradient ascent on the encodings to search for the worst case linear combination of item embeddings in training. As such, the embedding function learns robust item representations and the trained recommender is resistant to test-time adversarial examples. Extensive experiments show the effectiveness of both our attack and defense methods, which consistently outperform baselines by a significant margin across model architectures and datasets.
Towards the shortest DRAT proof of the Pigeonhole Principle
Venue: arXiv cs.LO 2022
Year: 2022

The Pigeonhole Principle (PHP) has been heavily studied in automated reasoning, both theoretically and in practice. Most solvers have exponential runtime and proof length, while some specialized techniques achieve polynomial runtime and proof length. Several decades ago, Cook manually constructed $O(n^4)$ extended resolution proofs, where $n$ denotes the number of pigeons.Existing automated techniques only surpass Cook's proofs in similar proof systems for large $n$. We construct the shortest known proofs of PHP in the standard proof format of modern SAT solving, DRAT. Using auxiliary variables and by recursively decomposing the original program into smaller sizes, we manually obtain proofs having length $O(n^3)$ and leading coefficient $5/2$.
A flexible PageRank-based graph embedding framework closely related to spectral eigenvector embeddings
Venue: arXiv cs.SI 2022
Year: 2022

We study a simple embedding technique based on a matrix of personalized PageRank vectors seeded on a random set of nodes. We show that the embedding produced by the element-wise logarithm of this matrix (1) are related to the spectral embedding for a class of graphs where spectral embeddings are significant, and hence useful representation of the data, (2) can be done for the entire network or a smaller part of it, which enables precise local representation, and (3) uses a relatively small number of PageRank vectors compared to the size of the networks. Most importantly, the general nature of this embedding strategy opens up many emerging applications, where eigenvector and spectral techniques may not be well established, to the PageRank-based relatives. For instance, similar techniques can be used on PageRank vectors from hypergraphs to get "spectral-like" embeddings.
CoqQ: Foundational Verification of Quantum Programs
Venue: arXiv cs.PL 2022
Year: 2022

CoqQ is a framework for reasoning about quantum programs in the Coq proof assistant. Its main components are: a deeply embedded quantum programming language, in which classic quantum algorithms are easily expressed, and an expressive program logic for proving properties of programs. CoqQ is foundational: the program logic is formally proved sound with respect to a denotational semantics based on state-of-art mathematical libraries (mathcomp and mathcomp analysis). CoqQ is also practical: assertions can use Dirac expressions, which eases concise specifications, and proofs can exploit local and parallel reasoning, which minimizes verification effort. We illustrate the applicability of CoqQ with many examples from the literature.