OpenRaiser/Intern-Atlas huggingface dataset:
Retrieval Augmentation for T5 Re-ranker using External Sources
Venue: arXiv cs.IR 2022
Year: 2022

Retrieval augmentation has shown promising improvements in different tasks. However, whether such augmentation can assist a large language model based re-ranker remains unclear. We investigate how to augment T5-based re-rankers using high-quality information retrieved from two external corpora -- a commercial web search engine and Wikipedia. We empirically demonstrate how retrieval augmentation can substantially improve the effectiveness of T5-based re-rankers for both in-domain and zero-shot out-of-domain re-ranking tasks.
Fine-Grained Image Style Transfer with Visual Transformers
Venue: arXiv cs.CV 2022
Year: 2022

With the development of the convolutional neural network, image style transfer has drawn increasing attention. However, most existing approaches adopt a global feature transformation to transfer style patterns into content images (e.g., AdaIN and WCT). Such a design usually destroys the spatial information of the input images and fails to transfer fine-grained style patterns into style transfer results. To solve this problem, we propose a novel STyle TRansformer (STTR) network which breaks both content and style images into visual tokens to achieve a fine-grained style transformation. Specifically, two attention mechanisms are adopted in our STTR. We first propose to use self-attention to encode content and style tokens such that similar tokens can be grouped and learned together. We then adopt cross-attention between content and style tokens that encourages fine-grained style transformations. To compare STTR with existing approaches, we conduct user studies on Amazon Mechanical Turk (AMT), which are carried out with 50 human subjects with 1,000 votes in total. Extensive evaluations demonstrate the effectiveness and efficiency of the proposed STTR in generating visually pleasing style transfer results.
Abstract interpretation of Michelson smart-contracts
Venue: arXiv cs.CR 2022
Year: 2022

Static analysis of smart-contracts is becoming more widespread on blockchain platforms. Analyzers rely on techniques like symbolic execution or model checking, but few of them can provide strong soundness properties and guarantee the analysis termination at the same time. As smart-contracts often manipulate economic assets, proving numerical properties beyond the absence of runtime errors is also desirable. Smart-contract execution models differ considerably from mainstream programming languages and vary from one blockchain to another, making state-of-the-art analyses hard to adapt. For instance, smart-contract calls may modify a persistent storage impacting subsequent calls. This makes it difficult for tools to infer invariants required to formally ensure the absence of exploitable vulnerabilities. The Michelson smart-contract language, used in the Tezos blockchain, is strongly typed, stack-based, and has a strict execution model leaving few opportunities for implicit runtime errors. We present a work in progress static analyzer for Michelson based on Abstract Interpretation and implemented within MOPSA, a modular static analyzer. Our tool supports the Michelson semantic features, including inner calls to external contracts. It can prove the absence of runtime errors and infer invariants on the persistent storage over an unbounded number of calls. It is also being extended to prove high-level numerical and security properties. CCS Concepts: $\bullet$ Security and privacy $\rightarrow$ Logic and verification; $\bullet$ Software and its engineering $\rightarrow$ Automated static analysis.
Fair and Efficient Multi-Resource Allocation for Cloud Computing
Venue: arXiv cs.GT 2022
Year: 2022

We study the problem of allocating multiple types of resources to agents with Leontief preferences. The classic Dominant Resource Fairness (DRF) mechanism satisfies several desired fairness and incentive properties, but is known to have poor performance in terms of social welfare approximation ratio. In this work, we propose a new approximation ratio measure, called \emph{\fratio}, which is defined as the worst-case ratio between the optimal social welfare (resp. utilization) among all \emph{fair} allocations and that by the mechanism, allowing us to break the lower bound barrier under the classic approximation ratio. We then generalize DRF and present several new mechanisms with two and multiple types of resources that satisfy the same set of properties as DRF but with better social welfare and utilization guarantees under the new benchmark. We also demonstrate the effectiveness of these mechanisms through experiments on both synthetic and real-world datasets.
Multi-site Diagnostic Classification Of Schizophrenia Using 3D CNN On Aggregated Task-based fMRI Data
Venue: arXiv cs.CV 2022
Year: 2022

In spite of years of research, the mechanisms that underlie the development of schizophrenia, as well as its relapse, symptomatology, and treatment, continue to be a mystery. The absence of appropriate analytic tools to deal with the variable and complicated nature of schizophrenia may be one of the factors that contribute to the development of this disorder. Deep learning is a subfield of artificial intelligence that was inspired by the nervous system. In recent years, deep learning has made it easier to model and analyse complicated, high-dimensional, and nonlinear systems. Research on schizophrenia is one of the many areas of study that has been revolutionised as a result of the outstanding accuracy that deep learning algorithms have demonstrated in classification and prediction tasks. Deep learning has the potential to become a powerful tool for understanding the mechanisms that are at the root of schizophrenia. In addition, a growing variety of techniques aimed at improving model interpretability and causal reasoning are contributing to this trend. Using multi-site fMRI data and a variety of deep learning approaches, this study seeks to identify different types of schizophrenia. Our proposed method of temporal aggregation of the 4D fMRI data outperforms existing work. In addition, this study aims to shed light on the strength of connections between various brain areas in schizophrenia individuals.
STSC-SNN: Spatio-Temporal Synaptic Connection with Temporal Convolution and Attention for Spiking Neural Networks
Venue: arXiv cs.NE 2022
Year: 2022

Spiking Neural Networks (SNNs), as one of the algorithmic models in neuromorphic computing, have gained a great deal of research attention owing to temporal information processing capability, low power consumption, and high biological plausibility. The potential to efficiently extract spatio-temporal features makes it suitable for processing the event streams. However, existing synaptic structures in SNNs are almost full-connections or spatial 2D convolution, neither of which can extract temporal dependencies adequately. In this work, we take inspiration from biological synapses and propose a spatio-temporal synaptic connection SNN (STSC-SNN) model, to enhance the spatio-temporal receptive fields of synaptic connections, thereby establishing temporal dependencies across layers. Concretely, we incorporate temporal convolution and attention mechanisms to implement synaptic filtering and gating functions. We show that endowing synaptic models with temporal dependencies can improve the performance of SNNs on classification tasks. In addition, we investigate the impact of performance vias varied spatial-temporal receptive fields and reevaluate the temporal modules in SNNs. Our approach is tested on neuromorphic datasets, including DVS128 Gesture (gesture recognition), N-MNIST, CIFAR10-DVS (image classification), and SHD (speech digit recognition). The results show that the proposed model outperforms the state-of-the-art accuracy on nearly all datasets.
Constrained Deployment Optimization in Integrated Access and Backhaul Networks
Venue: arXiv cs.NI 2022
Year: 2022

Integrated access and backhaul (IAB) is one of the promising techniques for 5G networks and beyond (6G), in which the same node/hardware is used to provide both backhaul and cellular services in a multi-hop fashion. Due to the sensitivity of the backhaul links with high rate/reliability demands, proper network planning is needed to make the IAB network performing appropriately and as good as possible. In this paper, we study the effect of deployment optimization on the coverage of IAB networks. We concentrate on the cases where, due to either geographical or interference management limitations, unconstrained IAB node placement is not feasible in some areas. To that end, we propose various millimeter wave (mmWave) blocking-aware constrained deployment optimization approaches. Our results indicate that, even with limitations on deployment optimization, network planning boosts the coverage of IAB networks considerably.
Computer Vision based inspection on post-earthquake with UAV synthetic dataset
Venue: arXiv cs.CV 2022
Year: 2022

The area affected by the earthquake is vast and often difficult to entirely cover, and the earthquake itself is a sudden event that causes multiple defects simultaneously, that cannot be effectively traced using traditional, manual methods. This article presents an innovative approach to the problem of detecting damage after sudden events by using an interconnected set of deep machine learning models organized in a single pipeline and allowing for easy modification and swapping models seamlessly. Models in the pipeline were trained with a synthetic dataset and were adapted to be further evaluated and used with unmanned aerial vehicles (UAVs) in real-world conditions. Thanks to the methods presented in the article, it is possible to obtain high accuracy in detecting buildings defects, segmenting constructions into their components and estimating their technical condition based on a single drone flight.
Learning Control Policies for Stochastic Systems with Reach-avoid Guarantees
Venue: arXiv cs.LG 2022
Year: 2022

We study the problem of learning controllers for discrete-time non-linear stochastic dynamical systems with formal reach-avoid guarantees. This work presents the first method for providing formal reach-avoid guarantees, which combine and generalize stability and safety guarantees, with a tolerable probability threshold $p\in[0,1]$ over the infinite time horizon. Our method leverages advances in machine learning literature and it represents formal certificates as neural networks. In particular, we learn a certificate in the form of a reach-avoid supermartingale (RASM), a novel notion that we introduce in this work. Our RASMs provide reachability and avoidance guarantees by imposing constraints on what can be viewed as a stochastic extension of level sets of Lyapunov functions for deterministic systems. Our approach solves several important problems -- it can be used to learn a control policy from scratch, to verify a reach-avoid specification for a fixed control policy, or to fine-tune a pre-trained policy if it does not satisfy the reach-avoid specification. We validate our approach on $3$ stochastic non-linear reinforcement learning tasks.
Enhancing Branch-and-Bound for Multi-Objective 0-1 Programming
Venue: arXiv cs.DS 2022
Year: 2022

In the bi-objective branch-and-bound literature, a key ingredient is objective branching, i.e. to create smaller and disjoint sub-problems in the objective space, obtained from the partial dominance of the lower bound set by the upper bound set. When considering three or more objective functions, however, applying objective branching becomes more complex, and its benefit has so far been unclear. In this paper, we investigate several ingredients which allow to better exploit objective branching in a multi-objective setting. We extend the idea of probing to multiple objectives, enhance it in several ways, and show that when coupled with objective branching, it results in significant speed-ups in terms of CPU times. We also investigate cut generation based on the objective branching constraints. Besides, we generalize the best-bound idea for node selection to multiple objectives and we show that the proposed rules outperform the, in the multi-objective literature, commonly employed depth-first and breadth-first strategies. We also analyze problem specific branching rules. We test the proposed ideas on available benchmark instances for three problem classes with three and four objectives, namely the capacitated facility location problem, the uncapacitated facility location problem, and the knapsack problem. Our enhanced multi-objective branch-and-bound algorithm outperforms the best existing branch-and-bound based approach and is the first to obtain competitive and even slightly better results than a state-of-the-art objective space search method on a subset of the problem classes.
Block Format Error Bounds and Optimal Block Size Selection
Venue: arXiv cs.LG 2022
Year: 2022

The amounts of data that need to be transmitted, processed, and stored by the modern deep neural networks have reached truly enormous volumes in the last few years calling for the invention of new paradigms both in hardware and software development. One of the most promising and rapidly advancing frontiers here is the creation of new numerical formats. In this work we focus on the family of block floating point numerical formats due to their combination of wide dynamic range, numerical accuracy, and efficient hardware implementation of inner products using simple integer arithmetic. These formats are characterized by a block of mantissas with a shared scale factor. The basic Block Floating Point (BFP) format quantizes the block scales into the nearest powers of two on the right. Its simple modification - Scaled BFP (SBFP) - stores the same scales in full precision and thus allows higher accuracy. In this paper, we study the statistical behavior of both these formats rigorously. We develop asymptotic bounds on the inner product error in SBFP- and BFP-quantized normally distributed vectors. Next, we refine those asymptotic results to finite dimensional settings and derive high-dimensional tight bounds for the same errors. Based on the obtained results we introduce a performance measure assessing accuracy of any block format. This measure allows us to determine the optimal parameters, such as the block size, yielding highest accuracy. In particular, we show that if the precision of the BFP format is fixed at 4 bits, the optimal block size becomes 64. All theoretical derivations are supported by numerical experiments and studies on the weights of publicly available pretrained neural networks.
On the Interpolation of Contextualized Term-based Ranking with BM25 for Query-by-Example Retrieval
Venue: arXiv cs.IR 2022
Year: 2022

Term-based ranking with pre-trained transformer-based language models has recently gained attention as they bring the contextualization power of transformer models into the highly efficient term-based retrieval. In this work, we examine the generalizability of two of these deep contextualized term-based models in the context of query-by-example (QBE) retrieval in which a seed document acts as the query to find relevant documents. In this setting -- where queries are much longer than common keyword queries -- BERT inference at query time is problematic as it involves quadratic complexity. We investigate TILDE and TILDEv2, both of which leverage BERT tokenizer as their query encoder. With this approach, there is no need for BERT inference at query time, and also the query can be of any length. Our extensive evaluation on the four QBE tasks of SciDocs benchmark shows that in a query-by-example retrieval setting TILDE and TILDEv2 are still less effective than a cross-encoder BERT ranker. However, we observe that BM25 could show a competitive ranking quality compared to TILDE and TILDEv2 which is in contrast to the findings about the relative performance of these three models on retrieval for short queries reported in prior work. This result raises the question about the use of contextualized term-based ranking models being beneficial in QBE setting. We follow-up on our findings by studying the score interpolation between the relevance score from TILDE (TILDEv2) and BM25. We conclude that these two contextualized term-based ranking models capture different relevance signals than BM25 and combining the different term-based rankers results in statistically significant improvements in QBE retrieval. Our work sheds light on the challenges of retrieval settings different from the common evaluation benchmarks.
What does a deep neural network confidently perceive? The effective dimension of high certainty class manifolds and their low confidence boundaries
Venue: arXiv cs.LG 2022
Year: 2022

Deep neural network classifiers partition input space into high confidence regions for each class. The geometry of these class manifolds (CMs) is widely studied and intimately related to model performance; for example, the margin depends on CM boundaries. We exploit the notions of Gaussian width and Gordon's escape theorem to tractably estimate the effective dimension of CMs and their boundaries through tomographic intersections with random affine subspaces of varying dimension. We show several connections between the dimension of CMs, generalization, and robustness. In particular we investigate how CM dimension depends on 1) the dataset, 2) architecture (including ResNet, WideResNet \& Vision Transformer), 3) initialization, 4) stage of training, 5) class, 6) network width, 7) ensemble size, 8) label randomization, 9) training set size, and 10) robustness to data corruption. Together a picture emerges that higher performing and more robust models have higher dimensional CMs. Moreover, we offer a new perspective on ensembling via intersections of CMs. Our code is at https://github.com/stanislavfort/slice-dice-optimize/
Multi-Agent Distributed and Decentralized Geometric Task Allocation
Venue: arXiv cs.MA 2022
Year: 2022

We consider the general problem of geometric task allocation, wherein a large, decentralised swarm of simple mobile agents must detect the locations of tasks in the plane and position themselves nearby. The tasks are represented by an a priori unknown demand profile $Φ(x,y)$ that determines how many agents are needed in each location. The agents are autonomous, oblivious and indistinguishable, and have finite sensing range. They must configure themselves according to $Φ$ using only local information about $Φ$ and about the positions of nearby agents. All agents act according to the same local sensing-based rule of motion, and cannot explicitly communicate nor share information. We propose an optimization-based approach to the problem which results in attraction-repulsion dynamics. Repulsion encourages agents to spread out and explore the region so as to find the tasks, and attraction causes them to accumulate at task locations. We derive this approach via gradient descent over an appropriate ``error'' functional, and test it extensively through numerical simulations. The figures in this work are snapshots of simulations that can be viewed online at https://youtu.be/kyUiGYSaaoQ.
Combining Relaying and Reflective Surfaces: Power Consumption and Energy Efficiency Analysis
Venue: arXiv cs.IT 2022
Year: 2022

Hybrid relaying networks (HRNs) combining both a relay and an intelligent reflective surface (IRS) can lead to enhanced rate performance compared to non-hybrid relaying schemes, where only either an IRS or a relay is utilized. However, utilizing both the relay and the IRS simultaneously results in higher power consumption for the HRNs compared to their counterpart. In this work, we study the required transmit power levels and the energy efficiency (EE) of HRNs utilizing both a half-duplex decode-and-forward (HD-DF) relay and an IRS, and compare their performance with non-hybrid relaying schemes. The impact of the required channel estimation overheads is considered when the reflective beamforming design (RBD) at the IRS is carried out under both instantaneous and statistical channel state information models. Also, the investigation is performed for both slow- and fast-changing environments. In terms of the transmit power requirements, our results show that HRNs can lead to higher power savings if the number of reflective elements at the IRS is not very large. However, non-hybrid relaying schemes are shown to be more energy-efficient, unless the targeted rate is high and the IRS is distant from both transmitter and receiver but within a close proximity to the relay.
Observability Analysis of Graph SLAM-Based Joint Calibration of Multiple Microphone Arrays and Sound Source Localization
Venue: arXiv cs.RO 2022
Year: 2022

Multiple microphone arrays have many applications in robot audition, including sound source localization, audio scene perception and analysis, etc. However, accurate calibration of multiple microphone arrays remains a challenge because there are many unknown parameters to be identified, including the Euler angles, geometry, asynchronous factors between the microphone arrays. This paper is concerned with joint calibration of multiple microphone arrays and sound source localization using graph simultaneous localization and mapping (SLAM). By using a Fisher information matrix (FIM) approach, we focus on the observability analysis of the graph SLAM framework for the above-mentioned calibration problem. We thoroughly investigate the identifiability of the unknown parameters, including the Euler angles, geometry, asynchronous effects between the microphone arrays, and the sound source locations. We establish necessary/sufficient conditions under which the FIM and the Jacobian matrix have full column rank, which implies the identifiability of the unknown parameters. These conditions are closely related to the variation in the motion of the sound source and the configuration of microphone arrays, and have intuitive and physical interpretations. We also discover several scenarios where the unknown parameters are not uniquely identifiable. All theoretical findings are demonstrated using simulation data.
An Experimental Study on Private Aggregation of Teacher Ensemble Learning for End-to-End Speech Recognition
Venue: arXiv cs.SD 2022
Year: 2022

Differential privacy (DP) is one data protection avenue to safeguard user information used for training deep models by imposing noisy distortion on privacy data. Such a noise perturbation often results in a severe performance degradation in automatic speech recognition (ASR) in order to meet a privacy budget $\varepsilon$. Private aggregation of teacher ensemble (PATE) utilizes ensemble probabilities to improve ASR accuracy when dealing with the noise effects controlled by small values of $\varepsilon$. We extend PATE learning to work with dynamic patterns, namely speech utterances, and perform a first experimental demonstration that it prevents acoustic data leakage in ASR training. We evaluate three end-to-end deep models, including LAS, hybrid CTC/attention, and RNN transducer, on the open-source LibriSpeech and TIMIT corpora. PATE learning-enhanced ASR models outperform the benchmark DP-SGD mechanisms, especially under strict DP budgets, giving relative word error rate reductions between 26.2% and 27.5% for an RNN transducer model evaluated with LibriSpeech. We also introduce a DP-preserving ASR solution for pretraining on public speech corpora.
On Landau's Eigenvalue Theorem for Line-of-Sight MIMO Channels
Venue: arXiv cs.IT 2022
Year: 2022

An alternative derivation is provided for the degrees of freedom (DOF) formula on line-of-sight (LOS) channels via Landau's eigenvalue theorem for bandlimited signals. Compared to other approaches, Landau's theorem provides a general framework to compute the DOF in arbitrary environments, this framework is herein specialized to LOS propagation. The development shows how the spatially bandlimited nature of the channel relates to its geometry under the paraxial approximation that applies to most LOS settings of interest.