OpenRaiser/Intern-Atlas huggingface dataset:
On Comparison of Encoders for Attention based End to End Speech Recognition in Standalone and Rescoring Mode
Venue: arXiv cs.SD 2022
Year: 2022

The streaming automatic speech recognition (ASR) models are more popular and suitable for voice-based applications. However, non-streaming models provide better performance as they look at the entire audio context. To leverage the benefits of the non-streaming model in streaming applications like voice search, it is commonly used in second pass re-scoring mode. The candidate hypothesis generated using steaming models is re-scored using a non-streaming model. In this work, we evaluate the non-streaming attention-based end-to-end ASR models on the Flipkart voice search task in both standalone and re-scoring modes. These models are based on Listen-Attend-Spell (LAS) encoder-decoder architecture. We experiment with different encoder variations based on LSTM, Transformer, and Conformer. We compare the latency requirements of these models along with their performance. Overall we show that the Transformer model offers acceptable WER with the lowest latency requirements. We report a relative WER improvement of around 16% with the second pass LAS re-scoring with latency overhead under 5ms. We also highlight the importance of CNN front-end with Transformer architecture to achieve comparable word error rates (WER). Moreover, we observe that in the second pass re-scoring mode all the encoders provide similar benefits whereas the difference in performance is prominent in standalone text generation mode.
Multi-Access Distributed Computing
Venue: arXiv cs.IT 2022
Year: 2022

Coded distributed computing (CDC) is a new technique proposed with the purpose of decreasing the intense data exchange required for parallelizing distributed computing systems. Under the famous MapReduce paradigm, this coded approach has been shown to decrease this communication overhead by a factor that is linearly proportional to the overall computation load during the mapping phase. Nevertheless, it is widely accepted that this overhead remains a main bottleneck in distributed computing. To address this, we take a new approach and we explore a new system model which, for the same aforementioned overall computation load of the mapping phase, manages to provide astounding reductions of the communication overhead and, perhaps counterintuitively, a substantial increase of the computational parallelization. In particular, we propose multi-access distributed computing (MADC) as a novel generalization of the original CDC model, where now mappers and reducers are distinct computing nodes that are connected through a multi-access network topology. Focusing on the MADC setting with combinatorial topology, which implies $Λ$ mappers and $K$ reducers such that there is a unique reducer connected to any $α$ mappers, we propose a novel coded scheme and a novel information-theoretic converse, which jointly identify the optimal inter-reducer communication load to within a constant gap of $1.5$. Additionally, a modified coded scheme and converse identify the optimal max-link communication load across all existing links to within a gap of $4$. The unparalleled coding gains reported here should not be simply credited to having access to more mapped data, but rather to the powerful role of topology in effectively aligning mapping outputs. This realization raises the open question of which multi-access network topology guarantees the best possible performance in distributed computing.
Contextual embedding and model weighting by fusing domain knowledge on Biomedical Question Answering
Venue: arXiv cs.CL 2022
Year: 2022

Biomedical Question Answering aims to obtain an answer to the given question from the biomedical domain. Due to its high requirement of biomedical domain knowledge, it is difficult for the model to learn domain knowledge from limited training data. We propose a contextual embedding method that combines open-domain QA model \aoa and \biobert model pre-trained on biomedical domain data. We adopt unsupervised pre-training on large biomedical corpus and supervised fine-tuning on biomedical question answering dataset. Additionally, we adopt an MLP-based model weighting layer to automatically exploit the advantages of two models to provide the correct answer. The public dataset \biomrc constructed from PubMed corpus is used to evaluate our method. Experimental results show that our model outperforms state-of-the-art system by a large margin.
Edge Direction-invariant Graph Neural Networks for Molecular Dipole Moments Prediction
Venue: arXiv cs.LG 2022
Year: 2022

The dipole moment is a physical quantity indicating the polarity of a molecule and is determined by reflecting the electrical properties of constituent atoms and the geometric properties of the molecule. Most embeddings used to represent graph representations in traditional graph neural network methodologies treat molecules as topological graphs, creating a significant barrier to the goal of recognizing geometric information. Unlike existing embeddings dealing with equivariance, which have been proposed to handle the 3D structure of molecules properly, our proposed embeddings directly express the physical implications of the local contribution of dipole moments. We show that the developed model works reasonably even for molecules with extended geometries and captures more interatomic interaction information, significantly improving the prediction results with accuracy comparable to ab-initio calculations.
On the Impossibility of Decomposing Binary Matroids
Venue: arXiv cs.DS 2022
Year: 2022

We show that there exist $k$-colorable matroids that are not $(b,c)$-decomposable when $b$ and $c$ are constants. A matroid is $(b,c)$-decomposable, if its ground set of elements can be partitioned into sets $X_1, X_2, \ldots, X_l$ with the following two properties. Each set $X_i$ has size at most $ck$. Moreover, for all sets $Y$ such that $|Y \cap X_i| \leq 1$ it is the case that $Y$ is $b$-colorable. A $(b,c)$-decomposition is a strict generalization of a partition decomposition and, thus, our result refutes a conjecture from arXiv:1911.10485v2 .
FAIR-BFL: Flexible and Incentive Redesign for Blockchain-based Federated Learning
Venue: arXiv cs.DC 2022
Year: 2022

Vanilla Federated learning (FL) relies on the centralized global aggregation mechanism and assumes that all clients are honest. This makes it a challenge for FL to alleviate the single point of failure and dishonest clients. These impending challenges in the design philosophy of FL call for blockchain-based federated learning (BFL) due to the benefits of coupling FL and blockchain (e.g., democracy, incentive, and immutability). However, one problem in vanilla BFL is that its capabilities do not follow adopters' needs in a dynamic fashion. Besides, vanilla BFL relies on unverifiable clients' self-reported contributions like data size because checking clients' raw data is not allowed in FL for privacy concerns. We design and evaluate a novel BFL framework, and resolve the identified challenges in vanilla BFL with greater flexibility and incentive mechanism called FAIR-BFL. In contrast to existing works, FAIR-BFL offers unprecedented flexibility via the modular design, allowing adopters to adjust its capabilities following business demands in a dynamic fashion. Our design accounts for BFL's ability to quantify each client's contribution to the global learning process. Such quantification provides a rational metric for distributing the rewards among federated clients and helps discover malicious participants that may poison the global model.
Disambiguating Disinformation: Extending Beyond the Veracity of Online Content
Venue: arXiv cs.SI 2022
Year: 2022

Following the 2016 US presidential election and the now overwhelming evidence of Russian interference, there has been an explosion of interest in the phenomenon of "fake news". To date, research on false news has centered around detecting content from low-credibility sources and analyzing how this content spreads across online platforms. Misinformation poses clear risks, yet research agendas that overemphasize veracity miss the opportunity to truly understand the Kremlin-led disinformation campaign that shook so many Americans. In this paper, we present a definition for disinformation - a set or sequence of orchestrated, agenda-driven information actions with the intent to deceive - that is useful in contextualizing Russian interference in 2016 and disinformation campaigns more broadly. We expand on our ongoing work to operationalize this definition and demonstrate how detecting disinformation must extend beyond assessing the credibility of a specific publisher, user, or story.
Non-Parametric Style Transfer
Venue: arXiv cs.CV 2022
Year: 2022

Recent feed-forward neural methods of arbitrary image style transfer mainly utilized encoded feature map upto its second-order statistics, i.e., linearly transformed the encoded feature map of a content image to have the same mean and variance (or covariance) of a target style feature map. In this work, we extend the second-order statistical feature matching into a general distribution matching based on the understanding that style of an image is represented by the distribution of responses from receptive fields. For this generalization, first, we propose a new feature transform layer that exactly matches the feature map distribution of content image into that of target style image. Second, we analyze the recent style losses consistent with our new feature transform layer to train a decoder network which generates a style transferred image from the transformed feature map. Based on our experimental results, it is proven that the stylized images obtained with our method are more similar with the target style images in all existing style measures without losing content clearness.
Visualizing Non-Fungible Token Ethics: A Case Study On CryptoPunks
Venue: arXiv cs.HC 2022
Year: 2022

As a blockchain-based application, Non-Fungible Token (NFT) has received worldwide attention over the past few years. Digital artwork is the main form of NFT that can be stored on different blockchains. Although the NFT market is rapidly developing, we observed potential ethical and racial fairness issues in the design of NFT artworks due to a lack of ethical guidelines or censorship. Therefore, we investigated CryptoPunks, the most famous collection in the NFT market, to explore and visualize its potential ethical issues. We explored the ethical issues from three aspects: design, trading transactions, and related topics on Twitter. We scraped data from Twitter and Dune Analytics using python libraries, Twitter crawler, and sentiment analysis tools. Our five visualizations implied that 1.6 times more male punks were created in the initial design process than the female ones. And the male ones have a higher average selling price than females; lighter-skinned punks tend to sell for higher prices. The results of our study and visualizations provide a preliminary exploration of CryptoPunks and further inspire future ethical-related investigation and research in the NFT domain.
Object Detection and Tracking with Autonomous UAV
Venue: arXiv cs.RO 2022
Year: 2022

In this paper, a combat Unmanned Air Vehicle (UAV) is modeled in the simulation environment. The rotary wing UAV is successfully performed various tasks such as locking on the targets, tracking, and sharing the relevant data with surrounding vehicles. Different software technologies such as API communication, ground control station configuration, autonomous movement algorithms, computer vision, and deep learning are employed.
Explaining the root causes of unit-level changes
Venue: arXiv cs.LG 2022
Year: 2022

Existing methods of explainable AI and interpretable ML cannot explain change in the values of an output variable for a statistical unit in terms of the change in the input values and the change in the "mechanism" (the function transforming input to output). We propose two methods based on counterfactuals for explaining unit-level changes at various input granularities using the concept of Shapley values from game theory. These methods satisfy two key axioms desirable for any unit-level change attribution method. Through simulations, we study the reliability and the scalability of the proposed methods. We get sensible results from a case study on identifying the drivers of the change in the earnings for individuals in the US.
Locked and unlocked smooth embeddings of surfaces
Venue: arXiv cs.CG 2022
Year: 2022

We study the continuous motion of smooth isometric embeddings of a planar surface in three-dimensional Euclidean space, and two related discrete analogues of these embeddings, polygonal embeddings and flat foldings without interior vertices, under continuous changes of the embedding or folding. We show that every star-shaped or spiral-shaped domain is unlocked: a continuous motion unfolds it to a flat embedding. However, disks with two holes can have locked embeddings that are topologically equivalent to a flat embedding but cannot reach a flat embedding by continuous motion.
Normalized/Clipped SGD with Perturbation for Differentially Private Non-Convex Optimization
Venue: arXiv cs.LG 2022
Year: 2022

By ensuring differential privacy in the learning algorithms, one can rigorously mitigate the risk of large models memorizing sensitive training data. In this paper, we study two algorithms for this purpose, i.e., DP-SGD and DP-NSGD, which first clip or normalize \textit{per-sample} gradients to bound the sensitivity and then add noise to obfuscate the exact information. We analyze the convergence behavior of these two algorithms in the non-convex optimization setting with two common assumptions and achieve a rate $\mathcal{O}\left(\sqrt[4]{\frac{d\log(1/δ)}{N^2ε^2}}\right)$ of the gradient norm for a $d$-dimensional model, $N$ samples and $(ε,δ)$-DP, which improves over previous bounds under much weaker assumptions. Specifically, we introduce a regularizing factor in DP-NSGD and show that it is crucial in the convergence proof and subtly controls the bias and noise trade-off. Our proof deliberately handles the per-sample gradient clipping and normalization that are specified for the private setting. Empirically, we demonstrate that these two algorithms achieve similar best accuracy while DP-NSGD is comparatively easier to tune than DP-SGD and hence may help further save the privacy budget when accounting the tuning effort.
Monitoring Shortcut Learning using Mutual Information
Venue: arXiv cs.LG 2022
Year: 2022

The failure of deep neural networks to generalize to out-of-distribution data is a well-known problem and raises concerns about the deployment of trained networks in safety-critical domains such as healthcare, finance and autonomous vehicles. We study a particular kind of distribution shift $\unicode{x2013}$ shortcuts or spurious correlations in the training data. Shortcut learning is often only exposed when models are evaluated on real-world data that does not contain the same spurious correlations, posing a serious dilemma for AI practitioners to properly assess the effectiveness of a trained model for real-world applications. In this work, we propose to use the mutual information (MI) between the learned representation and the input as a metric to find where in training, the network latches onto shortcuts. Experiments demonstrate that MI can be used as a domain-agnostic metric for monitoring shortcut learning.
DPOAD: Differentially Private Outsourcing of Anomaly Detection through Iterative Sensitivity Learning
Venue: arXiv cs.CR 2022
Year: 2022

Outsourcing anomaly detection to third-parties can allow data owners to overcome resource constraints (e.g., in lightweight IoT devices), facilitate collaborative analysis (e.g., under distributed or multi-party scenarios), and benefit from lower costs and specialized expertise (e.g., of Managed Security Service Providers). Despite such benefits, a data owner may feel reluctant to outsource anomaly detection without sufficient privacy protection. To that end, most existing privacy solutions would face a novel challenge, i.e., preserving privacy usually requires the difference between data entries to be eliminated or reduced, whereas anomaly detection critically depends on that difference. Such a conflict is recently resolved under a local analysis setting with trusted analysts (where no outsourcing is involved) through moving the focus of differential privacy (DP) guarantee from "all" to only "benign" entries. In this paper, we observe that such an approach is not directly applicable to the outsourcing setting, because data owners do not know which entries are "benign" prior to outsourcing, and hence cannot selectively apply DP on data entries. Therefore, we propose a novel iterative solution for the data owner to gradually "disentangle" the anomalous entries from the benign ones such that the third-party analyst can produce accurate anomaly results with sufficient DP guarantee. We design and implement our Differentially Private Outsourcing of Anomaly Detection (DPOAD) framework, and demonstrate its benefits over baseline Laplace and PainFree mechanisms through experiments with real data from different application domains.
A Coupled Neural Circuit Design for Guillain-Barre Syndrome
Venue: arXiv cs.NE 2022
Year: 2022

Guillain-Barre syndrome is a rare neurological condition in which the human immune system attacks the peripheral nervous system. A peripheral nervous system appears as a diffusively connected system of mathematical models of neuron models, and the system's period becomes shorter than the periods of each neural circuit. The stimuli in the conduction path that will address the myelin sheath that has lost its function are received by the axons and are conveyed externally to the target organ, aiming to solve the problem of decreased nerve conduction. In the NEURON simulation environment, one can create a neuron model and define biophysical events that take place within the system for study. In this environment, signal transmission between cells and dendrites is obtained graphically. The simulated potassium and sodium conductance are replicated adequately, and the electronic action potentials are quite comparable to those measured experimentally. In this work, we propose an analog and digital coupled neuron model comprising individual excitatory and inhibitory neural circuit blocks for a low-cost and energy-efficient system. Compared to digital design, our analog design performs in lower frequency but gives a 32.3\% decreased energy efficiency. Thus, the resulting coupled analog hardware neuron model can be a proposed model for the simulation of reduced nerve conduction. As a result, the analog coupled neuron, (even with its greater design complexity) serious contender for the future development of a wearable sensor device that could help with Guillain-Barre syndrome and other neurologic diseases.
Personalized recommendation system based on social relationships and historical behaviors
Venue: arXiv cs.SI 2022
Year: 2022

Previous studies show that recommendation algorithms based on historical behaviors of users can provide satisfactory recommendation performance. Many of these algorithms pay attention to the interest of users, while ignore the influence of social relationships on user behaviors. Social relationships not only carry intrinsic information of similar consumption tastes or behaviors, but also imply the influence of individual to its neighbors. In this paper, we assume that social relationships and historical behaviors of users are related to the same factors. Based on this assumption, we propose an algorithm to focus on social relationships useful for recommendation systems through mutual constraints from both types of information. We test the performance of our algorithm on four types of users, including all users, active users, inactive users and cold-start users. Results show that the proposed algorithm outperforms benchmarks in four types of scenarios subject to recommendation accuracy and diversity metrics. We further design a randomization model to explore the contribution of social relationships to recommendation performance, and the result shows that the contribution of social relationships in the proposed algorithm depends on the coupling strength of social relationships and historical behaviors.
SearchMorph:Multi-scale Correlation Iterative Network for Deformable Registration
Venue: arXiv cs.CV 2022
Year: 2022

Deformable image registration can obtain dynamic information about images, which is of great significance in medical image analysis. The unsupervised deep learning registration method can quickly achieve high registration accuracy without labels. However, these methods generally suffer from uncorrelated features, poor ability to register large deformations and details, and unnatural deformation fields. To address the issues above, we propose an unsupervised multi-scale correlation iterative registration network (SearchMorph). In the proposed network, we introduce a correlation layer to strengthen the relevance between features and construct a correlation pyramid to provide multi-scale relevance information for the network. We also design a deformation field iterator, which improves the ability of the model to register details and large deformations through the search module and GRU while ensuring that the deformation field is realistic. We use single-temporal brain MR images and multi-temporal echocardiographic sequences to evaluate the model's ability to register large deformations and details. The experimental results demonstrate that the method in this paper achieves the highest registration accuracy and the lowest folding point ratio using a short elapsed time to state-of-the-art.
Dynamic Bank Learning for Semi-supervised Federated Image Diagnosis with Class Imbalance
Venue: arXiv cs.CV 2022
Year: 2022

Despite recent progress on semi-supervised federated learning (FL) for medical image diagnosis, the problem of imbalanced class distributions among unlabeled clients is still unsolved for real-world use. In this paper, we study a practical yet challenging problem of class imbalanced semi-supervised FL (imFed-Semi), which allows all clients to have only unlabeled data while the server just has a small amount of labeled data. This imFed-Semi problem is addressed by a novel dynamic bank learning scheme, which improves client training by exploiting class proportion information. This scheme consists of two parts, i.e., the dynamic bank construction to distill various class proportions for each local client, and the sub-bank classification to impose the local model to learn different class proportions. We evaluate our approach on two public real-world medical datasets, including the intracranial hemorrhage diagnosis with 25,000 CT slices and skin lesion diagnosis with 10,015 dermoscopy images. The effectiveness of our method has been validated with significant performance improvements (7.61% and 4.69%) compared with the second-best on the accuracy, as well as comprehensive analytical studies. Code is available at https://github.com/med-air/imFedSemi.
Adversarial Example Detection in Deployed Tree Ensembles
Venue: arXiv cs.LG 2022
Year: 2022

Tree ensembles are powerful models that are widely used. However, they are susceptible to adversarial examples, which are examples that purposely constructed to elicit a misprediction from the model. This can degrade performance and erode a user's trust in the model. Typically, approaches try to alleviate this problem by verifying how robust a learned ensemble is or robustifying the learning process. We take an alternative approach and attempt to detect adversarial examples in a post-deployment setting. We present a novel method for this task that works by analyzing an unseen example's output configuration, which is the set of predictions made by an ensemble's constituent trees. Our approach works with any additive tree ensemble and does not require training a separate model. We evaluate our approach on three different tree ensemble learners. We empirically show that our method is currently the best adversarial detection method for tree ensembles.