OpenRaiser/Intern-Atlas huggingface dataset:
Improving Adversarial Robustness to Sensitivity and Invariance Attacks with Deep Metric Learning
Venue: arXiv cs.LG 2022
Year: 2022

Intentionally crafted adversarial samples have effectively exploited weaknesses in deep neural networks. A standard method in adversarial robustness assumes a framework to defend against samples crafted by minimally perturbing a sample such that its corresponding model output changes. These sensitivity attacks exploit the model's sensitivity toward task-irrelevant features. Another form of adversarial sample can be crafted via invariance attacks, which exploit the model underestimating the importance of relevant features. Previous literature has indicated a tradeoff in defending against both attack types within a strictly L_p bounded defense. To promote robustness toward both types of attacks beyond Euclidean distance metrics, we use metric learning to frame adversarial regularization as an optimal transport problem. Our preliminary results indicate that regularizing over invariant perturbations in our framework improves both invariant and sensitivity defense.
Reservoir Computing via Quantum Recurrent Neural Networks
Venue: arXiv cs.NE 2022
Year: 2022

Recent developments in quantum computing and machine learning have propelled the interdisciplinary study of quantum machine learning. Sequential modeling is an important task with high scientific and commercial value. Existing VQC or QNN-based methods require significant computational resources to perform the gradient-based optimization of a larger number of quantum circuit parameters. The major drawback is that such quantum gradient calculation requires a large amount of circuit evaluation, posing challenges in current near-term quantum hardware and simulation software. In this work, we approach sequential modeling by applying a reservoir computing (RC) framework to quantum recurrent neural networks (QRNN-RC) that are based on classical RNN, LSTM and GRU. The main idea to this RC approach is that the QRNN with randomly initialized weights is treated as a dynamical system and only the final classical linear layer is trained. Our numerical simulations show that the QRNN-RC can reach results comparable to fully trained QRNN models for several function approximation and time series prediction tasks. Since the QRNN training complexity is significantly reduced, the proposed model trains notably faster. In this work we also compare to corresponding classical RNN-based RC implementations and show that the quantum version learns faster by requiring fewer training epochs in most cases. Our results demonstrate a new possibility to utilize quantum neural network for sequential modeling with greater quantum hardware efficiency, an important design consideration for noisy intermediate-scale quantum (NISQ) computers.
Resource-Efficient Transfer Learning From Speech Foundation Model Using Hierarchical Feature Fusion
Venue: arXiv cs.LG 2022
Year: 2022

Self-supervised pre-training of a speech foundation model, followed by supervised fine-tuning, has shown impressive quality improvements on automatic speech recognition (ASR) tasks. Fine-tuning separate foundation models for many downstream tasks are expensive since the foundation model is usually very big. Parameter-efficient fine-tuning methods (e.g. adapter, sparse update methods) offer an alternative paradigm where a small set of parameters are updated to adapt the foundation model to new tasks. However, these methods still suffer from a high computational memory cost and slow training speed because they require backpropagation through the entire neural network at each step. In the paper, we analyze the performance of features at different layers of a foundation model on the speech recognition task and propose a novel hierarchical feature fusion method for resource-efficient transfer learning from speech foundation models. Experimental results show that the proposed method can achieve better performance on speech recognition task than existing algorithms with fewer number of trainable parameters, less computational memory cost and faster training speed. After combining with Adapters at all layers, the proposed method can achieve the same performance as fine-tuning the whole model with $97\%$ fewer trainable encoder parameters and $53\%$ faster training speed.
On The Modelling and Performance Analysis of Lower Layer Mobility in 5G-Advanced
Venue: arXiv cs.NI 2022
Year: 2022

One of upcoming mobility enhancements in 5G-Advanced networks is to execute handover based on Layer 1 (L1) measurements using the so called lower layer mobility procedure. In this paper, we provide a system model for lower layer mobility procedure and we evaluate it against existing higher layer mobility procedures, such as baseline and conditional handover, using system level simulations. The benefits and drawbacks of lower layer mobility procedure are analyzed and compared against higher layer handover mechanisms using the relevant mobility key performance indicators. It has been shown that lower layer mobility procedure outperforms the existing handover mechanisms with respect to radio communication reliability at the expense of higher number of handovers and ping-pongs. To tackle these drawbacks, additional filtering for the L1 measurements used in handover decision is introduced to reduce the fluctuations caused by fast fading and measurement errors. Moreover, lower layer mobility procedure is enhanced with dynamic switching mechanism enabling the UE to change cells without being reconfigured by the network. The evaluations have shown that the introduction of such techniques is beneficial in reducing the number of ping-pongs and signaling overhead at the expense of an increase in the delay to react to rapid signal degradation and resource reservation overhead, respectively.
Bridging HPC Communities through the Julia Programming Language
Venue: arXiv cs.DC 2022
Year: 2022

The Julia programming language has evolved into a modern alternative to fill existing gaps in scientific computing and data science applications. Julia leverages a unified and coordinated single-language and ecosystem paradigm and has a proven track record of achieving high performance without sacrificing user productivity. These aspects make Julia a viable alternative to high-performance computing's (HPC's) existing and increasingly costly many-body workflow composition strategy in which traditional HPC languages (e.g., Fortran, C, C++) are used for simulations, and higher-level languages (e.g., Python, R, MATLAB) are used for data analysis and interactive computing. Julia's rapid growth in language capabilities, package ecosystem, and community make it a promising universal language for HPC. This paper presents the views of a multidisciplinary group of researchers from academia, government, and industry that advocate for an HPC software development paradigm that emphasizes developer productivity, workflow portability, and low barriers for entry. We believe that the Julia programming language, its ecosystem, and its community provide modern and powerful capabilities that enable this group's objectives. Crucially, we believe that Julia can provide a feasible and less costly approach to programming scientific applications and workflows that target HPC facilities. In this work, we examine the current practice and role of Julia as a common, end-to-end programming model to address major challenges in scientific reproducibility, data-driven AI/machine learning, co-design and workflows, scalability and performance portability in heterogeneous computing, network communication, data management, and community education. As a result, the diversification of current investments to fulfill the needs of the upcoming decade is crucial as more supercomputing centers prepare for the exascale era.
Extension of Simple Algorithms to the Matroid Secretary Problem
Venue: arXiv cs.DS 2022
Year: 2022

Whereas there are simple algorithms that are proven to be optimal for the Classical and the Multiple Choice Secretary Problem, the Matroid Secretary Problem is less thoroughly understood. This paper proposes the generalization of some simple algorithms from the Classical and Multiple Choice versions on the Matroid Secretary Problem. Out of two algorithms that make decisions based on samples, like the Dynkin's algorithm, one is proven to be an instance of Greedy Algorithm (Bahrani et al., 2022), while the other is not. A generalized version of the Virtual Algorithm (Babaioff et al., 2018) obtains a constant competitive ratio for the Hat Graph, the adversarial example for Greedy Algorithms, but fails to do so when a slight modificiation is introduced to the graph. We show that there is no algorithm with Strong Forbidden Sets (Soto et al., 2021) of size 1 on all graphic matroids.
Quantitative Assessment of Drought Impacts Using XGBoost based on the Drought Impact Reporter
Venue: arXiv cs.LG 2022
Year: 2022

Under climate change, the increasing frequency, intensity, and spatial extent of drought events lead to higher socio-economic costs. However, the relationships between the hydro-meteorological indicators and drought impacts are not identified well yet because of the complexity and data scarcity. In this paper, we proposed a framework based on the extreme gradient model (XGBoost) for Texas to predict multi-category drought impacts and connected a typical drought indicator, Standardized Precipitation Index (SPI), to the text-based impacts from the Drought Impact Reporter (DIR). The preliminary results of this study showed an outstanding performance of the well-trained models to assess drought impacts on agriculture, fire, society & public health, plants & wildlife, as well as relief, response & restrictions in Texas. It also provided a possibility to appraise drought impacts using hydro-meteorological indicators with the proposed framework in the United States, which could help drought risk management by giving additional information and improving the updating frequency of drought impacts. Our interpretation results using the Shapley additive explanation (SHAP) interpretability technique revealed that the rules guiding the predictions of XGBoost comply with domain expertise knowledge around the role that SPI indicators play around drought impacts.
Query Processing at Snapchat: How we Handle Query Completion, Suggestion and Localization
Venue: arXiv cs.IR 2022
Year: 2022

From the Publisher:Software is a commodity being sold across diverse language and cultural groups, whether in the commercial marketplace, or as customized applications. Developers must structure their applications so that they can be readily and cheaply localized for sale in this range of markets. Obvious differences such as scripts and languages must be understood as well as a range of more subtle cultural conventions. Further topics covered include: the overall architecture for internationalized products and an outline of an internationalization API; the use of computational linguistics methods; quality assurance, testing and documentation. Appendices contain summaries of the facilities available for localization on major platforms, characteristics of European languages, commercial tools and further reading. The book is aimed at small and medium sized software producers, and the IT departments of multinational corporations.
Modified EDAS Method Based on Cumulative Prospect Theory for Multiple Attributes Group Decision Making with Interval-valued Intuitionistic Fuzzy Information
Venue: arXiv cs.AI 2022
Year: 2022

The Interval-valued intuitionistic fuzzy sets (IVIFSs) based on the intuitionistic fuzzy sets combines the classical decision method is in its research and application is attracting attention. After comparative analysis, there are multiple classical methods with IVIFSs information have been applied into many practical issues. In this paper, we extended the classical EDAS method based on cumulative prospect theory (CPT) considering the decision makers (DMs) psychological factor under IVIFSs. Taking the fuzzy and uncertain character of the IVIFSs and the psychological preference into consideration, the original EDAS method based on the CPT under IVIFSs (IVIF-CPT-MABAC) method is built for MAGDM issues. Meanwhile, information entropy method is used to evaluate the attribute weight. Finally, a numerical example for project selection of green technology venture capital has been given and some comparisons is used to illustrate advantages of IVIF-CPT-MABAC method and some comparison analysis and sensitivity analysis are applied to prove this new methods effectiveness and stability.
A Robust and Low Complexity Deep Learning Model for Remote Sensing Image Classification
Venue: arXiv cs.CV 2022
Year: 2022

In this paper, we present a robust and low complexity deep learning model for Remote Sensing Image Classification (RSIC), the task of identifying the scene of a remote sensing image. In particular, we firstly evaluate different low complexity and benchmark deep neural networks: MobileNetV1, MobileNetV2, NASNetMobile, and EfficientNetB0, which present the number of trainable parameters lower than 5 Million (M). After indicating best network architecture, we further improve the network performance by applying attention schemes to multiple feature maps extracted from middle layers of the network. To deal with the issue of increasing the model footprint as using attention schemes, we apply the quantization technique to satisfy the maximum of 20 MB memory occupation. By conducting extensive experiments on the benchmark datasets NWPU-RESISC45, we achieve a robust and low-complexity model, which is very competitive to the state-of-the-art systems and potential for real-life applications on edge devices.
Degradation Prediction of Semiconductor Lasers using Conditional Variational Autoencoder
Venue: arXiv cs.LG 2022
Year: 2022

Semiconductor lasers have been rapidly evolving to meet the demands of next-generation optical networks. This imposes much more stringent requirements on the laser reliability, which are dominated by degradation mechanisms (e.g., sudden degradation) limiting the semiconductor laser lifetime. Physics-based approaches are often used to characterize the degradation behavior analytically, yet explicit domain knowledge and accurate mathematical models are required. Building such models can be very challenging due to a lack of a full understanding of the complex physical processes inducing the degradation under various operating conditions. To overcome the aforementioned limitations, we propose a new data-driven approach, extracting useful insights from the operational monitored data to predict the degradation trend without requiring any specific knowledge or using any physical model. The proposed approach is based on an unsupervised technique, a conditional variational autoencoder, and validated using vertical-cavity surface-emitting laser (VCSEL) and tunable edge emitting laser reliability data. The experimental results confirm that our model (i) achieves a good degradation prediction and generalization performance by yielding an F1 score of 95.3%, (ii) outperforms several baseline ML based anomaly detection techniques, and (iii) helps to shorten the aging tests by early predicting the failed devices before the end of the test and thereby saving costs
Density of triangulated ternary disc packings
Venue: arXiv cs.CG 2022
Year: 2022

We consider ternary disc packings of the plane, i.e. the packings using discs of three different radii. Packings in which each ''hole'' is bounded by three pairwise tangent discs are called triangulated. There are 164 pairs $(r,s)$, $1{>}r{>}s$, allowing triangulated packings by discs of radii 1, $r$ and $s$. In this paper, we enhance existing methods of dealing with maximal-density packings in order to find ternary triangulated packings which maximize the density among all the packings with the same disc radii. We showed for 16 pairs that the density is maximized by a triangulated ternary packing; for 15 other pairs, we proved the density to be maximized by a triangulated packing using only two sizes of discs; for 40 pairs, we found non-triangulated packings strictly denser than any triangulated one; finally, we classified the remaining cases where our methods are not applicable.
ON-DEMAND-FL: A Dynamic and Efficient Multi-Criteria Federated Learning Client Deployment Scheme
Venue: arXiv cs.AI 2022
Year: 2022

In this paper, we increase the availability and integration of devices in the learning process to enhance the convergence of federated learning (FL) models. To address the issue of having all the data in one location, federated learning, which maintains the ability to learn over decentralized data sets, combines privacy and technology. Until the model converges, the server combines the updated weights obtained from each dataset over a number of rounds. The majority of the literature suggested client selection techniques to accelerate convergence and boost accuracy. However, none of the existing proposals have focused on the flexibility to deploy and select clients as needed, wherever and whenever that may be. Due to the extremely dynamic surroundings, some devices are actually not available to serve as clients in FL, which affects the availability of data for learning and the applicability of the existing solution for client selection. In this paper, we address the aforementioned limitations by introducing an On-Demand-FL, a client deployment approach for FL, offering more volume and heterogeneity of data in the learning process. We make use of the containerization technology such as Docker to build efficient environments using IoT and mobile devices serving as volunteers. Furthermore, Kubernetes is used for orchestration. The Genetic algorithm (GA) is used to solve the multi-objective optimization problem due to its evolutionary strategy. The performed experiments using the Mobile Data Challenge (MDC) dataset and the Localfed framework illustrate the relevance of the proposed approach and the efficiency of the on-the-fly deployment of clients whenever and wherever needed with less discarded rounds and more available data.
Predicting Treatment Adherence of Tuberculosis Patients at Scale
Venue: arXiv cs.LG 2022
Year: 2022

Tuberculosis (TB), an infectious bacterial disease, is a significant cause of death, especially in low-income countries, with an estimated ten million new cases reported globally in $2020$. While TB is treatable, non-adherence to the medication regimen is a significant cause of morbidity and mortality. Thus, proactively identifying patients at risk of dropping off their medication regimen enables corrective measures to mitigate adverse outcomes. Using a proxy measure of extreme non-adherence and a dataset of nearly $700,000$ patients from four states in India, we formulate and solve the machine learning (ML) problem of early prediction of non-adherence based on a custom rank-based metric. We train ML models and evaluate against baselines, achieving a $\sim 100\%$ lift over rule-based baselines and $\sim 214\%$ over a random classifier, taking into account country-wide large-scale future deployment. We deal with various issues in the process, including data quality, high-cardinality categorical data, low target prevalence, distribution shift, variation across cohorts, algorithmic fairness, and the need for robustness and explainability. Our findings indicate that risk stratification of non-adherent patients is a viable, deployable-at-scale ML solution. As the official AI partner of India's Central TB Division, we are working on multiple city and state-level pilots with the goal of pan-India deployment.
Privacy-Preserving Models for Legal Natural Language Processing
Venue: arXiv cs.CL 2022
Year: 2022

Pre-training large transformer models with in-domain data improves domain adaptation and helps gain performance on the domain-specific downstream tasks. However, sharing models pre-trained on potentially sensitive data is prone to adversarial privacy attacks. In this paper, we asked to which extent we can guarantee privacy of pre-training data and, at the same time, achieve better downstream performance on legal tasks without the need of additional labeled data. We extensively experiment with scalable self-supervised learning of transformer models under the formal paradigm of differential privacy and show that under specific training configurations we can improve downstream performance without sacrifying privacy protection for the in-domain data. Our main contribution is utilizing differential privacy for large-scale pre-training of transformer language models in the legal NLP domain, which, to the best of our knowledge, has not been addressed before.
Decision Making by a Neuromorphic Network of Volatile Resistive Switching Memories
Venue: arXiv cs.ET 2022
Year: 2022

The necessity of having an electronic device working in relevant biological time scales with a small footprint boosted the research of a new class of emerging memories. Ag-based volatile resistive switching memories (RRAMs) feature a spontaneous change of device conductance with a similarity to biological mechanisms. They rely on the formation and self-disruption of a metallic conductive filament through an oxide layer, with a retention time ranging from a few milliseconds to several seconds, greatly tunable according to the maximum current which is flowing through the device. Here we prove a neuromorphic system based on volatile-RRAMs able to mimic the principles of biological decision-making behavior and tackle the Two-Alternative Forced Choice problem, where a subject is asked to make a choice between two possible alternatives not relying on a precise knowledge of the problem, rather on noisy perceptions.
LG-Hand: Advancing 3D Hand Pose Estimation with Locally and Globally Kinematic Knowledge
Venue: arXiv cs.CV 2022
Year: 2022

3D hand pose estimation from RGB images suffers from the difficulty of obtaining the depth information. Therefore, a great deal of attention has been spent on estimating 3D hand pose from 2D hand joints. In this paper, we leverage the advantage of spatial-temporal Graph Convolutional Neural Networks and propose LG-Hand, a powerful method for 3D hand pose estimation. Our method incorporates both spatial and temporal dependencies into a single process. We argue that kinematic information plays an important role, contributing to the performance of 3D hand pose estimation. We thereby introduce two new objective functions, Angle and Direction loss, to take the hand structure into account. While Angle loss covers locally kinematic information, Direction loss handles globally kinematic one. Our LG-Hand achieves promising results on the First-Person Hand Action Benchmark (FPHAB) dataset. We also perform an ablation study to show the efficacy of the two proposed objective functions.