OpenRaiser/Intern-Atlas huggingface dataset:
L2C2: Last-Level Compressed-Cache NVM and a Procedure to Forecast Performance and Lifetime
Venue: arXiv cs.AR 2022
Year: 2022

Several emerging non-volatile (NV) memory technologies are rising as interesting alternatives to build the Last-Level Cache (LLC). Their advantages, compared to SRAM memory, are higher density and lower static power, but write operations wear out the bitcells to the point of eventually losing their storage capacity. In this context, this paper presents a novel LLC organization designed to extend the lifetime of the NV data array and a procedure to forecast in detail the capacity and performance of such an NV-LLC over its lifetime. From a methodological point of view, although different approaches are used in the literature to analyze the degradation of an NV-LLC, none of them allows to study in detail its temporal evolution. In this sense, this work proposes a forecast procedure that combines detailed simulation and prediction, allowing an accurate analysis of the impact of different cache control policies and mechanisms (replacement, wear-leveling, compression, etc.) on the temporal evolution of the indices of interest, such as the effective capacity of the NV-LLC or the system IPC. We also introduce L2C2, a LLC design intended for implementation in NV memory technology that combines fault tolerance, compression, and internal write wear leveling for the first time. Compression is not used to store more blocks and increase the hit rate, but to reduce the write rate and increase the lifetime during which the cache supports near-peak performance. It has affordable hardware overheads compared to that of a baseline NV-LLC without compression in terms of area, latency and energy consumption, and increases up to 6-37 times the time in which 50\% of the effective capacity is degraded, depending on the variability in the manufacturing process.
BSAL: A Framework of Bi-component Structure and Attribute Learning for Link Prediction
Venue: arXiv cs.LG 2022
Year: 2022

Given the ubiquitous existence of graph-structured data, learning the representations of nodes for the downstream tasks ranging from node classification, link prediction to graph classification is of crucial importance. Regarding missing link inference of diverse networks, we revisit the link prediction techniques and identify the importance of both the structural and attribute information. However, the available techniques either heavily count on the network topology which is spurious in practice or cannot integrate graph topology and features properly. To bridge the gap, we propose a bicomponent structural and attribute learning framework (BSAL) that is designed to adaptively leverage information from topology and feature spaces. Specifically, BSAL constructs a semantic topology via the node attributes and then gets the embeddings regarding the semantic view, which provides a flexible and easy-to-implement solution to adaptively incorporate the information carried by the node attributes. Then the semantic embedding together with topology embedding is fused together using an attention mechanism for the final prediction. Extensive experiments show the superior performance of our proposal and it significantly outperforms baselines on diverse research benchmarks.
Evaluating Commit Message Generation: To BLEU Or Not To BLEU?
Venue: arXiv cs.SE 2022
Year: 2022

Commit messages play an important role in several software engineering tasks such as program comprehension and understanding program evolution. However, programmers neglect to write good commit messages. Hence, several Commit Message Generation (CMG) tools have been proposed. We observe that the recent state of the art CMG tools use simple and easy to compute automated evaluation metrics such as BLEU4 or its variants. The advances in the field of Machine Translation (MT) indicate several weaknesses of BLEU4 and its variants. They also propose several other metrics for evaluating Natural Language Generation (NLG) tools. In this work, we discuss the suitability of various MT metrics for the CMG task. Based on the insights from our experiments, we propose a new variant specifically for evaluating the CMG task. We re-evaluate the state of the art CMG tools on our new metric. We believe that our work fixes an important gap that exists in the understanding of evaluation metrics for CMG research.
A Comprehensive Study of Accelerating IPv6 Deployment
Venue: arXiv cs.NI 2022
Year: 2022

Since the lack of IPv6 network development, China is currently accelerating IPv6 deployment. In this scenario, traffic and network structure show a huge shift. However, due to the long-term prosperity, we are ignorant of the problems behind such outbreak of traffic and performance improvement events in accelerating deployment. IPv6 development in some regions will still face similar challenges in the future. To contribute to solving this problem, in this paper, we produce a new measurement framework and implement a 5-month passive measurement on the IPv6 network during the accelerating deployment in China. We combine 6 global-scale datasets to form the normal status of IPv6 network, which is against to the accelerating status formed by the passive traffic. Moreover, we compare with the traffic during World IPv6 Day 2011 and Launch 2012 to discuss the common nature of accelerating deployment. Finally, the results indicate that the IPv6 accelerating deployment is often accompanied by an unbalanced network status. It exposes unresolved security issues including the challenge of user privacy and inappropriate access methods. According to the investigation, we point the future IPv6 development after accelerating deployment.
Extracting Impact Model Narratives from Social Services' Text
Venue: arXiv cs.CL 2022
Year: 2022

Named entity recognition (NER) is an important task in narration extraction. Narration, as a system of stories, provides insights into how events and characters in the stories develop over time. This paper proposes an architecture for NER on a corpus about social purpose organizations. This is the first NER task specifically targeted at social service entities. We show how this approach can be used for the sequencing of services and impacted clients with information extracted from unstructured text. The methodology outlines steps for extracting ontological representation of entities such as needs and satisfiers and generating hypotheses to answer queries about impact models defined by social purpose organizations. We evaluate the model on a corpus of social service descriptions with empirically calculated score.
Predictive Accuracy of a Hybrid Generalized Long Memory Model for Short Term Electricity Price Forecasting
Venue: arXiv cs.LG 2022
Year: 2022

Accurate electricity price forecasting is the main management goal for market participants since it represents the fundamental basis to maximize the profits for market players. However, electricity is a non-storable commodity and the electricity prices are affected by some social and natural factors that make the price forecasting a challenging task. This study investigates the predictive performance of a new hybrid model based on the Generalized long memory autoregressive model (k-factor GARMA), the Gegenbauer Generalized Autoregressive Conditional Heteroscedasticity(G-GARCH) process, Wavelet decomposition, and Local Linear Wavelet Neural Network (LLWNN) optimized using two different learning algorithms; the Backpropagation algorithm (BP) and the Particle Swarm optimization algorithm (PSO). The performance of the proposed model is evaluated using data from Nord Pool Electricity markets. Moreover, it is compared with some other parametric and non-parametric models in order to prove its robustness. The empirical results prove that the proposed method performs well than other competing techniques.
A Survey and Perspective on Artificial Intelligence for Security-Aware Electronic Design Automation
Venue: arXiv cs.LG 2022
Year: 2022

Artificial intelligence (AI) and machine learning (ML) techniques have been increasingly used in several fields to improve performance and the level of automation. In recent years, this use has exponentially increased due to the advancement of high-performance computing and the ever increasing size of data. One of such fields is that of hardware design; specifically the design of digital and analog integrated circuits~(ICs), where AI/ ML techniques have been extensively used to address ever-increasing design complexity, aggressive time-to-market, and the growing number of ubiquitous interconnected devices (IoT). However, the security concerns and issues related to IC design have been highly overlooked. In this paper, we summarize the state-of-the-art in AL/ML for circuit design/optimization, security and engineering challenges, research in security-aware CAD/EDA, and future research directions and needs for using AI/ML for security-aware circuit design.
Build a Robust QA System with Transformer-based Mixture of Experts
Venue: arXiv cs.CL 2022
Year: 2022

In this paper, we aim to build a robust question answering system that can adapt to out-of-domain datasets. A single network may overfit to the superficial correlation in the training distribution, but with a meaningful number of expert sub-networks, a gating network that selects a sparse combination of experts for each input, and careful balance on the importance of expert sub-networks, the Mixture-of-Experts (MoE) model allows us to train a multi-task learner that can be generalized to out-of-domain datasets. We also explore the possibility of bringing the MoE layers up to the middle of the DistilBERT and replacing the dense feed-forward network with a sparsely-activated switch FFN layers, similar to the Switch Transformer architecture, which simplifies the MoE routing algorithm with reduced communication and computational costs. In addition to model architectures, we explore techniques of data augmentation including Easy Data Augmentation (EDA) and back translation, to create more meaningful variance among the small out-of-domain training data, therefore boosting the performance and robustness of our models. In this paper, we show that our combination of best architecture and data augmentation techniques achieves a 53.477 F1 score in the out-of-domain evaluation, which is a 9.52% performance gain over the baseline. On the final test set, we reported a higher 59.506 F1 and 41.651 EM. We successfully demonstrate the effectiveness of Mixture-of-Expert architecture in a Robust QA task.
Radiology Text Analysis System (RadText): Architecture and Evaluation
Venue: arXiv cs.CL 2022
Year: 2022

Analyzing radiology reports is a time-consuming and error-prone task, which raises the need for an efficient automated radiology report analysis system to alleviate the workloads of radiologists and encourage precise diagnosis. In this work, we present RadText, an open-source radiology text analysis system developed by Python. RadText offers an easy-to-use text analysis pipeline, including de-identification, section segmentation, sentence split and word tokenization, named entity recognition, parsing, and negation detection. RadText features a flexible modular design, provides a hybrid text processing schema, and supports raw text processing and local processing, which enables better usability and improved data privacy. RadText adopts BioC as the unified interface, and also standardizes the input / output into a structured representation compatible with Observational Medical Outcomes Partnership (OMOP) Common Data Model (CDM). This allows for a more systematic approach to observational research across multiple, disparate data sources. We evaluated RadText on the MIMIC-CXR dataset, with five new disease labels we annotated for this work. RadText demonstrates highly accurate classification performances, with an average precision of, a recall of 0.94, and an F-1 score of 0.92. We have made our code, documentation, examples, and the test set available at https://github.com/bionlplab/radtext .
Factors Hindering the Adoption of DevOps in the Saudi Software Industry
Venue: arXiv cs.SE 2022
Year: 2022

DevOps has gained high importance in the global software industry due to the ease of software development, testing and deployment it provides. However, the Saudi software industry has not been able to adopt DevOps at a great pace due to various factors. This study, thus, aimed to examine different factors that hindered the adoption of DevOps in the Saudi software industry. Also, recommendations are provided at the end for Saudi Arabia to enhance the adoption of DevOps in its software industry. To accomplish the aims, this study used a literature review and interviews to gather data and examine it to produce the findings. The findings of the study highlight lack of support from organizational management and lack of laws as the major factors for the adoption of DevOps in the Saudi software industry.
LAMNER: Code Comment Generation Using Character Language Model and Named Entity Recognition
Venue: arXiv cs.CL 2022
Year: 2022

Code comment generation is the task of generating a high-level natural language description for a given code method or function. Although researchers have been studying multiple ways to generate code comments automatically, previous work mainly considers representing a code token in its entirety semantics form only (e.g., a language model is used to learn the semantics of a code token), and additional code properties such as the tree structure of a code are included as an auxiliary input to the model. There are two limitations: 1) Learning the code token in its entirety form may not be able to capture information succinctly in source code, and 2) The code token does not contain additional syntactic information, inherently important in programming languages. In this paper, we present LAnguage Model and Named Entity Recognition (LAMNER), a code comment generator capable of encoding code constructs effectively and capturing the structural property of a code token. A character-level language model is used to learn the semantic representation to encode a code token. For the structural property of a token, a Named Entity Recognition model is trained to learn the different types of code tokens. These representations are then fed into an encoder-decoder architecture to generate code comments. We evaluate the generated comments from LAMNER and other baselines on a popular Java dataset with four commonly used metrics. Our results show that LAMNER is effective and improves over the best baseline model in BLEU-1, BLEU-2, BLEU-3, BLEU-4, ROUGE-L, METEOR, and CIDEr by 14.34%, 18.98%, 21.55%, 23.00%, 10.52%, 1.44%, and 25.86%, respectively. Additionally, we fused LAMNER's code representation with the baseline models, and the fused models consistently showed improvement over the non-fused models. The human evaluation further shows that LAMNER produces high-quality code comments.
Optimize_Prime@DravidianLangTech-ACL2022: Abusive Comment Detection in Tamil
Venue: arXiv cs.CL 2022
Year: 2022

This paper tries to address the problem of abusive comment detection in low-resource indic languages. Abusive comments are statements that are offensive to a person or a group of people. These comments are targeted toward individuals belonging to specific ethnicities, genders, caste, race, sexuality, etc. Abusive Comment Detection is a significant problem, especially with the recent rise in social media users. This paper presents the approach used by our team - Optimize_Prime, in the ACL 2022 shared task "Abusive Comment Detection in Tamil." This task detects and classifies YouTube comments in Tamil and Tamil- English Codemixed format into multiple categories. We have used three methods to optimize our results: Ensemble models, Recurrent Neural Networks, and Transformers. In the Tamil data, MuRIL and XLM-RoBERTA were our best performing models with a macro-averaged f1 score of 0.43. Furthermore, for the Code-mixed data, MuRIL and M-BERT provided sub-lime results, with a macro-averaged f1 score of 0.45.
Coverage Control for a Multi-robot Team with Heterogeneous Capabilities using Block Coordinate Descent (BCD) Method
Venue: arXiv cs.RO 2022
Year: 2022

In this paper, we propose a coverage control system for a multi-robot team with heterogeneous capabilities to patrol or monitor a bounded environment. The capability could be defined as any criterion of robots like remaining power or mobile speed, depending on the purpose. The proposed control system aims to allocate different portions of the environment to the robots according to their capabilities, i.e., the robot with higher capability takes a larger portion of the environment while the robot with lower capability takes a smaller one. We use the block coordinate descent (BCD) method to optimize the location of portions and the partitioning method alternately. A centralized machine is used to synchronize the robots and the gradient of each robot can be computed in a distributed manner. Simulation results are provided to illustrate the performance of the proposed control system.
A majorization-minimization algorithm for nonnegative binary matrix factorization
Venue: arXiv cs.LG 2022
Year: 2022

This paper tackles the problem of decomposing binary data using matrix factorization. We consider the family of mean-parametrized Bernoulli models, a class of generative models that are well suited for modeling binary data and enables interpretability of the factors. We factorize the Bernoulli parameter and consider an additional Beta prior on one of the factors to further improve the model's expressive power. While similar models have been proposed in the literature, they only exploit the Beta prior as a proxy to ensure a valid Bernoulli parameter in a Bayesian setting; in practice it reduces to a uniform or uninformative prior. Besides, estimation in these models has focused on costly Bayesian inference. In this paper, we propose a simple yet very efficient majorization-minimization algorithm for maximum a posteriori estimation. Our approach leverages the Beta prior whose parameters can be tuned to improve performance in matrix completion tasks. Experiments conducted on three public binary datasets show that our approach offers an excellent trade-off between prediction performance, computational complexity, and interpretability.
Inferring ice sheet damage models from limited observations using CRIKit: the Constitutive Relation Inference Toolkit
Venue: arXiv cs.CE 2022
Year: 2022

We examine the prospect of learning ice sheet damage models from observational data. Our approach, implemented in CRIKit (the Constitutive Relation Inference Toolkit), is to model the material time derivative of damage as a frame-invariant neural network, and to optimize the parameters of the model from simulations of the flow of an ice dome. Using the model of Albrecht and Levermann as the ground truth to generate synthetic observations, we measure the difference of optimized neural network models from that model to try to understand how well this process generates models that can then transfer to other ice sheet simulations. The use of so-called "deep-learning" models for constitutive equations, equations of state, sub-grid-scale processes, and other pointwise relations that appear in systems of PDEs has been successful in other disciplines, yet our inference setting has some confounding factors. The first is the type of observations that are available: we compare the quality of the inferred models when the loss of the numerical simulations includes observation misfits throughout the ice, which is unobtainable in real settings, to losses that include only combinations of surface and borehole observations. The second confounding factor is the evolution of damage in an ice sheet, which is advection dominated. The non-local effect of perturbations in a damage models results in loss functions that have both many local minima and many parameter configurations for which the system is unsolvable. Our experience suggests that basic neural networks have several deficiencies that affect the quality of the optimized models. We suggest several approaches to incorporating additional inductive biases into neural networks which may lead to better performance in future work.
Path Planning Algorithms for Robotic Aquaculture Monitoring
Venue: arXiv cs.RO 2022
Year: 2022

Aerial drones have great potential to monitor large areas quickly and efficiently. Aquaculture is an industry that requires continuous water quality data to successfully grow and harvest fish. The Hybrid Aerial Underwater Robotic System (HAUCS) is designed to collect water quality data of aquaculture ponds to reduce labor costs for farmers. The routing of drones to cover each fish pond on an aquaculture farm can be reduced to the Vehicle Routing Problem. A dataset is created to simulate the distribution of ponds on a farm and is used to assess the HAUCS Path Planning Algorithm (HPP). Its performance is compared with the Google Linear Optimization Package (GLOP) and a Graph Attention Model (AM) for routing problems. GLOP is the most efficient solver for 50 to 200 ponds at the expense of long run times, while HPP outperforms the other methods in solution quality and run time for instances larger than 200 ponds.
Multi-Scale Features and Parallel Transformers Based Image Quality Assessment
Venue: arXiv cs.CV 2022
Year: 2022

With the increase in multimedia content, the type of distortions associated with multimedia is also increasing. This problem of image quality assessment is expanded well in the PIPAL dataset, which is still an open problem to solve for researchers. Although, recently proposed transformers networks have already been used in the literature for image quality assessment. At the same time, we notice that multi-scale feature extraction has proven to be a promising approach for image quality assessment. However, the way transformer networks are used for image quality assessment until now lacks these properties of multi-scale feature extraction. We utilized this fact in our approach and proposed a new architecture by integrating these two promising quality assessment techniques of images. Our experimentation on various datasets, including the PIPAL dataset, demonstrates that the proposed integration technique outperforms existing algorithms. The source code of the proposed algorithm is available online: https://github.com/KomalPal9610/IQA
Multiply-and-Fire (MNF): An Event-driven Sparse Neural Network Accelerator
Venue: arXiv cs.AR 2022
Year: 2022

Machine learning, particularly deep neural network inference, has become a vital workload for many computing systems, from data centers and HPC systems to edge-based computing. As advances in sparsity have helped improve the efficiency of AI acceleration, there is a continued need for improved system efficiency for both high-performance and system-level acceleration. This work takes a unique look at sparsity with an event (or activation-driven) approach to ANN acceleration that aims to minimize useless work, improve utilization, and increase performance and energy efficiency. Our analytical and experimental results show that this event-driven solution presents a new direction to enable highly efficient AI inference for both CNN and MLP workloads. This work demonstrates state-of-the-art energy efficiency and performance centring on activation-based sparsity and a highly-parallel dataflow method that improves the overall functional unit utilization (at 30 fps). This work enhances energy efficiency over a state-of-the-art solution by 1.46$\times$. Taken together, this methodology presents a novel, new direction to achieve high-efficiency, high-performance designs for next-generation AI acceleration platforms.