Reviewerly/exHarmony huggingface dataset:
Air-to-Air Collaborative Learning: A Multi-Task Orchestration in Federated Aerial Computing

Recent research on edge computing (EC) has proposed federated or collaborative learning technique, where machine learning models are shared among participating edge deployments, thereby benefiting from all available datasets without exchanging them. In addition, EC systems are currently exploiting attaching portable edge devices on drones for data processing close to the sources, to achieve high performance, fast response times and real-time insights. Existing researches lack the potential to federate edge resources and manage corresponding service entities running across multiple drones, thus resulting to sub-optimal performance. Therefore, we introduce Aerial Edge, a federated learning-based orchestration framework for a federated aerial EC system. We propose a federated multi-output linear regression model to estimate multi-task resource requirements and execution time, to select the closest drone deployment having congruent resource availability and flight time to execute ready tasks at any given time. For better utilization of resources, we propose a variant bin-packing optimization approach through gang-scheduling of multi-dependent containerized tasks that co-schedules and co-locates tasks tightly on nodes to fully utilize available resources. Extensive experiments on real-world data-trace from Alibaba cluster trace with information on task dependencies show the effectiveness, fast executions, and resource efficiency of our approach.
Diacritics generation and application in hate speech detection on Vietnamese social networks

One of the challenging problems in text processing is diacritics generation where one needs to generate diacritic marks for non-accented text. With an ever increasing amount of informal text without accents such as short text messages, emails or blog posts on social media, a software system which is capable of generating diacritic marks accurately is very useful and necessary in many situations. This paper presents an approach to improve the accuracy of diacritics generation for Vietnamese text. We propose two novel deep learning models which leverage a plausible conceptual representation for the phonetic structure of Vietnamese syllables. Experimental results on real-world datasets show that our models achieve a significant improvement as compared to the state-of-the-art methods for diacritics generation. We also demonstrate that the proposed models can be applied efficiently to improve the accuracy of hate speech detection on Vietnamese social networks.
Research on Object Detection Model Based on Feature Network Optimization

As the object detection dataset scale is smaller than the image recognition dataset ImageNet scale, transfer learning has become a basic training method for deep learning object detection models, which pre-trains the backbone network of the object detection model on an ImageNet dataset to extract features for detection tasks. However, the classification task of detection focuses on the salient region features of an object, while the location task of detection focuses on the edge features, so there is a certain deviation between the features extracted by a pretrained backbone network and those needed by a localization task. To solve this problem, a decoupled self-attention (DSA) module is proposed for one-stage object-detection models in this paper. A DSA includes two decoupled self-attention branches, so it can extract appropriate features for different tasks. It is located between the Feature Pyramid Networks (FPN) and head networks of subtasks, and used to independently extract global features for different tasks based on FPN-fused features. Although the DSA network module is simple, it can effectively improve the performance of object detection, and can easily be embedded in many detection models. Our experiments are based on the representative one-stage detection model RetinaNet. In the Common Objects in Context (COCO) dataset, when ResNet50 and ResNet101 are used as backbone networks, the detection performances can be increased by 0.4 and 0.5% AP, respectively. When the DSA module and object confidence task are both applied in RetinaNet, the detection performances based on ResNet50 and ResNet101 can be increased by 1.0 and 1.4% AP, respectively. The experiment results show the effectiveness of the DSA module.
Recommending Bug-fixing Comments from Issue Tracking Discussions in Support of Bug Repair

In practice, developers search for related earlier bugs and their associated discussion threads when faced with a new bug to repair. Typically, these discussion threads consist of comments and even bug-fixing comments intended to capture clues for facilitating the investigation and root cause of a new bug report. Over time, these discussions can become extensively lengthy and difficult to understand. Inevitably, these discussion threads lead to instances where bug-fixing comments intermingle with seemingly-unrelated comments. This task, however, poses further challenges when dealing with high volumes of bug reports. Large software systems are plagued by thousands of bug reports daily. Hence, it becomes time-consuming to investigate these bug reports efficiently. To address this gap, this paper builds a ranked-based automated tool that we refer it to as RETRORANK. Specifically, RETRORANK recommends bug-fixing comments from issue tracking discussion threads in the context of user query relevance, the use of positive language, and semantic relevance among comments. By using a combination of Vector Space Model (VSM), Sentiment Analysis (SA), and the TextRank Model (TR) we show how that past fixed bugs and their associated bug-fixing comments with relatively positive sentiments can semantically connect to investigate the root cause of a new bug. We evaluated our approach via a synthetic study and a user study. Results indicate that RETRORANK significantly improved performance when compared to the baseline VSM.
PP-RCNN: Point-Pillars Feature Set Abstraction for 3D Real-time Object Detection

3D object detection in point cloud data is an important aspect of computer vision systems, especially for autonomous driving applications. Recent literature suggests two methods of point cloud encoders; grid-based methods tend to be fast but sacrifice accuracy, while point-based methods that are learned from raw data are more accurate, but slower. In this work, we present a novel and real-time two-stage 3D object detection framework, named PointPillars-RCNN (PP-RCNN). In the first stage, we use pillars network to encode the point cloud and generate high-qulaity 3D proposals. Benefiting from the pillars network, our framework realizes real-time detection. In the second stage, we use the Point-Pillars Feature Set Abstraction (PPSA) module to extract the point-based features from raw point cloud and pillars features, and then we use the RoI-grid feature abstraction for proposals refinement. All our detection pipelines are trained end-to-end. Extensive experiments on the KITTI benchmark shows that our approach has better performance than the one-stage PointPillars algorithm, and faster than current two-stage state-of-the-art algorithms.
This! Identifying New Sentiment Slang Through Orthographic Pleonasm Online: Yasss Slay Gorg Queen Ilysm

Identifying neologisms is important for natural language processing of social web text when informal language is standard and youth slang is common. For example, failing to identify neologisms can reduce the accuracy of the lexical sentiment analysis if opinions are frequently expressed in words that are too new to be in the sentiment dictionary. This article proposes a method based on orthographic pleonasm to identify emotion-related neologisms in the social web: finding words with the most different letter repetition spelling variations. For this method, nondictionary words are extracted from a large social web corpus, spelling standardization is applied, and then words are ranked in a decreasing order of spelling variation frequency. Words with the most spelling variations are then keyword-in-context-analyzed for semantic context. Applied to a collection of comments on YouTube influencers, this method founds neologisms such as slay and early as positive terms, mixed with traditional sentiment words, exclamations, and nouns. Although orthographic pleonasm was originally used to express the speaker's rhythm and tone of voice, it is also used for initialisms in a way that is difficult to vocalize. The method is, therefore, a practical way to identify new sentiment slang, including both normal words and initialisms.
A New Classification Method in Ultrasound Images of Benign and Malignant Thyroid Nodules Based on Transfer Learning and Deep Convolutional Neural Network

Thyroid nodule is a clinical disorder with a high incidence rate, with large number of cases being detected every year globally. Early analysis of a benign or malignant thyroid nodule using ultrasound imaging is of great importance in the diagnosis of thyroid cancer. Although the b-mode ultrasound can be used to find the presence of a nodule in the thyroid, there is no existing method for an accurate and automatic diagnosis of the ultrasound image. In this pursuit, the present study envisaged the development of an ultrasound diagnosis method for the accurate and efficient identification of thyroid nodules, based on transfer learning and deep convolutional neural network. Initially, the Total Variation- (TV-) based self-adaptive image restoration method was adopted to preprocess the thyroid ultrasound image and remove the boarder and marks. With data augmentation as a training set, transfer learning with the trained GoogLeNet convolutional neural network was performed to extract image features. Finally, joint training and secondary transfer learning were performed to improve the classification accuracy, based on the thyroid images from open source data sets and the thyroid images collected from local hospitals. The GoogLeNet model was established for the experiments on thyroid ultrasound image data sets. Compared with the network established with LeNet5, VGG16, GoogLeNet, and GoogLeNet (Improved), the results showed that using GoogLeNet (Improved) model enhanced the accuracy for the nodule classification. The joint training of different data sets and the secondary transfer learning further improved its accuracy. The results of experiments on the medical image data sets of various types of diseased and normal thyroids showed that the accuracy rate of classification and diagnosis of this method was 96.04%, with a significant clinical application value.
On the Selection of Loss Functions Under Known Weak Label Models

This paper considers the problem of constructing proper loss functions for learning from weak labels by means of linear transformations of proper losses based on true labels. Recent works have shown that linear transformations defined by a left inverse of the transition matrix of the weak labelling process, transforms a true-label proper loss into a weak-label proper loss. In this paper, we show that the choice of both the true-label loss and the left inverse has a major influence on the performance of the learning algorithm, and we propose a novel method to optimize the loss selection. Some simulation results demonstrate the advantages of the proposed method.
Inherent Discriminability of BERT Towards Racial Minority Associated Data

AI and BERT (Bidirectional Encoder Representations from Transformers) have been increasingly adopted in the human resources (HR) industry for recruitment. The increased efficiency (e.g., fairness) will help remove biases in machine learning, help organizations find a qualified candidate, and remove bias in the labor market. BERT has further improved the performance of language representation models by using an auto-encoding model which incorporates larger bidirectional contexts. However, BERT’s underlying mechanisms that enhance its effectiveness, such as tokenization, masking, and leveraging the attention mechanism to compute vector score, are not well understood.This research analyzes how BERT’s architecture and its tokenization protocol affect the low number of occurrences of the minority-related data using the cosine similarity of its embeddings. In this project, by using a dataset of racially and gender-associated personal names and analyzing the interactions of transformers, we present the unfair prejudice of BERTs’ pre-trained network and autoencoding model. Furthermore, by analyzing the distance of an initial word’s token and its MASK replacement token using the cosine similarity, we will demonstrate the inherent discriminability during pre-training. Finally, this research will deliver potential solutions to mitigate discrimination and bias in BERT by examining its geometric properties.
RecSysOps: Best Practices for Operating a Large-Scale Recommender System

Ensuring the health of a modern large-scale recommendation system is a very challenging problem. To address this, we need to put in place proper logging, sophisticated exploration policies, develop ML-interpretability tools or even train new ML models to predict/detect issues of the main production model. In this talk, we shine a light on this less-discussed but important area and share some of the best practices, called RecSysOps, that we’ve learned while operating our increasingly complex recommender systems at Netflix. RecSysOps is a set of best practices for identifying issues and gaps as well as diagnosing and resolving them in a large-scale machine-learned recommender system. RecSysOps helped us to 1) reduce production issues and 2) increase recommendation quality by identifying areas of improvement and 3) make it possible to bring new innovations faster to our members by enabling us to spend more of our time on new innovations and less on debugging and firefighting issues.
Automatic Learning Path Recommendation for Open Source Projects Using Deep Learning on Knowledge Graphs

Open source is an important way for developers to collaborate on software development. More and more developers begin contributing to open-source projects. When a developer begins to contribute to an existing open source project, the first thing to do is to read and understand the project code. However, most current open source projects only provide API documentation, not project design documents for new developers. Developers can only understand the code based on scattered comments in the code, which are difficult for new comers. Therefore, developers need to find a learning path, which helps them understand the project and finish their contribution tasks quickly. In order to help developers find the learning path easily and quickly, this paper puts forward a method to automatically recommend learning paths of open source projects. It uses multiple data sources in an open source community to extract knowledge data and build knowledge graphs for open source projects. After that, based on a deep-learning-based knowledge graph embedding model and a path recommendation algorithm, the method recommends proper learning paths for developers. We select three well-known open source projects, including Lua, Memcached and TensorFlow, according to language, scope and community activity, as cases to verify our method, and do comparative experiments between the learning paths found by real developers and recommended by the method. Experiment results show that our method saves developers a lot of time while ensuring the accuracy of the recommended learning path.
Neural Architecture Search via Combinatorial Multi-Armed Bandit

Neural Architecture Search (NAS) has gained significant popularity as an effective tool for designing high performance deep neural networks (DNNs). NAS can be performed via reinforcement learning, evolutionary algorithms, differentiable architecture search or tree-search methods. While significant progress has been made for both reinforcement learning and differentiable architecture search, tree-search methods have so far failed to achieve comparable accuracy or search efficiency. In this paper, we formulate NAS as a Combinatorial Multi-Armed Bandit (CMAB) problem (CMAB-NAS). This allows the decomposition of a large search space into smaller blocks where tree-search methods can be applied more effectively and efficiently. We further leverage a tree-based method called Nested Monte-Carlo Search to tackle the CMAB-NAS problem. On CIFAR-10, our approach discovers a cell structure that achieves a low error rate that is comparable to the state-of-the-art, using only 0.58 GPU days, which is 20 times faster than current tree-search methods. Moreover, the discovered structure transfers well to large-scale datasets such as ImageNet.
On the number of zeros to the equation f(x1)+... + f(x)=a over finite fields

Let p be a prime, k a positive integer and let Fq be the finite field of q=pk elements. Let f(x) be a polynomial over Fq and a∈Fq. We denote by Ns(f,a) the number of zeros of f(x1)+⋯+f(xs)=a. In this paper, we show that∑s=1∞Ns(f,0)xs=x1−qx−xMf′(x)qMf(x), where Mf′(x) stands for the derivative of Mf(x) andMf(x):=∏m∈Fq⁎Sf,m≠0(x−1Sf,m) with Sf,m:=∑x∈FqζpTr(mf(x)), ζp being the p-th primitive unit root and Tr being the trace map from Fq to Fp. This extends Richman's theorem which treats the case of f(x) being a monomial. Moreover, we show that the generating series ∑s=1∞Ns(f,a)xs is a rational function in x and also present its explicit expression in terms of the first 2d+1 initial values N1(f,a),...,N2d+1(f,a), where d is a positive integer no more than q−1. From this result, the theorems of Chowla-Cowles-Cowles and of Myerson can be derived.
Fragmented but coherent: Lexical cohesion on a YouTube channel

This paper investigates the use and functions of lexical cohesion within and across modes as well as levels of interaction on a YouTube channel, arguing that lexical cohesion contributes to coherence by establishing links between the video, the comments and sources elsewhere on the internet in a cross-modal and intertextual manner. The analysis focuses on one video and its comments section, investigating the use of cohesion in maintaining coherence and unity across a lengthy YouTube interaction initiated by the video and continued in the comments. Cohesive ties are examined with regard to their functions in the process of meaning-making in this mediated and multimodal context. The results reveal that the use of cohesive ties is extensive within and across modes and levels of interaction. Chains of cohesion are created within the comments, with cross-modal links to the video. Despite the complex multimodality of YouTube and the fragmentation of the comments as many participants post short comments one after another, the YouTube interaction is shown to be coherent.
Implementation of Knowledge based Collaborative Filtering and Machine Learning for E-Commerce Recommendation System

Abstract This is the era of I-way. The development of high-speed computing and huge storage devices change the working culture of human. It affects the traditional business processes and shifted towards online business. It creates huge problems like overload and irrelevant information which are the causes of confusion both customers as well as enterprise. Recommendation system solves these problems. Design and development of efficient system is one of the key areas of the recent researchers. Collaborative filtering (CF) and content-based filtering algorithms are widely used in the implementation of such system. Collaborative used user’s features while content-based used item’s features. Most of the CFs are rating or review based processed homogeneous information. In this paper we proposed knowledge-based collaborative filtering algorithm for large data set that uses various activities done by users during interaction of item through E-commerce web site like clicks, select and purchase. The performance of the system is compared with the base models using real time Amazon E-commerce dataset using precession, recall and NDCG evaluation parameters in various combinations of activities performed by users on items.
Evidence Augment for Multiple-Choice Machine Reading Comprehension by Weak Supervision

Given a passage and a question, Multiple-choice Machine Reading Comprehension (MRC) requires to select the correct answer from several candidates. Existing methods consider more about the accuracy of the prediction of the final answer in a “black box”, which provides no concrete evidence to explain the choice making process. Intuitively, evidence is helpful in building a convincing multiple-choice MRC model. Due to the lack of the golden evidence labels and high cost of manual annotation, we realize weak evidence labels in an automatic way to integrate evidence extraction into the training of MRC models. This auxiliary task learns to select sentences that are more relevant to the question during training, and makes our MRC model interpretable. We come up with an end-to-end model called (EAM) Evidence Augment Model, which learns evidence extraction and answer prediction jointly. More accurate results can be obtained by our model without the need of additional manual annotation. Experimental results on RACE datasets show that we obtain an improvement in accuracy over the previous best result based on a fair Pre-trained Language Model (PrLM).
Continual Learning with Dual Regularizations

Continual learning (CL) has received a great amount of attention in recent years and a multitude of continual learning approaches arose. In this paper, we propose a continual learning approach with dual regularizations to alleviate the well-known issue of catastrophic forgetting in a challenging continual learning scenario – domain incremental learning. We reserve a buffer of past examples, dubbed memory set, to retain some information about previous tasks. The key idea is to regularize the learned representation space as well as the model outputs by utilizing the memory set based on interleaving the memory examples into the current training process. We verify our approach on four CL dataset benchmarks. Our experimental results demonstrate that the proposed approach is consistently superior to the compared methods on all benchmarks, especially in the case of small buffer size.
Active learning for the optimal design of multinomial classification in physics

Optimal design for model training is a critical topic in machine learning. Active learning aims at obtaining improved models by querying samples with maximum uncertainty according to the estimation model for artificially labeling; this has the additional advantage of achieving successful performances with a reduced number of labeled samples. We analyze its capability as an assistant for the design of experiments, extracting maximum information for learning with the minimal cost in fidelity loss, or reducing total operation costs of labeling in the laboratory. We present two typical applications as quantum information retrieval in qutrits and phase boundary prediction in many-body physics. For an equivalent multinomial classification problem, we achieve the correct rate of 99% with less than 2% of samples labeled. We reckon that active-learning-inspired physics experiments will remarkably save budget without loss of accuracy.