OpenRaiser/Intern-Atlas huggingface dataset:
Adaptive Probabilistic Model for Energy-Efficient Distance-based Clustering in WSNs (Adapt-P): A LEACH-based Analytical Study
Venue: arXiv cs.NI 2021
Year: 2021

Network lifetime and energy consumption of data transmission have been primary Quality of Service (QoS) obligations in Wireless Sensor Networks (WSNs). The environment of a WSN is often organized into clusters to mitigate the management complexity of such obligations. However, the distance between Sensor Nodes (SNs) and the number of clusters per round are vital factors that affect QoS performance of a WSN. A designer's conundrum resolves around the desire to sustain a balance between the limited residual energy of SNs and the demand for prolonged network lifetime. Any imbalance in controlling such objectives results in either QoS penalties due to draining SN energies, or an over-cost environment that is significantly difficult to distribute and operate. Low-Energy Adaptive Clustering Hierarchy (LEACH) is a distributed algorithm proposed to tackle such difficulties. Proposed LEACH-based algorithms focus on residual energies of SNs to compute a probability function that selects cluster-heads and an optimal energy-efficient path toward a destination SN. Nevertheless, these algorithms do not consider variations in network's state at run-time. Such a state changes in an adaptive manner according to existing network structures and conditions. Thus, cluster-heads per round are not elected adaptively depending on the state and distances between SNs. This paper proposes an energy-efficient adaptive distance-based clustering called Adapt-P, in which an adaptive probability function is developed to formulate clusters. A near-optimal distance between each cluster-head and its cluster-members is formulated so that energy consumption of the network is mitigated and network lifetime is maximized. The cluster-head selection probability is adapted at the end of each round based on the maximum number of cluster-heads permitted per round found a priori and the number of alive SNs in the network.
Exposure of occupations to technologies of the fourth industrial revolution
Venue: arXiv cs.CY 2021
Year: 2021

The fourth industrial revolution (4IR) is likely to have a substantial impact on the economy. Companies need to build up capabilities to implement new technologies, and automation may make some occupations obsolete. However, where, when, and how the change will happen remain to be determined. Robust empirical indicators of technological progress linked to occupations can help to illuminate this change. With this aim, we provide such an indicator based on patent data. Using natural language processing, we calculate patent exposure scores for more than 900 occupations, which represent the technological progress related to them. To provide a lens on the impact of the 4IR, we differentiate between traditional and 4IR patent exposure. Our method differs from previous approaches in that it both accounts for the diversity of task-level patent exposures within an occupation and reflects work activities more accurately. We find that exposure to 4IR patents differs from traditional patent exposure. Manual tasks, and accordingly occupations such as construction and production, are exposed mainly to traditional (non-4IR) patents but have low exposure to 4IR patents. The analysis suggests that 4IR technologies may have a negative impact on job growth; this impact appears 10 to 20 years after patent filing. Further, we compared the 4IR exposure to other automation and AI exposure scores. Whereas many measures refer to theoretical automation potential, our patent-based indicator reflects actual technology diffusion. Our work not only allows analyses of the impact of 4IR technologies as a whole, but also provides exposure scores for more than 300 technology fields, such as AI and smart office technologies. Finally, the work provides a general mapping of patents to tasks and occupations, which enables future researchers to construct individual exposure measures.
The Role of the Scrum Master in an Industry based University Course
Venue: arXiv cs.SE 2021
Year: 2021

Background: Scrum is the most commonly used agile software development method, and the role of the Scrum Master (SM) in a Scrum environment is vital. Therefore, through an industry based university course for final year undergraduate and masters students, we aimed to give students both theoretical and practical understanding of the role of SM via hands-on experience on playing the role in real-world Scrum contexts. Method: We asked them (92 students) to share their experiences and learnings on the role of SM through reflective surveys and essays. Students participated in reflective surveys (311 survey responses) over 5 weeks in the course, and they submitted essays (92 essays) at the end of the course. We used a mixed-methods approach using Socio-Technical Grounded Theory analysis techniques and trend and regression based statistical analysis to analyse the survey responses and the essays. Findings: We identified the key responsibilities and duties of the SM, common challenges faced by the SM and the team due to the role of the SM, root causes of the challenges, strategies used by the SM and the team to overcome the challenges, and the overall experience of the students. Based on the results, we present recommendations for educators.
Multi-Faceted Hierarchical Multi-Task Learning for a Large Number of Tasks with Multi-dimensional Relations
Venue: arXiv cs.LG 2021
Year: 2021

There has been many studies on improving the efficiency of shared learning in Multi-Task Learning(MTL). Previous work focused on the "micro" sharing perspective for a small number of tasks, while in Recommender Systems(RS) and other AI applications, there are often demands to model a large number of tasks with multi-dimensional task relations. For example, when using MTL to model various user behaviors in RS, if we differentiate new users and new items from old ones, there will be a cartesian product style increase of tasks with multi-dimensional relations. This work studies the "macro" perspective of shared learning network design and proposes a Multi-Faceted Hierarchical MTL model(MFH). MFH exploits the multi-dimension task relations with a nested hierarchical tree structure which maximizes the shared learning. We evaluate MFH and SOTA models in a large industry video platform of 10 billion samples and results show that MFH outperforms SOTA MTL models significantly in both offline and online evaluations across all user groups, especially remarkable for new users with an online increase of 9.1\% in app time per user and 1.85\% in next-day retention rate. MFH now has been deployed in a large scale online video recommender system. MFH is especially beneficial to the cold-start problems in RS where new users and new items often suffer from a "local overfitting" phenomenon. However, the idea is actually generic and widely applicable to other MTL scenarios.
ViDA-MAN: Visual Dialog with Digital Humans
Venue: arXiv cs.CV 2021
Year: 2021

We demonstrate ViDA-MAN, a digital-human agent for multi-modal interaction, which offers realtime audio-visual responses to instant speech inquiries. Compared to traditional text or voice-based system, ViDA-MAN offers human-like interactions (e.g, vivid voice, natural facial expression and body gestures). Given a speech request, the demonstration is able to response with high quality videos in sub-second latency. To deliver immersive user experience, ViDA-MAN seamlessly integrates multi-modal techniques including Acoustic Speech Recognition (ASR), multi-turn dialog, Text To Speech (TTS), talking heads video generation. Backed with large knowledge base, ViDA-MAN is able to chat with users on a number of topics including chit-chat, weather, device control, News recommendations, booking hotels, as well as answering questions via structured knowledge.
A critical review of the proposed reforms of the academic performance indicators applied in the assessment of researchers' performance in Hungary
Venue: arXiv cs.DL 2021
Year: 2021

The academic performance indicators of the Doctor of Science title, the highest and most prestigious qualification awarded by the Hungarian Academy of Sciences (HAS), are key in the national assessment system. The types of performance indicators, as well as their minimum values, are incorporated into the application requirements of academic promotions, scientific qualifications, and research scholarships. HAS has proposed a reform of these performance indicators, to align with the current national and global trends. The proposed modifications are generally based on arbitrary decisions and the consensus between academicians, namely, the representatives of the sections of HAS. This paper contains a bibliometric analysis of 25,000 publications produced between 2011 and 2020 by 683 researchers affiliated with HAS's Section of Earth Sciences. The bibliometric data of the publications are processed by integer and fractional counting, respectively. The main goal of the paper is to argue that discipline-specific co-authorship patterns should be accounted for in the assessment procedure. It is also shown that the homogenization of the performance indicators and the rigid use of the integer counting method favour hard natural science disciplines and disadvantage social science disciplines. Finally, I describe some components of an alternative publishing strategy which would be most prudent for researchers, given the proposed assessment criteria.
Quantitative Evaluation of Snapshot Graphs for the Analysis of Temporal Networks
Venue: arXiv cs.SI 2021
Year: 2021

One of the most common approaches to the analysis of dynamic networks is through time-window aggregation. The resulting representation is a sequence of static networks, i.e. the snapshot graph. Despite this representation being widely used in the literature, a general framework to evaluate the soundness of snapshot graphs is still missing. In this article, we propose two scores to quantify conflicting objectives: Stability measures how much stable the sequence of snapshots is, while Fidelity measures the loss of information compared to the original data. We also develop a technique of targeted filtering of the links, to simplify the original temporal network. Our framework is tested on datasets of proximity and face-to-face interactions.
Response-based Distillation for Incremental Object Detection
Venue: arXiv cs.CV 2021
Year: 2021

Traditional object detection are ill-equipped for incremental learning. However, fine-tuning directly on a well-trained detection model with only new data will leads to catastrophic forgetting. Knowledge distillation is a straightforward way to mitigate catastrophic forgetting. In Incremental Object Detection (IOD), previous work mainly focuses on feature-level knowledge distillation, but the different response of detector has not been fully explored yet. In this paper, we propose a fully response-based incremental distillation method focusing on learning response from detection bounding boxes and classification predictions. Firstly, our method transferring category knowledge while equipping student model with the ability to retain localization knowledge during incremental learning. In addition, we further evaluate the qualities of all locations and provides valuable response by adaptive pseudo-label selection (APS) strategies. Finally, we elucidate that knowledge from different responses should be assigned with different importance during incremental distillation. Extensive experiments conducted on MS COCO demonstrate significant advantages of our method, which substantially narrow the performance gap towards full training.
SEDML: Securely and Efficiently Harnessing Distributed Knowledge in Machine Learning
Venue: arXiv cs.CR 2021
Year: 2021

Training high-performing deep learning models require a rich amount of data which is usually distributed among multiple data sources in practice. Simply centralizing these multi-sourced data for training would raise critical security and privacy concerns, and might be prohibited given the increasingly strict data regulations. To resolve the tension between privacy and data utilization in distributed learning, a machine learning framework called private aggregation of teacher ensembles(PATE) has been recently proposed. PATE harnesses the knowledge (label predictions for an unlabeled dataset) from distributed teacher models to train a student model, obviating access to distributed datasets. Despite being enticing, PATE does not offer protection for the individual label predictions from teacher models, which still entails privacy risks. In this paper, we propose SEDML, a new protocol which allows to securely and efficiently harness the distributed knowledge in machine learning. SEDML builds on lightweight cryptography and provides strong protection for the individual label predictions, as well as differential privacy guarantees on the aggregation results. Extensive evaluations show that while providing privacy protection, SEDML preserves the accuracy as in the plaintext baseline. Meanwhile, SEDML's performance in computing and communication is 43 times and 1.23 times higher than the latest technology, respectively.
A Reinforcement Learning Approach for Re-allocating Drone Swarm Services
Venue: arXiv cs.MA 2021
Year: 2021

We propose a novel framework for the re-allocation of drone swarms for delivery services known as Swarm-based Drone-as-a-Service (SDaaS). The re-allocation framework ensures maximum profit to drone swarm providers while meeting the time requirement of service consumers. The constraints in the delivery environment (e.g., limited recharging pads) are taken into consideration. We utilize reinforcement learning (RL) to select the best allocation and scheduling of drone swarms given a set of requests from multiple consumers. We conduct a set of experiments to evaluate and compare the efficiency of the proposed approach considering the provider's profit and run-time efficiency.
Measuring the Effectiveness of Digital Hygiene using Historical DNS Data
Venue: arXiv cs.CR 2021
Year: 2021

This paper describes an ongoing experiment evaluating the efficacy of a digital safety intervention in six high-risk, low capacity Civil Society Organisations (CSOs) in Central Asia. The evaluation takes the form of statistical analysis of DNS traffic in each organisation, obtained via security tools installed by researchers. The hypothesis is that the digital safety intervention strengthens the overall digital security posture of the CSOs, as measured by number of malware attacks intercepted by a cloud-based DNS firewall installed on the CSOs networks. The research collects DNS traffic from CSOs that are participating in the digital safety intervention, and compares a treatment group consisting of four CSOs against DNS traffic from a second group of two CSOs in which the intervention has not yet taken place. This project is ongoing, with data collection underway at a number of Central Asian CSOs. In this paper we outline the experimental design of the project, and look at the early data coming out of the DNS firewall. This is done to support the ultimate question of whether DNS data such as this can be used to accurately assess the efficacy of digital hygiene efforts.
Dendritic Self-Organizing Maps for Continual Learning
Venue: arXiv cs.NE 2021
Year: 2021

Current deep learning architectures show remarkable performance when trained in large-scale, controlled datasets. However, the predictive ability of these architectures significantly decreases when learning new classes incrementally. This is due to their inclination to forget the knowledge acquired from previously seen data, a phenomenon termed catastrophic-forgetting. On the other hand, Self-Organizing Maps (SOMs) can model the input space utilizing constrained k-means and thus maintain past knowledge. Here, we propose a novel algorithm inspired by biological neurons, termed Dendritic-Self-Organizing Map (DendSOM). DendSOM consists of a single layer of SOMs, which extract patterns from specific regions of the input space accompanied by a set of hit matrices, one per SOM, which estimate the association between units and labels. The best-matching unit of an input pattern is selected using the maximum cosine similarity rule, while the point-wise mutual information is employed for class inference. DendSOM performs unsupervised feature extraction as it does not use labels for targeted updating of the weights. It outperforms classical SOMs and several state-of-the-art continual learning algorithms on benchmark datasets, such as the Split-MNIST and Split-CIFAR-10. We propose that the incorporation of neuronal properties in SOMs may help remedy catastrophic forgetting.
Model-based Reinforcement Learning for Service Mesh Fault Resiliency in a Web Application-level
Venue: arXiv cs.DC 2021
Year: 2021

Microservice-based architectures enable different aspects of web applications to be created and updated independently, even after deployment. Associated technologies such as service mesh provide application-level fault resilience through attribute configurations that govern the behavior of request-response service -- and the interactions among them -- in the presence of failures. While this provides tremendous flexibility, the configured values of these attributes -- and the relationships among them -- can significantly affect the performance and fault resilience of the overall application. Furthermore, it is impossible to determine the best and worst combinations of attribute values with respect to fault resiliency via testing, due to the complexities of the underlying distributed system and the many possible attribute value combinations. In this paper, we present a model-based reinforcement learning workflow towards service mesh fault resiliency. Our approach enables the prediction of the most significant fault resilience behaviors at a web application-level, scratching from single service to aggregated multi-service management with efficient agent collaborations.
Hybrid physics-based and data-driven modeling with calibrated uncertainty for lithium-ion battery degradation diagnosis and prognosis
Venue: arXiv cs.LG 2021
Year: 2021

Advancing lithium-ion batteries (LIBs) in both design and usage is key to promoting electrification in the coming decades to mitigate human-caused climate change. Inadequate understanding of LIB degradation is an important bottleneck that limits battery durability and safety. Here, we propose hybrid physics-based and data-driven modeling for online diagnosis and prognosis of battery degradation. Compared to existing battery modeling efforts, we aim to build a model with physics as its backbone and statistical learning techniques as enhancements. Such a hybrid model has better generalizability and interpretability together with a well-calibrated uncertainty associated with its prediction, rendering it more valuable and relevant to safety-critical applications under realistic usage scenarios.
Addressing out-of-distribution label noise in webly-labelled data
Venue: arXiv cs.CV 2021
Year: 2021

A recurring focus of the deep learning community is towards reducing the labeling effort. Data gathering and annotation using a search engine is a simple alternative to generating a fully human-annotated and human-gathered dataset. Although web crawling is very time efficient, some of the retrieved images are unavoidably noisy, i.e. incorrectly labeled. Designing robust algorithms for training on noisy data gathered from the web is an important research perspective that would render the building of datasets easier. In this paper we conduct a study to understand the type of label noise to expect when building a dataset using a search engine. We review the current limitations of state-of-the-art methods for dealing with noisy labels for image classification tasks in the case of web noise distribution. We propose a simple solution to bridge the gap with a fully clean dataset using Dynamic Softening of Out-of-distribution Samples (DSOS), which we design on corrupted versions of the CIFAR-100 dataset, and compare against state-of-the-art algorithms on the web noise perturbated MiniImageNet and Stanford datasets and on real label noise datasets: WebVision 1.0 and Clothing1M. Our work is fully reproducible https://git.io/JKGcj
Synchronous-Clock Range-Angle Relative Acoustic Navigation: A Unified Approach to Multi-AUV Localization, Command, Control and Coordination
Venue: arXiv cs.RO 2021
Year: 2021

This paper presents a scalable acoustic navigation approach for the unified command, control and coordination of multiple autonomous underwater vehicles (AUVs). Existing multi-AUV operations typically achieve coordination manually, by programming individual vehicles on the surface via radio communications, which becomes impractical with large vehicle numbers; or they require bi-directional inter-vehicle acoustic communications to achieve limited coordination when submerged, with limited scalability due to the physical properties of the acoustic channel. Our approach utilizes a single, periodically-broadcasting beacon acting as a navigation reference for the group of AUVs, each of which carries a chip-scale atomic clock (CSAC) and fixed ultra-short baseline (USBL) array of acoustic receivers. One-way travel-time (OWTT) from synchronized clocks and time-delays between signals received by each array element allows any number of vehicles within receive distance to determine range, angle, and thus determine their relative position to the beacon. The operator can command different vehicle behaviors by selecting between broadcast signals from a predetermined set, while coordination between AUVs is achieved without inter-vehicle communication, by defining individual vehicle behaviors within the context of the group. Vehicle behaviors are designed within a beacon-centric moving frame of reference, allowing the operator to control the absolute position of the AUV group by re-positioning the navigation beacon to survey the area of interest. Multiple deployments with a fleet of three miniature, low-cost SandShark AUVs performing closed-loop acoustic navigation in real-time provide experimental results validated against a secondary long-baseline (LBL) positioning system, demonstrating the capabilities and robustness of our approach with real-world data.
Leveraging Local Temporal Information for Multimodal Scene Classification
Venue: arXiv cs.CV 2021
Year: 2021

Robust video scene classification models should capture the spatial (pixel-wise) and temporal (frame-wise) characteristics of a video effectively. Transformer models with self-attention which are designed to get contextualized representations for individual tokens given a sequence of tokens, are becoming increasingly popular in many computer vision tasks. However, the use of Transformer based models for video understanding is still relatively unexplored. Moreover, these models fail to exploit the strong temporal relationships between the neighboring video frames to get potent frame-level representations. In this paper, we propose a novel self-attention block that leverages both local and global temporal relationships between the video frames to obtain better contextualized representations for the individual frames. This enables the model to understand the video at various granularities. We illustrate the performance of our models on the large scale YoutTube-8M data set on the task of video categorization and further analyze the results to showcase improvement.