OpenRaiser/Intern-Atlas huggingface dataset:
Retrieval-Augmented Transformer-XL for Close-Domain Dialog Generation
Venue: arXiv cs.CL 2021
Year: 2021

Transformer-based models have demonstrated excellent capabilities of capturing patterns and structures in natural language generation and achieved state-of-the-art results in many tasks. In this paper we present a transformer-based model for multi-turn dialog response generation. Our solution is based on a hybrid approach which augments a transformer-based generative model with a novel retrieval mechanism, which leverages the memorized information in the training data via k-Nearest Neighbor search. Our system is evaluated on two datasets made by customer/assistant dialogs: the Taskmaster-1, released by Google and holding high quality, goal-oriented conversational data and a proprietary dataset collected from a real customer service call center. Both achieve better BLEU scores over strong baselines.
Measuring the technological pedagogical content knowledge (TPACK) of in-service teachers of computer science who teach algorithms and programming in upper secondary education
Venue: arXiv cs.CY 2021
Year: 2021

Based on the Technological Pedagogical and Content Knowledge (TPACK) framework (Mishra & Koehler, 2006) and the Schmidt et al. (2009) instrument which explore TPACK, this study examines a national sample of 1032 secondary teachers of computer science and measures their knowledge with respect to technology, pedagogy, content knowledge and the combination of each of these areas. Findings indicate that content knowledge and technology knowledge rating are high (average 4.38 and 4.16 respectively) and it seems that secondary teachers are less confident with their pedagogical content knowledge and their technological content knowledge (average 3.51 and 3.68 respectively).
A Decade of Research for Image Compression In Multimedia Laboratory
Venue: arXiv cs.MM 2021
Year: 2021

With the advancement of technology, we have supercomputers with high processing power and affordable prices. In addition, using multimedia expanded all around the world. This caused a vast use of images and videos in different fields. As this kind of data consists of a large amount of information, there is a need to use compression methods to store, manage or transfer them better and faster. One effective technique, which was introduced is variable resolution. This technique stimulates human vision and divides regions in pictures into two different parts, including the area of interest that needs more detail and periphery parts with less detail. This results in better compression. The variable resolution was used for image, video, and 3D motion data compression. This paper investigates the mentioned technique and some other research in this regard.
Lessons Learned Addressing Dataset Bias in Model-Based Candidate Generation at Twitter
Venue: arXiv cs.IR 2021
Year: 2021

Traditionally, heuristic methods are used to generate candidates for large scale recommender systems. Model-based candidate generation promises multiple potential advantages, primarily that we can explicitly optimize the same objective as the downstream ranking model. However, large scale model-based candidate generation approaches suffer from dataset bias problems caused by the infeasibility of obtaining representative data on very irrelevant candidates. Popular techniques to correct dataset bias, such as inverse propensity scoring, do not work well in the context of candidate generation. We first explore the dynamics of the dataset bias problem and then demonstrate how to use random sampling techniques to mitigate it. Finally, in a novel application of fine-tuning, we show performance gains when applying our candidate generation system to Twitter's home timeline.
Improved Product-Based High-Dimensional Expanders
Venue: arXiv cs.DM 2021
Year: 2021

High-dimensional expanders generalize the notion of expander graphs to higher-dimensional simplicial complexes. In contrast to expander graphs, only a handful of high-dimensional expander constructions have been proposed, and no elementary combinatorial construction with near-optimal expansion is known. In this paper, we introduce an improved combinatorial high-dimensional expander construction, by modifying a previous construction of Liu, Mohanty, and Yang (ITCS 2020), which is based on a high-dimensional variant of a tensor product. Our construction achieves a spectral gap of $Ω(\frac{1}{k^2})$ for random walks on the $k$-dimensional faces, which is only quadratically worse than the optimal bound of $Θ(\frac{1}{k})$. Previous combinatorial constructions, including that of Liu, Mohanty, and Yang, only achieved a spectral gap that is exponentially small in $k$. We also present reasoning that suggests our construction is optimal among similar product-based constructions.
Stochastic Coordination in Heterogeneous Load Balancing Systems
Venue: arXiv cs.DC 2021
Year: 2021

Current-day data centers and high-volume cloud services employ a broad set of heterogeneous servers. In such settings, client requests typically arrive at multiple entry points, and dispatching them to servers is an urgent distributed systems problem. This paper presents an efficient solution to the load balancing problem in such systems that improves on and overcomes problems of previous solutions. The load balancing problem is formulated as a stochastic optimization problem, and an efficient algorithmic solution is obtained based on a subtle mathematical analysis of the problem. Finally, extensive evaluation of the solution on simulated data shows that it outperforms previous solutions. Moreover, the resulting dispatching policy can be computed very efficiently, making the solution practically viable.
Separation of Powers in Federated Learning
Venue: arXiv cs.CR 2021
Year: 2021

Federated Learning (FL) enables collaborative training among mutually distrusting parties. Model updates, rather than training data, are concentrated and fused in a central aggregation server. A key security challenge in FL is that an untrustworthy or compromised aggregation process might lead to unforeseeable information leakage. This challenge is especially acute due to recently demonstrated attacks that have reconstructed large fractions of training data from ostensibly "sanitized" model updates. In this paper, we introduce TRUDA, a new cross-silo FL system, employing a trustworthy and decentralized aggregation architecture to break down information concentration with regard to a single aggregator. Based on the unique computational properties of model-fusion algorithms, all exchanged model updates in TRUDA are disassembled at the parameter-granularity and re-stitched to random partitions designated for multiple TEE-protected aggregators. Thus, each aggregator only has a fragmentary and shuffled view of model updates and is oblivious to the model architecture. Our new security mechanisms can fundamentally mitigate training reconstruction attacks, while still preserving the final accuracy of trained models and keeping performance overheads low.
Optimal Distributed Bandwidth Allocation in NB-IoT Networks
Venue: arXiv cs.NI 2021
Year: 2021

In this paper, we investigate a key problem of Narrowband-Internet of Things (NB-IoT) in the context of 5G with Mobile Edge Computing (MEC). We address the challenge that IoT devices may have different priorities when demanding bandwidth for data transmission in specific applications and services. Due to the scarcity of bandwidth in an MEC enabled IoT network, our objective is to optimize bandwidth allocation for a group of NB-IoT devices in a way that the group can work collaboratively to maximize their overall utility. To this end, we design an optimal distributed algorithm and use simulations to demonstrate its efficacy to effectively manage various IoT data streams in a fully distributed framework.
Quantifying sources of uncertainty in drug discovery predictions with probabilistic models
Venue: arXiv cs.LG 2021
Year: 2021

Knowing the uncertainty in a prediction is critical when making expensive investment decisions and when patient safety is paramount, but machine learning (ML) models in drug discovery typically provide only a single best estimate and ignore all sources of uncertainty. Predictions from these models may therefore be over-confident, which can put patients at risk and waste resources when compounds that are destined to fail are further developed. Probabilistic predictive models (PPMs) can incorporate uncertainty in both the data and model, and return a distribution of predicted values that represents the uncertainty in the prediction. PPMs not only let users know when predictions are uncertain, but the intuitive output from these models makes communicating risk easier and decision making better. Many popular machine learning methods have a PPM or Bayesian analogue, making PPMs easy to fit into current workflows. We use toxicity prediction as a running example, but the same principles apply for all prediction models used in drug discovery. The consequences of ignoring uncertainty and how PPMs account for uncertainty are also described. We aim to make the discussion accessible to a broad non-mathematical audience. Equations are provided to make ideas concrete for mathematical readers (but can be skipped without loss of understanding) and code is available for computational researchers (https://github.com/stanlazic/ML_uncertainty_quantification).
Futuristic Intelligent Transportation System
Venue: arXiv cs.NI 2021
Year: 2021

The emerging autonomous vehicles (AVs) will inevitably revolutionize the transportation systems. This is because of a key feature of AVs; instead of being managed by human drivers as the conventional vehicles, AVs are of the complete capability to manage the driving by themselves. As a result, the futuristic intelligent transportation system (FITS) can be a centrally managed and optimized system with the fully coordinated driving of vehicles, which is impossible by the current transportation systems controlled by humans. In this article, we envision the operation of such FITS when AVs, advanced vehicular networks (VANETs) and artificial intelligence (AI) are adopted. Specifically, we first develop the autonomous vehicular networks (AVNs) based on the advanced development of AVs and heterogeneous vehicular communication technologies to achieve global data collection and real-time data sharing. With this network architecture, we then integrate AVNs and AI based on the intelligent digital twin (IDT) to design the FITS with the target of setting up an accurate and efficient global traffic scheduling system. After that, compared with the conventional schemes, a customized path planning case is studied to evaluate the performance of the proposed FITS. Finally, we highlight the emerging issues related to the FITS for future research.
A hybrid algorithm based on Community Detection and Multi-Attribute Decision-Making for Influence Maximization
Venue: arXiv cs.SI 2021
Year: 2021

The influence maximization problem is trying to identify a set of K nodes by which the spread of influence, diseases, or information is maximized. The optimization of influence by finding such a set is an NP-hard problem and a key issue in analyzing complex networks. In this paper, a new greedy and hybrid approach based on a community detection algorithm and a MADM technique (TOPSIS) is proposed to cope with the problem, called, Greedy TOPSIS and Community-Based (GTaCB) algorithm. The paper concisely introduces community detection and the TOPSIS technique, then it presents the pseudo-code of the proposed algorithm. Afterward, it compares the performance of the solution which is found by GTaCB with some well-known greedy algorithms, based on Degree Centrality, Closeness Centrality, Betweenness Centrality, PageRank as well as TOPSIS, from two aspects: diffusion quality and diffusion speed. In order to evaluate the performance of GTaCB, computational experiments on nine different types of real-world networks are provided. The tests are conducted via one of the renowned epidemic diffusion models, namely, Susceptible-Infected-Recovered (SIR) model. The simulations exhibit that in most of the cases the proposed algorithm significantly outperforms the others, chiefly as the number of initial nodes or probability of infection increases.
Fast Nonblocking Persistence for Concurrent Data Structures
Venue: arXiv cs.DC 2021
Year: 2021

We present a fully lock-free variant of the recent Montage system for persistent data structures. Our variant, nbMontage, adds persistence to almost any nonblocking concurrent structure without introducing significant overhead or blocking of any kind. Like its predecessor, nbMontage is buffered durably linearizable: it guarantees that the state recovered in the wake of a crash will represent a consistent prefix of pre-crash execution. Unlike its predecessor, nbMontage ensures wait-free progress of the persistence frontier, thereby bounding the number of recent updates that may be lost on a crash, and allowing a thread to force an update of the frontier (i.e., to perform a sync operation) without the risk of blocking. As an extra benefit, the helping mechanism employed by our wait-free sync significantly reduces its latency. Performance results for nonblocking queues, skip lists, trees, and hash tables rival custom data structures in the literature -- dramatically faster than achieved with prior general-purpose systems, and generally within 50% of equivalent non-persistent structures placed in DRAM.
Understanding the Perceived Relevance of Capability Measures: A Survey of Agile Software Development Practitioners
Venue: arXiv cs.SE 2021
Year: 2021

Context: In the light of the swift and iterative nature of Agile Software Development (ASD) practices, establishing deeper insights into capability measurement within the context of team formation is crucial, as the capability of individuals and teams can affect team performance and productivity. Although a former Systematic Literature Review (SLR) synthesized the state of the art in relation to capability measurement in ASD with a focus on selecting individuals to agile teams, and capabilities related to team performance and success, determining to what degree the SLR's results apply to practice can provide progressive insights to both research and practice. Objective: Our study investigates how agile practitioners perceive the relevance of individual and team level measures for characterizing the capability of an agile team and its members. Furthermore, to scrutinize variations in practitioners' perceptions, our study further analyzes perceptions across stratified demographic groups. Method: We undertook a Web-based survey using a questionnaire built based on the capability measures identified from a previously conducted SLR. Results: Our survey responses (60) indicate that 127 individual and 28 team capability measures were considered as relevant by the majority of practitioners. We also identified seven individual and one team capability measure that have not been previously characterized by our SLR. The surveyed practitioners suggested that an agile team member's responsibility and questioning skills significantly represent the member's capability. Conclusion: Results from our survey align with our SLR's findings. Measures associated with social aspects were observed to be dominant compared to technical and innovative aspects. Our results can support agile practitioners in their team composition decisions.
From Verification to Causality-based Explications
Venue: arXiv cs.LO 2021
Year: 2021

In view of the growing complexity of modern software architectures, formal models are increasingly used to understand why a system works the way it does, opposed to simply verifying that it behaves as intended. This paper surveys approaches to formally explicate the observable behavior of reactive systems. We describe how Halpern and Pearl's notion of actual causation inspired verification-oriented studies of cause-effect relationships in the evolution of a system. A second focus lies on applications of the Shapley value to responsibility ascriptions, aimed to measure the influence of an event on an observable effect. Finally, formal approaches to probabilistic causation are collected and connected, and their relevance to the understanding of probabilistic systems is discussed.
5G Deployment Strategies for High Positioning Accuracy in Indoor Environments
Venue: arXiv cs.NI 2021
Year: 2021

Indoor positioning is currently recognized as one of the important features in emergency, commercial and industrial applications. The 5G network enhances mobility, flexibility, reliability, and security to new higher levels which greatly benefit the IoT and industrial applications. Industrial IoT (IIoT) use-cases are characterized by ambitious system requirements for positioning accuracy in many verticals. For example, on the factory floor, it is important to locate assets and moving objects such as forklifts. The deployment design for different IIoT environments has a significant impact on the positioning per-performance in terms of both accuracy and availability of the service. Indoor factory (InF) and indoor open office (IOO) are two available and standardized Third Generation Partnership Project (3GPP) scenarios for evaluation of indoor channel models and positioning performance in IIoT use cases. This paper aims to evaluate the positioning performance in terms of accuracy and availability while considering different deployment strategies. Our simulation-based evaluation shows that deployment plays a vital role when it comes to achieving high accuracy positioning performance. It is for example favorable to deploy the 5G Transmission and Reception Points (TRPs) on the walls of the factory halls than deploying them attached to the ceiling.
Database Matching Under Column Deletions
Venue: arXiv cs.IT 2021
Year: 2021

De-anonymizing user identities by matching various forms of user data available on the internet raises privacy concerns. A fundamental understanding of the privacy leakage in such scenarios requires a careful study of conditions under which correlated databases can be matched. Motivated by synchronization errors in time indexed databases, in this work, matching of random databases under random column deletion is investigated. Adapting tools from information theory, in particular ones developed for the deletion channel, conditions for database matching in the absence and presence of deletion location information are derived, showing that partial deletion information significantly increases the achievable database growth rate for successful matching. Furthermore, given a batch of correctly-matched rows, a deletion detection algorithm that provides partial deletion information is proposed and a lower bound on the algorithm's deletion detection probability in terms of the column size and the batch size is derived. The relationship between the database size and the batch size required to guarantee a given deletion detection probability using the proposed algorithm suggests that a batch size growing double-logarithmic with the row size is sufficient for a nonzero detection probability guarantee.
TF-IDF vs Word Embeddings for Morbidity Identification in Clinical Notes: An Initial Study
Venue: arXiv cs.CL 2021
Year: 2021

Today, we are seeing an ever-increasing number of clinical notes that contain clinical results, images, and textual descriptions of patient's health state. All these data can be analyzed and employed to cater novel services that can help people and domain experts with their common healthcare tasks. However, many technologies such as Deep Learning and tools like Word Embeddings have started to be investigated only recently, and many challenges remain open when it comes to healthcare domain applications. To address these challenges, we propose the use of Deep Learning and Word Embeddings for identifying sixteen morbidity types within textual descriptions of clinical records. For this purpose, we have used a Deep Learning model based on Bidirectional Long-Short Term Memory (LSTM) layers which can exploit state-of-the-art vector representations of data such as Word Embeddings. We have employed pre-trained Word Embeddings namely GloVe and Word2Vec, and our own Word Embeddings trained on the target domain. Furthermore, we have compared the performances of the deep learning approaches against the traditional tf-idf using Support Vector Machine and Multilayer perceptron (our baselines). From the obtained results it seems that the latter outperforms the combination of Deep Learning approaches using any word embeddings. Our preliminary results indicate that there are specific features that make the dataset biased in favour of traditional machine learning approaches.
Modelling DVFS and UFS for Region-Based Energy Aware Tuning of HPC Applications
Venue: arXiv cs.DC 2021
Year: 2021

Energy efficiency and energy conservation are one of the most crucial constraints for meeting the 20MW power envelope desired for exascale systems. Towards this, most of the research in this area has been focused on the utilization of user-controllable hardware switches such as per-core dynamic voltage frequency scaling (DVFS) and software controlled clock modulation at the application level. In this paper, we present a tuning plugin for the Periscope Tuning Framework which integrates fine-grained autotuning at the region level with DVFS and uncore frequency scaling (UFS). The tuning is based on a feed-forward neural network which is formulated using Performance Monitoring Counters (PMC) supported by x86 systems and trained using standardized benchmarks. Experiments on five standardized hybrid benchmarks show an energy improvement of 16.1% on average when the applications are tuned according to our methodology as compared to 7.8% for static tuning.
Unreliable Sensors for Reliable Efficient Robots
Venue: arXiv cs.CG 2021
Year: 2021

The vast majority of existing Distributed Computing literature about mobile robotic swarms considers computability issues: characterizing the set of system hypotheses that enables problem solvability. By contrast, the focus of this work is to investigate complexity issues: obtaining quantitative results about a given problem that admits solutions. Our quantitative measurements rely on a newly developed simulation framework to benchmark pen and paper designs. First, we consider the maximum traveled distance when gathering robots at a given location, not known beforehand (both in the two robots and in the n robots settings) in the classical OBLOT model, for the FSYNC, SSYNC, and ASYNC schedulers. This particular metric appears relevant as it correlates closely to what would be real world fuel consumption. Then, we introduce the possibility of errors in the vision of robots, and assess the behavior of known rendezvous (aka two robots gathering) and leader election protocols when sensors are unreliable. We also introduce two new algorithms, one for fuel efficient convergence, and one for leader election, that operate reliably despite unreliable sensors.