OpenRaiser/Intern-Atlas huggingface dataset:
Understanding and Improving the Role of Projection Head in Self-Supervised Learning
Venue: arXiv cs.LG 2022
Year: 2022

Self-supervised learning (SSL) aims to produce useful feature representations without access to any human-labeled data annotations. Due to the success of recent SSL methods based on contrastive learning, such as SimCLR, this problem has gained popularity. Most current contrastive learning approaches append a parametrized projection head to the end of some backbone network to optimize the InfoNCE objective and then discard the learned projection head after training. This raises a fundamental question: Why is a learnable projection head required if we are to discard it after training? In this work, we first perform a systematic study on the behavior of SSL training focusing on the role of the projection head layers. By formulating the projection head as a parametric component for the InfoNCE objective rather than a part of the network, we present an alternative optimization scheme for training contrastive learning based SSL frameworks. Our experimental study on multiple image classification datasets demonstrates the effectiveness of the proposed approach over alternatives in the SSL literature.
Confidence-Aware Paced-Curriculum Learning by Label Smoothing for Surgical Scene Understanding
Venue: arXiv cs.CV 2022
Year: 2022

Curriculum learning and self-paced learning are the training strategies that gradually feed the samples from easy to more complex. They have captivated increasing attention due to their excellent performance in robotic vision. Most recent works focus on designing curricula based on difficulty levels in input samples or smoothing the feature maps. However, smoothing labels to control the learning utility in a curriculum manner is still unexplored. In this work, we design a paced curriculum by label smoothing (P-CBLS) using paced learning with uniform label smoothing (ULS) for classification tasks and fuse uniform and spatially varying label smoothing (SVLS) for semantic segmentation tasks in a curriculum manner. In ULS and SVLS, a bigger smoothing factor value enforces a heavy smoothing penalty in the true label and limits learning less information. Therefore, we design the curriculum by label smoothing (CBLS). We set a bigger smoothing value at the beginning of training and gradually decreased it to zero to control the model learning utility from lower to higher. We also designed a confidence-aware pacing function and combined it with our CBLS to investigate the benefits of various curricula. The proposed techniques are validated on four robotic surgery datasets of multi-class, multi-label classification, captioning, and segmentation tasks. We also investigate the robustness of our method by corrupting validation data into different severity levels. Our extensive analysis shows that the proposed method improves prediction accuracy and robustness.
Scaffolding Generation using a 3D Physarum Polycephalum Simulation
Venue: arXiv cs.GR 2022
Year: 2022

In this demo, we present a novel technique for approximating topologically optimal scaffoldings for 3D printed objects using a Monte Carlo algorithm based on the foraging behavior of the Physarum polycephalum slime mold. As a case study, we have created a biologically inspired bicycle helmet using this technique that is designed to be effective in resisting impacts. We have created a prototype of this helmet and propose further studies that measure the effectiveness and validity of the design.
SHLE: Devices Tracking and Depth Filtering for Stereo-based Height Limit Estimation
Venue: arXiv cs.CV 2022
Year: 2022

Recently, over-height vehicle strike frequently occurs, causing great economic cost and serious safety problems. Hence, an alert system which can accurately discover any possible height limiting devices in advance is necessary to be employed in modern large or medium sized cars, such as touring cars. Detecting and estimating the height limiting devices act as the key point of a successful height limit alert system. Though there are some works research height limit estimation, existing methods are either too computational expensive or not accurate enough. In this paper, we propose a novel stereo-based pipeline named SHLE for height limit estimation. Our SHLE pipeline consists of two stages. In stage 1, a novel devices detection and tracking scheme is introduced, which accurately locate the height limit devices in the left or right image. Then, in stage 2, the depth is temporally measured, extracted and filtered to calculate the height limit device. To benchmark the height limit estimation task, we build a large-scale dataset named "Disparity Height", where stereo images, pre-computed disparities and ground-truth height limit annotations are provided. We conducted extensive experiments on "Disparity Height" and the results show that SHLE achieves an average error below than 10cm though the car is 70m away from the devices. Our method also outperforms all compared baselines and achieves state-of-the-art performance. Code is available at https://github.com/Yang-Kaixing/SHLE.
Transmitter Selection for Secrecy in Frequency-Selective Fading with Multiple Eavesdroppers and Wireless Backhaul Links
Venue: arXiv cs.IT 2022
Year: 2022

This paper investigates the secrecy performance of sub-optimal transmitter selection (SS) and optimal selection (OS) schemes in a system comprising multiple transmitters, multiple eavesdroppers, and a single destination in frequency-selective fading channels with single carrier cyclic prefix modulation. Considering unreliable backhaul links between the access point and the transmitters, we analyze secrecy performance in two scenarios: when backhaul activity knowledge is available (KA) and when backhaul activity knowledge is unavailable (KU). Closed-form expressions for the secrecy outage probability (SOP) and ergodic secrecy rate (ESR) are derived. We also provide the corresponding asymptotic expressions when the backhaul links are unreliable as well as when the backhaul links are active in both KA and KU scenarios. We show that in the backhaul KA scenario, the asymptotic SOP and the slope of the ESR are governed by the number of transmitters and the backhaul reliability factor. In contrast, only the backhaul reliability factor influences these two parameters in the backhaul KU scenario. These parameters are independent of the number of eavesdroppers in both backhaul KA and KU scenarios. We also observe that the secrecy diversity order of both selection schemes is identical when all backhaul links are active.
Sequential Decision Problems with Weak Feedback
Venue: arXiv cs.LG 2022
Year: 2022

This thesis considers sequential decision problems, where the loss/reward incurred by selecting an action may not be inferred from observed feedback. A major part of this thesis focuses on the unsupervised sequential selection problem, where one can not infer the loss incurred for selecting an action from observed feedback. We also introduce a new setup named Censored Semi Bandits, where the loss incurred for selecting an action can be observed under certain conditions. Finally, we study the channel selection problem in the communication networks, where the reward for an action is only observed when no other player selects that action to play in the round. These problems find applications in many fields like healthcare, crowd-sourcing, security, adaptive resource allocation, among many others. This thesis aims to address the above-described sequential decision problems by exploiting specific structures these problems exhibit. We develop provably optimal algorithms for each of these setups with weak feedback and validate their empirical performance on different problem instances derived from synthetic and real datasets.
Towards Mechanised Proofs in Double-Pushout Graph Transformation
Venue: arXiv cs.LO 2022
Year: 2022

We formalise the basics of the double-pushout approach to graph transformation in the proof assistant Isabelle/HOL and provide associated machine-checked proofs. Specifically, we formalise graphs, graph morphisms and rules, and a definition of direct derivations based on deletion and gluing. We then formalise graph pushouts and prove with Isabelle's help that both deletions and gluings are pushouts. We also prove that pushouts are unique up to isomorphism. The formalisation comprises around 2000 lines of source text. Our motivation is to pave the way for rigorous, machine-checked proofs in the theory of the double-pushout approach, and to lay the foundations for verifying graph transformation systems and rule-based graph programs by interactive theorem proving.
Time and Space Measures for a Complete Graph Computation Model
Venue: arXiv cs.PL 2022
Year: 2022

We present a computation model based on a subclass of GP 2 graph programs which can simulate any off-line Turing machine of space complexity O(s(n) log s(n)) in space O(s(n)). The simulation only requires a quadratic time overhead. Our model shares this property with Schönhage's storage modification machines and Kolmogorov-Uspenskii machines. These machines use low-level pointer instructions whereas our GP 2-based model uses pattern-based transformation rules and high-level control constructs.
Monolith Development History for Microservices Identification: a Comparative Analysis
Venue: arXiv cs.SE 2022
Year: 2022

Recent research has proposed different approaches on the automated identification of candidate microservices on monolith systems, which vary on the monolith representation, similarity criteria, and quality metrics used. On the other hand, they are generally limited in the number of codebases and decompositions evaluated, and few comparisons between approaches exist. Considering the emerging trend in software engineering in techniques based on the analysis of codebases' evolution, we compare a representation based on the monolith code structure, in particular the sequences of accesses to domain entities, with representations based on the monolith development history (file changes and changes authorship). From the analysis on a total of 468k decompositions of 28 codebases, using five quality metrics that evaluate modularity, minimization of the number of transactions per functionality, and reduction of teams and communication, we conclude that the best decompositions on each metric were made by combining data from the sequences of accesses and the development history representations. We also found that the changes authorship representation of codebases with many authors achieves comparable or better results than the sequence of accesses representation of codebases with few authors with respect to minimization of the number of transactions per functionality and the reduction of teams.
Exploring Student Engagement and Outcomes: Experiences from Three Cycles of an Undergraduate Module
Venue: arXiv cs.CY 2022
Year: 2022

Many studies in educational data mining address specific learner groups, such as first-in-family to attend Higher Education, or focus on differences in characteristics such as gender or ethnicity, with the aim of predicting performance and designing interventions to improve outcomes. For Higher Education, this is reflected in significant interest in institutional-level analysis of student cohorts and in tools being promoted to Higher Education Institutions to support collection, integration and analysis of data. For those leading modules/units on degree programmes, however, the reality can be far removed from the seemingly well-supported and increasingly sophisticated approaches advocated in centrally-led data analysis. Module leaders often find themselves working with a number of student-data systems that are not integrated, may contain conflicting data and where significant effort is required to extract, clean and meaningfully analyse the data. This paper suggests that important lessons may be learned from experiences at module level in this context and from subsequent analysis of related data collected across multiple years. The changes made each year are described and a range of data analysis methods are applied, post hoc, to identify findings in relation to the four areas of focus. The key findings are that non-engagement with the Virtual Learning Environment in the first three weeks was the strongest predictor of failure and that early engagement correlated most strongly with final grade. General recommendations are drawn from the findings which should be valuable to module leaders in environments where access to integrated, up-to-date student information remains a day-to-day challenge, and insights will be presented into how such bottom-up activities might inform institutional/top-down planning in the use of relevant technologies.
Blockchain Scalability and Security: Communications Among Fast-Changing Committees Made Simple
Venue: arXiv cs.CR 2022
Year: 2022

For permissionless blockchains, scalability is paramount. While current technologies still fail to address this problem fully, many research works propose sharding or other techniques that extensively adopt parallel processing of transactions. In these approaches, a potentially large number of committees of nodes independently perform consensus and process new transactions. Hence, in addition to regular intra-committee communication, (1) new transactions have to be delivered to the right committee, (2) committees need to communicate to process inter-shard transactions or (3) to exchange intermediate results. To contrast slowly adaptive adversaries, committees should be frequently changed. However, efficient communication to frequently-changing committees is hard. We propose a simple approach that allows us to implicitly select committee members and effectively deliver messages to all members of a specific committee, even when committees are changed frequently. The aim of our design is to provide a committee selection procedure and a committee-targeted communication primitive to be applied in most of the scalable blockchain architectures that are currently proposed in literature. We provide a theoretical proof of the security of our approach and first experimental results that shows that our approach might be feasible in practice.
Model Based Co-clustering of Mixed Numerical and Binary Data
Venue: arXiv cs.LG 2022
Year: 2022

Co-clustering is a data mining technique used to extract the underlying block structure between the rows and columns of a data matrix. Many approaches have been studied and have shown their capacity to extract such structures in continuous, binary or contingency tables. However, very little work has been done to perform co-clustering on mixed type data. In this article, we extend the latent block models based co-clustering to the case of mixed data (continuous and binary variables). We then evaluate the effectiveness of the proposed approach on simulated data and we discuss its advantages and potential limits.
Mind Your Heart: Stealthy Backdoor Attack on Dynamic Deep Neural Network in Edge Computing
Venue: arXiv cs.CR 2022
Year: 2022

Transforming off-the-shelf deep neural network (DNN) models into dynamic multi-exit architectures can achieve inference and transmission efficiency by fragmenting and distributing a large DNN model in edge computing scenarios (e.g., edge devices and cloud servers). In this paper, we propose a novel backdoor attack specifically on the dynamic multi-exit DNN models. Particularly, we inject a backdoor by poisoning one DNN model's shallow hidden layers targeting not this vanilla DNN model but only its dynamically deployed multi-exit architectures. Our backdoored vanilla model behaves normally on performance and cannot be activated even with the correct trigger. However, the backdoor will be activated when the victims acquire this model and transform it into a dynamic multi-exit architecture at their deployment. We conduct extensive experiments to prove the effectiveness of our attack on three structures (ResNet-56, VGG-16, and MobileNet) with four datasets (CIFAR-10, SVHN, GTSRB, and Tiny-ImageNet) and our backdoor is stealthy to evade multiple state-of-the-art backdoor detection or removal methods.
RescureService: A Benchmark Microservice System for the Research of Mobile Edge and Cloud Computing
Venue: arXiv cs.DC 2022
Year: 2022

The dramatic development of cloud and edge computing allows for better Quality of Service (QoS) in many scenarios by deploying services on cloud and edge servers. Microservice technology is also adopted in these scenarios to decompose complex business logic into many small independent services. Meanwhile, as microservice systems continue to grow, providing stable QoS in these systems becomes a challenge, and many different approaches have been proposed for stable QoS. However, the microservice systems used in the experiments of these work have problems such as the low number of services and a single type of service. Therefore, we developed the open-source benchmark microservice system RescureService with 20+ services, including database, front-end, business logic, data processing, and artificial intelligence services in the disaster relief scenario. Measuring tools are provided to measure the service properties to help researchers prepare experimental data, and the service properties pre-measured are also presented. Meanwhile, the fulfillment of benchmark requirements is detailed, and the results show that our RescureService meets the requirements of a benchmark system in research. Moreover, instructions are given to describe adopting our system in service computing as examples.
Keeping Mutation Test Suites Consistent and Relevant with Long-Standing Mutants
Venue: arXiv cs.SE 2022
Year: 2022

Mutation testing has been demonstrated to be one of the most powerful fault-revealing tools in the tester's tool kit. Much previous work implicitly assumed it to be sufficient to re-compute mutant suites per release. Sadly, this makes mutation results inconsistent; mutant scores from each release cannot be directly compared, making it harder to measure test improvement. Furthermore, regular code change means that a mutant suite's relevance will naturally degrade over time. We measure this degradation in relevance for 143,500 mutants in 4 non-trivial systems finding that, on overage, 52% degrade. We introduce a mutant brittleness measure and use it to audit software systems and their mutation suites. We also demonstrate how consistent-by-construction long-standing mutant suites can be identified with a 10x improvement in mutant relevance over an arbitrary test suite. Our results indicate that the research community should avoid the re-computation of mutant suites and focus, instead, on long-standing mutants, thereby improving the consistency and relevance of mutation testing.
Automatically Annotating Indoor Images with CAD Models via RGB-D Scans
Venue: arXiv cs.CV 2022
Year: 2022

We present an automatic method for annotating images of indoor scenes with the CAD models of the objects by relying on RGB-D scans. Through a visual evaluation by 3D experts, we show that our method retrieves annotations that are at least as accurate as manual annotations, and can thus be used as ground truth without the burden of manually annotating 3D data. We do this using an analysis-by-synthesis approach, which compares renderings of the CAD models with the captured scene. We introduce a 'cloning procedure' that identifies objects that have the same geometry, to annotate these objects with the same CAD models. This allows us to obtain complete annotations for the ScanNet dataset and the recent ARKitScenes dataset.
A High-Level Comparison of Recent Technologies for Massive MIMO Architectures
Venue: arXiv cs.IT 2022
Year: 2022

Since the introduction of massive MIMO (mMIMO), the design of a transceiver with feasible complexity has been a challenging problem. Initially, it was believed that the main issue in this respect is the overall RF-cost. However, as mMIMO is becoming more and more a key technology for future wireless networks, it is realized, that the RF-cost is only one of many implementational challenges and design trade-offs. In this paper, we present, analyze and compare various novel mMIMO architectures, considering recent emerging technologies such as intelligent surface-assisted and Rotman lens based architectures. These are compared to the conventional fully digital (FD) and hybrid analog-digital beamforming (HADB) approaches. To enable a fair comparison, we account for various hardware imperfections and losses and utilize a novel, universal algorithm for signal precoding. Based on our thorough investigations, we draw a generic efficiency to quality trade-off for various mMIMO architectures. We find that in a typical cellular communication setting the reflect/transmit array based architectures sketch the best overall trade-off. Further, we show that in a qualitative ranking the power efficiency of the considered architectures is independent of the frequency range.
Multilingual News Location Detection using an Entity-Based Siamese Network with Semi-Supervised Contrastive Learning and Knowledge Base
Venue: arXiv cs.CL 2022
Year: 2022

Early detection of relevant locations in a piece of news is especially important in extreme events such as environmental disasters, war conflicts, disease outbreaks, or political turmoils. Additionally, this detection also helps recommender systems to promote relevant news based on user locations. Note that, when the relevant locations are not mentioned explicitly in the text, state-of-the-art methods typically fail to recognize them because these methods rely on syntactic recognition. In contrast, by incorporating a knowledge base and connecting entities with their locations, our system successfully infers the relevant locations even when they are not mentioned explicitly in the text. To evaluate the effectiveness of our approach, and due to the lack of datasets in this area, we also contribute to the research community with a gold-standard multilingual news-location dataset, NewsLOC. It contains the annotation of the relevant locations (and their WikiData IDs) of 600+ Wikinews articles in five different languages: English, French, German, Italian, and Spanish. Through experimental evaluations, we show that our proposed system outperforms the baselines and the fine-tuned version of the model using semi-supervised data that increases the classification rate. The source code and the NewsLOC dataset are publicly available for being used by the research community at https://github.com/vsuarezpaniagua/NewsLocation.