OpenRaiser/Intern-Atlas huggingface dataset:
Representing Shape Collections with Alignment-Aware Linear Models
Venue: arXiv cs.CV 2021
Year: 2021

In this paper, we revisit the classical representation of 3D point clouds as linear shape models. Our key insight is to leverage deep learning to represent a collection of shapes as affine transformations of low-dimensional linear shape models. Each linear model is characterized by a shape prototype, a low-dimensional shape basis and two neural networks. The networks take as input a point cloud and predict the coordinates of a shape in the linear basis and the affine transformation which best approximate the input. Both linear models and neural networks are learned end-to-end using a single reconstruction loss. The main advantage of our approach is that, in contrast to many recent deep approaches which learn feature-based complex shape representations, our model is explicit and every operation occurs in 3D space. As a result, our linear shape models can be easily visualized and annotated, and failure cases can be visually understood. While our main goal is to introduce a compact and interpretable representation of shape collections, we show it leads to state of the art results for few-shot segmentation.
Privacy Issues in Voice Assistant Ecosystems
Venue: arXiv cs.CR 2021
Year: 2021

Voice assistants have become quite popular lately while in parallel they are an important part of smarthome systems. Through their voice assistants, users can perform various tasks, control other devices and enjoy third party services. The assistants are part of a wider ecosystem. Their function relies on the users voice commands, received through original voice assistant devices or companion applications for smartphones and tablets, which are then sent through the internet to the vendor cloud services and are translated into commands. These commands are then transferred to other applications and services. As this huge volume of data, and mainly personal data of the user, moves around the voice assistant ecosystem, there are several places where personal data is temporarily or permanently stored and thus it is easy for a cyber attacker to tamper with this data, bringing forward major privacy issues. In our work we present the types and location of such personal data artifacts within the ecosystems of three popular voice assistants, after having set up our own testbed, and using IoT forensic procedures. Our privacy evaluation includes the companion apps of the assistants, as we also compare the permissions they require before their installation on an Android device.
Multimodal Detection of COVID-19 Symptoms using Deep Learning & Probability-based Weighting of Modes
Venue: arXiv cs.LG 2021
Year: 2021

The COVID-19 pandemic is one of the most challenging healthcare crises during the 21st century. As the virus continues to spread on a global scale, the majority of efforts have been on the development of vaccines and the mass immunization of the public. While the daily case numbers were following a decreasing trend, the emergent of new virus mutations and variants still pose a significant threat. As economies start recovering and societies start opening up with people going back into office buildings, schools, and malls, we still need to have the ability to detect and minimize the spread of COVID-19. Individuals with COVID-19 may show multiple symptoms such as cough, fever, and shortness of breath. Many of the existing detection techniques focus on symptoms having the same equal importance. However, it has been shown that some symptoms are more prevalent than others. In this paper, we present a multimodal method to predict COVID-19 by incorporating existing deep learning classifiers using convolutional neural networks and our novel probability-based weighting function that considers the prevalence of each symptom. The experiments were performed on an existing dataset with respect to the three considered modes of coughs, fever, and shortness of breath. The results show considerable improvements in the detection of COVID-19 using our weighting function when compared to an equal weighting function.
A Novel IoT Sensor Authentication Using HaLo Extraction Method and Memory Chip Variability
Venue: arXiv cs.CR 2021
Year: 2021

In this paper, we propose flash-based hardware security primitives as a viable solution to meet the security challenges of the IoT and specifically telehealth markets. We have created a novel solution, called the High and Low (HaLo) method, that generates physical unclonable function (PUF) signatures based on process variations within flash memory in order to uniquely identify and authenticate remote sensors. The HaLo method consumes 60% less power than conventional authentication schemes, has an average latency of only 39ms for signature generation, and can be readily implemented through firmware on ONFI 2.2 compliant off-the-shelf NAND flash memory chips. The HaLo method generates 512 bit signatures with an average error rate of 5.9 * 10^-4, while also adapting for flash chip aging. Due to its low latency, low error rate, and high power efficiency, the HaLo method could help progress the field of remote patient monitoring by accurately and efficiently authenticating remote health sensors.
Conceptualising Cloud Migration Lifecycle
Venue: arXiv cs.SE 2021
Year: 2021

Many enterprise software systems supporting IT services are characterised by a need for a high computing capability and resource consumption (Armbrust et al. 2010; Buyya et al. 2008; Koçak et al. 2013). Cloud Computing initiatives have received a significant attention as a viable solution to address these requirements through offering a wide range of services, which are universally accessible, acquirable and releasable in a dynamic fashion, and payable on the basis of service usage. Hence, organisations view the cloud services as an opportunity to empower their legacy systems.
Hybrid Contrastive Learning of Tri-Modal Representation for Multimodal Sentiment Analysis
Venue: arXiv cs.AI 2021
Year: 2021

The wide application of smart devices enables the availability of multimodal data, which can be utilized in many tasks. In the field of multimodal sentiment analysis (MSA), most previous works focus on exploring intra- and inter-modal interactions. However, training a network with cross-modal information (language, visual, audio) is still challenging due to the modality gap, and existing methods still cannot ensure to sufficiently learn intra-/inter-modal dynamics. Besides, while learning dynamics within each sample draws great attention, the learning of inter-class relationships is neglected. Moreover, the size of datasets limits the generalization ability of existing methods. To address the afore-mentioned issues, we propose a novel framework HyCon for hybrid contrastive learning of tri-modal representation. Specifically, we simultaneously perform intra-/inter-modal contrastive learning and semi-contrastive learning (that is why we call it hybrid contrastive learning), with which the model can fully explore cross-modal interactions, preserve inter-class relationships and reduce the modality gap. Besides, a refinement term is devised to prevent the model falling into a sub-optimal solution. Moreover, HyCon can naturally generate a large amount of training pairs for better generalization and reduce the negative effect of limited datasets. Extensive experiments on public datasets demonstrate that our proposed method outperforms existing works.
A Comprehensive Approach for UAV Small Object Detection with Simulation-based Transfer Learning and Adaptive Fusion
Venue: arXiv cs.CV 2021
Year: 2021

Precisely detection of Unmanned Aerial Vehicles(UAVs) plays a critical role in UAV defense systems. Deep learning is widely adopted for UAV object detection whereas researches on this topic are limited by the amount of dataset and small scale of UAV. To tackle these problems, a novel comprehensive approach that combines transfer learning based on simulation data and adaptive fusion is proposed. Firstly, the open-source plugin AirSim proposed by Microsoft is used to generate mass realistic simulation data. Secondly, transfer learning is applied to obtain a pre-trained YOLOv5 model on the simulated dataset and fine-tuned model on the real-world dataset. Finally, an adaptive fusion mechanism is proposed to further improve small object detection performance. Experiment results demonstrate the effectiveness of simulation-based transfer learning which leads to a 2.7% performance increase on UAV object detection. Furthermore, with transfer learning and adaptive fusion mechanism, 7.1% improvement is achieved compared to the original YOLO v5 model.
X-Driven Methodologies for SOA System Development -- A Survey
Venue: arXiv cs.SE 2021
Year: 2021

This study aims to evaluate four service-oriented architecture (SOA) system software development methodologies: business-driven development, model-driven development, event-driven development, and domain-driven development. These methods, generically labelled as x-driven methodologies (XDMs), are commonly used in a general software development context, but software architects can also apply them in an SOA-based system. Each XDM typically focus on a specific aspect that drives its processes and steps. This aspect is indicated by its label. An evaluation method called qualitative screening mode is used in this study. XDMs are analysed based on their features to determine the suitability or support for service-oriented solutions. Criteria used to appraise each method are taken from SOA characteristics and SOA manifesto points. Of the four discussed XDMs, business-driven development is the best-suited approach to implement a service-oriented system shown by its conformity with the selected assessment criteria. Nevertheless, the other three XDMs have also their own strengths. Model-driven development is excellent for productivity, event-driven development is preferential for a quick response and asynchronous work, while domain-driven development is distinctive to describe problems precisely. The originality of this research is in the assessment general software development approaches of XDMs to be applied to SOA approach. The results can help developers in considering suitable methods to construct a prospective software system. Previous studies only investigate on methodologies designed intentionally for service-oriented systems.
Representation Learning for Efficient and Effective Similarity Search and Recommendation
Venue: arXiv cs.IR 2021
Year: 2021

How data is represented and operationalized is critical for building computational solutions that are both effective and efficient. A common approach is to represent data objects as binary vectors, denoted \textit{hash codes}, which require little storage and enable efficient similarity search through direct indexing into a hash table or through similarity computations in an appropriate space. Due to the limited expressibility of hash codes, compared to real-valued representations, a core open challenge is how to generate hash codes that well capture semantic content or latent properties using a small number of bits, while ensuring that the hash codes are distributed in a way that does not reduce their search efficiency. State of the art methods use representation learning for generating such hash codes, focusing on neural autoencoder architectures where semantics are encoded into the hash codes by learning to reconstruct the original inputs of the hash codes. This thesis addresses the above challenge and makes a number of contributions to representation learning that (i) improve effectiveness of hash codes through more expressive representations and a more effective similarity measure than the current state of the art, namely the Hamming distance, and (ii) improve efficiency of hash codes by learning representations that are especially suited to the choice of search method. The contributions are empirically validated on several tasks related to similarity search and recommendation.
GOHOME: Graph-Oriented Heatmap Output for future Motion Estimation
Venue: arXiv cs.CV 2021
Year: 2021

In this paper, we propose GOHOME, a method leveraging graph representations of the High Definition Map and sparse projections to generate a heatmap output representing the future position probability distribution for a given agent in a traffic scene. This heatmap output yields an unconstrained 2D grid representation of agent future possible locations, allowing inherent multimodality and a measure of the uncertainty of the prediction. Our graph-oriented model avoids the high computation burden of representing the surrounding context as squared images and processing it with classical CNNs, but focuses instead only on the most probable lanes where the agent could end up in the immediate future. GOHOME reaches 2$nd$ on Argoverse Motion Forecasting Benchmark on the MissRate$_6$ metric while achieving significant speed-up and memory burden diminution compared to Argoverse 1$^{st}$ place method HOME. We also highlight that heatmap output enables multimodal ensembling and improve 1$^{st}$ place MissRate$_6$ by more than 15$\%$ with our best ensemble on Argoverse. Finally, we evaluate and reach state-of-the-art performance on the other trajectory prediction datasets nuScenes and Interaction, demonstrating the generalizability of our method.
A Case Study of Agent-Based Models for Evolutionary Game Theory
Venue: arXiv cs.GT 2021
Year: 2021

Evolutionary game theory is a mathematical toolkit to analyse the interactions that an individual agent has in a population and how the composition of strategies in this population evolves over time. While it can provide neat solutions to simple problems, in more complicated situations where assumptions such as infinite population size may be relaxed, deriving analytic solutions can be intractable. In this short paper, we present a game with complex interactions and examine how an agent-based model may be used as a heuristic technique to find evolutionarily stable states.
Direct Construction of Program Alignment Automata for Equivalence Checking
Venue: arXiv cs.SE 2021
Year: 2021

The problem of checking whether two programs are semantically equivalent or not has a diverse range of applications, and is consequently of substantial importance. There are several techniques that address this problem, chiefly by constructing a product program that makes it easier to derive useful invariants. A novel addition to these is a technique that uses alignment predicates to align traces of the two programs, in order to construct a program alignment automaton. Being guided by predicates is not just beneficial in dealing with syntactic dissimilarities, but also in staying relevant to the property. However, there are also drawbacks of a trace-based technique. Obtaining traces that cover all program behaviors is difficult, and any under-approximation may lead to an incomplete product program. Moreover, an indirect construction of this kind is unaware of the missing behaviors, and has no control over the aforesaid incompleteness. This paper, addressing these concerns, presents an algorithm to construct the program alignment automaton directly instead of relying on traces.
Sparse Spatial Attention Network for Semantic Segmentation
Venue: arXiv cs.CV 2021
Year: 2021

The spatial attention mechanism captures long-range dependencies by aggregating global contextual information to each query location, which is beneficial for semantic segmentation. In this paper, we present a sparse spatial attention network (SSANet) to improve the efficiency of the spatial attention mechanism without sacrificing the performance. Specifically, a sparse non-local (SNL) block is proposed to sample a subset of key and value elements for each query element to capture long-range relations adaptively and generate a sparse affinity matrix to aggregate contextual information efficiently. Experimental results show that the proposed approach outperforms other context aggregation methods and achieves state-of-the-art performance on the Cityscapes, PASCAL Context and ADE20K datasets.
Audio-Visual Transformer Based Crowd Counting
Venue: arXiv cs.CV 2021
Year: 2021

Crowd estimation is a very challenging problem. The most recent study tries to exploit auditory information to aid the visual models, however, the performance is limited due to the lack of an effective approach for feature extraction and integration. The paper proposes a new audiovisual multi-task network to address the critical challenges in crowd counting by effectively utilizing both visual and audio inputs for better modalities association and productive feature extraction. The proposed network introduces the notion of auxiliary and explicit image patch-importance ranking (PIR) and patch-wise crowd estimate (PCE) information to produce a third (run-time) modality. These modalities (audio, visual, run-time) undergo a transformer-inspired cross-modality co-attention mechanism to finally output the crowd estimate. To acquire rich visual features, we propose a multi-branch structure with transformer-style fusion in-between. Extensive experimental evaluations show that the proposed scheme outperforms the state-of-the-art networks under all evaluation settings with up to 33.8% improvement. We also analyze and compare the vision-only variant of our network and empirically demonstrate its superiority over previous approaches.
Iterative Threshold Decoding of Spatially Coupled, Parallel-Concatenated Codes
Venue: arXiv cs.IT 2021
Year: 2021

Spatially coupled, parallel concatenated codes (SC-PCCs) have been shown to approach channel capacity when decoded using optimal iterative methods. However, under complexity constraints such decoding strategies can result in unacceptable power and latency costs. In this work, we employ convolutional self-orthogonal component codes along with low-complexity, suboptimal a posteriori probability (APP) threshold decoders with SC-PCCs to reduce decoding complexity. The proposed code design is faster, more energy efficient, and easier to implement than optimal methods, while offering significant coding gain over existing threshold decodable, turbo-like constructions of similar latency and complexity. The design also serves to further illustrate the advantages spatial coupling can provide to existing code constructions and decoder implementations.
Post-Quantum VRF and its Applications in Future-Proof Blockchain System
Venue: arXiv cs.CR 2021
Year: 2021

A verifiable random function (VRF in short) is a powerful pseudo-random function that provides a non-interactively public verifiable proof for the correctness of its output. Recently, VRFs have found essential applications in blockchain design, such as random beacons and proof-of-stake consensus protocols. To our knowledge, the first generation of blockchain systems used inherently inefficient proof-of-work consensuses, and the research community tried to achieve the same properties by proposing proof-of-stake schemes where resource-intensive proof-of-work is emulated by cryptographic constructions. Unfortunately, those most discussed proof-of-stake consensuses (e.g., Algorand and Ouroborous family) are not future-proof because the building blocks are secure only under the classical hard assumptions; in particular, their designs ignore the advent of quantum computing and its implications. In this paper, we propose a generic compiler to obtain the post-quantum VRF from the simple VRF solution using symmetric-key primitives (e.g., non-interactive zero-knowledge system) with an intrinsic property of quantum-secure. Our novel solution is realized via two efficient zero-knowledge systems ZKBoo and ZKB++, respectively, to validate the compiler correctness. Our proof-of-concept implementation indicates that even today, the overheads introduced by our solution are acceptable in real-world deployments. We also demonstrate potential applications of a quantum-secure VRF, such as quantum-secure decentralized random beacon and lottery-based proof of stake consensus blockchain protocol.
Tolerating Adversarial Attacks and Byzantine Faults in Distributed Machine Learning
Venue: arXiv cs.CR 2021
Year: 2021

Adversarial attacks attempt to disrupt the training, retraining and utilizing of artificial intelligence and machine learning models in large-scale distributed machine learning systems. This causes security risks on its prediction outcome. For example, attackers attempt to poison the model by either presenting inaccurate misrepresentative data or altering the models' parameters. In addition, Byzantine faults including software, hardware, network issues occur in distributed systems which also lead to a negative impact on the prediction outcome. In this paper, we propose a novel distributed training algorithm, partial synchronous stochastic gradient descent (ParSGD), which defends adversarial attacks and/or tolerates Byzantine faults. We demonstrate the effectiveness of our algorithm under three common adversarial attacks again the ML models and a Byzantine fault during the training phase. Our results show that using ParSGD, ML models can still produce accurate predictions as if it is not being attacked nor having failures at all when almost half of the nodes are being compromised or failed. We will report the experimental evaluations of ParSGD in comparison with other algorithms.
GTG-Shapley: Efficient and Accurate Participant Contribution Evaluation in Federated Learning
Venue: arXiv cs.AI 2021
Year: 2021

Federated Learning (FL) bridges the gap between collaborative machine learning and preserving data privacy. To sustain the long-term operation of an FL ecosystem, it is important to attract high quality data owners with appropriate incentive schemes. As an important building block of such incentive schemes, it is essential to fairly evaluate participants' contribution to the performance of the final FL model without exposing their private data. Shapley Value (SV)-based techniques have been widely adopted to provide fair evaluation of FL participant contributions. However, existing approaches incur significant computation costs, making them difficult to apply in practice. In this paper, we propose the Guided Truncation Gradient Shapley (GTG-Shapley) approach to address this challenge. It reconstructs FL models from gradient updates for SV calculation instead of repeatedly training with different combinations of FL participants. In addition, we design a guided Monte Carlo sampling approach combined with within-round and between-round truncation to further reduce the number of model reconstructions and evaluations required, through extensive experiments under diverse realistic data distribution settings. The results demonstrate that GTG-Shapley can closely approximate actual Shapley values, while significantly increasing computational efficiency compared to the state of the art, especially under non-i.i.d. settings.