OpenRaiser/Intern-Atlas huggingface dataset:
Automating the Communication of Cybersecurity Knowledge: Multi-Case Study
Venue: arXiv cs.CY 2021
Year: 2021

Cybersecurity is essential for the protection of companies against cyber threats. Traditionally, cybersecurity experts assess and improve a company's capabilities. However, many small and medium-sized businesses (SMBs) consider such services not to be affordable. We explore an alternative do-it-yourself (DIY) approach to bringing cybersecurity to SMBs. Our method and tool, CYSEC, implements the Self-Determination Theory (SDT) to guide and motivate SMBs to adopt good cybersecurity practices. CYSEC uses assessment questions and recommendations to communicate cybersecurity knowledge to the end-user SMBs and encourage self-motivated change. In this paper, the operationalisation of SDT in CYSEC is presented and the results of a multi-case study shown that offer insight into how SMBs adopted cybersecurity practices with CYSEC. Effective automated cybersecurity communication depended on the SMB's hands-on skills, tools adaptedness, and the users' willingness to documenting confidential information. The SMBs wanted to learn in simple, incremental steps, allowing them to understand what they do. An SMB's motivation to improve security depended on the fitness of assessment questions and recommendations with the SMB's business model and IT infrastructure. The results of this study indicate that automated counselling can help many SMBs in security adoption. The final publication is available at Springer via https://link.springer.com/chapter/10.1007%2F978-3-030-59291-2_8
A family of metrics from the truncated smoothing of Reeb graphs
Venue: arXiv cs.CG 2021
Year: 2021

In this paper, we introduce an extension of smoothing on Reeb graphs, which we call truncated smoothing; this in turn allows us to define a new family of metrics which generalize the interleaving distance for Reeb graphs. Intuitively, we "chop off" parts near local minima and maxima during the course of smoothing, where the amount cut is controlled by a parameter $τ$. After formalizing truncation as a functor, we show that when applied after the smoothing functor, this prevents extensive expansion of the range of the function, and yields particularly nice properties (such as maintaining connectivity) when combined with smoothing for $0 \leq τ\leq 2\varepsilon$, where $\varepsilon$ is the smoothing parameter. Then, for the restriction of $τ\in [0,\varepsilon]$, we have additional structure which we can take advantage of to construct a categorical flow for any choice of slope $m \in [0,1]$. Using the infrastructure built for a category with a flow, this then gives an interleaving distance for every $m \in [0,1]$, which is a generalization of the original interleaving distance, which is the case $m=0$. While the resulting metrics are not stable, we show that any pair of these for $m,m' \in [0,1)$ are strongly equivalent metrics, which in turn gives stability of each metric up to a multiplicative constant. We conclude by discussing implications of this metric within the broader family of metrics for Reeb graphs.
Leveraging Category Information for Single-Frame Visual Sound Source Separation
Venue: arXiv cs.CV 2021
Year: 2021

Visual sound source separation aims at identifying sound components from a given sound mixture with the presence of visual cues. Prior works have demonstrated impressive results, but with the expense of large multi-stage architectures and complex data representations (e.g. optical flow trajectories). In contrast, we study simple yet efficient models for visual sound separation using only a single video frame. Furthermore, our models are able to exploit the information of the sound source category in the separation process. To this end, we propose two models where we assume that i) the category labels are available at the training time, or ii) we know if the training sample pairs are from the same or different category. The experiments with the MUSIC dataset show that our model obtains comparable or better performance compared to several recent baseline methods. The code is available at https://github.com/ly-zhu/Leveraging-Category-Information-for-Single-Frame-Visual-Sound-Source-Separation
GRADE: Graph Dynamic Embedding
Venue: arXiv cs.LG 2021
Year: 2021

Representation learning of static and more recently dynamically evolving graphs has gained noticeable attention. Existing approaches for modelling graph dynamics focus extensively on the evolution of individual nodes independently of the evolution of mesoscale community structures. As a result, current methods do not provide useful tools to study and cannot explicitly capture temporal community dynamics. To address this challenge, we propose GRADE - a probabilistic model that learns to generate evolving node and community representations by imposing a random walk prior over their trajectories. Our model also learns node community membership which is updated between time steps via a transition matrix. At each time step link generation is performed by first assigning node membership from a distribution over the communities, and then sampling a neighbor from a distribution over the nodes for the assigned community. We parametrize the node and community distributions with neural networks and learn their parameters via variational inference. Experiments demonstrate GRADE outperforms baselines in dynamic link prediction, shows favourable performance on dynamic community detection, and identifies coherent and interpretable evolving communities.
Unseen Object Instance Segmentation for Robotic Environments
Venue: arXiv cs.CV 2021
Year: 2021

In order to function in unstructured environments, robots need the ability to recognize unseen objects. We take a step in this direction by tackling the problem of segmenting unseen object instances in tabletop environments. However, the type of large-scale real-world dataset required for this task typically does not exist for most robotic settings, which motivates the use of synthetic data. Our proposed method, UOIS-Net, separately leverages synthetic RGB and synthetic depth for unseen object instance segmentation. UOIS-Net is comprised of two stages: first, it operates only on depth to produce object instance center votes in 2D or 3D and assembles them into rough initial masks. Secondly, these initial masks are refined using RGB. Surprisingly, our framework is able to learn from synthetic RGB-D data where the RGB is non-photorealistic. To train our method, we introduce a large-scale synthetic dataset of random objects on tabletops. We show that our method can produce sharp and accurate segmentation masks, outperforming state-of-the-art methods on unseen object instance segmentation. We also show that our method can segment unseen objects for robot grasping.
Data-driven effective model shows a liquid-like deep learning
Venue: arXiv cs.LG 2021
Year: 2021

The geometric structure of an optimization landscape is argued to be fundamentally important to support the success of deep neural network learning. A direct computation of the landscape beyond two layers is hard. Therefore, to capture the global view of the landscape, an interpretable model of the network-parameter (or weight) space must be established. However, the model is lacking so far. Furthermore, it remains unknown what the landscape looks like for deep networks of binary synapses, which plays a key role in robust and energy efficient neuromorphic computation. Here, we propose a statistical mechanics framework by directly building a least structured model of the high-dimensional weight space, considering realistic structured data, stochastic gradient descent training, and the computational depth of neural networks. We also consider whether the number of network parameters outnumbers the number of supplied training data, namely, over- or under-parametrization. Our least structured model reveals that the weight spaces of the under-parametrization and over-parameterization cases belong to the same class, in the sense that these weight spaces are well-connected without any hierarchical clustering structure. In contrast, the shallow-network has a broken weight space, characterized by a discontinuous phase transition, thereby clarifying the benefit of depth in deep learning from the angle of high dimensional geometry. Our effective model also reveals that inside a deep network, there exists a liquid-like central part of the architecture in the sense that the weights in this part behave as randomly as possible, providing algorithmic implications. Our data-driven model thus provides a statistical mechanics insight about why deep learning is unreasonably effective in terms of the high-dimensional weight space, and how deep networks are different from shallow ones.
Community aware group testing
Venue: arXiv cs.IT 2021
Year: 2021

In this paper, we propose algorithms that leverage a known community structure to make group testing more efficient. We consider a population organized in disjoint communities: each individual participates in a community, and its infection probability depends on the community (s)he participates in. Use cases include families, students who participate in several classes, and workers who share common spaces. Group testing reduces the number of tests needed to identify the infected individuals by pooling diagnostic samples and testing them together. We show that if we design the testing strategy taking into account the community structure, we can significantly reduce the number of tests needed for adaptive and non-adaptive group testing, and can improve the reliability in cases where tests are noisy.
Robust adaptive steganography based on dither modulation and modification with re-compression
Venue: arXiv cs.MM 2021
Year: 2021

Traditional adaptive steganography is a technique used for covert communication with high security, but it is invalid in the case of stego images are sent to legal receivers over networks which is lossy, such as JPEG compression of channels. To deal with such problem, robust adaptive steganography is proposed to enable the receiver to extract secret messages from the damaged stego images. Previous works utilize reverse engineering and compression-resistant domain constructing to implement robust adaptive steganography. In this paper, we adopt modification with re-compression scheme to improve the robustness of stego sequences in stego images. To balance security and robustness, we move the embedding domain to the low frequency region of DCT (Discrete Cosine Transform) coefficients to improve the security of robust adaptive steganography. In addition, we add additional check codes to further reduce the average extraction error rate based on the framework of E-DMAS (Enhancing Dither Modulation based robust Adaptive Steganography). Compared with GMAS (Generalized dither Modulation based robust Adaptive Steganography) and E-DMAS, experiment results show that our scheme can achieve strong robustness and improve the security of robust adaptive steganography greatly when the channel quality factor is known.
FADACS: A Few-shot Adversarial Domain Adaptation Architecture for Context-Aware Parking Availability Sensing
Venue: arXiv cs.LG 2021
Year: 2021

Existing research on parking availability sensing mainly relies on extensive contextual and historical information. In practice, the availability of such information is a challenge as it requires continuous collection of sensory signals. In this study, we design an end-to-end transfer learning framework for parking availability sensing to predict parking occupancy in areas in which the parking data is insufficient to feed into data-hungry models. This framework overcomes two main challenges: 1) many real-world cases cannot provide enough data for most existing data-driven models, and 2) it is difficult to merge sensor data and heterogeneous contextual information due to the differing urban fabric and spatial characteristics. Our work adopts a widely-used concept, adversarial domain adaptation, to predict the parking occupancy in an area without abundant sensor data by leveraging data from other areas with similar features. In this paper, we utilise more than 35 million parking data records from sensors placed in two different cities, one a city centre and the other a coastal tourist town. We also utilise heterogeneous spatio-temporal contextual information from external resources, including weather and points of interest. We quantify the strength of our proposed framework in different cases and compare it to the existing data-driven approaches. The results show that the proposed framework is comparable to existing state-of-the-art methods and also provide some valuable insights on parking availability prediction.
Tractability Beyond $β$-Acyclicity for Conjunctive Queries with Negation
Venue: arXiv cs.DB 2021
Year: 2021

Numerous fundamental database and reasoning problems are known to be NP-hard in general but tractable on instances where the underlying hypergraph structure is $β$-acyclic. Despite the importance of many of these problems, there has been little success in generalizing these results beyond acyclicity. In this paper, we take on this challenge and propose nest-set width, a novel generalization of hypergraph $β$-acyclicity. We demonstrate that nest-set width has desirable properties and algorithmic significance. In particular, evaluation of boolean conjunctive queries with negation is tractable for classes with bounded nest-set width. Furthermore, propositional satisfiability is fixed-parameter tractable when parameterized by nest-set width.
Self-Supervised Learning of Context-Aware Pitch Prosody Representations
Venue: arXiv cs.SD 2021
Year: 2021

In music and speech, meaning is derived at multiple levels of context. Affect, for example, can be inferred both by a short sound token and by sonic patterns over a longer temporal window such as an entire recording. In this letter, we focus on inferring meaning from this dichotomy of contexts. We show how contextual representations of short sung vocal lines can be implicitly learned from fundamental frequency ($F_0$) and thus be used as a meaningful feature space for downstream Music Information Retrieval (MIR) tasks. We propose three self-supervised deep learning paradigms which leverage pseudotask learning of these two levels of context to produce latent representation spaces. We evaluate the usefulness of these representations by embedding unseen pitch contours into each space and conducting downstream classification tasks. Our results show that contextual representation can enhance downstream classification by as much as 15\% as compared to using traditional statistical contour features.
Phase-Noise Compensation for OFDM Systems Exploiting Coherence Bandwidth: Modeling, Algorithms, and Analysis
Venue: arXiv cs.IT 2021
Year: 2021

Phase-noise (PN) estimation and compensation are crucial in millimeter-wave (mmWave) communication systems to achieve high reliability. The PN estimation, however, suffers from high computational complexity due to its fundamental characteristics, such as spectral spreading and fast-varying fluctuations. In this paper, we propose a new framework for low-complexity PN compensation in orthogonal frequency-division multiplexing systems. The proposed framework also includes a pilot allocation strategy to minimize its overhead. The key ideas are to exploit the coherence bandwidth of mmWave systems and to approximate the actual PN spectrum with its dominant components, resulting in a non-iterative solution by using linear minimum mean squared-error estimation. The proposed method obtains a reduction of more than 2.5x in total complexity, as compared to the existing methods. Furthermore, we derive closed-form expressions for normalized mean squared-errors (NMSEs) as a function of critical system parameters, which help in understanding the NMSE behavior in low and high signal-to-noise ratio regimes. Lastly, we study a trade-off between performance and pilot-overhead to provide insight into an appropriate approximation of the PN spectrum.
Modern Random Access: an Age of Information Perspective on Irregular Repetition Slotted ALOHA
Venue: arXiv cs.NI 2021
Year: 2021

Age of information (AoI) is gaining attention as a valuable performance metric for many IoT systems, in which a large number of devices report time-stamped updates to a central gateway. This is the case, for instance, of remote sensing, monitoring, or tracking, with broad applications in the industrial, vehicular, and environmental domain. In these settings, AoI provides insights that are complementary to those offered by throughput or latency, capturing the ability of the system to maintain an up-to-date view of the status of each transmitting device. From this standpoint, while a good understanding of the metric has been reached for point-to-point links, relatively little attention has been devoted to the impact that link layer solutions employed in IoT systems may have on AoI. In particular, no result is available for modern random access protocols, which have recently emerged as promising solutions to support massive machine-type communications. To start addressing this gap we provide in this paper the first study of the AoI of a scheme in this family, namely irregular repetition slotted ALOHA (IRSA). By means of a Markovian analysis, we track the AoI evolution at the gateway, prove that the process is ergodic, and derive a compact closed form expression for its stationary distribution. Leaning on this, we compute exact formulations for the average AoI and the age violation probability. The study reveals non-trivial design trade-offs for IRSA and highlights the key role played by the protocol operating frame size. Moreover, a comparison with the performance of a simpler slotted ALOHA strategy highlights a remarkable potential for modern random access schemes in terms of information freshness.
NPCFace: Negative-Positive Collaborative Training for Large-scale Face Recognition
Venue: arXiv cs.CV 2021
Year: 2021

The training scheme of deep face recognition has greatly evolved in the past years, yet it encounters new challenges in the large-scale data situation where massive and diverse hard cases occur. Especially in the range of low false accept rate (FAR), there are various hard cases in both positives (intra-class) and negatives (inter-class). In this paper, we study how to make better use of these hard samples for improving the training. The literature approaches this by margin-based formulation in either positive logit or negative logits. However, the correlation between hard positive and hard negative is overlooked, and so is the relation between the margins in positive and negative logits. We find such correlation is significant, especially in the large-scale dataset, and one can take advantage from it to boost the training via relating the positive and negative margins for each training sample. To this end, we propose an explicit collaboration between positive and negative margins sample-wisely. Given a batch of hard samples, a novel Negative-Positive Collaboration loss, named NPCFace, is formulated, which emphasizes the training on both negative and positive hard cases via the collaborative-margin mechanism in the softmax logits, and also brings better interpretation of negative-positive hardness correlation. Besides, the emphasis is implemented with an improved formulation to achieve stable convergence and flexible parameter setting. We validate the effectiveness of our approach on various benchmarks of large-scale face recognition, and obtain advantageous results especially in the low FAR range.
Best Viewpoints for External Robots or Sensors Assisting Other Robots
Venue: arXiv cs.RO 2021
Year: 2021

This work creates a model of the value of different external viewpoints of a robot performing tasks. The current state of the practice is to use a teleoperated assistant robot to provide a view of a task being performed by a primary robot; however, the choice of viewpoints is ad hoc and does not always lead to improved performance. This research applies a psychomotor approach to develop a model of the relative quality of external viewpoints using Gibsonian affordances. In this approach, viewpoints for the affordances are rated based on the psychomotor behavior of human operators and clustered into manifolds of viewpoints with the equivalent value. The value of 30 viewpoints is quantified in a study with 31 expert robot operators for 4 affordances (Reachability, Passability, Manipulability, and Traversability) using a computer-based simulator of two robots. The adjacent viewpoints with similar values are clustered into ranked manifolds using agglomerative hierarchical clustering. The results show the validity of the affordance-based approach by confirming that there are manifolds of statistically significantly different viewpoint values, viewpoint values are statistically significantly dependent on the affordances, and viewpoint values are independent of a robot. Furthermore, the best manifold for each affordance provides a statistically significant improvement with a large Cohen's d effect size (1.1-2.3) in performance (improving time by 14%-59% and reducing errors by 87%-100%) and improvement in performance variation over the worst manifold. This model will enable autonomous selection of the best possible viewpoint and path planning for the assistant robot.
The Effects of Approximate Multiplication on Convolutional Neural Networks
Venue: arXiv cs.LG 2021
Year: 2021

This paper analyzes the effects of approximate multiplication when performing inferences on deep convolutional neural networks (CNNs). The approximate multiplication can reduce the cost of the underlying circuits so that CNN inferences can be performed more efficiently in hardware accelerators. The study identifies the critical factors in the convolution, fully-connected, and batch normalization layers that allow more accurate CNN predictions despite the errors from approximate multiplication. The same factors also provide an arithmetic explanation of why bfloat16 multiplication performs well on CNNs. The experiments are performed with recognized network architectures to show that the approximate multipliers can produce predictions that are nearly as accurate as the FP32 references, without additional training. For example, the ResNet and Inception-v4 models with Mitch-$w$6 multiplication produces Top-5 errors that are within 0.2% compared to the FP32 references. A brief cost comparison of Mitch-$w$6 against bfloat16 is presented, where a MAC operation saves up to 80% of energy compared to the bfloat16 arithmetic. The most far-reaching contribution of this paper is the analytical justification that multiplications can be approximated while additions need to be exact in CNN MAC operations.
Navigating the Trade-Off between Multi-Task Learning and Learning to Multitask in Deep Neural Networks
Venue: arXiv cs.LG 2021
Year: 2021

The terms multi-task learning and multitasking are easily confused. Multi-task learning refers to a paradigm in machine learning in which a network is trained on various related tasks to facilitate the acquisition of tasks. In contrast, multitasking is used to indicate, especially in the cognitive science literature, the ability to execute multiple tasks simultaneously. While multi-task learning exploits the discovery of common structure between tasks in the form of shared representations, multitasking is promoted by separating representations between tasks to avoid processing interference. Here, we build on previous work involving shallow networks and simple task settings suggesting that there is a trade-off between multi-task learning and multitasking, mediated by the use of shared versus separated representations. We show that the same tension arises in deep networks and discuss a meta-learning algorithm for an agent to manage this trade-off in an unfamiliar environment. We display through different experiments that the agent is able to successfully optimize its training strategy as a function of the environment.