OpenRaiser/Intern-Atlas huggingface dataset:
PPMN: Pixel-Phrase Matching Network for One-Stage Panoptic Narrative Grounding
Venue: arXiv cs.CV 2022
Year: 2022

Panoptic Narrative Grounding (PNG) is an emerging task whose goal is to segment visual objects of things and stuff categories described by dense narrative captions of a still image. The previous two-stage approach first extracts segmentation region proposals by an off-the-shelf panoptic segmentation model, then conducts coarse region-phrase matching to ground the candidate regions for each noun phrase. However, the two-stage pipeline usually suffers from the performance limitation of low-quality proposals in the first stage and the loss of spatial details caused by region feature pooling, as well as complicated strategies designed for things and stuff categories separately. To alleviate these drawbacks, we propose a one-stage end-to-end Pixel-Phrase Matching Network (PPMN), which directly matches each phrase to its corresponding pixels instead of region proposals and outputs panoptic segmentation by simple combination. Thus, our model can exploit sufficient and finer cross-modal semantic correspondence from the supervision of densely annotated pixel-phrase pairs rather than sparse region-phrase pairs. In addition, we also propose a Language-Compatible Pixel Aggregation (LCPA) module to further enhance the discriminative ability of phrase features through multi-round refinement, which selects the most compatible pixels for each phrase to adaptively aggregate the corresponding visual context. Extensive experiments show that our method achieves new state-of-the-art performance on the PNG benchmark with 4.0 absolute Average Recall gains.
Searching for chromate replacements using natural language processing and machine learning algorithms
Venue: arXiv cs.CL 2022
Year: 2022

The past few years has seen the application of machine learning utilised in the exploration of new materials. As in many fields of research - the vast majority of knowledge is published as text, which poses challenges in either a consolidated or statistical analysis across studies and reports. Such challenges include the inability to extract quantitative information, and in accessing the breadth of non-numerical information. To address this issue, the application of natural language processing (NLP) has been explored in several studies to date. In NLP, assignment of high-dimensional vectors, known as embeddings, to passages of text preserves the syntactic and semantic relationship between words. Embeddings rely on machine learning algorithms and in the present work, we have employed the Word2Vec model, previously explored by others, and the BERT model - applying them towards a unique challenge in materials engineering. That challenge is the search for chromate replacements in the field of corrosion protection. From a database of over 80 million records, a down-selection of 5990 papers focused on the topic of corrosion protection were examined using NLP. This study demonstrates it is possible to extract knowledge from the automated interpretation of the scientific literature and achieve expert human level insights.
Using Infant Limb Movement Data to Control Small Aerial Robots
Venue: arXiv cs.RO 2022
Year: 2022

Promoting exploratory movements through contingent feedback can positively influence motor development in infancy. Our ongoing work gears toward the development of a robot-assisted contingency learning environment through the use of small aerial robots. This paper examines whether aerial robots and their associated motion controllers can be used to achieve efficient and highly-responsive robot flight for our purpose. Infant kicking kinematic data were extracted from videos and used in simulation and physical experiments with an aerial robot. The efficacy of two standard of practice controllers was assessed: a linear PID and a nonlinear geometric controller. The ability of the robot to match infant kicking trajectories was evaluated qualitatively and quantitatively via the mean squared error (to assess overall deviation from the input infant leg trajectory signals), and dynamic time warping algorithm (to quantify the signal synchrony). Results demonstrate that it is in principle possible to track infant kicking trajectories with small aerials robots, and identify areas of further development required to improve the tracking quality.
Assessing the Unitary RNN as an End-to-End Compositional Model of Syntax
Venue: arXiv cs.CL 2022
Year: 2022

We show that both an LSTM and a unitary-evolution recurrent neural network (URN) can achieve encouraging accuracy on two types of syntactic patterns: context-free long distance agreement, and mildly context-sensitive cross serial dependencies. This work extends recent experiments on deeply nested context-free long distance dependencies, with similar results. URNs differ from LSTMs in that they avoid non-linear activation functions, and they apply matrix multiplication to word embeddings encoded as unitary matrices. This permits them to retain all information in the processing of an input string over arbitrary distances. It also causes them to satisfy strict compositionality. URNs constitute a significant advance in the search for explainable models in deep learning applied to NLP.
A Model of Anaphoric Ambiguities using Sheaf Theoretic Quantum-like Contextuality and BERT
Venue: arXiv cs.CL 2022
Year: 2022

Ambiguities of natural language do not preclude us from using it and context helps in getting ideas across. They, nonetheless, pose a key challenge to the development of competent machines to understand natural language and use it as humans do. Contextuality is an unparalleled phenomenon in quantum mechanics, where different mathematical formalisms have been put forwards to understand and reason about it. In this paper, we construct a schema for anaphoric ambiguities that exhibits quantum-like contextuality. We use a recently developed criterion of sheaf-theoretic contextuality that is applicable to signalling models. We then take advantage of the neural word embedding engine BERT to instantiate the schema to natural language examples and extract probability distributions for the instances. As a result, plenty of sheaf-contextual examples were discovered in the natural language corpora BERT utilises. Our hope is that these examples will pave the way for future research and for finding ways to extend applications of quantum computing to natural language processing.
Learning Point Processes using Recurrent Graph Network
Venue: arXiv cs.LG 2022
Year: 2022

We present a novel Recurrent Graph Network (RGN) approach for predicting discrete marked event sequences by learning the underlying complex stochastic process. Using the framework of Point Processes, we interpret a marked discrete event sequence as the superposition of different sequences each of a unique type. The nodes of the Graph Network use LSTM to incorporate past information whereas a Graph Attention Network (GAT Network) introduces strong inductive biases to capture the interaction between these different types of events. By changing the self-attention mechanism from attending over past events to attending over event types, we obtain a reduction in time and space complexity from $\mathcal{O}(N^2)$ (total number of events) to $\mathcal{O}(|\mathcal{Y}|^2)$ (number of event types). Experiments show that the proposed approach improves performance in log-likelihood, prediction and goodness-of-fit tasks with lower time and space complexity compared to state-of-the art Transformer based architectures.
Path-aware Siamese Graph Neural Network for Link Prediction
Venue: arXiv cs.LG 2022
Year: 2022

In this paper, we propose a Path-aware Siamese Graph neural network(PSG) for link prediction tasks. First, PSG captures both nodes and edge features for given two nodes, namely the structure information of k-neighborhoods and relay paths information of the nodes. Furthermore, a novel multi-task GNN framework with self-supervised contrastive learning is proposed for differentiation of positive links and negative links while content and behavior of nodes can be captured simultaneously. We evaluate the proposed algorithm PSG on two link property prediction datasets, ogbl-ddi and ogbl-collab. PSG achieves top 1 performance on ogbl-ddi until submission and top 3 performance on ogbl-collab. The experimental results verify the superiority of our proposed PSG
HERO: HiErarchical spatio-tempoRal reasOning with Contrastive Action Correspondence for End-to-End Video Object Grounding
Venue: arXiv cs.MM 2022
Year: 2022

Video Object Grounding (VOG) is the problem of associating spatial object regions in the video to a descriptive natural language query. This is a challenging vision-language task that necessitates constructing the correct cross-modal correspondence and modeling the appropriate spatio-temporal context of the query video and caption, thereby localizing the specific objects accurately. In this paper, we tackle this task by a novel framework called HiErarchical spatio-tempoRal reasOning (HERO) with contrastive action correspondence. We study the VOG task at two aspects that prior works overlooked: (1) Contrastive Action Correspondence-aware Retrieval. Notice that the fine-grained video semantics (e.g., multiple actions) is not totally aligned with the annotated language query (e.g., single action), we first introduce the weakly-supervised contrastive learning that classifies the video as action-consistent and action-independent frames relying on the video-caption action semantic correspondence. Such a design can build the fine-grained cross-modal correspondence for more accurate subsequent VOG. (2) Hierarchical Spatio-temporal Modeling Improvement. While transformer-based VOG models present their potential in sequential modality (i.e., video and caption) modeling, existing evidence also indicates that the transformer suffers from the issue of the insensitive spatio-temporal locality. Motivated by that, we carefully design the hierarchical reasoning layers to decouple fully connected multi-head attention and remove the redundant interfering correlations. Furthermore, our proposed pyramid and shifted alignment mechanisms are effective to improve the cross-modal information utilization of neighborhood spatial regions and temporal frames. We conducted extensive experiments to show our HERO outperforms existing techniques by achieving significant improvement on two benchmark datasets.
HiTailor: Interactive Transformation and Visualization for Hierarchical Tabular Data
Venue: arXiv cs.HC 2022
Year: 2022

Tabular visualization techniques integrate visual representations with tabular data to avoid additional cognitive load caused by splitting users' attention. However, most of the existing studies focus on simple flat tables instead of hierarchical tables, whose complex structure limits the expressiveness of visualization results and affects users' efficiency in visualization construction. We present HiTailor, a technique for presenting and exploring hierarchical tables. HiTailor constructs an abstract model, which defines row/column headings as biclustering and hierarchical structures. Based on our abstract model, we identify three pairs of operators, Swap/Transpose, ToStacked/ToLinear, Fold/Unfold, for transformations of hierarchical tables to support users' comprehensive explorations. After transformation, users can specify a cell or block of interest in hierarchical tables as a TableUnit for visualization, and HiTailor recommends other related TableUnits according to the abstract model using different mechanisms. We demonstrate the usability of the HiTailor system through a comparative study and a case study with domain experts, showing that HiTailor can present and explore hierarchical tables from different viewpoints. HiTailor is available at https://github.com/bitvis2021/HiTailor.
Face Morphing Attacks and Face Image Quality: The Effect of Morphing and the Unsupervised Attack Detection by Quality
Venue: arXiv cs.CV 2022
Year: 2022

Morphing attacks are a form of presentation attacks that gathered increasing attention in recent years. A morphed image can be successfully verified to multiple identities. This operation, therefore, poses serious security issues related to the ability of a travel or identity document to be verified to belong to multiple persons. Previous works touched on the issue of the quality of morphing attack images, however, with the main goal of quantitatively proofing the realistic appearance of the produced morphing attacks. We theorize that the morphing processes might have an effect on both, the perceptual image quality and the image utility in face recognition (FR) when compared to bona fide samples. Towards investigating this theory, this work provides an extensive analysis of the effect of morphing on face image quality, including both general image quality measures and face image utility measures. This analysis is not limited to a single morphing technique, but rather looks at six different morphing techniques and five different data sources using ten different quality measures. This analysis reveals consistent separability between the quality scores of morphing attack and bona fide samples measured by certain quality measures. Our study goes further to build on this effect and investigate the possibility of performing unsupervised morphing attack detection (MAD) based on quality scores. Our study looks intointra and inter-dataset detectability to evaluate the generalizability of such a detection concept on different morphing techniques and bona fide sources. Our final results point out that a set of quality measures, such as MagFace and CNNNIQA, can be used to perform unsupervised and generalized MAD with a correct classification accuracy of over 70%.
Automatic Hybrid-Precision Quantization for MIMO Detectors
Venue: arXiv cs.AR 2022
Year: 2022

In the design of wireless systems, quantization plays a critical role in hardware, which directly affects both area efficiency and energy efficiency. Being an enabling technique, the wide applications of multiple-input multiple-output (MIMO) heavily relies on efficient implementations balancing both performance and complexity. However, most of the existing detectors uniformly quantize all variables, resulting in high redundancy and low flexibility. Requiring both expertise and efforts, an in-depth tailored quantization usually asks for prohibitive costs and is not considered by conventional MIMO detectors. In this paper, a general framework named the automatic hybrid-precision quantization (AHPQ) is proposed with two parts: integral quantization determined by probability density function (PDF), and fractional quantization by deep reinforcement learning (DRL). Being automatic, AHPQ demonstrates high efficiency in figuring out good quantizations for a set of algorithmic parameters. For the approximate message passing (AMP) detector, AHPQ achieves up to $58.7\%$ lower average bitwidth than the unified quantization (UQ) one with almost no performance sacrifice. The feasibility of AHPQ has been verified by implementation with $65$ nm CMOS technology. Compared with its UQ counterpart, AHPQ exhibits $2.97\times$ higher throughput-to-area ratio (TAR) with $19.3\%$ lower energy dissipation. Moreover, by node compression and strength reduction, the AHPQ detector outperforms the state-of-the-art (SOA) in both throughput ($17.92$ Gb/s) and energy efficiency ($7.93$ pJ/b). The proposed AHPQ framework is also applicable for other digital signal processing algorithms.
Absolute Security in High-Frequency Wireless Links
Venue: arXiv cs.IT 2022
Year: 2022

Security against eavesdropping is one of the key concerns in the design of any communication system. Many common considerations of the security of a wireless communication channel rely on comparing the signal level measured by Bob (the intended receiver) to that accessible to Eve (an eavesdropper). Frameworks such as Wyner's wiretap model ensure the security of a link, in an average sense, when Bob's signal-to-noise ratio exceeds Eve's. Unfortunately, because these guarantees rely on statistical assumptions about noise, Eve can still occasionally succeed in decoding information. The goal of achieving exactly zero probability of intercept over an engineered region of the broadcast sector, which we term absolute security, remains elusive. Here, we describe the first architecture for a wireless link which provides absolute security. Our approach relies on the inherent properties of broadband and high-gain antennas, and is therefore ideally suited for implementation in millimeter-wave and terahertz wireless systems, where such antennas will generally be employed. We exploit spatial minima of the antenna pattern at different frequencies, the union of which defines a wide region where Eve is guaranteed to fail regardless of her computational capabilities, and regardless of the noise in the channels. Unlike conventional zero-forcing beam forming methods, we show that, for realistic assumptions about the antenna configuration and power budget, this absolute security guarantee can be achieved over most possible eavesdropper locations. Since we use relatively simple frequency-multiplexed coding, together with the underlying physics of a diffracting aperture, this idea is broadly applicable in many contexts.
Privacy Preservation Among Honest-but-Curious Edge Nodes: A Survey
Venue: arXiv cs.CR 2022
Year: 2022

Users care greatly about preserving the privacy of their personal data gathered during their use of information systems. This extends to both the data they actively provide in exchange for services as well as the metadata passively generated in many aspects of their computing experiences. However, new technologies are at a great risk of being inadequate to protect a user's privacy if researchers focus primarily on the use cases of these technologies without giving sufficient consideration to incorporating privacy at a fundamental level. Edge computing has been introduced as a promising networking paradigm for processing the incredible magnitude of data produced by modern IoT networks. As edge computing is still considered a relatively new technology, the edge computing community has a responsibility to ensure privacy protection is interwoven into its implementations at a foundational level. In this paper, I first introduce the concepts of user privacy and edge computing; I then provide a state-of-the-art overview of current literature as it relates to privacy preservation in honest-but-curious edge computing. Finally, I provide future research recommendations in the hope that the edge computing research and development community will be inspired to ensure strong privacy protections in their current and future work.
Modular interface for managing cognitive bias in experts
Venue: arXiv cs.HC 2022
Year: 2022

Expert knowledge is required to interpret data across a range of fields. Experts bridge gaps that often exists in our knowledge about relationships between data and the parameters of interest. This is especially true in geoscientific applications, where knowledge of the Earth is derived from interpretations of observable features and relies on predominantly unproven but widely accepted theories. Thus, experts facilitate solutions to otherwise unsolvable problems. However, experts are inherently subjective, and susceptible to cognitive biases and adverse external effects. This work examines this problem within geoscience. Three compelling examples are provided of the prevalence of cognitive biases from previous work. The problem is then formally defined, and a set of design principles which ensure that any solution is sufficiently flexible to be readily applied to the range of geoscientific problems. No solutions exist that reliably capture and reduce cognitive bias in experts. However, formal expert elicitation methods can be used to assess expert variation, and a variety of approaches exist that may help to illuminate uncertainties, avoid misunderstandings, and reduce herding behaviours or single-expert over-dominance. This work combines existing and future approaches to reduce expert suboptimality through a flexible modular design where each module provides a specific function. The design centres around action modules that force a stop-and-perform step into interpretation tasks. A starter-pack of modules is provided as an example of the conceptual design. This simple bias-reduction system may readily be applied in organisations and during everyday interpretations through to tasks for major commercial ventures.
Security of IoT Device: Perspective Forensic/Anti-Forensic Issues on Invalid Area of NAND Flash Memory
Venue: arXiv cs.CR 2022
Year: 2022

NAND flash memory-based IoT device can potentially still leave behind original personal data in an invalid area even if the data has been deleted. In this paper, we raise the forensic issue of original data remaining in unmanaged blocks caused by NAND flash memory and introduce methods for secure deletion of such data in the invalid area. We also propose a verification technique for secure deletion that is performed based on cell count information, which refers to the difference in bits between personal data and data stored in the block. The pass/fail of the verification technique according to the cell count information is determined in consideration of error correction capabilities. With the forensic issue of de-identification being a vital theme in the big data industry, the threat of serious privacy breaches coupled with our proposal to prevent these attacks will prove to be critical technological necessities in the future.
On Zero-Divisor Graph of the ring $\mathbb{F}_p+u\mathbb{F}_p+u^2 \mathbb{F}_p$
Venue: arXiv cs.IT 2022
Year: 2022

In this article, we discussed the zero-divisor graph of a commutative ring with identity $\mathbb{F}_p+u\mathbb{F}_p+u^2 \mathbb{F}_p$ where $u^3=0$ and $p$ is an odd prime. We find the clique number, chromatic number, vertex connectivity, edge connectivity, diameter and girth of a zero-divisor graph associated with the ring. We find some of topological indices and the main parameters of the code derived from the incidence matrix of the zero-divisor graph $Γ(R).$ Also, we find the eigenvalues, energy and spectral radius of both adjacency and Laplacian matrices of $Γ(R).$