OpenRaiser/Intern-Atlas huggingface dataset:
E Pluribus Unum Interpretable Convolutional Neural Networks
Venue: arXiv cs.CV 2022
Year: 2022

The adoption of Convolutional Neural Network (CNN) models in high-stake domains is hindered by their inability to meet society's demand for transparency in decision-making. So far, a growing number of methodologies have emerged for developing CNN models that are interpretable by design. However, such models are not capable of providing interpretations in accordance with human perception, while maintaining competent performance. In this paper, we tackle these challenges with a novel, general framework for instantiating inherently interpretable CNN models, named E Pluribus Unum Interpretable CNN (EPU-CNN). An EPU-CNN model consists of CNN sub-networks, each of which receives a different representation of an input image expressing a perceptual feature, such as color or texture. The output of an EPU-CNN model consists of the classification prediction and its interpretation, in terms of relative contributions of perceptual features in different regions of the input image. EPU-CNN models have been extensively evaluated on various publicly available datasets, as well as a contributed benchmark dataset. Medical datasets are used to demonstrate the applicability of EPU-CNN for risk-sensitive decisions in medicine. The experimental results indicate that EPU-CNN models can achieve a comparable or better classification performance than other CNN architectures while providing humanly perceivable interpretations.
A Comparison of Spatiotemporal Visualizations for 3D Urban Analytics
Venue: arXiv cs.HC 2022
Year: 2022

Recent technological innovations have led to an increase in the availability of 3D urban data, such as shadow, noise, solar potential, and earthquake simulations. These spatiotemporal datasets create opportunities for new visualizations to engage experts from different domains to study the dynamic behavior of urban spaces in this under explored dimension. However, designing 3D spatiotemporal urban visualizations is challenging, as it requires visual strategies to support analysis of time-varying data referent to the city geometry. Although different visual strategies have been used in 3D urban visual analytics, the question of how effective these visual designs are at supporting spatiotemporal analysis on building surfaces remains open. To investigate this, in this paper we first contribute a series of analytical tasks elicited after interviews with practitioners from three urban domains. We also contribute a quantitative user study comparing the effectiveness of four representative visual designs used to visualize 3D spatiotemporal urban data: spatial juxtaposition, temporal juxtaposition, linked view, and embedded view. Participants performed a series of tasks that required them to identify extreme values on building surfaces over time. Tasks varied in granularity for both space and time dimensions. Our results demonstrate that participants were more accurate using plot-based visualizations (linked view, embedded view) but faster using color-coded visualizations (spatial juxtaposition, temporal juxtaposition). Our results also show that, with increasing task complexity, plot-based visualizations perform better in preserving efficiency (time, accuracy) compared to color-coded visualizations. Based on our findings, we present a set of takeaways with design recommendations for 3D spatiotemporal urban visualizations for researchers and practitioners.
A Note on the Computational Complexity of Selfmate and Reflexmate Chess Problems
Venue: arXiv cs.CC 2022
Year: 2022

A selfmate is a Chess problem in which White, moving first, needs to force Black to checkmate within a specified number of moves. The reflexmate is a derivative of the selfmate in which White compels Black to checkmate with the added condition that if either player can checkmate, they must do that (when this condition applies only to Black, it is a semi-reflexmate). We slightly modify the reduction of EXPTIME-hardness of Chess and apply the reduction to these Chess problems. It is proved that selfmate, reflexmate, and semi-reflexmate are all EXPTIME-complete.
ATLAS: Universal Function Approximator for Memory Retention
Venue: arXiv cs.LG 2022
Year: 2022

Artificial neural networks (ANNs), despite their universal function approximation capability and practical success, are subject to catastrophic forgetting. Catastrophic forgetting refers to the abrupt unlearning of a previous task when a new task is learned. It is an emergent phenomenon that hinders continual learning. Existing universal function approximation theorems for ANNs guarantee function approximation ability, but do not predict catastrophic forgetting. This paper presents a novel universal approximation theorem for multi-variable functions using only single-variable functions and exponential functions. Furthermore, we present ATLAS: a novel ANN architecture based on the new theorem. It is shown that ATLAS is a universal function approximator capable of some memory retention, and continual learning. The memory of ATLAS is imperfect, with some off-target effects during continual learning, but it is well-behaved and predictable. An efficient implementation of ATLAS is provided. Experiments are conducted to evaluate both the function approximation and memory retention capabilities of ATLAS.
The Analysis about Building Cross-lingual Sememe Knowledge Base Based on Deep Clustering Network
Venue: arXiv cs.CL 2022
Year: 2022

A sememe is defined as the minimum semantic unit of human languages. Sememe knowledge bases (KBs), which contain words annotated with sememes, have been successfully applied to many NLP tasks, and we believe that by learning the smallest unit of meaning, computers can more easily understand human language. However, Existing sememe KBs are built on only manual annotation, human annotations have personal understanding biases, and the meaning of vocabulary will be constantly updated and changed with the times, and artificial methods are not always practical. To address the issue, we propose an unsupervised method based on a deep clustering network (DCN) to build a sememe KB, and you can use any language to build a KB through this method. We first learn the distributed representation of multilingual words, use MUSE to align them in a single vector space, learn the multi-layer meaning of each word through the self-attention mechanism, and use a DNC to cluster sememe features. Finally, we completed the prediction using only the 10-dimensional sememe space in English. We found that the low-dimensional space can still retain the main feature of the sememes.
Sequence Feature Extraction for Malware Family Analysis via Graph Neural Network
Venue: arXiv cs.CR 2022
Year: 2022

Malicious software (malware) causes much harm to our devices and life. We are eager to understand the malware behavior and the threat it made. Most of the record files of malware are variable length and text-based files with time stamps, such as event log data and dynamic analysis profiles. Using the time stamps, we can sort such data into sequence-based data for the following analysis. However, dealing with the text-based sequences with variable lengths is difficult. In addition, unlike natural language text data, most sequential data in information security have specific properties and structure, such as loop, repeated call, noise, etc. To deeply analyze the API call sequences with their structure, we use graphs to represent the sequences, which can further investigate the information and structure, such as the Markov model. Therefore, we design and implement an Attention Aware Graph Neural Network (AWGCN) to analyze the API call sequences. Through AWGCN, we can obtain the sequence embeddings to analyze the behavior of the malware. Moreover, the classification experiment result shows that AWGCN outperforms other classifiers in the call-like datasets, and the embedding can further improve the classic model's performance.
SSDBCODI: Semi-Supervised Density-Based Clustering with Outliers Detection Integrated
Venue: arXiv cs.LG 2022
Year: 2022

Clustering analysis is one of the critical tasks in machine learning. Traditionally, clustering has been an independent task, separate from outlier detection. Due to the fact that the performance of clustering can be significantly eroded by outliers, a small number of algorithms try to incorporate outlier detection in the process of clustering. However, most of those algorithms are based on unsupervised partition-based algorithms such as k-means. Given the nature of those algorithms, they often fail to deal with clusters of complex, non-convex shapes. To tackle this challenge, we have proposed SSDBCODI, a semi-supervised density-based algorithm. SSDBCODI combines the advantage of density-based algorithms, which are capable of dealing with clusters of complex shapes, with the semi-supervised element, which offers flexibility to adjust the clustering results based on a few user labels. We also merge an outlier detection component with the clustering process. Potential outliers are detected based on three scores generated during the process: (1) reachability-score, which measures how density-reachable a point is to a labeled normal object, (2) local-density-score, which measures the neighboring density of data objects, and (3) similarity-score, which measures the closeness of a point to its nearest labeled outliers. Then in the following step, instance weights are generated for each data instance based on those three scores before being used to train a classifier for further clustering and outlier detection. To enhance the understanding of the proposed algorithm, for our evaluation, we have run our proposed algorithm against some of the state-of-art approaches on multiple datasets and separately listed the results of outlier detection apart from clustering. Our results indicate that our algorithm can achieve superior results with a small percentage of labels.
Memorizing Complementation Network for Few-Shot Class-Incremental Learning
Venue: arXiv cs.CV 2022
Year: 2022

Few-shot Class-Incremental Learning (FSCIL) aims at learning new concepts continually with only a few samples, which is prone to suffer the catastrophic forgetting and overfitting problems. The inaccessibility of old classes and the scarcity of the novel samples make it formidable to realize the trade-off between retaining old knowledge and learning novel concepts. Inspired by that different models memorize different knowledge when learning novel concepts, we propose a Memorizing Complementation Network (MCNet) to ensemble multiple models that complements the different memorized knowledge with each other in novel tasks. Additionally, to update the model with few novel samples, we develop a Prototype Smoothing Hard-mining Triplet (PSHT) loss to push the novel samples away from not only each other in current task but also the old distribution. Extensive experiments on three benchmark datasets, e.g., CIFAR100, miniImageNet and CUB200, have demonstrated the superiority of our proposed method.
Visual Haptic Reasoning: Estimating Contact Forces by Observing Deformable Object Interactions
Venue: arXiv cs.RO 2022
Year: 2022

Robotic manipulation of highly deformable cloth presents a promising opportunity to assist people with several daily tasks, such as washing dishes; folding laundry; or dressing, bathing, and hygiene assistance for individuals with severe motor impairments. In this work, we introduce a formulation that enables a collaborative robot to perform visual haptic reasoning with cloth -- the act of inferring the location and magnitude of applied forces during physical interaction. We present two distinct model representations, trained in physics simulation, that enable haptic reasoning using only visual and robot kinematic observations. We conducted quantitative evaluations of these models in simulation for robot-assisted dressing, bathing, and dish washing tasks, and demonstrate that the trained models can generalize across different tasks with varying interactions, human body sizes, and object shapes. We also present results with a real-world mobile manipulator, which used our simulation-trained models to estimate applied contact forces while performing physically assistive tasks with cloth. Videos can be found at our project webpage.
Towards Automated Key-Point Detection in Images with Partial Pool View
Venue: arXiv cs.CV 2022
Year: 2022

Sports analytics has been an up-and-coming field of research among professional sporting organizations and academic institutions alike. With the insurgence and collection of athlete data, the primary goal of such analysis is to improve athletes' performance in a measurable and quantifiable manner. This work is aimed at alleviating some of the challenges encountered in the collection of adequate swimming data. Past works on this subject have shown that the detection and tracking of swimmers is feasible, but not without challenges. Among these challenges are pool localization and determining the relative positions of the swimmers relative to the pool. This work presents two contributions towards solving these challenges. First, we present a pool model with invariant key-points relevant for swimming analytics. Second, we study the detectability of such key-points in images with partial pool view, which are challenging but also quite common in swimming race videos.
A Trust-Based Malicious RSU Detection Mechanism in Edge-Enabled Vehicular Ad Hoc Networks
Venue: arXiv cs.CR 2022
Year: 2022

Edge-enabled Vehicular Ad Hoc Network (VANET) introduces real-time services and storage, computation, and communication facilities to the vehicles through Roadside Units (RSUs). Nevertheless, RSUs are often easy targets for security assaults due to their placement in an open, unprotected environment and resource-constrained nature. The malicious RSUs compromised by security attacks impose threats to human safety by impeding the operations of VANETs. Hence, an effective malevolent RSU detection mechanism is crucial for VANETs. Existing trust-based detection mechanisms assign trust scores to RSUs based on their interactions with moving vehicles where precise detection of rogue RSUs depends on the accuracy of trust scores. However, brief interaction of RSUs with the running vehicles permits inadequate time to estimate trust accurately. Besides, current works use only vehicle speed and density in beacon messages to assess trust without considering the sensor-detected data in the same messages. Nonetheless, sensor data is useful for traffic management, and neglecting them creates inaccuracy in trust estimation. In this paper, we address these limitations and propose a trust-based scheme to detect malicious RSUs that uses stable and frequent RSU-to-RSU (R2R) interaction to precisely analyze the behavior of an RSU. We also offer a mechanism to detect alteration of sensor-detected data in beacon content and incorporate this scheme in the trust calculation of RSUs. The experimental results show that the proposed solution effectively detects approximately 92% malicious RSUs, even in the presence of hostile vehicles. Moreover, integrating the proposed solution with the VANET routing protocols improves routing efficiency.
A Cooperative Positioning Flamework for Robot and Smart Phone Based on Visible Light Communication
Venue: arXiv cs.RO 2022
Year: 2022

A cooperative positioning flamework of human and robots based on visible light communication (VLC) is proposed. Based on the experiment system, we demonstrated it is feasible and has high-accuracy and real-time performance.
Improving Tourist Experience Through an IoT Application Based on FatBeacons
Venue: arXiv cs.CR 2022
Year: 2022

This paper describes the use of a new extension of the Bluetooth connection protocol, called FatBeacon, which faces the problem of obtaining information where no Internet connection is available. Rather than advertising a URL to load a web page, the FatBeacon protocol has the ability to broadcast any basic web contents actually hosted on the device. In particular, FatBeacons are here used to improve the tourist experience in places with no Internet coverage through a new application of the Internet of Things (IoT). Thanks to the fact that the web content is emitted by the own FatBeacon, any smartphone with Bluetooth Low Energy (BLE) can be used to receive touristic information, even in uncovered areas, such as rural or mountain destinations. This work does not only show the applicability of the new FatBeacon protocol, but it also presents a performance comparison of different BLE technologies used for similar touristic applications.
Learning Computation Bounds for Branch-and-Bound Algorithms to k-plex Extraction
Venue: arXiv cs.DS 2022
Year: 2022

k-plex is a representative definition of communities in networks. While the cliques is too stiff to applicate to real cases, the k-plex relaxes the notion of the clique, allowing each node to miss up to k connections. Although k-plexes are more flexible than cliques, finding them is more challenging as their number is greater. In this paper, we aim to detect the k-plex under the size and time constraints, leveraging the new vision of automated learning bounding strategy. We introduce the constraint learning concept to learn the bound strategy from the branch and bound process and develop it into a Mixed Integer Programming framework. While most of the work is dedicated on learn the branch strategy in branch and bound-based algorithms, we focus on the learn to bound strategy which needs to handle the problem that learned strategy might not examine the feasible solution. We adopted the MILP framework and design a set of variables relative to the k-plex property as our constraint space to learn the strategy. The learn to bound strategy learning the original strategy function also reduces the computation load of the bound process to accelerate the branch and bound algorithm. Note that the learn to bound concept can apply to any branch and bound based algorithm with the appropriate framework. We conduct the experiment on different networks, the results show that our learn to branch and bound method does accelerate the original branch and bound method and outperforms other baselines, while also being able to generalize on different graph properties.
Proposals for Resolving Consenting Issues with Signals and User-side Dialogues
Venue: arXiv cs.CY 2022
Year: 2022

Consent dialogues are a source of annoyance, malicious intent, dark patterns, illegal practices and a plethora of other issues. This work presents known problems based on GDPR requirements grouped into two categories: (i) UI/UX for consenting; and (ii) power imbalance in expressing consent. To resolve this, it presents two proposals: First, the use of automation through privacy signals to better govern consenting processes and to reduce `consent-fatigue'. Second, as generation of consent dialogues on the user side and its practicalities for both websites as well as users and agents (e.g. web browsers). Both proposals are discussed in terms of possibilities for implementation and suitability for stakeholders. The article concludes with a discussion on the difficulties in achieving such solutions owing to the conflicts of interest between `web-enablers' and `web-consumers', and the necessity for the EU to take a direct stance in addressing these in their future laws.
Transparent and Tamper-Proof Event Ordering in the Internet of Things Platforms
Venue: arXiv cs.CR 2022
Year: 2022

Today, the audit and diagnosis of the causal relationships between the events in a trigger-action-based event chain (e.g., why is a light turned on in a smart home?) in the Internet of Things (IoT) platforms are untrustworthy and unreliable. The current IoT platforms lack techniques for transparent and tamper-proof ordering of events due to their device-centric logging mechanism. In this paper, we develop a framework that facilitates tamper-proof transparency and event order in an IoT platform by proposing a Blockchain protocol and adopting the vector clock system, both tailored for the resource-constrained heterogeneous IoT devices, respectively. To cope with the unsuited storage (e.g., ledger) and computing power (e.g., proof of work puzzle) requirements of the Blockchain in the commercial off-the-shelf IoT devices, we propose a partial consistent cut protocol and engineer a modular arithmetic-based lightweight proof of work puzzle, respectively. To the best of our knowledge, this is the first Blockchain designed for resource-constrained heterogeneous IoT platforms. Our event ordering protocol based on the vector clock system is also novel for the IoT platforms. We implement our framework using an IoT gateway and 30 IoT devices. We experiment with 10 concurrent trigger-action-based event chains while each chain involves 20 devices, and each device participates in 5 different chains. The results show that our framework may order these events in 2.5 seconds while consuming only 140 mJ of energy per device. The results hence demonstrate the proposed platform as a practical choice for many IoT applications such as smart home, traffic monitoring, and crime investigation.
Optimizing Irregular-Shaped Matrix-Matrix Multiplication on Multi-Core DSPs
Venue: arXiv cs.DC 2022
Year: 2022

General Matrix Multiplication (GEMM) has a wide range of applications in scientific simulation and artificial intelligence. Although traditional libraries can achieve high performance on large regular-shaped GEMMs, they often behave not well on irregular-shaped GEMMs, which are often found in new algorithms and applications of high-performance computing (HPC). Due to energy efficiency constraints, low-power multi-core digital signal processors (DSPs) have become an alternative architecture in HPC systems. Targeting multi-core DSPs in FT-m7032, a prototype CPU-DSPs heterogeneous processor for HPC, an efficient implementation - ftIMM - for three types of irregular-shaped GEMMs is proposed. FtIMM supports automatic generation of assembly micro-kernels, two parallelization strategies, and auto-tuning of block sizes and parallelization strategies. The experiments show that ftIMM can get better performance than the traditional GEMM implementations on multi-core DSPs in FT-m7032, yielding on up to 7.2x performance improvement, when performing on irregular-shaped GEMMs. And ftIMM on multi-core DSPs can also far outperform the open source library on multi-core CPUs in FT-m7032, delivering up to 3.1x higher efficiency.
Covert Beamforming Design for Integrated Radar Sensing and Communication Systems
Venue: arXiv cs.IT 2022
Year: 2022

We propose covert beamforming design frameworks for integrated radar sensing and communication (IRSC) systems, where the radar can covertly communicate with legitimate users under the cover of the probing waveforms without being detected by the eavesdropper. Specifically, by jointly designing the target detection beamformer and communication beamformer, we aim to maximize the radar detection mutual information (MI) (or the communication rate) subject to the covert constraint, the communication rate constraint (or the radar detection MI constraint), and the total power constraint. For the perfect eavesdropper's channel state information (CSI) scenario, we transform the covert beamforming design problems into a series of convex subproblems, by exploiting semidefinite relaxation, which can be solved via the bisection search method. Considering the high complexity of iterative optimization, we further propose a single-iterative covert beamformer design scheme based on the zero-forcing criterion. For the imperfect eavesdropper's CSI scenario, we develop a relaxation and restriction method to tackle the robust covert beamforming design problems. Simulation results demonstrate the effectiveness of the proposed covert beamforming schemes for perfect and imperfect CSI scenarios.