OpenRaiser/Intern-Atlas huggingface dataset:
Context-based Deep Learning Architecture with Optimal Integration Layer for Image Parsing
Venue: arXiv cs.CV 2022
Year: 2022

Deep learning models have been efficient lately on image parsing tasks. However, deep learning models are not fully capable of exploiting visual and contextual information simultaneously. The proposed three-layer context-based deep architecture is capable of integrating context explicitly with visual information. The novel idea here is to have a visual layer to learn visual characteristics from binary class-based learners, a contextual layer to learn context, and then an integration layer to learn from both via genetic algorithm-based optimal fusion to produce a final decision. The experimental outcomes when evaluated on benchmark datasets are promising. Further analysis shows that optimized network weights can improve performance and make stable predictions.
Performance Analysis of Wireless Network Aided by Discrete-Phase-Shifter IRS
Venue: arXiv cs.IT 2022
Year: 2022

Discrete phase shifters of intelligent reflecting surface (IRS) generates phase quantization error (QE) and degrades the receive performance at the receiver. To make an analysis of the performance loss caused by IRS with phase QE, based on the law of large numbers, the closed-form expressions of signal-to-noise ratio (SNR) performance loss (PL), achievable rate (AR), and bit error rate (BER) are successively derived under line-of-sight (LoS) channels and Rayleigh channels. Moreover, based on the Taylor series expansion, the approximate simple closed form of PL of IRS with approximate QE is also given. The simulation results show that the performance losses of SNR and AR decrease as the number of quantization bits increase, while they gradually increase with the number of IRS phase shifter elements increase. Regardless of LoS channels or Rayleigh channels, when the number of quantization bits is larger than or equal to 3, the performance losses of SNR and AR are less than 0.23dB and 0.08bits/s/Hz, respectively, and the BER performance degradation is trivial. In particular, the performance loss difference between IRS with QE and IRS with approximate QE is negligible when the number of quantization bits is not less than 2.
Electromagnetic Quasistatic Field Formulations of Darwin Type
Venue: arXiv cs.CE 2022
Year: 2022

Electromagnetic quasistatic (EMQS) fields, where radiation effects are neglected, while Ohmic losses and electric and magnetic field energies are considered, can be modeled using Darwin-type field models as an approximation to the full Maxwell equations. Commonly formulated in terms of magnetic vector and electric scalar potentials, these EMQS formulations are not gauge invariant. Several EMQS formulations resulting from different gauge equations are considered and analyzed in terms of their structural properties and their modeling capabilities and limitations. Associated discrete field formulations in the context of the Maxwell-grid equations of the Finite Integration Technique are considered in frequency and time domain and are studied with respect to their algebraic properties. A comparison of numerical simulation results w.r.t. reference solutions obtained with established formulations for the full Maxwell equations are presented.
Automated Atomic Silicon Quantum Dot Circuit Design via Deep Reinforcement Learning
Venue: arXiv cs.ET 2022
Year: 2022

Robust automated design tools are crucial for the proliferation of any computing technology. We introduce the first automated design tool for the silicon dangling bond quantum dot computing technology, which is an extremely versatile and flexible single-atom computing circuitry framework. The automated designer is capable of navigating the complex, hyperdimensional design spaces of arbitrarily sized design areas and truth tables by employing a tabula rasa double-deep Q-learning reinforcement learning algorithm. Robust policy convergence is demonstrated for a wide range of two-input, one-output logic circuits and a two-input, two-output half-adder, designed with an order of magnitude fewer SiDBs in several orders of magnitude less time than the only other half-adder demonstrated in the literature. We anticipate that reinforcement learning-based automated design tools will accelerate the development of the SiDB quantum dot computing technology, leading to its eventual adoption in specialized computing hardware.
Swarm Modelling with Dynamic Mode Decomposition
Venue: arXiv cs.NE 2022
Year: 2022

Modelling biological or engineering swarms is challenging due to the inherently high dimension of the system, despite the often low-dimensional emergent dynamics. Most existing swarm modelling approaches are based on first principles and often result in swarm-specific parameterizations that do not generalize to a broad range of applications. In this work, we apply a purely data-driven method to (1) learn local interactions of homogeneous swarms through observation data and to (2) generate similar swarming behaviour using the learned model. In particular, a modified version of dynamic mode decomposition with control, called swarmDMD, is developed and tested on the canonical Vicsek swarm model. The goal is to use swarmDMD to learn inter-agent interactions that give rise to the observed swarm behaviour. We show that swarmDMD can faithfully reconstruct the swarm dynamics, and the model learned by swarmDMD provides a short prediction window for data extrapolation with a trade-off between prediction accuracy and prediction horizon. We also provide a comprehensive analysis on the efficacy of different observation data types on the modelling, where we find that inter-agent distance yields the most accurate models. We believe the proposed swarmDMD approach will be useful for studying multi-agent systems found in biology, physics, and engineering.
AHP: Learning to Negative Sample for Hyperedge Prediction
Venue: arXiv cs.LG 2022
Year: 2022

Hypergraphs (i.e., sets of hyperedges) naturally represent group relations (e.g., researchers co-authoring a paper and ingredients used together in a recipe), each of which corresponds to a hyperedge (i.e., a subset of nodes). Predicting future or missing hyperedges bears significant implications for many applications (e.g., collaboration and recipe recommendation). What makes hyperedge prediction particularly challenging is the vast number of non-hyperedge subsets, which grows exponentially with the number of nodes. Since it is prohibitive to use all of them as negative examples for model training, it is inevitable to sample a very small portion of them, and to this end, heuristic sampling schemes have been employed. However, trained models suffer from poor generalization capability for examples of different natures. In this paper, we propose AHP, an adversarial training-based hyperedge-prediction method. It learns to sample negative examples without relying on any heuristic schemes. Using six real hypergraphs, we show that AHP generalizes better to negative examples of various natures. It yields up to 28.2% higher AUROC than the best existing methods and often even outperforms its variants with sampling schemes tailored to test sets.
Efficient Re-parameterization Operations Search for Easy-to-Deploy Network Based on Directional Evolutionary Strategy
Venue: arXiv cs.AI 2022
Year: 2022

Structural re-parameterization (Rep) methods has achieved significant performance improvement on traditional convolutional network. Most current Rep methods rely on prior knowledge to select the reparameterization operations. However, the performance of architecture is limited by the type of operations and prior knowledge. To break this restriction, in this work, an improved re-parameterization search space is designed, which including more type of re-parameterization operations. Concretely, the performance of convolutional networks can be further improved by the search space. To effectively explore this search space, an automatic re-parameterization enhancement strategy is designed based on neural architecture search (NAS), which can search a excellent re-parameterization architecture. Besides, we visualize the output features of the architecture to analyze the reasons for the formation of the re-parameterization architecture. On public datasets, we achieve better results. Under the same training conditions as ResNet, we improve the accuracy of ResNet-50 by 1.82% on ImageNet-1k.
Building Markovian Generative Architectures over Pretrained LM Backbones for Efficient Task-Oriented Dialog Systems
Venue: arXiv cs.CL 2022
Year: 2022

Recently, Transformer based pretrained language models (PLMs), such as GPT2 and T5, have been leveraged to build generative task-oriented dialog (TOD) systems. A drawback of existing PLM-based models is their non-Markov architectures across turns, i.e., the whole history is used as the conditioning input at each turn. First, this brings inefficiencies in memory and computation. Furthermore, using the whole history increases model complexity and may hurt the training efficiency, especially when facing small amounts of labeled training data (the low-resource setting). In this paper, motivated by the observation that dialog states could be viewed as Markov states, we propose to build Markovian Generative Architectures (MGA) over PLM backbones for efficient TOD systems. Experiments on MultiWOZ2.1 show that in the rich-resource setting, the proposed Markov models reduce memory and time costs without performance degradation; in the low-resource setting, the training efficiency of the Markov models is more significant.
Does depth estimation help object detection?
Venue: arXiv cs.CV 2022
Year: 2022

Ground-truth depth, when combined with color data, helps improve object detection accuracy over baseline models that only use color. However, estimated depth does not always yield improvements. Many factors affect the performance of object detection when estimated depth is used. In this paper, we comprehensively investigate these factors with detailed experiments, such as using ground-truth vs. estimated depth, effects of different state-of-the-art depth estimation networks, effects of using different indoor and outdoor RGB-D datasets as training data for depth estimation, and different architectural choices for integrating depth to the base object detector network. We propose an early concatenation strategy of depth, which yields higher mAP than previous works' while using significantly fewer parameters.
A pipeline and comparative study of 12 machine learning models for text classification
Venue: arXiv cs.IR 2022
Year: 2022

Text-based communication is highly favoured as a communication method, especially in business environments. As a result, it is often abused by sending malicious messages, e.g., spam emails, to deceive users into relaying personal information, including online accounts credentials or banking details. For this reason, many machine learning methods for text classification have been proposed and incorporated into the services of most email providers. However, optimising text classification algorithms and finding the right tradeoff on their aggressiveness is still a major research problem. We present an updated survey of 12 machine learning text classifiers applied to a public spam corpus. A new pipeline is proposed to optimise hyperparameter selection and improve the models' performance by applying specific methods (based on natural language processing) in the preprocessing stage. Our study aims to provide a new methodology to investigate and optimise the effect of different feature sizes and hyperparameters in machine learning classifiers that are widely used in text classification problems. The classifiers are tested and evaluated on different metrics including F-score (accuracy), precision, recall, and run time. By analysing all these aspects, we show how the proposed pipeline can be used to achieve a good accuracy towards spam filtering on the Enron dataset, a widely used public email corpus. Statistical tests and explainability techniques are applied to provide a robust analysis of the proposed pipeline and interpret the classification outcomes of the 12 machine learning models, also identifying words that drive the classification results. Our analysis shows that it is possible to identify an effective machine learning model to classify the Enron dataset with an F-score of 94%.
Bayesian Negative Sampling for Recommendation
Venue: arXiv cs.IR 2022
Year: 2022

How to sample high quality negative instances from unlabeled data, i.e., negative sampling, is important for training implicit collaborative filtering and contrastive learning models. Although previous studies have proposed some approaches to sample informative instances, few has been done to discriminating false negative from true negative for unbiased negative sampling. On the basis of our order relation analysis of negatives' scores, we first derive the class conditional density of true negatives and that of false negatives. We next design a Bayesian classifier for negative classification, from which we define a model-agnostic posterior probability estimate of an instance being true negative as a quantitative negative signal measure. We also propose a Bayesian optimal sampling rule to sample high-quality negatives. The proposed Bayesian Negative Sampling (BNS) algorithm has a linear time complexity. Experimental studies validate the superiority of BNS over the peers in terms of better sampling quality and better recommendation performance.
Improving generalization to new environments and removing catastrophic forgetting in Reinforcement Learning by using an eco-system of agents
Venue: arXiv cs.AI 2022
Year: 2022

Adapting a Reinforcement Learning (RL) agent to an unseen environment is a difficult task due to typical over-fitting on the training environment. RL agents are often capable of solving environments very close to the trained environment, but when environments become substantially different, their performance quickly drops. When agents are retrained on new environments, a second issue arises: there is a risk of catastrophic forgetting, where the performance on previously seen environments is seriously hampered. This paper proposes a novel approach that exploits an eco-system of agents to address both concerns. Hereby, the (limited) adaptive power of individual agents is harvested to build a highly adaptive eco-system.
Observability Analysis of Visual-Inertial Odometry with Online Calibration of Velocity-Control Based Kinematic Motion Models
Venue: arXiv cs.RO 2022
Year: 2022

In this paper, we analyze the observability of the visual-inertial odometry (VIO) using stereo cameras with a velocity-control based kinematic motion model. Previous work shows that in general case the global position and yaw are unobservable in VIO system, additionally the roll and pitch become also unobservable if there is no rotation. We prove that by integrating a planar motion constraint roll and pitch become observable. We also show that the parameters of the motion model are observable.
Explicit caching HYB: a new high-performance SpMV framework on GPGPU
Venue: arXiv cs.DC 2022
Year: 2022

Sparse Matrix-Vector Multiplication (SpMV) is a critical operation for the iterative solver of Finite Element Methods on computer simulation. Since the SpMV operation is a memory-bound algorithm, the efficiency of data movements heavily influenced the performance of the SpMV on GPU. In recent years, many research is conducted in accelerating the performance of SpMV on the graphic processing units (GPU). The performance optimization methods used in existing studies focus on the following areas: improve the load balancing between GPU processors, and reduce the execution divergence between GPU threads. Although some studies have made preliminary optimization on the input vector fetching, the effect of explicitly caching the input vector on GPU base SpMV has not been studied in depth yet. In this study, we are trying to minimize the data movements cost for GPU-based SpMV using a new framework named "explicit caching Hybrid (EHYB)". The EHYB framework achieved significant performance improvement by using the following methods: 1. Improve the speed of data movements by partitioning and explicitly caching the input vector to the shared memory of the CUDA kernel. 2. Reduce the volume of data movements by storing the major part of the column index with a compact format. We tested our implementation with sparse matrices derived from FEM applications in different areas. The experiment results show that our implementation can overperform the state-of-the-arts implementation with significant speedup, and leads to higher FLOPs than the theoryperformance up-boundary of the existing GPU-based SpMV implementations.
GM-TOuNN: Graded Multiscale Topology Optimization using Neural Networks
Venue: arXiv cs.CE 2022
Year: 2022

Multiscale topology optimization (M-TO) entails generating an optimal global topology, and an optimal set of microstructures at a smaller scale, for a physics-constrained problem. With the advent of additive manufacturing, M-TO has gained significant prominence. However, generating optimal microstructures at various locations can be computationally very expensive. As an alternate, graded multiscale topology optimization (GM-TO) has been proposed where one or more pre-selected and graded (parameterized) microstructural topologies are used to fill the domain optimally. This leads to a significant reduction in computation while retaining many of the benefits of M-TO. A successful GM-TO framework must: (1) be capable of efficiently handling numerous pre-selected microstructures, (2) be able to continuously switch between these microstructures during optimization, (3) ensure that the partition of unity is satisfied, and (4) discourage microstructure mixing at termination. In this paper, we propose to meet these requirements by exploiting the unique classification capacity of neural networks. Specifically, we propose a graded multiscale topology optimization using neural-network (GM-TOuNN) framework with the following features: (1) the number of design variables is only weakly dependent on the number of pre-selected microstructures, (2) it guarantees partition of unity while discouraging microstructure mixing, and (3) it supports automatic differentiation, thereby eliminating manual sensitivity analysis. The proposed framework is illustrated through several examples.
Bilateral Inversion Principles
Venue: arXiv cs.LO 2022
Year: 2022

This paper formulates a bilateral account of harmony that is an alternative to one proposed by Francez. It builds on an account of harmony for unilateral logic proposed by Kürbis and the observation that reading the rules for the connectives of bilateral logic bottom up gives the grounds and consequences of formulas with the opposite speech act. I formulate a process I call 'inversion' which allows the determination of assertive elimination rules from assertive introduction rules, and rejective elimination rules from rejective introduction rules, and conversely. It corresponds to Francez's notion of vertical harmony. I also formulate a process I call 'conversion', which allows the determination of rejective introduction rules from assertive elimination rules and conversely, and the determination of assertive introduction rules from rejective elimination rules and conversely. It corresponds to Francez's notion of horizontal harmony. The account has a number of features that distinguishes it from Francez's.
The Forms of Categorical Proposition
Venue: arXiv cs.LO 2022
Year: 2022

An exhaustive survey of categorical propositions is proposed in the present paper, both with respect to their nature and the logical problems raised by them. Through a comparative analysis of Term Logic and First-Order Logic, it is shown that the famous problem of existential import may be solved in two ways: with a model-adaptive strategy, in which the square of opposition is validated by restricting the models; with a language-adaptive strategy, in which the logical form of categorical propositions is extended in order to validate the square in every model. The latter strategy is advocated in the name of logic, which means truth in every model. Finally, the present paper needs some automatic process in order to determine the nature of logical relations between any pair of the available 256 categorical propositions. This requires the implementation of a programming machine in the style of Prolog.
Unsupervised Domain Adaptation with Implicit Pseudo Supervision for Semantic Segmentation
Venue: arXiv cs.CV 2022
Year: 2022

Pseudo-labelling is a popular technique in unsuper-vised domain adaptation for semantic segmentation. However, pseudo labels are noisy and inevitably have confirmation bias due to the discrepancy between source and target domains and training process. In this paper, we train the model by the pseudo labels which are implicitly produced by itself to learn new complementary knowledge about target domain. Specifically, we propose a tri-learning architecture, where every two branches produce the pseudo labels to train the third one. And we align the pseudo labels based on the similarity of the probability distributions for each two branches. To further implicitly utilize the pseudo labels, we maximize the distances of features for different classes and minimize the distances for the same classes by triplet loss. Extensive experiments on GTA5 to Cityscapes and SYNTHIA to Cityscapes tasks show that the proposed method has considerable improvements.
HCFL: A High Compression Approach for Communication-Efficient Federated Learning in Very Large Scale IoT Networks
Venue: arXiv cs.LG 2022
Year: 2022

Federated learning (FL) is a new artificial intelligence concept that enables Internet-of-Things (IoT) devices to learn a collaborative model without sending the raw data to centralized nodes for processing. Despite numerous advantages, low computing resources at IoT devices and high communication costs for exchanging model parameters make applications of FL in massive IoT networks very limited. In this work, we develop a novel compression scheme for FL, called high-compression federated learning (HCFL), for very large scale IoT networks. HCFL can reduce the data load for FL processes without changing their structure and hyperparameters. In this way, we not only can significantly reduce communication costs, but also make intensive learning processes more adaptable on low-computing resource IoT devices. Furthermore, we investigate a relationship between the number of IoT devices and the convergence level of the FL model and thereby better assess the quality of the FL process. We demonstrate our HCFL scheme in both simulations and mathematical analyses. Our proposed theoretical research can be used as a minimum level of satisfaction, proving that the FL process can achieve good performance when a determined configuration is met. Therefore, we show that HCFL is applicable in any FL-integrated networks with numerous IoT devices.