OpenRaiser/Intern-Atlas huggingface dataset:
A Universality-Individuality Integration Model for Dialog Act Classification
Venue: arXiv cs.CL 2022
Year: 2022

Dialog Act (DA) reveals the general intent of the speaker utterance in a conversation. Accurately predicting DAs can greatly facilitate the development of dialog agents. Although researchers have done extensive research on dialog act classification, the feature information of classification has not been fully considered. This paper suggests that word cues, part-of-speech cues and statistical cues can complement each other to improve the basis for recognition. In addition, the different types of the three lead to the diversity of their distribution forms, which hinders the mining of feature information. To solve this problem, we propose a novel model based on universality and individuality strategies, called Universality-Individuality Integration Model (UIIM). UIIM not only deepens the connection between the clues by learning universality, but also utilizes the learning of individuality to capture the characteristics of the clues themselves. Experiments were made over two most popular benchmark data sets SwDA and MRDA for dialogue act classification, and the results show that extracting the universalities and individualities between cues can more fully excavate the hidden information in the utterance, and improve the accuracy of automatic dialogue act recognition.
Probing for Constituency Structure in Neural Language Models
Venue: arXiv cs.CL 2022
Year: 2022

In this paper, we investigate to which extent contextual neural language models (LMs) implicitly learn syntactic structure. More concretely, we focus on constituent structure as represented in the Penn Treebank (PTB). Using standard probing techniques based on diagnostic classifiers, we assess the accuracy of representing constituents of different categories within the neuron activations of a LM such as RoBERTa. In order to make sure that our probe focuses on syntactic knowledge and not on implicit semantic generalizations, we also experiment on a PTB version that is obtained by randomly replacing constituents with each other while keeping syntactic structure, i.e., a semantically ill-formed but syntactically well-formed version of the PTB. We find that 4 pretrained transfomer LMs obtain high performance on our probing tasks even on manipulated data, suggesting that semantic and syntactic knowledge in their representations can be separated and that constituency information is in fact learned by the LM. Moreover, we show that a complete constituency tree can be linearly separated from LM representations.
A New Robot Arm Calibration Method Based on Cubic Interpolated Beetle Antennae Search Approach
Venue: arXiv cs.RO 2022
Year: 2022

Industrial robot arms are extensively important for intelligent manufacturing. An industrial robot arm commonly enjoys its high repetitive positioning accuracy while suffering from its low absolute positioning accuracy, which greatly restricts its application in high-precision manufacture, like automobile manufacture. Aiming at addressing this hot issue, this work proposes a novel robot arm calibration method based on cubic interpolated beetle antennae search (CIBAS). This study has three ideas: a) developing a novel CIBAS algorithm, which can effectively addresses the issue of local optimum in a Beetle Antennae Search algorithm; b) utilizing a particle filter to reduce the influence of non-Gaussian noises; and c) proposing a new calibration method incorporating CIBAS algorithm and particle filter to searching the optimal kinematic parameters. Experimental results on an ABB IRB120 industrial robot arm demonstrate that the proposed method achieves much higher calibration accuracy than several state-of-the-art calibration methods.
An Advancing Ensemble with Diversified Algorithms for Robot Arm Calibration
Venue: arXiv cs.RO 2022
Year: 2022

Recently, industrial robots plays a significant role in intelligent manufacturing. Hence, it is an urgent issue to ensure the robot with the high positioning precision. To address this hot issue, a novel calibration method based on an powerful ensemble with various algorithms is proposed. This paper has two ideas: a) developing eight calibration methods to identify the kinematic parameter errors; 2) establishing an effective ensemble to search calibrated kinematic parameters. Enough experimental results show that this ensemble can achieve: 1) higher calibration accuracy for the robot; 2) model diversity; 3) strong generalization ability.
A Novel Quadratic Interpolated Beetle Antennae Search for Manipulator Calibration
Venue: arXiv cs.RO 2022
Year: 2022

Over the past decades, industrial manipulators play a vital role in in various fields, like aircraft manufacturing and automobile manufacturing. However, an industrial manipulator without calibration suffers from its low absolute positioning accuracy, which extensively restricts its application in high-precision intelligent manufacture. Recent manipulator calibration methods are developed to address this issue, while they frequently encounter long-tail convergence and low calibration accuracy. To address this thorny issue, this work proposes a novel manipulator calibration method incorporating an extended Kalman filter with a Quadratic Interpolated Beetle Antennae Search algorithm. This paper has three-fold ideas: a) proposing a new Quadratic Interpolated Beetle Antennae Search algorithm to deal with the issue of local optimum and low convergence rate in a Beetle Antennae Search algorithm; b) adopting an extended Kalman filter algorithm to suppress non-Gaussian noises and c) developing a new manipulator calibration method incorporating an extended Kalman filter with a Quadratic Interpolated Beetle Antennae Search algorithm to calibrating a manipulator. Extensively experimental results on an ABB IRB120 industrial manipulator demonstrate that the proposed method achieves much higher calibration accuracy than several state-of-the-art calibration methods.
Research on Intellectual Property Resource Profile and Evolution Law
Venue: arXiv cs.DL 2022
Year: 2022

In the era of big data, intellectual property-oriented scientific and technological resources show the trend of large data scale, high information density and low value density, which brings severe challenges to the effective use of intellectual property resources, and the demand for mining hidden information in intellectual property is increasing. This makes intellectual property-oriented science and technology resource portraits and analysis of evolution become the current research hotspot. This paper sorts out the construction method of intellectual property resource intellectual portrait and its pre-work property entity extraction and entity completion from the aspects of algorithm classification and general process, and directions for improvement of future methods.
Assessing IT Architecture Evolution using Enriched Enterprise Architecture Models
Venue: arXiv cs.SE 2022
Year: 2022

Enterprise Architecture (EA) help companies to keep the evolution of their IT architecture aligned with their business evolution using a set of complementary models ranging from vision to infrastructure. In this paper, we explore how such models can be exploited to best drive this co-evolution by helping in validating different change management strategies. Considering state of the art techniques in service oriented and Cloud architecture, we propose to enrich and improve the quality of the information about the applications and software components on various qualities (e.g. performance, scalability, security, technical debt) in order to include them in the assessment process already at the Enterprise Architecture level rather than discovering them later in the change implementation phase. Our work is experimented on the LabNaf EA framework and is illustrated on a partial case study taken from an industrial project.
Neural Operator with Regularity Structure for Modeling Dynamics Driven by SPDEs
Venue: arXiv cs.LG 2022
Year: 2022

Stochastic partial differential equations (SPDEs) are significant tools for modeling dynamics in many areas including atmospheric sciences and physics. Neural Operators, generations of neural networks with capability of learning maps between infinite-dimensional spaces, are strong tools for solving parametric PDEs. However, they lack the ability to modeling SPDEs which usually have poor regularity due to the driving noise. As the theory of regularity structure has achieved great successes in analyzing SPDEs and provides the concept model feature vectors that well-approximate SPDEs' solutions, we propose the Neural Operator with Regularity Structure (NORS) which incorporates the feature vectors for modeling dynamics driven by SPDEs. We conduct experiments on various of SPDEs including the dynamic Phi41 model and the 2d stochastic Navier-Stokes equation, and the results demonstrate that the NORS is resolution-invariant, efficient, and achieves one order of magnitude lower error with a modest amount of data.
Towards A Critical Evaluation of Robustness for Deep Learning Backdoor Countermeasures
Venue: arXiv cs.CR 2022
Year: 2022

Since Deep Learning (DL) backdoor attacks have been revealed as one of the most insidious adversarial attacks, a number of countermeasures have been developed with certain assumptions defined in their respective threat models. However, the robustness of these countermeasures is inadvertently ignored, which can introduce severe consequences, e.g., a countermeasure can be misused and result in a false implication of backdoor detection. For the first time, we critically examine the robustness of existing backdoor countermeasures with an initial focus on three influential model-inspection ones that are Neural Cleanse (S&P'19), ABS (CCS'19), and MNTD (S&P'21). Although the three countermeasures claim that they work well under their respective threat models, they have inherent unexplored non-robust cases depending on factors such as given tasks, model architectures, datasets, and defense hyper-parameter, which are \textit{not even rooted from delicate adaptive attacks}. We demonstrate how to trivially bypass them aligned with their respective threat models by simply varying aforementioned factors. Particularly, for each defense, formal proofs or empirical studies are used to reveal its two non-robust cases where it is not as robust as it claims or expects, especially the recent MNTD. This work highlights the necessity of thoroughly evaluating the robustness of backdoor countermeasures to avoid their misleading security implications in unknown non-robust cases.
Edge Labelled Graphs and Property Graphs; a comparison from the user perspective
Venue: arXiv cs.HC 2022
Year: 2022

This study compares participant acceptance of the property graph and edge-labelled graph paradigms, as represented by Cypher and the proposed extensions to the W3C standards, RDF* and SPARQL*. In general, modelling preferences are consistent across the two paradigms. When presented with location information, participants preferred to create nodes to represent cities, rather than use metadata; although the preference was less marked for Cypher. In Cypher, participants showed little difference in preference between representing dates or population size as nodes. In RDF*, this choice was not necessary since both could be represented as literals. However, there was a significant preference for using the date as metadata to describe a triple containing population size, rather than vice versa. There was no significant difference overall in accuracy of interpretation of queries in the two paradigms; although in one specific case, the use of a reverse arrow in Cypher was interpreted significantly more accurately than the ^ symbol in SPARQL. Based on our results and on the comments of participants, we make some recommendations for modellers. Techniques for reifing RDF have attracted a great deal of research. Recently, a hybrid approach, employing some of the features of property graphs, has claimed to offer an improved technique for RDF reification. Query-time reasoning is also a requirement which has prompted a number of proposed extensions to SPARQL and which is only possible to a limited extent in the property graph paradigm. Another recent development, the hypergraph paradigm enables more powerful query-time reasoning. There is a need for more research into the user acceptance of these various more powerful approaches to modelling and querying. Such research should take account of complex modelling situations.
A "parallel universe" scheme for crack nucleation in the phase field approach to fracture
Venue: arXiv cs.CE 2022
Year: 2022

Crack nucleation is crucial in many industrial applications. The phase field method for fracture transforms the crack nucleation problem into a minimization problem of the sum of the elastic potential energy and the crack surface energy. Due to the polyconvexity of the formulation, starting from a crackless solid, a standard Newton iteration may lead to a solution with no crack, even though a cracked solution has a lower total energy. As such, the critical load for cracking is highly overestimated. Here, we propose an algorithm termed "parallel universe" algorithm to capture the global minimum. This algorithm has two key ingredients: (a) a necessary condition for cracking solely based on the current crackless solution, and (b) beginning from when this condition is met, Newton iteration with two initial guesses, a crackles one and a cracked one, will both be performed and the converged candidate solution with lower energy is accepted as the solution at that load step. Once the cracked candidate solution is accepted, the crackless one is discarded, i.e., only one universe is retained. This cracked initial guess is obtained only once for all load steps by solving a series of similar minimization problems with a progressively reduced critical crack energy release rate. Numerical examples with isotropic and anisotropic critical crack energy release rates indicate that the proposed algorithm is more reliable (as there is no need to retrace) and more efficient than the standard Newton iteration and a well-known backtracking algorithm.
Empathy-Centric Design At Scale
Venue: arXiv cs.HC 2022
Year: 2022

EmpathiCH aims at bringing together and blend different expertise to develop new research agenda in the context of "Empathy-Centric Design at Scale". The main research question is to investigate how new technologies can contribute to the elicitation of empathy across and within multiple stakeholders at scale; and how empathy can be used to design solutions to societal problems that are not only effective but also balanced, inclusive, and aware of their effect on society. Through presentations, participatory sessions, and a living experiment -- where data about the peoples' interactions is collected throughout the event -- we aim to make this workshop the ideal venue to foster collaboration, build networks, and shape the future direction of "Empathy-Centric Design at Scale".
Coverage and Capacity Optimization in STAR-RISs Assisted Networks: A Machine Learning Approach
Venue: arXiv cs.IT 2022
Year: 2022

Coverage and capacity are the important metrics for performance evaluation in wireless networks, while the coverage and capacity have several conflicting relationships, e.g. high transmit power contributes to large coverage but high inter-cell interference reduces the capacity performance. Therefore, in order to strike a balance between the coverage and capacity, a novel model is proposed for the coverage and capacity optimization of simultaneously transmitting and reflecting reconfigurable intelligent surfaces (STAR-RISs) assisted networks. To solve the coverage and capacity optimization (CCO) problem, a machine learning-based multi-objective optimization algorithm, i.e., the multi-objective proximal policy optimization (MO-PPO) algorithm, is proposed. In this algorithm, a loss function-based update strategy is the core point, which is able to calculate weights for both loss functions of coverage and capacity by a min-norm solver at each update. The numerical results demonstrate that the investigated update strategy outperforms the fixed weight-based MO algorithms.
Label Augmentation with Reinforced Labeling for Weak Supervision
Venue: arXiv cs.LG 2022
Year: 2022

Weak supervision (WS) is an alternative to the traditional supervised learning to address the need for ground truth. Data programming is a practical WS approach that allows programmatic labeling data samples using labeling functions (LFs) instead of hand-labeling each data point. However, the existing approach fails to fully exploit the domain knowledge encoded into LFs, especially when the LFs' coverage is low. This is due to the common data programming pipeline that neglects to utilize data features during the generative process. This paper proposes a new approach called reinforced labeling (RL). Given an unlabeled dataset and a set of LFs, RL augments the LFs' outputs to cases not covered by LFs based on similarities among samples. Thus, RL can lead to higher labeling coverage for training an end classifier. The experiments on several domains (classification of YouTube comments, wine quality, and weather prediction) result in considerable gains. The new approach produces significant performance improvement, leading up to +21 points in accuracy and +61 points in F1 scores compared to the state-of-the-art data programming approach.
Receptive Field Analysis of Temporal Convolutional Networks for Monaural Speech Dereverberation
Venue: arXiv cs.SD 2022
Year: 2022

Speech dereverberation is often an important requirement in robust speech processing tasks. Supervised deep learning (DL) models give state-of-the-art performance for single-channel speech dereverberation. Temporal convolutional networks (TCNs) are commonly used for sequence modelling in speech enhancement tasks. A feature of TCNs is that they have a receptive field (RF) dependent on the specific model configuration which determines the number of input frames that can be observed to produce an individual output frame. It has been shown that TCNs are capable of performing dereverberation of simulated speech data, however a thorough analysis, especially with focus on the RF is yet lacking in the literature. This paper analyses dereverberation performance depending on the model size and the RF of TCNs. Experiments using the WHAMR corpus which is extended to include room impulse responses (RIRs) with larger T60 values demonstrate that a larger RF can have significant improvement in performance when training smaller TCN models. It is also demonstrated that TCNs benefit from a wider RF when dereverberating RIRs with larger RT60 values.
Study of Indian English Pronunciation Variabilities relative to Received Pronunciation
Venue: arXiv cs.CL 2022
Year: 2022

Analysis of Indian English (IE) pronunciation variabilities are useful in building systems for Automatic Speech Recognition (ASR) and Text-to-Speech (TTS) synthesis in the Indian context. Typically, these pronunciation variabilities have been explored by comparing IE pronunciation with Received Pronunciation (RP). However, to explore these variabilities, it is required to have labelled pronunciation data at the phonetic level, which is scarce for IE. Moreover, versatility of IE stems from the influence of a large diversity of the speakers' mother tongues and demographic region differences. Prior linguistic works have characterised features of IE variabilities qualitatively by reporting phonetic rules that represent such variations relative to RP. The qualitative descriptions often lack quantitative descriptors and data-driven analysis of diverse IE pronunciation data to characterise IE on the phonetic level. To address these issues, in this work, we consider a corpus, Indic TIMIT, containing a large set of IE varieties from 80 speakers from various regions of India. We present an analysis to obtain the new set of phonetic rules representing IE pronunciation variabilities relative to RP in a data-driven manner. We do this using 15,974 phonetic transcriptions, of which 13,632 were obtained manually in addition to those part of the corpus. Furthermore, we validate the rules obtained from the analysis against the existing phonetic rules to identify the relevance of the obtained phonetic rules and test the efficacy of Grapheme-to-Phoneme (G2P) conversion developed based on the obtained rules considering Phoneme Error Rate (PER) as the metric for performance.
DL4SciVis: A State-of-the-Art Survey on Deep Learning for Scientific Visualization
Venue: arXiv cs.GR 2022
Year: 2022

Since 2016, we have witnessed the tremendous growth of artificial intelligence+visualization (AI+VIS) research. However, existing survey papers on AI+VIS focus on visual analytics and information visualization, not scientific visualization (SciVis). In this paper, we survey related deep learning (DL) works in SciVis, specifically in the direction of DL4SciVis: designing DL solutions for solving SciVis problems. To stay focused, we primarily consider works that handle scalar and vector field data but exclude mesh data. We classify and discuss these works along six dimensions: domain setting, research task, learning type, network architecture, loss function, and evaluation metric. The paper concludes with a discussion of the remaining gaps to fill along the discussed dimensions and the grand challenges we need to tackle as a community. This state-of-the-art survey guides SciVis researchers in gaining an overview of this emerging topic and points out future directions to grow this research.
Learning Self-Modulating Attention in Continuous Time Space with Applications to Sequential Recommendation
Venue: arXiv cs.IR 2022
Year: 2022

User interests are usually dynamic in the real world, which poses both theoretical and practical challenges for learning accurate preferences from rich behavior data. Among existing user behavior modeling solutions, attention networks are widely adopted for its effectiveness and relative simplicity. Despite being extensively studied, existing attentions still suffer from two limitations: i) conventional attentions mainly take into account the spatial correlation between user behaviors, regardless the distance between those behaviors in the continuous time space; and ii) these attentions mostly provide a dense and undistinguished distribution over all past behaviors then attentively encode them into the output latent representations. This is however not suitable in practical scenarios where a user's future actions are relevant to a small subset of her/his historical behaviors. In this paper, we propose a novel attention network, named self-modulating attention, that models the complex and non-linearly evolving dynamic user preferences. We empirically demonstrate the effectiveness of our method on top-N sequential recommendation tasks, and the results on three large-scale real-world datasets show that our model can achieve state-of-the-art performance.