OpenRaiser/Intern-Atlas huggingface dataset:
Equine Pain Behavior Classification via Self-Supervised Disentangled Pose Representation
Venue: arXiv cs.CV 2021
Year: 2021

Timely detection of horse pain is important for equine welfare. Horses express pain through their facial and body behavior, but may hide signs of pain from unfamiliar human observers. In addition, collecting visual data with detailed annotation of horse behavior and pain state is both cumbersome and not scalable. Consequently, a pragmatic equine pain classification system would use video of the unobserved horse and weak labels. This paper proposes such a method for equine pain classification by using multi-view surveillance video footage of unobserved horses with induced orthopaedic pain, with temporally sparse video level pain labels. To ensure that pain is learned from horse body language alone, we first train a self-supervised generative model to disentangle horse pose from its appearance and background before using the disentangled horse pose latent representation for pain classification. To make best use of the pain labels, we develop a novel loss that formulates pain classification as a multi-instance learning problem. Our method achieves pain classification accuracy better than human expert performance with 60% accuracy. The learned latent horse pose representation is shown to be viewpoint covariant, and disentangled from horse appearance. Qualitative analysis of pain classified segments shows correspondence between the pain symptoms identified by our model, and equine pain scales used in veterinary practice.
Making the Invisible Visible: Risks and Benefits of Disclosing Metadata in Visualization
Venue: arXiv cs.HC 2021
Year: 2021

Accompanying a data visualization with metadata may benefit readers by facilitating content understanding, strengthening trust, and providing accountability. However, providing this kind of information may also have negative, unintended consequences, such as biasing readers' interpretations, a loss of trust as a result of too much transparency, and the possibility of opening visualization creators with minoritized identities up to undeserved critique. To help future visualization researchers and practitioners decide what kinds of metadata to include, we discuss some of the potential benefits and risks of disclosing five kinds of metadata: metadata about the source of the underlying data; the cleaning and processing conducted; the marks, channels, and other design elements used; the people who directly created the visualization; and the people for whom the visualization was created. We conclude by proposing a few open research questions related to how to communicate metadata about visualizations.
The effects of data size on Automated Essay Scoring engines
Venue: arXiv cs.CL 2021
Year: 2021

We study the effects of data size and quality on the performance on Automated Essay Scoring (AES) engines that are designed in accordance with three different paradigms; A frequency and hand-crafted feature-based model, a recurrent neural network model, and a pretrained transformer-based language model that is fine-tuned for classification. We expect that each type of model benefits from the size and the quality of the training data in very different ways. Standard practices for developing training data for AES engines were established with feature-based methods in mind, however, since neural networks are increasingly being considered in a production setting, this work seeks to inform us as to how to establish better training data for neural networks that will be used in production.
VTLayout: Fusion of Visual and Text Features for Document Layout Analysis
Venue: arXiv cs.IR 2021
Year: 2021

Documents often contain complex physical structures, which make the Document Layout Analysis (DLA) task challenging. As a pre-processing step for content extraction, DLA has the potential to capture rich information in historical or scientific documents on a large scale. Although many deep-learning-based methods from computer vision have already achieved excellent performance in detecting \emph{Figure} from documents, they are still unsatisfactory in recognizing the \emph{List}, \emph{Table}, \emph{Text} and \emph{Title} category blocks in DLA. This paper proposes a VTLayout model fusing the documents' deep visual, shallow visual, and text features to localize and identify different category blocks. The model mainly includes two stages, and the three feature extractors are built in the second stage. In the first stage, the Cascade Mask R-CNN model is applied directly to localize all category blocks of the documents. In the second stage, the deep visual, shallow visual, and text features are extracted for fusion to identify the category blocks of documents. As a result, we strengthen the classification power of different category blocks based on the existing localization technique. The experimental results show that the identification capability of the VTLayout is superior to the most advanced method of DLA based on the PubLayNet dataset, and the F1 score is as high as 0.9599.
Extracting Qualitative Causal Structure with Transformer-Based NLP
Venue: arXiv cs.LG 2021
Year: 2021

Qualitative causal relationships compactly express the direction, dependency, temporal constraints, and monotonicity constraints of discrete or continuous interactions in the world. In everyday or academic language, we may express interactions between quantities (e.g., sleep decreases stress), between discrete events or entities (e.g., a protein inhibits another protein's transcription), or between intentional or functional factors (e.g., hospital patients pray to relieve their pain). This paper presents a transformer-based NLP architecture that jointly identifies and extracts (1) variables or factors described in language, (2) qualitative causal relationships over these variables, and (3) qualifiers and magnitudes that constrain these causal relationships. We demonstrate this approach and include promising results from in two use cases, processing textual inputs from academic publications, news articles, and social media.
DNNFusion: Accelerating Deep Neural Networks Execution with Advanced Operator Fusion
Venue: arXiv cs.LG 2021
Year: 2021

Deep Neural Networks (DNNs) have emerged as the core enabler of many major applications on mobile devices. To achieve high accuracy, DNN models have become increasingly deep with hundreds or even thousands of operator layers, leading to high memory and computational requirements for inference. Operator fusion (or kernel/layer fusion) is key optimization in many state-of-the-art DNN execution frameworks, such as TensorFlow, TVM, and MNN. However, these frameworks usually adopt fusion approaches based on certain patterns that are too restrictive to cover the diversity of operators and layer connections. Polyhedral-based loop fusion techniques, on the other hand, work on a low-level view of the computation without operator-level information, and can also miss potential fusion opportunities. To address this challenge, this paper proposes a novel and extensive loop fusion framework called DNNFusion. The basic idea of this work is to work at an operator view of DNNs, but expand fusion opportunities by developing a classification of both individual operators and their combinations. In addition, DNNFusion includes 1) a novel mathematical-property-based graph rewriting framework to reduce evaluation costs and facilitate subsequent operator fusion, 2) an integrated fusion plan generation that leverages the high-level analysis and accurate light-weight profiling, and 3) additional optimizations during fusion code generation. DNNFusion is extensively evaluated on 15 DNN models with varied types of tasks, model sizes, and layer counts. The evaluation results demonstrate that DNNFusion finds up to 8.8x higher fusion opportunities, outperforms four state-of-the-art DNN execution frameworks with 9.3x speedup. The memory requirement reduction and speedups can enable the execution of many of the target models on mobile devices and even make them part of a real-time application.
A Mathematical Walkthrough and Discussion of the Free Energy Principle
Venue: arXiv cs.AI 2021
Year: 2021

The Free-Energy-Principle (FEP) is an influential and controversial theory which postulates a deep and powerful connection between the stochastic thermodynamics of self-organization and learning through variational inference. Specifically, it claims that any self-organizing system which can be statistically separated from its environment, and which maintains itself at a non-equilibrium steady state, can be construed as minimizing an information-theoretic functional -- the variational free energy -- and thus performing variational Bayesian inference to infer the hidden state of its environment. This principle has also been applied extensively in neuroscience, and is beginning to make inroads in machine learning by spurring the construction of novel and powerful algorithms by which action, perception, and learning can all be unified under a single objective. While its expansive and often grandiose claims have spurred significant debates in both philosophy and theoretical neuroscience, the mathematical depth and lack of accessible introductions and tutorials for the core claims of the theory have often precluded a deep understanding within the literature. Here, we aim to provide a mathematically detailed, yet intuitive walk-through of the formulation and central claims of the FEP while also providing a discussion of the assumptions necessary and potential limitations of the theory. Additionally, since the FEP is a still a living theory, subject to internal controversy, change, and revision, we also present a detailed appendix highlighting and condensing current perspectives as well as controversies about the nature, applicability, and the mathematical assumptions and formalisms underlying the FEP.
Trustworthy AI for Process Automation on a Chylla-Haase Polymerization Reactor
Venue: arXiv cs.AI 2021
Year: 2021

In this paper, genetic programming reinforcement learning (GPRL) is utilized to generate human-interpretable control policies for a Chylla-Haase polymerization reactor. Such continuously stirred tank reactors (CSTRs) with jacket cooling are widely used in the chemical industry, in the production of fine chemicals, pigments, polymers, and medical products. Despite appearing rather simple, controlling CSTRs in real-world applications is quite a challenging problem to tackle. GPRL utilizes already existing data from the reactor and generates fully automatically a set of optimized simplistic control strategies, so-called policies, the domain expert can choose from. Note that these policies are white-box models of low complexity, which makes them easy to validate and implement in the target control system, e.g., SIMATIC PCS 7. However, despite its low complexity the automatically-generated policy yields a high performance in terms of reactor temperature control deviation, which we empirically evaluate on the original reactor template.
Exploring Multi-Tasking Learning in Document Attribute Classification
Venue: arXiv cs.CV 2021
Year: 2021

In this work, we adhere to explore a Multi-Tasking learning (MTL) based network to perform document attribute classification such as the font type, font size, font emphasis and scanning resolution classification of a document image. To accomplish these tasks, we operate on either segmented word level or on uniformed size patches randomly cropped out of the document. Furthermore, a hybrid convolution neural network (CNN) architecture "MTL+MI", which is based on the combination of MTL and Multi-Instance (MI) of patch and word is used to accomplish joint learning for the classification of the same document attributes. The contribution of this paper are three fold: firstly, based on segmented word images and patches, we present a MTL based network for the classification of a full document image. Secondly, we propose a MTL and MI (using segmented words and patches) based combined CNN architecture ("MTL+MI") for the classification of same document attributes. Thirdly, based on the multi-tasking classifications of the words and/or patches, we propose an intelligent voting system which is based on the posterior probabilities of each words and/or patches to perform the classification of document's attributes of complete document image.
Full-Cycle Energy Consumption Benchmark for Low-Carbon Computer Vision
Venue: arXiv cs.CV 2021
Year: 2021

The energy consumption of deep learning models is increasing at a breathtaking rate, which raises concerns due to potential negative effects on carbon neutrality in the context of global warming and climate change. With the progress of efficient deep learning techniques, e.g., model compression, researchers can obtain efficient models with fewer parameters and smaller latency. However, most of the existing efficient deep learning methods do not explicitly consider energy consumption as a key performance indicator. Furthermore, existing methods mostly focus on the inference costs of the resulting efficient models, but neglect the notable energy consumption throughout the entire life cycle of the algorithm. In this paper, we present the first large-scale energy consumption benchmark for efficient computer vision models, where a new metric is proposed to explicitly evaluate the full-cycle energy consumption under different model usage intensity. The benchmark can provide insights for low carbon emission when selecting efficient deep learning algorithms in different model usage scenarios.
ConVIScope: Visual Analytics for Exploring Patient Conversations
Venue: arXiv cs.HC 2021
Year: 2021

The proliferation of text messaging for mobile health is generating a large amount of patient-doctor conversations that can be extremely valuable to health care professionals. We present ConVIScope, a visual text analytic system that tightly integrates interactive visualization with natural language processing in analyzing patient-doctor conversations. ConVIScope was developed in collaboration with healthcare professionals following a user-centered iterative design. Case studies with six domain experts suggest the potential utility of ConVIScope and reveal lessons for further developments.
A Convolutional Neural Network-based Approach to Field Reconstruction
Venue: arXiv cs.LG 2021
Year: 2021

This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible. In many applications, the spatial distribution of a field needs to be carefully monitored to detect spikes, discontinuities or dangerous heterogeneities, but invasive monitoring approaches cannot be used. Besides, technical specifications about the process might not be available by preventing the adoption of an accurate model of the system. In this work, a physics-informed, data-driven algorithm that allows addressing these requirements is presented. The approach is based on the implementation of a boundary element method (BEM)-scheme within a convolutional neural network. Thanks to the capability of representing any continuous mathematical function with a reduced number of parameters, the network allows predicting the field value in any point of the domain, given the boundary conditions and few measurements within the domain. The proposed approach was applied to reconstruct a field described by the Helmholtz equation over a three-dimensional domain. A sensitivity analysis was also performed by investigating different physical conditions and different network configurations. Since the only assumption is the applicability of BEM, the current approach can be applied to the monitoring of a wide range of processes, from the localization of the source of pollutant within a water reservoir to the monitoring of the neutron flux in a nuclear reactor.
Upper bounds on the length function for covering codes with covering radius $R$ and codimension $tR+1$
Venue: arXiv cs.IT 2021
Year: 2021

The length function $\ell_q(r,R)$ is the smallest length of a $ q $-ary linear code with codimension (redundancy) $r$ and covering radius $R$. In this work, new upper bounds on $\ell_q(tR+1,R)$ are obtained in the following forms: \begin{equation*} \begin{split} &(a)~\ell_q(r,R)\le cq^{(r-R)/R}\cdot\sqrt[R]{\ln q},~ R\ge3,~r=tR+1,~t\ge1, &\phantom{(a)~} q\text{ is an arbitrary prime power},~c\text{ is independent of }q. \end{split} \end{equation*} \begin{equation*} \begin{split} &(b)~\ell_q(r,R)< 3.43Rq^{(r-R)/R}\cdot\sqrt[R]{\ln q},~ R\ge3,~r=tR+1,~t\ge1, &\phantom{(b)~} q\text{ is an arbitrary prime power},~q\text{ is large enough}. \end{split} \end{equation*} In the literature, for $q=(q')^R$ with $q'$ a prime power, smaller upper bounds are known; however, when $q$ is an arbitrary prime power, the bounds of this paper are better than the known ones. For $t=1$, we use a one-to-one correspondence between $[n,n-(R+1)]_qR$ codes and $(R-1)$-saturating $n$-sets in the projective space $\mathrm{PG}(R,q)$. A new construction of such saturating sets providing sets of small size is proposed. Then the $[n,n-(R+1)]_qR$ codes, obtained by geometrical methods, are taken as the starting ones in the lift-constructions (so-called "$q^m$-concatenating constructions") for covering codes to obtain infinite families of codes with growing codimension $r=tR+1$, $t\ge1$.
Spike time displacement based error backpropagation in convolutional spiking neural networks
Venue: arXiv cs.NE 2021
Year: 2021

We recently proposed the STiDi-BP algorithm, which avoids backward recursive gradient computation, for training multi-layer spiking neural networks (SNNs) with single-spike-based temporal coding. The algorithm employs a linear approximation to compute the derivative of the spike latency with respect to the membrane potential and it uses spiking neurons with piecewise linear postsynaptic potential to reduce the computational cost and the complexity of neural processing. In this paper, we extend the STiDi-BP algorithm to employ it in deeper and convolutional architectures. The evaluation results on the image classification task based on two popular benchmarks, MNIST and Fashion-MNIST datasets with the accuracies of respectively 99.2% and 92.8%, confirm that this algorithm has been applicable in deep SNNs. Another issue we consider is the reduction of memory storage and computational cost. To do so, we consider a convolutional SNN (CSNN) with two sets of weights: real-valued weights that are updated in the backward pass and their signs, binary weights, that are employed in the feedforward process. We evaluate the binary CSNN on two datasets of MNIST and Fashion-MNIST and obtain acceptable performance with a negligible accuracy drop with respect to real-valued weights (about $0.6%$ and $0.8%$ drops, respectively).
Gray Cycles of Maximum Length Related to k-Character Substitutions
Venue: arXiv cs.CL 2021
Year: 2021

Given a word binary relation $τ$ we define a $τ$-Gray cycle over a finite language X to be a permutation w [i] 0$\le$i$\le$|X|--1 of X such that each word wi is an image of the previous word wi--1 by $τ$. In that framework, we introduce the complexity measure $λ$(n), equal to the largest cardinality of a language X having words of length at most n, and such that a $τ$-Gray cycle over X exists. The present paper is concerned with the relation $τ$ = $σ$ k , the so-called k-character substitution, where (u, v) belongs to $σ$ k if, and only if, the Hamming distance of u and v is k. We compute the bound $λ$(n) for all cases of the alphabet cardinality and the argument n.
Plan-then-Generate: Controlled Data-to-Text Generation via Planning
Venue: arXiv cs.CL 2021
Year: 2021

Recent developments in neural networks have led to the advance in data-to-text generation. However, the lack of ability of neural models to control the structure of generated output can be limiting in certain real-world applications. In this study, we propose a novel Plan-then-Generate (PlanGen) framework to improve the controllability of neural data-to-text models. Extensive experiments and analyses are conducted on two benchmark datasets, ToTTo and WebNLG. The results show that our model is able to control both the intra-sentence and inter-sentence structure of the generated output. Furthermore, empirical comparisons against previous state-of-the-art methods show that our model improves the generation quality as well as the output diversity as judged by human and automatic evaluations.
DLPFS: The Data Leakage Prevention FileSystem
Venue: arXiv cs.CR 2021
Year: 2021

Shared folders are still a common practice for granting third parties access to data files, regardless of the advances in data sharing technologies. Services like Google Drive, Dropbox, Box, and others, provide infrastructures and interfaces to manage file sharing. The human factor is the weakest link and data leaks caused by human error are regrettable common news. This takes place as both mishandled data, for example stored to the wrong directory, or via misconfigured or failing applications dumping data incorrectly. We present Data Leakage Prevention FileSystem (DLPFS), a first attempt to systematically protect against data leakage caused by misconfigured application or human error. This filesystem interface provides a privacy protection layer on top of the POSIX filesystem interface, allowing for seamless integration with existing infrastructures and applications, simply augmenting existing security controls. At the same time, DLPFS allows data administrators to protect files shared within an organisation by preventing unauthorised parties to access potentially sensitive content. DLPFS achieves this by transparently integrating with existing access control mechanisms. We empirically evaluate the impact of DLPFS on system's performances to demonstrate the feasibility of the proposed solution.
Network psychometrics and cognitive network science open new ways for detecting, understanding and tackling the complexity of math anxiety: A review
Venue: arXiv cs.SI 2021
Year: 2021

Math anxiety is a clinical pathology impairing cognitive processing in math-related contexts. Originally thought to affect only inexperienced, low-achieving students, recent investigations show how math anxiety is vastly diffused even among high-performing learners. This review of data-informed studies outlines math anxiety as a complex system that: (i) cripples well-being, self-confidence and information processing on both conscious and subconscious levels, (ii) can be transmitted by social interactions, like a pathogen, and worsened by distorted perceptions, (iii) affects roughly 20% of students in 63 out of 64 worldwide educational systems but correlates weakly with academic performance, and (iv) poses a concrete threat to students' well-being, computational literacy and career prospects in science. These patterns underline the crucial need to go beyond performance for estimating math anxiety. Recent advances with network psychometrics and cognitive network science provide ideal frameworks for detecting, interpreting and intervening upon such clinical condition. Merging education research, psychology and data science, the approaches reviewed here reconstruct psychological constructs as complex systems, represented either as multivariate correlation models (e.g. graph exploratory analysis) or as cognitive networks of semantic/emotional associations (e.g. free association networks or forma mentis networks). Not only can these interconnected networks detect otherwise hidden levels of math anxiety but - more crucially - they can unveil the specific layout of interacting factors, e.g. key sources and targets, behind math anxiety in a given cohort. As discussed here, these network approaches open concrete ways for unveiling students' perceptions, emotions and mental well-being, and can enable future powerful data-informed interventions untangling math anxiety.