OpenRaiser/Intern-Atlas huggingface dataset:
TabAug: Data Driven Augmentation for Enhanced Table Structure Recognition
Venue: arXiv cs.CV 2021
Year: 2021

Table Structure Recognition is an essential part of end-to-end tabular data extraction in document images. The recent success of deep learning model architectures in computer vision remains to be non-reflective in table structure recognition, largely because extensive datasets for this domain are still unavailable while labeling new data is expensive and time-consuming. Traditionally, in computer vision, these challenges are addressed by standard augmentation techniques that are based on image transformations like color jittering and random cropping. As demonstrated by our experiments, these techniques are not effective for the task of table structure recognition. In this paper, we propose TabAug, a re-imagined Data Augmentation technique that produces structural changes in table images through replication and deletion of rows and columns. It also consists of a data-driven probabilistic model that allows control over the augmentation process. To demonstrate the efficacy of our approach, we perform experimentation on ICDAR 2013 dataset where our approach shows consistent improvements in all aspects of the evaluation metrics, with cell-level correct detections improving from 92.16% to 96.11% over the baseline.
Axiomatizations and Computability of Weighted Monadic Second-Order Logic
Venue: arXiv cs.LO 2021
Year: 2021

Weighted monadic second-order logic is a weighted extension of monadic second-order logic that captures exactly the behaviour of weighted automata. Its semantics is parameterized with respect to a semiring on which the values that weighted formulas output are evaluated. Gastin and Monmege (2018) gave abstract semantics for a version of weighted monadic second-order logic to give a more general and modular proof of the equivalence of the logic with weighted automata. We focus on the abstract semantics of the logic and we give a complete axiomatization both for the full logic and for a fragment without general sum, thus giving a more fine-grained understanding of the logic. We discuss how common decision problems for logical languages can be adapted to the weighted setting, and show that many of these are decidable, though they inherit bad complexity from the underlying first- and second-order logics. However, we show that a weighted adaptation of satisfiability is undecidable for the logic when one uses the abstract interpretation.
Source Seeking Control of Unicycle Robots with 3D-printed Flexible Piezoresistive Sensors
Venue: arXiv cs.RO 2021
Year: 2021

We present the design and experimental validation of source seeking control algorithms for a unicycle mobile robot that is equipped with novel 3D-printed flexible graphene-based piezoresistive airflow sensors. Based solely on a local gradient measurement from the airflow sensors, we propose and analyze a projected gradient ascent algorithm to solve the source seeking problem. In the case of partial sensor failure, we propose a combination of Extremum-Seeking Control with our projected gradient ascent algorithm. For both control laws, we prove the asymptotic convergence of the robot to the source. Numerical simulations were performed to validate the algorithms and experimental validations are presented to demonstrate the efficacy of the proposed methods.
Relevance Detection in Cataract Surgery Videos by Spatio-Temporal Action Localization
Venue: arXiv cs.CV 2021
Year: 2021

In cataract surgery, the operation is performed with the help of a microscope. Since the microscope enables watching real-time surgery by up to two people only, a major part of surgical training is conducted using the recorded videos. To optimize the training procedure with the video content, the surgeons require an automatic relevance detection approach. In addition to relevance-based retrieval, these results can be further used for skill assessment and irregularity detection in cataract surgery videos. In this paper, a three-module framework is proposed to detect and classify the relevant phase segments in cataract videos. Taking advantage of an idle frame recognition network, the video is divided into idle and action segments. To boost the performance in relevance detection, the cornea where the relevant surgical actions are conducted is detected in all frames using Mask R-CNN. The spatiotemporally localized segments containing higher-resolution information about the pupil texture and actions, and complementary temporal information from the same phase are fed into the relevance detection module. This module consists of four parallel recurrent CNNs being responsible to detect four relevant phases that have been defined with medical experts. The results will then be integrated to classify the action phases as irrelevant or one of four relevant phases. Experimental results reveal that the proposed approach outperforms static CNNs and different configurations of feature-based and end-to-end recurrent networks.
Multi-class Text Classification using BERT-based Active Learning
Venue: arXiv cs.IR 2021
Year: 2021

Text Classification finds interesting applications in the pickup and delivery services industry where customers require one or more items to be picked up from a location and delivered to a certain destination. Classifying these customer transactions into multiple categories helps understand the market needs for different customer segments. Each transaction is accompanied by a text description provided by the customer to describe the products being picked up and delivered which can be used to classify the transaction. BERT-based models have proven to perform well in Natural Language Understanding. However, the product descriptions provided by the customers tend to be short, incoherent and code-mixed (Hindi-English) text which demands fine-tuning of such models with manually labelled data to achieve high accuracy. Collecting this labelled data can prove to be expensive. In this paper, we explore Active Learning strategies to label transaction descriptions cost effectively while using BERT to train a transaction classification model. On TREC-6, AG's News Corpus and an internal dataset, we benchmark the performance of BERT across different Active Learning strategies in Multi-Class Text Classification.
Computing Possible and Necessary Equilibrium Actions (and Bipartisan Set Winners)
Venue: arXiv cs.GT 2021
Year: 2021

In many multiagent environments, a designer has some, but limited control over the game being played. In this paper, we formalize this by considering incompletely specified games, in which some entries of the payoff matrices can be chosen from a specified set. We show that it is NP-hard for the designer to make these choices optimally, even in zero-sum games. In fact, it is already intractable to decide whether a given action is (potentially or necessarily) played in equilibrium. We also consider incompletely specified symmetric games in which all completions are required to be symmetric. Here, hardness holds even in weak tournament games (symmetric zero-sum games whose entries are all -1, 0, or 1) and in tournament games (symmetric zero-sum games whose non-diagonal entries are all -1 or 1). The latter result settles the complexity of the possible and necessary winner problems for a social-choice-theoretic solution concept known as the bipartisan set. We finally give a mixed-integer linear programming formulation for weak tournament games and evaluate it experimentally.
Bridging the gap between streaming and non-streaming ASR systems bydistilling ensembles of CTC and RNN-T models
Venue: arXiv cs.CL 2021
Year: 2021

Streaming end-to-end automatic speech recognition (ASR) systems are widely used in everyday applications that require transcribing speech to text in real-time. Their minimal latency makes them suitable for such tasks. Unlike their non-streaming counterparts, streaming models are constrained to be causal with no future context and suffer from higher word error rates (WER). To improve streaming models, a recent study [1] proposed to distill a non-streaming teacher model on unsupervised utterances, and then train a streaming student using the teachers' predictions. However, the performance gap between teacher and student WERs remains high. In this paper, we aim to close this gap by using a diversified set of non-streaming teacher models and combining them using Recognizer Output Voting Error Reduction (ROVER). In particular, we show that, despite being weaker than RNN-T models, CTC models are remarkable teachers. Further, by fusing RNN-T and CTC models together, we build the strongest teachers. The resulting student models drastically improve upon streaming models of previous work [1]: the WER decreases by 41% on Spanish, 27% on Portuguese, and 13% on French.
BlockColdChain: Vaccine Cold Chain Blockchain
Venue: arXiv cs.CR 2021
Year: 2021

In this paper, we propose a blockchain-based cold chain technology for vaccine cooling track. The COVID-19 pandemic has caused the death of millions of people. An important step towards ending the pandemic is vaccination. Vaccines must be kept under control temperature during the whole process, from fabrication to the hands of the health professionals who will immunize the population. However, there are numerous reports of vaccine loss due to temperature variations, and, currently, people getting vaccinated have no control if their vaccine was kept safe. Blockchain is a technology solution that can provide public and verifiable records. We review the World Health Organization (WHO) cool chain and Blockchain technology. Moreover, we describe current IoT temperature monitoring devices and propose Blockcoldchain to track vaccine cold chain using blockchain, thus proving an unalterable vaccine temperature history. Our experimental results using smart contracts demonstrate the system's feasibility.
Learning Robust Variational Information Bottleneck with Reference
Venue: arXiv cs.LG 2021
Year: 2021

We propose a new approach to train a variational information bottleneck (VIB) that improves its robustness to adversarial perturbations. Unlike the traditional methods where the hard labels are usually used for the classification task, we refine the categorical class information in the training phase with soft labels which are obtained from a pre-trained reference neural network and can reflect the likelihood of the original class labels. We also relax the Gaussian posterior assumption in the VIB implementation by using the mutual information neural estimation. Extensive experiments have been performed with the MNIST and CIFAR-10 datasets, and the results show that our proposed approach significantly outperforms the benchmarked models.
A 0.502$\cdot$MaxCut Approximation using Quadratic Programming
Venue: arXiv cs.DS 2021
Year: 2021

We study the MaxCut problem for graphs $G=(V,E)$. The problem is NP-hard, there are two main approximation algorithms with theoretical guarantees: (1) the Goemans \& Williamson algorithm uses semi-definite programming to provide a 0.878MaxCut approximation (which, if the Unique Games Conjecture is true, is the best that can be done in polynomial time) and (2) Trevisan proposed an algorithm using spectral graph theory from which a 0.614MaxCut approximation can be obtained. We discuss a new approach using a specific quadratic program and prove that its solution can be used to obtain at least a 0.502MaxCut approximation. The algorithm seems to perform well in practice.
Dynamical prediction of two meteorological factors using the deep neural network and the long short-term memory $(2)$
Venue: arXiv cs.LG 2021
Year: 2021

This paper presents the predictive accuracy using two-variate meteorological factors, average temperature and average humidity, in neural network algorithms. We analyze result in five learning architectures such as the traditional artificial neural network, deep neural network, and extreme learning machine, long short-term memory, and long-short-term memory with peephole connections, after manipulating the computer-simulation. Our neural network modes are trained on the daily time-series dataset during seven years (from 2014 to 2020). From the trained results for 2500, 5000, and 7500 epochs, we obtain the predicted accuracies of the meteorological factors produced from outputs in ten metropolitan cities (Seoul, Daejeon, Daegu, Busan, Incheon, Gwangju, Pohang, Mokpo, Tongyeong, and Jeonju). The error statistics is found from the result of outputs, and we compare these values to each other after the manipulation of five neural networks. As using the long-short-term memory model in testing 1 (the average temperature predicted from the input layer with six input nodes), Tonyeong has the lowest root mean squared error (RMSE) value of 0.866 $(%)$ in summer from the computer-simulation in order to predict the temperature. To predict the humidity, the RMSE is shown the lowest value of 5.732 $(%)$, when using the long short-term memory model in summer in Mokpo in testing 2 (the average humidity predicted from the input layer with six input nodes). Particularly, the long short-term memory model is is found to be more accurate in forecasting daily levels than other neural network models in temperature and humidity forecastings. Our result may provide a computer-simuation basis for the necessity of exploring and develping a novel neural network evaluation method in the future.
Machine-Learning Assisted Optimization Strategies for Phase Change Materials Embedded within Electronic Packages
Venue: arXiv cs.CE 2021
Year: 2021

Leveraging the latent heat of phase change materials (PCMs) can reduce the peak temperatures and transient variations in temperature in electronic devices. But as the power levels increase, the thermal conduction pathway from the heat source to the heat sink limits the effectiveness of these systems. In this work, we evaluate embedding the PCM within the silicon device layer of an electronic device to minimize the thermal resistance between the source and the PCM to minimize this thermal resistance and enhance the thermal performance of the device. The geometry and material properties of the embedded PCM regions are optimized using a combination of parametric and machine learning algorithms. For a fixed geometry, considering commercially available materials, Solder 174 significantly outperforms other organic and metallic PCMs. Also with a fixed geometry, the optimal melting points to minimize the peak temperature is higher than the optimal melting point to minimize the amplitude of the transient temperature oscillation, and both optima increase with increasing heater power. Extending beyond conventional optimization strategies, genetic algorithms and particle swarm optimization with and without neural network surrogate models are used to enable optimization of many geometric and material properties. For the test case evaluated, the optimized geometries and properties are similar between all ML-assisted algorithms, but the computational time depends on the technique. Ultimately, the optimized design with embedded phase change materials reduces the maximum temperature rise by 19% and the fluctuations by up to 88% compared to devices without PCM.
Towards a practical lip-to-speech conversion system using deep neural networks and mobile application frontend
Venue: arXiv cs.CV 2021
Year: 2021

Articulatory-to-acoustic (forward) mapping is a technique to predict speech using various articulatory acquisition techniques as input (e.g. ultrasound tongue imaging, MRI, lip video). The advantage of lip video is that it is easily available and affordable: most modern smartphones have a front camera. There are already a few solutions for lip-to-speech synthesis, but they mostly concentrate on offline training and inference. In this paper, we propose a system built from a backend for deep neural network training and inference and a fronted as a form of a mobile application. Our initial evaluation shows that the scenario is feasible: a top-5 classification accuracy of 74% is combined with feedback from the mobile application user, making sure that the speaking impaired might be able to communicate with this solution.
Experts, Errors, and Context: A Large-Scale Study of Human Evaluation for Machine Translation
Venue: arXiv cs.CL 2021
Year: 2021

Human evaluation of modern high-quality machine translation systems is a difficult problem, and there is increasing evidence that inadequate evaluation procedures can lead to erroneous conclusions. While there has been considerable research on human evaluation, the field still lacks a commonly-accepted standard procedure. As a step toward this goal, we propose an evaluation methodology grounded in explicit error analysis, based on the Multidimensional Quality Metrics (MQM) framework. We carry out the largest MQM research study to date, scoring the outputs of top systems from the WMT 2020 shared task in two language pairs using annotations provided by professional translators with access to full document context. We analyze the resulting data extensively, finding among other results a substantially different ranking of evaluated systems from the one established by the WMT crowd workers, exhibiting a clear preference for human over machine output. Surprisingly, we also find that automatic metrics based on pre-trained embeddings can outperform human crowd workers. We make our corpus publicly available for further research.
Questioning causality on sex, gender and COVID-19, and identifying bias in large-scale data-driven analyses: the Bias Priority Recommendations and Bias Catalog for Pandemics
Venue: arXiv cs.CY 2021
Year: 2021

The COVID-19 pandemic has spurred a large amount of observational studies reporting linkages between the risk of developing severe COVID-19 or dying from it, and sex and gender. By reviewing a large body of related literature and conducting a fine grained analysis based on sex-disaggregated data of 61 countries spanning 5 continents, we discover several confounding factors that could possibly explain the supposed male vulnerability to COVID-19. We thus highlight the challenge of making causal claims based on available data, given the lack of statistical significance and potential existence of biases. Informed by our findings on potential variables acting as confounders, we contribute a broad overview on the issues bias, explainability and fairness entail in data-driven analyses. Thus, we outline a set of discriminatory policy consequences that could, based on such results, lead to unintended discrimination. To raise awareness on the dimensionality of such foreseen impacts, we have compiled an encyclopedia-like reference guide, the Bias Catalog for Pandemics (BCP), to provide definitions and emphasize realistic examples of bias in general, and within the COVID-19 pandemic context. These are categorized within a division of bias families and a 2-level priority scale, together with preventive steps. In addition, we facilitate the Bias Priority Recommendations on how to best use and apply this catalog, and provide guidelines in order to address real world research questions. The objective is to anticipate and avoid disparate impact and discrimination, by considering causality, explainability, bias and techniques to mitigate the latter. With these, we hope to 1) contribute to designing and conducting fair and equitable data-driven studies and research; and 2) interpret and draw meaningful and actionable conclusions from these.
A General Katsuno-Mendelzon-Style Characterization of AGM Belief Base Revision for Arbitrary Monotonic Logics
Venue: arXiv cs.AI 2021
Year: 2021

The AGM postulates by Alchourrón, Gärdenfors, and Makinson continue to represent a cornerstone in research related to belief change. We generalize the approach of Katsuno and Mendelzon (KM) for characterizing AGM base revision from propositional logic to the setting of (multiple) base revision in arbitrary monotonic logics. Our core result is a representation theorem using the assignment of total - yet not transitive - "preference" relations to belief bases. We also provide a characterization of all logics for which our result can be strengthened to preorder assignments (as in KM's original work).
State-level Racially Motivated Hate Crimes Contrast Public Opinion on the #StopAsianHate and #StopAAPIHate Movement
Venue: arXiv cs.SI 2021
Year: 2021

#StopAsianHate and #StopAAPIHate are two of the most commonly used hashtags that represent the current movement to end hate crimes against the Asian American and Pacific Islander community. We conduct a social media study of public opinion on the #StopAsianHate and #StopAAPIHate movement based on 46,058 Twitter users across 30 states in the United States ranging from March 18 to April 11, 2021. The movement attracts more participation from women, younger adults, Asian and Black communities. 51.56% of the Twitter users show direct support, 18.38% are news about anti-Asian hate crimes, while 5.43% show a negative attitude towards the movement. Public opinion varies across user characteristics. Furthermore, among the states with most racial bias motivated hate crimes, the negative attitude towards the #StopAsianHate and #StopAAPIHate movement is the weakest. To our best knowledge, this is the first large-scale social media-based study to understand public opinion on the #StopAsianHate and #StopAAPIHate movement. We hope our study can provide insights and promote research on anti-Asian hate crimes, and ultimately help address such a serious societal issue for the common benefits of all communities.
Towards Fair Classifiers Without Sensitive Attributes: Exploring Biases in Related Features
Venue: arXiv cs.LG 2021
Year: 2021

Despite the rapid development and great success of machine learning models, extensive studies have exposed their disadvantage of inheriting latent discrimination and societal bias from the training data. This phenomenon hinders their adoption on high-stake applications. Thus, many efforts have been taken for developing fair machine learning models. Most of them require that sensitive attributes are available during training to learn fair models. However, in many real-world applications, it is usually infeasible to obtain the sensitive attributes due to privacy or legal issues, which challenges existing fair-ensuring strategies. Though the sensitive attribute of each data sample is unknown, we observe that there are usually some non-sensitive features in the training data that are highly correlated with sensitive attributes, which can be used to alleviate the bias. Therefore, in this paper, we study a novel problem of exploring features that are highly correlated with sensitive attributes for learning fair and accurate classifiers. We theoretically show that by minimizing the correlation between these related features and model prediction, we can learn a fair classifier. Based on this motivation, we propose a novel framework which simultaneously uses these related features for accurate prediction and enforces fairness. In addition, the model can dynamically adjust the regularization weight of each related feature to balance its contribution on model classification and fairness. Experimental results on real-world datasets demonstrate the effectiveness of the proposed model for learning fair models with high classification accuracy.