OpenRaiser/Intern-Atlas huggingface dataset:
DocFormerv2: Local Features for Document Understanding
Venue: arXiv cs.CV 2023
Year: 2023

We propose DocFormerv2, a multi-modal transformer for Visual Document Understanding (VDU). The VDU domain entails understanding documents (beyond mere OCR predictions) e.g., extracting information from a form, VQA for documents and other tasks. VDU is challenging as it needs a model to make sense of multiple modalities (visual, language and spatial) to make a prediction. Our approach, termed DocFormerv2 is an encoder-decoder transformer which takes as input - vision, language and spatial features. DocFormerv2 is pre-trained with unsupervised tasks employed asymmetrically i.e., two novel document tasks on encoder and one on the auto-regressive decoder. The unsupervised tasks have been carefully designed to ensure that the pre-training encourages local-feature alignment between multiple modalities. DocFormerv2 when evaluated on nine datasets shows state-of-the-art performance over strong baselines e.g. TabFact (4.3%), InfoVQA (1.4%), FUNSD (1%). Furthermore, to show generalization capabilities, on three VQA tasks involving scene-text, Doc- Formerv2 outperforms previous comparably-sized models and even does better than much larger models (such as GIT2, PaLi and Flamingo) on some tasks. Extensive ablations show that due to its pre-training, DocFormerv2 understands multiple modalities better than prior-art in VDU.
UMDFood: Vision-language models boost food composition compilation
Venue: arXiv cs.CV 2023
Year: 2023

Nutrition information is crucial in precision nutrition and the food industry. The current food composition compilation paradigm relies on laborious and experience-dependent methods. However, these methods struggle to keep up with the dynamic consumer market, resulting in delayed and incomplete nutrition data. In addition, earlier machine learning methods overlook the information in food ingredient statements or ignore the features of food images. To this end, we propose a novel vision-language model, UMDFood-VL, using front-of-package labeling and product images to accurately estimate food composition profiles. In order to empower model training, we established UMDFood-90k, the most comprehensive multimodal food database to date, containing 89,533 samples, each labeled with image and text-based ingredient descriptions and 11 nutrient annotations. UMDFood-VL achieves the macro-AUCROC up to 0.921 for fat content estimation, which is significantly higher than existing baseline methods and satisfies the practical requirements of food composition compilation. Meanwhile, up to 82.2% of selected products' estimated error between chemical analysis results and model estimation results are less than 10%. This performance sheds light on generalization towards other food and nutrition-related data compilation and catalyzation for the evolution of generative AI-based technology in other food applications that require personalization.
A Survey of Explainable AI and Proposal for a Discipline of Explanation Engineering
Venue: arXiv cs.AI 2023
Year: 2023

In this survey paper, we deep dive into the field of Explainable Artificial Intelligence (XAI). After introducing the scope of this paper, we start by discussing what an "explanation" really is. We then move on to discuss some of the existing approaches to XAI and build a taxonomy of the most popular methods. Next, we also look at a few applications of these and other XAI techniques in four primary domains: finance, autonomous driving, healthcare and manufacturing. We end by introducing a promising discipline, "Explanation Engineering," which includes a systematic approach for designing explainability into AI systems.
Towards a Technology-Driven Adaptive Decision Support System for Integrated Pavement and Maintenance strategies (TDADSS-IPM): focus on risk assessment framework for climate change adaptation
Venue: arXiv cs.AI 2023
Year: 2023

Decision Support Systems for pavement and maintenance strategies have traditionally been designed as silos led to local optimum systems. Moreover, since big data usage didn't exist as result of Industry 4.0 as of today, DSSs were not initially designed adaptive to the sources of uncertainties led to rigid decisions. Motivated by the vulnerability of the road assets to the climate phenomena, this paper takes a visionary step towards introducing a Technology-Driven Adaptive Decision Support System for Integrated Pavement and Maintenance activities called TDADSS-IPM. As part of such DSS, a bottom-up risk assessment model is met via Bayesian Belief Networks (BBN) to realize the actual condition of the Danish roads due to weather condition. Such model fills the gaps in the knowledge domain and develops a platform that can be trained over time, and applied in real-time to the actual event.
Beyond Rankings: Exploring the Impact of SERP Features on Organic Click-through Rates
Venue: arXiv cs.IR 2023
Year: 2023

Search Engine Result Pages (SERPs) serve as the digital gateways to the vast expanse of the internet. Past decades have witnessed a surge in research primarily centered on the influence of website ranking on these pages, to determine the click-through rate (CTR). However, during this period, the landscape of SERPs has undergone a dramatic evolution: SERP features, encompassing elements such as knowledge panels, media galleries, FAQs, and more, have emerged as an increasingly prominent facet of these result pages. Our study examines the crucial role of these features, revealing them to be not merely aesthetic components, but strongly influence CTR and the associated behavior of internet users. We demonstrate how these features can significantly modulate web traffic, either amplifying or attenuating it. We dissect these intricate interaction effects leveraging a unique dataset of 67,000 keywords and their respective Google SERPs, spanning over 40 distinct US-based e-commerce domains, generating over 6 million clicks from 24 million views. This cross-website dataset, unprecedented in its scope, enables us to assess the impact of 24 different SERP features on organic CTR. Through an ablation study modeling CTR, we illustrate the incremental predictive power these features hold.
Comparative Analysis of Widely use Object-Oriented Languages
Venue: arXiv cs.PL 2023
Year: 2023

Programming is an integral part of computer science discipline. Every day the programming environment is not only rapidly growing but also changing and languages are constantly evolving. Learning of object-oriented paradigm is compulsory in every computer science major so the choice of language to teach object-oriented principles is very important. Due to large pool of object-oriented languages, it is difficult to choose which should be the first programming language in order to teach object-oriented principles. Many studies shown which should be the first language to tech object-oriented concepts but there is no method to compare and evaluate these languages. In this article we proposed a comprehensive framework to evaluate the widely used object-oriented languages. The languages are evaluated basis of their technical and environmental features.
Concurrent Classifier Error Detection (CCED) in Large Scale Machine Learning Systems
Venue: arXiv cs.LG 2023
Year: 2023

The complexity of Machine Learning (ML) systems increases each year, with current implementations of large language models or text-to-image generators having billions of parameters and requiring billions of arithmetic operations. As these systems are widely utilized, ensuring their reliable operation is becoming a design requirement. Traditional error detection mechanisms introduce circuit or time redundancy that significantly impacts system performance. An alternative is the use of Concurrent Error Detection (CED) schemes that operate in parallel with the system and exploit their properties to detect errors. CED is attractive for large ML systems because it can potentially reduce the cost of error detection. In this paper, we introduce Concurrent Classifier Error Detection (CCED), a scheme to implement CED in ML systems using a concurrent ML classifier to detect errors. CCED identifies a set of check signals in the main ML system and feeds them to the concurrent ML classifier that is trained to detect errors. The proposed CCED scheme has been implemented and evaluated on two widely used large-scale ML models: Contrastive Language Image Pretraining (CLIP) used for image classification and Bidirectional Encoder Representations from Transformers (BERT) used for natural language applications. The results show that more than 95 percent of the errors are detected when using a simple Random Forest classifier that is order of magnitude simpler than CLIP or BERT. These results illustrate the potential of CCED to implement error detection in large-scale ML models.
Fast $(1+\varepsilon)$-Approximation Algorithms for Binary Matrix Factorization
Venue: arXiv cs.DS 2023
Year: 2023

We introduce efficient $(1+\varepsilon)$-approximation algorithms for the binary matrix factorization (BMF) problem, where the inputs are a matrix $\mathbf{A}\in\{0,1\}^{n\times d}$, a rank parameter $k>0$, as well as an accuracy parameter $\varepsilon>0$, and the goal is to approximate $\mathbf{A}$ as a product of low-rank factors $\mathbf{U}\in\{0,1\}^{n\times k}$ and $\mathbf{V}\in\{0,1\}^{k\times d}$. Equivalently, we want to find $\mathbf{U}$ and $\mathbf{V}$ that minimize the Frobenius loss $\|\mathbf{U}\mathbf{V} - \mathbf{A}\|_F^2$. Before this work, the state-of-the-art for this problem was the approximation algorithm of Kumar et. al. [ICML 2019], which achieves a $C$-approximation for some constant $C\ge 576$. We give the first $(1+\varepsilon)$-approximation algorithm using running time singly exponential in $k$, where $k$ is typically a small integer. Our techniques generalize to other common variants of the BMF problem, admitting bicriteria $(1+\varepsilon)$-approximation algorithms for $L_p$ loss functions and the setting where matrix operations are performed in $\mathbb{F}_2$. Our approach can be implemented in standard big data models, such as the streaming or distributed models.
Multifunctionality in a Connectome-Based Reservoir Computer
Venue: arXiv cs.LG 2023
Year: 2023

Multifunctionality describes the capacity for a neural network to perform multiple mutually exclusive tasks without altering its network connections; and is an emerging area of interest in the reservoir computing machine learning paradigm. Multifunctionality has been observed in the brains of humans and other animals: particularly, in the lateral horn of the fruit fly. In this work, we transplant the connectome of the fruit fly lateral horn to a reservoir computer (RC), and investigate the extent to which this 'fruit fly RC' (FFRC) exhibits multifunctionality using the 'seeing double' problem as a benchmark test. We furthermore explore the dynamics of how this FFRC achieves multifunctionality while varying the network's spectral radius. Compared to the widely-used Erdös-Renyi Reservoir Computer (ERRC), we report that the FFRC exhibits a greater capacity for multifunctionality; is multifunctional across a broader hyperparameter range; and solves the seeing double problem far beyond the previously observed spectral radius limit, wherein the ERRC's dynamics become chaotic.
Discrete-time Robust PD Controlled System with DOB/CDOB Compensation for High Speed Autonomous Vehicle Path Following
Venue: arXiv cs.RO 2023
Year: 2023

Autonomous vehicle path following performance is one of significant consideration. This paper presents discrete time design of robust PD controlled system with disturbance observer (DOB) and communication disturbance observer (CDOB) compensation to enhance autonomous vehicle path following performance. Although always implemented on digital devices, DOB and CDOB structure are usually designed in continuous time in the literature and also in our previous work. However, it requires high sampling rate for continuous-time design block diagram to automatically convert to corresponding discrete-time controller using rapid controller prototyping systems. In this paper, direct discrete time design is carried out. Digital PD feedback controller is designed based on the nominal plant using the proposed parameter space approach. Zero order hold method is applied to discretize the nominal plant, DOB and CDOB structure in continuous domain. Discrete time DOB is embedded into the steering to path following error loop for model regulation in the presence of uncertainty in vehicle parameters such as vehicle mass, vehicle speed and road-tire friction coefficient and rejecting external disturbance like crosswind force. On the other hand, time delay from CAN bus based sensor and actuator command interfaces results in degradation of system performance since large negative phase angles are added to the plant frequency response. Discrete time CDOB compensated control system can be used for time delay compensation where the accurate knowledge of delay time value is not necessary. A validated model of our lab Ford Fusion hybrid automated driving research vehicle is used for the simulation analysis while the vehicle is driving at high speed. Simulation results successfully demonstrate the improvement of autonomous vehicle path following performance with the proposed discrete time DOB and CDOB structure.
EdGCon: Auto-assigner of Iconicity Ratings Grounded by Lexical Properties to Aid in Generation of Technical Gestures
Venue: arXiv cs.HC 2023
Year: 2023

Gestures that share similarities in their forms and are related in their meanings, should be easier for learners to recognize and incorporate into their existing lexicon. In that regard, to be more readily accepted as standard by the Deaf and Hard of Hearing community, technical gestures in American Sign Language (ASL) will optimally share similar in forms with their lexical neighbors. We utilize a lexical database of ASL, ASL-LEX, to identify lexical relations within a set of technical gestures. We use automated identification for 3 unique sub-lexical properties in ASL- location, handshape and movement. EdGCon assigned an iconicity rating based on the lexical property similarities of the new gesture with an existing set of technical gestures and the relatedness of the meaning of the new technical word to that of the existing set of technical words. We collected 30 ad hoc crowdsourced technical gestures from different internet websites and tested them against 31 gestures from the DeafTEC technical corpus. We found that EdGCon was able to correctly auto-assign the iconicity ratings 80.76% of the time.
A Novel Correlation-optimized Deep Learning Method for Wind Speed Forecast
Venue: arXiv cs.LG 2023
Year: 2023

The increasing installation rate of wind power poses great challenges to the global power system. In order to ensure the reliable operation of the power system, it is necessary to accurately forecast the wind speed and power of the wind turbines. At present, deep learning is progressively applied to the wind speed prediction. Nevertheless, the recent deep learning methods still reflect the embarrassment for practical applications due to model interpretability and hardware limitation. To this end, a novel deep knowledge-based learning method is proposed in this paper. The proposed method hybridizes pre-training method and auto-encoder structure to improve data representation and modeling of the deep knowledge-based learning framework. In order to form knowledge and corresponding absorbers, the original data is preprocessed by an optimization model based on correlation to construct multi-layer networks (knowledge) which are absorbed by sequence to sequence (Seq2Seq) models. Specifically, new cognition and memory units (CMU) are designed to reinforce traditional deep learning framework. Finally, the effectiveness of the proposed method is verified by three wind prediction cases from a wind farm in Liaoning, China. Experimental results show that the proposed method increases the stability and training efficiency compared to the traditional LSTM method and LSTM/GRU-based Seq2Seq method for applications of wind speed forecasting.
Span Identification of Epistemic Stance-Taking in Academic Written English
Venue: arXiv cs.CL 2023
Year: 2023

Responding to the increasing need for automated writing evaluation (AWE) systems to assess language use beyond lexis and grammar (Burstein et al., 2016), we introduce a new approach to identify rhetorical features of stance in academic English writing. Drawing on the discourse-analytic framework of engagement in the Appraisal analysis (Martin & White, 2005), we manually annotated 4,688 sentences (126,411 tokens) for eight rhetorical stance categories (e.g., PROCLAIM, ATTRIBUTION) and additional discourse elements. We then report an experiment to train machine learning models to identify and categorize the spans of these stance expressions. The best-performing model (RoBERTa + LSTM) achieved macro-averaged F1 of .7208 in the span identification of stance-taking expressions, slightly outperforming the intercoder reliability estimates before adjudication (F1 = .6629).
Incorporating Deep Syntactic and Semantic Knowledge for Chinese Sequence Labeling with GCN
Venue: arXiv cs.CL 2023
Year: 2023

Recently, it is quite common to integrate Chinese sequence labeling results to enhance syntactic and semantic parsing. However, little attention has been paid to the utility of hierarchy and structure information encoded in syntactic and semantic features for Chinese sequence labeling tasks. In this paper, we propose a novel framework to encode syntactic structure features and semantic information for Chinese sequence labeling tasks with graph convolutional networks (GCN). Experiments on five benchmark datasets, including Chinese word segmentation and part-of-speech tagging, demonstrate that our model can effectively improve the performance of Chinese labeling tasks.
Identifying Subgroups of ICU Patients Using End-to-End Multivariate Time-Series Clustering Algorithm Based on Real-World Vital Signs Data
Venue: arXiv cs.LG 2023
Year: 2023

This study employed the MIMIC-IV database as data source to investigate the use of dynamic, high-frequency, multivariate time-series vital signs data, including temperature, heart rate, mean blood pressure, respiratory rate, and SpO2, monitored first 8 hours data in the ICU stay. Various clustering algorithms were compared, and an end-to-end multivariate time series clustering system called Time2Feat, combined with K-Means, was chosen as the most effective method to cluster patients in the ICU. In clustering analysis, data of 8,080 patients admitted between 2008 and 2016 was used for model development and 2,038 patients admitted between 2017 and 2019 for model validation. By analyzing the differences in clinical mortality prognosis among different categories, varying risks of ICU mortality and hospital mortality were found between different subgroups. Furthermore, the study visualized the trajectory of vital signs changes. The findings of this study provide valuable insights into the potential use of multivariate time-series clustering systems in patient management and monitoring in the ICU setting.
Hierarchical Multiresolution Feature- and Prior-based Graphs for Classification
Venue: arXiv cs.CV 2023
Year: 2023

To incorporate spatial (neighborhood) and bidirectional hierarchical relationships as well as features and priors of the samples into their classification, we formulated the classification problem on three variants of multiresolution neighborhood graphs and the graph of a hierarchical conditional random field. Each of these graphs was weighted and undirected and could thus incorporate the spatial or hierarchical relationships in all directions. In addition, each variant of the proposed neighborhood graphs was composed of a spatial feature-based subgraph and an aspatial prior-based subgraph. It expanded on a random walker graph by using novel mechanisms to derive the edge weights of its spatial feature-based subgraph. These mechanisms included implicit and explicit edge detection to enhance detection of weak boundaries between different classes in spatial domain. The implicit edge detection relied on the outlier detection capability of the Tukey's function and the classification reliabilities of the samples estimated by a hierarchical random forest classifier. Similar mechanism was used to derive the edge weights and thus the energy function of the hierarchical conditional random field. This way, the classification problem boiled down to a system of linear equations and a minimization of the energy function which could be done via fast and efficient techniques.
Effects of Imperfections on Quantum Algorithms: A Software Engineering Perspective
Venue: arXiv cs.ET 2023
Year: 2023

Quantum computers promise considerable speedups over classical approaches, which has raised interest from many disciplines. Since any currently available implementations suffer from noise and imperfections, achieving concrete speedups for meaningful problem sizes remains a major challenge. Yet, imperfections and noise may remain present in quantum computing for a long while. Such limitations play no role in classical software computing, and software engineers are typically not well accustomed to considering such imperfections, albeit they substantially influence core properties of software and systems. In this paper, we show how to model imperfections with an approach tailored to (quantum) software engineers. We intuitively illustrate, using numerical simulations, how imperfections influence core properties of quantum algorithms on NISQ systems, and show possible options for tailoring future NISQ machines to improve system performance in a co-design approach. Our results are obtained from a software framework that we provide in form of an easy-to-use reproduction package. It does not require computer scientists to acquire deep physical knowledge on noise, yet provide tangible and intuitively accessible means of interpreting the influence of noise on common software quality and performance indicators.
Evaluating Emotion Arcs Across Languages: Bridging the Global Divide in Sentiment Analysis
Venue: arXiv cs.CL 2023
Year: 2023

Emotion arcs capture how an individual (or a population) feels over time. They are widely used in industry and research; however, there is little work on evaluating the automatically generated arcs. This is because of the difficulty of establishing the true (gold) emotion arc. Our work, for the first time, systematically and quantitatively evaluates automatically generated emotion arcs. We also compare two common ways of generating emotion arcs: Machine-Learning (ML) models and Lexicon-Only (LexO) methods. By running experiments on 18 diverse datasets in 9 languages, we show that despite being markedly poor at instance level emotion classification, LexO methods are highly accurate at generating emotion arcs when aggregating information from hundreds of instances. We also show, through experiments on six indigenous African languages, as well as Arabic, and Spanish, that automatic translations of English emotion lexicons can be used to generate high-quality emotion arcs in less-resource languages. This opens up avenues for work on emotions in languages from around the world; which is crucial for commerce, public policy, and health research in service of speakers often left behind. Code and resources: https://github.com/dteodore/EmotionArcs