OpenRaiser/Intern-Atlas huggingface dataset:
Characterizing the Generalization Error of Gibbs Algorithm with Symmetrized KL information
Venue: arXiv cs.LG 2021
Year: 2021

Bounding the generalization error of a supervised learning algorithm is one of the most important problems in learning theory, and various approaches have been developed. However, existing bounds are often loose and lack of guarantees. As a result, they may fail to characterize the exact generalization ability of a learning algorithm. Our main contribution is an exact characterization of the expected generalization error of the well-known Gibbs algorithm in terms of symmetrized KL information between the input training samples and the output hypothesis. Such a result can be applied to tighten existing expected generalization error bound. Our analysis provides more insight on the fundamental role the symmetrized KL information plays in controlling the generalization error of the Gibbs algorithm.
An Online Question Answering System based on Sub-graph Searching
Venue: arXiv cs.AI 2021
Year: 2021

Knowledge graphs (KGs) have been widely used for question answering (QA) applications, especially the entity based QA. However, searching an-swers from an entire large-scale knowledge graph is very time-consuming and it is hard to meet the speed need of real online QA systems. In this pa-per, we design a sub-graph searching mechanism to solve this problem by creating sub-graph index, and each answer generation step is restricted in the sub-graph level. We use this mechanism into a real online QA chat system, and it can bring obvious improvement on question coverage by well answer-ing entity based questions, and it can be with a very high speed, which en-sures the user experience of online QA.
Efficient Human Pose Estimation by Maximizing Fusion and High-Level Spatial Attention
Venue: arXiv cs.CV 2021
Year: 2021

In this paper, we propose an efficient human pose estimation network -- SFM (slender fusion model) by fusing multi-level features and adding lightweight attention blocks -- HSA (High-Level Spatial Attention). Many existing methods on efficient network have already taken feature fusion into consideration, which largely boosts the performance. However, its performance is far inferior to large network such as ResNet and HRNet due to its limited fusion operation in the network. Specifically, we expand the number of fusion operation by building bridges between two pyramid frameworks without adding layers. Meanwhile, to capture long-range dependency, we propose a lightweight attention block -- HSA, which computes second-order attention map. In summary, SFM maximizes the number of feature fusion in a limited number of layers. HSA learns high precise spatial information by computing the attention of spatial attention map. With the help of SFM and HSA, our network is able to generate multi-level feature and extract precise global spatial information with little computing resource. Thus, our method achieve comparable or even better accuracy with less parameters and computational cost. Our SFM achieve 89.0 in PCKh@0.5, 42.0 in PCKh@0.1 on MPII validation set and 71.7 in AP, 90.7 in AP@0.5 on COCO validation with only 1.7G FLOPs and 1.5M parameters. The source code will be public soon.
Design-Driven Requirements for Computationally Co-Creative Game AI Design Tools
Venue: arXiv cs.HC 2021
Year: 2021

Game AI designers must manage complex interactions between the AI character, the game world, and the player, while achieving their design visions. Computational co-creativity tools can aid them, but first, AI and HCI researchers must gather requirements and determine design heuristics to build effective co-creative tools. In this work, we present a participatory design study that categorizes and analyzes game AI designers' workflows, goals, and expectations for such tools. We evince deep connections between game AI design and the design of co-creative tools, and present implications for future co-creativity tool research and development.
Viewpoint-Invariant Exercise Repetition Counting
Venue: arXiv cs.CV 2021
Year: 2021

Counting the repetition of human exercise and physical rehabilitation is a common task in rehabilitation and exercise training. The existing vision-based repetition counting methods less emphasize the concurrent motions in the same video. This work presents a vision-based human motion repetition counting applicable to counting concurrent motions through the skeleton location extracted from various pose estimation methods. The presented method was validated on the University of Idaho Physical Rehabilitation Movements Data Set (UI-PRMD), and MM-fit dataset. The overall mean absolute error (MAE) for mm-fit was 0.06 with off-by-one Accuracy (OBOA) 0.94. Overall MAE for UI-PRMD dataset was 0.06 with OBOA 0.95. We have also tested the performance in a variety of camera locations and concurrent motions with conveniently collected video with overall MAE 0.06 and OBOA 0.88. The proposed method provides a view-angle and motion agnostic concurrent motion counting. This method can potentially use in large-scale remote rehabilitation and exercise training with only one camera.
Break, Perturb, Build: Automatic Perturbation of Reasoning Paths Through Question Decomposition
Venue: arXiv cs.CL 2021
Year: 2021

Recent efforts to create challenge benchmarks that test the abilities of natural language understanding models have largely depended on human annotations. In this work, we introduce the "Break, Perturb, Build" (BPB) framework for automatic reasoning-oriented perturbation of question-answer pairs. BPB represents a question by decomposing it into the reasoning steps that are required to answer it, symbolically perturbs the decomposition, and then generates new question-answer pairs. We demonstrate the effectiveness of BPB by creating evaluation sets for three reading comprehension (RC) benchmarks, generating thousands of high-quality examples without human intervention. We evaluate a range of RC models on our evaluation sets, which reveals large performance gaps on generated examples compared to the original data. Moreover, symbolic perturbations enable fine-grained analysis of the strengths and limitations of models. Last, augmenting the training data with examples generated by BPB helps close the performance gaps, without any drop on the original data distribution.
Why You Should Try the Real Data for the Scene Text Recognition
Venue: arXiv cs.CV 2021
Year: 2021

Recent works in the text recognition area have pushed forward the recognition results to the new horizons. But for a long time a lack of large human-labeled natural text recognition datasets has been forcing researchers to use synthetic data for training text recognition models. Even though synthetic datasets are very large (MJSynth and SynthTest, two most famous synthetic datasets, have several million images each), their diversity could be insufficient, compared to natural datasets like ICDAR and others. Fortunately, the recently released text-recognition annotation for OpenImages V5 dataset has comparable with synthetic dataset number of instances and more diverse examples. We have used this annotation with a Text Recognition head architecture from the Yet Another Mask Text Spotter and got comparable to the SOTA results. On some datasets we have even outperformed previous SOTA models. In this paper we also introduce a text recognition model. The model's code is available.
A Sketch of a Proof-Theoretic Semantics for Necessity
Venue: arXiv cs.LO 2021
Year: 2021

This paper considers proof-theoretic semantics for necessity within Dummett's and Prawitz's framework. Inspired by a system of Pfenning's and Davies's, the language of intuitionist logic is extended by a higher order operator which captures a notion of validity. A notion of relative necessary is defined in terms of it, which expresses a necessary connection between the assumptions and the conclusion of a deduction.
Improving Robustness and Accuracy via Relative Information Encoding in 3D Human Pose Estimation
Venue: arXiv cs.CV 2021
Year: 2021

Most of the existing 3D human pose estimation approaches mainly focus on predicting 3D positional relationships between the root joint and other human joints (local motion) instead of the overall trajectory of the human body (global motion). Despite the great progress achieved by these approaches, they are not robust to global motion, and lack the ability to accurately predict local motion with a small movement range. To alleviate these two problems, we propose a relative information encoding method that yields positional and temporal enhanced representations. Firstly, we encode positional information by utilizing relative coordinates of 2D poses to enhance the consistency between the input and output distribution. The same posture with different absolute 2D positions can be mapped to a common representation. It is beneficial to resist the interference of global motion on the prediction results. Second, we encode temporal information by establishing the connection between the current pose and other poses of the same person within a period of time. More attention will be paid to the movement changes before and after the current pose, resulting in better prediction performance on local motion with a small movement range. The ablation studies validate the effectiveness of the proposed relative information encoding method. Besides, we introduce a multi-stage optimization method to the whole framework to further exploit the positional and temporal enhanced representations. Our method outperforms state-of-the-art methods on two public datasets. Code is available at https://github.com/paTRICK-swk/Pose3D-RIE.
"Excavating AI" Re-excavated: Debunking a Fallacious Account of the JAFFE Dataset
Venue: arXiv cs.CY 2021
Year: 2021

Twenty-five years ago, my colleagues Miyuki Kamachi and Jiro Gyoba and I designed and photographed JAFFE, a set of facial expression images intended for use in a study of face perception. In 2019, without seeking permission or informing us, Kate Crawford and Trevor Paglen exhibited JAFFE in two widely publicized art shows. In addition, they published a nonfactual account of the images in the essay "Excavating AI: The Politics of Images in Machine Learning Training Sets." The present article recounts the creation of the JAFFE dataset and unravels each of Crawford and Paglen's fallacious statements. I also discuss JAFFE more broadly in connection with research on facial expression, affective computing, and human-computer interaction.
PKSpell: Data-Driven Pitch Spelling and Key Signature Estimation
Venue: arXiv cs.SD 2021
Year: 2021

We present PKSpell: a data-driven approach for the joint estimation of pitch spelling and key signatures from MIDI files. Both elements are fundamental for the production of a full-fledged musical score and facilitate many MIR tasks such as harmonic analysis, section identification, melodic similarity, and search in a digital music library. We design a deep recurrent neural network model that only requires information readily available in all kinds of MIDI files, including performances, or other symbolic encodings. We release a model trained on the ASAP dataset. Our system can be used with these pre-trained parameters and is easy to integrate into a MIR pipeline. We also propose a data augmentation procedure that helps retraining on small datasets. PKSpell achieves strong key signature estimation performance on a challenging dataset. Most importantly, this model establishes a new state-of-the-art performance on the MuseData pitch spelling dataset without retraining.
Explanatory Journeys: Visualising to Understand and Explain Administrative Justice Paths of Redress
Venue: arXiv cs.HC 2021
Year: 2021

Administrative justice concerns the relationships between individuals and the state. It includes redress and complaints on decisions of a child's education, social care, licensing, planning, environment, housing and homelessness. However, if someone has a complaint or an issue, it is challenging for people to understand different possible redress paths and explore what path is suitable for their situation. Explanatory visualisation has the potential to display these paths of redress in a clear way, such that people can see, understand and explore their options. The visualisation challenge is further complicated because information is spread across many documents, laws, guidance and policies and requires judicial interpretation. Consequently, there is not a single database of paths of redress. In this work we present how we have co-designed a system to visualise administrative justice paths of redress. Simultaneously, we classify, collate and organise the underpinning data, from expert workshops, heuristic evaluation and expert critical reflection. We make four contributions: (i) an application design study of the explanatory visualisation tool (Artemus), (ii) coordinated and co-design approach to aggregating the data, (iii) two in-depth case studies in housing and education demonstrating explanatory paths of redress in administrative law, and (iv) reflections on the expert co-design process and expert data gathering and explanatory visualisation for administrative justice and law.
ProtoTransformer: A Meta-Learning Approach to Providing Student Feedback
Venue: arXiv cs.CY 2021
Year: 2021

High-quality computer science education is limited by the difficulty of providing instructor feedback to students at scale. While this feedback could in principle be automated, supervised approaches to predicting the correct feedback are bottlenecked by the intractability of annotating large quantities of student code. In this paper, we instead frame the problem of providing feedback as few-shot classification, where a meta-learner adapts to give feedback to student code on a new programming question from just a few examples annotated by instructors. Because data for meta-training is limited, we propose a number of amendments to the typical few-shot learning framework, including task augmentation to create synthetic tasks, and additional side information to build stronger priors about each task. These additions are combined with a transformer architecture to embed discrete sequences (e.g. code) to a prototypical representation of a feedback class label. On a suite of few-shot natural language processing tasks, we match or outperform state-of-the-art performance. Then, on a collection of student solutions to exam questions from an introductory university course, we show that our approach reaches an average precision of 88% on unseen questions, surpassing the 82% precision of teaching assistants. Our approach was successfully deployed to deliver feedback to 16,000 student exam-solutions in a programming course offered by a tier 1 university. This is, to the best of our knowledge, the first successful deployment of a machine learning based feedback to open-ended student code.
A Checklist for Explainable AI in the Insurance Domain
Venue: arXiv cs.CY 2021
Year: 2021

Artificial intelligence (AI) is a powerful tool to accomplish a great many tasks. This exciting branch of technology is being adopted increasingly across varying sectors, including the insurance domain. With that power arise several complications. One of which is a lack of transparency and explainability of an algorithm for experts and non-experts alike. This brings into question both the usefulness as well as the accuracy of the algorithm, coupled with an added difficulty to assess potential biases within the data or the model. In this paper, we investigate the current usage of AI algorithms in the Dutch insurance industry and the adoption of explainable artificial intelligence (XAI) techniques. Armed with this knowledge we design a checklist for insurance companies that should help assure quality standards regarding XAI and a solid foundation for cooperation between organisations. This checklist extends an existing checklist of SIVI, the standardisation institute for digital cooperation and innovation in Dutch insurance.
Audit and Assurance of AI Algorithms: A framework to ensure ethical algorithmic practices in Artificial Intelligence
Venue: arXiv cs.CY 2021
Year: 2021

Algorithms are becoming more widely used in business, and businesses are becoming increasingly concerned that their algorithms will cause significant reputational or financial damage. We should emphasize that any of these damages stem from situations in which the United States lacks strict legislative prohibitions or specified protocols for measuring damages. As a result, governments are enacting legislation and enforcing prohibitions, regulators are fining businesses, and the judiciary is debating whether or not to make artificially intelligent computer models as the decision-makers in the eyes of the law. From autonomous vehicles and banking to medical care, housing, and legal decisions, there will soon be enormous amounts of algorithms that make decisions with limited human interference. Governments, businesses, and society would have an algorithm audit, which would have systematic verification that algorithms are lawful, ethical, and secure, similar to financial audits. A modern market, auditing, and assurance of algorithms developed to professionalize and industrialize AI, machine learning, and related algorithms. Stakeholders of this emerging field include policymakers and regulators, along with industry experts and entrepreneurs. In addition, we foresee audit thresholds and frameworks providing valuable information to all who are concerned with governance and standardization. This paper aims to review the critical areas required for auditing and assurance and spark discussion in this novel field of study and practice.
Measuring Domain Knowledge for Early Prediction of Student Performance: A Semantic Approach
Venue: arXiv cs.CY 2021
Year: 2021

The growing popularity of data mining catalyses the researchers to explore various exciting aspects of education. Early prediction of student performance is an emerging area among them. The researchers have used various predictors in performance modelling studies. Although prior cognition can affect student performance, establishing their relationship is still an open research challenge. Quantifying the knowledge from readily available data is the major challenge here. We have proposed a semantic approach for this purpose. Association mining on nearly 0.35 million observations establishes that prior cognition impacts the student performance. The proposed approach of measuring domain knowledge can help the early performance modelling studies to use it as a predictor.
Collaboration Planning of Stakeholders for Sustainable City Logistics Operations
Venue: arXiv cs.CY 2021
Year: 2021

City logistics involves movements of goods in urban areas respecting the municipal and administrative guidelines. The importance of city logistics is growing over the years especially with its role in minimizing traffic congestion and freeing up of public space for city residents. Collaboration is key to managing city logistics operations efficiently. Collaboration can take place in the form of goods consolidation, sharing of resources, information sharing, etc. We investigate the problems of collaboration planning of stakeholders to achieve sustainable city logistics operations. Two categories of models are proposed to evaluate the collaboration strategies. At the macro level, we have the simplified collaboration square model and advance collaboration square model and at the micro level we have the operational level model. These collaboration decision making models, with their mathematical elaborations on business-to-business, business-to-customer, customer-to-business, and customer-to-customer provide roadmaps for evaluating the collaboration strategies of stakeholders for achieving sustainable city logistics operations attainable under non-chaotic situation and presumptions of human levity tendency. City logistics stakeholders can strive to achieve effective collaboration strategies for sustainable city logistics operations by mitigating the uncertainty effect and understanding the theories behind the moving nature of the individual complexities of a city. To investigate system complexity, we propose axioms of uncertainty and use spider networks and system dynamics modeling to investigate system elements and their behavior over time.
The Need and Status of Sea Turtle Conservation and Survey of Associated Computer Vision Advances
Venue: arXiv cs.CV 2021
Year: 2021

For over hundreds of millions of years, sea turtles and their ancestors have swum in the vast expanses of the ocean. They have undergone a number of evolutionary changes, leading to speciation and sub-speciation. However, in the past few decades, some of the most notable forces driving the genetic variance and population decline have been global warming and anthropogenic impact ranging from large-scale poaching, collecting turtle eggs for food, besides dumping trash including plastic waste into the ocean. This leads to severe detrimental effects in the sea turtle population, driving them to extinction. This research focusses on the forces causing the decline in sea turtle population, the necessity for the global conservation efforts along with its successes and failures, followed by an in-depth analysis of the modern advances in detection and recognition of sea turtles, involving Machine Learning and Computer Vision systems, aiding the conservation efforts.
Structure and Performance of Fully Connected Neural Networks: Emerging Complex Network Properties
Venue: arXiv cs.LG 2021
Year: 2021

Understanding the behavior of Artificial Neural Networks is one of the main topics in the field recently, as black-box approaches have become usual since the widespread of deep learning. Such high-dimensional models may manifest instabilities and weird properties that resemble complex systems. Therefore, we propose Complex Network (CN) techniques to analyze the structure and performance of fully connected neural networks. For that, we build a dataset with 4 thousand models and their respective CN properties. They are employed in a supervised classification setup considering four vision benchmarks. Each neural network is approached as a weighted and undirected graph of neurons and synapses, and centrality measures are computed after training. Results show that these measures are highly related to the network classification performance. We also propose the concept of Bag-Of-Neurons (BoN), a CN-based approach for finding topological signatures linking similar neurons. Results suggest that six neuronal types emerge in such networks, independently of the target domain, and are distributed differently according to classification accuracy. We also tackle specific CN properties related to performance, such as higher subgraph centrality on lower-performing models. Our findings suggest that CN properties play a critical role in the performance of fully connected neural networks, with topological patterns emerging independently on a wide range of models.