Reviewerly/exHarmony huggingface dataset:
Tracking by Deblatting

Abstract Objects moving at high speed along complex trajectories often appear in videos, especially videos of sports. Such objects travel a considerable distance during exposure time of a single frame, and therefore, their position in the frame is not well defined. They appear as semi-transparent streaks due to the motion blur and cannot be reliably tracked by general trackers. We propose a novel approach called Tracking by Deblatting based on the observation that motion blur is directly related to the intra-frame trajectory of an object. Blur is estimated by solving two intertwined inverse problems, blind deblurring and image matting, which we call deblatting. By postprocessing, non-causal Tracking by Deblatting estimates continuous, complete, and accurate object trajectories for the whole sequence. Tracked objects are precisely localized with higher temporal resolution than by conventional trackers. Energy minimization by dynamic programming is used to detect abrupt changes of motion, called bounces. High-order polynomials are then fitted to smooth trajectory segments between bounces. The output is a continuous trajectory function that assigns location for every real-valued time stamp from zero to the number of frames. The proposed algorithm was evaluated on a newly created dataset of videos from a high-speed camera using a novel Trajectory-IoU metric that generalizes the traditional Intersection over Union and measures the accuracy of the intra-frame trajectory. The proposed method outperforms the baselines both in recall and trajectory accuracy. Additionally, we show that from the trajectory function precise physical calculations are possible, such as radius, gravity, and sub-frame object velocity. Velocity estimation is compared to the high-speed camera measurements and radars. Results show high performance of the proposed method in terms of Trajectory-IoU, recall, and velocity estimation.
Detection and Classification of Cross-language Code Clone Types by Filtering the Nodes of ANTLR-generated Parse Tree

A complete and accurate cross-language clone detection tool can support software forking process that reuses the more reliable algorithms of legacy systems from one language code base to other. Cross-language clone detection also helps in building code recommendation system. This paper proposes a new technique to detect and classify cross-language clones of C and C++ programs by filtering the nodes of ANTLR-generated parse tree using a common grammar file, CPP14.g4. Parsing the input files using CPP14.g4 provides all the lexical and semantic information of input source code. Selective filtering of nodes performs serialization of two parse trees. Vector representation using term frequency inverse document frequency (TF-IDF) of the resultant tree is given as an input to cosine similarity to classify the clone types. Filtered parse tree of C and C++ increases the precision from 51% to 61%, and matching based on renaming the input/output expressions provides average precision of 91.97% and 95.37% for small scale and large scale repositories respectively. The proposed cross-language clone detection exhibits the highest precision of 95.37% in finding all types of clones (1, 2, 3 and 4) for 16,032 semantically similar clone pairs of C and CPP codes.
On the Diversity and Limits of Human Explanations

A growing effort in NLP aims to build datasets of human explanations.However, it remains unclear whether these datasets serve their intended goals.This problem is exacerbated by the fact that the term explanation is overloaded and refers to a broad range of notions with different properties and ramifications.Our goal is to provide an overview of the diversity of explanations, discuss human limitations in providing explanations, and ultimately provide implications for collecting and using human explanations in NLP.Inspired by prior work in psychology and cognitive sciences, we group existing human explanations in NLP into three categories: proximal mechanism, evidence, and procedure.These three types differ in nature and have implications for the resultant explanations.For instance, procedure is not considered explanation in psychology and connects with a rich body of work on learning from instructions.The diversity of explanations is further evidenced by proxy questions that are needed for annotators to interpret and answer "why is [input] assigned [label]".Finally, giving explanations may require different, often deeper, understandings than predictions, which casts doubt on whether humans can provide valid explanations in some tasks.
Inter-annotator agreement in spoken language annotation: Applying uα-family coefficients to discourse segmentation

As databases make Corpus Linguistics a common tool for most linguists, corpus annotation becomes an increasingly important process. Corpus users do not need only raw data, but also annotated data, submitted to tagging or parsing processes through annotation protocols. One problem with corpus annotation lies in its reliability, that is, in the probability that its results can be replicable by independent researchers. Inter-annotation agreement (IAA) is the process which evaluates the probability that, applying the same protocol, different annotators reach similar results. To measure agreement, different statistical metrics are used. This study applies IAA for the first time to the Valencia Espaol Coloquial (Val.Es.Co.) discourse segmentation model, designed for segmenting and labelling spoken language into discourse units. Whereas most IAA studies merely label a set of in advance pre-defined units, this study applies IAA to the Val.Es.Co. protocol, which involves a more complex two-fold process: first, the speech continuum needs to be divided into units; second, the units have to be labelled. Kripendorffs u -family statistical metrics (Krippendorff et al. 2016) allow measuring IAA in both segmentation and labelling tasks. Three expert annotators segmented a spontaneous conversation into subacts, the minimal discursive unit of the Val.Es.Co. model, and labelled the resulting units according to a set of 10 subact categories. Kripendorffs u coefficients were applied in several rounds to elucidate whether the inclusion of a bigger number of categories and their distinction had an impact on the agreement results. The conclusions show high levels of IAA, especially in the annotation of procedural subact categories, where results reach coefficients over 0.8. This study validates the Val.Es.Co. model as an optimal method to fully analyze a conversation into pragmatically-based discourse units.
Verb Sense Clustering using Contextualized Word Representations for Semantic Frame Induction

Contextualized word representations have proven useful for various natural language processing tasks.However, it remains unclear to what extent these representations can cover hand-coded semantic information such as semantic frames, which specify the semantic role of the arguments associated with a predicate.In this paper, we focus on verbs that evoke different frames depending on the context, and we investigate how well contextualized word representations can recognize the difference of frames that the same verb evokes.We also explore which types of representation are suitable for semantic frame induction.In our experiments, we compare seven different contextualized word representations for two English frame-semantic resources, FrameNet and PropBank.We demonstrate that several contextualized word representations, especially BERT and its variants, are considerably informative for semantic frame induction.Furthermore, we examine the extent to which the contextualized representation of a verb can estimate the number of frames that the verb can evoke.
Corpus-Level Evaluation for Event QA: The IndiaPoliceEvents Corpus Covering the 2002 Gujarat Violence

Automated event extraction in social science applications often requires corpus-level evaluations: for example, aggregating text predictions across metadata and unbiased estimates of recall.We combine corpus-level evaluation requirements with a real-world, social science setting and introduce the INDIAPO-LICEEVENTS corpus-all 21,391 sentences from 1,257 English-language Times of India articles about events in the state of Gujarat during March 2002.Our trained annotators read and label every document for mentions of police activity events, allowing for unbiased recall evaluations.In contrast to other datasets with structured event representations, we gather annotations by posing natural questions, and evaluate off-the-shelf models for three different tasks: sentence classification, document ranking, and temporal aggregation of target events.We present baseline results from zero-shot BERT-based models fine-tuned on natural language inference and passage retrieval tasks.Our novel corpus-level evaluations and annotation approach can guide creation of similar social-science-oriented resources in the future.
Dynamic Dual-Peak Network: A real-time human detection network in crowded scenes

Human detection in crowded scenes is challenging since the objects occlude and overlap each other. Compared to general pedestrian detection, there is also more variation in human posture. This paper proposes a real-time human detection network, Dynamic Dual-Peak Network (DDPNet), which specifically addresses human object detection in overlapping and crowded scenes. We design a deep cascade fusion module to enhance the feature extraction capability of the anchor-free model for small objects in crowded scenes. In the meantime, the head–body dual-peak activation module is used to improve the prediction score of the central region of the occluded individual through low occlusion components. By this improvement strategy, the network’s ability is enhanced to discriminate individuals in crowded scenes and alleviate the problem caused by individual posture variation. Ultimately, we propose a novel Exhale–Inhale method to adjust the feature mapping ranges for various scale objects dynamically. In the process of ground truth mapping, the overlapping of individual feature information is reduced. Our DDPNet achieves competitive performance on the CrowdHuman dataset and executes real-time inference of almost 3x∼7x faster than competitive methods.
An Enhanced Advising Model in Teacher-Student Framework using State Categorization

The teacher-student framework aims to improve the sample efficiency of RL algorithms by deploying an advising mechanism in which a teacher helps a student by guiding its exploration. Prior work in this field has considered an advising mechanism where the teacher advises the student about the optimal action to take in a given state. However, real-world teachers can leverage domain expertise to provide more informative signals. Using this insight, we propose to extend the current advising framework wherein the teacher would provide not only the optimal action but also a qualitative assessment of the state. We introduce a novel architecture, namely Advice Replay Memory (ARM), to effectively reuse the advice provided by the teacher. We demonstrate the robustness of our approach by showcasing our experiments on multiple Atari 2600 games using a fixed set of hyper-parameters. Additionally, we show that a student taking help even from a sub-optimal teacher can achieve significant performance boosts and eventually outperform the teacher. Our approach outperforms the baselines even when provided with comparatively suboptimal teachers and an advising budget, which is smaller by orders of magnitude. The contributions of our paper are 4-fold (a) effectively leveraging a teacher's knowledge by richer advising (b) introduction of ARM to effectively reuse the advice throughout learning (c) ability to achieve significant performance boost even with a coarse state categorization (d) enabling the student to outperform the teacher.
Learning Instance-Level N-Ary Semantic Knowledge At Scale For Robots Operating in Everyday Environments

Robots operating in everyday environments need to effectively perceive, model, and infer semantic properties of objects. Existing knowledge reasoning frameworks only model binary relations between an object’s class label and its semantic properties, unable to collectively reason about object properties detected by different perception algorithms and grounded in diverse sensory modalities. We bridge the gap between multimodal perception and knowledge reasoning by introducing an n-ary representation that models complex, inter-related object properties. To tackle the problem of collecting n-ary semantic knowledge at scale, we propose a transformer neural network that directly generalizes knowledge from observations of object instances. The learned model can reason at different levels of abstraction, effectively predicting unknown properties of objects in different environmental contexts given different amounts of observed information. We quantitatively validate our approach against five prior methods on LINK, a unique dataset we contribute that contains 1457 situated object instances with 15 multimodal properties types and 200 total properties. Compared to the prior state of the art Markov Logic Network, our model obtains a 10% improvement in predicting unknown properties of novel object instances while reducing training and inference time by 150 times. Additionally, we apply our work to a mobile manipulation robot, demonstrating its ability to leverage n-ary reasoning to retrieve objects and actively detect object properties.
Texture Browser: Feature‐based Texture Exploration

Abstract Texture is a key characteristic in the definition of the physical appearance of an object and a crucial element in the creation process of 3D artists. However, retrieving a texture that matches an intended look from an image collection is difficult. Contrary to most photo collections, for which object recognition has proven quite useful, syntactic descriptions of texture characteristics is not straightforward, and even creating appropriate metadata is a very difficult task. In this paper, we propose a system to help explore large unlabeled collections of texture images. The key insight is that spatially grouping textures sharing similar features can simplify navigation. Our system uses a pre‐trained convolutional neural network to extract high‐level semantic image features, which are then mapped to a 2‐dimensional location using an adaptation of t‐SNE, a dimensionality‐reduction technique. We describe an interface to visualize and explore the resulting distribution and provide a series of enhanced navigation tools, our prioritized t‐SNE, scalable clustering, and multi‐resolution embedding, to further facilitate exploration and retrieval tasks. Finally, we also present the results of a user evaluation that demonstrates the effectiveness of our solution.
Transformation of the Human Image in the Paradigm of Knowledge Evolution

Purpose. The knowledge influence analysis on the formation process of new anthropological images of man in the contexts of scientific achievements and innovative technologies is the basis of this study. It involves the solution of the following tasks: 1) explication of the ontological content of knowledge in the anthropo-cultural senses of the epoch; 2) analysis of the knowledge influence on the process of forming a new type of man; 3) characteristics of the modern anthropological situation in the context of digital culture; 4) substantiation of interrelation of phatic communication with post-truth society in the dimensions of anthropo-social transformations of the present. Theoretical basis. Ontological content of knowledge determines the anthropo-cultural context of the epoch by forming a system of intellectual, value, social meanings of human life. The assertion of new anthropological types of man is caused by the changes in social and cultural space in the context of the growing influence of achievements in scientific knowledge and technology. Digital revolution as the process of expanding the possibilities of informational-digital reality, the substitution of knowledge for information gives rise to Homo digitalis – the digital man. He focuses on phatic communication, which in its intellectual meaninglessness is commensurate with the anthropological dimensions of post-truth society. Originality. It is substantiated that Homo digitalis is the result of a complex set of heterogeneous effects of scientific knowledge, which in modern post-truth conditions appears as different communicative practices. A condition for the formation of a new anthropological type of man in the perspective of scientific and technological progress is the affirmation of ethical wisdom. Conclusions. Knowledge in the process of evolution of socio-cultural life acquires constitutive significance for the process of formation of anthropological situations that manifest themselves in new images of man. Today, he is commensurate with the demands of digital culture, in which human life practices are increasingly becoming information and digital clusters embedded in the reality of a post-truth society. The means of overcoming the passivity of digital man is the formation of a new anthropological type based on a rethinking of the value system.
Attention-based Contextual Language Model Adaptation for Speech Recognition

Language modeling (LM) for automatic speech recognition (ASR) does not usually incorporate utterance level contextual information.For some domains like voice assistants, however, additional context, such as time at which an utterance was spoken, provides a rich input signal.We introduce an attention mechanism for training neural speech recognition language models on both text and nonlinguistic contextual data 1 .When applied to a large de-identified dataset of utterances collected by a popular voice assistant platform, our method reduces perplexity by 7.0% relative over a standard LM that does not incorporate contextual information.When evaluated on utterances extracted from the long tail of the dataset, our method improves perplexity by 9.0% relative over a standard LM and by over 2.8% relative when compared to a state-of-theart model for contextual LM.
AgreeSum: Agreement-Oriented Multi-Document Summarization

We aim to renew interest in a particular multidocument summarization (MDS) task which we call AgreeSum: agreement-oriented multidocument summarization.Given a cluster of articles, the goal is to provide abstractive summaries that represent information common and faithful to all input articles.Given the lack of existing datasets, we create a dataset for AgreeSum, and provide annotations on article-summary entailment relations for a subset of the clusters in the dataset.We aim to create strong baselines for the task by applying the top-performing pretrained singledocument summarization model PEGASUS onto AgreeSum, leveraging both annotated clusters by supervised losses, and unannotated clusters by T5-based entailment-related and language-related losses.Compared to other baselines, both automatic evaluation and human evaluation show better article-summary and cluster-summary entailment in generated summaries.On a separate note, we hope that our article-summary entailment annotations contribute to the community's effort in improving abstractive summarization faithfulness.
The Educational Community in the Face of COVID-19. Discursive Analysis on Vulnerability and Education

This article presents the results of research the objective of which was to know the opinion of directors of different educational centers about the management carried out in their centers and the possible difficulties encountered during the suspension of face-to-face classes caused by the first wave of the COVID-19 pandemic. The research method is qualitative, exploratory, descriptive, and inductive. The study sample is made up of 43 managers of educational centers, who were selected by applying an intentional sampling based on criteria of age, experience, ownership of the center, teachings that are taught, and the socioeconomic context where it is located. A questionnaire was used to collect the information, previously designed and validated through the judgment of experts, to inquire about the organization and management from the management team, the development of the academic task with the students, and the relationship with the families during the course period of cessation of classroom activities. To make an adequate approximation to the reality studied, an exhaustive content analysis of the speeches issued by the participants was carried out. Among the main conclusions of the research are the difficulties they have experienced during the closure of schools in relation to the lack of strategic planning to reconvert face-to-face education to the online format, scarcity of technological resources in centers and homes, training deficit in digital skills, increasing the digital divide, attention to students with special educational needs or communication problems with students and their families.
Application of web 3.0 technologies in distance education (by levels of higher education)

The purpose of the survey is to identify the need for Web 3.0-technologies in distance education among higher education seekers initial level (short cycle), first (bachelor's) level, second (master's) level, third (educational-scientific/educational-creative) level, scientific level among 438 applicants for higher education. Features (open source software (OSS) for developing, sharing and configuring programs for global use and application, built-in algorithms for analyzing and interpreting large amounts of data) and the benefits of Web 3.0 in higher education (the ability to organize collaboration on a social network, encourage globalization, improve data management, stimulate creativity and innovation, support the availability of mobile Internet). The correlation between the functions of Web 3.0-technologies in distance education and learning outcomes at all levels of higher education is established. Intelligence indicates a lack of comprehensive scientific research in the relevant field. The practical significance of the results of intelligence lies in the correlation of the functions of Web 3.0-technologies in distance education and learning outcomes at all levels of higher education.
The 3rd Grand Challenge of Lightweight 106-Point Facial Landmark Localization on Masked Faces

Facial landmark localization is a fundamental component in various face related applications, such as face recognition, 3D face reconstruction, facial pose estimation, face synthesis and forensics, etc. However, due to the global pandemic of COVID-19, the facial mask has become a common way to suppress the transmission of the virus. This apparently makes conventional facial landmark localization unfaithful and inefficient because of occlusion. To lead the cutting-edge algorithms on masked faces, we host the 3rd grand challenge of lightweight 106-point facial landmark localization in conjunction with ICME 2021, aiming to improve the accuracy and robustness of facial landmark localization in real-world situations, especially on masked faces. Specifically, we construct a new dataset, named JD-landmark-mask, on the basis of the previous two competitions. It contains about 27,000 face images of two kinds, with real and virtual masks, which are largely varied in identity, head pose, facial expression, and occlusion. Besides, the strict limitations of model size (2M) and computational complexity (100M Flops) are set up for the lightweight model. Finally, more than 80 worldwide universities and research institutes took part in the competition. We will briefly introduce the information of the competition, as well as algorithms and results of the top three teams in this paper.