pwc-archive/papers-with-abstracts huggingface dataset:
Extending a Model for Animating Adverbs of Manner in American Sign Language
Authors: Robyn Moncrief
Date: 2020-05-01 00:00:00
Proceeding: LREC 2020 5

The goal of this work is to show that a model produced to characterize adverbs of manner can be applied to a variety of neutral animated signs to be used towards avatar sign language synthesis. This case study presents the extension of a new approach that was first presented at SLTAT 2019 in Hamburg for modeling language processes that manifest themselves as modifications to the manual channel. This work discusses additions to the model to be effective for one-handed and two-handed signs, repeating and non-repeating signs, and signs with contact.
From Dictionary to Corpus and Back Again -- Linking Heterogeneous Language Resources for DGS
Authors: Anke M{\"u}ller, Thomas Hanke, Reiner Konrad, Gabriele Langer, Sabrina W{\"a}hl
Date: 2020-05-01 00:00:00
Proceeding: LREC 2020 5

The Public DGS Corpus is published in two different formats, that is subtitled videos for lay persons and lemmatized and annotated transcripts and videos for experts. In addition, a draft version with the first set of preliminary entries of the DGS dictionary (DW-DGS) to be completed in 2023 is now online. The Public DGS Corpus and the DW-DGS are conceived of as stand-alone products, but are nevertheless closely interconnected to offer additional and complementary informative functions. In this paper we focus on linking the published products in order to provide users access to corpus and corpus-based dictionary in various, interrelated ways. We discuss which links are thought to be useful and what challenges the linking of the products poses. In addition we address the inclusion of links to other, older lexical resources (LSP dictionaries).
Automatic Classification of Handshapes in Russian Sign Language
Authors: Medet Mukushev, Alfarabi Imashev, Vadim Kimmelman, S, Anara ygulova
Date: 2020-05-01 00:00:00
Proceeding: LREC 2020 5
Tasks: Classification, Clustering, General Classification, Hand Detection

Handshapes are one of the basic parameters of signs, and any phonological or phonetic analysis of a sign language must account for handshapes. Many sign languages have been carefully analysed by sign language linguists to create handshape inventories. This has theoretical implications, but also applied use, as it is important due to the need of generating corpora for sign languages that can be searched, filtered, sorted by different sign components (such as handshapes, orientation, location, movement, etc.). However, it is a very time-consuming process, thus only a handful of sign languages have such inventories. This work proposes a process of automatically generating such inventories for sign languages by applying automatic hand detection, cropping, and clustering techniques. We applied our proposed method to a commonly used resource: the Spreadthesign online dictionary (www.spreadthesign.com), in particular to Russian Sign Language (RSL). We then manually verified the data to be able to perform classification. Thus, the proposed pipeline can serve as an alternative approach to manual annotation, and can help linguists in answering numerous research questions in relation to handshape frequencies in sign languages.
Design and Evaluation for a Prototype of an Online Tool to Access Mathematics Notions in Sign Language
Authors: Camille Nadal, Christophe Collet
Date: 2020-05-01 00:00:00
Proceeding: LREC 2020 5
Tasks: Navigate

The Sign{'}Maths project aims at giving access to pedagogical resources in Sign Language (SL). It will provide Deaf students and teachers with mathematics vocabulary in SL, this in order to contribute to the standardisation of the vocabulary used at school. The work conducted led to Sign{'}Maths, an online interactive tool that gives Deaf students access to mathematics definitions in SL. A group of mathematics teachers for Deafs and teachers experts in SL collaborated to create signs to express mathematics concepts, and to produce videos of definitions, examples and illustrations for these concepts. In parallel, we are working on the conception and the design of Sign{'}Maths software and user interface. Our research work investigated ways to include SL in pedagogical resources in order to present information but also to navigate through the content. User tests revealed that users appreciate the use of SL in a pedagogical resource. However, they pointed out that SL content should be complemented with French to support bilingual education. Our final solution takes advantage of the complementarity of SL, French and visual content to provide an interface that will suit users no matter what their education background is. Future work will investigate a tool for text and signs{'} search within Sign{'}Maths.
STS-korpus: A Sign Language Web Corpus Tool for Teaching and Public Use
Authors: Zrajm {\"O}qvist, Nikolaus Riemer Kankkonen, Johanna Mesch
Date: 2020-05-01 00:00:00
Proceeding: LREC 2020 5
Tasks: STS

In this paper we describe STS-korpus, a web corpus tool for Swedish Sign Language (STS) which we have built during the past year, and which is now publicly available on the internet. STS-korpus uses the data of Swedish Sign Language Corpus (SSLC) and is primarily intended for teachers and students of sign language. As such it is created to be simple and user-friendly with no download or setup required. The user interface allows for searching {--} with search results displayed as a simple concordance {--} and viewing of videos with annotations. Each annotation also provides additional data and links to the corresponding entry in the online Swedish Sign Language Dictionary. We describe the corpus, its appearance and search syntax, as well as more advanced features like access control and dynamic content. Finally we say a word or two about the role we hope it will play in the classroom, and something about the development process and the software used. STS-korpus is available here: https://teckensprakskorpus.su.se
Unsupervised Term Discovery for Continuous Sign Language
Authors: Korhan Polat, Murat Sara{\c{c}}lar
Date: 2020-05-01 00:00:00
Proceeding: LREC 2020 5
Tasks: Sign Language Recognition

Most of the sign language recognition (SLR) systems rely on supervision for training and available annotated sign language resources are scarce due to the difficulties of manual labeling. Unsupervised discovery of lexical units would facilitate the annotation process and thus lead to better SLR systems. Inspired by the unsupervised spoken term discovery in speech processing field, we investigate whether a similar approach can be applied in sign language to discover repeating lexical units. We adapt an algorithm that is designed for spoken term discovery by using hand shape and pose features instead of speech features. The experiments are run on a large scale continuous sign corpus and the performance is evaluated using gloss level annotations. This work introduces a new task for sign language processing that has not been addressed before.
The Corpus of Finnish Sign Language
Authors: Juhana Salonen, Antti Kronqvist, Tommi Jantunen
Date: 2020-05-01 00:00:00
Proceeding: LREC 2020 5

This paper presents the Corpus of Finnish Sign Language (Corpus FinSL), a structured and annotated collection of Finnish Sign Language (FinSL) videos published in May 2019 in FIN-CLARIN{'}s Language Bank of Finland. The corpus is divided into two subcorpora, one of which comprises elicited narratives and the other conversations. All of the FinSL material has been annotated using ELAN and the lexical database Finnish Signbank. Basic annotation includes ID-glosses and translations into Finnish. The anonymized metadata of Corpus FinSL has been organized in accordance with the IMDI standard. Altogether, Corpus FinSL contains nearly 15 hours of video material from 21 FinSL users. Corpus FinSL has already been exploited in FinSL research and teaching, and it is predicted that in the future it will have a significant positive impact on these fields as well as on the status of the sign language community in Finland. Keywords: Corpus of Finnish Sign Language, Language Bank of Finland, Finnish Signbank, annotation, metadata, research, teaching
Tools for the Use of SignWriting as a Language Resource
Authors: Antonio F. G. Sevilla, Alberto D{\'\i}az Esteban, Jos{\'e} Mar{\'\i}a Lahoz-Bengoechea
Date: 2020-05-01 00:00:00
Proceeding: LREC 2020 5

Representation of linguistic data is an issue of utmost importance when developing language resources, but the lack of a standard written form in sign languages presents a challenge. Different notation systems exist, but only SignWriting seems to have some use in the native signer community. It is, however, a difficult system to use computationally, not based on a linear sequence of characters. We present the project {``}VisSE{''}, which aims to develop tools for the effective use of SignWriting in the computer. The first of these is an application which uses computer vision to interpret SignWriting, understanding the meaning of new or existing transcriptions, or even hand-written images. Two additional tools will be able to consume the result of this recognizer: first, a textual description of the features of the transcription will make it understandable for non-signers. Second, a three-dimensional avatar will be able to reproduce the configurations and movements contained within the transcription, making it understandable for signers even if not familiar with SignWriting. Additionally, the project will result in a corpus of annotated SignWriting data which will also be of use to the computational linguistics community.
Video-to-HamNoSys Automated Annotation System
Authors: Victor Skobov, Yves Lepage
Date: 2020-05-01 00:00:00
Proceeding: LREC 2020 5

The Hamburg Notation System (HamNoSys) was developed for movement annotation of any sign language (SL) and can be used to produce signing animations for a virtual avatar with the JASigning platform. This provides the potential to use HamNoSys, i.e., strings of characters, as a representation of an SL corpus instead of video material. Processing strings of characters instead of images can significantly contribute to sign language research. However, the complexity of HamNoSys makes it difficult to annotate without a lot of time and effort. Therefore annotation has to be automatized. This work proposes a conceptually new approach to this problem. It includes a new tree representation of the HamNoSys grammar that serves as a basis for the generation of grammatical training data and classification of complex movements using machine learning. Our automatic annotation system relies on HamNoSys grammar structure and can potentially be used on already existing SL corpora. It is retrainable for specific settings such as camera angles, speed, and gestures. Our approach is conceptually different from other SL recognition solutions and offers a developed methodology for future research.
Cross-Lingual Keyword Search for Sign Language
Authors: Nazif Can Tamer, Murat Sara{\c{c}}lar
Date: 2020-05-01 00:00:00
Proceeding: LREC 2020 5
Tasks: Information Retrieval, Retrieval

Sign language research most often relies on exhaustively annotated and segmented data, which is scarce even for the most studied sign languages. However, parallel corpora consisting of sign language interpreting are rarely explored. By utilizing such data for the task of keyword search, this work aims to enable information retrieval from sign language with the queries from the translated written language. With the written language translations as labels, we train a weakly supervised keyword search model for sign language and further improve the retrieval performance with two context modeling strategies. In our experiments, we compare the gloss retrieval and cross language retrieval performance on RWTH-PHOENIX-Weather 2014T dataset.
One Side of the Coin: Development of an ASL-English Parallel Corpus by Leveraging SRT Files
Authors: Rafael Trevi{\~n}o, Julie A. Hochgesang, Emily P. Shaw, Nic Willow
Date: 2020-05-01 00:00:00
Proceeding: LREC 2020 5
Tasks: Translation

We report on a method used to develop a sizable parallel corpus of English and American Sign Language (ASL). The effort is part of the Gallaudet University Documentation of ASL (GUDA) project, which is currently coordinated by an interdisciplinary team from the Department of Linguistics and the Department of Interpretation and Translation at Gallaudet University. Creation of the parallel corpus makes use of the available SRT (SubRip Subtitle) files of ASL videos that have been interpreted into or from English, or captioned into English. The corpus allows for one-way searches based on the English translation or interpretation, which is useful for translators, interpreters, and those conducting comparative analyses. We conclude with a discussion of important considerations for this method of constructing a parallel corpus, as well as next steps that will help to refine the development and utility of this type of corpus.
Fewer is More: Image Segmentation Based Weakly Supervised Object Detection with Partial Aggregation
Authors: Ce Ge, Jingyu Wang, Qi Qi, Haifeng Sun, Jianxin Liao
Date: 2018-09-04 00:00:00
Proceeding: BMVC 2018 9
Tasks: Image Segmentation, Object, object-detection, Object Detection, Semantic Segmentation, Weakly Supervised Object Detection

We consider addressing the major failures in weakly supervised object detectors. As most weakly supervised object detection methods are based on pre-generated proposals, they often show two false detections: (i) group multiple object instances with one bounding box, and (ii) focus on only parts rather than the whole objects. We propose an image segmentation framework to help correctly detect individual instances. The input images are first segmented into several sub-images based on the proposal overlaps to uncouple the grouping objects. Then the batch of sub-images are fed into the convolutional network to train an object detector. Within each sub-image, a partial aggregation strategy is adopted to dynamically select a portion of the proposal-level scores to produce the sub-image-level output. This regularizes the model to learn context knowledge about the object content. Finally, the outputs of the sub-images are pooled together as the model prediction. The ideas are implemented with VGG-D backbone to be comparable with recent state-of-the-art weakly supervised methods. Extensive experiments on PASCAL VOC datasets show the superiority of our design. The proposed model outperforms other alternatives on detection, localization, and classification tasks.
Towards automatic visual inspection: A weakly supervised learning method for industrial applicable object detection
Authors: Ce Ge, Jing Wang, Jingyu Wang, Qi Qi, Haifeng Sun, Jianxin Liao:
Date: 2020-10-01 00:00:00
Proceeding: Computers in Industry 2020 10
Tasks: Deep Learning, Diversity, object-detection, Object Detection, Weakly-supervised Learning, Weakly Supervised Object Detection

Industrial visual detection is an essential part in modern industry for equipment maintenance and inspection. With the recent progress of deep learning, advanced industrial object detectors are built for smart industrial applications. However, deep learning methods are known data-hungry: the processes of data collection and annotation are labor-intensive and time-consuming. It is especially impractical in industrial scenarios to collect publicly available datasets due to the inherent diversity and privacy. In this paper, we explore automation of industrial visual inspection and propose a segmentation-aggregation framework to learn object detectors from weakly annotated visual data. The used minimum annotation is only image-level category labels without bounding boxes. The method is implemented and evaluated on collected insulator images and public PASCAL VOC benchmarks to verify its effectiveness. The experiments show that our models achieve high detection accuracy and can be applied in industry to achieve automatic visual inspection with minimum annotation cost.
DetectoRS: Detecting Objects with Recursive Feature Pyramid and Switchable Atrous Convolution
Authors: Siyuan Qiao, Liang-Chieh Chen, Alan Yuille
Date: 2020-06-03 00:00:00
Proceeding: CVPR 2021 1
Tasks: Instance Segmentation, Object, object-detection, Object Detection, Panoptic Segmentation, Segmentation, Semantic Segmentation

Many modern object detectors demonstrate outstanding performances by using the mechanism of looking and thinking twice. In this paper, we explore this mechanism in the backbone design for object detection. At the macro level, we propose Recursive Feature Pyramid, which incorporates extra feedback connections from Feature Pyramid Networks into the bottom-up backbone layers. At the micro level, we propose Switchable Atrous Convolution, which convolves the features with different atrous rates and gathers the results using switch functions. Combining them results in DetectoRS, which significantly improves the performances of object detection. On COCO test-dev, DetectoRS achieves state-of-the-art 55.7% box AP for object detection, 48.5% mask AP for instance segmentation, and 50.0% PQ for panoptic segmentation. The code is made publicly available.
Equivariant Flows: Exact Likelihood Generative Learning for Symmetric Densities
Authors: Jonas Köhler, Leon Klein, Frank Noé
Date: 2020-06-03 00:00:00

Normalizing flows are exact-likelihood generative neural networks which approximately transform samples from a simple prior distribution to samples of the probability distribution of interest. Recent work showed that such generative models can be utilized in statistical mechanics to sample equilibrium states of many-body systems in physics and chemistry. To scale and generalize these results, it is essential that the natural symmetries in the probability density -- in physics defined by the invariances of the target potential -- are built into the flow. We provide a theoretical sufficient criterion showing that the distribution generated by \textit{equivariant} normalizing flows is invariant with respect to these symmetries by design. Furthermore, we propose building blocks for flows which preserve symmetries which are usually found in physical/chemical many-body particle systems. Using benchmark systems motivated from molecular physics, we demonstrate that those symmetry preserving flows can provide better generalization capabilities and sampling efficiency.
Learning to Scan: A Deep Reinforcement Learning Approach for Personalized Scanning in CT Imaging
Authors: Ziju Shen, YuFei Wang, Dufan Wu, Xu Yang, Bin Dong
Date: 2020-06-03 00:00:00
Tasks: compressed sensing, Computed Tomography (CT), CT Reconstruction, Deep Reinforcement Learning, Reinforcement Learning (RL)

Computed Tomography (CT) takes X-ray measurements on the subjects to reconstruct tomographic images. As X-ray is radioactive, it is desirable to control the total amount of dose of X-ray for safety concerns. Therefore, we can only select a limited number of measurement angles and assign each of them limited amount of dose. Traditional methods such as compressed sensing usually randomly select the angles and equally distribute the allowed dose on them. In most CT reconstruction models, the emphasize is on designing effective image representations, while much less emphasize is on improving the scanning strategy. The simple scanning strategy of random angle selection and equal dose distribution performs well in general, but they may not be ideal for each individual subject. It is more desirable to design a personalized scanning strategy for each subject to obtain better reconstruction result. In this paper, we propose to use Reinforcement Learning (RL) to learn a personalized scanning policy to select the angles and the dose at each chosen angle for each individual subject. We first formulate the CT scanning process as an MDP, and then use modern deep RL methods to solve it. The learned personalized scanning strategy not only leads to better reconstruction results, but also shows strong generalization to be combined with different reconstruction algorithms.
Emergent Multi-Agent Communication in the Deep Learning Era
Authors: Angeliki Lazaridou, Marco Baroni
Date: 2020-06-03 00:00:00
Tasks: Deep Learning

The ability to cooperate through language is a defining feature of humans. As the perceptual, motory and planning capabilities of deep artificial networks increase, researchers are studying whether they also can develop a shared language to interact. From a scientific perspective, understanding the conditions under which language evolves in communities of deep agents and its emergent features can shed light on human language evolution. From an applied perspective, endowing deep networks with the ability to solve problems interactively by communicating with each other and with us should make them more flexible and useful in everyday life. This article surveys representative recent language emergence studies from both of these two angles.
DGSAC: Density Guided Sampling and Consensus
Authors: Lokender Tiwari, Saket Anand
Date: 2020-06-03 00:00:00
Tasks: Clustering, Model Selection, Motion Segmentation

Robust multiple model fitting plays a crucial role in many computer vision applications. Unlike single model fitting problems, the multi-model fitting has additional challenges. The unknown number of models and the inlier noise scale are the two most important of them, which are in general provided by the user using ground-truth or some other auxiliary information. Mode seeking/ clustering-based approaches crucially depend on the quality of model hypotheses generated. While preference analysis based guided sampling approaches have shown remarkable performance, they operate in a time budget framework, and the user provides the time as a reasonable guess. In this paper, we deviate from the mode seeking and time budget framework. We propose a concept called Kernel Residual Density (KRD) and apply it to various components of a multiple-model fitting pipeline. The Kernel Residual Density act as a key differentiator between inliers and outliers. We use KRD to guide and automatically stop the sampling process. The sampling process stops after generating a set of hypotheses that can explain all the data points. An explanation score is maintained for each data point, which is updated on-the-fly. We propose two model selection algorithms, an optimal quadratic program based, and a greedy. Unlike mode seeking approaches, our model selection algorithms seek to find one representative hypothesis for each genuine structure present in the data. We evaluate our method (dubbed as DGSAC) on a wide variety of tasks like planar segmentation, motion segmentation, vanishing point estimation, plane fitting to 3D point cloud, line, and circle fitting, which shows the effectiveness of our method and its unified nature.
Explaining Multi-stage Tasks by Learning Temporal Logic Formulas from Suboptimal Demonstrations
Authors: Glen Chou, Necmiye Ozay, Dmitry Berenson
Date: 2020-06-03 00:00:00

We present a method for learning multi-stage tasks from demonstrations by learning the logical structure and atomic propositions of a consistent linear temporal logic (LTL) formula. The learner is given successful but potentially suboptimal demonstrations, where the demonstrator is optimizing a cost function while satisfying the LTL formula, and the cost function is uncertain to the learner. Our algorithm uses the Karush-Kuhn-Tucker (KKT) optimality conditions of the demonstrations together with a counterexample-guided falsification strategy to learn the atomic proposition parameters and logical structure of the LTL formula, respectively. We provide theoretical guarantees on the conservativeness of the recovered atomic proposition sets, as well as completeness in the search for finding an LTL formula consistent with the demonstrations. We evaluate our method on high-dimensional nonlinear systems by learning LTL formulas explaining multi-stage tasks on 7-DOF arm and quadrotor systems and show that it outperforms competing methods for learning LTL formulas from positive examples.