pwc-archive/papers-with-abstracts huggingface dataset:
Optimised Preprocessing for Automatic Mouth Gesture Classification
Authors: Maren Brumm, Rolf-Rainer Grigat
Date: 2020-05-01 00:00:00
Proceeding: LREC 2020 5
Tasks: Classification, General Classification, Gesture Recognition, Sign Language Translation, Translation

Mouth gestures are facial expressions in sign language, that do not refer to lip patterns of a spoken language. Research on this topic has been limited so far. The aim of this work is to automatically classify mouth gestures from video material by training a neural network. This could render time-consuming manual annotation unnecessary and help advance the field of automatic sign language translation. However, it is a challenging task due to the little data available as training material and the similarity of different mouth gesture classes. In this paper we focus on the preprocessing of the data, such as finding the area of the face important for mouth gesture recognition. Furthermore we analyse the duration of mouth gestures and determine the optimal length of video clips for classification. Our experiments show, that this can improve the classification results significantly and helps to reach a near human accuracy.
PE2LGP Animator: A Tool To Animate A Portuguese Sign Language Avatar
Authors: Pedro Cabral, Matilde Gon{\c{c}}alves, Hugo Nicolau, Lu{\'\i}sa Coheur, Ruben Santos
Date: 2020-05-01 00:00:00
Proceeding: LREC 2020 5

Software for the production of sign languages is much less common than for spoken languages. Such software usually relies on 3D humanoid avatars to produce signs which, inevitably, necessitates the use of animation. One barrier to the use of popular animation tools is their complexity and steep learning curve, which can be hard to master for inexperienced users. Here, we present PE2LGP, an authoring system that features a 3D avatar that signs Portuguese Sign Language. Our Animator is designed specifically to craft sign language animations using a key frame method, and is meant to be easy to use and learn to users without animation skills. We conducted a preliminary evaluation of the Animator, where we animated seven Portuguese Sign Language sentences and asked four sign language users to evaluate their quality. This evaluation revealed that the system, in spite of its simplicity, is indeed capable of producing comprehensible messages.
Translating an Aesop's Fable to Filipino Sign Language through 3D Animation
Authors: Mark Cueto, Winnie He, Rei Untiveros, Josh Zu{\~n}iga, Joanna Pauline Rivera
Date: 2020-05-01 00:00:00
Proceeding: LREC 2020 5

According to the National Statistics Office (2003) in the 2000 Population Census, the deaf community in the Philippines numbered to about 121,000 deaf and hard of hearing Filipinos. Deaf and hard of hearing Filipinos in these communities use the Filipino Sign Language (FSL) as the main method of manual communication. Deaf and hard of hearing children experience difficulty in developing reading and writing skills through traditional methods of teaching used primarily for hearing children. This study aims to translate an Aesop{'}s fable to Filipino Sign Language with the use of 3D animation resulting to a video output. The video created contains a 3D animated avatar performing the sign translations to FSL (mainly focusing on hand gestures which includes hand shape, palm orientation, location, and movement) on screen beside their English text equivalent and related images. The final output was then evaluated by FSL deaf signers. Evaluation results showed that the final output can potentially be used as a learning material. In order to make it more effective as a learning material, it is very important to consider the animation{'}s appearance, speed, naturalness, and accuracy. In this paper, the common action units were also listed for easier construction of animations of the signs.
LSE\_UVIGO: A Multi-source Database for Spanish Sign Language Recognition
Authors: Laura Doc{\'\i}o-Fern{\'a}ndez, Jos{\'e} Luis Alba-Castro, Soledad Torres-Guijarro, Eduardo Rodr{\'\i}guez-Banga, Manuel Rey-Area, Ania P{\'e}rez-P{\'e}rez, Sonia Rico-Alonso, Carmen Garc{\'\i}a-Mateo
Date: 2020-05-01 00:00:00
Proceeding: LREC 2020 5
Tasks: Sign Language Recognition

This paper presents LSE{\_}UVIGO, a multi-source database designed to foster research on Sign Language Recognition. It is being recorded and compiled for Spanish Sign Language (LSE acronym in Spanish) and contains also spoken Galician language, so it is very well fitted to research on these languages, but also quite useful for fundamental research in any other sign language. LSE{\_}UVIGO is composed of two datasets: LSE{\_}Lex40{\_}UVIGO, a multi-sensor and multi-signer dataset acquired from scratch, designed as an incremental dataset, both in complexity of the visual content and in the variety of signers. It contains static and co-articulated sign recordings, fingerspelled and gloss-based isolated words, and sentences. Its acquisition is done in a controlled lab environment in order to obtain good quality videos with sharp video frames and RGB and depth information, making them suitable to try different approaches to automatic recognition. The second subset, LSE{\_}TVGWeather{\_}UVIGO is being populated from the regional television weather forecasts interpreted to LSE, as a faster way to acquire high quality, continuous LSE recordings with a domain-restricted vocabulary and with a correspondence to spoken sentences.
Elicitation and Corpus of Spontaneous Sign Language Discourse Representation Diagrams
Authors: Michael Filhol
Date: 2020-05-01 00:00:00
Proceeding: LREC 2020 5
Tasks: Form, Translation

While Sign Languages have no standard written form, many signers do capture their language in some form of spontaneous graphical form. We list a few use cases (discourse preparation, deverbalising for translation, etc.) and give examples of diagrams. After hypothesising that they contain regular patterns of significant value, we propose to build a corpus of such productions. The main contribution of this paper is the specification of the elicitation protocol, explaining the variables that are likely to affect the diagrams collected. We conclude with a report on the current state of a collection following this protocol, and a few observations on the collected contents. A first prospect is the standardisation of a scheme to represent SL discourse in a way that would make them sharable. A subsequent longer-term prospect is for this scheme to be owned by users and with time be shaped into a script for their language.
The Synthesis of Complex Shape Deployments in Sign Language
Authors: Michael Filhol, John C. McDonald
Date: 2020-05-01 00:00:00
Proceeding: LREC 2020 5

Proform constructs such as classifier predicates and size and shape specifiers are essential elements of Sign Language communication, but have remained a challenge for synthesis due to their highly variable nature. In contrast to frozen signs, which may be pre-animated or recorded, their variability necessitates a new approach both to their linguistic description and to their synthesis in animation. Though the specification and animation of classifier predicates was covered in previous works, size and shape specifiers have to this date remain unaddressed. This paper presents an efficient method for linguistically describing such specifiers using a small number of rules that cover a large range of possible constructs. It continues to show that with a small number of services in a signing avatar, these descriptions can be synthesized in a natural way that captures the essential gestural actions while also including the subtleties of human motion that make the signing legible.
Signing as Input for a Dictionary Query: Matching Signs Based on Joint Positions of the Dominant Hand
Authors: Manolis Fragkiadakis, Victoria Nyst, Peter van der Putten
Date: 2020-05-01 00:00:00
Proceeding: LREC 2020 5
Tasks: Dynamic Time Warping

This study presents a new methodology to search sign language lexica, using a full sign as input for a query. Thus, a dictionary user can look up information about a sign by signing the sign to a webcam. The recorded sign is then compared to potential matching signs in the lexicon. As such, it provides a new way of searching sign language dictionaries to complement existing methods based on (spoken language) glosses or phonological features, like handshape or location. The method utilizes OpenPose to extract the body and finger joint positions. Dynamic Time Warping (DTW) is used to quantify the variation of the trajectory of the dominant hand and the average trajectories of the fingers. Ten people with various degrees of sign language proficiency have participated in this study. Each subject viewed a set of 20 signs from the newly compiled Ghanaian sign language lexicon and was asked to replicate the signs. The results show that DTW can predict the matching sign with 87{\%} and 74{\%} accuracy at the Top-10 and Top-5 ranking level respectively by using only the trajectory of the dominant hand. Additionally, more proficient signers obtain 90{\%} accuracy at the Top-10 ranking. The methodology has the potential to be used also as a variation measurement tool to quantify the difference in signing between different signers or sign languages in general.
Extending the Public DGS Corpus in Size and Depth
Authors: Thomas Hanke, Marc Schulder, Reiner Konrad, Elena Jahn
Date: 2020-05-01 00:00:00
Proceeding: LREC 2020 5

In 2018 the DGS-Korpus project published the first full release of the Public DGS Corpus. This event marked a change of focus for the project. While before most attention had been on increasing the size of the corpus, now an increase in its depth became the priority. New data formats were added, corpus annotation conventions were released and OpenPose pose information was published for all transcripts. The community and research portal websites of the corpus also received upgrades, including persistent identifiers, archival copies of previous releases and improvements to their usability on mobile devices.The research portal was enhanced even further, improving its transcript web viewer, adding a KWIC concordance view, introducing cross-references to other linguistic resources of DGS and making its entire interface available in German in addition to English. This article provides an overview of these changes, chronicling the evolution of the Public DGS Corpus from its first release in 2018, through its second release in 2019 until its third release in 2020.
SignHunter -- A Sign Elicitation Tool Suitable for Deaf Events
Authors: Thomas Hanke, Elena Jahn, Sabrina W{\"a}hl, Oliver B{\"o}se, Lutz K{\"o}nig
Date: 2020-05-01 00:00:00
Proceeding: LREC 2020 5

This paper presents SignHunter, a tool for collecting isolated signs, and discusses application possibilities. SignHunter is successfully used within the DGS-Korpus project to collect name signs for places and cities. The data adds to the content of a German Sign Language (DGS) {--} German dictionary which is currently being developed, as well as a freely accessible subset of the DGS Corpus, the Public DGS Corpus. We discuss reasons to complement a natural language corpus by eliciting concepts without context and present an application example of SignHunter.
An Isolated-Signing RGBD Dataset of 100 American Sign Language Signs Produced by Fluent ASL Signers
Authors: Saad Hassan, Larwan Berke, Elahe Vahdani, Longlong Jing, YingLi Tian, Matt Huenerfauth
Date: 2020-05-01 00:00:00
Proceeding: LREC 2020 5

We have collected a new dataset consisting of color and depth videos of fluent American Sign Language (ASL) signers performing sequences of 100 ASL signs from a Kinect v2 sensor. This directed dataset had originally been collected as part of an ongoing collaborative project, to aid in the development of a sign-recognition system for identifying occurrences of these 100 signs in video. The set of words consist of vocabulary items that would commonly be learned in a first-year ASL course offered at a university, although the specific set of signs selected for inclusion in the dataset had been motivated by project-related factors. Given increasing interest among sign-recognition and other computer-vision researchers in red-green-blue-depth (RBGD) video, we release this dataset for use by the research community. In addition to the RGB video files, we share depth and HD face data as well as additional features of face, hands, and body produced through post-processing of this data.
Approaches to the Anonymisation of Sign Language Corpora
Authors: Amy Isard
Date: 2020-05-01 00:00:00
Proceeding: LREC 2020 5
Tasks: Ethics

In this paper we survey the state of the art for the anonymisation of sign language corpora. We begin by exploring the motivations behind anonymisation and the close connection with the issue of ethics and informed consent for corpus participants. We detail how the the names which should be anonymised can be identified. We then describe the processes which can be used to anonymise both the video and the annotations belonging to a corpus, and the variety of ways in which these can be carried out. We provide examples for all of these processes from three sign language corpora in which anonymisation of the data has been performed.
Sign Language Motion Capture Dataset for Data-driven Synthesis
Authors: Pavel Jedli{\v{c}}ka, Zden{\v{e}}k Kr{\v{n}}oul, Jakub Kanis, Milo{\v{s}} {\v{Z}}elezn{\'y}
Date: 2020-05-01 00:00:00
Proceeding: LREC 2020 5

This paper presents a new 3D motion capture dataset of Czech Sign Language (CSE). Its main purpose is to provide the data for further analysis and data-based automatic synthesis of CSE utterances. The content of the data in the given limited domain of weather forecasts was carefully selected by the CSE linguists to provide the necessary utterances needed to produce any new weather forecast. The dataset was recorded using the state-of-the-art motion capture (MoCap) technology to provide the most precise trajectories of the motion. In general, MoCap is a device capable of accurate recording of motion directly in 3D space. The data contains trajectories of body, arms, hands and face markers recorded at once to provide consistent data without the need for the time alignment.
A survey of Shading Techniques for Facial Deformations on Sign Language Avatars
Authors: Ronan Johnson, Rosalee Wolfe
Date: 2020-05-01 00:00:00
Proceeding: LREC 2020 5

Of the five phonemic parameters in sign language (handshape, location, palm orientation, movement and nonmanual expressions), the one that still poses the most challenges for effective avatar display is nonmanual signals. Facial nonmanual signals carry a rich combination of linguistic and pragmatic information, but current techniques have yet to portray these in a satisfactory manner. Due to the complexity of facial movements, additional considerations must be taken into account for rendering in real time. Of particular interest is the shading areas of facial deformations to improve legibility. In contrast to more physically-based, compute-intensive techniques that more closely mimic nature, we propose using a simple, classic, Phong illumination model with a dynamically modified layered texture. To localize and control the desired shading, we utilize an opacity channel within the texture. The new approach, when applied to our avatar {``}Paula{''}, results in much quicker render times than more sophisticated, computationally intensive techniques.
Use Cases for a Sign Language Concordancer
Authors: Marion Kaczmarek, Michael Filhol
Date: 2020-05-01 00:00:00
Proceeding: LREC 2020 5
Tasks: Translation

This article treats about a Sign Language concordancer. In the past years, the need for content translated into Sign Language has been growing, and is still growing nowadays. Yet, unlike their text-to-text counterparts, Sign Language translators are not equipped with computer-assisted translation software. As we aim to provide them with such software, we explore the possibilities offered by a first tool: a Sign Language concordancer. It includes designing an alignments database as well as a search function to browse it. Testing sessions with professionals highlight relevant use cases for their professional practices. It can either comfort the translator when the results are identical, or show the importance of context when the results are different for a same expression. This concordancer is available online, and aim to be a collaborative tool. Though our current database is small, we hope for translators to invest themselves and help us to keep it expanding.
Towards Kurdish Text to Sign Translation
Authors: Zina Kamal, Hossein Hassani
Date: 2020-05-01 00:00:00
Proceeding: LREC 2020 5
Tasks: Translation

The resources and technologies for Sign language processing of resourceful languages are emerging, while the low-resource languages are falling behind. Kurdish is a multi-dialect language, and it is considered a low-resource language. It is spoken by approximately 30 million people in several countries, which denotes that it has a large community with hearing-impairments as well. This paper reports on a project which aims to develop the necessary data and tools to process the Sign language for Sorani as one of the spoken Kurdish dialects. We present the results of developing a dataset in HamNoSys and its corresponding SiGML form for the Kurdish Sign lexicon. We use this dataset to implement a sign-supported Kurdish tool to check the accuracy of the Sign lexicon. We tested the tool by presenting it to hearing-impaired individuals. The experiment showed that 100{\%} of the translated letters were understandable by a hearing-impaired person. The percentages were 65{\%} for isolated words, and approximately 30{\%} for the words in sentences. The data is publicly available at \url{https://github.com/KurdishBLARK/KurdishSignLanguage} for non-commercial use under the CC BY-NC-SA 4.0 licence
Recognition of Static Features in Sign Language Using Key-Points
Authors: Ioannis Koulierakis, Georgios Siolas, Eleni Efthimiou, Evita Fotinea, Andreas-Georgios Stafylopatis
Date: 2020-05-01 00:00:00
Proceeding: LREC 2020 5

In this paper we report on a research effort focusing on recognition of static features of sign formation in single sign videos. Three sequential models have been developed for handshape, palm orientation and location of sign formation respectively, which make use of key-points extracted via OpenPose software. The models have been applied to a Danish and a Greek Sign Language dataset, providing results around 96{\%}. Moreover, during the reported research, a method has been developed for identifying the time-frame of real signing in the video, which allows to ignore transition frames during sign recognition processing.
Collocations in Sign Language Lexicography: Towards Semantic Abstractions for Word Sense Discrimination
Authors: Gabriele Langer, Marc Schulder
Date: 2020-05-01 00:00:00
Proceeding: LREC 2020 5

In general monolingual lexicography a corpus-based approach to word sense discrimination (WSD) is the current standard. Automatically generated lexical profiles such as Word Sketches provide an overview on typical uses in the form of collocate lists grouped by their part of speech categories and their syntactic dependency relations to the base item. Collocates are sorted by their typicality according to frequency-based rankings. With the advancement of sign language (SL) corpora, SL lexicography can finally be based on actual language use as reflected in corpus data. In order to use such data effectively and gain new insights on sign usage, automatically generated collocation profiles need to be developed under the special conditions and circumstances of the SL data available. One of these conditions is that many of the prerequesites for the automatic syntactic parsing of corpora are not yet available for SL. In this article we describe a collocation summary generated from DGS Corpus data which is used for WSD as well as in entry-writing. The summary works based on the glosses used for lemmatisation. In addition, we explore how other resources can be utilised to add an additional layer of semantic grouping to the collocation analysis. For this experimental approach we use glosses, concepts, and wordnet supersenses.
Machine Learning for Enhancing Dementia Screening in Ageing Deaf Signers of British Sign Language
Authors: Xing Liang, Bencie Woll, Kapetanios Epaminondas, Anastasia Angelopoulou, Reda Al-Batat
Date: 2020-05-01 00:00:00
Proceeding: LREC 2020 5
Tasks: BIG-bench Machine Learning

Ageing trend in populations is correlated with increased prevalence of acquired cognitive impairments such as dementia. Although there is no cure for dementia, a timely diagnosis helps in obtaining necessary support and appropriate medication. With this in mind, researchers are working urgently to develop effective technological tools that can help doctors undertake early identification of cognitive disorder. In this paper, we introduce an automatic dementia screening system for ageing Deaf signers of British Sign Language (BSL), using Convolutional Neural Networks (CNN), by analysing the sign space envelope and facial expression of BSL signers using normal 2D videos from BSL corpus. Our approach firstly establishes an accurate real-time hand trajectory tracking model together with a real-time landmark facial motion analysis model to identify differences in sign space envelope and facial movement as the keys to identifying language changes associated with dementia. Based on the differences in patterns obtained from facial and trajectory motion data, CNN models (ResNet50/VGG16) are fine-tuned using Keras deep learning models to incrementally identify and improve dementia recognition rates. We report the results for two methods using different modalities (sign trajectory and facial motion), together with the performance comparisons between different deep learning CNN models in ResNet50 and VGG16. The experiments show the effectiveness of our deep learning based approach in terms of sign space tracking, facial motion tracking and early stage dementia performance assessment tasks. The results are validated against cognitive assessment scores as of our ground truth data with a test set performance of 87.88{\%}. The proposed system has potential for economical, simple, flexible, and adaptable assessment of other acquired neurological impairments associated with motor changes, such as stroke and Parkinson{'}s disease in both hearing and Deaf people.
Machine Translation from Spoken Language to Sign Language using Pre-trained Language Model as Encoder
Authors: Taro Miyazaki, Yusuke Morita, Masanori Sano
Date: 2020-05-01 00:00:00
Proceeding: LREC 2020 5
Tasks: Language Modeling, Language Modelling, Machine Translation, Translation

Sign language is the first language for those who were born deaf or lost their hearing in early childhood, so such individuals require services provided with sign language. To achieve flexible open-domain services with sign language, machine translations into sign language are needed. Machine translations generally require large-scale training corpora, but there are only small corpora for sign language. To overcome this data-shortage scenario, we developed a method that involves using a pre-trained language model of spoken language as the initial model of the encoder of the machine translation model. We evaluated our method by comparing it to baseline methods, including phrase-based machine translation, using only 130,000 phrase pairs of training data. Our method outperformed the baseline method, and we found that one of the reasons of translation error is from pointing, which is a special feature used in sign language. We also conducted trials to improve the translation quality for pointing. The results are somewhat disappointing, so we believe that there is still room for improving translation quality, especially for pointing.