PopSign ASL v1.0: An Isolated American Sign Language Dataset Collected via Smartphones
Authors: Thad Starner
Sean Forbes
Matthew So
David Martin
Rohit Sridhar
Gururaj Deshpande
Sam Sepah
Sahir Shahryar
Khushi Bhardwaj
Tyler Kwok
Daksh Sehgal
Saad Hassan
Bill Neubauer
Sofia Anandi Vempala
Alec Tan
Jocelyn Heath
Unnathi Utpal Kumar
Priyanka Vijayaraghavan Mosur
Tavenner M. Hall
Rajandeep Singh
Christopher Zhang Cui
Glenn Cameron
Sohier Dane
Garrett Tanzer
Venue: NeurIPS 2023 Datasets and Benchmarks Poster (2023)
Keywords: American Sign Language
gesture
dataset
TL;DR: Introduces PopSign ASL v1.0: a public isolated American Sign Language dataset of ~200,000 examples of 250 signs

Abstract: PopSign is a smartphone-based bubble-shooter game that helps hearing parents
of deaf infants learn sign language. To help parents practice their ability to sign,
PopSign is integrating sign language recognition as part of its gameplay. For
training the recognizer, we introduce the PopSign ASL v1.0 dataset that collects
examples of 250 isolated American Sign Language (ASL) signs using Pixel 4A
smartphone selfie cameras in a variety of environments. It is the largest publicly
available, isolated sign dataset by number of examples and is the first dataset to
focus on one-handed, smartphone signs. We collected over 210,000 examples
at 1944x2592 resolution made by 47 consenting Deaf adult signers for whom
American Sign Language is their primary language. We manually reviewed 217,866
of these examples, of which 175,022 (approximately 700 per sign) were the sign
intended for the educational game. 39,304 examples were recognizable as a sign
but were not the desired variant or were a different sign. We provide a training set
of 31 signers, a validation set of eight signers, and a test set of eight signers. A
baseline LSTM model for the 250-sign vocabulary achieves 82.1% accuracy (81.9%
class-weighted F1 score) on the validation set and 84.2% (83.9% class-weighted
F1 score) on the test set. Gameplay suggests that accuracy will be sufficient for
creating educational games involving sign language recognition.

PopSign ASL v1.0: An Isolated American Sign Language Dataset Collected via Smartphones
Thad Starner' Sean Forbes? Matthew So
David Martin'
Rohit Sridhar
Gururaj Deshpande' Sam Sepah Sahir Shahryar!
Khushi Bhardwaj²
Tyler Kwok Daksh Sehgal Saad Hassan Bill Neubauer
Sofia Anandi Vempala' Alec Tan Jocelyn Heath Unnathi Utpal Kumar
Priyanka Vijayaraghavan Mosur Tavenner M. Hall' Rajandeep Singh'
Christopher Zhang Cui Glenn Cameron Sohier Dane Garrett Tanzer'
Georgia Institute of Technology Deaf Professional Arts Network
Google "Rochester Institute of Technology
(thad,
mwm.tattt.wseo..m "thad,matthew.cc wtt.://ttitiiioo..aaaaaooooooott wwo.atttiiinn..koktttsiiddddddd org.:/oooorrrrooao..oo...nnnnnnn
edu
tkwok7 ( htt.
(khushi
a atan70, Jheatht hatpaaahhha..mmaiiitaaaaaagagaaa
sing
sean&dpan. tv, mm253@rit.edu, sahirshahryar@gmail.co
tavennerh@yahoo.co com, com,
{sepah, glencam, sohier, gtanzer]@google.com com
Abstract
PopSign is a smartphone-basee bubble-shooter game that helps hearing parents of deaf infants learn sign language. To help parents practice their ability to sign, PopSign is integrating sign language recognition as part of its gameplay. For training the recognizer, we introduce the PopSign ASL v1.0 dataset that collects examples of 250 isolated American Sign Language (ASL) signs using Pixel 4A smartphone selfie cameras in variety of environments. It is the largest publicly available, isolated sign dataset by number of examples and is the first dataset to focus on one-handed, smartphone signs. We collected over 210,000 examples at 1944x2592 resolution made by 47 consenting Deaf adult signers for whom American Sign Language their primary language. manually reviewed 217,866 of these examples, of which 175,022 (approximately 700 per sign) were the sign intended for the educational game. 39,304 examples were recognizable as a sign but were not the desired variant were different sign. We provide a training set 31 signers, a validation set of eight signers, and a test set of eight signers. A baseline LSTM model for the 250-sign vocabulary achieves 82.1 % accuracy (81.9% class- weighte FI score) on the validation set and 84.2% (83.9% class-weighted F1 score) the test set. Gameplay suggests that accuracy will be sufficient for creating educational games involving sign language recognition.
Introduction
PopSign is a bubble-shooter smartphone-basee game that helps hearing parents of deaf infants learn American Sign Language. It builds on significant past user studies on such games [Xu] 2013| Summet 2010]. PopSign focuses on vocabulary from the MacArthhu-Barte
Communicative Development
Inventories (CDI) [Fenson and Marchman, 2007], which are the first concepts one teaches a child in any language. Originally, PopSign required players recognize a short video of a sign and match it to one of five types of bubbles labeled with English words. The player shoots a bubble and, if it strikes two or more contiguous bubbles of the same type, all the bubbles disappear. The goal is to clear the screen. The initial game was enjoyable but had limitations as it solely emphasized receptive
37th Conference on Neural Information Processing Systems (NeurIPS 2023) Track on Datasets and Benchmarks.

---

How To Play
Sevies
Fire!
Practice
Rly
Put down your phone and practice the words!
"iging buunch the buitee
frog
30 moves left
Figure I: PopSignAI incorporates sign recognition as part of PopSign's bubble shooter game play. Left: Instructions. Middle: Practicing five signs before each level. Right: The player holds their phone and touches the screen with their thumb to aim the bubble. The other hand performs the sign to select which type of bubble will be shot. Here, the player is aiming at the red balls and signing FROG. Google's 's MediaPipe hand tracking overlays the player's selfie camera video in real-time.
skills. Incorporating sign language recognition into the game (see Figure [ allows parents practice signing themselves, developing expressive rather than just receptive signing skills. These expressive skills are necessary in order for hearing parents to teach sign language to their infants.
To create a version of PopSign that includes sign recognition, we collected large number of examples of 250 individual signs from people for whom ASL their primary language. Since PopSign only engages five English/ASL concepts at time, the expectation is that recognition engine, coupled with hand tracker such Google's ' MediaPipe, could run sufficiently well smartphone to support gameplay [Lugaresi et al., 2019]. This assumption has proven correct. The eventual goal a general-purpose recognition engine developed for the full 250-sign vocabulary as a plug-in for popular game creation engines, such as Unity, to support development of educational games in sign.
2 Background and Motivation
Ninety-five percent of deaf children are born to hearing parents [Moores 2000). and most of those parents never learn enough sign language to teach their children. The majority of deaf children of hearing parents remain significantly delayed in language development throughout their lives when compared with hearing children and deaf children of deaf parents [Johnson et al., 1989 Spencer b). The children's slow development of language has been attributed to incomplete language models and interaction [THmmilonn and Gillo-Martin 1986 Lederberg and Everhart 1998). An environment without access to language results in language deprivation, which leads to health, education and quality-of-life issues such as social isolation [Crowe] 2003 Twersky Glasner and Miller 2010). mental health problems (2-7x that of hearing children) Dammeyer 2009], and suicide (6-60x hearing) [Turner et al.) 2007 You 2017 In many cases, deaf children of hearing parents (in the United States) are fluent neither in English nor ASL [Woll 1998 Kannapell] 1989 For these
deaf individuals, language sometimes a life-long struggle Jean F. Andrews 2019]. Language acquisition is dependent upon the availability of that language and the opportunities a child [Clark]
jet al. 2001] or an adult learner [Kramsch] 2000 Schutz] 2005) have for interacting with skilled users of the language. Deaf children of hearing parents typically grow up in linguistically impoverished surroundings due to the inability of family members to use sign (Goldin-Meadww 1999) Goldin
Meadow and Mylander 1990]. The quantity and quality of adult-child language interaction at

---

early age has also been shown to affect the language development and subsequent school success of hearing children [Panagos] 1998]
Once a decision is made to use sign language, which 75% of parents eventually do Meadoww -rlans et al. 2003]. the family faces the task of learning new language in modality vriuull-gsstrrll that is foreign to them as the main avenue of communication. Local ASL classes may not focus on the topics and vocabulary that new parents need; one-on-one family government services may be helpful but are often infrequent; classes may not be accessible in terms of cost or location; and books can be dry, demotivating. and not the best method for conveying ASL [Summet 2010 Given these daunting obstacles, many families may opt to follow the advice of doctors and adopt cochlear implant and spoken language-only approaches Hall| [2017). However, it has been demonstrated that not only is some level of ASL exposure for deaf children empirically not harmful to their development Hall et al. [2099] and vastly superior to no language exposure (Singleton and Newport, 2005], but in fact, sign language is more effective at reducing cognitive delays and improving communication skills than hearing-basee based approaches Hall] [2077]. particularly among the many children whose brains are unable to understand speech through cochlear implants Humphries et al. [2012]. Further, while vocabulary knowledge alone does not constitute fluency or command of ASL, it has been shown that a medgedge of approximately 150 words is enough to increase the rate at which new words and grammatical skills are acquired (Marchman and Bates 1994). Even "survival level" signing is a worthwhile endeavor for families with deaf children Berger et al. (023). PopSign seeks to help address this silent crisis by providing hearing parents a tool by which they may learn ASL vocabulary.
2.1 Isolated Sign Language Recognition (ISLR) Datasets
PopSign ASL v1.0 collects examples of 250 isolated American Sign Language signs using the selfie camera on Pixel 4A smartphones in a variety of environments. It is the largest isolated sign language dataset publicly available, the first to focus on one-handed signing with smartphones, and one of the few of its size that has been manually reviewed.
The majority of isolated sign datasets collected in laboratory settings with controlled lighting and angles. Some have been collected by users with their own cameras in private settings. Sign language data collection carries its own unique challenges. For one, the data collected is usually in video format [Quer and Steinbach 2019). Video introduces variables such as lighting, background textures, field of view, resolution, participant diversity, and cost. Since participants are directly recorded, there are additional challenges in acquiring consent [Quer and Steinbach] 2019]. Finally, ASL has a vocabulary with many thousands of signs and varies considerably by region, ethnicity, gender, and even by household [LeMaster and Monaghan 2005).
T major publicly available ISLR datasets. ASLLVD, Purdu:86RVL-SLLI and RWTH-
BOSTON 50 were all collected in a studio setting [Athitsos et al. 2008 Martinez et al.) 2002 Zahedi et al.
2005 More recent efforts include WLASL, ASL-TOO-RGBD, MS-ASL, and ASL Citizen [L]
et al. 2020 Joze and Koller, 2018 Desai et al. 2023]. ASLLVD features a large number of signs, but the dataset has few videos per sign. Furthermore, since there are six signers at most per sign, generalization across users is difficult. RWTH-Bostor 50 and WLASL suffer from the same problem of having few videos per sign but not the same degree. Purdue RVL-SILL addresses the issue of having too few videos per sign, but it does not feature many signers. All of those datasets have been collected using studios for recording, which too prohibitively expensive for scalable data collection. ASL Citizen fixes many of these issues by collecting very large number of videos over many signs and features enough participants increase generalizability over users. The signs are collected via webcam, making large scale data collection less expensive. MS-ASL is cost-effective in collection, since it is scraped from public videos. However, participants do not explicitly consent to such scraped collections, and annotation can be difficult (as highlighted by |Bragg et al. (202T].) There are many datasets for other sign languages, some of which are listed in Table We note that these datasets suffer from the same issues in existing ASL datasets. In particular, there are often too few signs and signers or too few examples per sign for useful downstream applications. Furthermore, these sets are recorded under tightly controlled studio setups. limiting data collection and utility.
3

---

Dataset
Lang
Signs
Signers
Videos
Videos/Sign
Source
Citation
ASLLVD
ASL
3000
6
9800
3.27
Studio
2008
Purdue RVL-SLLL
ASL
101
14
2576
25.50
Studio
2002
RWTH-BOSTON
ASL
50
483
9.66
Studio
2005
WLASL
ASL
2000
119
21083
10.54
Studio
2020
MS-ASL
ASL
1000
222
25513
25.51
Web
2018
ASL Citizen
ASL
2731
52
83912
30.73
Webcam
2023
PopSign ASL v1.0
ASL
250
47
214326
857.30
Smartphone
BSLDict
BSL
9283
148
14210
1,53
Studio
2020
DEVISIGN-G
CSL
36
432
12.00
Studio
2013
DEVISIGN-D
CSL
500
6000
12.00
Studio
2013
DEVISIGN-L
CSL
2000
8
24000
12.00
Studio
2013
500
CSL
500
50
125000
250.00
Studio
2019
DGS Kinect 40
DGS
40
14
2800
70.00
Studio
2012
SMILE
DSGS
100
Studio
2018
GSL 982
GSL
982
4910
5.00
Studio
2012
INCLUDE
ISL
263
4287
16.30
Studio
2020
LSA64
LSA
64
10
3200
50.00
Studio
2016
LSE-Sign
LSE
2400
2400
1.00
Studio
2013
LSFB-ISOL
LSFB
395
100
47551
120.38
Studio
202T
BosphorusSign
TSL
855
10
>51300
>60.00
Studio
2016
BosphorusSign22K
TSL
744
6
22542
30.30
Studio
2020
AUTSL
TSL
226
43
38336
169.63
Studio
2020
Table I: Isolated Sign Language Datasets
2.2 Smartphone Signing
We took an approach similar to ASL Citizen but used smartphone selfie cameras for data collection. Smartphone cameras tend be of higher quality than webcams or cameras embedded laptop. Laptop and webcam cameras often have smaller field-of-view than recent smartphones, making it difficult for signer interact with keyboard or mouse during data collection and still be distant enough to capture the full sigming box (the volume of space in which signers move their hands while signing). Our ASI. Recorder data collection smartphone app (see below) requires the user to hold and interact with the phone with one hand while signing with the other. This posture is surprisingly commonplace for Deaf signers, who often make video calls from their smartphones. Signers either hold the phone or rest it desk their body while communicating. Signers are used to aligning the camera so that the lighting sufficient and the camera can see the signing hand and the face. While many signs are two-handed, smartphone signing Morris [2022b] has become common that many signers will adjust their signs so that they can be made with one hand or fingerspell a sign if the one-handed version is ambiguous. From collection standpoint, smartphones are simple ship to participants, the collection system self-contained, and most participants are familiar with how to smartphone.
3 Collection Methodology
The Deaf Professional Arts Network (DPAN) recruited 50 Deaf signers for whom ASL was their primary language, and 47 completed the task. DPAN shipped Pixel 4A smartphones to each participant with the ASLRccorder open-source data collection smartphone app and participants were given guidance on using iP) Participants returned the phones after completion. Upon receipt, DPAN confirmed that all recorded videos had been uploaded to an on-line repository.
3.1 Choosing Lemmas
Examples of processes for choosing which lemmas to include in a sign language dataset can be found for British Sign Language Fenlon et al. (2015). Sign Language of the Netherlands (NGT) Schülllr
hhtps://gihuu..oo/mmiii http://hhh...co///tahhowssss//// Recorder/ bttps///inyall.oom22299999

---

et al. [02T], ASL Hochgesang et al. (2018) and others. The closest for our purposes is the ASL CDI 2.0 Caselli et al. 2020] which was constructed by three expert signers updating the CDI for C.seell
concepts used with Deaf infants. For selecting the 250 one-handed signs in our subset of the CDI,
DPAN created reference videos based on the English version of all 563 concepts in the CDI as used
in Xu's research on ASL games Xu [2033), which was taken from Anderson and Reilly (2002). Five DPAN members, who have signed since childhood, agreed on how to sign each concept one-handed as if they were on a video conference and holding a smartphone with one hand. These five signers come from diverse regions and schools. Example videos were made for each agreed-upon sign. From these 563 videos, 250 signs were chosen in an attempt to avoid ones that are too similar (with limited success, see below). The PopSign game uses these reference videos to teach players how to sign.
3.2 ASeRecorder App
The ASLRecorder app was designed to terminate after 200 signs to encourage regular breaks throughout the task. Participants could restart the app to continue. Each signer provided approximately 5000 examples (20 examples of 250 individual signs). Participants were encouraged to sign in varying lighting conditions and with varying backdrops. Participants were discouraged from including others in their videos. The example video of each sign from DPAN could be accessed by the participants by clicking on the icon at the top of the recording screen.
ASL Recordrr attempts to capture participants' signing in manner similar to how the PopSign game is played. PopSign uses a hold-to-aim mechanism where the player signs the name of the intended bubble before releasing the screen; therefore, ASLRecoder implements a hold-to-record mechanism. Because phones cannot immediately begin and stop recording, we decided to record continuously. Saving the press and release events as timestamps allows us to easily adjust the start and end point of each clip later. We stored timestamps in the image description (EXIF) metadata of a thumbnail for imposed
each video. In order manage the file size of a continuous recording, limit on the recording, after which the video stops automatically. Vidcos are recorded using the selfie camera of the phone at 1944x2592 resolution at 30 frames per second. To improve the reliability of ASI. Recorder and reduce infrastructure overhead, we used Google Photos's 's automatic cloud backup service store the recordings. Each signer has a separate Google account associated with their phone that provides backup folder and silos each volunteer's 's data from that of the other participants. The phone automatically connects to the signer's Wi-Fi network and uploads the videos whenever the phone is powered externally. This feature allows DPAN staff to observe a participant's progress and monitor for any difficulties remotely.
ASLRecorder has 10 mandomly-selected signs per session that are presented in a sliding user interface, allowing users to freely go back and forth between the selected signs. The interface automatically moves to the next sign after the user releases the Record button, but users can swipe back to the previous sign if they wish to make a better recording. Because we only ask them to make one recording for each sign in a session, we assume that only the last recording for each sign is valid (within that session). At the end of each session, signers have the opportunity to review their videos and manually mark them as invalid if they felt they had performed the sign improperly. We kept timestamps for invalid clips, but they were not included in the training data.
Since the individual sign videos were recorded as continuous sessions, we decoded the videos and split them into individual videos for each recording using custom- made script Decoding strategies generally followed the assumption that, by instruction, users held the record button for the duration of their sign. We added buffer of 0.5 seconds by default on either side of the recording as users tend to press too late and release to early. Of 47 initial signers recruited, 31 signers could be decoded with this approach, splitting the 200 sign videos (plus redos) into individual videos for review. The recordings of 10 signers had to be split after making some manual adjustments to the buffer size. The remaining six signers tapped the button to start signing, rather than holding the button. Generally, these participants completed each sign in two seconds, allowing segmentation with modified script.
3.3 Review Procedure
We designed a custom annotation engine to review the sign videos, given in Figure 2 We classify our data into three categories:
py
5

---

For Each Sign Video
Load Annotation Engine
Sign Video
Unretogniaeblle Gesture?
Curfains Game Sign?
Removed Signs
PaySign Training Videas
Reserved for Lingustic Analysls
Figure 2: Review process.
the sign to be taught in the PopSign game
incorrect signs unintended variants of the game version of the sign
unrecognizable videos
video marked unrecognizable if the video is unplayable, empty, or does not include the hands fingers for most of the video. Vidcos with identifiable person (i.e., the face recognizable) besides the participant were also rejected. We mark videos with signs that do not match the DPAN reference sign as variant. These include signs that are distinct but have the same meaning the original sign, as well as signs for the same gloss word, but with different meanings. Repetitive motions within signs were accepted they common in ASL and understood by viewer. An example is SHOWER, with the fingers transitioning from closed to spread once, twice or three times.
Since the DPAN reference signs will be used for teaching vocabulary in the PopSign game, we use these videos for judging whether an example should be accepted. Reviewers include 13 undergraduate and graduate students from Georgia Tech, a volunteer, and professor. The team is supervised by Deaf researcher who advises on linguistic variants of the signs. Three signers on the team split the vocabulary into 130 signs that were expected to have low number of variants and 120 that were expected to have a high number of variants. Reviewers with familiarity of ASL were assigned to the signs with greater number of variants. Review was performed one sign time using custom Along with classification,, reviewers made notes in a coordination spreadsheet as
annotation engine F] to the types of variants they observed. Reviewers also provided status reports, bugs, feedback and feature requests to improve the annotation software.
Dataset Composition
217,865 videos were reviewed. Of these, 3,539 videos were rejected, and 175,022 were considered to be the desired sign, making an average of 700 correct examples per sign. 39,304 videos were judged be variants of the intended sign or different sign. As the variants can be informative for creating sign recognition systems for other purposes, we include them in the dataset [ resulting in a total of 2143226 total videos. PEN has 907 total examples, making the most represented in the dataset. CAR is the least represented with 467 total examples. Counting the number of examples to be used for training the game, SCISSORS, with 888 examples, most represented, while SHOWER, with 96, is the least represented. More information can be found in the supplementary material. Of the 47 signers in the dataset, 16 identify as male and 31 as female, and 13 are left-handed and 34 are right-handed. The dataset represents mix of skin tones and ages. Some participants would switch their signing hand during collection.
There were some homophones and some near-homophones in the dataset. FINGER/WAIT and MOUTH/LIPS are only differentiated by mouth movements in the reference videos. TOOTHIGLASSWINDOW and CHIN/LIPS have minor position differences relative to the lips. WAKE/AWAKE differ by an initial F-hand shape versus A-hand shape. THAT/STAY differ by wrist rotation. The requirement sign with one hand resulted in some pairs of signs that normally would be distinct to become very similar. For example, PERSON is performed with two flattened hands,
'htpss//gituu..com/Beee hwtp://wh.oooo///ece.e/ 23/hotey
"htp://signndaa.cccgaeeee

---

moving in parallel high-to-low on the body. When performed one-handed, the sign can become similar BESIDE.
The high number of unintended variants in the dataset (roughly 20%) was influenced by using an English gloss for prompting the participants during data collection. Signers may assume they know the sign desired based on the gloss and not refer to the reference example provided. Alternatively, while providing 5000 examples, is easy to forget which sign was desired, and we observed situations where participants would alternate between two signs. To further complicate matters, some of the English prompts have two meanings which caused some signers at times to provide the sign with the unintended meaning. Examples include CLOSE ("close the doors" versus "close to me") and DUCK ("swimming duck" versus "be sure to duck the overhead pipe"). For future data collection, a smaller sized reference video for each sign playing on the collection screen itself may improve consistency. However, even then, common variations which have small distinctions may be overlooked. For example, we requested THERE to be made with an upward facing palm pointing gesture, though it commonplace to use an index finger pointing gesture similar to how HE/SHE/IT is formed. The distinction could easily be overlooked in the reference video.
On the other hand, prompting with the English gloss has the benefit of collecting more variations of each sign, which could be valuable in making a more general-purpose sign recognition system (though labeling all of these unintended signs future work). In this dataset, we observed five classes with significant variants: Compound signs such as BEDROOM, BACKYARD, and REFRIGERATOR may vary from the reference example by the ordering of the parts or the formation of one or both of the constituent parts. I Lexicalization results in the parsing of the first letter of a sign's 's closest gloss and using the manual alphabet as the handshape of the sign. For example, a signer might use a R-hand shape for RED. Regional and cultural variations are commonplace. The sign for GARBAGE in this dataset would be understood as CABBAGE in many areas of the United States. PIZZA has a surprising number of regional variations. Iconic signs in the vocabulary often had variations. For example, TOOTHBRUSH may be formed by using the index finger as the toothbrush to mimic the back and forth motion. Alternatively, TOOTHBRUSH may be signed by mimicking holding toothbrush in the palm. VACUUM may be formed by imitating the motion made while vacuuming or by using the hand to mimic the internal fan of the device. Contextual understanding of the English gloss also caused variations. BEFORE in time is signed differently than when something comes sequentially BEFORE something else. Other examples include PRETEND, SAME, and CLOSE.
We assign 31 signers to the training set, eight to the validation set, and eight to the test set. The eight chosen for the validation set capture the broadest possible demographic and linguistic range in order to make validation results as representative of real-world performance as possible. Of the eight users, four were noted for exhibiting high variance (differed from DPAN's example sign), while one was noted for exhibiting middle levels of variance. In addition to variability, we classified signers according to the Fitzpatrick scale. numerical classification of human skin color. Of the validation signers, five had skin phenotype I, two had skin phenotype II, and two had skin phenotype V (a lower-numbered phenotype represents fairer skin). Finally, the gender of the validation users is evenly split between male and female. The test set has demographics proportional to the training set.
5 Experiments on PopSign ASL v1.0
Georgia Tech's PACE computing cluster PACE 2017 was used to extract features with MediaPipe Hands with detection confidence of 0.5 and a tracking confidence of 0. MediaPipe Hands extracts 21 key points, each containing an x, y and coordinate [Lugaresi et al. 2019). Preprocessing includes removing NaNs for each key point. All videos where left hand was used sign were flipped along the vertical axis to convert them into higdt-handed signs. The PopSign game recognizes when a player signing with the left hand and also flips the axis. Since there are much fewer left hand dominant signers, this technique can improve recognition rates for playing with either hand. All videos with zero frames of tracking were removed. To deal with variable sequence length, videos with less than 60 frames were padded with row with values set to -1 at the end of the sample. For videos with more than 60 frames, the middle 60 frames were kept while the rest were discarded. Each of the labels was converted into one-hot encoding.
7

---

% Times Confused (% of Dataset)
% Times confused of Dataset)
Confused Pair
Confused Pair
WAKE, AWAKE
47.0 (2.7)
PRETEND, HELLO
15.4 (0.8)
GLASSWINDOW, TOOTH
25.1 (1.2)
NAPKIN, FLOWER
26.0 (0.8)
EYE, TOOTH CHIN, SAY
19.8 (1.0)
INTO, ELEPHANT
11.6 (0.6)
19.9 (1.0)
FINGER, WAIT THAT, STAY
11.4 (0.6)
BESIDE, PERSON
18.8 (0.8)
10.8 (0.4)
Table 2: Most confused signs
Using Keras [ the model is built using two bidirectional LSTM layers followed by dropout layer and the output layer of 250 units with a softmax activation. The first bidirectional LSTM layer has 128 units and has return sequences set to True to allow timestep information to be preserved and passed to the second bidirectional LSTM layer. Default Keras LSTM parameter values are used. The input shape is manually set to (60, 63). The second bidirectional LSTM has 128 units and the default parameter values required for the LSTM layer in Keras remain. The third layer is a dropout layer with rate set to 0.5. The last layer is the 250 unit softmax output layer. During model training, a batch size of 32 is used along with the Adam optimizer and Categorical Cross Entropy Loss. The model set to train for 40 epochs, and the validation data used for early callback stopping if the validation loss converges. The weights that results in the lowest validation loss are restored. The model is trained in 30 minutes on one PACE GPU node (4x NVIDIA RTX 6000).
The recognition system achieved 82. % accuracy (81 (81.9% FI score) on the verification set and 84.2% accuracy (83.9 (3.99 F1 score) on the test set for all files that had least one frame of MediaPipe tracking. When including the files where tracking failed completely (109/29,96) 9499 validation; 171/33,667 test), accuracy decreases 81.9% validation and 83.8% test set, respectively. Upon inspection, the test set has some signers who are highly consistent with their signing, which explains the surprising increase in accuracy between verification and test. The top most confused pair of signs can be seen in Table 2 Many are near homophones mentioned earlier. For example, the video for AWAKE is often labeled WAKE, and vice versa. EYE versus TOOTH confusion is the result of Mediapipe Hands's features not being measured relative to the face. Adding Mediapipe Pose or switching Mediapipe Holistic, we can make run efficiently enough with Unity for PopSign, would significantly improve results.
6 Discussion and Future work
With 200,686 total videos, PopSign ASL v1.0 is the largest isolated sign language dataset publicly available. Besides residing at signdata.cc. ceaeittt.ccggtcch.cce.. dataset at RIT/NTID, UIUC, and DPAN, Unlike previous datasets, PopSign focuses on one-handed signing captured from smartphone selfie cameras. The face and signing hand represent significant amount of the area of the image. Each video is reviewed manually, with a large majority being labeled as the expected sign. The videos show large variety of lighting conditions and backgrounds.
expect to host the 1.1TB PopSign ASL v1.0
The PopSign game only considers five signs at a time. Thus, sets of signs can be chosen to minimize potential confusion by the recognition system. Seven sets of signs were chosen to create PopSignAI Preview, an initial version of the PopSign game with embedded sign recognition that can be found on the Android Play and Apple App stores For this croof-of-concepp a 2D-CNN was trained for each set of five signs and limited to approximately IMB in size in order to run efficiently on older smartphones such as the Pixel 4AA. In informal testing, this limited task increased recognition accuracy to approximately 95% when played by an experienced signer. Gameplay is quite compelling, suggesting that the PopSign ASL v1.0 dataset is suitable for creating sign language recognition based games. Of course, a 250-sign recognition system will allow many more levels of the PopSign game to be created, As the recognition system matures, it will be integrated plug-in for game creation engines such as Unity to enable more programmers to create games based on sign language.
"htts://githuu..om// www//eoaooe/eeeeeeeeeeeeeee -eeerassteem/keras
The app is available http:h/hw.g.eeecccccooooeooooeld
8

---

The lexicon was chosen with a particular application in mind: teaching hearing parents of deaf infants basic sign vocabulary. However, there are approximately 313 concepts in the MacArthur- -attes CDI not covered in the current dataset. A similar collection effort is underway to address these signs, which should expand the usefulness of the data. In speech recognition, small vocabulary isolated recognizers are used for voice dialing ("call" "father"), interactive phone menus ("press one or say 'balance' to get account balances"), multimodal control of windowing systems ("close" while pointing to window), searching voice mail by voice, and controlling wearable computers ("ok Glass" "show" "compass"), We hope this dataset can support similarly creative use cases for ASL.
Since collection is performed remotely by mailing smartphones to participants, researchers can use our tools and procedures to purposely sample different variants of signing. For example, one can ask participants to intentionally provide different versions of the sign based on their background. Signers who were raised with Black ASI. (Valli and Lucas 2000) or with a local sign language like Hawaiian Sign (Shroyer and Shroyer, 1984) may have used a different set of signs at home than at university. Such collection allows sign linguists to gain better understanding of regional signs and enables a general purpose sign recognizer to be more inclusive.
7 Ethics and Safety Discussion
"Nothing about us without us" is an often-heard phrase when discussing accessibility with those in the community. Without guidance, such as can be found in the FATE paper [Bragg et al. [202T) the many subtleties of sign collection can be confusing. Unfortunately, most machine learning researchers do not know about the existence of the Deaf culture, the fact that ASL has different grammar than English, or even that there are many distinct sign languages. Here we follow the convention that "deaf" refers to the medical condition, whereas "Deaf" refers to the community, which focuses sign language. Without Deaf members on the team, researchers can stray into creating technologies that are not useful or are even considered harmful by the community (Erard 2017 Hill 2020]. Deaf team members and the Deaf community have informed PopSign at every step. The idea that an educational game might be the first useful application of sign language recognition was posited in 2000 by Dr. Harley Hamilton, a sign linguist and technology coordinator at the Atlanta Area School for the Deaf. After several iterations of desktop games tested at Deaf schools in Georgia, student researcher suggested changing the focus to smartphones and introduced the team to the idea that ASL evolving due to smartphone technology [Morris] 2022a]. PopSign provides the ML community with clear need and a viable dataset for goal that can be achieved by sign recognition in the near term. Signers for the dataset were recruited by Deaf researchers and informed before beginning participation about the intent to create a public video database. DPAN's consent procedures were informed by those used by Georgia Tech's 's IRB. The risks of the dataset videos are similar to any identifiable video made available on the Internet. While names are not affiliated with any of the videos, it was clear in the consent process that participants' faces would be identifiable. Participants were paid $300 for providing 5000 signs, process that would take approximately six hours. Signers could stop participation without penalty; compensation is prorated. PopSign ASL v1.0 is provided under Creative Commons CC-BY 4.0 license, specifically so both academies and industry may use it. Software created as part of the collection and review process is covered by the standard MIT license. The dataset is not meant to be representative of ASSL, though there is a risk users might use it that way. The next section warns about the dataset's limitations.
8 Limitations
While ASL is the most common sign language used in the United States, there are many sign languages both in the United States (c.g., Plains Indian Sign Language and Hawaïan Sign Language) and in the world at large (c.g.. British Sign Language, French Sign Language, German Sign Language. Hindi Sign Language, etc.). In addition, there are many regional and cultural accents associated with sign in the United States, including Black American Sign Language. The PopSign dataset is designed for teaching one variant of signed concept, focusing on one-handed smartphone signing, but does not capture a representative sample of all the sign variations that would be commonly understood in conversation. ASL. has grammar that is very different from English, and isolated signs do not reflect its richness. A larger number of signers is necessary to better represent skin tones, hand features, ages and different levels of signing fluency. PopSign assumes one- mechaddedd sign from
9

---

the smartphone viewpoint and field of view, whereas a more general recognition system would also require two-handed signs and a greater variety of viewpoints and camera parameters. As collection required the use of a smartphone, communication via email, and viable shipping address, it biases the dataset towards younger, more affluent signers with facility in English.
Conclusion
The Popsign ASL v1.0 dataset provides large, reviewed dataset of signs with many examples per sign that demonstrate the concepts first taught to infants. It focuses on signing captured by the selfie camera on smartphone, and the data has already proven viable in creating a sign language recognition based educational game for helping hearing parents of deaf children learn ASL.
Acknowledgments and Disclosure of Funding
We would like to thank all of the signers who contributed to the project and especially Nathan Qualls and Michaela Jitaru at DPAN for their hard work data collection. Max Shengelia RIT helped guide the project toward smartphone signing. based on his observations in the community, and performed initial reviews of the sign. Graham MacKenzie provided perspective as an ASL interpreter on how one-handed smartphone signing is becoming commonplace and provided of which concepts in the CDI could be signed with one hand. RIT/NTID CAT lab is our partner in creating the next PopSign games. Finally, thanks the Kaggle team (Mark Sherwood, Ashley Chow, and Phil Culliton) for all their advice on how to best create useful datasets. This research was supported in part through research cyberinfrastructure resources and services provided by the Partnership for an Advanced Computing Environment (PACE) at the Georgia Institute of Technollogy, Atlanta, Georgia, USA.
initial analysis
DPAN is a non-profit, and its funding for this project came through non-restrictive gifts by private and public entities. Georgia Tech's contributions are through classwork and volunteer efforts. Google- affiliated personnel play an advisory role and participate on a volunteer basis.
References
YouTube, May 2017. httptp/swwww..pptttt..c/////////
D. Anderson and Reilly. The macarthur communicative development inventory: normative data for american sign language. Journal of deaf studies and deaf education, 7(2):83-106, 2002.
V. Athitsos, C. Neidle, Sclaroff, J. Nash, A. Stefan, Q. Yuan, and A. Thangali. The American Sign Language Lexicon Video Dataset. In 2008 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, pages 1-8, 2008. doi: 1009/CVPR..200884566633
L. Berger, J. Pyers, A. Lieberman, and N. Caselli. Parent american sign language skills correlate with child-but not toddler-asl vocabulary size. Language Acquisition, pages 1-15, 2023.
D. Bragg, N. Caselli, J. A. Hochgesang, M. Huenerfauth, L. Hatz-Herr,,dez, O. Koller, R. Kushal- nagar, C. Vogler, and R. E. Ladner. The FATE Landscape of Sign Language AI Datasets: An Interdisciplinary Perspective. ACM Trans. Access. Comput.. 14(2), July 2021. ISSN 1936-7228. doi: 10. 10.1145/3436996 URL /httt:::///oo USA Publisher: Association for Computing Machinery.
org/10.1r 1145/3436996 Place: New York, NY,
N. C. Camgöz, A. Kındıroglu, Karabükküü M. Kelepir, A. S. Özsoy, and L. Akarun. Bosphorus- Sign: A Turkish Sign Language Recognition Corpus in Health and Finance Domains. In Proceed- ings of the Tenth International Conference on Language Resources and Evaluation (LREC' 16), pages 1383-1388 Portorož, Slovenia, May 2016. 6. European Language Resources Association (ELRA). URL NtR./tttps///aaaathooll www.oollooooogogg.ggoooooo org/L16-/2200
N. K. Caselli, A. M. Lieberman, and J. E. Pyers. The asl-cdi 2.0: An updated, normed adaptation of the macarthur bates communicative development inventory for american sign language. Behavior Research Methods, 52:2071-2084, 2020.
10

---

X. Chai, G. Li. Y. Lin, Z. Xu, Y. B. Tang, and X. Chen. Sign Language Recognition and Translation with Kinect, 2013.
M. Clark, M. Marschark, and M. A. Karchmer. Context, cognition, and deafness. 2001.
Crowe. Self-esteem scores among deaf college students: An examination of gender and parents' hearing status and signing ability. Journal of deaf studies and deaf education, 8:199-206, 02 2003. doi: 10. 1093/deafeddeeg@003.
Dammeyer. Psychosocial development in danish population children with cochlear implants and deaf and hard-of of-hearin children. Journal of deaf studies and deaf education, 15:50-8, 10 2009. doi: 10. 1...0///eafeee/eep222.....
A. Desai, L. Berger, F. O. Minakov, V. Milan, C. Singh, Pumphrey, R. E. Ladner, H. D. au2, A. X. Lu, N. Caselli, and D. Bragg. ASL Citizen: A Hommunity-Sourcees Dataset for Advancing Isolated Sign Language Recognition, 2023, _eprint: 2304.05934.
Ebling, N. C. Camgöz, P. Boyes Braem, K. Tissi, Sidler-Miserez, S. Stoll, S. Hadfield, T. Haug. R. Bowden, S. Tornay, M. Razavi, and M. Magimai-Doss, SMILE Swiss German Sign Language Dataset. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018), Miyazaki, Japan, May 2018. European Language Resources Association (ELRA). URL htttttas:/aaaaattlllloooooo SELAAA,,ULh/pp::/aaaatlooooo org/ Barg/L18/ L-1-
M. Erard. Why sign-language gloves don't help deaf people. The Atlantic, 2017.
Fenlon, K. Cormier, and A. Schembri. Building bsl signbank: The lemma dilemma revisited.
International Journal of Lexicography, 28(2):1692206, 2015.
L. Fenson and V. Marchman. MacArthur-Baud Bates Communicative Development Inventories: User's ' Guide and Technical Manual. Paul H. Brookes Publishing Company, 2007. ISBN 9781557668844. URL
wwwpooooooooooo........oooooo... wtteeeeccooooooooeeeeoeeeeeeoeee bbt.//oooos///// wwwaa//oooossooooouuoaa .d=yu-KAAAAAAAA.
J. Fink, B. Frénay, L. Meurant, and A. Cleve. LSFB-CONE and LSFB-ISOL: Two New Datasets for RisoedBBased Sign Language Recognition, 2021 International Joint Conference on Neural Networks (IJCN), pages 1-8, 2021. doi: 10. 10991ICNN5533332222 2011.93336....
S. Goldin-Meadow. What children contribute to language-learning. Science Progress, 82(1): 89-102, 1999. doi: 10. 10.17700365009990000000
URL https://ddi.oorg/10
0033885449988200005
PMID: 10445008.
S. Goldin-Meadow and C. Mylander. The role of parental input in the development of morphological
system. Journal of Child Language, 17(3):527-563, 1990. doi: 10.1017/500000000000003333
E. Gutiérrez-Sigut, B. Costello, Baus, and M. Carreiras. LSE-Sign: A lexical database for Spanish Sign Language. Behavior research methods, 48, Jan. 2015. doi: 10.37558/13428-0144-000
M. L. Hall, W. C. Hall, and N. K. Caselli. Deaf children need language, not (just) speech. First Language, 39(4)3367-395, 2019. doi: 10. 10.117770142723719884418 /tt/tttts:://oo..rr//// A177/014272379983411
W. C. Hall. What you don't don't know can hurt you: The risk of language deprivation by impairing sign language development deaf children. Maternal and child health journal,, 21(5):961 -966, 2017.
H. Hamilton and D. C. Lillo-Martin. Imitative production of verbs of movement and location: comparative study. Sign Language Studies, 50:29 - 57, 1986.
J. C. Hill. Do deaf communities actually want sign language gloves? Nature Electronics, 3:512 513, 2020. URL httphhtts///pp..... wwt.:tiiscaccnooo..o..mm www./orrgrrrrreee 225565924
Hochgesang, O. Crasborn, and D. Lillo-Martin. Building the asl signbank. lemmatization principles for asl. 2018.
J. Huang, W. Zhou, H. Li, and W. Li. Attention-Based 3D-CNNs for Large- VoceblVoyabulary Sign Language Recognition. JEEE Transactions on Circuits and Systems for Video Technology. 29(9)22222-28) 2019. doi: 10. 10.1199/TCSVTT220888877777

---

T. Humphries, P. Kushalnagar, G. Mathur, D. J. Napoli. C. Padden, C. Rathmann, and S. R. Smith. Language acquisition for deaf children: Reducing the harms of zero tolerance the use of alternative approaches. Harm Reduction Journal, 9(1):1-9, 2012.
M. L. Jean F. Andrews, McCay Vernon. The deaf suspect/defendant and the bill of rights., 2019. of-rights.pdff
http:///aaaaaaaoo/aaaaaaaaaaaaaa Dhee----eeeaa--uueee--eeedaaaa Muaspect-Defeeddtt-bllll
R. Johnson, S. Liddell, and C. Erting. Unlocking the curriculum: Principles for achieving access in deaf education. 01 1989.
H. R. Joze and Koller. MS-ASL: A Large-Scale Data Set and Benchmark Understanding American Sign Language. CoRR, abs/ 2018. URL http:/tt////r http:////////////11///// 01053, arXiv: 1812.01053.
B. M. Kannapell. An examination of deaf college students' attitudes toward asl and english. 1989.
C. Kramsch. Second language acquisition, applied linguistics, and the teaching of foreign languages.
The Modern Language Journal, 84(3):311-322 2000. ISSN 00267902, 15404781. httht
//www jawor org/oog//e sttbbe/33be3356
A. Lederberg and V. Everhart. Communication between deaf children and their hearing mothers. Journal of Speech Language and Hearing Research, 41:887, 08 1998. doi: 10. 41044//shhr.414....
B. LeMaster and L. Monaghan. Variation in Sign Languages. In Companion
to Linguistic Anthropology, pages 141-165. John Wiley & Sons, Ltd, 2005. ISBN 978 278-0-470-99655-22 doi: https://doi.org/100 http:///.0.0..000000009992222220 on onlinelibbary wi www.ywllly...... com wwtp/////////.o..o.oo.oo/ wwwpellibrrrrwwwleee........
URL https://
http:///bbabb...11000//700000220 ch7 Section: _eprint:
D. LI, C. Rodriguez, X. Yu, and H. LI. Word-level Deep Sign Language Recognition from Video: A New Large-scale Dataset and Methods Comparison. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), Mar. 2020.
C. Lugaresi, J. Tang, H. Nash, C. McClanahan, E. Uboweja, M. Hays, F. Zhang, C.-L. Chang, M. G. Yong. J. Lee, W.-T. Chang, W. Hua, M. Georg, and M. Grundmann. Mediapipe: A framework for building perception pipelines. ArXiv, abs/1906.00$ 12,00011100108 2019.
V. Marchman and E. Bates, Continuity lexical and morphological development: test of the critical mass hypothesis. Journal of child language, 21:339-66, 06 1994. doi: 10.1017/ $0305000900009300.
A. Martinez, R. Wilbur, R. Shay, and A. Kak. Purdue RVL-SLLL ASL database for automatic recognition of American Sign Language. In Proceedings. Fourth IEEE International Conference Multimodal Interfaces, pages 167-172, 2002. doi: D0.100//CMM...20211166
K. Meadow-Orlans, D. Mertens, and M. Sass-Lehrer. Parents and their deaf children: The early years. Bibliovault OAI Repository, the University of Chicago Press, 01 2003.
O. Mercanoglu and H. Keles. AUTSL: A Large Scale Multi-Modal Turkish Sign Language Dataset and Baseline Methods. IEEE Access, 01888340-111355 Jan. 2020. doi: 10.1 C0.1109/ 3028072.
L. Momeni, G. Varol, S. Albanie, T. Afouras, and Zisserman. Watch, read and lookup: learning to spot signs from multiple supervisors. Oct. 2020.
D. Moores. Educating the deaf:: Psychology, principles, and practices. 2000.
A. Morris. How sign language evolves as our world does the new york times. https://www ameri htt.///mmeeeeemnna-llaaanuaaaeae changes
w http:///mmtttrrraaaee/////////// 2022a.
html html
A. Morris. How visual language evolves as world does. The New York Times, 2022b.
12

---

E.-J. Ong, H. Cooper, N. Pugeault, and R. Bowden. Sign Language Recognition using Sequential Pattern Trees. June 2012. doi: 10.1009CVV..2212222299....
PACE. (Pjartnership for an [A]dvanced [CJomputing (E]nvironment ((PACE)), 2017. URL
http::/www www.wpppcc.... maace.gatech... edu
R. J. Panagos. Meaningful differences in the everyday experience of young american children. Remedial and Special Education, 19(6):3113771 1998. doi: 10.11770044193259800000 https://dot org/ 10. 1177 1177/0741992599019600000
URL
J. Quer and M. Steinbach. Handling Sign Language Data: The Impact of Modality. Frontiers in Psychology, 10, Mar. 2019. doi: 200.8//9/ssy.2219004483
Ronchetti, M. Quiroga, C. Estrebou, L. Lanzarini, and A. Rosete. LSA64: An Argentinian Sign Language Dataset. 2016.
A. Schüller al. The lemma dilemma: Finding relevant lemmas include in the communicative development inventory for sign language of the netherlands (ngt-cdi). Master's thesis, 2021.
R. Schutz. Stephen krashen's theory of second language acquisition, 2005. http://aap..cc...tpppeesisnneeee htgpreeemmeeeiilaaaaaoosesss2a1.
E. H. Shroyer and S. P. Shroyer. Signs across America: A look at regional differences in American Sign Language. Gallaudet University Press, 1984.
Singleton and E. Newport When learners surpass their models: The acquisition of american sign language from inconsistent input. Cognitive psychology, 49:370-407, 01 2005, doi: 10.1016/j. stgpsych.200.05..00
P. Spencer. Communication behaviors of infants with hearing loss and their hearing mothers. Journal of speech and hearing research, 36 2:311-21, 1993a.
P. Spencer. The expressive communication of hearing mothers and deaf infants. American Annals of the Deaf, 138:275 283, 1993b.
A. Sridhar, R. G. Ganesan, P. Kumar, and M. Khapra. INCLUDE: A Large Scale Dataset for Indian Sign Language Recognition. Proceedings of the 28th ACM International Conference on Multimedia, MM '20. pages 1375, New York, NY, USA, 2020. Association for Computing Machinery. ISBN 978-1 450---5500--
doi: 10.1145/3394171.41133 org/10. 1141/1199/3311111. 771.111....411122 3413528 event-place: Seattle, WA, USA.
URL https:///ooi
V. Summet. Facilitating communication for deaf individuals with mobile technologies. 01 2010.
O. Turner, K. Windfuhr, and N. Kapur. Suicide in deaf populations: a literature review. Annals of General Psychiatry, 6:1-9. 1-9, 2007.
A. Twersky Glasner and K. Miller. Communication isolation as reported by group of deaf texas inmates. Western Criminology Review, 11, 07 2010.
C. Valli and C. Lucas. Linguistics of American sign language: An introduction. Gallaudet University Press, 2000.
B. Woll. Cultural and language diversity and the deaf experience. International Journal of Bilingual- ism, 2:104 105, 1998.
K. A. Xu. Facilitating american sign language learning for hearing parents of deaf children via mo- bile devices, Apr 2013. URL htppat/s///rppsiiooyy....a 472401d5- 4ef Sfaa- 1977736d5468
4ef
M. Zahedi, D. Keysers, T. Deselaers, and Ney. Combination of Tangent Distance and an Image Distortion Model for Appearance-Basee Sign Language Recognition. In W. G. Kropatsch, R. Sab- latnig, and A. Hanbury, editors, Pattern Recognition, pages 401-408, Berlin, Heidelberg, 2005. Springer Berlin Heidelberg. ISBN 978-3-540-31942-9,
O. Özdemir, A. A. Kindıroglu, N. Cihan Camgoz, and L. Akarun. BosphorusSign22k Sign Language Recognition Dataset. In Proceedings of the LREC2020 9th Workshop on the Representation and Processing of Sign Languagex: Sign Language Resources in the Service of the Language Community, Technological Challenges and Application Perspectives, 2020.
13


Reviews:
[{"review_id": "jKw3Cw4lwj", "reviewer": "Reviewer_HKTi", "summary": "", "questions": "", "limitations": "Yes. The authors have discussed limitations.", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "1. It is interesting to enable sign recognition for this kind of educational application. It may help to attract more users.\n2. It consists of over 200K videos with a high resolution, which is the largest one to the best of my knowledge.\n3. The videos are manually reviewed leading to high quality.", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": ""}, {"review_id": "YMzUB5t7ud", "reviewer": "Reviewer_cDoQ", "summary": "", "questions": "", "limitations": "The limitations are addressed, such as the different accents of sign language. ", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "The introduced dataset comprises a remarkable number of sign videos compared to other ASL datasets, and the review process for these videos is comprehensive.\n\nThe designed app is user-friendly and incorporates gamification, which could expedite parents' learning of sign language. During sign practice, the selfie camera is activated, enabling real-time hand recognition of the user. Additionally, rendered line approximations overlaid on the signing hand aid the user.", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": ""}, {"review_id": "2yakiDktd0", "reviewer": "Reviewer_BJjx", "summary": "", "questions": "", "limitations": "I am not sure how to interpret the accuracy rate in light of this sentence in the discussion \"The PopSign game only considers five signs at a time.\" Is the accuracy based on the task of choosing the correct match out of five possible options, or is it based on the full set of 250 signs? Much more detail is needed in section 5, and if the accuracy is out of five options that should be explicit in the abstract, as it is not analogous to other ISLR benchmarks.\n\n\"Largest publicly available dataset\" is true considering the number of examples, but includes a very small vocabulary of only 250 signs.", "rating": 7, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "This paper addresses an important area of research. I appreciate that there was clearly inclusion of deaf people in the research process, as authors and meaningful contributors. The contributions from DPAN are a particular strength. The specific context of supporting hearing parents in learning ASL is a really excellent one. I commend the authors on choosing an application that is not only interesting from a ML perspective, but is also valuable to the deaf community.", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": ""}]

Author rebuttal:

Meta-review: The paper introduces an isolated American sign language (ASL) dataset, PopSign ASL v1.0. It aims to enable sign recognition for an ASL educational mobile application, PopSign, which helps parents whose infants are deaf to learn ASL when playing a game. The dataset is the largest one in terms of the video number among ASL datasets.

The Ethics Reviews are generally positive.

One reviewer has concerns on dictionary size, one-handed and recognition accuracy, the rebuttal is sucessful and the reviewer leans to accept the paper though the score is 6. The other two reviewers are positive and one reviewer is satisfied with the rebuttal.

Based on the comments, rebuttals and discussions, this work is recommended for accept as poster.
Decision: Accept (Poster)