Detecting Sentences that May be Harmful to Children with Special Needs Children and adults with special needs may find it difficult to recognize danger and threats as well as socially complex situations. Thus, they are under the risk of being victims of exploitation, violence and attacks. In addition, they may find themselves unintendedly insulting their friends, relatives or caregivers. In this paper, we propose an autonomous agent to assist the special needs person (child or adult) in the goal of recognizing risky or insulting situations. The autonomous agent will detect these situations and will signal them to the user (by text, speech, or other signaling forms). We composed a dataset containing 13,490 sentences, categorized into one of four classes: a u0027normalu0027 sentence, an insulting sentence, a negative sentence about a different person, or a risky sentence that may indicate a dangerous situation for the special needs person, which requires immediate intervention. We used several machine learning methods, and we found that the most accurate methods were the random forest method with 100 estimators, a voting method using several classifiers, and a convolutional neural network (CNN) with embedding. All of these mechanisms reached an accuracy close to 70% in classifying the sentences in the test set. Finally, using an ensemble method comprising a panel of the 5 best CNN based methods, improves the accuracy of the results and the F1score. Our results demonstrate the feasibility of building an assisting agent that will accompany the special needs children and adults, and assist them in their daily social interactions.

Detecting Sensitive Content in Spoken Language Spoken language can include sensitive topics including profanity, insults, political and offensive speech. In order to engage in contextually appropriate conversations, it is essential for voice services such as Alexa, Google Assistant, Siri, etc. to detect sensitive topics in the conversations and react appropriately. A simple approach to detect sensitive topics is to use regular expression or keyword based rules. However, keyword based rules have several drawbacks: (1) coverage (recall) depends on the exhaustiveness of the keywords, and (2) rules do not scale and generalize well even for minor variations of the keywords. Machine learning (ML) approaches provide the potential benefit of generalization, but require large volumes of training data, which is difficult to obtain for sparse data problems. This paper describes: (1) a ML based solution that uses training data (2.1M dataset), obtained from synthetic generation and semisupervised learning techniques, to detect sensitive content in spoken language; and (2) the results of evaluating its performance on several million test instances of live utterances. The results show that our ML models have very high precision (u003eu003e90%). Moreover, in spite of relying on synthetic training data, the ML models are able to generalize beyond the training data to identify significantly higher amounts (2x for Logistic Regression, and 4x6x for a Neural Network models such as BiLSTM and CNN) of the test stream as sensitive in comparison to a baseline approach using the training data ( 1 Million examples) as rules. We are able to train our models with very few manual annotations. The percentage share of sensitive examples in our training dataset from synthetic generation using templates and manual annotations are 98.04% and 1.96%, respectively. The percentage share of nonsensitive examples in our training dataset from synthetic generation using templates, automated labeling via semisupervised techniques, and manual annotations are 15.35%, 83.75%, and 0.90%, respectively. The neural network models (BiLSTM and CNN) also use lower memory footprint (22.5% lower than baseline and 80% lower than Logistic Regression) while giving improved accuracy.