Reviewerly/exHarmony huggingface dataset:
Multiple Machine Learning Algorithms for Human Smoking Behavior Detection

Due to the harm and accidents caused by smoking behavior, people have to spend a great number of efforts to detect those behavior in many public places by people's eyes. This way of detection is sometimes inaccurate and exhausting. In this paper, three models were proposed to find smoking behavior automatically. The dataset in this paper has three classes: smoking, calling and normal (neither smoking nor calling). Smoking behavior detection is different from cigarette detection or smog detection since it needs us to find a smoker first. Therefore face detection was used to determine if there is a person in the picture. Then a screenshot will be made on the picture, and this screenshot is supposed to include the person's face and the area around the face. After that this screenshot will be turned to grayscale and be resized to 64× 64. Data augmentation is used as well to make proposed model more robust. There are 3 different models used in this study: support vector machine (SVM), Random Forest and convolutional neural network (CNN). Compared to SVM and Random Forest, in CNN model the study got the best performance with an accuracy of 94.59% in testing dataset which consists of 522 images. Experiments show the high efficiency of our method, and smoking behavior can be detected accurately.
Reconstruction-based Multi-Scale Anomaly Detection for Cyber-Physical Systems

This paper considers anomaly detection for cyber-physical systems, in which the multivariate time series data collected from different sensors have complex temporal dependencies and inter-sensor correlations. We firstly propose an improved unsupervised anomaly detection framework which extracts the temporal and spatial patterns based on the autoencoder and the attention-based convolutional long-short term memory networks. In particular, the original data are fused into the input signature matrices to avoid information loss and an improved sample-based threshold setting approach is proposed to estimate the optimal threshold automatically. Finally, the experiments on two sensor datasets illustrate that our model achieves superior performance over state-of-the-art methods.
Unsupervised Disentangled Graph Learning for OCM

In Chapter 2 and Chapter 3, we proposed two methods for outfit compatibility modeling. They still suffer from two key limitations. (1) They evaluate the outfit compatibility based on the single latent compatibility space. The outfit compatibility is essentially affected by multiple complementary hidden factors, such as the color, style, shape, and material. Therefore, we argue that previous methods can only achieve the suboptimal solution, as it entangles all the factors in a single latent space. (2) They focus on learning the representation of each composing item and based on that, calculate the outfit compatibility. We argue that this method still fails to authentically treat the outfit as a whole, namely, it overlooks the global outfit representation learning. Therefore, in this chapter, we aim to estimate the compatibility of the outfit by considering the multiple hidden spaces and the global outfit graph representation learning.
Influence of Degradation on Modified Full Reference Image Quality Metrics

This paper presents the influence of six different types of image degradation on the modified objective full-reference PSNR quality metrics. The types of degradation used are different levels of Contrast, Median filter, JPEG compression, Gaussian blur, Motion blur as well as white Gaussian noise with zero standard deviation and variable mean. The modified PSNR quality metric uses MAE, Euclidean distance and Canberra distance as error estimators instead of MSE. The results show that for all types of degradation the quality metric with the lowest deviation is PSNR modified with Euclidean distance. The Gaussian noise with mean shift is the most appropriate degradation to be estimated by all modifications, whereas median filter and contrast produce the worst results. The results by PSNR with Euclidean distance can be used with standard uncertainty when the quality is degraded by Gaussian blur, Motion blur, Median and positive mean due to normality of the data.
Augmenting Conversations With Comic-Style Word Balloons

We propose a novel approach for enabling comic-style conversation in mixed reality to assist face-to-face conversation on-site or remotely. Our approach brings word balloons of comic-style conversation to the real world. The word balloons can adapt to mixed reality scenes, such as the 3-D head motion of the speaker, the comic styles, and the speech. During the conversation, our approach updates the word balloons continuously in the object space and discretely in the image space, guided by a field learned from comics. Quantitative experiments and perceptual studies were conducted to evaluate and compare our approach with alternatives. The results from the user study and ablation study demonstrated that our approach turns out to be practical for assisting face-to-face conversation.
Preventing Adversarial Attacks Against Deep Learning-Based Intrusion Detection System

Deep learning (DL) applications in network intrusion detection systems (NIDS) are increasingly popular in protecting IoT networks against cyber threats. However, these systems are threatened by adversarial attacks that can evade detection and disrupt the network. Preventing such attacks is highly challenging due to their variation and the resource-constrained nature of IoT devices. Therefore, in this paper, we first evaluate the impact of adversarial attacks on a novel DL-based NIDS designed for IoT networks. Then, we propose an adversarial detector powered by a light gradient boosted algorithm against adversarial attacks. The superiority of our proposal is to detect several types of adversarial attacks with high accuracy while ensuring low additional latency. The evaluation results on practical datasets show that our model effectively detects adversarial attacks, with an overall F-score of 99.66% and much higher than that of its competitors.
Look After: A Context-Aware Mobile Computing Based Smart Agent to Manage Personal Events

Look-After is a mobile-based smart assistant which ensures effectiveness and simplicity to use. It enables the user to keep track of all of his tasks and events in a personalized manner. This virtual agent learns about users' context based on the information acquired from their email and contacts in terms of their proper authorization and permission. By utilizing these learnings, it automatically detects any future events, schedules, or appointments the user has and dynamically generates a task in the application to act as an online guide and notify the user on time. Along with other features, this application can serve as a one-stop service for all its user schedule management. With progressively being shared our communication and information online, this type of smart agent which can dynamically understand its user's schedule would be a great help and streamlined. Due to the simple UI/UX and requirements, any user or even organization can easily adapt to the system. When it gets merely the read-only rights of a user's email, the smart app can work on detecting keywords with the help of our pre-defined Regular Expression (RegEx) to identify any upcoming events/tasks. This also leaves room for the use of Machine Learning to generate more precise keywords, that can better identify possible schedules of the user. Our work suggests a context-aware mobile computing-based environment, where the user will be made aware of his schedules beforehand, and be part of a social environment, where activities can be shared and discussed with others according to convenience.
Grokking-like effects in counterfactual inference

We show that a typical neural network, which ignores any covariate/feature re-balancing, can be as effective as any explicit counterfactual method. We adopt the architecture of TARNet—a simple neural network with two heads (one for treatment, one for control) which is trained with a relatively high batch size. Combined with ensemble methods, this produces competitive results in four counterfactual inference benchmarks: IHDP, NEWS, JOBS, and TWINS. Our results indicate that relatively simple methods might be good enough for counterfactual prediction, with quality constraints coming from hyperparameter tuning. Our analysis indicates that the reason behind the observed phenomenon might be “grokking”, a recently developed theory.
Internet of Things Applications in Healthcare During COVID-19 Pandemic: Perspectives

The Internet of Things (IoT) can offer extraordinary innovations in healthcare. During the COVID-19 pandemic, the need to study the different applications of IoT-enabled healthcare has emerged. This paper focuses on extrapolating people’s opinions about the use of IoT technologies in healthcare. A questionnaire was used to obtain and analyze the results of a sample from Benghazi Medical Hospital in Libya. The questionnaire was distributed to two groups of patients, their companions, and medical staff in the hospital, where the data obtained were analyzed using the SPSS software. The results indicate that the patients and their companions see that there are advantages to using the IoT in the health sector because it reduces the difficulty of booking and overcrowding, although there are disadvantages, the most important of which is the lack of knowledge of the elderly in these technologies. Medical staff also believe that the use of the IoT has several advantages, including limiting the spread of infectious diseases, although hospitals do not provide the applications, equipment, and maintenance that this technology needs. Based on the results obtained, the study came up with some important recommendations.
When Few-Shot Learning Meets Video Object Detection

Different from static images, videos contain additional temporal and spatial information for better object detection. However, it is costly to obtain a large number of videos with bounding box annotations that are required for supervised deep learning. Although humans can easily learn to recognize new objects by watching only a few video clips, deep learning usually suffers from overfitting. This leads to an important question: how to effectively learn a video object detector from only a few labeled video clips? In this paper, we study the new problem of few-shot learning for video object detection. We first define the few-shot setting and create a new benchmark dataset for few-shot video object detection derived from the widely used ImageNet VID dataset. We employ a transfer-learning framework to effectively train the video object detector on a large number of base-class objects and a few video clips of novel-class objects. By analyzing the results of two methods under this framework (Joint and Freeze) on our designed weak and strong base datasets, we reveal insufficiency and overfitting problems. A simple but effective method, called Thaw, is naturally developed to trade off the two problems and validate our analysis. Extensive experiments on our proposed benchmark datasets with different scenarios demonstrate the effectiveness of our novel analysis in this new few-shot video object detection problem.
Attention Economy in Metaverse: An NFT Value Perspective

With the blooming of NFT platforms such as Opensea, metaverse has become the most booming concept in both industry and academia all around the universe. Compre-hending the way metaverse economy operates and the intrinsic value propping it has become the key to the essence of metaverse. We try to reveal the value basis of metaverse via two academic resources, media theory and attention economics, to describe the principle of value creation in metaverse economy. To better examine how the value of NFT projects is affected by human time and attention devoted to metaverse, we study a system with structured rules, e.g. blockchain games, to measure the players' time spent and quantify the players' rewards. A representative project, Aavegotchi is examined as a case. We analyse the data of Aavegotchi pixel ghosts such as price, rarity and player's time devoted. We find various interesting results which demonstrate how the value of NFTs is affected by various factors, and acquire some meaningful conclusions on the value of time in metaverse.
Quantitative BAN Logic Based on Belief Degree

Authentication protocols are the basis for secure communication in many distributed systems but are highly prone to errors in their design, preventing them from working properly. It is therefore necessary to analyze an authentication protocol to determine whether the designed protocol meets the requirements. Much attention has been paid to mathematical logic to analyze cryptographic protocols, particularly the logic proposed by Burrows, Abadi, and Needham (BAN logic). This logic has been successful in identifying weaknesses in various examples of authentication protocols. In this paper, we give a concept of “belief” for BAN logic based on the idea of possibility computation and further propose a quantitative BAN logic. It is also applied to the formal analysis and computation of a Radio Frequency Identification (RFID) authentication protocol to show how it works. The quantitative results on belief show that the proposed quantitative approach of BAN logic based on belief can more objectively reflect the security property of the authentication protocol.
Tripod: Harmonizing Job Scheduling and Data Caching for Analytics Frameworks

Modern data analytics platforms are often coupled with external data storage services such as Amazon S3, resulting in storage bottlenecks. Existing caching and prefetching solutions use higher-level information from data analytics frameworks, such as job dependency graphs(e.g., DAGs) and historical run time information, to predict future data accesses and then prefetch data into the cache and manage the cache contents based on those predictions.However, in doing so, they are not taking advantage of a fundamental opportunity: rather than caching data given a prediction of job execution, we can actually influence the job execution order to enable more effective caching and prefetching. With this key insight, we devise a set of novel heuristics and then design a system Tripod, which harmonizes job scheduling and data caching for analytics frameworks. With the higher-level information from analytics frameworks, Tripod explores a best-suited job execution order for prefetching and caching guided by the devised heuristics.We have implemented Tripod as extensions to Apache YARN and Tez. Our evaluation using standard analytic benchmarks (TPC-H and TPC-DS) shows that Tripod achieves up to 1.7x speedup over state-of-the-art approaches.
A Collaborative Augmented Reality Platform for Interactive and Immersive Education

Augmented reality (AR) integrates virtual objects with the physical world. This technology allows us to create immersive experiences that can enhance our lives, especially through education. In combination with networking, we are able to create collaborative AR experiences which can be used to enhance the learning experience for students or people of any age. The goal of this demonstration is to show one of these systems in action. The purpose is to show students how we are able to calculate the Astronomical Unit (AU) by using the position of Venus relative to Earth and the Sun. We use AR Foundation to realize basic AR functions and rendering. The server for this multi-user demonstration is the teacher, where the teacher is able to change different parameters of the demo, which will appear on the user's device. This demonstration is made for mobile devices to reach the broadest audience. We measure the frame rates of multiple users and the round-trip time between the server and users. The frame rate is about 60 frames per second during the whole lifetime of the demo, providing a smooth and stable visual performance. The round-trip time increases from 13ms to 24ms as the number of connected users grows, which means the communication latency is less than 10ms most of the time.
A Study for Precise Financial Support Model for Undergraduates with Financial Difficulties Using Machine Learning

At present, the government pays more attention on student financial assistance in colleges and universities than before. In order to achieve advance prediction of the degree of difficulty of students and reduce the burden of relevant staff, we used the data of the school card consumption records and the data of subsidised students as the data set to achieve it. Then, we established the model by using machine learning algorithms. Compared the difficulty level of economically disadvantaged students with the real data of subsidised students, it was concluded that the accuracy of the optimal logistic regression model reached at 100% and the model ran faster compared with other models at around 0.5s. The model can be better applied to student financial support work and improve the efficiency of working at the same time.
A Study on Experiencing Chinese Culture Through TikTok – Focusing on Cultural Exchange in the COVID-19 Era from the Perspective of a User with 500,000 Followers

Recently, unprecedented social (physical) distancing measures are being imposed around the world to curb the spread of COVID-19. Social media has become more important during the ongoing COVID-19 pandemic due to the increase of non-face-to-face activities. Considering that reactions to foreign culture are a part of cultural exchange, this study investigates why foreign reaction videos are popular from a cultural perspective and examines the positive effects of cultural exchange between Korea and China through TikTok video content. All of the most viewed and popular videos were about reactions to trending videos. This trend is presumed to be due to the curiosity and expectation of positive responses that videos popular in China will also be popular in Korea. Similar reactions can create cross-border consensus and relations, which increases familiarity. Many people were happy and proud to see positive comments about Chinese culture. Others wanted to learn the Korean language and said that their misunderstanding and prejudice against Koreans disappeared. Cultural exchanges through reaction videos show the positive effects of TikTok in increasing friendly emotions.
A survey on word representation in natural language processing

This survey explores several techniques and language models for the representation of raw text data. These models are considered the feature extraction phase in Natural Language Processing (NLP). The traditional approaches such as Bag-of-Words, N-Gram, and TF-IDF are simple to implement. Moreover, the recently proposed models in the literature such as Word2Vec, Glove, Fast text, ELMO, and BERT have a different effect on semantics and syntactic for the representation of words and sentences. Also, these models are included in word embedding terminology. Finally, our survey presents a comprehensive review of the state-of-the-art in the field of NLP and we believe this is of interest for the researchers in this field.
Basics of artificial neural networks

Artificial neural networks (ANNs) have been reported as useful predictive tools to model complex engineering systems. ANNs mimic the natural behavior of the human brain in handling different problems instead of solving intricate mathematical models. They are used as a black-box with excellent capabilities to learn the nonlinear relation between the inputs and outputs of a certain system. They have also enhanced the generalization capability to handle unseen data after the learning process. In this chapter, a review of the basics of ANNs is presented. In general, ANNs have received increased attention in recent years since they have been applied to numerous real-world applications. They have many advantages, such as simplicity and efficiency. In this chapter, the authors introduce the basic mathematical concepts of the multilayer perceptron, the wavelet neural network, radial basis function, and the Elman neural network.
Consumer Grievance Handler

Myriad of consumer complaints has subjected to the difficulty in classifying consumer's grievances. Grievances usually comprises of lengthy texts which takes lots of manpower and time. Complaints can be filed into wrong categories. Difficulty in going through every sole grievance and directing them to relevant departments is to be dealt. To solve these issues, we have an idea of using machine learning algorithms to learn and classify the complaints into their respective categories and perform sentimental analysis on the customer complaints to obtain the priority of each complaint. Python FLASK API is used to enable application interaction. The user should enter the consumer complaint in the application, and the sentimental analysis and categorization of consumer complaints is done and the accuracy of the complaint classified is displayed.
HF-Defend: Defending Against Adversarial Examples Based on Halftoning

How to deal with the adversarial examples attracts a lot of interest recently. In this paper, we propose HF-Defend: a novel method to defend against the adversarial examples based on halftoning. Unlike the existing schemes, HF-Defend thoroughly removes the adversarial perturbations by transforming a 8-bit grayscale image (or one of the RGB channels in a color image) into a 1-bit halftoned image. To maintain the image quality and content, we propose a reconstruction module for the recovery of both the main contents and fine details of the image. In particular, we newly design a nonlinear low-pass filter to extract the main contents, and a FilterNet to establish a high-pass filter for the reconstruction of fine details. Experimental results demonstrate the advantage of our HF-Defend over the existing schemes.
Affective Behavior Learning for Social Robot Haru with Implicit Evaluative Feedback

We propose a human-in-the-loop reinforcement learning mechanism to help robots learn emotional behavior. Unlike the previous methods of providing explicit feedback via pressing keyboard buttons or mouse clicks, we provide a more natural way for ordinary people to train social robots how to perform social tasks according to their preferences - facial expressions. The whole experiment is carried out on the desktop robot Haru, which is mainly used for the research of emotion and empathy participation. Our experimental results show that through learning from implicit feedback of facial features, Haru can quickly understand and dynamically adapt to individual preferences, and obtain a similar performance to learning from explicit feedback. In addition, we observe that the recognition error of human feedback will cause a “temporary regress” of the robot's learning performance, which is more obvious at the beginning of the training process. This phenomenon is shown to be correlated with the accuracy of recognizing negative implicit feedback.
The Phantom Menace: Unmasking Security Issues in Evolving Software

Software security concerns the creation of secure software starting from its initial development phases, i.e., software that can withstand malicious attacks. To this end, several automated and not-automated solutions have been developed that support developers in identifying and assessing security issues, e.g., software vulnerabilities. However, most solutions were not meant to cooperate synergically or continuously run in the context of evolving software, i.e., software subject to frequent maintenance and evolution activities. In this scenario, developers have trouble setting up an effective defensive line against security issues arising in their projects. This research fills this gap by investigating how vulnerabilities affect evolving software projects and by proposing novel solutions to improve and simplify the security verification and validation process. The paper concludes by presenting the open challenges in the field of software security we framed while conducting our research.