Reviewerly/exHarmony huggingface dataset:
Real-time facial expression recognition using CNN

Enhancing modern day machines or computers to recognize various facial expressions and to understand human emotions from them in real time is an exigent research subject. Through this paper, I put forward a solution to recognize emotions by understanding different facial expressions by collecting live video through a Flask App created. I deploy a Flask App to video stream live feed captured through the local camera attached to the machine or computer system. The video captured is fed to various image extraction techniques. The facial features are identified by different operations provided by OpenCV and the region consisting of parts of the face are made to surround or enclose by a contour. This region, enclosed by the contour is used as an input to Convolutional Neural Network (CNN). The CNN model created consists of six activation layers, of which four are convolution layers and two are fully controlled layers. Each layer is designed to undergo several training techniques. The main objective of this project is to demonstrate the accuracy of Convolutional Neural Network model designed. The paper is concluded by discussing the outcomes of our project and the ways to improve the efficiency of the model. The scope of this project is also analyzed to enhance technologies developed in the near future.
Mobile Centernet for Embedded Deep Learning Object Detection

Object detection is a fundamental task in computer vision with wide application prospect. And recent years, many novel methods are proposed to tackle this task. However, most algorithms suffer from high computation cost and long inference time, which makes them impossible to be deployed on embedded devices in real industrial application scenarios. In this paper, we propose the Mobile CenterNet to solve this problem. Our method is based on CenterNet but with some key improvements. To enhance detection performance, we adopt HRNet as a powerful backbone and introduce a categorybalanced focal loss to deal with category imbalance problem. Moreover, to compress the model size as well as reduce inference time, knowledge distillation is employed to transfer knowledge from cumbersome model to a compact one. We conduct experiments on a large traffic detection dataset BDD100K and validate the effectiveness of all the modifications. Finally, our method achieves the 1st place in the Embedded Deep Learning Object Detection Model Compression Competition held in ICME 2020.
Enriched In-Order Linearization for Faster Sequence-to-Sequence Constituent Parsing

Sequence-to-sequence constituent parsing requires a linearization to represent trees as sequences.Top-down tree linearizations, which can be based on brackets or shift-reduce actions, have achieved the best accuracy to date.In this paper, we show that these results can be improved by using an inorder linearization instead.Based on this observation, we implement an enriched inorder shift-reduce linearization inspired by Vinyals et al. ( 2015)'s approach, achieving the best accuracy to date on the English PTB dataset among fully-supervised single-model sequence-to-sequence constituent parsers.Finally, we apply deterministic attention mechanisms to match the speed of state-of-theart transition-based parsers, thus showing that sequence-to-sequence models can match them, not only in accuracy, but also in speed.
Co-Saliency Detection Using Collaborative Feature Extraction And High-To-Low Feature Integration

Co-saliency detection, as a developing research branch of saliency detection, devotes to identify the common salient objects in a group of related images. The major challenge of co-saliency detection is how to effectively represent features considering both intra-image and inter-image information. In this paper, we propose a co-saliency detection model using collaborative feature extraction and high-to-low feature integration. We first feed the target image and its co-images into the Individual Feature Extraction Module (IFEM) to produce multi-level individual features. Then, to capture the collaborative inter-image information, the Collaborative Feature Extraction Module (CFEM) is applied to all highest-level individual features, generating the collaborative feature. Finally, we build a High-to-low Feature Integration Module (HFIM), which integrates the collaborative feature and multi-level individual features of the target image, to enrich the collaborative feature with individual intra-image information. Extensive experiments on two public datasets demonstrate that the proposed model achieves the state-of-the-art performance.
Handling Black Swan Events in Deep Learning with Diversely Extrapolated Neural Networks

By virtue of their expressive power, neural networks (NNs) are well suited to fitting large, complex datasets, yet they are also known to produce similar predictions for points outside the training distribution. As such, they are, like humans, under the influence of the Black Swan theory: models tend to be extremely "surprised" by rare events, leading to potentially disastrous consequences, while justifying these same events in hindsight. To avoid this pitfall, we introduce DENN, an ensemble approach building a set of Diversely Extrapolated Neural Networks that fits the training data and is able to generalize more diversely when extrapolating to novel data points. This leads DENN to output highly uncertain predictions for unexpected inputs. We achieve this by adding a diversity term in the loss function used to train the model, computed at specific inputs. We first illustrate the usefulness of the method on a low-dimensional regression problem. Then, we show how the loss can be adapted to tackle anomaly detection during classification, as well as safe imitation learning problems.
Video Caption Based Searching Using End-to-End Dense Captioning and Sentence Embeddings

Traditionally, searching for videos on popular streaming sites like YouTube is performed by taking the keywords, titles, and descriptions that are already tagged along with the video into consideration. However, the video content is not utilized for searching of the user’s query because of the difficulty in encoding the events in a video and comparing them to the search query. One solution to tackle this problem is to encode the events in a video and then compare them to the query in the same space. A method of encoding meaning to a video could be video captioning. The captioned events in the video can be compared to the query of the user, and we can get the optimal search space for the videos. There have been many developments over the course of the past few years in modeling video-caption generators and sentence embeddings. In this paper, we exploit an end-to-end video captioning model and various sentence embedding techniques that collectively help in building the proposed video-searching method. The YouCook2 dataset was used for the experimentation. Seven sentence embedding techniques were used, out of which the Universal Sentence Encoder outperformed over all the other six, with a median percentile score of 99.51. Thus, this method of searching, when integrated with traditional methods, can help improve the quality of search results.
Hierarchically Robust Representation Learning

With the tremendous success of deep learning in visual tasks, the representations extracted from intermediate layers of learned models, that is, deep features, attract much attention of researchers. Previous empirical analysis shows that those features can contain appropriate semantic information. Therefore, with a model trained on a large-scale benchmark data set (e.g., ImageNet), the extracted features can work well on other tasks. In this work, we investigate this phenomenon and demonstrate that deep features can be suboptimal due to the fact that they are learned by minimizing the empirical risk. When the data distribution of the target task is different from that of the benchmark data set, the performance of deep features can degrade. Hence, we propose a hierarchically robust optimization method to learn more generic features. Considering the example-level and concept-level robustness simultaneously, we formulate the problem as a distributionally robust optimization problem with Wasserstein ambiguity set constraints, and an efficient algorithm with the conventional training pipeline is proposed. Experiments on benchmark data sets demonstrate the effectiveness of the robust deep representations.
Learning Optimal Tree Models under Beam Search

Retrieving relevant targets from an extremely large target set under computational limits is a common challenge for information retrieval and recommendation systems. Tree models, which formulate targets as leaves of a tree with trainable node-wise scorers, have attracted a lot of interests in tackling this challenge due to their logarithmic computational complexity in both training and testing. Tree-based deep models (TDMs) and probabilistic label trees (PLTs) are two representative kinds of them. Though achieving many practical successes, existing tree models suffer from the training-testing discrepancy, where the retrieval performance deterioration caused by beam search in testing is not considered in training. This leads to an intrinsic gap between the most relevant targets and those retrieved by beam search with even the optimally trained node-wise scorers. We take a first step towards understanding and analyzing this problem theoretically, and develop the concept of Bayes optimality under beam search and calibration under beam search as general analyzing tools for this purpose. Moreover, to eliminate the discrepancy, we propose a novel algorithm for learning optimal tree models under beam search. Experiments on both synthetic and real data verify the rationality of our theoretical analysis and demonstrate the superiority of our algorithm compared to state-of-the-art methods.
Empowering Active Learning to Jointly Optimize System and User Demands

Existing approaches to active learning maximize the system performance by sampling unlabeled instances for annotation that yield the most efficient training.However, when active learning is integrated with an end-user application, this can lead to frustration for participating users, as they spend time labeling instances that they would not otherwise be interested in reading.In this paper, we propose a new active learning approach that jointly optimizes the seemingly counteracting objectives of the active learning system (training efficiently) and the user (receiving useful instances).We study our approach in an educational application, which particularly benefits from this technique as the system needs to rapidly learn to predict the appropriateness of an exercise to a particular user, while the users should receive only exercises that match their skills.We evaluate multiple learning strategies and user types with data from real users and find that our joint approach better satisfies both objectives when alternative methods lead to many unsuitable exercises for end users. 1
Variance Reduction and Quasi-Newton for Particle-Based Variational Inference

Particle-based Variational Inference methods (ParVIs), like Stein Variational Gradient Descent, are nonparametric variational inference methods that optimize a set of particles to best approximate a target distribution. ParVIs have been proposed as efficient approximate inference algorithms and as potential alternatives to MCMC methods. However, to our knowledge, the quality of the posterior approximation of particles from ParVIs has not been examined before for large-scale Bayesian inference problems. We conduct this analysis and evaluate the sample quality of particles produced by ParVIs, and we find that existing ParVI approaches using stochastic gradients converge insufficiently fast under sample quality metrics. We propose a novel variance reduction and quasiNewton preconditioning framework for ParVIs, by leveraging the Riemannian structure of the Wasserstein space and advanced Riemannian optimization algorithms. Experimental results demonstrate the accelerated convergence of variance reduction and quasi-Newton methods for ParVIs for accurate posterior inference in large-scale and ill-conditioned problems.
Chatbot Design Method Using Hybrid Word Vector Expression Model Based on Real Telemarketing Data

In the development of commercial promotion, chatbot is known as one of significant skill by application of natural language processing (NLP). Conventional design methods are using bag-of-words model (BOW) alone based on Google database and other online corpus. For one thing, in the bag-of-words model, the vectors are Irrelevant to one another. Even though this method is friendly to discrete features, it is not conducive to the machine to understand continuous statements due to the loss of the connection between words in the encoded word vector. For other thing, existing methods are used to test in state-of-the-art online corpus but it is hard to apply in real applications such as telemarketing data. In this paper, we propose an improved chatbot design way using hybrid bag-of-words model and skip-gram model based on the real telemarketing data. Specifically, we first collect the real data in the telemarketing field and perform data cleaning and data classification on the constructed corpus. Second, the word representation is adopted hybrid bag-of-words model and skip-gram model. The skip-gram model maps synonyms in the vicinity of vector space. The correlation between words is expressed, so the amount of information contained in the word vector is increased, making up for the shortcomings caused by using bag-of-words model alone. Third, we use the term frequency-inverse document frequency (TF-IDF) weighting method to improve the weight of key words, then output the final word expression. At last, the answer is produced using hybrid retrieval model and generate model. The retrieval model can accurately answer questions in the field. The generate model can supplement the question of answering the open domain, in which the answer to the final reply is completed by long-short term memory (LSTM) training and prediction. Experimental results show which the hybrid word vector expression model can improve the accuracy of the response and the whole system can communicate with humans.
Biased Feature Learning for Occlusion Invariant Face Recognition

To address the challenges posed by unknown occlusions, we propose a Biased Feature Learning (BFL) framework for occlusion-invariant face recognition. We first construct an extended dataset using a multi-scale data augmentation method. For model training, we modify the label loss to adjust the impact of normal and occluded samples. Further, we propose a biased guidance strategy to manipulate the optimization of a network so that the feature embedding space is dominated by non-occluded faces. BFL not only enhances the robustness of a network to unknown occlusions but also maintains or even improves its performance for normal faces. Experimental results demonstrate its superiority as well as the generalization capability with different network architectures and loss functions.
Neural Topic Modeling with Continual Lifelong Learning

Lifelong learning has recently attracted attention in building machine learning systems that continually accumulate and transfer knowledge to help future learning. Unsupervised topic modeling has been popularly used to discover topics from document collections. However, the application of topic modeling is challenging due to data sparsity, e.g., in a small collection of (short) documents and thus, generate incoherent topics and sub-optimal document representations. To address the problem, we propose a lifelong learning framework for neural topic modeling that can continuously process streams of document collections, accumulate topics and guide future topic modeling tasks by knowledge transfer from several sources to better deal with the sparse data. In the lifelong process, we particularly investigate jointly: (1) sharing generative homologies (latent topics) over lifetime to transfer prior knowledge, and (2) minimizing catastrophic forgetting to retain the past learning via novel selective data augmentation, co-training and topic regularization approaches. Given a stream of document collections, we apply the proposed Lifelong Neural Topic Modeling (LNTM) framework in modeling three sparse document collections as future tasks and demonstrate improved performance quantified by perplexity, topic coherence and information retrieval task.
GURU MENGINOVASI BAHAN AJAR SEBAGAI LANGKAH UNTUK MENINGKATKAN HASIL BELAJAR SISWA

Teachers are professions that require special expertise in their main tasks such as educating, teaching, guiding, directing, training, assessing and evaluating students in early childhood education in formal, primary and secondary education. Teachers apart from the assignment also design learning to help students in the learning process, so teachers need special expertise and high ability to develop teaching materials that are appropriate to the needs of students with the diversity of students' competence. The approach used in this research is qualitative with descriptive research type. Data collection is done by observation techniques and strengthened by theoretical studies from experts. Based on the data carried out by finding the learning process conducted by the teacher in class with teaching materials that have been developed by the teacher before the learning process takes place. Based on observations that the teaching material developed by the teacher has been able to provide convenience in students understanding the subject matter that the teacher conveys and the teaching material that the teacher has developed makes students eager to learn and solve the problems and problems the teacher provides, and is no less important than the development of teaching materials the teacher has made the learning outcomes obtained by students become visible from the results of the midterm exams of 32 students there are only 4 students whose grades are below the minimum graduation standard.
Explainable Inference on Sequential Data via Memory-Tracking

In this paper we present a novel mechanism to get explanations that allow to better understand network predictions when dealing with sequential data. Specifically, we adopt memory-based networks — Differential Neural Computers — to exploit their capability of storing data in memory and reusing it for inference. By tracking both the memory access at prediction time, and the information stored by the network at each step of the input sequence, we can retrieve the most relevant input steps associated to each prediction. We validate our approach (1) on a modified T-maze, which is a non-Markovian discrete control task evaluating an algorithm’s ability to correlate events far apart in history, and (2) on the Story Cloze Test, which is a commonsense reasoning framework for evaluating story understanding that requires a system to choose the correct ending to a four-sentence story. Our results show that we are able to explain agent’s decisions in (1) and to reconstruct the most relevant sentences used by the network to select the story ending in (2). Additionally, we show not only that by removing those sentences the network prediction changes, but also that the same are sufficient to reproduce the inference.
Learning and Solving Regular Decision Processes

Regular Decision Processes (RDPs) are a recently introduced model that extends MDPs with non-Markovian dynamics and rewards. The non-Markovian behavior is restricted to depend on regular properties of the history. These can be specified using regular expressions or formulas in linear dynamic logic over finite traces. Fully specified RDPs can be solved by compiling them into an appropriate MDP. Learning RDPs from data is a challenging problem that has yet to be addressed, on which we focus in this paper. Our approach rests on a new representation for RDPs using Mealy Machines that emit a distribution and an expected reward for each state-action pair. Building on this representation, we combine automata learning techniques with history clustering to learn such a Mealy machine and solve it by adapting MCTS to it. We empirically evaluate this approach, demonstrating its feasibility.
Highway Transformer: Self-Gating Enhanced Self-Attentive Networks

Self-attention mechanisms have made striking state-of-the-art (SOTA) progress in various sequence learning tasks, standing on the multiheaded dot product attention by attending to all the global contexts at different locations.Through a pseudo information highway, we introduce a gated component self-dependency units (SDU) that incorporates LSTM-styled gating units to replenish internal semantic importance within the multi-dimensional latent space of individual representations.The subsidiary content-based SDU gates allow for the information flow of modulated latent embeddings through skipped connections, leading to a clear margin of convergence speed with gradient descent algorithms.We may unveil the role of gating mechanism to aid in the contextbased Transformer modules, with hypothesizing that SDU gates, especially on shallow layers, could push it faster to step towards suboptimal points during the optimization process.
IoT Blockchain for Smart Sensor

The Internet of Things (IoT) started as one of the main driving forces of Industry 4.0. Having had its initial phase with results clearly visible still today, we are now beginning to witness the maturity at which this technology can reach. This sophistication of IoT will be made possible with the help of other emerging technologies. Among these technologies is Blockchain. This technology is the ideal candidate to counter the flaws of IoT, among which there is security. This paper proposes a functional infrastructure model that employs IoT smart sensor nodes to monitor environmental parameters such as temperature and pressure. On top of this base architecture a Blockchain network is deployed to mitigate the native vulnerabilities of IoT and to ensure that data collected is decentralized, accessible, transparent, lightweight and scalable.
Towards Secure, Flexible and Efficient Role Based Hospital’s Cloud Management System: Case Study

INTRODUCTION: Many organizations of health care have recognized that quality of service can be improved by maintaining e-records of patient's reports, medical histories, surgeries recordings, etc. over the multimedia cloud servers.But, the data breach is always a matter of stake for an organization as well as for the patients.OBJECTIVES: This work has considered a qualitative scenario related to multimedia e-content management for a multiforte hospital's cloud server.METHODS: An End-to-End Encryption with the cryptographic algorithm is applied along with an access control framework for secure transmission and storage of e-records.RESULTS: The results are presented in the form of bar graphs for the time taken to perform encryption/decryption of various media files and combo graphs for different scenarios of access control.CONCLUSION: This can provide prevention from many attacks/threats, ensures authentication and privacy along with limiting the server usage and cost.