Reviewerly/exHarmony huggingface dataset:
Exploiting the Logits: Joint Sign Language Recognition and Spell-Correction

Machine learning techniques have excelled in the automatic semantic analysis of images, reaching human-level performances on challenging benchmarks. Yet, the semantic analysis of videos remains challenging due to the significantly higher dimensionality of the input data, respectively, the significantly higher need for annotated training examples. By studying the automatic recognition of German sign language videos, we demonstrate that on the relatively scarce training data of 2.800 videos, modern deep learning architectures for video analysis (such as ResNeXt) along with transfer learning on large gesture recognition tasks, can achieve about 75% character accuracy. Considering that this leaves us with a probability of under 25% that a 5 letter word is spelled correctly, spell-correction systems are crucial for producing readable outputs. The contribution of this paper is to propose a convolutional neural network for spell-correction that expects the softmax outputs of the character recognition network (instead of a misspelled word) as an input. We demonstrate that purely learning on softmax inputs in combination with scarce training data yields overfitting as the network learns the inputs by heart. In contrast, training the network on several variants of the logits of the classification output i.e. scaling by a constant factor, adding of random noise, mixing of softmax and hardmax inputs or purely training on hardmax inputs, leads to better generalization while benefitting from the significant information hidden in these outputs (that have 98% top-5 accuracy), yielding a readable text despite the comparably low character accuracy.
The Surprising Effectiveness of Linear Unsupervised Image-to-Image Translation

Unsupervised image-to-image translation is an inherently ill-posed problem. Recent methods based on deep encoder-decoder architectures have shown impressive results, but we show that they only succeed due to a strong locality bias, and they fail to learn very simple nonlocal transformations (e.g. mapping upside down faces to upright faces). When the locality bias is removed, the methods are too powerful and may fail to learn simple local transformations. In this paper we introduce linear encoder-decoder architectures for unsupervised image to image translation. We show that learning is much easier and faster with these architectures and yet the results are surprisingly effective. In particular, we show a number of local problems for which the results of the linear methods are comparable to those of state-of-the-art architectures but with a fraction of the training time, and a number of nonlocal problems for which the state-of-the-art fails while linear methods succeed.
RBNN: Memory-Efficient Reconfigurable Deep Binary Neural Network With IP Protection for Internet of Things

Though deep neural network models exhibit outstanding performance for various applications, their large model size and extensive floating-point operations render deployment on mobile computing platforms a major challenge, and, in particular, on Internet of Things devices. One appealing solution is model quantization that reduces the model size and uses integer operations commonly supported by microcontrollers . To this end, a 1-bit quantized DNN model or deep binary neural network maximizes the memory efficiency, where each parameter in a BNN model has only 1-bit. In this paper, we propose a reconfigurable BNN (RBNN) to further amplify the memory efficiency for resource-constrained IoT devices. Generally, the RBNN can be reconfigured on demand to achieve any one of M (M>1) distinct tasks with the same parameter set, thus only a single task determines the memory requirements. In other words, the memory utilization is improved by times M. Our extensive experiments corroborate that up to seven commonly used tasks can co-exist (the value of M can be larger). These tasks with a varying number of classes have no or negligible accuracy drop-off on three binarized popular DNN architectures including VGG, ResNet, and ReActNet. The tasks span across different domains, e.g., computer vision and audio domains validated herein, with the prerequisite that the model architecture can serve those cross-domain tasks. To protect the intellectual property of an RBNN model, the reconfiguration can be controlled by both a user key and a device-unique root key generated by the intrinsic hardware fingerprint. By doing so, an RBNN model can only be used per paid user per authorized device, thus benefiting both the user and the model provider.
A Comparative Study of Learning Outcomes for Online Learning Platforms

Personalization and active learning help educational systems to close the gap between students with varying abilities. We run a comparative head-to-head study of learning outcomes for two popular online platforms: Platform A, which delivers content over lecture videos and multiple-choice quizzes, and Platform B, which provides interactive problem-solving exercises and personalized feedback. We observe a statistically significant increase in the learning outcomes on Platform B. Further, the results of the self-assessment questionnaire suggest that participants using Platform B improve their metacognition.
An Online Learning Algorithm for Non-stationary Imbalanced Data by Extra-Charging Minority Class

Online learning is one of the trending areas of machine learning in recent years. How to update the model based on new data is the core question in developing an online classifier. When new data arrives, the classifier should keep its model up-to-date by (1) learn new knowledge, (2) keep relevant learned knowledge, and (3) forget obsolete knowledge. This problem becomes more challenging in imbalanced non-stationary scenarios. Previous approaches save arriving instances, then utilize up/down sampling techniques to balance preserved samples and update their models. However, this strategy comes with two drawbacks: first, a delay in updating the models, and second, the up/down sampling causes information loss for the majority classes and introduces noise for the minority classes. To address these drawbacks, we propose the Hyper-Ellipses-Extra-Margin model (HEEM), which properly addresses the class imbalance challenge in online learning by reacting to every new instance as it arrives. HEEM keeps an ensemble of hyper-extended-ellipses for the minority class. Misclassified instances of the majority class are then used to shrink the ellipse, and correctly predicted instances of the minority class are used to enlarge the ellipse. Experimental results show that HEEM mitigates the class imbalance problem and outperforms the state-of-the-art methods.
Feature-Dependent Cross-Connections in Multi-Path Neural Networks

Learning a particular task from a dataset, samples in which originate from diverse contexts, is challenging, and usually addressed by deepening or widening standard neural networks. As opposed to conventional network widening, multipath architectures restrict the quadratic increment of complexity to a linear scale. However, existing multi-column/path networks or model ensembling methods do not consider any feature-dependent allocation of parallel resources, and therefore, tend to learn redundant features. Given a layer in a multi-path network, if we restrict each path to learn a context-specific set of features and introduce a mechanism to intelligently allocate incoming feature maps to such paths, each path can specialize in a certain context, reducing the redundancy and improving the quality of extracted features. This eventually leads to better-optimized usage of parallel resources. To do this, we propose inserting feature-dependent cross-connections between parallel sets of feature maps in successive layers. The weighting coefficients of these cross-connections are computed from the input features of the particular layer. Our multi-path networks show improved image recognition accuracy at a similar complexity compared to conventional and state-of-the-art methods for deepening, widening and adaptive feature extracting, in both small and large scale datasets.
Virtual Reality Game for Visual Dimension Understanding

Engineering drawing is one of the important fundamental subjects in the engineering course that requires imagination and artistic skill for understanding. Engineering student needs to practice a lot of exercises to understand three-dimensions representation and its two dimensional orthographic in top view, front view and side view. However, some students cannot understand 3D objects by just only paper and pencil. This process makes them feel bored to practice. Therefore, they cannot understand well and might have a long-term issue. Therefore, this paper proposes virtual reality 3D games as a learning media by using Oculus Rift and Oculus Touch. The objective of this game is to help students or people who have a previously mentioned problem to improve their skills. This game will increase the enjoyment of learning and possibly motivate them to educate the understanding of 3D objects and their visual dimensions. In this game, the player must match 3D objects with orthographic projections and create the object by themselves to match an orthographic projection. A player can register and check their improvement from history score and time spent. The goal of this virtual reality learning game is to motivate the player to practice with happiness. The experimental results showed players used shorter time and had got higher accuracy when continue playing. Post-questionnaire results shown they feel most enjoying when creating 3D virtual reality objects by themselves. Moreover, the results could imply students below grade C+ in Engineering Drawing subject may have problems with 3D objects that were cut into triangular in prismatic shapes, and students below grade D+ may have problems with dashed and solid lines.
Particle Gibbs Sampling for Regime-Switching State-Space Models

Regime-switching state-space models (RS-SSMs) are an important class of statistical models that can be used to represent real-world phenomena. Unlike regular state-space models, RS-SSMs allow for dynamic uncertainty in the state transition and observations distributions, making them much more expressive. Unfortunately, there are no existing Bayesian inference techniques for joint estimation of regimes, states, and model parameters in generic RS-SSMs. In this work, we develop a particle Gibbs sampling algorithm for Bayesian learning in RS-SSMs. We demonstrate the proposed inference approach on a synthetic data experiment related to an ecological application, where the goal is in estimating the abundance and demographic rates of penguins in the Antarctic.
OntoSpect: IoT Ontology Inspection by Concept Extraction and Natural Language Generation

One of the main challenges in the Internet of Things (IoT) is the lack of semantic interoperability between heterogeneous sources. In the Semantic Web domain, ontologies are one way to achieve semantic interoperability by using a common vocabulary that represents heterogeneous sources. However, recent studies have shown that the amount of concept reuse from existing IoT ontologies is low. As the number of IoT ontologies increases, encouraging users to reuse existing ontologies instead of creating new concepts becomes important. Ontology catalogues are a prominent approach to discover and inspect existing ontologies for reuse. However, such catalogues inspect the ontologies using general criteria which is not enough to understand the content of the ontology. In this paper, we propose a method for automatic ontology inspection (OntoSpect) of IoT ontologies from different application domains based on a generic set of content-related concepts. OntoSpect consists of two main steps: first it extracts the set of IoT concepts, and then generates human-understandable descriptions using a Model-driven Engineering (MDE) approach. We evaluate the quality of concept extraction and natural language description generation with 84 ontologies retrieved from the LOV4IoT catalogue and report on quality metrics. In addition, we conduct an empirical study with 28 ontology users to further assess the quality of the generated descriptions. The results demonstrate the capability of OntoSpect to support ontology users inspecting IoT ontologies.
Efficient Shadow Detection and Removal using Synthetic Data with Domain Adaptation

In recent years, learning based shadow detection and removal approaches have shown prospects and, in most cases, yielded state-of-the-art results. The performance of these approaches, however, relies heavily on the construction of training database of shadow images, shadow-free versions, and shadow maps as the ground truth. This conventional data gathering is time-consuming, expensive, or even practically intractable to realize especially for outdoor scenes with complicated shadow patterns, thus limiting the size of the data available for training. In this paper, we leverage on large high quality synthetic image database and domain adaptation to mitigate the bottlenecks resulting from insufficient training samples and domain bias. Specifically, our approach utilizes adversarial training to predict near-pixel-perfect shadow map from synthetic shadow image for downstreaming shadow removal steps. At inference time, we capitalize on domain adaptation via image style transfer to map the style of real-world scene to that of synthetic scene for the purpose of detecting and subsequently removing shadow. Comprehensive experiments indicate that our approach outperforms state-of-the-art methods on selected benchmark datasets.
Document Clustering on COVID literature using Machine Learning

Coronavirus, which is recently declared as a pandemic outbreak is initially started in Wuhan, China in the month of December, where it has extended towards every edge of the globe. Many individuals across the globe are affected by the virus, and countless others have passed away. There are about 116 million cases, and the population is yet to be treated. The study and progress in the research of coronavirus have reduced researchers' ability to get relevant information from the internet. As mutations of infections and microbes are inescapable, the danger for such pandemics still remains high. To guarantee that, the harm is done is lesser, where a literature clustering technique can be used to group the relevant information together by making accessibility easier for researchers and enthusiasts. Furthermore, k-means clustering is used for labeling the clusters and principal component analysis for dimensionality reduction, t-SNE for better visualization, and a scatter plot may be used to represent the grouped collection of literature. On this bokeh plot, articles that are similar share a tag, whereas articles that are not similar are placed so far apart. Latent Dirichlet Allocation will be used to extract keywords from each cluster in order to find credibility in the groups.
Mental Map-Preserving Visualization through a Genetic Algorithm

The video game industry has evolved significantly, with different genres becoming popular over time, but how to visualize such information by curating data into a form that makes it easier to identify and understand the trends is quite an interesting research topic. This research focuses on producing an animation of aesthetically pleasing two-dimensional (2D) undirected graphs based on PC video game datasets. The data are further analyzed for developing a web-based application giving users the ability to control and create the animation of a graph. To make it easier to understand the animation of a graph, the changes between the displays of the previous and the following periods are set as small as possible, allowing a user to grasp the differences of the graph’s structure faster. A genetic algorithm-based undirected graph drawing that minimizes both the aesthetic criteria and mental map cost are proposed in this research to tackle this problem. Furthermore, based on our experiments, we could find the best period to start with, so we do not necessarily need to start from the first period to calculate the animation result. Our experiment results proved that a smoother animation could be achieved, and information is better preserved throughout the animation.
A Preliminary Investigation of Avatar Use in Video-Conferencing

Avatar use on video-conference platforms has found dual purpose in recent times as a potential method for ensuring privacy and improving subjective engagement with remote meeting, provided one can also ensure a minimal loss in the quality of social interaction and sense of personal presence. This preliminary study focuses on interaction through virtual avatars in a video conferencing context.
Customer Review Analysis by Hybrid Unsupervised Learning Applying Weight on Priority Data

This paper describes a method which deals with the assurance of E-commerce product quality by analyzing the customer reviews using unsupervised machine learning algorithms. The increasing E-commerce business involves people more engaging in online shopping though the assurance of the product quality is a big concern. For quality products, people now depend on the other customers' reviews such as comments, emotions, hashtags, etc. The purpose of this study is to automatically extract the polarity of reviews in Bangla as positive or negative sentiment conducted by customers to unleash online product quality. Here, we have tried to implement several unsupervised machine learning techniques for clustering the customers' reviews such as K-Means, Density-Based Spatial (DBSCAN), Mean Shift, Agglomerative, ensemble approach combined with different clustering, etc. It is observed from the experiments that the proposed method provides 98% accuracy for the experimental corpus by employing PWWA (Priority Word Weight Assignment) in data processing stage and an ensemble clustering algorithm, K-Means and Agglomerative Hierarchical approach, in post-processing stage.
Pose Recognition Based Wheelchair Propulsion for Elderly Convenience

aging is an inevitable, natural process affecting both inanimate and animate objects. While inanimate objects crumble, the bodies of animate beings struggle to handle the daily routine after reaching a certain age. Many tasks become too difficult to handle and the body barely hangs on as it ages. One such task is the mobility for the geriatric society. The purpose of this research is to improve the quality of life for the senior citizens. The research focuses on the design of such a robotic wheelchair which incorporates everything from hardware, software to sensor technology, computer processing and power distribution. When the senior citizen wishes to sit, they will perform a set of specified Poses that elicits a response from the wheelchair in the form of forward movement making it easy for them to be seated. During the experimentation phase, the wheelchair is triggered and reacts appropriately as and when the Poses are performed. Results from the performance testing demonstrate its functionality as a possible care-giving solution to the senior citizens.
Lux

Smart speakers are increasingly appearing in homes, enterprises, and businesses including hotels. These systems serve as hubs for other IoT devices and deliver content from streaming media services. However, such an arrangement creates a number of security concerns. For instance, providing such devices with long-term secrets is problematic with regards to vulnerable devices and fails to capture the increasingly transient nature of the relationship between users and the devices (e.g., in hotel or airbnb settings, this device is not owned by the customer and may only be used for a single day). Moreover, the limited interfaces available to such speakers make entering such credentials in a safe manner difficult. We address these problems with Lux, a system to provide ephemeral, fine-grained authorization to smart speakers which can be automatically revoked when the user and hub are no longer in the same location. We develop protocols using the LED/light channel available to many smart speaker devices to help users properly identify the device with which they are communicating, and demonstrate through a formally validated protocol that such authorization takes only a few seconds in practice. Through this effort, we demonstrate that Lux can safely authorize devices to access user accounts while limiting any long-term exposure to compromise.
Application of Smart Phone for Industrial Barcode Scanner

The barcode system is used in various purpose. In industrial sector, barcode is used for identifying product in the manufacturing process. However, the barcode systems in industrial factory requires special barcode reader and the industrial barcode reader is more complicated and expensive than the conventional barcode reader.Recently, all smart phone can be applied for barcode reader application but there is no study how to use the smart phone barcode reader application in industrial factory. Therefore, this study would consider how to use the smart phone for industrial barcode reader system. The Android smart phone was used to develop barcode reader application by cooperating with the barcode reader library and the barcode reader system in manufacturing process was simulated. The barcodes for this experiment were ID barcode and QR code. Moreover, the damaged barcodes were tested in the experiments and each experimental was continue tested nonstop for 24 hours.The result found that smart phone can read ID barcode and QR code. The readable rates of ID barcode (complete and damage barcode) were 100%. The readable rates of QR code at 30 and 50 photos per minute were 92.5% and 85%. Thus, the smart phone has an opportunity to use as ID barcode reader in industrial factory. For QR code, the readable rate result is low. However, there are many advantages to use smart phone for barcode reader such as price, flexibility, easy to use and reinstall. So, it is a challenge to use smart phone for industrial barcode reader.
Human Perceptuality-Aware Tone-Mapping-Based Dynamic Voltage Scaling for an AMOLED Display Smartphone

In recent years, people have wanted to watch high dynamic range imagery which can give high human visual satisfaction on smartphones and demand longer smartphone battery time. However, compression of dynamic range using tone-mapping operators is required in smartphones because most smartphone displays currently have a low dynamic range, and this causes loss of local contrast and details to compress dynamic range. Thus, in this paper we propose a novel dynamic voltage scaling scheme tightly coupled with a modified tone-mapping operator to achieve high power saving as well as good human perceptuality on an AMOLED display smartphone. In order to perform a human perceptuality-aware voltage control, we control display panel voltage to save power consumption and use a well-adjusted global tone-mapping operator to convert image brightness and unsharp masking to enhance local contrast and details and control. We implement the proposed scheme on the AMOLED display Android smartphone and experiment with various high dynamic range image databases. Experimental results show that not only tone-mapped images but also general images are improved in terms of human visual satisfaction and power saving, compared to conventional techniques.