Reviewerly/exHarmony huggingface dataset:
CNN-Based Autonomous Traffic Detection on Unknown Home Security Cameras

As a popular IoT device, home security cameras (HSCs) generate large amounts of traffic, but the security mechanisms are weak, making them often exploited to launch DDoS attacks. Moreover, the lurking HSC device significantly threatens user privacy, so it is necessary to detect the HSC traffic. With the current service upgrade of HSC devices and the popularity of the open-source DIY community, the HSC types have become numerous. However, current traffic classification methods can only rely on complex feature extraction, expertise, and static datasets for identification, making it challenging to detect unknown devices in the open world. To solve this problem, we propose an autonomous update framework, called CATD-HSC, to constantly improve the classifier’s detection ability on unknown HSC devices. First, we use a CNN-based classifier model to train known HSC traffic. Then, we collect the classifier’s output as a new feature and use the threshold filtering approach to filter the unknown HSC traffic. Finally, we cluster the unknown HSC traffic by connecting the original feature to the generated feature. Moreover, we make the process autonomous to reduce the repeated trivial training. We evaluate the performance of CATD-HSC on both the UNSW open dataset and the real-world HSC traces. The updated model achieves an average accuracy rate of 99.3% and a recall rate of 99.4% for unknown HSC traffic in three interference environments. Our further evaluation shows that CATD-HSC can successfully implement the classifier’s autonomous update and dynamic classification on unknown HSC devices.
Privacy-Preserving and Verifiable Outsourcing Message Transmission and Authentication Protocol in IoT

With the popularity of Internet of Things (IoT) and 5G, privacy-preserving message transmission and authentication have become an indispensable part in the field of data collection and analysis. There exist many protocols based on the public key cryptosystem, which allow the users to utilize their own identity as the public key to carry out data encryption and digital signature, which is very suitable for applying in the IoT environment with a large number of terminal devices. However, these protocols usually involve some complex cryptographic operations, which hinder their application on the resource-constrained IoT devices. In this paper, we design a privacy-preserving and verifiable outsourcing message transmission and authentication protocol, which allows the resource-constrained users to delegate some complex operations to the two untrusted edge servers and reduce the computational burden on the users side. The designed protocol contains several secure and novel outsourcing algorithms for modular exponentiation, bilinear pairing and scalar multiplication. For the different operations in the different situations, we design several different blinding techniques and verification methods, which not only protect the users’ private information, but also ensure the users can verify the correctness of results. Finally, we carry out some experiments to show that our proposed protocol is efficient.
Privacy-Preserving Robust Federated Learning with Distributed Differential Privacy

Federated Learning (FL) has attracted significant interest, as it provides a distributed machine learning paradigm to share data resources during model training process. However, sharing the gradients or model weights uploaded by clients or the final model aggregated by the server can lead to privacy disclosures and executing correctness issues. Specifically, the original data can be easily inferred through analyzing the shared gradients, and malicious users can disrupt the model aggregation to result in a destruction of the model accuracy. To address these issues, we propose a novel FL scheme with providing both privacy protection and robust aggregation. By using the distributed differential privacy and range proof technologies, the proposed scheme resists semi-honest servers and malicious users, while protecting the global model and providing the high accuracy. Both privacy analysis and experiments are given to demonstrate the effectiveness of our scheme.
Exploring Transformers for Multi-Label Classification of Java Vulnerabilities

Deep learning (DL) techniques have demonstrated potential in reasoning complex patterns of vulnerable code from high-level abstractions. Recent advancements in the area, such as the introduction of transformer-based models, like BERT, help overcome the problem of the available vulnerability detection datasets being too small to enable most DL models to capture all relevant patterns. They mitigate the challenge by leveraging knowledge from a general domain to solve problems in specific domains. In this paper, we explore different BERT-based models for multi-label classification of vulnerabilities in Java on a synthetic dataset. The models yield up to 99% in accuracy and 94% in f1-score. We remove biases in the training dataset and observe drops of up to 13% of the f1-score. We further assess the generalizability of the models on realistic samples and notice that one model, in particular, predicted unknown vulnerabilities with an f1-score of nearly 85%.
A Comparative Study of Deep Learning and Machine Learning Algorithm for Sentiment Analysis

Sentiment analysis is a classification procedure where we apply machine learning and deep learning algorithms to analyze the sentiment of the dataset, which consists of text, e.g., a message that can be of positive or negative sentiment. In this study, an attempt has been made to investigate which sentiment analysis techniques are feasible for product reviews. Here, the Amazon reviews dataset is used to compare, train, and test various machine learning and deep learning methods having product reviews from Amazon, which are chosen randomly from an open-source repository. The dataset comprises 4 million reviews. Comparison of several algorithms' performances, i.e., RFC, XGBC, LGBM, MNB, GBC, DTC, and Bi-LSTM, amongst which Bi-LSTM gives the highest performance among the algorithms used for classification. It was also applied to the other reviews from the Amazon dataset to predict the sentiment of the reviews, as well as a fresh Amazon scraped dataset comprising product reviews from several categories. This resulted in a very accurate classification, with the best results for test reviews on the amazon dataset. In conclusion, Bi-LSTM networks are excellent for categorizing customer sentiment on product reviews, and the results do not differ considerably across categories.
Real-Time Sign Language to Text and Speech Translation and Hand Gesture Recognition using the LSTM Model

Communication is very important between People to people. People who are “Specially Abled,” with speech or hearing impairments, “Mute” or “Deaf” individuals, respectively, constantly rely on visual communication. Due to a lack of knowledge in sign language, people without visual or hearing impairments occasionally encounter problems and cannot interact with those seeking their help. They accept sign language well and make use of it to communicate. Building a system that can translate gestures into text and voice is necessary to enable two-way communication between persons with disabilities and the broader population. Translating these sign languages into text and voice can be highly beneficial since it will link the deaf/mute and the rest of society by enabling communication between the two groups. This research paper proposes a method for recording hand movements (sign language). It makes use of the Mediapipe platform, which successfully recognizes the hand and extracts the relevant information to feed into the model. After the model recognizes the sign language, words and alphabets are merged to form a sentence, which is then converted to speech. With LSTM, the suggested framework has demonstrated effective accuracy. According to a survey, CNN and LSTM produce results that are superior in quality and accuracy.
Regression Analysis &amp; Visualization of Twitch Dataset

This paper examines Twitch dataset in order to determine and unveil various aspects and relationship among attributes of dataset. Twitch is an online platform used for video game live streaming. The models used by authors in their analysis are Logistic regression, Naïve Bayes, K-Nearest Neighbour and Decision tree. As demonstrated by the authors, K-Nearest Neighbour and logistic regressions outperform other model by yielding an accuracy of 97.7 <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">%</sup> and 97.4 %.
Aspect Analysis of Dementia Patients

Early diagnosis of Mental Cognitive Disorders (MCD) is crucial in establishing timely treatment to delay further progression of the disease. This paper demonstrates a data visualization and statistical data analysis of dataset collected from Kaggle. In this paper the authors have derived an analysis of a dataset based on Mild cognitive impairment (MCI) disorder. Most of the data that is brought to work is not clean and requires substantial amount of Data cleaning. Hence, for pre-processing and cleaning of data, we imported many python libraries and dealt with NULL values and outliers. These NULL values and outliers were treated using statistical methods and were replaced by their means and medians. Data visualization is the pictorial depiction of data by using visual elements like charts, graphs and maps. When data-processing and cleaning was completed, data was visualized with varied graphs and charts to create appealing, informative and detailed plots for presenting data in the most effective and easy way. The results focused on the visualization of demented and non-demented people of different genders along with other parameters to lay out a clear picture for the reader.
Sentiment Analysis Using Machine Learning and Recurrent Neural Networks

Sentiment Analysis is an NLP problem dealing with the understanding of emotions and assigning the tag of negative or positive to the tweets. For this purpose we have used carefully picked classification models inclusive of Bernoulli NB which yields accuracy 78.25%, Multinomial NB which yields accuracy 78.34%, Support vector Machines which yields accuracy 78.25% and Logistic Regression with best accuracy 79.63% among the ML models and the DL model Recurrent Neural Networks which gave accuracy 81.45% and through in-depth analysis of our data set we have preprocessed it in such a way that the dataset is clean and set to be trained in the models without underfitting or overfitting.
Performance Issues and Research Challenges in Designing a Secure IoT with Blockchain

Now a day, Internet of Things (IoT) are being used in several areas such as vehicular systems, smart city, healthcare, and supply chain management. Sensor data is stored in the cloud in an IoT system. Therefore, there is always a security concern with this data. There are several techniques for maintaining security, but most of them work in a centralized manner. However, blockchain is an emerging technology that works in a distributed manner and is based on peer-to-peer computing. Security can be enhanced using blockchain in any system. Blockchain technology offers decentralised security and privacy. Sensors used in loT systems are resource constrained. Therefore, implementing blockchain in any loT system creates several research challenges. This paper provides a discussion on such performance issues and research challenges while integrating blockchain with loT. The paper also gives some insights to overcome such issues.
A Drone Video Clip Dataset and its Applications in Automated Cinematography

Abstract Drones became popular video capturing tools. Drone videos in the wild are first captured and then edited by humans to contain aesthetically pleasing camera motions and scenes. Therefore, edited drone videos have extremely useful information for cinematography and for applications such as camera path planning to capture aesthetically pleasing shots. To design intelligent camera path planners, learning drone camera motions from these edited videos is essential. However, first, this requires to filter drone clips and extract their camera motions out of these edited videos that commonly contain both drone and non‐drone content. Moreover, existing video search engines return the whole edited video as a semantic search result and cannot return only drone clips inside an edited video. To address this problem, we proposed the first approach that can automatically retrieve drone clips from an unlabeled video collection using high‐level search queries, such as “drone clips captured outdoor in daytime from rural places”. The retrieved clips also contain camera motions, camera view, and 3D reconstruction of a scene that can help develop intelligent camera path planners. To train our approach, we needed numerous examples of edited drone videos. To this end, we introduced the first large‐scale dataset composed of edited drone videos. This dataset is also used for training and validating our drone video filtering algorithm. Both quantitative and qualitative evaluations have confirmed the validity of our method.
Real‐time Deep Radiance Reconstruction from Imperfect Caches

Abstract Real‐time global illumination is a highly desirable yet challenging task in computer graphics. Existing works well solving this problem are mostly based on some kind of precomputed data (caches), while the final results depend significantly on the quality of the caches. In this paper, we propose a learning‐based pipeline that can reproduce a wide range of complex light transport phenomena, including high‐frequency glossy interreflection, at any viewpoint in real time (&gt; 90 frames per‐second), using information from imperfect caches stored at the barycentre of every triangle in a 3D scene. These caches are generated at a precomputation stage by a physically‐based offline renderer at a low sampling rate (e.g., 32 samples per‐pixel) and a low image resolution (e.g., 64×16). At runtime, a deep radiance reconstruction method based on a dedicated neural network is then involved to reconstruct a high‐quality radiance map of full global illumination at any viewpoint from these imperfect caches, without introducing noise and aliasing artifacts. To further improve the reconstruction accuracy, a new feature fusion strategy is designed in the network to better exploit useful contents from cheap G‐buffers generated at runtime. The proposed framework ensures high‐quality rendering of images for moderate‐sized scenes with full global illumination effects, at the cost of reasonable precomputation time. We demonstrate the effectiveness and efficiency of the proposed pipeline by comparing it with alternative strategies, including real‐time path tracing and precomputed radiance transfer.
Classifier Guided Temporal Supersampling for Real‐time Rendering

Abstract We present a learning based temporal supersampling algorithm for real‐time rendering. Different from existing learning‐based approaches that adopt an end‐to‐end training of a ‘black‐box’ neural network, we design a ‘white‐box’ solution that first classifies the pixels into different categories and then generates the supersampling result based on classification. Our key observation is that the core problem in temporal supersampling for rendering is to distinguish the pixels that consist of occlusion, aliasing, or shading changes. Samples from these pixels exhibit similar temporal radiance change but require different composition strategies to produce the correct supersampling result. Based on this observation, our method first classifies the pixels into several classes. Based on the classification results, our method then blends the current frame with the warped last frame via a learned weight map to get the supersampling results. We design compact neural networks for each step and develop dedicated loss functions for pixels belonging to different classes. Compared to existing learning based methods, our classifier‐based supersampling scheme takes less computational and memory cost for real‐time supersampling and generates visually compelling temporal supersampling results with fewer flickering artifacts. We evaluate the performance and generality of our method on several rendered game sequences and our method can upsample the rendered frames from 1080P to 2160P in just 13.39ms on a single Nvidia 3090GPU.
Effective Eyebrow Matting with Domain Adaptation

Abstract We present the first synthetic eyebrow matting datasets and a domain adaptation eyebrow matting network for learning domain‐robust feature representation using synthetic eyebrow matting data and unlabeled in‐the‐wild images with adversarial learning. Different from existing matting methods that may suffer from the lack of ground‐truth matting datasets, which are typically labor‐intensive to annotate or even worse, unable to obtain, we train the matting network in a semi‐supervised manner using synthetic matting datasets instead of ground‐truth matting data while achieving high‐quality results. Specifically, we first generate a large‐scale synthetic eyebrow matting dataset by rendering avatars and collect a real‐world eyebrow image dataset while maximizing the data diversity as much as possible. Then, we use the synthetic eyebrow dataset to train a multi‐task network, which consists of a regression task to estimate the eyebrow alpha mattes and an adversarial task to adapt the learned features from synthetic data to real data. As a result, our method can successfully train an eyebrow matting network using synthetic data without the need to label any real data. Our method can accurately extract eyebrow alpha mattes from in‐the‐wild images without any additional prior and achieves state‐of‐the‐art eyebrow matting performance. Extensive experiments demonstrate the superior performance of our method with both qualitative and quantitative results.
MINERVAS: Massive INterior EnviRonments VirtuAl Synthesis

Abstract With the rapid development of data‐driven techniques, data has played an essential role in various computer vision tasks. Many realistic and synthetic datasets have been proposed to address different problems. However, there are lots of unresolved challenges: (1) the creation of dataset is usually a tedious process with manual annotations, (2) most datasets are only designed for a single specific task, (3) the modification or randomization of the 3D scene is difficult, and (4) the release of commercial 3D data may encounter copyright issue. This paper presents MINERVAS, a Massive INterior EnviRonments VirtuAl Synthesis system, to facilitate the 3D scene modification and the 2D image synthesis for various vision tasks. In particular, we design a programmable pipeline with Domain‐Specific Language, allowing users to select scenes from the commercial indoor scene database, synthesize scenes for different tasks with customized rules, and render various types of imagery data, such as color images, geometric structures, semantic labels. Our system eases the difficulty of customizing massive scenes for different tasks and relieves users from manipulating fine‐grained scene configurations by providing user‐controllable randomness using multilevel samplers. Most importantly, it empowers users to access commercial scene databases with millions of indoor scenes and protects the copyright of core data assets, e.g., 3D CAD models. We demonstrate the validity and flexibility of our system by using our synthesized data to improve the performance on different kinds of computer vision tasks. The project page is at https://coohom.github.io/MINERVAS .
Contrastive Semantic‐Guided Image Smoothing Network

Abstract Image smoothing is a fundamental low‐level vision task that aims to preserve salient structures of an image while removing insignificant details. Deep learning has been explored in image smoothing to deal with the complex entanglement of semantic structures and trivial details. However, current methods neglect two important facts in smoothing: 1) naive pixel‐level regression supervised by the limited number of high‐quality smoothing ground‐truth could lead to domain shift and cause generalization problems towards real‐world images; 2) texture appearance is closely related to object semantics, so that image smoothing requires awareness of semantic difference to apply adaptive smoothing strengths. To address these issues, we propose a novel Contrastive Semantic‐Guided Image Smoothing Network (CSGIS‐Net) that combines both contrastive prior and semantic prior to facilitate robust image smoothing. The supervision signal is augmented by leveraging undesired smoothing effects as negative teachers, and by incorporating segmentation tasks to encourage semantic distinctiveness. To realize the proposed network, we also enrich the original VOC dataset with texture enhancement and smoothing labels, namely VOC‐smooth, which first bridges image smoothing and semantic segmentation. Extensive experiments demonstrate that the proposed CSGIS‐Net outperforms state‐of‐the‐art algorithms by a large margin. Code and dataset are available at https://github.com/wangjie6866/CSGIS-Net .
Learning Dynamic 3D Geometry and Texture for Video Face Swapping

Abstract Face swapping is the process of applying a source actor's appearance to a target actor's performance in a video. This is a challenging visual effect that has seen increasing demand in film and television production. Recent work has shown that data‐driven methods based on deep learning can produce compelling effects at production quality in a fraction of the time required for a traditional 3D pipeline. However, the dominant approach operates only on 2D imagery without reference to the underlying facial geometry or texture, resulting in poor generalization under novel viewpoints and little artistic control. Methods that do incorporate geometry rely on pre‐learned facial priors that do not adapt well to particular geometric features of the source and target faces. We approach the problem of face swapping from the perspective of learning simultaneous convolutional facial autoencoders for the source and target identities, using a shared encoder network with identity‐specific decoders. The key novelty in our approach is that each decoder first lifts the latent code into a 3D representation, comprising a dynamic face texture and a deformable 3D face shape, before projecting this 3D face back onto the input image using a differentiable renderer. The coupled autoencoders are trained only on videos of the source and target identities, without requiring 3D supervision. By leveraging the learned 3D geometry and texture, our method achieves face swapping with higher quality than when using off‐the‐shelf monocular 3D face reconstruction, and overall lower FID score than state‐of‐the‐art 2D methods. Furthermore, our 3D representation allows for efficient artistic control over the result, which can be hard to achieve with existing 2D approaches.
Hierarchical Association Features Learning for Network Traffic Recognition

With the development of network technology, identifying specific traffic has become important in network monitoring and security. However, designing feature sets that can accurately describe network traffic is still an urgent problem. Most of existing researches cannot realize effectively the identification of targets, and don't perform well in the complex and dynamic network environment. Aiming at these problems, we propose a novel method in this paper, which learns correlation features of network traffic based on the hierarchical structure. Firstly, the method learns the spatial-temporal features using convolutional neural networks (CNNs) and the bidirectional long short-term memory networks (Bi-LSTMs), then builds network topology to capture dependency characteristics between sessions and learns the context-related features through the graph attention networks (GATs). Finally, the network traffic session is classified using a fully connected network. The experimental results show that our method can effectively improve the detection ability and achieve a better classification performance overall.