Reviewerly/exHarmony huggingface dataset:
Implementation of Security Access Control using American Sign Language Recognition via Deep Learning Approach

Sign language is a kind of conversation that consists of a set of gestures or postures used to converse with the deaf and mute. It is usually accomplished with hands, which implies profound signals, especially when both the receiver and sender are well-versed in the subject. Signals generated by hand gestures can also be used in a variety of applications such as augmented reality (AR), gaming, robotics, and vision-based applications. However, sign language interpretation via computer vision has yet to be implemented as a security access control, which could provide a significantly greater authentication method and better statutory provisions. The trained model’s use as a security access control system was also taken into consideration. It is done by creating a Python-based GUI that takes a single frame from a camera. A layer loss of 2.803 and an mAP of 98.69 % were the final results after 14 epochs. The study shows that when compared to earlier comparable research pursuing the same objective, this study’s validation accuracy is the highest.
Detailed Eye Region Capture and Animation

Abstract Even if the appearance and geometry of the human eye have been extensively studied during the last decade, the geometrical correlation between gaze direction, eyelids aperture and eyelids shape has not been empirically modeled. In this paper, we propose a data‐driven approach for capturing and modeling the subtle features of the human eye region, such as the inner eye corner and the skin bulging effect due to globe orientation. Our approach consists of an original experimental setup to capture the eye region geometry variations combined with a 3D reconstruction method. Regarding the eye region capture, we scanned 55 participants doing 36 eyes poses. To animate a participant's eye region, we register the different poses to a vertex wise correspondence before blending them in a trilinear fashion. We show that our 3D animation results are visually pleasant and realistic while bringing novel eye features compared to state of the art models.
MP‐NeRF: Neural Radiance Fields for Dynamic Multi‐person synthesis from Sparse Views

Abstract Multi‐person novel view synthesis aims to generate free‐viewpoint videos for dynamic scenes of multiple persons. However, current methods require numerous views to reconstruct a dynamic person and only achieve good performance when only a single person is present in the video. This paper aims to reconstruct a multi‐person scene with fewer views, especially addressing the occlusion and interaction problems that appear in the multi‐person scene. We propose MP‐NeRF, a practical method for multi‐person novel view synthesis from sparse cameras without the pre‐scanned template human models. We apply a multi‐person SMPL template as the identity and human motion prior. Then we build a global latent code to integrate the relative observations among multiple people, so we could represent multiple dynamic people into multiple neural radiance representations from sparse views. Experiments on multi‐person dataset MVMP show that our method is superior to other state‐of‐the‐art methods.
Steganalysis for Small-Scale Training Image Pairs with Cover-Stego Feature Difference Model

Current steganalytic classifiers always need a large number of cover-stego image pairs for training. However in this paper we focus on a scenario where steganalysts have a few cover-stego image pairs in hand. Meanwhile steganalysts have no knowledge of the embedding algorithm, and cannot generate corresponding stego images after collecting additional cover images. Hence in this scenario steganalysts cannot match more cover-stego image pairs to augment the training set. To address this issue, we propose a stego feature simulation method to artificially generate cover-stego feature pairs for training. First, we design a cover-stego feature difference model to build the relationship between cover and stego features in pairs. Then, we estimate the model parameters from a few existing cover-stego image pairs in hand. Finally, after extracting steganalytic features from additionally collected cover images, we simulate corresponding stego features with the cover-stego feature difference model to match artificial feature pairs. The experimental results demonstrate that our method can effectively mitigate the shortage of training image pairs by adding adequate artificial feature pairs into training.
IntJect: Vulnerability Intent Bug Seeding

Studying and exposing software vulnerabilities is important to ensure software security, safety, and reliability. Software engineers often inject vulnerabilities into their programs to test the reliability of their test suites, vulnerability detectors, and security measures. However, state-of-the-art vulnerability injection methods only capture code syntax/patterns, they do not learn the intent of the vulnerability and are limited to the syntax of the original dataset. To address this challenge, we propose the first intent-based vulnerability injection method that learns both the program syntax and vulnerability intent. Our approach applies a combination of NLP methods and semantic-preserving program mutations (at the bytecode level) to inject code vulnerabilities. Given a dataset of known vulnerabilities (containing benign and vulnerable code pairs), our approach proceeds by employing semantic-preserving program mutations to transform the existing dataset to semantically similar code. Then, it learns the intent of the vulnerability via neural machine translation (Seq2Seq) models. The key insight is to employ Seq2Seq to learn the intent (context) of the vulnerable code in a manner that is agnostic of the specific program instance. We evaluate the performance of our approach using 1275 vulnerabilities belonging to five (5) CWEs from the Juliet test suite. We examine the effectiveness of our approach in producing compilable and vulnerable code. Our results show that IntJECT is effective, almost all (99%) of the code produced by our approach is vulnerable and compilable. We also demonstrate that the vulnerable programs generated by IntJECT are semantically similar to the withheld original vulnerable code. Finally, we show that our mutation-based data transformation approach outperforms its alternatives, namely data obfuscation and using the original data.
Marking based Obfuscation Strategy to resist Side Channel Attack in Cross-User Deduplication for Cloud Storage

Cross-user deduplication is an effective technique means to eliminate redundant uploading in using cloud storage by returning a specific response to indicate existence of the data in request. However, the deterministic response actually provides a side channel to attackers, which could be utilized to steal existence privacy of the target data in cloud storage. Because of the difficulty to achieve complete obfuscation in response for chunk level deduplication, such kind of attack still cannot be well resisted, let alone random chunks generation attack, a more sophisticated form to assist statistical attacks. Even though dirty chunks processing is able to resist such attacks under normal circumstances, a lot of communication overhead is introduced inevitably. To deal with these challenges, we propose a marking based obfuscation strategy in cross-user deduplication for cloud storage, which takes the lead to well improve security in cross-user deduplication under side channel attack as well as random chunks generation attack. Specifically, we first calculate the number of chunks to be marked to cover at least one duplicate chunk with great probability. Then perform marking strategy to the certain number of randomly chosen chunks, to consider them as inexistent before generating a response regardless of their real existence. To achieve light weight dirty chunks processing with security guaranteed, as long as at least a dirty chunk is found to be involved in the request, once more marking is introduced to achieve another obfuscation. The experimental results show that, the proposed scheme is lightweight in achieving security comparing with the state-of-the art.
Evaluation of planetary boundary layer parameterizations with the WRF model in the Metropolitan Region of Salvador.

Evaluation of the parameterization of the PBL Planetary Boundary Layer in the Metropolitan Region of Salvador using the Weather Research and Forecasting (WRF) model.The performance of numerical simulations was evaluated using radiosonde data.The physical parameterizations used in the model showed good agreement for temporal comparisons of vertical and horizontal wind speeds and wind directions predicted by the WRF model.The main objective of this initial study is to evaluate the performance of the physical parameterization of the model for the behavior in the PBL.The proposed initial study shows good agreement between the simulated and measured data, but the evaluation of the PBL for the period does not suggest a characterization of the Internal Boundary Layer.
Privacy-Preserving Smart Lock System for IoT-enabled Rental House Markets

Smart locks are one of the most popular Internet of things devices, which help ease the burden of physical key management in smart home, hotel, and especially in short-term rental house markets. Existing smart lock systems either suffer from the state consistency attacks that allow unauthorized access, or rely on a remote server to maintain an access list, which could be altered by compromised servers. Moreover, privacy risks could be posed to the guests when they reserve a property by the rental platform, which is usually integrated with the smart lock system. To address the above issues, we propose a smart lock system that provides security and privacy guarantees in IoT-enabled rental house markets. The proposed system resists state consistency attacks by utilizing a one-way hash chain to replace the access list. Meanwhile, we take advantage of dynamic accumulators and signature of knowledge to realize user authentication and privacy protection of the guests. Finally, we make a theoretical security and privacy analysis of the proposed system, and provide a comprehensive evaluation of the performance to show the efficiency of the proposed system.
Monocular Facial Performance Capture Via Deep Expression Matching

Abstract Facial performance capture is the process of automatically animating a digital face according to a captured performance of an actor. Recent developments in this area have focused on high‐quality results using expensive head‐scanning equipment and camera rigs. These methods produce impressive animations that accurately capture subtle details in an actor's performance. However, these methods are accessible only to content creators with relatively large budgets. Current methods using inexpensive recording equipment generally produce lower quality output that is unsuitable for many applications. In this paper, we present a facial performance capture method that does not require facial scans and instead animates an artist‐created model using standard blendshapes. Furthermore, our method gives artists high‐level control over animations through a workflow similar to existing commercial solutions. Given a recording, our approach matches keyframes of the video with corresponding expressions from an animated library of poses. A Gaussian process model then computes the full animation by interpolating from the set of matched keyframes. Our expression‐matching method computes a low‐dimensional latent code from an image that represents a facial expression while factoring out the facial identity. Images depicting similar facial expressions are identified by their proximity in the latent space. In our results, we demonstrate the fidelity of our expression‐matching method. We also compare animations generated with our approach to animations generated with commercially available software.
Improved Zero-Knowledge Proofs for Commitments from Learning Parity with Noise

Zero-knowledge proof for any relation amongst committed values is crucial and widely applicable in the design of high level cryptographic schemes, especially in privacy-preserving protocols. Besides quantum resistance, efficiency is what we are most concerned about, including asymptotic efficiency and concrete efficiency. Jain et al. proposed a simple string commitment scheme based on the Learning Parity with Noise (LPN) problem (JKPT12), and then designed zero-knowledge proofs for valid opening, linear relation and multiplicative relation of committed values. As a result, they got an efficient zero-knowledge proof for any circuit C, with communication complexity $\mathcal{O}(t|C|\ell \log \ell )$, where t is a security parameter measuring soundness and ℓ is the secret length of the LPN problem. In this work, we improve the concrete communication complexity by combining some commitments in JKPT12 together. The proofs of linear relation and multiplicative relation are shortened by (6α + 4)ℓ and (42α+28)ℓ respectively, where ℓ is the size of LPN secret. As a result, the communication cost of the protocol proving arbitrary relation is reduced by a constant level.
CGDF-GNN: Cascaded GNN fraud detector with dual features facing imbalanced graphs with camouflaged fraudsters

Due to the rich relational information of graph-structured data, graph neural networks (GNNs) have been widely used for fraud detection tasks, where the suspiciousness of nodes is identified by aggregating information about the neighbors of different relations. To bypass such detection, fraudsters camouflage themselves by providing seemingly legitimate feedback (i.e., feature camouflage) or connecting many legitimate users (i.e., relation camouflage). Moreover, when the label distribution of nodes is severely skewed, GNN-based algorithms may have limitations. To counteract the effect of disguised fraudsters under imbalanced graphs on fraud detection, we propose a cascaded GNN fraud detector with dual features (CGDF-GNN) in this paper. It tackles the impact of feature camouflage by applying a dual-feature parallel aggregation method and uses a cascaded neighbor aggregator to deal with the single-level learning problem arising from relational camouflage. Experiments on graph-based fraud detection tasks on two real-world datasets demonstrate the effectiveness of our model compared to current state-of-the-art baselines.
Mutation Testing based Safety Testing and Improving on DNNs

In recent years, deep neural networks (DNNs) have made great progress in people’s daily life since it becomes easier for data accessing and labeling. However, DNN has been proven to behave uncertainly, especially when facing small perturbations in their input data, which becomes a limitation for its application in self-driving and other safety-critical fields. Those human-made attacks like adversarial attacks would cause extremely serious consequences. In this work, we design and evaluate a safety testing method for DNNs based on mutation testing, and propose an adversarial training method based on testing results and joint optimization. First, we conduct an adversarial mutation on the test datasets and measure the performance of models in response to the adversarial samples by mutation scores. Next, we evaluate the validity of mutation scores as a quantitative indicator of safety by comparing DNN models and their updated versions. Finally, we construct a joint optimization problem with safety scores for adversarial training, thus improving the safety of the model as well as the generalizability of the defense capability.
MCFM: Discover Sensitive Behavior from Encrypted Traffic in Industrial Control System

To tackle with advanced persistent threats against industrial control system, Siemens has developed S7CommPlus- TLS, a new version of the encrypted protocol challenging traditional DPI-based anomaly detection methods. However, the communication mode of industrial control system leads to the overlapping of periodic traffic and sensitive behavior traffic, and thus makes mainstream encrypted traffic classification methods exhibit a poor performance in S7CommPlus-TLS protocol. Therefore, we design a multiple clustering framework called MCFM, which can automatically extract sensitive behavior of S7CommPlus-TLS from network traffic. The first-clustering is used as a pre-processing model to separate and remove periodic traffic from overlapping flows according to the communication mode of industrial control system. Besides, we employ the second- clustering as a generator to extract the fingerprint of sensitive behaviors. Our comprehensive experiments on the simulation dataset covering six sensitive behaviors indicate that MCFM achieves an excellent performance, and outperforms present cutting-edge methods. To the best of our knowledge, this is the first work analyzing industrial control system from the perspective of encrypted traffic analysis.
An Empirical Study Towards SAR Adversarial Examples

Adversarial attack and adversarial detection have become a hot issue in the field of deep learning based image forensics. However, current researches mainly focus on optical images. Synthetic aperture radar (SAR) images are quite different from the optical images in both imaging mechanism and data structure. This paper aims to study adversarial attack and adversarial detection for SAR images. Firstly, we analyze the distribution characteristics of SAR adversarial examples (AEs) in both output space and feature space by simply transferring optical attacks. In order to match the digital perturbation with the scattering energy of target, we then propose a generation method of SAR AEs with regional constraint. Experiments show that the proposed method decreases the attack performance of four classic adversarial attacks but leads to difficult detection. Finally, we point out an open issue that decreasing the perturbation scale leads to the degradation of adversarial detection against both optical AEs and SAR AEs.
Privacy-Preserving Recommendation with Debiased Obfuscaiton

As people enjoy the personalized services recommended by Recommender Systems (RSs), the privacy disclosure risk increases with frequent interactions. Malicious adversary often collects public information online to infer private information for illicit profit. As privacy concerns grew, researchers introduced data obfuscation into recommender systems. However, there still exists several limitations in current work. First, although the existing methods effectively reduce the risk of privacy disclosure, they can be detrimental to the quality of the recommendation service. Second, a range of practical issues under the application of recommendation systems are not considered, e.g., long-tail, density, etc. To address those challenges, we propose a novel framework named Want User Defending Inference (WUDI), a high-performance privacy-preserving debiased framework based on data obfuscation. Unlike the original strategies, i.e., adding or removing user ratings, we introduced some novel strategies to generate an obfuscated matrix. Firstly, we define a new method called Cluster Recommend for alleviating the long-tail skewness and data sparsity in RSs. Then we investigate the gender bias in obfuscation and apply a bias mitigating strategy to RSs. Experiments on public datasets demonstrate that WUDI can outperform the state-of-the-art baselines in obfuscation.
CBSDI: Cross-Architecture Binary Code Similarity Detection based on Index Table

Binary code similarity detection for cross-platform is widely used in plagiarism detection, malware detection and vulnerability search, aiming to detect whether two binary functions over different platforms are similar. Existing cross-architecture approaches mainly rely on the approximate matching calculation of complex high-dimensional features, such as graph, which are inevitably slow and unsuitable for large-scale applications. To solve this problem, we propose a novel approach based on index table called CBSDI, improving efficiency by screening a batch of mismatched functions before similarity detection. We select three features and compare them across architectures to select the most appropriate one to construct the index table, and this table can be embedded in other tools. The evaluation shows that the index table can roughly cut the computational costs in half when there are few errors. Moreover, compared with the related works in the literature, our proposed approach can improve not only the efficiency but also the accuracy.
An Exploratory Study for GUI Posts on Stack Overflow

Graphical User Interface (GUI) has become one of the most effective human-computer communication medium today. The quality of GUI is essential to the success of apps, especially for mobile apps. Developers not only have to understand the interaction of various components, but also follow the principles of design and implementation. It is helpful for developers to understand the challenges via analyzing the questions and answers (Q&A) on GUI development. However, there is no large-scale study on the GUI development posts on Stack Overflow. In this paper, we conduct an exploratory study on 23,741 posts related to GUI development on Stack Overflow. We first extract 20 topics related to GUI development using topic modeling. After manually classifying these GUI topics into 5 categories, we further quantitatively analyze the popularity and difficulty of GUI topics, the correlation between these two aspects, and qualitatively analyze the distribution of question types in posts. Finally, we have some interesting findings. These findings contain that the topic "tool selection" is the most popular topic, the topic "thread" has the highest percentage of unaccepted answers, and the topic "client/server" answer takes the longest time to be accepted. In addition, we discuss about possible inspirations of our research to GUI development stakeholders.
A Lightweight Graph-based Method to Detect Pornographic and Gambling Websites with Imperfect Datasets

With the widespread abuse of information technology, pornographic and gambling websites develop rapidly. They affect the physical and mental health of children and endanger personal property. Therefore, it is necessary to detect them. However, the existing detection methods ignored that imperfect datasets are common in the scenario of pornographic and gambling websites which are hence adverse to the detection. Those imperfections specifically include sparse samples, mismatch and imbalanced datasets. In addition, over-reliance on visual features incurred high overhead.To overcome these shortcomings, we innovatively propose a lightweight graph-based method to detect pornographic and gambling websites through semi-supervised learning of textual content. The semi-supervised learning is to solve sparse samples and mismatch datasets, while the graph-based approach can combine the semi-supervised part with community discovery to deal with imbalanced datasets. Specifically, we perform the detection process with the utilization of modified TF-IDF and Louvain during the iteration and updating by the EM algorithm. The experimental results show that our method achieves the best 92.01% Macro-Avg-F1 with the shortest CPU time and outperforms all baselines. We also illustrate that the designed components in our model do contribute to the detection.