Reviewerly/exHarmony huggingface dataset:
Motion detection in moving camera videos using background modeling and FlowNet

Real-time moving object detection is challenging for moving cameras due to the moving background. Many studies use homography matrix to compensate for global motion by warping the background model to the current frame. Then, the pixel difference between the current frame and the background model is used for background subtraction. Moving pixels are extracted by applying adaptive threshold and some post-processing techniques. On the other hand, deep learning-based dense optical flow can be efficient enough to extract the moving pixels, but it increases computational cost. This study proposes a method to enhance a classical background modeling method with deep learning-based dense optical flow. The main contribution of this paper is to propose a fusing algorithm for dense optical flow and background modeling approach. The background modeling methods are error-prone, especially with continuous camera movement, while the optical flow method alone may not always be efficient. Our hybrid method fuses both techniques to improve the detection accuracy. We propose a software architecture to run background modeling and dense optical flow methods in parallel processes. The proposed implementation approach significantly increases the method's working speed, while the proposed fusion and combining strategy improve detection results. The experimental results show that the proposed method can run at high speed and has satisfying performance against the methods in the literature.
A Blockchain-Based Anti-Counterfeit and Traceable NBA Digital Trading Card Management System

NBA (National Basketball Association) trading cards are a hot collector’s item, with sales increasing rapidly every year. However, with the popularity of online trading, some sellers have started to intentionally and unintentionally sell imitation trading cards, and even PwC (Pricewaterhouse Coopers) is not immune. However, the PSA (Professional Sports Authenticator), which is the authentication agency, is not liable for this. Faced with the above situation, we moved trading cards online and proposed a blockchain-based anti-counterfeit and traceable NBA digital trading card management system, using blockchain technology to protect digital trading cards, and special digital copyright, to move from relying on other regulators to achieve the fight against counterfeit cards and maintain the security of the digital trading card market. Finally, we analyzed the security of the system and compared it with other methods. Our system uses Hyperledger Fabric to share data while protecting corporate privacy. Proxy re-encryption enables secure and trusted access authorization for digital transaction cards. Asymmetric encryption protects the data and uses signatures to achieve traceability and non-repudiation. Overall, our system solves the problem of counterfeiting and traceability that can occur in the digital trading card process from production to purchase.
Adaptive safety-aware semi-supervised clustering

Recently, safe semi-supervised clustering (S3C) has become an emerging topic in machine learning field. S3C aims to reduce the performance degradation probability of wrong prior knowledge. The existing S3C methods tries to artificially estimate the risk or safety degrees through a predefined formula. The performance of S3C heavily depends on the quality of the formula. To alleviate the influence of the artificially defined formula, it is meaningful to develop an novel S3C version which can adaptively estimate the safety degrees. To achieve the goal, we propose an adaptive safety-aware semi-supervised fuzzy c -means algorithm (AS3FCM) which is used to deal with mislabeled instances. In order to adaptively estimate the safety degrees, AS3FCM employs a local consistency strategy to build a regularization term. Meanwhile, two regularization terms are constructed to constrain the outputs of the labeled instances. The constructed regularization terms are then embedded into the objective function of FCM. Finally, an alternating iterative optimization strategy is utilized to solve the optimization problem. In each iteration, the safety degrees can be adaptively computed and updated. To evaluate the performance of our algorithm, some experiments are conducted on toy and several benchmark datasets. As the mislabeled percentages change, AS3FCM can obtain the clustering accuracy from 81.23% to 82.87% on Heart and 86.84% to 88.12% on WDBC while the other S3C methods yield the accuracy from 80.09% to 82.41% on Heart and 86.34% to 87.94% on WDBC. It verifies that AS3FCM can reasonably estimate the safety degrees and achieve a safe exploration of the mislabeled instances. • A S3C method is developed to adaptively estimate the safety degrees. • Three terms are constructed to safely explore the mislabeled instances. • An effective strategy is used to solve the optimization problem. • Our algorithm yields the best performance compared to the other S3C methods.
Predicting A User's Demographic Identity from Leaked Samples of Health-Tracking Wearables and Understanding Associated Risks

Recently, market wearables, such as smartwatches with their powerful sensing and computing capabilities, are helping us to real-time monitoring activity, sleep quality, mental health, heart and blood disorders, among many others. Ad-ditionally’ these wearables support various services, including unlocking smartphones or cars, checking emails, and managing financial payments. Since most of these services are based on users' personal data, such as demographics, researchers have been trying to secure those personal data using authentication mechanisms based on biometric, such as heart rate, step count, calorie burn, and blood oxygen saturation values. However, leaking these biometric samples can reveal a user's demographic information, leading to user vulnerability both in cyberspace and physical space. In this work, we demonstrate that biometric samples from single and paired wearables can be utilized to predict a user's two types of attributes or demographic information, i.e., discrete or categorical attribute (ethnicity) and continuous attribute (age) using classifiers and regression models. From our detailed analysis of 40 subjects, we can predict the ethnicity of an unknown user with an average accuracy of up to 0.73 ± 0.06 and the age of an unknown user with an average root mean square error of up to 7.85 + 0.14.
ITUnet: Integration Of Transformers And Unet For Organs-At-Risk Segmentation

Recently, convolutional neural network(CNN) has achieved great success in medical image segmentation. However, due to the limitation of convolutional receptive field, the pure convolutional neural network is difficult to further improve its performance. Given the outstanding ability of transformers in extracting the long-range dependency, some works have successfully applied it to computer vision and achieved better results than CNN in some tasks. Based on transformers could remedy the shortage of CNN, in this paper, we propose ITUnet, a segmentation network using CNN and transformers as features extractor. The combination of CNN and transformers enables the network to learn both short- and long-range dependency of features, which is beneficial to segmentation tasks. We evaluate our method on a head-and-neck CT dataset which has 18 kinds of organs to be segmented. The experimental results demonstrate that our proposed method shows better accuracy and robustness, the proposed methods achieve the Dice score of 77.72 and the 95% Hausdorff Distance of 2.31, outperforming the existing methods.
Focal U-Net: A Focal Self-attention based U-Net for Breast Lesion Segmentation in Ultrasound Images

Accurate breast lesion segmentation in ultrasound images helps radiologists to make exact diagnoses and treatments, which is important to increase the survival rate of breast cancer patients. Recently, deep learning-based methods have demonstrated remarkable results in breast lesion segmentation. However, the blurry breast lesion boundaries and noise artifacts in ultrasound images still limit the performance of the deep learning-based methods. In this paper, we propose a novel segmentation network equipped with a focal self-attention block for improving the performance of breast lesion segmentation. The focal self-attention block can incorporate fine-grained local and coarse-grained global information. The fine-grained local information is useful to enhance features of breast lesion boundaries, while the coarse-grained global information effectively reduces noise interference. To verify the performance of our network, we implement breast lesion segmentation on our collected dataset of 9836 ultrasound images. The results demonstrate that the focal self-attention block enhances features of breast lesion boundaries and improves the accuracy of breast lesion segmentation.
Enabling Automatic Cinematography with Reinforcement Learning

Rule-based automated cinematography has proven to be a very effective means of speeding up the filmmaking process and reducing costs by incorporating the rules of many camera shots into the optimization process. However, it is observed that many well-known directors developed their style of lens language and sometimes even violate the conventional rules. It is a very challenging task to exploit these directors' versatile styles into the automatic cinematography framework due to two main reasons: (i) it is not trivial to translate these styles into a library of rules; (ii) the data can be collected from existing films are very limited and the accuracy is insufficient for learning purposes. In this paper, we propose a novel unified Reinforcement learning-based Text to Animation (RT2A) framework that can apply reinforcement learning to automatic cinematography. In RT2A, the decisions by the director on the camera settings are recorded and can be utilized for the future auto cinematography agent training process. A well-designed reward functionality has been proposed that guides the algorithm to find the best policy and mimic the human director's decision-making process for the camera selection of each scene. The experimental results show that the proposed RT2A can effectively imitate the directors' usage of lens language patterns. Compared to the reference algorithm, RT2A can achieve a gain of up to 50% in camera placement acceptance rate and 80% in imitating the rhythm of camera switching.
FEAT: A Federated Approach for Privacy-Preserving Network Traffic Classification in Heterogeneous Environments

Network traffic classification is the foundation for many network security and network management applications. Recently, to preserve the privacy of the data which are generated in the mobile ends, federated learning (FL)-based classification methods are being proposed. Unfortunately, the performance of FL-based methods can seriously degrade when the client data have skewness. This is particularly true for mobile network traffic classification where the environments in the mobile ends are highly heterogeneous. In this article, we first conduct a measurement study on traffic classification accuracy through FL using real-world network traffic trace and we observe serious accuracy degradation due to heterogeneous environments. We propose a novel federated analytics (FA) approach, FEAT, to improve the accuracy. Note that FL emphasizes on model training, yet our FA performs local analytic tasks that can estimate traffic data skewness and select appropriate clients for FL model training. Our analytics tasks are performed locally and in a federated manner; thus, we preserve privacy as well. Our approach has strong theoretical properties where we exploit Hoeffding inequality to infer traffic data skewness and we leverage the Thompson Sampling for client selection. We evaluate our approach through extensive experiments using real-world traffic data sets QUIC and ISCX. The extensive experiments demonstrate that FEAT can improve traffic classification accuracy in heterogeneous environments.
Making Resets away from Targets: POI aware Redirected Walking

Rapidly developing Redirected Walking (ROW) technologies have enabled VR applications to immerse users in large virtual environments (VE) while actually walking in relatively small physical environments (PE). When an unavoidable collision emerges in a PE, the ROW controller suspends the user's immersive experience and resets the user to a new direction in PE. Existing ROW methods mainly aim to reduce the number of resets. However, from the perspective of the user experience, when users are about to reach a point of interest (POI) in a VE, reset interruptions are more likely to have an impact on user experience. In this paper, we propose a new ROW method, aiming to keep resets occurring at a longer distance from the virtual target, as well as to reduce the number of resets. Simulation experiments and real user studies demonstrate that our method outperforms state-of-the-art ROW methods in the number of resets and dramatically increases the distance between the reset locations and the virtual targets.
Multi-View Graph Neural Architecture Search for Biomedical Entity and Relation Extraction

Recently, graph neural architecture search (GNAS) frameworks have been successfully used to automatically design the optimal neural architectures for many problems such as node classification and graph classification. In the existing GNAS frameworks, the designed graph neural network (GNN) architectures learn the representation of homogenous graphs with one type of relationship connecting two nodes. However, multi-view graphs, where each view represents a type of relationship among nodes, are ubiquitous in the real world. The traditional GNAS frameworks learn the graph representation without considering the interactions between nodes and multiple relationships, so they fail to solve multi-view graph-based problems, such as multi-view graphs modelling the biomedical entity and relation extraction tasks. In this paper, we propose MVGNAS, a multi-view graph neural network automatic modelling framework for biomedical entity and relation extraction, to resolve this challenge. In MVGNAS, we propose an automatic multi-view representation learning to learn low-dimensional representations of nodes that capture multiple relationships in a multi-view graph, representing the first research work in literature to solve the problem of multi-view graph representation learning architecture search for biomedical entity and relation extraction tasks. The experimental results demonstrate that MVGNAS can achieve the best performance in biomedical entity and relation extraction tasks against the state-of-the-art baseline methods.
PU-GEN: Enhancing generative commonsense reasoning for language models with human-centered knowledge

Generative commonsense reasoning refers to the ability of a language model to generate a sentence with a given concept-set based on compositional generalization and commonsense reasoning. In the CommonGen challenge, which evaluates the capability of generative commonsense reasoning, language models continue to exhibit low performances and struggle to leverage knowledge representation from humans. Therefore, we propose PU-GEN to leverage human-centered knowledge in language models to enhance compositional generalization and commonsense reasoning considering the human language generation process. To incorporate human-centered knowledge, PU-GEN reinterprets two linguistic philosophies from Wittgenstein: picture theory and use theory. First, we retrieve scene knowledge to reflect picture theory such that a model can describe a general situation as if it were being painted. Second, we extend relational knowledge to consider use theory for understanding various contexts. PU-GEN demonstrates superior performance in qualitative and quantitative evaluations over baseline models in CommonGen and generates convincing evidence for CommonsenseQA. Moreover, it outperforms the state-of-the-art model used in the previous CommonGen challenge.
ContextKT: A Context-Based Method for Knowledge Tracing

Knowledge tracing, which is used to predict students’ performance based on their previous practices, has attracted many researchers’ attention. Especially in this rising period of intelligent education, many knowledge tracing methods have been developed. However, most of the existing knowledge tracing methods focus on the personality of practices and knowledge concepts but ignore the contexts related to the studying process. In this paper, we propose a context-based knowledge tracing model, which combines students’ historical performance and their studying contexts during knowledge mastery. To be specific, we first define five studying contexts for performance prediction. The basic context is the current knowledge state of a student, which is described by their practice sequences. Then, a QR-matrix is defined to represent the relationship among questions, knowledge concepts, and responses, which describes the contexts of questions and knowledge. Furthermore, an improved LSTM model is proposed to capture the context of students’ memory and forgetness, and a multi-head attention mechanism is designed to capture the context of students’ behaviors. Finally, based on the captured contexts, the prediction model ContextKT is established. Our prediction model is evaluated on two real educational datasets. The experimental results show our model is effective and efficient in student performance prediction, and it outperforms the other existing methods.
Digital morphology in hematology diagnosis and education: The experience of the European <scp>LeukemiaNet WP10</scp>

Hematological diagnostics is based on increasingly precise techniques of cellular and molecular analysis. The correct interpretation of the blood and bone marrow smears observed under an optical microscope still represents a cornerstone. Precise quantitative and qualitative cytomorphological criteria have recently been codified by up-to-date guidelines for diagnosing hematopoietic neoplasms. Morphological analysis has found formidable support in digital reproduction techniques, which have simplified the circulation of images for educational or consultation purposes. From 2007 to 2019, the Working Group WP10 of European LeukemiaNet (ELN) used, in annual exercises, digital images to support training in cytomorphology and verify harmonization and comparability in the interpretation of blood and bone marrow smears. We describe the design, development, and results of this program, which had 741 participants in-person or remotely, to which 2055 questions were submitted regarding the interpretation of cytomorphological images. We initially used circulation and presentation of digital microphotographs and then introduced a virtual microscopy (VM). Virtual slides were obtained using a whole slide imaging technique, similar to the one largely used in histopathology, to produce digitized scans of consecutive microscopic fields and reassembles them to obtain a complete virtual smear by stitching. Participants were required to identify cells in labeled fields of view of the virtual slides to obtain a morphological diagnosis. This work has demonstrated substantial improvements in diagnostic accuracy and harmonization with the VM technique. Between-observer concordance increased from 62.5% to 83.0%. The integrity of the digitalized film image, which provides a general context for cell abnormalities, was the main factor for this outcome.
A unified framework of transformations based on the Jordan–Wigner transformation

Quantum simulation of chemical Hamiltonians enables the efficient calculation of chemical properties. Mapping is one of the essential steps in simulating fermionic systems on quantum computers. In this work, a unified framework of transformations mapping fermionic systems to qubit systems is presented and many existing transformations-such as Jordan-Wigner, Bravyi-Kitaev, and parity transformations-are included in this framework. Based on this framework, the Multilayer Segmented Parity (MSP) transformation is proposed. The MSP transformation is a general mapping with an adjustable parameter vector, which can be viewed as a generalization of the above-mentioned mappings. Furthermore, the MSP transformation can adjust flexibly when dealing with different systems. Applying these mappings to the electronic structure Hamiltonians of various molecules, the MSP transformation is found to perform better on a number of Pauli operators and gates needed in the circuit of Hamiltonian simulation. The MSP transformation will reduce the qubit gate requirement for Hamiltonian simulation on noisy intermediate-scale quantum devices, and it will provide a much wider choice of mappings for researchers.
An adaptive multilevel thresholding method with chaotically-enhanced Rao algorithm

Multilevel image thresholding is a well-known technique for image segmentation. Recently, various metaheuristic methods have been proposed for the determination of the thresholds for multilevel image segmentation. These methods are mainly based on metaphors and they have high complexity and their convergences are comparably slow. In this paper, a multilevel image thresholding approach is proposed that simplifies the thresholding problem by using a simple optimization technique instead of metaphor-based algorithms. More specifically, in this paper, Chaotic enhanced Rao (CER) algorithms are developed where eight chaotic maps namely Logistic, Sine, Sinusoidal, Gauss, Circle, Chebyshev, Singer, and Tent are used. Besides, in the developed CER algorithm, the number of thresholds is determined automatically, instead of manual determination. The performances of the developed CER algorithms are evaluated based on different statistical analysis metrics namely BDE, PRI, VOI, GCE, SSIM, FSIM, RMSE, PSNR, NK, AD, SC, MD, and NAE. The experimental works and the related evaluations are carried out on the BSDS300 dataset. The obtained experimental results demonstrate that the proposed CER algorithm outperforms the compared methods based on PRI, SSIM, FSIM, PSNR, RMSE, AD, and NAE metrics. In addition, the proposed method provides better convergence regarding speed and accuracy.
SiamAGN: Siamese attention-guided network for visual tracking

Most Siamese-based trackers utilize cross-correlation to calculate the similarity scores between the target template and the search image, which may cause the loss of spatial information and lead to inaccurate target estimation. To address this issue, we propose an attention-guided model under the Siamese framework for object tracking, named SiamAGN. Specifically, we combine the template and search image through the proposed feature fusion module. It contains a self-context interaction (SCI) module, cross-context interaction (CCI) module, and target location module (TLM). SCI based on self-attention learns global context by emphasizing channel-wise complementary features. CCI based on cross-attention explores rich dynamic context via the channel interaction between the template and the search image. TLM based on cross-attention reformulates the template according to the pixel-level similarity scores between the template and the search image, which can keep as much spatial information as possible and enable our model to predict more precise bounding boxes. Extensive experimental results on the GOT10k, OTB100, VOT2016, VOT2018, UAV123, and LaSOT benchmarks indicate that the proposed tracker SiamAGN achieves competitive performance compared with state-of-the-art trackers.