Reviewerly/exHarmony huggingface dataset:
Medical health data privacy preservation using smart contract

This paper describes a blockchain-based privacy preservation scheme for medical health records. IoT, e-health, and cloud-based hospital management systems have all become important components of our healthcare system. Hospital patient data is extremely sensitive and preserving patient data privacy appears to be difficult. The current method for data storage is to use a cloud storage system. All of this is done through a centralized system, and some management people control everything, which increases the risk of a data breach. We propose a smart contract-based blockchain architecture for the privacy of medical health data. The key features of blockchain are immutability and user confidentiality. We use IoT applications to take advantage of blockchain technology and eliminate its disadvantages, such as low throughput, high overhead, and latency. To that end, we're clustering the BC miners and sending data to the nearest patient cluster with various keywords and multi-functionality, it supports secure data management.
Zero-Shot Attribute Attacks on Fine-Grained Recognition Models

Zero-shot fine-grained recognition is an important classification task, whose goal is to recognize visually very similar classes, including the ones without training images. Despite recent advances on the development of zero-shot fine-grained recognition methods, the robustness of such models to adversarial attacks is not well understood. On the other hand, adversarial attacks have been widely studied for conventional classification with visually distinct classes. Such attacks, in particular, universal perturbations that are class-agnostic and ideally should generalize to unseen classes, however, cannot leverage or capture small distinctions among fine-grained classes. Therefore, we propose a compositional attribute-based framework for generating adversarial attacks on zero-shot fine-grained recognition models. To generate attacks that capture small differences between fine-grained classes, generalize well to previously unseen classes and can be applied in real-time, we propose to learn and compose multiple attribute-based universal perturbations (AUPs). Each AUP corresponds to an image-agnostic perturbation on a specific attribute. To build our attack, we compose AUPs with weights obtained by learning a class-attribute compatibility function. To learn the AUPs and the parameters of our model, we minimize a loss, consisting of a ranking loss and a novel utility loss, which ensures AUPs are effectively learned and utilized. By extensive experiments on three datasets for zero-shot fine-grained recognition, we show that our attacks outperform conventional universal classification attacks and transfer well between different recognition architectures.
Interpretable and High-Performance Hate and Offensive Speech Detection

The spread of information through social media platforms can create environments possibly hostile to vulnerable communities and silence certain groups in society. To mitigate such instances, several models have been developed to detect hate and offensive speech. Since detecting hate and offensive speech in social media platforms could incorrectly exclude individuals from social media platforms, which can reduce trust, there is a need to create explainable and interpretable models. Thus, we build an explainable and interpretable high performance model based on the XGBoost algorithm, trained on Twitter data. For unbalanced Twitter data, XGboost outperformed the LSTM, AutoGluon, and ULMFiT models on hate speech detection with an F1 score of 0.75 compared to 0.38 and 0.37, and 0.38 respectively. When we down-sampled the data to three separate classes of approximately 5,000 tweets, XGBoost performed better than LSTM, AutoGluon, and ULMFiT; with F1 scores for hate speech detection of 0.79 vs 0.69, 0.77, and 0.66 respectively. XGBoost also performed better than LSTM, AutoGluon, and ULMFiT in the down-sampled version for offensive speech detection with F1 score of 0.83 vs 0.88, 0.82, and 0.79 respectively. We use Shapley Additive Explanations (SHAP) on our XGBoost models’ outputs to makes it explainable and interpretable compared to LSTM, AutoGluon and ULMFiT that are black-box models.
Unpaired Image Captioning by Image-Level Weakly-Supervised Visual Concept Recognition

The goal of unpaired image captioning (UIC) is to describe images without using image-caption pairs in the training phase. Although challenging, we expect the task can be accomplished by leveraging images aligned with visual concepts. Most existing studies use off-the-shelf algorithms to obtain the visual concepts because the Bounding Box (BBox) labels or relationship-triplet labels used for training are expensive to acquire. To avoid exhaustive annotations, we propose a novel approach to achieve cost-effective UIC. Specifically, we adopt image-level labels to optimize the UIC model in a weakly-supervised manner. For each image, we assume that only the image-level labels are available without specific locations and numbers. The image-level labels are utilized to train a weakly-supervised object recognition model to extract object information (e.g., instance), and the extracted instances are adopted to infer the relationships among different objects using an enhanced graph neural network (GNN). The proposed approach achieves comparable or even better performance compared with previous methods without expensive annotations. Furthermore, we design an unrecognized object (UnO) loss to improve the alignment of the inferred object and relationship information with the images. It can effectively alleviate the issue encountered by existing UIC models when generating sentences with nonexistent objects. To the best of our knowledge, this is the first attempt to address the problem of Weakly-Supervised visual concept recognition for UIC (WS-UIC) based only on image-level labels. Extensive experiments demonstrate that the proposed method achieves inspiring results on the COCO dataset while significantly reducing the labeling cost.
Study on UAV obstacle avoidance algorithm based on deep recurrent double Q network

The traditional reinforcement learning method has the problems of overestimation of value function and partial observability in the field of machine motion planning, especially in the obstacle avoidance problem of UAV, which lead to long training time and difficult convergence in the process of network training. This paper proposes an obstacle avoidance algorithm for UAVs based on a deep recurrent double Q network. By transforming the single-network structure into a dual-network structure, the optimal action selection and action value estimation are decoupled to reduce the overestimation of the value function. The fully connected layer introduces the GRU recurrent neural network module, and uses the GRU to process the time dimension information, enhance the analyzability of the real neural network, and improve the performance of the algorithm in some observable environments. On this basis, combining with the priority experience playback mechanism, the network convergence is accelerated. Finally, the original algorithm and the improved algorithm are tested in the simulation environment. The experimental results show that the algorithm has better performance in terms of training time, obstacle avoidance success rate and robustness.
Design of Google Apps-based helpdesk ticketing system at University using extreme programming approach

Academic activities require supporting devices that are always ready, both hardware and software, and networks. However, the device may be experiencing problems. Fast problem solving is needed so that academic activities continue to run well. The problem is that the asset data is not integrated with the helpdesk, the difficulty of prioritizing work, and there is no previous troubleshoot recording as reference material. The Extreme Programming approach is used to make the development process more efficient, with iterations carried out at each stage. The SWOT Analysis and Value Chain Analysis helps identify the requirements correctly. Using Google Apps as a development tool for the Helpdesk Ticketing System proves to be more efficient as there is no need to invest in server hardware for processing and storage. Google Apps Script is also reliable enough to complete various processes designed in this system. Implementation of Google Apps is also easier because users are familiar with Google Forms and Google Sheets interfaces. With a good helpdesk ticketing system process, the implementation of academic activities will also run well.
Eigencontours: Novel Contour Descriptors Based on Low-Rank Approximation

Novel contour descriptors, called eigencontours, based on low-rank approximation are proposed in this paper. First, we construct a contour matrix containing all object boundaries in a training set. Second, we decompose the contour matrix into eigencontours via the best rank-M approximation. Third, we represent an object boundary by a linear combination of the <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$M$</tex> eigencontours. We also incorporate the eigencontours into an instance segmentation framework. Experimental results demonstrate that the proposed eigencontours can represent object boundaries more effectively and more efficiently than existing descriptors in a low-dimensional space. Furthermore, the proposed algorithm yields meaningful performances on instance segmentation datasets.
The Application Research of Education Metaverse Under the Framework of SWOT Analysis

Education Metaverse provides new development directions and pursuit goals for the new education model in the future. In fact, under the impact of the new crown pneumonia epidemic, the advantages of Education Metaverse are reflected. Education Metaverse provides teachers, students, enterprises and participants. New possibilities for living and learning. This article analyzes the fusion of the metaverse and education in thought, technology, scene, and body, and conducts a SWOT analysis of the metaverse on education. It is found that the education metaverse is a driving metaverse that combines the needs of reality, market to demand, individual needs and the needs of the larger environment. Provide an immersive, authentic, interactive, innovative, diverse, and collaborative education metaverse, which can effectively to promote education and teaching, as well as reform and innovation, which has a better role in promoting the future development of education. Grasping the integration between the metaverse and education can better open up, actively promote the development of the educational metaverse, expand the metaverse of education and teaching, the metaverse of the campus, the metaverse of the society, and promote the interdisciplinary of the educational metaverse, the excellent development of production-university-research and hybrid models.
Object Detection in Computer Vision Using Machine Learning Algorithm For Visually Impaired People

Traditionally, blind individuals go to their destinations with a white cane or a guide dog. They were unable to quickly recognize their surroundings, though. Consequently, The development of a navigation system for visually impaired people in an interior environment is the topic of this study. In order to provide an efficient and user-friendly navigation tool, passive radio frequency identification (RFID) transponders are mounted on the floor, such as on tactile pavement, to establish such RFID networks. the navigational system created includes a digital compass to help persons who are blind or visually handicapped walk correctly and in the right direction, especially while turning. This system makes use of the ideas of localization and placement using a digital compass as well as vocal prompts for directions. Others looked into how to get blind people back onto the right path when they go off course and calibrate the digital compass for accuracy. Additionally, in order to determine the practicality of the developed navigation system, a comparison between two subjects—a mobile robot and a human—is done. The results of the experiment show that a mobile robot and a human move at different speeds. People who are blind or visually impaired will benefit from this project since the voice-activated navigation system will improve their travel experience and make it safer and more comfortable.
Stay in Grid: Improving Video Captioning via Fully Grid-Level Representation

Video captioning is a challenging task of automatically generating natural and meaningful textual descriptions given some context videos. The state-of-the-art methods aggregate the spatial-wise information in the video encoder at the early stage, which has two drawbacks: 1) Early aggregation in the encoder can cause considerable spatial details missing, which may consequently lead to incorrect word choices in the following text encoder. 2) The spatial attention learned in the video encoder may not be compelling enough without text guidance. To solve these problems, we propose a Stay-in-Grid video CAPtioning method SGCAP, which makes full use of the grid-level spatial features and consists of a Bilinear Sequential Attention Encoder (BSAE) and a Cross-modal Sequential Attention Decoder (CSAD). The former explores and retains fully grid-level discriminative representations in the video encoder, while the latter performs the late spatial aggregation in the decoder to attend to the most relevant regions with the supervision of the input words. Experimental results demonstrate the effectiveness of our method on three public datasets, showing its superior performance over multiple state-of-the-art video captioning models. Source codes and the pre-trained models will be made available to the public.
Operationalizing threats to MSR studies by simulation-based testing

Quantitative studies on the border between Mining Software Repository (MSR) and Empirical Software Engineering (ESE) apply data analysis methods, like regression modeling, statistic tests or correlation analysis, to commits or pulls to better understand the software development process. Such studies assure the validity of the reported results by following a sound methodology. However, with increasing complexity, parts of the methodology can still go wrong. This may result in MSR/ESE studies with undetected threats to validity. In this paper, we propose to systematically protect against threats by operationalizing their treatment using simulations. A simulation substitutes observed and unobserved data, related to an MSR/ESE scenario, with synthetic data, carefully defined according to plausible assumptions on the scenario. Within a simulation, unobserved data becomes transparent, which is the key difference to a real study, necessary to detect threats to an analysis methodology. Running an analysis methodology on synthetic data may detect basic technical bugs and misinterpretations, but it also improves the trust in the methodology. The contribution of a simulation is to operationalize testing the impact of important assumptions. Assumptions still need to be rated for plausibility. We evaluate simulation-based testing by operationalizing undetected threats in the context of four published MSR/ESE studies. We recommend that future research uses such more systematic treatment of threats, as a contribution against the reproducibility crisis.
VMEKNet: Visual Memory and External Knowledge Based Network for Medical Report Generation

The main purpose of the medical report generation task is to generate a medical report corresponding to a given medical image, which contains detailed information of body parts and diagnostic results from radiologists. The task not only greatly reduces the workload of radiologists, but also helps patients get medical treatment in time. However, there are still many limitations in this task. First, the gap between image semantic features and text semantic features hinders the accuracy of the generated medical reports. Second, there are a large number of similar features in different medical images, which are not utilized efficiently and adequately. In order to solve the problems mentioned above, we propose a medical report generation model VMEKNet that integrates visual memory and external knowledge into the task. Specifically, we propose two novel modules and introduce them into medical report generation. Among them, the TF-IDF Embedding (TIE) module incorporates external knowledge into the feature extraction stage via the TF-IDF algorithm, and the Visual Memory (VIM) module makes full use of previous image features to help the model extract more accurate medical image features. After that, a standard Transformer processes the image features and text features then generates full medical reports. Experimental results on benchmark datasets, IU X-Ray, have demonstrated that our proposed model outperforms previous works on both natural language generation metrics and practical clinical diagnosis.
3D Information Guided Motion Transfer via Sequential Image Based Human Model Refinement and Face-Attention GAN

Image and video based human motions can be regarded as the deformation processes of person appearances, so motion transfer is usually treated as a pose guided image generation task and implemented in the 2D image plane. However, the 2D plane image generation lacks guidance of the original 3D motion information, which results in blur and shape distortions of the generated motion images. Therefore, we propose to simulate the generation process of real motion images by projecting the 3D human models, which are reconstructed from the training motion images and driven with target poses, into the 2D plane. We then take the 2D projections as the pose representations and input them into the generation model as they naturally inherit the 3D information from the original motions. Considering the unreliability on the invisible surface of the single image based human model reconstruction, we propose a sequential image based human model refinement module which exploits the complementary information between adjacent motion frames to refine the 3D human model. Furthermore, we propose a face-attention GAN model to conduct the final motion transfer, in which we use the Gaussian distribution to match the elliptical face region and design a face enhancement loss function since the faces in the generated motion images influence the performances very much. The generated motion images with reliable depth information, accurate shapes and clear faces demonstrate the effectiveness of the proposed method.
Neural-Guided Runtime Prediction of Planners for Improved Motion and Task Planning with Graph Neural Networks

The past decade has amply demonstrated the remarkable functionality that can be realized by learning complex input/output relationships. Algorithmically, one of the most important and opaque relationships is that between a problem's structure and an effective solution method. Here, we quantitatively connect the structure of a planning problem to the performance of a given sampling-based motion planning (SBMP) algorithm. We demonstrate that the geometric relationships of motion planning problems can be well captured by graph neural networks (GNNs) to predict SBMP runtime. By using an algorithm portfolio we show that GNN predictions of runtime on particular problems can be leveraged to accelerate online motion planning in both navigation and manipulation tasks. Moreover, the problem-to-runtime map can be inverted to identify subproblems easier to solve by particular SBMPs. We provide a motivating example of how this knowledge may be used to improve integrated task and motion planning on simulated examples. These successes rely on the relational structure of GNNs to capture scalable generalization from low-dimensional navigation tasks to high degree-of-freedom manipulation tasks in 3d environments.
Online Graph Learning In Dynamic Environments

Inferring the graph topology that characterizes structured data is pivotal to many graph-based models when predefined graphs are not available. This paper focuses on learning graphs in the case of sequential data in dynamic environments. For sequential data, an online version of classic batch graph learning method is developed. To better track dynamic graphs, we assume the graphs evolve in some certain patterns such that dynamic priors might be embedded in the online graph learning framework. When we have no knowledge of the hidden patterns, a data-driven method is leveraged to predict the evolution of graphs. Furthermore, dynamic regret analysis of the proposed method is performed, illustrating theoretically that proper dynamic priors do reduce regret. Experimental results support that our method is superior to the state-of-the-art methods.
Image Synthesis Using Conditional GANs for Selective Laser Melting Additive Manufacturing

In-situ process monitoring for metals additive manufacturing is paramount to the successful build of an object for application in extreme or high stress environments. Yet in selective laser melting additive manufacturing, it is extremely difficult to evaluate the build process. The difficulty is that obtaining enough variety of data to quantify the internal microstructures for the evaluation of its physical properties is problematic, as the laser passes at high speeds over powder grains at a micrometer scale. Using generative models, a type of machine learning, has been shown here to provide new artificially generated data with the same properties as the experimental images. The Generative Adversarial Network (GAN) synthesized new computationally derived data through a process that learns the underlying features of images that correspond to the different laser process parameters in a generator network. While this technique was effective at delivering high-quality images that closely matched the training data when tested against holdout samples, modifications to the general form of the network through a conditional generative adversarial network (CGAN) showed improved capabilities at creating these new images. Using multiple evaluation metrics, it has been shown that generative models can be used to create new data for various laser process parameter combinations, thereby allowing a more comprehensive evaluation of ideal laser conditions for any particular build. The new data can supplement the experimental data, thereby growing the overall knowledge framework for build characteristics.
SRNet: Structural Relation-aware Network for Head Pose Estimation

Estimating head pose from a single RGB image has recently attracted considerable research attention. Prior arts employ a CNN backbone to process face images and then directly output Euler angles. We argue that they may ignore essential features that are highly correlated to head pose due to the non-global perspective, and the ambiguity and discontinuity issues of Euler angles representation could interfere with the performance of challenging samples. In this paper, we formulate the head pose estimation problem into quaternion representation space and propose a novel framework named Structural Relation-aware Network (SRNet). Different from previous methods, our SRNet explicitly explores the correlation among different regions of the face for mining global facial structure information. Furthermore, in order to boost robustness and generalization of the model, a hard example mining (HEM) strategy is designed to mitigate the data imbalance issue by adjusting the contributions of examples in different states to loss. Extensive experiments demonstrate that our method outperforms the current state-of-the-art alternatives on the public benchmark datasets: AFLW2000 and BIWI.