Reviewerly/exHarmony huggingface dataset:
Lightweight batch authentication and privacy-preserving scheme for online education system

Nowadays, the online education system (OES) attracts learners to improve their knowledge and skills through learning flexibility and efficient assessment procedures. The learner can learn the courses and obtain a degree in a comfortable place, no need to go to school or college. Physical verification in the traditional education system is replaced by a secure authentication process. Many authentication schemes are existing to provide authentication, but they are suffering from authentication burden, due to high computational cost. In this work, two batch authentication schemes are introduced to validate the multiple learners and batch of messages/materials simultaneously instead of authenticating one after another. It reduces the authentication burden during the learner registration and message/material distribution process. The security and performance analysis sections ensures that the proposed scheme provides the essential security features with less computational costs compared to that of existing competitive schemes.
Cooperative Task Offloading and Block Mining in Blockchain-Based Edge Computing With Multi-Agent Deep Reinforcement Learning

The convergence of mobile edge computing (MEC) and blockchain is transforming the current computing services in mobile networks, by offering task offloading solutions with security enhancement empowered by blockchain mining. Nevertheless, these important enabling technologies have been studied separately in most existing works. This article proposes a novel cooperative task offloading and block mining (TOBM) scheme for a blockchain-based MEC system where each edge device not only handles data tasks but also deals with block mining for improving the system utility. To address the latency issues caused by the blockchain operation in MEC, we develop a new Proof-of-Reputation consensus mechanism based on a lightweight block verification strategy. A multi-objective function is then formulated to maximize the system utility of the blockchain-based MEC system, by jointly optimizing offloading decision, channel selection, transmit power allocation, and computational resource allocation. We propose a novel distributed deep reinforcement learning-based approach by using a multi-agent deep deterministic policy gradient algorithm. We then develop a game-theoretic solution to model the offloading and mining competition among edge devices as a potential game, and prove the existence of a pure Nash equilibrium. Simulation results demonstrate the significant system utility improvements of our proposed scheme over baseline approaches.
Conceptual and Syntactical Cross-modal Alignment with Cross-level Consistency for Image-Text Matching

Image-Text Matching (ITM) is a fundamental and emerging task, which plays a key role in cross-modal understanding. It remains a challenge because prior works mainly focus on learning fine-grained (i.e. coarse and/or phrase) correspondence, without considering the syntactical correspondence. In theory, a sentence is not only a set of words or phrases but also a syntactic structure, consisting of a set of basic syntactic tuples (i.e.(attribute) object - predicate - (attribute) subject). Inspired by this, we propose a Conceptual and Syntactical Cross-modal Alignment with Cross-level Consistency (CSCC) for Image-text Matching by simultaneously exploring the multiple-level cross-modal alignments across the concept and syntactic with a consistency constraint. Specifically, a conceptual-level cross-modal alignment is introduced for exploring the fine-grained correspondence, while a syntactical-level cross-modal alignment is proposed to explicitly learn a high-level syntactic similarity function. Moreover, an empirical cross-level consistent attention loss is introduced to maintain the consistency between cross-modal attentions obtained from the above two cross-modal alignments. To justify our method, comprehensive experiments are conducted on two public benchmark datasets, i.e. MS-COCO (1K and 5K) and Flickr30K, which show that our CSCC outperforms state-of-the-art methods with fairly competitive improvements.
Swin-Depth: Using Transformers and Multi-Scale Fusion for Monocular-Based Depth Estimation

Depth estimation using monocular sensors is an important and basic task in computer vision. It has a wide range of applications in robot navigation, autonomous driving, etc., and has received extensive attention from researchers in recent years. For a long time before, monocular depth estimation was based on convolutional neural networks, but its inherent convolution operation showed limitations in modeling large-scale dependence. Using Transformers instead of convolutional neural networks to perform monocular depth estimation provides a good idea, but there is a problem that the calculation complexity is too high and the number of parameters is too large. In response to these problems, we proposed Swin-Depth, which is a Transformer-based monocular depth estimation method that uses hierarchical representation learning with linear complexity for images. In addition, there is an attention module based on multi-scale fusion in Swin-Depth to strengthen the network’s ability to capture global information. Our proposed method effectively reduces the excessive parameters in the monocular depth estimation using transformer, and a large number of research experiments show that Swin-Depth has achieved state-of-the-art in challenging datasets of indoor and outdoor scenes.
Deep structural information fusion for 3D object detection on LiDAR–camera system

3D object detection on LiDAR–camera system is a challenging task, for 3D LiDAR point and 2D RGB image have different data representation. In this paper, We consider that the geometrical consistency in the local 3D and 2D regions is helpful for the regression task in 3D object detection, and propose 3D–2D consistent feature. It is based on hand-crafted 3D and 2D descriptors, generates primary structure feature, and has stable performance in outdoor scenes. Considering that material feature can be used to distinguish different objects, material coefficients ratio (MCR) is proposed to generate primary semantic feature, benefiting the classification task in 3D object detection. It is based on Lambertian model. To take advantage of both 3D–2D consistent feature and MCR, we propose deep 3D–2D structural information fusion (SIF) for 3D object detection. It provides attentional structural voxel feature, used as the input of LiDAR voxel based 3D object detectors. SIF is a light, effective, and explainable module. In the outdoor 3D object detection dataset, extensive experiments demonstrate that SIF improves the performance for both LiDAR voxel based single stage and multi-stage 3D detectors.
Texture Is Important in Improving the Accuracy of Mapping Photovoltaic Power Plants: A Case Study of Ningxia Autonomous Region, China

Photovoltaic (PV) technology is becoming more popular due to climate change because it allows for replacing fossil-fuel power generation to reduce greenhouse gas emissions. Consequently, many countries have been attempting to generate electricity through PV power plants over the last decade. Monitoring PV power plants through satellite imagery, machine learning models, and cloud-based computing systems that may ensure rapid and precise locating with current status on a regional basis are crucial for environmental impact assessment and policy formulation. The effect of fusion of the spectral, textural with different neighbor sizes, and topographic features that may improve machine learning accuracy has not been evaluated yet in PV power plants’ mapping. This study mapped PV power plants using a random forest (RF) model on the Google Earth Engine (GEE) platform. We combined textural features calculated from the Grey Level Co-occurrence Matrix (GLCM), reflectance, thermal spectral features, and Normalized Difference Vegetation Index (NDVI), Normalized Difference Built-up Index (NDBI), and Modified Normalized Difference Water Index (MNDWI) from Landsat-8 imagery and elevation, slope, and aspect from Shuttle Radar Topography Mission (SRTM) as input variables. We found that the textural features from GLCM prominent enhance the accuracy of the random forest model in identifying PV power plants where a neighbor size of 30 pixels showed the best model performance. The addition of texture features can improve model accuracy from a Kappa statistic of 0.904 ± 0.05 to 0.938 ± 0.04 and overall accuracy of 97.45 ± 0.14% to 98.32 ± 0.11%. The topographic and thermal features contribute a slight improvement in modeling. This study extends the knowledge of the effect of various variables in identifying PV power plants from remote sensing data. The texture characteristics of PV power plants at different spatial resolutions deserve attention. The findings of our study have great significance for collecting the geographic information of PV power plants and evaluating their environmental impact.
Improved U-Net based on contour prediction for efficient segmentation of rectal cancer

Segmentation of rectal cancerous regions using 2D Magnetic Resonance Imaging (MRI) images is a critical step in radiation therapy. The shape of rectal cancer has significant variations and the shape of some surrounding organs is similar to that of rectal cancer; these conditions significantly affect the segmentation accuracy of rectal cancer and lead to incorrect segmentation. Therefore, automatic segmentation of rectal cancer is urgently needed, and it is a great challenge. For this task, the existing deep learning-based approaches have two shortcomings: 1) The U-Net network plays an important role in the field of medical segmentation. However, the designs of encoders and decoders in traditional U-Net networks are relatively simple and cannot extract good features, resulting in incorrect segmentation results. 2) Conventional neural networks extract high-level features that often do not include sufficient high-resolution contour information, resulting in ambiguity in contour segmentation. In this paper, we propose an improved U-Net network based on contour prediction, aiming at effective segmentation of rectal cancer.We designed a new U-Net network by improving the traditional U-Net network. We made four improvements: 1) We replaced the encoders with the SENet network. 2) A global pooling layer was added after the last encoder. 3) We added the Spatial and Channel Squeeze & Excitation (SCSE) attention mechanism module to each decoder. 4) We concatenated the output results of each decoder. In addition, the model implemented content segmentation and contour segmentation for rectal cancer in parallel, so that both the content and contour information was learned by the network to enhance the segmentation accuracy.Our data were obtained from the Shanxi Provincial Cancer Hospital and included 3773 2D MRI rectal cancer images. The proposed method achieved an Mean Intersection over Union of 0.894 (MIoU) on the test set. Compared with state-of-the-art methods, our method had the best performance on the test set, and its MIoU metric was 0.123 higher than that of the second-best model. At the same time, the effectiveness of the improvements to our method was demonstrated through ablation experiments.Our method can help radiologists to segment effectively, save their time and energy, and enable them to focus on cases that are not easily segmented because of the complex shape of rectal cancer.
Context-Aware Recommendation Systems in the IoT Environment (IoT-CARS)–A Comprehensive Overview

An essential goal of recommendation systems is to provide users with accurate and personalized recommendations that meet their preferences. With the rapid growth of IoT-connected sensors, the availability of contextual information has increased, and this has necessitated the fast development of Context-Aware Recommendation Systems (CARS). Context-Aware recommenders are different from traditional recommenders because of their ability to predict the ratings of target users/items by exploiting the knowledge of contextual information. Context-aware recommenders define the context as any information that characterizes the situations of items and users at a particular interaction. They are essential for some contexts where prediction can be more precise in generating specific personalized recommendations. This paper provides a comprehensive review of context-based recommendation systems in IoT environments, namely IoT-CARS, and sheds light on their requirements, characteristics, and applications. We characterize context-aware recommenders in terms of the different IoT contexts and how these contexts are modeled. We also highlight the used metrics to evaluate the performance of various context-based recommenders.
Fine-Grained Recognition for Oriented Ship Against Complex Scenes in Optical Remote Sensing Images

Remote sensing ship recognition is widely used in civil and military applications, such as national defense construction, fishery management, and navigation supervision. Unfortunately, limited by the lack of public datasets for fine-grained ship recognition, current studies mainly focus on ship detection or coarse-grained ship recognition, whereas fine-grained recognition task is left out. The main challenges for fine-grained ship recognition include: 1) complex scenes and 2) ship characteristics of arbitrary orientation, dense distribution, and huge scale and appearance variation. Aiming at the challenges above, we propose a novel efficient information reuse network (EIRNet) and establish a public 20-class Dataset for Oriented Ship Recognition (DOSR). In our EIRNet, considering recognition robustness to multiscale ships, a dense feature fusion network (DFF-Net) with two fusion directions is designed to maximize the utilization of multilayer information and reduce information redundancy. Then, the fused feature maps are refined by a dual-mask attention module (DMAM) to improve performance in dense and clutter scenes by enhancing the distinction between ships and suppressing clutter. Furthermore, Mask-RPN improves the efficiency of generating proposals by reusing the attention mask. Finally, we introduce a concept of upper level class to mine interclass relationships, which further improves the recognition accuracy. Extensive experiments demonstrate that our EIRNet achieves state-of-the-art performance on DOSR and another popular public dataset called HRSC2016.
Reparo: Publicly Verifiable Layer to Repair Blockchains

Although blockchains aim for immutability as their core feature, several instances have exposed the harms with perfect immutability. The permanence of illicit content inserted in Bitcoin poses a challenge to law enforcement agencies like Interpol, and millions of dollars were lost in buggy smart contracts in Ethereum. A line of research then spawned on redactable blockchains with the aim of solving the problem of redacting illicit contents from both permissioned and permissionless blockchains. However, all the existing proposals follow the build-new-chain approach for redactions, and cannot be integrated with existing running blockchains, such as Bitcoin and Ethereum. This work demonstrates that the traditional build-new-chain approach for blockchain redactions is not necessary. We present $$\mathsf {Reparo}$$ (In the Harry Potter universe, ‘ $$\mathsf {Reparo}$$ ’ is a spell that repairs objects), a publicly verifiable layer on top of any blockchain to perform repairs, ranging from fixing buggy contracts to removing illicit contents from the chain. We present an efficient instantiation of $$\mathsf {Reparo}$$ over Ethereum (with proof of work based consensus) for repairing smart contract bugs. In this protocol, any Ethereum user may propose a repair and a deliberation process ensues resulting in a decision that complies with the repair policy of the chain and is publicly verifiable. A repair operation (for instance, fixing a bug in a contract) is then performed according to the repair proposal and the state of Ethereum is updated accordingly. $$\mathsf {Reparo}$$ ’s advantages are multi-fold: (i) Since $$\mathsf {Reparo}$$ follows a layer design, it helps facilitate additional functionalities for Ethereum while maintaining the same provable security guarantees; (ii) $$\mathsf {Reparo}$$ can be easily tailored to different consensus requirements (like proof of stake), does not require heavy cryptographic machinery, and thus, can be integrated with other existing blockchains (such as Bitcoin, Cardano) as well. We evaluate $$\mathsf {Reparo}$$ with Ethereum mainnet and show that the cost of fixing several prominent smart contract bugs is almost negligible. For instance, the cost of repairing the prominent Parity Multisig wallet bug with $$\mathsf {Reparo}$$ is as low as $$0.00005\%$$ of the Ethers that can be retrieved after the fix.
Transformer Meets Part Model: Adaptive Part Division for Person Re-Identification

Part model is one of the key factors to high performance person re-identification (ReID) task. In recent studies, there are mainly two streams for part model. The first one is to divide a person image into several fixed parts to obtain their local information, but it may cause performance degradation in case of misalignment. The other one is to explore external resources like pose estimation or human parsing to locate local parts, but it costs extra storage and computation. Inspired by recent successful transformers on spatial similarity modeling, we propose a novel Adaptive Part Division (APD) model to better extract local features. More specifically, APD mainly consists of two crucial modules: a Transformer-based Part Merge (TPM) module and a Part Mask Generation (PMG) module. In particular, TPM first adaptively assigns the patch tokens of the same semantic object to the identical part. Then, PMG takes these identical parts together and generates several non-overlapping masks for robust part division. We have conducted extensive evaluations on four popular benchmarks, i.e. Market-1501, CUHK03, DukeMTMC-ReID and MSMT17, and the experimental results show that our proposed method achieves the state-of-the-art performance.
Blockchain-based Framework for Reducing Fake or Vicious News Spread on Social Media/Messaging Platforms

With social media becoming the most frequently used mode of modern-day communications, the propagation of fake or vicious news through such modes of communication has emerged as a serious problem. The scope of the problem of fake or vicious news may range from rumour-mongering, with intent to defame someone, to manufacturing false opinions/trends impacting elections and stock exchanges to much more alarming and mala fide repercussions of inciting violence by bad actors, especially in sensitive law-and-order situations. Therefore, curbing fake or vicious news and identifying the source of such news to ensure strict accountability is the need of the hour. Researchers have been working in the area of using text analysis, labelling, artificial intelligence, and machine learning techniques for detecting fake news, but identifying the source or originator of such news for accountability is still a big challenge for which no concrete approach exists as of today. Also, there is another common problematic trend on social media whereby targeted vicious content goes viral to mobilize or instigate people with malicious intent to destabilize normalcy in society. In the proposed solution, we treat both problems of fake news and vicious news together. We propose a blockchain and keyed watermarking-based framework for social media/messaging platforms that will allow the integrity of the posted content as well as ensure accountability on the owner/user of the post. Intrinsic properties of blockchain-like transparency and immutability are advantageous for curbing fake or vicious news. After identification of fake or vicious news, its spread will be immediately curbed through backtracking as well as forward tracking. Also, observing transactions on the blockchain, the density and rate of forwarding of a particular original message going beyond a threshold can easily be checked, which could be identified as a possible malicious attempt to spread objectionable content. If the content is deemed dangerous or inappropriate, its spread will be curbed immediately. The use of the Raft consensus algorithm and bloXroute servers is proposed to enhance throughput and network scalability, respectively. Thus, the framework offers a proactive as well as reactive, practically feasible, and effective solution for curtailment of fake or vicious news on social media/messaging platforms. The proposed work is a framework for solving fake or vicious news spread problems on social media; the complete design specifications are beyond scope of the current work and will be addressed in the future.
Contrastive Pre-Training of GNNs on Heterogeneous Graphs

While graph neural networks (GNNs) emerge as the state-of-the-art representation learning methods on graphs, they often require a large amount of labeled data to achieve satisfactory performance, which is often expensive or unavailable. To relieve the label scarcity issue, some pre-training strategies have been devised for GNNs, to learn transferable knowledge from the universal structural properties of the graph. However, existing pre-training strategies are only designed for homogeneous graphs, in which each node and edge belongs to the same type. In contrast, a heterogeneous graph embodies rich semantics, as multiple types of nodes interact with each other via different kinds of edges, which are neglected by existing strategies. In this paper, we propose a novel Contrastive Pre-Training strategy of GNNs on Heterogeneous Graphs (CPT-HG), to capture both the semantic and structural properties in a self-supervised manner. Specifically, we design semantic-aware pre-training tasks at both the relation- and subgraph-levels, and further enhance their representativeness by employing contrastive learning. We conduct extensive experiments on three real-world heterogeneous graphs, and promising results demonstrate the superior ability of our CPT-HG to transfer knowledge to various downstream tasks via pre-training.
Multi-triage: A multi-task learning framework for bug triage

Assigning developers and allocating issue types are two important tasks in the bug triage process. Existing approaches tackle these two tasks separately, which is time-consuming due to repetition of effort and negating the values of correlated information between tasks. In this paper, a multi-triage model is proposed that resolves both tasks simultaneously via multi-task learning (MTL). First, both tasks can be regarded as a classification problem, based on historical issue reports. Second, performances on both tasks can be improved by jointly interpreting the representations of the issue report information. To do so, a text encoder and abstract syntax tree (AST) encoder are used to extract the feature representation of bug descriptions and code snippets accordingly. Finally, due to the disproportionate ratio of class labels in training datasets, the contextual data augmentation approach is introduced to generate syntactic issue reports to balance the class labels. Experiments were conducted on eleven open-source projects to demonstrate the effectiveness of this model compared with state-of-the-art methods.