Reviewerly/exHarmony huggingface dataset:
Highly Accurate CNN Inference Using Approximate Activation Functions over Homomorphic Encryption

In the big data era, cloud-based machine learning as a service (MLaaS) has attracted considerable attention. However, when handling sensitive data, such as financial and medical data, a privacy issue emerges, because the cloud server can access clients' raw data. A common method of handling sensitive data in the cloud uses homomorphic encryption, which allows computation over encrypted data without decryption. Previous research adopted a low-degree polynomial mapping function, such as the square function, for data classification. However, this technique results in low classification accuracy. This study seeks to improve the classification accuracy for inference processing in a convolutional neural network (CNN) while using homomorphic encryption. We apply various orders of the polynomial approximations of Google's Swish and ReLU activation functions. We also adopt batch normalization to normalize the inputs for the approximated activation functions to fit the input range to minimize the error. We implemented CNN inference labeling over homomorphic encryption using the Microsoft's Simple Encrypted Arithmetic Library (SEAL) for the Cheon-Kim-Kim-Song (CKKS) scheme. The experimental evaluations confirmed classification accuracies of 99.29% and 81.06% for MNIST and CIFAR-10, respectively, which entails 0.11% and 4.69% improvements, respectively, over previous methods.
Alleviating Modality Bias Training for Infrared-Visible Person Re-Identification

The task of infrared-visible person re-identification (IV-reID) is to recognize people across two modalities ( <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">i.e.</i> , RGB and IR). Existing cutting-edge approaches normally use a pair of images that have the same IDs ( <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">i.e.</i> , ID-tied cross-modality image pairs) and input them into an ImageNet-trained ResNet50. The ResNet50 backbone model can learn shared features across modalities to tolerate modality discrepancies between RGB and IR. This work will unveil a Modality Bias Training (MBT) problem that is less discussed in IV-reID, which will demonstrate that MBT significantly compromises the performance of IV-reID. Due to MBT, IR information can be overwhelmed by RGB information during training when the ResNet50 model is pretrained based on a large amount of RGB images from ImageNet. Thus, the trained models are more inclined to RGB information. Accordingly, the cross-modality generalization ability of the model is also compromised. To tackle this issue, we present a Dual-level Learning Strategy (DLS) that 1) enforces the focus of the network on ID-exclusive (rather than ID-tied) labels of cross-modality image pairs to mitigate the problem of MBT and 2) introduces third modality data that contain both RGB and IR information to further prevent the information from the IR modality from being overwhelmed during training. Our third modality images are generated by a generative adversarial network. A dynamic ID-exclusive Smooth (dIDeS) label is proposed for the generated third modality data. In experiments, comprehensive experiments are carried out to demonstrate the success of DLS in tackling the MBT issue exposed in IV-reID.
A security framework for Ethereum smart contracts

The use of blockchain and smart contracts have not stopped growing in recent years. Like all software that begins to expand its use, it is also beginning to be targeted by hackers who will try to exploit vulnerabilities in both the underlying technology and the smart contract code itself. While many tools already exist for analyzing vulnerabilities in smart contracts, the heterogeneity and variety of approaches and differences in providing the analysis data makes the learning curve for the smart contract developer steep. In this article the authors present ESAF (Ethereum Security Analysis Framework), a framework for analysis of smart contracts that aims to unify and facilitate the task of analyzing smart contract vulnerabilities which can be used as a persistent security monitoring tool for a set of target contracts as well as a classic vulnerability analysis tool among other uses.
Comparing and Combining Virtual Hand and Virtual Ray Pointer Interactions for Data Manipulation in Immersive Analytics

In this work, we evaluate two standard interaction techniques for Immersive Analytics environments: virtual hands, with actions such as grabbing and stretching, and virtual ray pointers, with actions assigned to controller buttons. We also consider a third option: seamlessly integrating both modes and allowing the user to alternate between them without explicit mode switches. Easy-to-use interaction with data visualizations in Virtual Reality enables analysts to intuitively query or filter the data, in addition to the benefit of multiple perspectives and stereoscopic 3D display. While many VR-based Immersive Analytics systems employ one of the studied interaction modes, the effect of this choice is unknown. Considering that each has different advantages, we compared the three conditions through a controlled user study in the spatio-temporal data domain. We did not find significant differences between hands and ray-casting in task performance, workload, or interactivity patterns. Yet, 60% of the participants preferred the mixed mode and benefited from it by choosing the best alternative for each low-level task. This mode significantly reduced completion times by 23% for the most demanding task, at the cost of a 5% decrease in overall success rates.
Basket Recommendation with Multi-Intent Translation Graph Neural Network

The problem of basket recommendation (BR) is to recommend a ranking list of items to the current basket. Existing methods solve this problem by assuming the items within the same basket are correlated by one semantic relation, thus optimizing the item embeddings. However, this assumption breaks when there exist multiple intents within a basket. For example, assuming a basket contains {bread, cereal, yogurt, soap, detergent} where {bread, cereal, yogurt} are correlated through the "breakfast" intent, while {soap, detergent} are of "cleaning" intent, ignoring multiple relations among the items spoils the ability of the model to learn the embeddings. To resolve this issue, it is required to discover the intents within the basket. However, retrieving a multi-intent pattern is rather challenging, as intents are latent within the basket. Additionally, intents within the basket may also be correlated. Moreover, discovering a multi-intent pattern requires modeling high-order interactions, as the intents across different baskets are also correlated. To this end, we propose a new framework named as Multi-Intent Translation Graph Neural Network (MITGNN). MITGNN models T intents as tail entities translated from one corresponding basket embedding via T relation vectors. The relation vectors are learned through multi-head aggregators to handle user and item information. Additionally, MITGNN propagates multiple intents across our defined basket graph to learn the embeddings of users and items by aggregating neighbors. Extensive experiments on two real-world datasets prove the effectiveness of our proposed model on both transductive and inductive BR. The code <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> is available online.
A decision‐making methodology for risk‐informed earthquake early warning

Abstract To maximize the potential of earthquake early warning (EEW) as a credible tool for seismic resilience promotion, it should be combined with next‐generation decision‐support tools that use advanced risk‐based predictions and account for unavoidable malfunctions of the system (i.e., false alarms) to determine whether or not alerts/mitigation actions should be triggered. This work contributes to the required effort by developing a novel end‐user‐oriented approach for decision making related to very short‐term earthquake risk management. The proposed methodology unifies earthquake‐engineering‐related performance assessment procedures/metrics (for end‐user‐focused damage and consequence estimation) with multicriteria decision‐making tools (to consider end‐user preferences toward different types of risks). It is demonstrated for EEW in a hypothetical school building, to specifically investigate the optimal decisions (i.e., “trigger”/“do not trigger” alerts) for a range of earthquake scenarios with varying parameter uncertainties. In particular, it is found that the best action for a given ground‐shaking intensity can depend on stakeholder (end‐user) preferences.
Performance Validation of Yolo Variants for Object Detection

Object detection is a core part of an intelligent surveillance system and a fundamental algorithm in the field of identity identification, which is of great practical importance. Since the YOLO series algorithms have good results in terms of accuracy and speed, YOLO and each subsequent version have been surpassing. Thus, in this paper, it carries out experiments on three versions of popular YOLO models such as yolov3, yolov4, and yolov5 (yolov5l, yolov5m, yolov5s, yolov5x). The performance of the three versions of YOLO model is analyzed and summarized by training and predicting the public VOC dataset. Results showed that the yolov4 model is higher than the yolov3 model in terms of mAP values, but slightly lower in terms of speed, while the yolov5 series model is better than the yolov3 and yolov4 models both in terms of mAP values and speed.
Margin Preserving Self-Paced Contrastive Learning Towards Domain Adaptation for Medical Image Segmentation

To bridge the gap between the source and target domains in unsupervised domain adaptation (UDA), the most common strategy puts focus on matching the marginal distributions in the feature space through adversarial learning. However, such category-agnostic global alignment lacks of exploiting the class-level joint distributions, causing the aligned distribution less discriminative. To address this issue, we propose in this paper a novel margin preserving self-paced contrastive Learning (MPSCL) model for cross-modal medical image segmentation. Unlike the conventional construction of contrastive pairs in contrastive learning, the domain-adaptive category prototypes are utilized to constitute the positive and negative sample pairs. With the guidance of progressively refined semantic prototypes, a novel margin preserving contrastive loss is proposed to boost the discriminability of embedded representation space. To enhance the supervision for contrastive learning, more informative pseudo-labels are generated in target domain in a self-paced way, thus benefiting the category-aware distribution alignment for UDA. Furthermore, the domain-invariant representations are learned through joint contrastive learning between the two domains. Extensive experiments on cross-modal cardiac segmentation tasks demonstrate that MPSCL significantly improves semantic segmentation performance, and outperforms a wide variety of state-of-the-art methods by a large margin.
DeeperThings: Fully Distributed CNN Inference on Resource-Constrained Edge Devices

Abstract Performing inference of Convolutional Neural Networks (CNNs) on Internet of Things (IoT) edge devices ensures both privacy of input data and possible run time reductions when compared to a cloud solution. As most edge devices are memory- and compute-constrained, they cannot store and execute complex CNNs. Partitioning and distributing layer information across multiple edge devices to reduce the amount of computation and data on each device presents a solution to this problem. In this article, we propose DeeperThings, an approach that supports a full distribution of CNN inference tasks by partitioning fully-connected as well as both feature- and weight-intensive convolutional layers. Additionally, we jointly optimize memory, computation and communication demands. This is achieved using techniques to combine both feature and weight partitioning with a communication-aware layer fusion method, enabling holistic optimization across layers. For a given number of edge devices, the schemes are applied jointly using Integer Linear Programming (ILP) formulations to minimize data exchanged between devices, to optimize run times and to find the entire model’s minimal memory footprint. Experimental results from a real-world hardware setup running four different CNN models confirm that the scheme is able to evenly balance the memory footprint between devices. For six devices on 100 Mbit/s connections the integration of layer fusion additionally leads to a reduction of communication demands by up to 28.8%. This results in run time speed-up of the inference task by up to 1.52x compared to layer partitioning without fusing.
P2DEX: Privacy-Preserving Decentralized Cryptocurrency Exchange

Cryptocurrency exchange services are either trusted central entities that have been routinely hacked (losing over 8 billion USD), or decentralized services that make all orders public before they are settled. The latter allows market participants to “front run” each other, an illegal operation in most jurisdictions. We extend the “Insured MPC” approach of Baum et al. (FC 2020) to construct an efficient universally composable privacy preserving decentralized exchange where a set of servers run private cross-chain exchange order matching in an outsourced manner, while being financially incentivised to behave honestly. Our protocol allows for exchanging assets over multiple public ledgers, given that users have access to a ledger that supports standard public smart contracts. If parties behave honestly, the on-chain complexity of our construction is as low as that of performing the transactions necessary for a centralized exchange. In case malicious behavior is detected, users are automatically refunded by malicious servers at low cost. Thus, an actively corrupted majority can only mount a denial-of-service attack that makes exchanges fail, in which case the servers are publicly identified and punished, while honest clients do not to lose their funds. For the first time in this line of research, we report experimental results on the MPC building block, showing the approach is efficient enough to be used in practice.
Transformer-Based Direct Speech-To-Speech Translation with Transcoder

Traditional speech translation systems use a cascade manner that concatenates speech recognition (ASR), machine translation (MT), and text-to-speech (TTS) synthesis to translate speech from one language to another language in a step-by-step manner. Unfortunately, since those components are trained separately, MT often struggles to handle ASR errors, resulting in unnatural translation results. Recently, one work attempted to construct direct speech translation in a single model. The model used a multi-task scheme that learns to predict not only the target speech spectrograms directly but also the source and target phoneme transcription as auxiliary tasks. However, that work was only evaluated Spanish-English language pairs with similar syntax and word order. With syntactically distant language pairs, speech translation requires distant word order, and thus direct speech frame-to-frame alignments become difficult. Another direction was to construct a single deep-learning framework while keeping the step-by-step translation process. However, such studies focused only on speech-to-text translation. Furthermore, all of these works were based on a recurrent neural net-work (RNN) model. In this work, we propose a step-by-step scheme to a complete end-to-end speech-to-speech translation and propose a Transformer-based speech translation using Transcoder. We compare our proposed and multi-task model using syntactically similar and distant language pairs.
Joint Iterative Color Correction and Dehazing for Underwater Image Enhancement

The captured underwater images suffer from color cast and haze effect caused by absorption and scattering. These interdependent phenomena jointly degrade images, resulting in failure of autonomous machines to recognize image contents. Most existing learning-based methods for underwater image enhancement (UIE) treat the degraded process as a whole and ignore the interaction between color correction and dehazing. Thus, they often obtain unnatural results. To this end, we propose a novel joint network to optimize the results of color correction and dehazing in multiple iterations. Firstly, a novel triplet-based color correction module is proposed to obtain color-balanced images with identical distribution of color channels. By means of inherent constraints of the triplet structure, the information of channel with less distortion is utilized to recover the information of other channels. Secondly, a recurrent dehazing module is designed to alleviate haze effect in images, where the Gated Recurrent Unit (GRU) as the memory module optimizes the results in multiple cycles to deal with severe underwater distortions. Finally, an iterative mechanism is proposed to jointly optimize the color correction and dehazing. By learning transform coefficients from dehazing features, color features and basic features of raw images are progressively refined, which maintains color balanced during the dehazing process and further improves clarity of images. Experimental results show that our network is superior to the existing state-of-the-art approaches for UIE and provides improved performance for underwater object detection.
Blockchain-Based Digital Rights Management Scheme via Multiauthority Ciphertext-Policy Attribute-Based Encryption and Proxy Re-Encryption

In order to ensure the confidentiality of digital contents, improve the fairness of digital copyright transactions, and reduce the time and management overhead of digital copyright owners, we proposed a blockchain-based digital rights management scheme. First, we designed a new multiauthority ciphertext-policy attribute-based encryption (MA-CPABE) scheme and showed that the new MA-CPABE has the indistinguishability of plaintext under adaptively chosen plaintext attack (IND-CPA) security and good performance. By combining the MA-CPABE and proxy re-encryption, the rights owner can flexibly sell the copyright to different users with once encryption by an agent who cannot access any information related to digital content when changing the ciphertext access policy as required. By using the smart contract of Ethereum, a fair trade of the decryption keys between the rights owner and rights requester is implemented. In order to further improve fairness, another blockchain is used as a ledge to store information related to digital rights, which greatly reduces the storage overhead in public blockchain. Security analysis shows that our scheme can provide IND-CPA security, resist collusion attacks, and protect the user’s privacy. Performance analysis shows that our scheme can provide a wealth of features to meet the various needs of users. The simulation results show that our scheme is very efficient compared to other schemes.
Enhanced prototypical network for few-shot relation extraction

Most existing methods for relation extraction tasks depend heavily on large-scale annotated data; they cannot learn from existing knowledge and have low generalization ability. It is urgent for us to solve the above problems by further developing few-shot learning methods. Because of the limitations of the most commonly used CNN model which is not good at sequence labeling and capturing long-range dependencies, we proposed a novel model that integrates the transformer model into a prototypical network for more powerful relation-level feature extraction. The transformer connects tokens directly to adapt to long sequence learning without catastrophic forgetting and is able to gain more enhanced semantic information by learning from several representation subspaces in parallel for each word. We evaluate our method on three tasks, including in-domain, cross-domain and cross-sentence tasks. Our method achieves a trade-off between performance and computation and has an approximately 8% improvement in different settings over the state-of-the-art prototypical network. In addition, our experiments also show that our approach is competitive when considering cross-domain transfer and cross-sentence relation extraction in few-shot learning methods.
Supervised mineral exploration targeting and the challenges with the selection of deposit and non-deposit sites thereof

Selection of non-deposit sites is a challenging issue affecting the application of supervised algorithms for modeling mineral exploration targets. For this, equal number of deposit and non-deposit sites has been widely applied for training purposes. In this paper, we investigated the effect of changes in the number of non-deposit sites on the effectiveness of exploration targeting models while the number of deposit sites is constant. The results obtained demonstrated that exploration targeting models are affected by the ratio of non-deposit and deposit sites. Thus, balancing between the number of deposit and non-deposit sites is an efficient way to produce more-reliable exploration targets when supervised algorithms are applied for modeling. The idea of this research came from the fact that mineralization is a rare event, and therefore, in a region of interest number of non-deposit sites is much more than that of deposit events. To illustrate the procedure proposed, we used an exploration dataset of porphyry Cu mineralization in Chahargonbad area, SE Iran. A sequence application of self-organizing map and multilayer perceptron neural network algorithm was applied to better illustration of the changing effects of the number of non-deposit sites on the ensuing exploration targeting models.
Meta-Intelligence: Understanding, Control, and Interactivity between Creative, Analytical, Practical, and Wisdom-Based Approaches in Problem Solving

A deeper understanding of the processes leading to problem framing and behind finding solutions to problems should help explain variability in the quality of the solutions to those problems. Using Sternberg’s WICS model as the conceptual basis of problem solving, this article discusses the relations between creative, analytical, practical, and wisdom-based approaches as bases for solutions to problems. We use a construct of meta-intelligence to encompass understanding, control, and coordination between these constructs. We propose that constraints can act at each of three levels—individual, contextual, and interactive. Individual constraints include the metacomponents (executive processes) that underpin each of the four kinds of solutions. Contextual constraints direct which of the four approaches are preferred under what circumstances. Finally, interactive constraints involve individual and contextual constraints directly impacting each other’s actions. The model of meta-intelligence and its functioning helps to explain the variability in the ways that individuals frame problems and, as a consequence, in the solutions that are found. The model of meta-intelligence also helps explain why some solutions to problems are so much more comprehensive, and often better, than others.