Reviewerly/exHarmony huggingface dataset:
ML-based group method of data handling: an improvement on the conventional GMDH

Abstract Machine learning (ML) has been recognized as a feasible and reliable technique for the modeling of multi-parametric datasets. In real applications, there are different relationships with various complexities between sets of inputs and their corresponding outputs. As a result, various models have been developed with different levels of complexity in the input–output relationships. The group method of data handling (GMDH) employs a family of inductive algorithms for computer-based mathematical modeling grounded on a combination of quadratic and higher neurons in a certain number of variable layers. In this method, a vector of input features is mapped to the expected response by creating a multistage nonlinear pattern. Usually, each neuron of the GMDH is considered a quadratic partial function. In this paper, the basic structure of the GMDH technique is adapted by changing the partial functions to enhance the complexity modeling ability. To accomplish this, popular ML models that have shown reasonable function approximation performance, such as support vector regression and random forest, are used, and the basic polynomial functions in the GMDH are replaced by these ML models. The regression feasibility and validity of the ML-based GMDH models are confirmed by computer simulation.
Encryption Based Image Watermarking Algorithm in 2DWT-DCT Domains

This paper proposes an encryption-based image watermarking scheme using a combination of second-level discrete wavelet transform (2DWT) and discrete cosine transform (DCT) with an auto extraction feature. The 2DWT has been selected based on the analysis of the trade-off between imperceptibility of the watermark and embedding capacity at various levels of decomposition. DCT operation is applied to the selected area to gather the image coefficients into a single vector using a zig-zig operation. We have utilized the same random bit sequence as the watermark and seed for the embedding zone coefficient. The quality of the reconstructed image was measured according to bit correction rate, peak signal-to-noise ratio (PSNR), and similarity index. Experimental results demonstrated that the proposed scheme is highly robust under different types of image-processing attacks. Several image attacks, e.g., JPEG compression, filtering, noise addition, cropping, sharpening, and bit-plane removal, were examined on watermarked images, and the results of our proposed method outstripped existing methods, especially in terms of the bit correction ratio (100%), which is a measure of bit restoration. The results were also highly satisfactory in terms of the quality of the reconstructed image, which demonstrated high imperceptibility in terms of peak signal-to-noise ratio (PSNR ≥ 40 dB) and structural similarity (SSIM ≥ 0.9) under different image attacks.
Expressivity of quantum neural networks

In this work, we address the question whether a sufficiently deep quantum neural network can approximate a target function as accurate as possible. We start with typical physical situations that the target functions are physical observables, and then we extend our discussion to situations that the learning targets are not directly physical observables, but can be expressed as physical observables in an enlarged Hilbert space with multiple replicas, such as the Loschmidt echo and the Rényi entropy. The main finding is that an accurate approximation is possible only when all the input wave functions in the dataset do not span the entire Hilbert space that the quantum circuit acts on, and more precisely, the Hilbert space dimension of the former has to be less than half of the Hilbert space dimension of the latter. In some cases, this requirement can be satisfied automatically because of the intrinsic properties of the dataset, for instance, when the input wave function has to be symmetric between different replicas. And if this requirement cannot be satisfied by the dataset, we show that the expressivity capabilities can be restored by adding one ancillary qubit at which the wave function is always fixed at input. Our studies point toward establishing a quantum neural network analogy of the universal approximation theorem that lays the foundation for expressivity of classical neural networks.Received 24 April 2021Accepted 9 August 2021DOI:https://doi.org/10.1103/PhysRevResearch.3.L032049Published by the American Physical Society under the terms of the Creative Commons Attribution 4.0 International license. Further distribution of this work must maintain attribution to the author(s) and the published article's title, journal citation, and DOI.Published by the American Physical SocietyPhysics Subject Headings (PhySH)Research AreasMachine learningGeneral PhysicsQuantum Information
Towards Balanced Learning for Instance Recognition

Instance recognition is rapidly advanced along with the developments of deep convolutional neural networks. Compared to the model architectures the training process, which is also crucial to the success of detectors, has received relatively less attention. In this work, we carefully revisit the standard training practice of detectors, and find that the detection performance is often limited by the imbalance during the training process, which generally consists in three levels—sample level, feature level, and objective level. To mitigate the adverse effects caused thereby, we propose Libra R-CNN, a simple yet effective framework towards balanced learning for instance recognition. It integrates IoU-balanced sampling, balanced feature pyramid, and objective re-weighting, respectively for reducing the imbalance at sample, feature, and objective level. Extensive experiments conducted on MS COCO, LVIS and Pascal VOC datasets prove the effectiveness of the overall balanced design.
Information Leaks in Federated Learning

With the surge in data collection and analytics, concerns are raised with regards to the privacy of the individuals represented by the data.In settings where the data is distributed over several data holders, federated learning offers an alternative to learn from the data without the need to centralize it in the first place.This is achieved by exchanging only model parameters learned locally at each data holder.This greatly limits the amount of data to be transferred, reduces the impact of data breaches, and helps to preserve the individual's privacy.Federated learning thus becomes a viable alternative in IoT and Edge Computing settings, especially if the data collected is sensitive.However, risks for data or information leaks still persist, if information can be inferred from the models exchanged.This can e.g.be in the form of membership inference attacks.In this paper, we investigate how successful such attacks are in the setting of sequential federated learning.The cyclic nature of model learning and exchange might enable attackers with more information to observe the dynamics of the learning process, and thus perform a more powerful attack.
Reinforcement learning-enhanced protocols for coherent population-transfer in three-level quantum systems

Abstract We deploy a combination of reinforcement learning-based approaches and more traditional optimization techniques to identify optimal protocols for population transfer in a multi-level system. We constrain our strategy to the case of fixed coupling rates but time-varying detunings, a situation that would simplify considerably the implementation of population transfer in relevant experimental platforms, such as semiconducting and superconducting ones. Our approach is able to explore the space of possible control protocols to reveal the existence of efficient protocols that, remarkably, differ from (and can be superior to) standard Raman, stimulated Raman adiabatic passage or other adiabatic schemes. The new protocols that we identify are robust against both energy losses and dephasing.
Economic and environmental assessment of an unreliable supply chain management

The retailers within a supply chain management are the most important player as it has direct contact with the customers. Customers are the source of the money and information for the entire supply chain management. The equilibrium condition of the forward and backward supply chain is tight until it is reliable. But sometimes the retailers hide information from the manufacturer and this unreliability starts to propagate to the entire supply chain management. Blockchain technology is one of the secure decentralized systems to prevent unreliability. This study eliminates the unreliability of the system. A mathematical model is derived where the vendor-managed inventory policy is used by the single manufacturer to deal with multiple unreliable retailers. The entire process of manufacturing emits carbon. The carbon emission is controlled by the cap and trade policy and by paying the carbon tax. The results are found by the classical optimization technique and the numerical studies open up a way to the blockchain to deal with unreliable retailers. The decentralized system is a probable choice to prevent unreliability and thus the blockchain.
MultiDoc2Dial: Modeling Dialogues Grounded in Multiple Documents

We propose MultiDoc2Dial, a new task and dataset on modeling goal-oriented dialogues grounded in multiple documents. Most previous works treat document-grounded dialogue modeling as a machine reading comprehension task based on a single given document or passage. In this work, we aim to address more realistic scenarios where a goal-oriented information-seeking conversation involves multiple topics, and hence is grounded on different documents. To facilitate such a task, we introduce a new dataset that contains dialogues grounded in multiple documents from four different domains. We also explore modeling the dialogue-based and document-based context in the dataset. We present strong baseline approaches and various experimental results, aiming to support further research efforts on such a task.
Rotation Equivariant Feature Image Pyramid Network for Object Detection in Optical Remote Sensing Imagery

Detection of objects is extremely important in various aerial vision-based applications. Over the last few years, the methods based on convolution neural networks have made substantial progress. However, because of the large variety of object scales, densities, and arbitrary orientations, the current detectors struggle with the extraction of semantically strong features for small-scale objects by a predefined convolution kernel. To address this problem, we propose the rotation equivariant feature image pyramid network (REFIPN), an image pyramid network based on rotation equivariance convolution. The proposed model adopts single-shot detector in parallel with a lightweight image pyramid module to extract representative features and generate regions of interest in an optimization approach. The proposed network extracts feature in a wide range of scales and orientations by using novel convolution filters. These features are used to generate vector fields and determine the weight and angle of the highest-scoring orientation for all spatial locations on an image. By this approach, the performance for small-sized object detection is enhanced without sacrificing the performance for large-sized object detection. The performance of the proposed model is validated on two commonly used aerial benchmarks and the results show our proposed model can achieve state-of-the-art performance with satisfactory efficiency.
Accurate 3D Shape Reconstruction from Single Structured-Light Image via Fringe-to-Fringe Network

Accurate three-dimensional (3D) shape reconstruction of objects from a single image is a challenging task, yet it is highly demanded by numerous applications. This paper presents a novel 3D shape reconstruction technique integrating a high-accuracy structured-light method with a deep neural network learning scheme. The proposed approach employs a convolutional neural network (CNN) to transform a color structured-light fringe image into multiple triple-frequency phase-shifted grayscale fringe images, from which the 3D shape can be accurately reconstructed. The robustness of the proposed technique is verified, and it can be a promising 3D imaging tool in future scientific and industrial applications.
UniPELT: A Unified Framework for Parameter-Efficient Language Model Tuning

Recent parameter-efficient language model tuning (PELT) methods manage to match the performance of fine-tuning with much fewer trainable parameters and perform especially well when training data is limited.However, different PELT methods may perform rather differently on the same task, making it nontrivial to select the most appropriate method for a specific task, especially considering the fast-growing number of new PELT methods and tasks.In light of model diversity and the difficulty of model selection, we propose a unified framework, UNIPELT, which incorporates different PELT methods as submodules and learns to activate the ones that best suit the current data or task setup via gating mechanism.On the GLUE benchmark, UNIPELT consistently achieves 1~4% gains compared to the best individual PELT method that it incorporates and outperforms fine-tuning under different setups.Moreover, UNIPELT generally surpasses the upper bound that takes the best performance of all its submodules used individually on each task, indicating that a mixture of multiple PELT methods may be inherently more effective than single methods.
DYLE: Dynamic Latent Extraction for Abstractive Long-Input Summarization

Transformer-based models have achieved state-of-the-art performance on short-input summarization.However, they still struggle with summarizing longer text.In this paper, we present DYLE, a novel dynamic latent extraction approach for abstractive long-input summarization.DYLE jointly trains an extractor and a generator and treats the extracted text snippets as the latent variable, allowing dynamic snippet-level attention weights during decoding.To provide adequate supervision, we propose simple yet effective heuristics for oracle extraction as well as a consistency loss term, which encourages the extractor to approximate the averaged dynamic weights predicted by the generator.We evaluate our method on different long-document and long-dialogue summarization tasks: Gov-Report, QMSum, and arXiv.Experiment results show that DYLE outperforms all existing methods on GovReport and QMSum, with gains up to 6.1 ROUGE, while yielding strong results on arXiv.Further analysis shows that the proposed dynamic weights provide interpretability of our generation process.
Analysis of social media data for public emotion on the Wuhan lockdown event during the COVID-19 pandemic

With outbreaks of COVID-19 around the world, lockdown restrictions are routinely imposed to limit the spread of the virus. During periods of lockdown, social media has become the main channel for citizens to exchange information with others. Public emotions are being generated and shared rapidly online with citizens using internet platforms to reduce anxiety and stress, and stay connected while isolated. This study aims to explore the regularity of emotional evolution by examining public emotions expressed in online discussions about the Wuhan lockdown event in January 2020. Data related to the Wuhan lockdown was collected from Sina Weibo by web crawler. In this study, the Ortony, Clore, and Collins (OCC) model, Word2Vec, and Bi-directional Long Short-Term Memory model were employed to determine emotional types, train vectorization of words, and identify each text emotion for the training set. Latent Dirichlet Allocation models were also employed to mine the various topic categories, while topic emotional evolution was visualized. Seven types of emotions and four phases were categorized to describe emotional evolution on the Wuhan lockdown event. The study found that negative emotions such as blame and fear dominated in the early days, and public attitudes towards the lockdown gradually alleviated and reached a balance as the situation improved. Emotional expression about Wuhan lockdown event were significantly related to users’ gender, location, and whether or not their account was verified. There were statistically significant correlations between different emotions within the subtle emotional categories. In addition, the evolution of emotions presented a different path due to different topics. Multiple emotional categories were determined in our study, providing a detailed and explainable emotion analysis to explored emotional appeal of citizen. The public emotions were gradually easing related to the Wuhan lockdown event, there yet exists regional discrimination and post-traumatic stress disorder in this process, which would lead us to pay continuous attention to citizens lives and psychological status post-pandemic. In addition, this study provided an appropriate method and reference case for the government's public opinion control and emotional appeasement.
Double lockdown: The effects of digital exclusion on undocumented immigrants during the COVID-19 pandemic

The COVID-19 pandemic shifted many activities online. However, there is little research on the digital inclusion of undocumented immigrants and their experience of the pandemic in the United States. We conducted 32 interviews with undocumented Latino immigrants in the United States to examine how digital technologies mediated their experiences of the pandemic. We find that undocumented immigrants (1) face barriers to telehealth services, (2) are at high risk of COVID-19 misinformation, (3) experience difficulties in assessing privacy risks, and (4) experienced heterogeneous outcomes of technology use during the pandemic. Our analysis shows that digital technologies both supported and further marginalized undocumented immigrants during the pandemic. Future research on the digital inclusion of vulnerable populations should pay particular attention to the interaction between their underlying vulnerabilities, on one hand, and attitudes, uses, and outcomes associated with technology, on the other.
CBML

Session-based recommendation is to predict an anonymous user's next action based on the user's historical actions in the current session. However, the cold-start problem of limited number of actions at the beginning of an anonymous session makes it difficult to model the user's behavior, i.e., hard to capture the user's various and dynamic preferences within the session. This severely affects the accuracy of session-based recommendation. Although some existing meta-learning based approaches have alleviated the cold-start problem by borrowing preferences from other users, they are still weak in modeling the behavior of the current user. To tackle the challenge, we propose a novel cluster-based meta-learning model for session-based recommendation. Specially, we adopt a soft-clustering method and design a parameter gate to better transfer shared knowledge across similar sessions and preserve the characteristics of the session itself. Besides, we apply two self-attention blocks to capture the transition patterns of sessions in both item and feature aspects. Finally, comprehensive experiments are conducted on two real-world datasets and demonstrate the superior performance of CBML over existing approaches.
Design and implementation of transaction privacy by virtue of ownership and traceability in blockchain based supply chain

Blockchain was at the top of the 2016 Gartner hype cycle and has been integrated into business profiles by numerous start-ups. Since the emergence of blockchain through Bitcoin, studies have been conducted to increase blockchain applications for nonfinancial uses. A supply chain is a sector where blockchain is anticipated to have crucial applications. In a traditional supply chain, maintaining traceability and ownership remains a serious issue. In the supply chain, blockchain can increase trust, improve traceability, and eliminate the middle man. It makes the supply chain more transparent though, raising the privacy issue. In this paper, a new approach for transaction privacy is proposed by considering ownership and traceability. The proposed system retains the advantages of blockchain and centralised database server. Its novelty lies in achieving privacy by generating symmetric keys, employing product codes and current timestamps, and it uses asymmetric key elliptic curve cryptography for transaction validation and user identification. The proposed system allows product owners to trace the product and enables its transfer. It protects the supply chain from counterfeit products. The Hyperledger Sawtooth blockchain was used for experiments. Security and privacy analysis show that the proposed system can afford privacy without impinging on traceability and ownership. The results estimate that privacy incorporation introduces an overhead of 4.4%. In the experiment, the performance of the proposed system bettered the results of the existing techniques such as POMS and b_verify.
Universal Compiling and (No-)Free-Lunch Theorems for Continuous-Variable Quantum Learning

Quantum compiling, where a parameterized quantum circuit is trained to learn a target unitary, is an important primitive for quantum computing that can be used as a subroutine to obtain optimal circuits or as a tomographic tool to study the dynamics of an experimental system. While much attention has been paid to quantum compiling on discrete variable hardware, less has been paid to compiling in the continuous variable paradigm. Here we motivate several, closely related, short depth continuous variable algorithms for quantum compilation. We analyse the trainability of our proposed cost functions and numerically demonstrate our algorithms by learning arbitrary Gaussian operations and Kerr non-linearities. We further make connections between this framework and quantum learning theory in the continuous variable setting by deriving No-Free-Lunch theorems. These generalization bounds demonstrate a linear resource reduction for learning Gaussian unitaries using entangled coherent-Fock states and an exponential resource reduction for learning arbitrary unitaries using Two-Mode-Squeezed states.
DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharing

This paper presents a new pre-trained language model, DeBERTaV3, which improves the original DeBERTa model by replacing mask language modeling (MLM) with replaced token detection (RTD), a more sample-efficient pre-training task. Our analysis shows that vanilla embedding sharing in ELECTRA hurts training efficiency and model performance. This is because the training losses of the discriminator and the generator pull token embeddings in different directions, creating the "tug-of-war" dynamics. We thus propose a new gradient-disentangled embedding sharing method that avoids the tug-of-war dynamics, improving both training efficiency and the quality of the pre-trained model. We have pre-trained DeBERTaV3 using the same settings as DeBERTa to demonstrate its exceptional performance on a wide range of downstream natural language understanding (NLU) tasks. Taking the GLUE benchmark with eight tasks as an example, the DeBERTaV3 Large model achieves a 91.37% average score, which is 1.37% over DeBERTa and 1.91% over ELECTRA, setting a new state-of-the-art (SOTA) among the models with a similar structure. Furthermore, we have pre-trained a multi-lingual model mDeBERTa and observed a larger improvement over strong baselines compared to English models. For example, the mDeBERTa Base achieves a 79.8% zero-shot cross-lingual accuracy on XNLI and a 3.6% improvement over XLM-R Base, creating a new SOTA on this benchmark. We have made our pre-trained models and inference code publicly available at https://github.com/microsoft/DeBERTa.