Reviewerly/exHarmony huggingface dataset:
Rethinking Label-Wise Cross-Modal Retrieval from A Semantic Sharing Perspective

The main challenge of cross-modal retrieval is to learn the consistent embedding for heterogeneous modalities. To solve this problem, traditional label-wise cross-modal approaches usually constrain the inter-modal and intra-modal embedding consistency relying on the label ground-truths. However, the experiments reveal that different modal networks actually have various generalization capacities, thereby end-to-end joint training with consistency loss usually leads to sub-optimal uni-modal model, which in turn affects the learning of consistent embedding. Therefore, in this paper, we argue that what really needed for supervised cross-modal retrieval is a good shared classification model. In other words, we learn the consistent embedding by ensuring the classification performance of each modality on the shared model, without the consistency loss. Specifically, we consider a technique called Semantic Sharing, which directly trains the two modalities interactively by adopting a shared self-attention based classification model. We evaluate the proposed approach on three representative datasets. The results validate that the proposed semantic sharing can consistently boost the performance under NDCG metric.
Learning strange attractors with reservoir systems

Abstract This paper shows that the celebrated embedding theorem of Takens is a particular case of a much more general statement according to which, randomly generated linear state-space representations of generic observations of an invertible dynamical system carry in their wake an embedding of the phase space dynamics into the chosen Euclidean state space. This embedding coincides with a natural generalized synchronization that arises in this setup and that yields a topological conjugacy between the state-space dynamics driven by the generic observations of the dynamical system and the dynamical system itself. This result provides additional tools for the representation, learning, and analysis of chaotic attractors and sheds additional light on the reservoir computing phenomenon that appears in the context of recurrent neural networks.
Densely Connected Neural Networks for Nonlinear Regression

Densely connected convolutional networks (DenseNet) behave well in image processing. However, for regression tasks, convolutional DenseNet may lose essential information from independent input features. To tackle this issue, we propose a novel DenseNet regression model where convolution and pooling layers are replaced by fully connected layers and the original concatenation shortcuts are maintained to reuse the feature. To investigate the effects of depth and input dimensions of the proposed model, careful validations are performed by extensive numerical simulation. The results give an optimal depth (19) and recommend a limited input dimension (under 200). Furthermore, compared with the baseline models, including support vector regression, decision tree regression, and residual regression, our proposed model with the optimal depth performs best. Ultimately, DenseNet regression is applied to predict relative humidity, and the outcome shows a high correlation with observations, which indicates that our model could advance environmental data science.
Personas: A Market Segmentation Approach for Transportation Behavior Change

This paper proposes a market segmentation method applied in the field of transportation behavior change using GPS trajectories and socio-demographic data collected from the advanced demand management system “GoEzy” designed by Metropia. User attributes are extracted using several statistical methods such as dynamic time warping, density-based spatial clustering of applications with noise (DBSCAN), and signal processing method to infer users’ sensitivity to incentives, temporal, and spatial travel patterns. Ten personas were generated by K-means clustering, representing different types of people with various travel patterns and sensitivity to incentives. The experiment was conducted on 24 new users to test if the persona could be used as a tool to predict their willingness to change. The results showed that after creating personas for new users and providing them with new incentives, their modified departure time pattern according to the new incentives matched expectations from analysis of the 10 personas.
DBNet

The outbreak of the Coronavirus disease (COVID-19) pandemic has caused millions of deaths and put immense pressure on the health care system, especially the supply of mechanical ventilators. It is critical for clinicians to identify the patients in a timely manner whose status may deteriorate in the near future and therefore need mechanical ventilators. We propose a prediction model to estimate the probability of requiring mechanical ventilation for in-hospital patients at least 24 hours after their admission. Our model is a multi-modal encoder-decoder attention model that takes full usages of the electronic health record (EHR) database. The EHR database consists of heterogeneous data tables of different formats (diagnosis, drug administrations, medicine prescriptions, lab tests, vital sign observations, clinical procedures, and demographics). We leverage the attention mechanism to increase model performance and promote result interpretability. The attention mechanism also serves the role of the missing data imputation technique, which is often used on irregularly sampled temporal data. We name the model as DBNet as the model takes the database as input. DBNet is evaluated on a large cohort of COVID-19 patients and the result shows it outperforms the state-of-the-art baseline deep learning models in predicting the future requirement of mechanical ventilation. It also outperforms several machine learning models even with sophisticated feature engineering. Due to its ability to handle multiple tables and also longitudinal data, the DBNet can also is not limited to this single application and can be generalized to other healthcare prediction tasks.
Practical and Efficient in-Enclave Verification of Privacy Compliance

A trusted execution environment (TEE) such as Intel Software Guard Extension (SGX) runs attestation to prove to a data owner the integrity of the initial state of an enclave, including the program to operate on her data. For this purpose, the data-processing program is supposed to be open to the owner or a trusted third party, so its functionality can be evaluated before trust being established. In the real world, however, increasingly there are application scenarios in which the program itself needs to be protected (e.g., proprietary algorithm). So its compliance with privacy policies as expected by the data owner should be verified without exposing its code.To this end, this paper presents DEFLECTION, a new model for TEE-based delegated and flexible in-enclave code verification. Given that the conventional solutions do not work well under the resource-limited and TCB-frugal TEE, we come up with a new design inspired by Proof-Carrying Code. Our design strategically moves most of the workload to the code generator, which is responsible for producing easy-to-check code, while keeping the consumer simple. Also, the whole consumer can be made public and verified through a conventional attestation. We implemented this model on Intel SGX and demonstrate that it introduces a very small part of TCB. We also thoroughly evaluated its performance on micro-and macro-benchmarks and real-world applications, showing that the design only incurs a small overhead when enforcing several categories of security policies.
Table to text generation with accurate content copying

Generating fluent, coherent, and informative text from structured data is called table-to-text generation. Copying words from the table is a common method to solve the "out-of-vocabulary" problem, but it's difficult to achieve accurate copying. In order to overcome this problem, we invent an auto-regressive framework based on the transformer that combines a copying mechanism and language modeling to generate target texts. Firstly, to make the model better learn the semantic relevance between table and text, we apply a word transformation method, which incorporates the field and position information into the target text to acquire the position of where to copy. Then we propose two auxiliary learning objectives, namely table-text constraint loss and copy loss. Table-text constraint loss is used to effectively model table inputs, whereas copy loss is exploited to precisely copy word fragments from a table. Furthermore, we improve the text search strategy to reduce the probability of generating incoherent and repetitive sentences. The model is verified by experiments on two datasets and better results are obtained than the baseline model. On WIKIBIO, the result is improved from 45.47 to 46.87 on BLEU and from 41.54 to 42.28 on ROUGE. On ROTOWIRE, the result is increased by 4.29% on CO metric, and 1.93 points higher on BLEU.
Paraxial 3D shape measurement using parallel single-pixel imaging

Three-dimensional (3D) shape measurement with fringe projection technique and vertical scanning setup can alleviate the problem of shadow and occlusion. However, the shape-from-defocus based method suffers from limited sensitivity and low signal-to-noise ratio (SNR), whereas the projection-triangulation based is sensitive to the zero-phase detection. In this paper, we propose paraxial 3D shape measurement using parallel single-pixel imaging (PSI). The depth is encoded in the radial distance to the projector optical center, which is determined by the projection of light transport coefficients (LTCs). The third-order polynomial fitting is used for depth mapping and calibration. Experiments on 5 objects with different materials and textures are conducted, and standards are measured to test the accuracy. The results verified that the proposed method can achieve robust, dense reconstruction with depth accuracy at 20 μm while the root-mean-square error (RMSE) of plane fitting up to 43 μm.
Measurement-device-independent quantum key distribution with passive time-dependent source side channels

While measurement-device-independent (MDI) quantum key distribution (QKD) allows two trusted parties to establish a shared secret key from a distance without needing to trust a central detection node, their quantum sources must be well characterized, with side channels at the source posing the greatest loophole to the protocol's security. In this paper, we identify a time-dependent side channel in a common polarization-based QKD source that employs a Faraday mirror for phase stabilization. We apply a recently developed numerical proof technique [Phys. Rev. A 99, 062332 (2019)] to quantify the sensitivity of the secret key rate to the quantum optical model for the side channel, and to develop strategies to mitigate the information leakage. In particular, we find that the MDI three-state and BB84 protocols, while yielding the same key rate under ideal conditions, have diverging results in the presence of a side channel, with BB84 proving more advantageous. While we consider only a representative case example, we expect the strategies developed and key rate analysis method to be broadly applicable to other leaky sources.
Robotic grasping method of bolster spring based on image-based visual servoing with YOLOv3 object detection algorithm

In this paper, to address the problem of automatic positioning and grasping of bolster spring with complex geometric features and cluttered background, a novel image-based visual servoing (IBVS) control method based on the corner points features of YOLOv3 object detection bounding box is proposed and applied to the robotic grasping system of bolster spring. The YOLOv3 object detection model is used to detect and position the bolster spring and then based on the corner points features of the bolster spring bounding box, the IBVS controller is designed to drive the end effector of the robot to the desired pose. This method adopts the approach-align-grasp control strategy to achieve the grasping of the bolster spring, which is very robust to the calibration parameter errors of the camera and the robot model. With the help of Robotics and Machine Vision Toolboxes in Matlab, IBVS simulation analysis based on feature points is carried out. The results show that it is reasonable to use the corner points of the object detection bounding box as image features under the initial pose where the image plane is parallel to the object plane. The positioning and grasping experiments are conducted on the robotic grasping platform of bolster spring. The results show that this method is effective for automatic positioning and grasping of bolster spring with complex geometric features and cluttered background, and it has strong robustness to the change of illumination.
Blindfolded Attackers Still Threatening: Strict Black-Box Adversarial Attacks on Graphs

Adversarial attacks on graphs have attracted considerable research interests. Existing works assume the attacker is either (partly) aware of the victim model, or able to send queries to it. These assumptions are, however, unrealistic. To bridge the gap between theoretical graph attacks and real-world scenarios, in this work, we propose a novel and more realistic setting: strict black-box graph attack, in which the attacker has no knowledge about the victim model at all and is not allowed to send any queries. To design such an attack strategy, we first propose a generic graph filter to unify different families of graph-based models. The strength of attacks can then be quantified by the change in the graph filter before and after attack. By maximizing this change, we are able to find an effective attack strategy, regardless of the underlying model. To solve this optimization problem, we also propose a relaxation technique and approximation theories to reduce the difficulty as well as the computational expense. Experiments demonstrate that, even with no exposure to the model, the Macro-F1 drops 6.4% in node classification and 29.5% in graph classification, which is a significant result compared with existent works.
Streaming End-to-End ASR Based on Blockwise Non-Autoregressive Models

Non-autoregressive (NAR) modeling has gained more and more attention in speech processing. With recent state-of-the-art attention-based automatic speech recognition (ASR) structure, NAR can realize promising real-time factor (RTF) improvement with only small degradation of accuracy compared to the autoregressive (AR) models. However, the recognition inference needs to wait for the completion of a full speech utterance, which limits their applications on low latency scenarios. To address this issue, we propose a novel end-to-end streaming NAR speech recognition system by combining blockwise-attention and connectionist temporal classification with mask-predict (Mask-CTC) NAR. During inference, the input audio is separated into small blocks and then processed in a blockwise streaming way. To address the insertion and deletion error at the edge of the output of each block, we apply an overlapping decoding strategy with a dynamic mapping trick that can produce more coherent sentences. Experimental results show that the proposed method improves online ASR recognition in low latency conditions compared to vanilla Mask-CTC. Moreover, it can achieve a much faster inference speed compared to the AR attention-based models. All of our codes will be publicly available at https://github.com/espnet/espnet.
The Shapley Value of Tuples in Query Answering

We investigate the application of the Shapley value to quantifying the contribution of a tuple to a query answer. The Shapley value is a widely known numerical measure in cooperative game theory and in many applications of game theory for assessing the contribution of a player to a coalition game. It has been established already in the 1950s, and is theoretically justified by being the very single wealth-distribution measure that satisfies some natural axioms. While this value has been investigated in several areas, it received little attention in data management. We study this measure in the context of conjunctive and aggregate queries by defining corresponding coalition games. We provide algorithmic and complexity-theoretic results on the computation of Shapley-based contributions to query answers; and for the hard cases we present approximation algorithms.
Improving Streaming Transformer Based ASR Under a Framework of Self-Supervised Learning

Recently self-supervised learning has emerged as an effective approach to improve the performance of automatic speech recognition (ASR). Under such a framework, the neural network is usually pre-trained with massive unlabeled data and then fine-tuned with limited labeled data. However, the non-streaming architecture like bidirectional transformer is usually adopted by the neural network to achieve competitive results, which can not be used in streaming scenarios. In this paper, we mainly focus on improving the performance of streaming transformer under the self-supervised learning framework. Specifically, we propose a novel two-stage training method during fine-tuning, which combines knowledge distilling and self-training. The proposed training method achieves 16.3% relative word error rate (WER) reduction on Librispeech noisy test set. Finally, by only using the 100h clean subset of Librispeech as the labeled data and the rest (860h) as the unlabeled data, our streaming transformer based model obtains competitive WERs 3.5/8.7 on Librispeech clean/noisy test sets.
Secure User Authentication Leveraging Keystroke Dynamics via Wi-Fi Sensing

User authentication plays a critical role in access control of a man-machine system, where the knowledge factor, such as a personal identification number, constitutes the most widely used authentication element. However, knowledge factors are usually vulnerable to the spoofing attack. Recently, the inheritance factor, such as fingerprints, emerges as an efficient alternative resilient to malicious users, but it normally requires special equipment. To this end, in this article, we propose WiPass, a device-free authentication system only leveraging the pervasive Wi-Fi infrastructure to explore keystroke dynamics (manner and rhythm of keystrokes) captured by the channel state information to recognize legitimate users while rejecting spoofers. However, it remains an open challenge to characterize the behavioral features hidden in the human subtle motions, such as keystrokes. Therefore, we build a signal enhancement model using Ricean distribution to amplify user keystroke dynamics and a hybrid learning model for user authentication, which consists of two parts, i.e., convolutional neural network based feature extraction and support vector machine based classification. The former relies on visualizing the channel responses into time-series images to learn the behavioral features of keystrokes in energy and spectrum domains, whereas the latter exploits such behavioral features for user authentication. We prototype WiPass on the low-cost off-the-shelf Wi-Fi devices and verify its performance. Empirical results show that WiPass achieves on average 92.1% authentication accuracy, 5.9% false accept rate, and 6.3% false reject rate in three real environments.
Construction over operation? A study of the usage of digital humanities databases in China

Purpose Digital humanities database is one of the essential tools in digital humanities research area. Therefore, examining the usage of digital humanities database in academic papers is conducive to assessing the value of digital humanities database for scientific research activities and improving the construction of digital humanities infrastructure. Design/methodology/approach This paper constructs an evaluation system of digital humanities database from the perspective of academic influence and social influence, with mention frequency, usage motivation, platform access data, usage region and usage discipline as indicators and takes China Biographical Database Project as the empirical object to explore the usage of digital humanities database in China. Findings The data analysis result demonstrates that digital humanities databases are widely used and recognized in China. However, the problem of low actual usage remains. Originality/value This paper constructs the digital humanistic database's evaluation system and discusses applying the digital humanistic database in China, which provides a new perspective and method for the influence evaluation study of the digital humanistic database.
Efficient and Fully Secure Lattice-Based IBE with Equality Test

Identity-based encryption with equality test (IBEET), derived from public key encryption with equality test (PKEET), allows the equality test algorithm on two ciphertexts without decrypting the messages and simplifies the certificate management of PKEET. In response to the explosive growth of quantum computing, recently, some IBEET schemes based on the learning with errors (LWE) problem were proposed, which could survive under the quantum attacks. However, there are several unsatisfactory performances on either efficiency or security due to the key size and the security loss. In this work, we construct an efficient and fully secure IBEET scheme from the LWE problem in the standard model. Our construction favors weakly compact key size, which refers that our key size is independent of the bit length of identity \(\ell \) and only depends on a smaller value (i.e., the seed length \(\lambda \) of pseudo-random function (PRF)). Moreover, we achieve full security in the standard model without abort-resistant hash functions in security proof compared with prior adaptively secure lattice-based IBEET schemes, which reduces the losses of efficiency and security.
PARE: Part Attention Regressor for 3D Human Body Estimation

Despite significant progress, we show that state of the art 3D human pose and shape estimation methods remain sensitive to partial occlusion and can produce dramatically wrong predictions although much of the body is observable. To address this, we introduce a soft attention mechanism, called the Part Attention REgressor (PARE), that learns to predict body-part-guided attention masks. We observe that state-of-the-art methods rely on global feature representations, making them sensitive to even small occlusions. In contrast, PARE's part-guided attention mechanism overcomes these issues by exploiting information about the visibility of individual body parts while leveraging information from neighboring body-parts to predict occluded parts. We show qualitatively that PARE learns sensible attention masks, and quantitative evaluation confirms that PARE achieves more accurate and robust reconstruction results than existing approaches on both occlusion-specific and standard benchmarks. Code will be available for research purposes at this https URL.
Beyond Non-maximum Suppression - Detecting Lesions in Digital Breast Tomosynthesis Volumes

Detecting the specific locations of malignancy signs in a medical image is a non-trivial and time-consuming task for radiologists. A complex, 3D version of this task, was presented in the DBTex 2021 Grand Challenge on Digital Breast Tomosynthesis Lesion Detection. Teams from all over the world competed in an attempt to build AI models that predict the 3D locations that require biopsy. We describe a novel method to combine detection candidates from multiple models with minimum false positives. This method won the second place in the DBTex competition, with a very small margin from being first and a standout from the rest. We performed an ablation study to show the contribution of each one of the different new components in the proposed ensemble method, including additional performance improvements done after the competition.