Reviewerly/exHarmony huggingface dataset:
Pedestrian Motion Model Using Non-Parametric Trajectory Clustering and Discrete Transition Points

This paper presents a pedestrian motion model that includes both low level trajectory patterns, and high level discrete transitions. The inclusion of both levels creates a more general predictive model, allowing for more meaningful prediction and reasoning about pedestrian trajectories, as compared to the current state of the art. The model uses an iterative clustering algorithm with (1) Dirichlet Process Gaussian Processes to cluster trajectories into continuous motion patterns and (2) hypothesis testing to identify discrete transitions in the data called transition points. The model iteratively splits full trajectories into sub-trajectory clusters based on transition points, where pedestrians make discrete decisions. State transition probabilities are then learned over the transition points and trajectory clusters. The model is for online prediction of motions, and detection of anomalous trajectories. The proposed model is validated on the Duke MTMC dataset to demonstrate identification of low level trajectory clusters and high level transitions, and the ability to predict pedestrian motion and detect anomalies online with high accuracy.
Dual Learning for Semi-Supervised Natural Language Understanding

Natural language understanding (NLU) converts sentences into structured semantic forms. The paucity of annotated training samples is still a fundamental challenge of NLU. To solve this data sparsity problem, previous work based on semi-supervised learning mainly focuses on exploiting unlabeled sentences. In this work, we introduce a dual task of NLU, semantic-to-sentence generation (SSG), and propose a new framework for semi-supervised NLU with the corresponding dual model. The framework is composed of dual pseudo-labeling and dual learning method, which enables an NLU model to make full use of data (labeled and unlabeled) through a closed-loop of the primal and dual tasks. By incorporating the dual task, the framework can exploit pure semantic forms as well as unlabeled sentences, and further improve the NLU and SSG models iteratively in the closed-loop. The proposed approaches are evaluated on two public datasets (ATIS and SNIPS). Experiments in the semi-supervised setting show that our methods can outperform various baselines significantly, and extensive ablation studies are conducted to verify the effectiveness of our framework. Finally, our method can also achieve the state-of-the-art performance on the two datasets in the supervised setting. Our code is available at \url{https://github.com/rhythmcao/slu-dual-learning.git}.
Some Studies on Multidimensional Fourier Theory for Hilbert Transform, Analytic Signal and AM–FM Representation

In this paper, we propose the Fourier frequency vector (FFV), inherently, associated with multidimensional Fourier transform. With the help of FFV, we are able to provide physical meaning of so called negative frequencies in multidimensional Fourier transform (MDFT), which in turn provide multidimensional spatial and space-time series analysis. The complex exponential representation of sinusoidal function always yields two frequencies, negative frequency corresponding to positive frequency and vice versa, in the multidimensional Fourier spectrum. Thus, using the MDFT, we propose multidimensional Hilbert transform (MDHT) and associated multidimensional analytic signal (MDAS) with following properties: (a) the extra and redundant positive, negative, or both frequencies, introduced due to complex exponential representation of multidimensional Fourier spectrum, are suppressed, (b) real part of MDAS is original signal, (c) real and imaginary part of MDAS are orthogonal, and (d) the magnitude envelope of a original signal is obtained as the magnitude of its associated MDAS, which is the instantaneous amplitude of the MDAS. The proposed MDHT and associated DMAS are generalization of the 1D HT and AS, respectively. We also provide the decomposition of an image into the AM-FM image model by the Fourier method and obtain explicit expression for the analytic image computation by 2DDFT.
On the Convergence of Blockchain and Internet of Things (IoT) Technologies

The Internet of Things (IoT) technology will soon become an integral part of our daily lives to facilitate the control and monitoring of processes and objects and revolutionize the ways that human interacts with the physical world. For all features of IoT to become fully functional in practice, there are several obstacles on the way to be surmounted and critical challenges to be addressed. These include, but are not limited to cybersecurity, data privacy, energy consumption, and scalability. The Blockchain decentralized nature and its multi-faceted procedures offer a useful mechanism to tackle several of these IoT challenges. However, applying the Blockchain protocols to IoT without considering their tremendous computational loads, delays, and bandwidth overhead can let to a new set of problems. This review evaluates some of the main challenges we face in the integration of Blockchain and IoT technologies and provides insights and high-level solutions that can potentially handle the shortcomings and constraints of both IoT and Blockchain technologies.
Privacy-Protection Scheme Based on Sanitizable Signature for Smart Mobile Medical Scenarios

With the popularization of wireless communication and smart devices in the medical field, mobile medicine has attracted more and more attention because it can break through the limitations of time, space, and objects and provide more efficient and quality medical services. However, the characteristics of a mobile smart medical network make it more susceptible to security threats such as data integrity damage and privacy leakage than those of traditional wired networks. In recent years, many digital signature schemes have been proposed to alleviate some of these challenges. Unfortunately, traditional digital signatures cannot meet the diversity and privacy requirements of medical data applications. In response to this problem, this paper uses the unique security attributes of sanitizable signatures to carry out research on the security and privacy protection of medical data and proposes a data security and privacy protection scheme suitable for smart mobile medical scenarios. Security analysis and performance evaluation show that our new scheme effectively guarantees data security and user privacy while greatly reducing computation and communication costs, making it especially suitable for mobile smart medical application scenarios.
Kinship Identification Through Joint Learning Using Kinship Verification Ensembles

Kinship verification is a well-explored task: identifying whether or not two persons are kin. In contrast, kinship identification has been largely ignored so far. Kinship identification aims to further identify the particular type of kinship. An extension to kinship verification run short to properly obtain identification, because existing verification networks are individually trained on specific kinships and do not consider the context between different kinship types. Also, existing kinship verification datasets have biased positive-negative distributions which are different than real-world distributions. To this end, we propose a novel kinship identification approach based on joint training of kinship verification ensembles and classification modules. We propose to rebalance the training dataset to become more realistic. Large scale experiments demonstrate the appealing performance on kinship identification. The experiments further show significant performance improvement of kinship verification when trained on the same dataset with more realistic distributions.
S2DNAS: Transforming Static CNN Model for Dynamic Inference via Neural Architecture Search

Recently, dynamic inference has emerged as a promising way to reduce the computational cost of deep convolutional neural networks (CNNs). In contrast to static methods (e.g., weight pruning), dynamic inference adaptively adjusts the inference process according to each input sample, which can considerably reduce the computational cost on “easy” samples while maintaining the overall model performance. In this paper, we introduce a general framework, S2DNAS, which can transform various static CNN models to support dynamic inference via neural architecture search. To this end, based on a given CNN model, we first generate a CNN architecture space in which each architecture is a multi-stage CNN generated from the given model using some predefined transformations. Then, we propose a reinforcement learning based approach to automatically search for the optimal CNN architecture in the generated space. At last, with the searched multi-stage network, we can perform dynamic inference by adaptively choosing a stage to evaluate for each sample. Unlike previous works that introduce irregular computations or complex controllers in the inference or re-design a CNN model from scratch, our method can generalize to most of the popular CNN architectures and the searched dynamic network can be directly deployed using existing deep learning frameworks in various hardware devices.
Phase sensitivity approaching the quantum Cramér-Rao bound in a modified SU(1,1) interferometer

SU(1,1) interferometers, based on the usage of nonlinear elements, are superior to passive interferometers in phase sensitivity. However, the SU(1,1) interferometer cannot make full use of photons carrying phase information as the second nonlinear element annihilates some of the photons inside. Here we focus on improving phase sensitivity and propose a different protocol based on a modified SU(1,1) interferometer, where the second nonlinear element is replaced by a beam splitter. We utilize two coherent states as inputs and implement balanced homodyne measurement at the output. Our analysis suggests that the protocol we propose can achieve the sub-shot-noise-limited phase sensitivity and is robust against photon loss as well as background noise. Our work is important for practical quantum metrology using SU(1,1) interferometers.
Noticing Motion Patterns: A Temporal CNN With a Novel Convolution Operator for Human Trajectory Prediction

We propose a Convolutional Neural Network-based approach to learn, detect,and extract patterns in sequential trajectory data, known here as Social Pattern Extraction Convolution (Social-PEC). A set of experiments carried out on the human trajectory prediction problem shows that our model performs comparably to the state of the art and outperforms in some cases. More importantly,the proposed approach unveils the obscurity in the previous use of pooling layer, presenting a way to intuitively explain the decision-making process.
Suggest me a movie for tonight: Leveraging Knowledge Graphs for Conversational Recommendation

Conversational recommender systems focus on the task of suggesting products to users based on the conversation flow.Recently, the use of external knowledge in the form of knowledge graphs has shown to improve the performance in recommendation and dialogue systems.Information from knowledge graphs aids in enriching those systems by providing additional information such as closely related products and textual descriptions of the items.However, knowledge graphs are incomplete since they do not contain all factual information present on the web.Furthermore, when working on a specific domain, knowledge graphs in its entirety contribute towards extraneous information and noise.In this work, we study several subgraph construction methods and compare their performance across the recommendation task.We incorporate pre-trained embeddings from the subgraphs along with positional embeddings in our models.Extensive experiments show that our method has a relative improvement of at least 5.62% compared to the state-of-the-art on multiple metrics on the recommendation task.
Remote Reporting from Home for Primary Diagnosis in Surgical Pathology: A Tertiary Oncology Center Experience during the COVID-19 Pandemic

The COVID-19 pandemic accelerated the widespread adoption of digital pathology (DP) for primary diagnosis in surgical pathology. This paradigm shift is likely to influence how we function routinely in the postpandemic era. We present learnings from early adoption of DP for a live digital sign-out from home in a risk-mitigated environment.We aimed to validate DP for remote reporting from home in a real-time environment and evaluate the parameters influencing the efficiency of a digital workflow. Eighteen pathologists prospectively validated DP for remote use on 567 biopsy cases including 616 individual parts from 7 subspecialties over a duration from March 21, 2020, to June 30, 2020. The slides were digitized using Roche Ventana DP200 whole-slide scanner and reported from respective homes in a risk-mitigated environment.Following re-review of glass slides, there was no major discordance and 1.2% (n = 7/567) minor discordance. The deferral rate was 4.5%. All pathologists reported from their respective homes from laptops with an average network speed of 20 megabits per second.We successfully validated and adopted a digital workflow for remote reporting with available resources and were able to provide our patients, an undisrupted access to subspecialty expertise during these unprecedented times.
Multi-Label Learning With Visual-Semantic Embedded Knowledge Graph for Diagnosis of Radiology Imaging

A significant task of automatic diagnosis for radiology imaging, especially for chest X-rays, is to identify disease types, which can be viewed as a multi-label learning problem. Prior state-of-the-art approaches adopted the graph convolutional network to model the correlations among disease labels. However, the utilization of medical reports paired with radiology images is neglected in such approaches. Hence, at least two novel improvements are proposed in this paper. First, disease label embeddings are pre-trained on the total radiology reports, and these semantic features along with encoded X-ray features are fused in a transformer encoder for graph initialization. Second, to expand the representation ability of the graph, extra medical terms from radiology reports are mined and added to the graph model as auxiliary nodes without changing the size of the output space. Experiments conducted on two public chest-X-ray datasets demonstrate the outstanding performance over compared models and the advantages of the proposed improvements.
A Visual Support of Standard Procedures for Solar Radiation Quality Control

Solar irradiance data from high-quality ground-based measurements are primordial for different solar energy applications. In order to achieve the required accuracy, quality control procedures are of great benefit. A variety of approaches have been proposed. In this sense, some approaches propose a visual representation of the routine, while others only provide a time series of binary flag values, and do not propose any specific visualization of the flagged data as opposed to non-flagged ones. In this regard, the present paper puts forward a complete routine including several quality control procedures for solar irradiance measurements by providing visual support for these different approaches. The visual tool in question was validated using five years research data with 10 minutes resolution of the global, diffuse and direct components of solar irradiation collected from three ground-based weather stations in Morocco. This visual tool puts forth a more precise idea of the measurement quality by detecting various errors, such as time shifts, outliers identification; either with one or two components, or consistency tests between the three components of solar radiation when available. The proposed tool can be regarded as a means of improving the detection rate of abnormal data as a first step in diagnosing the prominent causes of error.
IR finite S-matrix by gauge invariant dressed states

Dressed states were proposed to define the infrared (IR) finite $S$-matrix in QED or gravity. We show that the original Kulish-Faddeev dressed states are not enough to cure the IR divergences. To illustrate this problem, we consider QED with background currents (Wilson lines). This theory is exactly solvable but shares the same IR problems as the full QED. We show that naive asymptotic states lead to IR divergences in the $S$-matrix and are also inconsistent with the asymptotic symmetry, even if we add the original Kulish-Faddeev dressing operators. We then propose new dressed states which are consistent with the asymptotic symmetry. We show that the $S$-matrix for the dressed states is IR finite. We finally conclude that appropriate dressed asymptotic states define the IR finite $S$-matrix in the full QED.
Abstract Neural Networks

Deep Neural Networks (DNNs) are rapidly being applied to safety-critical domains such as drone and airplane control, motivating techniques for verifying the safety of their behavior. Unfortunately, DNN verification is NP-hard, with current algorithms slowing exponentially with the number of nodes in the DNN. This paper introduces the notion of Abstract Neural Networks (ANNs), which can be used to soundly overapproximate DNNs while using fewer nodes. An ANN is like a DNN except weight matrices are replaced by values in a given abstract domain. We present a framework parameterized by the abstract domain and activation functions used in the DNN that can be used to construct a corresponding ANN. We present necessary and sufficient conditions on the DNN activation functions for the constructed ANN to soundly over-approximate the given DNN. Prior work on DNN abstraction was restricted to the interval domain and ReLU activation function. Our framework can be instantiated with other abstract domains such as octagons and polyhedra, as well as other activation functions such as Leaky ReLU, Sigmoid, and Hyperbolic Tangent.
Dual-mode ASR: Unify and Improve Streaming ASR with Full-context Modeling

Streaming automatic speech recognition (ASR) aims to emit each hypothesized word as quickly and accurately as possible, while full-context ASR waits for the completion of a full speech utterance before emitting completed hypotheses. In this work, we propose a unified framework, Dual-mode ASR, to train a single end-to-end ASR model with shared weights for both streaming and full-context speech recognition. We show that the latency and accuracy of streaming ASR significantly benefit from weight sharing and joint training of full-context ASR, especially with inplace knowledge distillation during the training. The Dual-mode ASR framework can be applied to recent state-of-the-art convolution-based and transformer-based ASR networks. We present extensive experiments with two state-of-the-art ASR networks, ContextNet and Conformer, on two datasets, a widely used public dataset LibriSpeech and a large-scale dataset MultiDomain. Experiments and ablation studies demonstrate that Dual-mode ASR not only simplifies the workflow of training and deploying streaming and full-context ASR models, but also significantly improves both emission latency and recognition accuracy of streaming ASR. With Dual-mode ASR, we achieve new state-of-the-art streaming ASR results on both LibriSpeech and MultiDomain in terms of accuracy and latency.
Deep Learning Image Analysis of Benign Breast Disease to Identify Subsequent Risk of Breast Cancer

Abstract Background New biomarkers of risk may improve breast cancer (BC) risk prediction. We developed a computational pathology method to segment benign breast disease (BBD) whole slide images into epithelium, fibrous stroma, and fat. We applied our method to the BBD BC nested case-control study within the Nurses’ Health Studies to assess whether computer-derived tissue composition or a morphometric signature was associated with subsequent risk of BC. Methods Tissue segmentation and nuclei detection deep-learning networks were established and applied to 3795 whole slide images from 293 cases who developed BC and 1132 controls who did not. Percentages of each tissue region were calculated, and 615 morphometric features were extracted. Elastic net regression was used to create a BC morphometric signature. Associations between BC risk factors and age-adjusted tissue composition among controls were assessed using analysis of covariance. Unconditional logistic regression, adjusting for the matching factors, BBD histological subtypes, parity, menopausal status, and body mass index evaluated the relationship between tissue composition and BC risk. All statistical tests were 2-sided. Results Among controls, direction of associations between BBD subtypes, parity, and number of births with breast composition varied by tissue region; select regions were associated with childhood body size, body mass index, age of menarche, and menopausal status (all P &amp;lt; .05). A higher proportion of epithelial tissue was associated with increased BC risk (odds ratio = 1.39, 95% confidence interval = 0.91 to 2.14, for highest vs lowest quartiles, Ptrend = .047). No morphometric signature was associated with BC. Conclusions The amount of epithelial tissue may be incorporated into risk assessment models to improve BC risk prediction.
Learning Inductive Attention Guidance for Partially Supervised Pancreatic Ductal Adenocarcinoma Prediction

Pancreatic ductal adenocarcinoma (PDAC) is the third most common cause of cancer death in the United States. Predicting tumors like PDACs (including both classification and segmentation) from medical images by deep learning is becoming a growing trend, but usually a large number of annotated data are required for training, which is very labor-intensive and time-consuming. In this paper, we consider a partially supervised setting, where cheap image-level annotations are provided for all the training data, and the costly per-voxel annotations are only available for a subset of them. We propose an Inductive Attention Guidance Network (IAG-Net) to jointly learn a global image-level classifier for normal/PDAC classification and a local voxel-level classifier for semi-supervised PDAC segmentation. We instantiate both the global and the local classifiers by multiple instance learning (MIL), where the attention guidance, indicating roughly where the PDAC regions are, is the key to bridging them: For global MIL based normal/PDAC classification, attention serves as a weight for each instance (voxel) during MIL pooling, which eliminates the distraction from the background; For local MIL based semi-supervised PDAC segmentation, the attention guidance is inductive, which not only provides bag-level pseudo-labels to training data without per-voxel annotations for MIL training, but also acts as a proxy of an instance-level classifier. Experimental results show that our IAG-Net boosts PDAC segmentation accuracy by more than 5% compared with the state-of-the-arts.