Reviewerly/exHarmony huggingface dataset:
Improving Uncertainty Estimations for Mammogram Classification using Semi-Supervised Learning

Computer aided diagnosis for mammogram images have seen positive results through the usage of deep learning architectures. However, limited sample sizes for the target datasets might prevent the usage of a deep learning model under real world scenarios. The usage of unlabeled data to improve the accuracy of the model can be an approach to tackle the lack of target data. Moreover, important model attributes for the medical domain as model uncertainty might be improved through the usage of unlabeled data. Therefore, in this work we explore the impact of using unlabeled data through the implementation of a recent approach known as MixMatch, for mammogram images. We evaluate the improvement on accuracy and uncertainty of the model using popular and simple approaches to estimate uncertainty. For this aim, we propose the usage of the uncertainty balanced accuracy metric.
Federated Learning With Unreliable Clients: Performance Analysis and Mechanism Design

Owing to the low communication costs and privacy-promoting capabilities, federated learning (FL) has become a promising tool for training effective machine learning models among distributed clients. However, with the distributed architecture, low-quality models could be uploaded to the aggregator server by unreliable clients, leading to a degradation or even a collapse of training. In this article, we model these unreliable behaviors of clients and propose a defensive mechanism to mitigate such a security risk. Specifically, we first investigate the impact on the models caused by unreliable clients by deriving a convergence upper bound on the loss function based on the gradient descent updates. Our bounds reveal that with a fixed amount of total computational resources, there exists an optimal number of local training iterations in terms of convergence performance. We further design a novel defensive mechanism, named deep neural network-based secure aggregation (DeepSA). Our experimental results validate our theoretical analysis. In addition, the effectiveness of DeepSA is verified by comparing with other state-of-the-art defensive mechanisms.
CENTRIS: A Precise and Scalable Approach for Identifying Modified Open-Source Software Reuse

Open-source software (OSS) is widely reused as it provides convenience and efficiency in software development. Despite evident benefits, unmanaged OSS components can introduce threats, such as vulnerability propagation and license violation. Unfortunately, however, identifying reused OSS components is a challenge as the reused OSS is predominantly modified and nested. In this paper, we propose CENTRIS, a precise and scalable approach for identifying modified OSS reuse. By segmenting an OSS code base and detecting the reuse of a unique part of the OSS only, CENTRIS is capable of precisely identifying modified OSS reuse in the presence of nested OSS components. For scalability, CENTRIS eliminates redundant code comparisons and accelerates the search using hash functions. When we applied CENTRIS on 10,241 widely-employed GitHub projects, comprising 229,326 versions and 80 billion lines of code, we observed that modified OSS reuse is a norm in software development, occurring 20 times more frequently than exact reuse. Nonetheless, CENTRIS identified reused OSS components with 91% precision and 94% recall in less than a minute per application on average, whereas a recent clone detection technique, which does not take into account modified and nested OSS reuse, hardly reached 10% precision and 40% recall.
Joint Masked CPC And CTC Training For ASR

Self-supervised learning (SSL) has shown promise in learning representations of audio that are useful for automatic speech recognition (ASR). But, training SSL models like wav2vec 2.0 requires a two-stage pipeline. In this paper we demonstrate a single-stage training of ASR models that can utilize both unlabeled and labeled data. During training, we alternately minimize two losses: an unsupervised masked Contrastive Predictive Coding (CPC) loss and the supervised audio-to-text alignment loss Connectionist Temporal Classification (CTC). We show that this joint training method directly optimizes performance for the downstream ASR task using unsupervised data while achieving similar word error rates to wav2vec 2.0 on the Librispeech 100-hours dataset. Finally, we postulate that solving the contrastive task is a regularization for the supervised CTC loss.
The Role of Social Presence for Cooperation in Augmented Reality on Head Mounted Devices

With growing interest regarding cooperation support using Augmented Reality (AR), social presence has become a popular measure of its quality. While this concept is established throughout cooperation research, its role in AR is still unclear: Some work uses social presence as an indicator for support quality, while others found no impact at all. To clarify this role, we conducted a literature review of recent publications that empirically investigated social presence in cooperative AR. After a thorough selection procedure, we analyzed 19 publications according to factors influencing social presence and the impact of social presence on cooperation support. We found that certain interventions support social presence better than others, that social presence has an influence on user's preferences and that the relation between social presence and cooperation quality may depend on the symmetry of the cooperation task. This contributes to existing research by clarifying the role of social presence for cooperative AR and deriving corresponding design recommendations.
A new convolutional neural network predictive model for the automatic recognition of hypogranulated neutrophils in myelodysplastic syndromes

Dysplastic neutrophils commonly show at least 2/3 reduction of the content of cytoplasmic granules by morphologic examination. Recognition of less granulated dysplastic neutrophils by human eyes is difficult and prone to inter-observer variability. To tackle this problem, we proposed a new deep learning model (DysplasiaNet) able to automatically recognize the presence of hypogranulated dysplastic neutrophils in peripheral blood. Eight models were generated by varying convolutional blocks, number of layer nodes and fully connected layers. Each model was trained for 20 epochs. The five most accurate models were selected for a second stage, being trained again from scratch for 100 epochs. After training, cut-off values were calculated for a granularity score that discerns between normal and dysplastic neutrophils. Furthermore, a threshold value was obtained to quantify the minimum proportion of dysplastic neutrophils in the smear to consider that the patient might have a myelodysplastic syndrome (MDS). The final selected model was the one with the highest accuracy (95.5%). We performed a final proof of concept with new patients not involved in previous steps. We reported 95.5% sensitivity, 94.3% specificity, 94% precision, and a global accuracy of 94.85%. The primary contribution of this work is a predictive model for the automatic recognition in an objective way of hypogranulated neutrophils in peripheral blood smears. We envision the utility of the model implemented as an evaluation tool for MDS diagnosis integrated in the clinical laboratory workflow.
Calling for a Plurality of Perspectives on Design Futuring: An Un-Manifesto

The Futures Cone, a prominent model in design futuring, is useful for promoting discussions about possible, plausible, probable, and preferable futures. Yet this model has limitations, such as representing diverse human experiences as a singular point of “the present” and implicitly embedding notions of linear progress. Responding to this, we argue that a plurality of perspectives is needed to engage imaginations that depict a diverse unfolding of potential futures. Through reflecting on our own cultural and professional backgrounds, we offer five perspectives for design futuring as a contribution to this plurality: Parallel Presents, “I Am Time”, Epithelial Metaphors, the Uncertainties Cone, and Meet (with) “Speculation”. These perspectives open alternative approaches to design futuring, move outside prevalent notions of technological progress, and foreground interdependent, relational agencies.
A survey on hardware security of DNN models and accelerators

As "deep neural networks" (DNNs) achieve increasing accuracy, they are getting employed in increasingly diverse applications, including security-critical applications such as medical and defense. This immense use of DNNs has motivated the researchers to scrutinizingly study their security vulnerability and propose countermeasures, especially in the context of hardware security. In this paper, we present a survey of techniques for the hardware security of DNNs. For the research works, we highlight the threat-model, key idea for launching attack and defense strategies. We organize the works on salient categories to highlight their strengths and limitations. This paper aims to equip researchers with the knowledge of recent advances in DNN security and motivate them to think of security as the first principle.
Multiplexed quantum repeaters based on dual-species trapped-ion systems

Trapped ions form an advanced technology platform for quantum information processing with long qubit coherence times, high-fidelity quantum logic gates, optically active qubits, and a potential to scale up in size while preserving a high level of connectivity between qubits. These traits make them attractive not only for quantum computing but also for quantum networking. Dedicated, special-purpose trapped-ion processors in conjunction with suitable interconnecting hardware can be used to form quantum repeaters that enable high-rate quantum communications between distant trapped-ion quantum computers in a network. In this regard, hybrid traps with two distinct species of ions, where one ion species can generate ion-photon entanglement that is useful for optically interfacing with the network and the other has long memory lifetimes, useful for qubit storage, have been proposed for entanglement distribution. We consider an architecture for a repeater based on such dual-species trapped-ion systems. We propose and analyze a protocol based on spatial and temporal mode multiplexing for entanglement distribution across a line network of such repeaters. Our protocol offers enhanced rates compared to rates previously reported for such repeaters. We determine the ion resources required at the repeaters to attain the enhanced rates, and the best rates attainable when constraints are placed on the number of repeaters and the number of ions per repeater. Our results bolster the case for near-term trapped-ion systems as quantum repeaters for long-distance quantum communications.
A Unimodal Reinforced Transformer With Time Squeeze Fusion for Multimodal Sentiment Analysis

Multimodal sentiment analysis refers to inferring sentiment from language, acoustic, and visual sequences. Previous studies focus on analyzing aligned sequences, while the unaligned sequential analysis is more practical in real-world scenarios. Due to the long-time dependency hidden in the multimodal unaligned sequence and time alignment information is not provided, exploring the time-dependent interactions within unaligned sequences is more challenging. To this end, we introduce the time squeeze fusion to automatically explore the time-dependent interactions by modeling the unimodal and multimodal sequences from the perspective of compressing the time dimension. Moreover, prior methods tend to fuse unimodal features into a multimodal embedding, based on which sentiment is inferred. However, we argue that the unimodal information may be lost or the generated multimodal embedding may be redundant. Addressing this issue, we propose a unimodal reinforced Transformer to progressively attend and distill unimodal information from the multimodal embedding, which enables the multimodal embedding to highlight the discriminative unimodal information. Extensive experiments suggest that our model reaches state-of-the-art performance in terms of accuracy and F1 score on MOSEI dataset.
Selection of Lee Filter Window Size Based on Despeckling Efficiency Prediction for Sentinel SAR Images

Radar imaging has many advantages. Meanwhile, SAR images suffer from a noise-like phenomenon called speckle. Many despeckling methods have been proposed to date but there is still no common opinion as to what the best filter is and/or what are its parameters (window or block size, thresholds, etc.). The local statistic Lee filter is one of the most popular and best-known despeckling techniques in radar image processing. Using this filter and Sentinel-1 images as a case study, we show how filter parameters, namely scanning window size, can be selected for a given image based on filter efficiency prediction. Such a prediction can be carried out using a set of input parameters that can be easily and quickly calculated and employing a trained neural network that allows determining one or several criteria of filtering efficiency with high accuracy. The statistical analysis of the obtained results is carried out. This characterizes improvements due to the adaptive selection of the filter window size, both potential and based on prediction. We also analyzed what happens if, due to prediction errors, erroneous decisions are undertaken. Examples for simulated and real-life images are presented.
Enabling Personal Computational Handweaving with a Low-Cost Jacquard Loom

We present an inexpensive tabletop loom that offers fully computational patterning while maintaining the flexibility of handweaving. Our loom can be assembled for under US$200 with 3D printed parts, and it can be controlled straightforwardly over USB. Our loom is explicitly a hand loom: that is, a weaver is required to operate the weaving process and may mediate row-by-row patterning and material specifics like yarn tension. Our approach combines the flexibility of fully analog handweaving with the computational affordances of digital fabrication: it enables the incorporation of special techniques and materials, as well as allowing for the possibility of computational and creative interventions in the weaving process itself. In taking this approach, we aim to serve a range of end users including artisans and researchers, whether for skill-building, for rapid prototyping, or for creative reflection.
Hierarchical Attention Fusion for Geo-Localization

Geo-localization is a critical task in computer vision. In this work, we cast the geo-localization as a 2D image retrieval task. Current state-of-the-art methods for 2D geo-localization are not robust to locate a scene with drastic scale variations because they only exploit features from one semantic level for image representations. To address this limitation, we introduce a hierarchical attention fusion network using multi-scale features for geo-localization. We extract the hierarchical feature maps from a convolutional neural network (CNN) and organically fuse the extracted features for image representations. Our training is self-supervised using adaptive weights to control the attention of feature emphasis from each hierarchical level. Evaluation results on the image retrieval and the large-scale geo-localization benchmarks indicate that our method outperforms the existing state-of-the-art methods. Code is available here: https://github.com/YanLiqi/HAF.
A Numerical Exploration of Signal Detector Arrangement in a Spin-Wave Reservoir Computing Device

This paper studies numerically how the signal detector arrangement influences the performance of reservoir computing using spin waves excited in a ferrimagnetic garnet film. This investigation is essentially important since the input information is not only conveyed but also transformed by the spin waves into high-dimensional information space when the waves propagate in the film in a spatially distributed manner. This spatiotemporal dynamics realizes a rich reservoir-computational functionality. First, we simulate spin waves in a rectangular garnet film with two input electrodes to obtain spatial distributions of the reservoir states in response to input signals, which are represented as spin vectors and used for a machine-learning waveform classification task. The detected reservoir states are combined through readout connection weights to generate a final output. We visualize the spatial distribution of the weights after training to discuss the number and positions of the output electrodes by arranging them at grid points, equiangularly circular points or at random. We evaluate the classification accuracy by changing the number of the output electrodes, and find that a high accuracy (>90%) is achieved with only several tens of output electrodes regardless of grid, circular or random arrangement. These results suggest that the spin waves possess sufficiently complex and rich dynamics for this type of tasks. Then we investigate in which area useful information is distributed more by arranging the electrodes locally on the chip. Finally, we show that this device has generalization ability for input wave-signal frequency in a certain frequency range. These results will lead to practical design of spin-wave reservoir devices for low-power intelligent computing in the near future.
St-Bert: Cross-Modal Language Model Pre-Training for End-to-End Spoken Language Understanding

Language model pre-training has shown promising results in various downstream tasks. In this context, we introduce a cross-modal pre-trained language model, called Speech-Text BERT (ST-BERT), to tackle end-to-end spoken language understanding (E2E SLU) tasks. Taking phoneme posterior and subword-level text as an input, ST-BERT learns a contextualized cross-modal alignment via our two proposed pre-training tasks: Cross-modal Masked Language Modeling (CM-MLM) and Cross-modal Conditioned Language Modeling (CM-CLM). Experimental results on three benchmarks present that our approach is effective for various SLU datasets and shows a surprisingly marginal performance degradation even when 1% of the training data are available. Also, our method shows further SLU performance gain via domain-adaptive pre-training with domain-specific speech-text pair data.
Transformer Language Models with LSTM-Based Cross-Utterance Information Representation

The effective incorporation of cross-utterance information has the potential to improve language models (LMs) for automatic speech recognition (ASR). To extract more powerful and robust cross-utterance representations for the Transformer LM (TLM), this paper proposes the R-TLM which uses hidden states in a long short-term memory (LSTM) LM. To encode the cross-utterance information, the R-TLM incorporates an LSTM module together with a segment-wise recurrence in some of the Transformer blocks. In addition to the LSTM module output, a shortcut connection using a fusion layer which bypasses the LSTM module is also investigated. The proposed system was evaluated on the AMI meeting corpus, the Eval2000 and the RT03 telephone conversation evaluation sets. The best R-TLM achieved 0.9%, 0.6% and 0.8% absolute WER reductions over the single-utterance TLM baseline, and 0.5%, 0.3%, 0.2% absolute WER reductions over a strong cross-utterance TLM baseline on the AMI evaluation set, Eval2000 and RT03 respectively. Improvements on Eval2000 and RT03 were further supported by significance tests. R-TLMs were found to have better LM scores on words where recognition errors are more likely to occur. The R-TLM WER can be further reduced by interpolation with an LSTM-LM.
Night Time Vehicle Detection and Tracking by Fusing Vehicle Parts From Multiple Cameras

Night time vehicle detection and tracking using traditional visible light cameras is a challenging task due to the limited visibility. The current state-of-the-art systems treat the vehicles at night time as paired vehicle headlights or taillights, with no ability to determine the contour of the vehicle or its spatial occupancy. Therefore, this paper proposes the first night time framework that combines the vehicle headlights and taillights to localize the vehicle contours. This new framework includes a novel multi-camera vehicle representation that groups and reconstructs vehicle headlights and taillights following mutual geometric distances between different vehicle components. This novel vehicle contour representation successfully removes duplicated vehicle lights and also compensates for the missing vehicle lights in the detection process. Eventually, vehicle headlight alignment and contour adjustment are used to further refine the vehicle contours. The proposed multi-camera system considers typical four-wheel vehicles, e.g., cars and SUVs, in the monitoring and might not be able to handle large trucks, e.g., 18-wheelers. The experiments are conducted on night time traffic videos under various scenarios and the proposed system attains an average of 0.896 in Multiple Object Tracking Accuracy (MOTA) and an average of 0.904 in Jaccard Coefficient (JC), which indicates 19.2% and 15.9% increases over the baseline system. The night time traffic datasets used in this paper are available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/JustMeZXX/Intelligent-Night-Time-Traffic-Surveillance</uri>
A partially hidden policy CP-ABE scheme against attribute values guessing attacks with online privacy-protective decryption testing in IoT assisted cloud computing

In recent years, to address the security defect that the explicit attribute values in access policies may reveal the privacy, a new variant of ciphertext-policy attribute-based encryption(CP-ABE)——hidden policy CP-ABE (HP-CP-ABE) is proposed in some recent works. But there are two tremendous flaws in most existing HP-CP-ABE schemes. The one issue is that an attacker can launch the attribute values guessing attacks (AVGA) to detect the attribute values in access policies of many HP-CP-ABE schemes. And another issue is that, if the HP-CP-ABE schemes are using the “Linear Secret Sharing Schemes (LSSS)” as their access structures, as the rows of the LSSS matrix grows, the time complexity of the decryption testing algorithm will boost rapidly which will greatly aggravate the computing burden of the user. So in this paper, we propose a partially HP-CP-ABE (PHP-CP-ABE) scheme which can perfectly withstand the attribute values guessing attacks (AVGA). As our access structure is using the LSSS, to alleviate the computing burden of the user, we design a online privacy-protective decryption testing algorithm for the users to privately and securely outsource the decryption testing phase to the cloud server. Our online testing algorithm is privacy-protective which means during running the privacy-protective decryption testing algorithm, the cloud server has no chance to know anything about the attribute values in the access policy and the attribute values of the user. This will prevent the privacy from leaking out to the third party cloud server. Then we rigorously prove that our scheme is selectively indistinguishable secure under chosen plaintext attacks (IND-CPA). Next, by reduction to the computational q-PBDHE assumption which is firstly proposed in our paper, we prove that our HP-CP-ABE scheme is indistinguishable secure under the attribute values guessing attacks (IND-AVGA). Finally through the comparison with the state-of-art HP-CP-ABE schemes from the perspective of functionality and efficiency, it is easily to observe that our scheme has high-security and high-efficiency. In appendix, we give a straightaway analysis to some relevant works to point out the security vulnerabilities in their schemes.