Reviewerly/exHarmony huggingface dataset:
MultiSiam: Self-supervised Multi-instance Siamese Representation Learning for Autonomous Driving

Autonomous driving has attracted much attention over the years but turns out to be harder than expected, probably due to the difficulty of labeled data collection for model training. Self-supervised learning (SSL), which leverages unlabeled data only for representation learning, might be a promising way to improve model performance. Existing SSL methods, however, usually rely on the single-centric-object guarantee, which may not be applicable for multi-instance datasets such as street scenes. To alleviate this limitation, we raise two issues to solve: (1) how to define positive samples for cross-view consistency and (2) how to measure similarity in multi-instance circumstances. We first adopt an IoU threshold during random cropping to transfer global-inconsistency to local-consistency. Then, we propose two feature alignment methods to enable 2D feature maps for multi-instance similarity measurement. Addition-ally, we adopt intra-image clustering with self-attention for further mining intra-image similarity and translation-invariance. Experiments show that, when pre-trained on Waymo dataset, our method called Multi-instance Siamese Network (MultiSiam) remarkably improves generalization ability and achieves state-of-the-art transfer performance on autonomous driving benchmarks, including Cityscapes and BDD100K, while existing SSL counterparts like MoCo, MoCo-v2, and BYOL show significant performance drop. By pre-training on SODA10M, a large-scale autonomous driving dataset, MultiSiam exceeds the ImageNet pre-trained MoCo-v2, demonstrating the potential of domain-specific pre-training. Code will be available at https://github.com/KaiChen1998/MultiSiam.
Inverse design of grating couplers using the policy gradient method from reinforcement learning

We present a proof-of-concept technique for the inverse design of electromagnetic devices motivated by the policy gradient method in reinforcement learning, named PHORCED (PHotonic Optimization using REINFORCE Criteria for Enhanced Design). This technique uses a probabilistic generative neural network interfaced with an electromagnetic solver to assist in the design of photonic devices, such as grating couplers. We show that PHORCED obtains better performing grating coupler designs than local gradient-based inverse design via the adjoint method, while potentially providing faster convergence over competing state-of-the-art generative methods. As a further example of the benefits of this method, we implement transfer learning with PHORCED, demonstrating that a neural network trained to optimize 8$^\circ$ grating couplers can then be re-trained on grating couplers with alternate scattering angles while requiring >10$\times$ fewer simulations than control cases.
T-SVDNet: Exploring High-Order Prototypical Correlations for Multi-Source Domain Adaptation

Most existing domain adaptation methods focus on adaptation from only one source domain, however, in practice there are a number of relevant sources that could be leveraged to help improve performance on target domain. We propose a novel approach named T-SVDNet to address the task of Multi-source Domain Adaptation (MDA), which is featured by incorporating Tensor Singular Value Decomposition (T-SVD) into a neural network’s training pipeline. Overall, high-order correlations among multiple domains and categories are fully explored so as to better bridge the domain gap. Specifically, we impose Tensor-Low-Rank (TLR) constraint on a tensor obtained by stacking up a group of prototypical similarity matrices, aiming at capturing consistent data structure across different domains. Furthermore, to avoid negative transfer brought by noisy source data, we propose a novel uncertainty-aware weighting strategy to adaptively assign weights to different source domains and samples based on the result of uncertainty estimation. Extensive experiments conducted on public benchmarks demonstrate the superiority of our model in addressing the task of MDA compared to state-of-the-art methods. Code is available at https://github.com/lslrh/T-SVDNet.
Secure Watermarking Scheme for Color DICOM Images in Telemedicine Applications

Teleradiology plays a vital role in the medical field, which permits transmitting medical and imaging data over a communication network. It ensures data reliability and provides convenient communication for clinical interpretation and diagnostic purposes. The transmission of this medical data over a network raises the problems of legal, ethical issues, privacy, and copyright authenticity. The copyright protection of medical images is a significant issue in the medical field. Watermarking schemes are used to address these issues. A gray-level or binary image is used as a watermark frequently in color image watermarking schemes. In this paper, the authors propose a novel non-blind medical image watermarking scheme based on 2-D Lifting Wavelet Transform (LWT), Multiresolution Singular Value Decomposition (MSVD), and LU factorization to improve the robustness and authenticity of medical images. In this scheme, multiple color watermarks are embedded into the colored DICOM (Digital Imaging and Communications in Medicine) images obtained from Color Doppler images (DICOM format), and the average results achieved by our proposed scheme is 46.84 db for Peak Signal-to-Noise Ratio (PSNR), 37.46 db for Signal-to-Noise Ratio (SNR), 0.99 for Quality of Image and 0.998 for Normalized Correlation for various image processing attacks. These results make our watermarking technique an ideal candidate for medical image watermarking.
Locked down through virtual disconnect: Navigating life by staying on/off the health QR code during COVID-19 in China

During the COVID-19 pandemic, China has implemented the most stringent lockdown measures across major cities during moments of heightened risk of infection. This mechanism is aided by the mandated Health QR Code to surveil and contain spread of the virus. This research scrutinizes the varying degrees and forms of disconnect in the context of the Health Code mandate. Informed by extensive focus group interviews of diverse resident groups, supplemented with thematic analysis of 136 newspaper news stories covering the enforcement of the Health Code, the research results offer insight over the multiplicity of voluntary and involuntary disconnective practices at both the individual/internal and societal/external level. These findings are contextualized for their theoretical relevance towards the overall line of inquiry into information technology non-use, and the pragmatic implications of the Health Code are discussed for contact tracing app design and utility in general.
Path independent stereo digital image correlation with high speed and analysis resolution

Path independent strategy is introduced into stereo digital image correlation, in which high accuracy is attained through the inverse compositional Gauss-Newton (IC-GN) algorithm with the second and the first order shape functions for stereo matching and temporal matching, respectively. To gain strong adaptibility to large and complex deformation or considerable parallax betweent the two views, two robust methods are developped to estimate the initial guess for the iterative IC-GN algorithm. One is aided by scale invariant feature transform (SIFT) features and the other is aided by epipolar constraint. Powered by parallel computing on GPU, the proposed stereo DIC demonstrates high computation efficiency. Real-time stereo DIC is realized and used to measure the large deformation at common video recording rate (∼24 fps) with high ananlysis resolution (∼6300 points of interest in region of interest).
Three-way Naive Bayesian collaborative filtering recommendation model for smart city

Smart city construction has penetrated all aspects of human life. People can continue to feel the convenience of life brought by new technologies and new services. In order to further optimize the smart city system, aiming at the sparsity of scoring data and the difficulty of two-way decisions to deal with uncertain decisions in the recommendation algorithm, a three-way Naive Bayesian Collaborative Filtering Recommendation model (3NBCFR) for smart city is constructed by integrating Naive Bayesian, three-way decisions and collaborative filtering algorithm. Firstly, we consider the influence of item attributes on user scoring. Naive Bayesian classifier is used to score unrated items, which can effectively fill in the missing values in the score matrix and solve the problem of data sparsity. Secondly, to ensure sustainable recommendations for uncertain goals, the three-way decisions is introduced into the collaborative filtering recommendation system, three-way recommendation rules are formulated, and 3NBCFR is constructed. Finally, the model is applied to the movie recommendation system. The experiment on Movielens shows that compared with the traditional collaborative filtering recommendation algorithm, 3NBCFR algorithm reduces the recommendation cost and improves the recommendation quality. At the same time, it lays a foundation for promoting the construction of smart city.
ATOM: Robustifying Out-of-Distribution Detection Using Outlier Mining

Detecting out-of-distribution (OOD) inputs is critical for safely deploying deep learning models in an open-world setting. However, existing OOD detection solutions can be brittle in the open world, facing various types of adversarial OOD inputs. While methods leveraging auxiliary OOD data have emerged, our analysis on illuminative examples reveals a key insight that the majority of auxiliary OOD examples may not meaningfully improve or even hurt the decision boundary of the OOD detector, which is also observed in empirical results on real data. In this paper, we provide a theoretically motivated method, Adversarial Training with informative Outlier Mining (ATOM), which improves the robustness of OOD detection. We show that, by mining informative auxiliary OOD data, one can significantly improve OOD detection performance, and somewhat surprisingly, generalize to unseen adversarial attacks. ATOM achieves state-of-the-art performance under a broad family of classic and adversarial OOD evaluation tasks. For example, on the CIFAR-10 in-distribution dataset, ATOM reduces the FPR (at TPR 95%) by up to 57.99% under adversarial OOD inputs, surpassing the previous best baseline by a large margin.
Delay Analysis of Wireless Federated Learning Based on Saddle Point Approximation and Large Deviation Theory

Federated learning (FL) is a collaborative machine learning paradigm, which enables deep learning model training over a large volume of decentralized data residing in mobile devices without accessing clients’ private data. Driven by the ever increasing demand for model training of mobile applications or devices, a vast majority of FL tasks are implemented over wireless fading channels. Due to the time-varying nature of wireless channels, however, random delay occurs in both the uplink and downlink transmissions of FL. How to analyze the overall time consumption of a wireless FL task, or more specifically, a FL’s delay distribution, becomes a challenging but important open problem, especially for delay-sensitive model training. In this paper, we present a unified framework to calculate the approximate delay distributions of FL over arbitrary fading channels. Specifically, saddle point approximation, extreme value theory (EVT), and large deviation theory (LDT) are jointly exploited to find the approximate delay distribution along with its tail distribution, which characterizes the quality-of-service of a wireless FL system. Simulation results will demonstrate that our approximation method achieves a small approximation error, which vanishes with the increase of training accuracy.
Temporal-Structure-Assisted Gradient Aggregation for Over-the-Air Federated Edge Learning

In this paper, we investigate over-the-air model aggregation in a federated edge learning (FEEL) system. We introduce a Markovian probability model to characterize the intrinsic temporal structure of the model aggregation series. With this temporal probability model, we formulate the model aggregation problem as to infer the desired aggregated update given all the past observations from a Bayesian perspective. We develop a message passing based algorithm, termed temporal-structure-assisted gradient aggregation (TSA-GA), to fulfil this estimation task with low complexity and near-optimal performance. We further establish the state evolution (SE) analysis to characterize the behaviour of the proposed TSA-GA algorithm, and derive an explicit bound of the expected loss reduction of the FEEL system under certain standard regularity conditions. In addition, we develop an expectation maximization (EM) strategy to learn the unknown parameters in the Markovian model. We show that the proposed TSA-GA significantly outperforms the state-of-the-art analog compression scheme, and is able to achieve comparable learning performance as the error-free benchmark in terms of final test accuracy.
End-to-End Detection and Pose Estimation of Two Interacting Hands

Three dimensional hand pose estimation has reached a level of maturity, enabling real-world applications for single-hand cases. However, accurate estimation of the pose of two closely interacting hands still remains a challenge as in this case, one hand often occludes the other. We present a new algorithm that accurately estimates hand poses in such a challenging scenario. The crux of our algorithm lies in a framework that jointly trains the estimators of interacting hands, leveraging their inter-dependence. Further, we employ a GAN-type discriminator of interacting hand pose that helps avoid physically implausible configurations, e.g. intersecting fingers, and exploit the visibility of joints to improve intermediate 2D pose estimation. We incorporate them into a single model that learns to detect hands and estimate their pose based on a unified criterion of pose estimation accuracy. To our knowledge, this is the first attempt to build an end-to-end network that detects and estimates the pose of two closely interacting hands (as well as single hands). In the experiments with three datasets representing challenging real-world scenarios, our algorithm demonstrated significant and consistent performance improvements over state-of-the-arts.
HiveMind: Towards Cellular Native Machine Learning Model Splitting

The increasing processing load of today&#x2019;s mobile machine learning (ML) application challenges the stringent computation budget of mobile user equipment (UE). With the wide deployment of 5G edge-cloud, a new ML offloading scheme called split ML is provisioned to enable computation-intensive mobile ML applications by splitting an ML model across mobile UE, edge, and cloud. However, the complex split assignment problems pose new challenges for split ML system design. In this paper, we introduce HiveMind, the first practical multi-split ML system tailored for 5G cellular networks. HiveMind reformulates the complicated multi-split problem to a min-cost graph search and optimizes the distributed algorithm to drastically reduce the signaling overhead. Benefit from its low overhead property, HiveMind makes the optimal split decision on multiple computing nodes in real-time and adapts the split decisions to the instantaneous network dynamics. HiveMind also incorporates a multi-objective mechanism that accommodates heterogeneous objectives for a single ML task. HiveMind adapts to a wide range of ML frameworks, including non-linear models like Recurrent Neural Network (RNN), Federated Learning (FL), and Multi-agent Reinforcement Learning (MARL). We evaluate HiveMind on 5G MEC network simulators with realistic traffic patterns and real-life MEC computation/communication profiles. Our experiments demonstrate that HiveMind achieves the optimal efficiency comparing to state-of-art split ML designs.
Application-Oriented Performance Benchmarks for Quantum Computing

In this work we introduce an open source suite of quantum application-oriented performance benchmarks that is designed to measure the effectiveness of quantum computing hardware at executing quantum applications.These benchmarks probe a quantum computer's performance on various algorithms and small applications as the problem size is varied, by mapping out the fidelity of the results as a function of circuit width and depth using the framework of volumetric benchmarking.In addition to estimating the fidelity of results generated by quantum execution, the suite is designed to benchmark certain aspects of the execution pipeline in order to provide end-users with a practical measure of both the quality of and the time to solution.Our methodology is constructed to anticipate advances in quantum computing hardware that are likely to emerge in the next five years.This benchmarking suite is designed to be readily accessible to a broad audience of users and provides benchmarks that correspond to many well-known quantum computing algorithms.
SignBERT: Pre-Training of Hand-Model-Aware Representation for Sign Language Recognition

Hand gesture serves as a critical role in sign language. Current deep-learning-based sign language recognition (SLR) methods may suffer insufficient interpretability and overfitting due to limited sign data sources. In this paper, we introduce the first self-supervised pre-trainable SignBERT with incorporated hand prior for SLR. Sign-BERT views the hand pose as a visual token, which is derived from an off-the-shelf pose extractor. The visual tokens are then embedded with gesture state, temporal and hand chirality information. To take full advantage of available sign data sources, SignBERT first performs self-supervised pre-training by masking and reconstructing visual tokens. Jointly with several mask modeling strategies, we attempt to incorporate hand prior in a model-aware method to better model hierarchical context over the hand sequence. Then with the prediction head added, SignBERT is fine-tuned to perform the downstream SLR task. To validate the effectiveness of our method on SLR, we perform extensive experiments on four public benchmark datasets, i.e., NMFs-CSL, SLR500, MSASL and WLASL. Experiment results demonstrate the effectiveness of both self-supervised learning and imported hand prior. Furthermore, we achieve state-of-the-art performance on all benchmarks with a notable gain.
An Empirical Study of Rule-Based and Learning-Based Approaches for Static Application Security Testing

Background: Static Application Security Testing (SAST) tools purport to assist developers in detecting security issues in source code. These tools typically use rule-based approaches to scan source code for security vulnerabilities. However, due to the significant shortcomings of these tools (i.e., high false positive rates), learning-based approaches for Software Vulnerability Prediction (SVP) are becoming a popular approach. Aims: Despite the similar objectives of these two approaches, their comparative value is unexplored. We provide an empirical analysis of SAST tools and SVP models, to identify their relative capabilities for source code security analysis. Method: We evaluate the detection and assessment performance of several common SAST tools and SVP models on a variety of vulnerability datasets. We further assess the viability and potential benefits of combining the two approaches. Results: SAST tools and SVP models provide similar detection capabilities, but SVP models exhibit better overall performance for both detection and assessment. Unification of the two approaches is difficult due to lacking synergies. Conclusions: Our study generates 12 main findings which provide insights into the capabilities and synergy of these two approaches. Through these observations we provide recommendations for use and improvement.
Intent-based networking and its application to optical networks [Invited Tutorial]

The intent-based networking (IBN) paradigm targets defining high-level abstractions so network operators can define what their desired outcomes are without specifying how they would be achieved. The latter can be achieved by leveraging network programmability, monitoring, and data analytics, as well as the key assurance component. In this tutorial, we introduce the IBN paradigm and its application to optical networking, highlighting the benefits that machine learning (ML) algorithms can provide to IBN. Because the deployment of ML applications requires a specific orchestrator to create ML functions that are connected as ML pipelines, we show an implementation of such an orchestrator. Some challenges and solutions are presented for the generation of accurate synthetic data, proactive self-configuration, and cooperative intent operation. Illustrative examples of intent-based operation and numerical results are presented, and the obtained performance is discussed.
SimTriplet: Simple Triplet Representation Learning with a Single GPU

Contrastive learning is a key technique of modern self-supervised learning. The broader accessibility of earlier approaches is hindered by the need of heavy computational resources (e.g., at least 8 GPUs or 32 TPU cores), which accommodate for large-scale negative samples or momentum. The more recent SimSiam approach addresses such key limitations via stop-gradient without momentum encoders. In medical image analysis, multiple instances can be achieved from the same patient or tissue. Inspired by these advances, we propose a simple triplet representation learning (SimTriplet) approach on pathological images. The contribution of the paper is three-fold: (1) The proposed SimTriplet method takes advantage of the multi-view nature of medical images beyond self-augmentation; (2) The method maximizes both intra-sample and inter-sample similarities via triplets from positive pairs, without using negative samples; and (3) The recent mix precision training is employed to advance the training by only using a single GPU with 16 GB memory. By learning from 79,000 unlabeled pathological patch images, SimTriplet achieved 10.58% better performance compared with supervised learning. It also achieved 2.13% better performance compared with SimSiam. Our proposed SimTriplet can achieve decent performance using only 1% labeled data. The code and data are available at https://github.com/hrlblab/SimTriplet.
Noisy Labels are Treasure: Mean-Teacher-Assisted Confident Learning for Hepatic Vessel Segmentation

Manually segmenting the hepatic vessels from Computer Tomography (CT) is far more expertise-demanding and laborious than other structures due to the low-contrast and complex morphology of vessels, resulting in the extreme lack of high-quality labeled data. Without sufficient high-quality annotations, the usual data-driven learning-based approaches struggle with deficient training. On the other hand, directly introducing additional data with low-quality annotations may confuse the network, leading to undesirable performance degradation. To address this issue, we propose a novel mean-teacher-assisted confident learning framework to robustly exploit the noisy labeled data for the challenging hepatic vessel segmentation task. Specifically, with the adapted confident learning assisted by a third party, i.e., the weight-averaged teacher model, the noisy labels in the additional low-quality dataset can be transformed from 'encumbrance' to 'treasure' via progressive pixel-wise soft-correction, thus providing productive guidance. Extensive experiments using two public datasets demonstrate the superiority of the proposed framework as well as the effectiveness of each component.