Reviewerly/exHarmony huggingface dataset:
Geometry-Free View Synthesis: Transformers and no 3D Priors

Is a geometric model required to synthesize novel views from a single image? Being bound to local convolutions, CNNs need explicit 3D biases to model geometric transformations. In contrast, we demonstrate that a transformer-based model can synthesize entirely novel views without any hand-engineered 3D biases. This is achieved by (i) a global attention mechanism for implicitly learning long-range 3D correspondences between source and target views, and (ii) a probabilistic formulation necessary to capture the ambiguity inherent in predicting novel views from a single image, thereby overcoming the limitations of previous approaches that are restricted to relatively small viewpoint changes. We evaluate various ways to integrate 3D priors into a transformer architecture. However, our experiments show that no such geometric priors are required and that the transformer is capable of implicitly learning 3D relationships between images. Furthermore, this approach outperforms the state of the art in terms of visual quality while covering the full distribution of possible realizations.
COMISR: Compression-Informed Video Super-Resolution

Most video super-resolution methods focus on restoring high-resolution video frames from low-resolution videos without taking into account compression. However, most videos on the web or mobile devices are compressed, and the compression can be severe when the bandwidth is limited. In this paper, we propose a new compression-informed video super-resolution model to restore high-resolution content without introducing artifacts caused by compression. The proposed model consists of three modules for video super-resolution: bi-directional recurrent warping, detail-preserving flow estimation, and Laplacian enhancement. All these three modules are used to deal with compression properties such as the location of the intra-frames in the input and smoothness in the output frames. For thorough performance evaluation, we conducted extensive experiments on standard datasets with a wide range of compression rates, covering many real video use cases. We showed that our method not only recovers high-resolution content on uncompressed frames from the widely-used benchmark datasets, but also achieves state-of-the-art performance in super-resolving compressed videos based on numerous quantitative metrics. We also evaluated the proposed method by simulating streaming from YouTube to demonstrate its effectiveness and robustness. The source codes and trained models are available at https://github.com/google-research/googleresearch/tree/master/comisr.
A novel convolutional neural network for kidney ultrasound images segmentation

Ultrasound imaging has been widely used in the screening of kidney diseases. The localization and segmentation of the kidneys in ultrasound images are helpful for the clinical diagnosis of diseases. However, it is a challenging task to segment the kidney accurately from ultrasound images due to the interference of various factors.In this paper, a novel multi-scale and deep-supervised CNN architecture is proposed to segment the kidney. The architecture consists of an encoder, a pyramid pooling module and a decoder. In the encoder, we design a multi-scale input pyramid with parallel branches to capture features at different scales. In the decoder, a multi-output supervision module is developed. The introduction of the multi-output supervision module enables the network to learn to predict more precise segmentation results scale-by-scale. In addition, we construct a kidney ultrasound dataset, which contains of 400 images and 400 labels.To highlight effectiveness of the proposed approach, we use six quantitative indicators to compare with several state-of-the-art methods on the same kidney ultrasound dataset. The results of our method on the six indicators of accuracy, dice, jaccard, precision, recall and ASSD are 98.86%, 95.86%, 92.18%, 96.38%, 95.47% and 0.3510, respectively.The analysis of evaluation indicators and segmentation results shows that our method achieves the best performance in kidney ultrasound image segmentation.
Predicting with Confidence on Unseen Distributions

Recent work has shown that the accuracy of machine learning models can vary substantially when evaluated on a distribution that even slightly differs from that of the training data. As a result, predicting model performance on previously unseen distributions without access to labeled data is an important challenge with implications for increasing the reliability of machine learning models. In the context of distribution shift, distance measures are often used to adapt models and improve their performance on new domains, however accuracy estimation is seldom explored in these investigations. Our investigation determines that common distributional distances such as Frechet distance or Maximum Mean Discrepancy, fail to induce reliable estimates of performance under distribution shift. On the other hand, we find that our proposed difference of confidences (DoC) approach yields successful estimates of a classifier’s performance over a variety of shifts and model architectures. Despite its simplicity, we observe that DoC outperforms other methods across synthetic, natural, and adversarial distribution shifts, reducing error by (> 46%) on several realistic and challenging datasets such as ImageNet-Vid-Robust and ImageNet-Rendition.
Hierarchical Object-to-Zone Graph for Object Navigation

The goal of object navigation is to reach the expected objects according to visual information in the unseen environments. Previous works usually implement deep models to train an agent to predict actions in real-time. However, in the unseen environment, when the target object is not in egocentric view, the agent may not be able to make wise decisions due to the lack of guidance. In this paper, we propose a hierarchical object-to-zone (HOZ) graph to guide the agent in a coarse-to-fine manner, and an online-learning mechanism is also proposed to update HOZ according to the real-time observation in new environments. In particular, the HOZ graph is composed of scene nodes, zone nodes and object nodes. With the pre-learned HOZ graph, the real-time observation and the target goal, the agent can constantly plan an optimal path from zone to zone. In the estimated path, the next potential zone is regarded as sub-goal, which is also fed into the deep reinforcement learning model for action prediction. Our methods are evaluated on the AI2-Thor simulator. In addition to widely used evaluation metrics SR and SPL, we also propose a new evaluation metric of SAE that focuses on the effective action rate. Experimental results demonstrate the effectiveness and efficiency of our proposed method. The code is available at https://github.com/sx-zhang/HOZ.git.
Motion Prediction using Trajectory Cues

Predicting human motion from a historical pose sequence is at the core of many applications in computer vision. Current state-of-the-art methods concentrate on learning motion contexts in the pose space, however, the high dimensionality and complex nature of human pose invoke inherent difficulties in extracting such contexts. In this paper, we instead advocate to model motion contexts in the joint trajectory space, as the trajectory of a joint is smooth, vectorial, and gives sufficient information to the model. Moreover, most existing methods consider only the dependencies between skeletal connected joints, disregarding prior knowledge and the hidden connections between geometrically separated joints. Motivated by this, we present a semi-constrained graph to explicitly encode skeletal connections and prior knowledge, while adaptively learn implicit dependencies between joints.We also explore the applications of our approach to a range of objects including human, fish, and mouse. Surprisingly, our method sets the new state-of-the-art performance on 4 different benchmark datasets, a remarkable highlight is that it achieves a 19.1% accuracy improvement over current state-of-the-art in average. To facilitate future research, we have released our code at https://github.com/Pose-Group/MPT.
Detecting Overlapped Objects in X-Ray Security Imagery by a Label-Aware Mechanism

One of the key challenges to the X-ray security check is to detect the overlapped items in backpacks or suitcases in the X-ray images. Most existing methods improve the robustness of models to the object overlapping problem by enhancing the underlying visual information such as colors and edges. However, this strategy ignores the situations that the objects have similar visual clues as to the background, and objects overlapping each other. Since the two cases rarely appear in existing datasets, we contribute a novel dataset – Cutters and Liquid Containers X-ray Dataset (CLCXray) to complete the related research. Furthermore, we propose a novel Label-aware Mechanism (LA) to tackle the object overlapping problem. Particularly, LA establishes the associations between feature channels and different labels and adjusts the features according to the assigned labels (or pseudo labels) to help improve the prediction results. Extensive experiments demonstrate that the LA is accurate and robust to detect overlapped objects, and also validate the effectiveness and the good generalization of the LA for arbitrary state-of-the-art (SOTA) methods. Furthermore, experimental results show that the network constructed by the LA is superior to the SOTA models on OPIXray and CLCXray, especially solving the challenges of the subset of the highly overlapped objects.
Improved Fault-Tolerant Consensus Based on the PBFT Algorithm

Nowadays Practical Byzantine Fault Tolerance (PBFT) algorithm has become the most extensive consensus algorithm in the alliance chain. However, the PBFT algorithm is usually only applicable to small networks due to high communication complexity and poor scalability. Although there have been many improved algorithms for PBFT in recent years, they ignore fault tolerance and democracy. Therefore, to meet the requirements of a high degree of decentralization and fault tolerance of blockchain-based scenarios. This paper proposes a high fault tolerance consensus algorithm NBFT, which follows the principle of decentralization and democratization of blockchain and ensures the improvement of performance in fault tolerance upper limit and scalability. First, we use the consistent hash algorithm to group the consensus nodes to avoid much communication between nodes, reduce the communication complexity of the network, and improve the scalability of the network. Second, to ensure the fault-tolerant ability of the grouping consensus, the nodal decision broadcast model and threshold vote-counting model are proposed first. Combined with the proposed two models, the joint fault analysis of nodes is carried out, and the fault tolerance upper limit is more than 1/3. Then, the Faulty Number Determined (FND) model is introduced to simulate the experiment, and the results are verified.
Dual Contrastive Loss and Attention for GANs

Generative Adversarial Networks (GANs) produce impressive results on unconditional image generation when powered with large-scale image datasets. Yet generated images are still easy to spot especially on datasets with high variance (e.g. bedroom, church). In this paper, we propose various improvements to further push the boundaries in image generation. Specifically, we propose a novel dual contrastive loss and show that, with this loss, discriminator learns more generalized and distinguishable representations to incentivize generation. In addition, we revisit attention and extensively experiment with different attention blocks in the generator. We find attention to be still an important module for successful image generation even though it was not used in the recent state-of-the-art models. Lastly, we study different attention architectures in the discriminator, and propose a reference attention mechanism. By combining the strengths of these remedies, we improve the compelling state-of-the-art Fréchet Inception Distance (FID) by at least 17.5% on several benchmark datasets. We obtain even more significant improvements on compositional synthetic scenes (up to 47.5% in FID).
An Implementation of the HDBSCAN* Clustering Algorithm

An implementation of the HDBSCAN* clustering algorithm, Tribuo Hdbscan, is presented in this work. The implementation is developed as a new feature of the Java machine learning library Tribuo. This implementation leverages concurrency and achieves better performance than the reference Java implementation. Tribuo Hdbscan provides prediction functionality, which is a novel technique to make fast predictions for unseen data points using an HDBSCAN* clustering model. Tribuo Hdbscan cluster results and performance measurements are also compared with the state-of-the-art HDBSCAN* implementation, the Python module hdbscan.
Improved deep convolutional embedded clustering with re-selectable sample training

• This paper proposes an improved deep convolutional embedded clustering algorithm using reliable samples. • This paper designs the new deep clustering model structure and corresponding loss function. • In this study, we select reliable samples with pseudo-labels and pass them to the convolutional neural network for training to get a better clustering model. • We conducted experimental tests on four standard data sets and show the better performance compared to the state-of-the-art clustering algorithms. The deep clustering algorithm can learn the latent features of the embedded subspace, and further realize the clustering of samples in the feature space. The existing deep clustering algorithms mostly integrate neural networks and traditional clustering algorithms. However, for sample sets with many noise points, the effect of the clustering remains unsatisfactory. To address this issue, we propose an improved deep convolutional embedded clustering algorithm using reliable samples (IDCEC) in this paper. The algorithm first uses the convolutional autoencoder to extract features and cluster the samples. Then we select reliable samples with pseudo-labels and pass them to the convolutional neural network for training to get a better clustering model. We construct a new loss function for backpropagation training and implement an unsupervised deep clustering method. To verify the performance of the method proposed in this paper, we conducted experimental tests on standard data sets such as MNIST and USPS. Experimental results show that our method has better performance compared to traditional clustering algorithms and the state-of-the-art deep clustering algorithm under four clustering metrics.
An Effective and Efficient Algorithm for K-Means Clustering With New Formulation

K-means is one of the most simple and popular clustering algorithms, which implemented as a standard clustering method in most of machine learning researches. The goal of K-means clustering is finding a set of cluster centers and minimizing the sum of squared distances between each sample and its nearest clustering center. In this paper, we proposed a novel K-means clustering algorithm, which reformulate the classical K-Means objective function as a trace maximization problem and then replace it with a new formulation. The proposed algorithm does not need to calculate the cluster centers in each iteration and requires fewer additional intermediate variables during the optimization process. In addition, we proposed an efficient iterative re-weighted algorithm to solve the involved optimization problem and provided the corresponding convergence analysis. The proposed algorithm keeps a consistent computational complexity as Lloyd's algorithm, <inline-formula><tex-math notation="LaTeX">$\mathcal {O}(ndk)$</tex-math></inline-formula> , but shows a faster convergence rate in experiments. Extensive experimental results on real world benchmark datasets show the effectiveness and efficiency of the proposed algorithm.
Medical Image Segmentation Using Transformer Networks

Deep learning models represent the state of the art in medical image segmentation. Most of these models are fully-convolutional networks (FCNs), namely each layer processes the output of the preceding layer with convolution operations. The convolution operation enjoys several important properties such as sparse interactions, parameter sharing, and translation equivariance. Because of these properties, FCNs possess a strong and useful inductive bias for image modeling and analysis. However, they also have certain important shortcomings, such as performing a fixed and pre-determined operation on a test image regardless of its content and difficulty in modeling long-range interactions. In this work we show that a different deep neural network architecture, based entirely on self-attention between neighboring image patches and without any convolution operations, can achieve more accurate segmentations than FCNs. Our proposed model is based directly on the transformer network architecture. Given a 3D image block, our network divides it into non-overlapping 3D patches and computes a 1D embedding for each patch. The network predicts the segmentation map for the block based on the self-attention between these patch embeddings. Furthermore, in order to address the common problem of scarcity of labeled medical images, we propose methods for pre-training this model on large corpora of unlabeled images. Our experiments show that the proposed model can achieve segmentation accuracies that are better than several state of the art FCN architectures on two datasets. Our proposed network can be trained using only tens of labeled images. Moreover, with the proposed pre-training strategies, our network outperforms FCNs when labeled training data is small.
Robust Quantum Optimal Control with Trajectory Optimization

The ability to engineer high-fidelity gates on quantum processors in the presence of systematic errors remains the primary barrier to achieving quantum advantage. Quantum optimal control methods have proven effective in experimentally realizing high-fidelity gates, but they require exquisite calibration to be performant. We apply robust trajectory optimization techniques to suppress gate errors arising from system parameter uncertainty. We propose a derivative-based approach that maintains computational efficiency by using forward-mode differentiation. Additionally, the effect of depolarization on a gate is typically modeled by integrating the Lindblad master equation, which is computationally expensive. We employ a computationally efficient model and utilize time-optimal control to achieve high-fidelity gates in the presence of depolarization. We apply these techniques to a fluxonium qubit and suppress simulated gate errors due to parameter uncertainty below ${10}^{\ensuremath{-}7}$ for static parameter deviations of the order of 1%.
Top Five Challenges in Software Supply Chain Security: Observations From 30 Industry and Government Organizations

Software is complex, not only due to the code within a given project, but also due to the vast ecosystem of dependencies and transitive dependencies upon which each project relies. Recent years have observed a sharp uptick of attacks on the software supply chain spurring invigorated interest by industry and government alike. We held three summits with a diverse set of organizations and report on the top five challenges in software supply chain security.
Deep-learning-based super-resolution reconstruction of high-speed imaging in fluids

In many fluid experiments, we can only obtain low-spatial high-temporal resolution flow images and high-spatial low-temporal resolution flow images due to the limitation of high-speed imaging systems. To solve this problem, we proposed a degradation and super-resolution attention model (D-SRA) using unsupervised machine learning to super-resolution reconstruct high resolution (HR) time-resolved fluid images from coarse data. Unlike the prior research to increase the resolution of coarse data artificially generated by simple bilinear down-sampling, our model that consists of a degradation neural network and a super-resolution neural network aims to learn the mappings between experimental low-resolution data and corresponding HR data. What is more, channel and spatial attention modules are also adopted in D-SRA to facilitate the restoration of abundant and critical details of flow fields. The proposed model is validated by two high-speed schlieren experiments of under-expanded impinging supersonic jets. The comprehensive capability of D-SRA is statistically analyzed based on the synthetic unpaired schlieren images. The spatial-resolution of coarse images can be successfully augmented by 42 times and 82 times with most physical details recovered perfectly, which outperforms the existing method. The D-SRA also exhibits considerable generalization and robustness against unknown-degenerated schlieren images. Moreover, the practicability of the proposed method is also further explored on real unpaired jets schlieren images. It is convincingly demonstrated that the present study successfully surpasses the performance limitations of high-speed cameras and has significant applications in various fluid experiments to obtain flow images with high spatial and temporal resolution.
Adoption of Health Mobile Apps during the COVID-19 Lockdown: A Health Belief Model Approach

This study aimed to investigate the adoption of the Sehha, Mawid, and Tetamman mobile health applications during the COVID-19 pandemic in Saudi Arabia. The present study investigated factors influencing app use intention based on the Health Belief Model (HBM) approach. This study was conducted using a sample of 176 participants from the Riyadh and Makkah regions during the lockdown in May 2020. This study uses structural equation modeling for data collected using SmartPLS 3.3.9 (GmbH, Oststeinbek, Germany) to examine the effect of constructs on the model. The most important predictor was the perceived benefits of the mobile health apps, followed by self-efficacy. The perceived barriers and cues to action have no significant effect on behavioral intention. The perceived benefits and self-efficacy as keys can provide an overview to the government and to health organizations for taking into account the most important factors of the adoption of mobile health apps, meaning that the developer must adjust to the characteristics of the community of people that need applications that provide many benefits and have an impact.
ResT-ReID: Transformer block-based residual learning for person re-identification

The Transformer has been applied into computer vision to explore long-range dependencies with multi-head self-attention strategy, therefore numerous Transformer-based methods for person re-identification (ReID) are designed for extracting effective as well as robust representation. However, the memory and computational complexity of scaled dot-product attention in Transformer cost vast overheads. To overcome these limitations, this paper presents ResT-ReID method, which designs a hybrid backbone Res-Transformer based on ResNet-50 and Transformer block for effective identify information. Specifically, we use global self-attention in place of depth-wise convolution in the fourth layer’s residual bottleneck of ResNet-50. For fully exploiting the entire knowledge of the person, we devise attention-guided Graph Convolution Networks (GCNs) with side information embedding (SIE-AGCN), which has an attention layer located into two GCN layers. The quantified experiments on two large-scale ReID benchmarks demonstrate that the proposed ResT-ReID achieves competitive results compared with several state-of-the-art approaches.