Reviewerly/exHarmony huggingface dataset:
Federated Learning via Unmanned Aerial Vehicle

To enable communication-efficient federated learning (FL), this paper studies an unmanned aerial vehicle (UAV)-enabled FL system, where the UAV coordinates distributed ground devices for a shared model training. Specifically, by exploiting the UAV's high altitude and mobility, the UAV can proactively establish short-distance line-of-sight links with devices and prevent any device from being a communication straggler. Thus, the model aggregation process can be accelerated while the cumulative model loss caused by device scheduling can be reduced, resulting in a decreased completion time. We first present the convergence analysis of FL without the assumption of convexity, demonstrating the effect of device scheduling on the global gradients. Based on the derived convergence bound, we further formulate the completion time minimization problem by jointly optimizing device scheduling, UAV trajectory, and time allocation. This problem explicitly incorporates the devices' energy budgets, dynamic channel conditions, and convergence accuracy of FL constraints. Despite the non-convexity of the formulated problem, we exploit its structure to decompose it into two sub-problems and further derive the optimal solutions via the Lagrange dual ascent method. Simulation results show that the proposed design significantly improves the tradeoff between completion time and prediction accuracy in practical FL settings compared to existing benchmarks.
Improving Segmentation of Breast Ultrasound Images: Semi Automatic Two Pointers Histogram Splitting Technique

Automatically segmenting lesion area in breast ultrasound (BUS) images is a challenging one due to its noise, speckle and artifacts. Edge-map of BUS images also does not help because in most cases the edge-map gives no information whatsoever. Almost all segmentation technique takes the edge-map of the image as its first step, though there are a few algorithms that try to avoid edge-maps as well. Improving the edge-map of breast ultrasound images theoretically improves the chances of automatic segmentation to be more precise. In this paper, we propose a semi-automatic technique of histogram splitting using two pointers. Here the user only has to select two initially guessed points denoting a circle on the region of interest (ROI). The method will automatically study the internal histogram and split it using two pointers. The output BUS image has improved edge-map and ultimately the segmentation on it is better compared to regular segmentation using same algorithm and same initialization. Also, we further processed the edge-map to have less edge-pixels to area ratio, improving the homogeneity and the chances of easy segmentation in the future.
Visual-Semantic Contrastive Alignment for Few-Shot Image Classification

Few-Shot learning aims to train and optimize a model that can adapt to unseen visual classes with only a few labeled examples. The existing few-shot learning (FSL) methods, heavily rely only on visual data, thus fail to capture the semantic attributes to learn a more generalized version of the visual concept from very few examples. However, it is a known fact that human visual learning benefits immensely from inputs from multiple modalities such as vision, language, and audio. Inspired by the human learning nature of encapsulating the existing knowledge of a visual category which is in the form of language, we introduce a contrastive alignment mechanism for visual and semantic feature vectors to learn much more generalized visual concepts for few-shot learning. Our method simply adds an auxiliary contrastive learning objective which captures the contextual knowledge of a visual category from a strong textual encoder in addition to the existing training mechanism. Hence, the approach is more generalized and can be plugged into any existing FSL method. The pre-trained semantic feature extractor (learned from a large-scale text corpora) we use in our approach provides a strong contextual prior knowledge to assist FSL. The experimental results done in popular FSL datasets show that our approach is generic in nature and provides a strong boost to the existing FSL baselines.
Single Image Super-Resolution Using Lightweight Networks Based on Swin Transformer

Image super-resolution reconstruction is an important task in the field of image processing technology, which can restore low resolution image to high quality image with high resolution. In recent years, deep learning has been applied in the field of image super-resolution reconstruction. With the continuous development of deep neural network, the quality of the reconstructed images has been greatly improved, but the model complexity has also been increased. In this paper, we propose two lightweight models named as MSwinSR and UGSwinSR based on Swin Transformer. The most important structure in MSwinSR is called Multi-size Swin Transformer Block (MSTB), which mainly contains four parallel multi-head self-attention (MSA) blocks. UGSwinSR combines U-Net and GAN with Swin Transformer. Both of them can reduce the model complexity, but MSwinSR can reach a higher objective quality, while UGSwinSR can reach a higher perceptual quality. The experimental results demonstrate that MSwinSR increases PSNR by $\mathbf{0.07dB}$ compared with the state-of-the-art model SwinIR, while the number of parameters can reduced by $\mathbf{30.68\%}$, and the calculation cost can reduced by $\mathbf{9.936\%}$. UGSwinSR can effectively reduce the amount of calculation of the network, which can reduced by $\mathbf{90.92\%}$ compared with SwinIR.
Neural Estimation of Submodular Functions with Applications to Differentiable Subset Selection

Submodular functions and variants, through their ability to characterize diversity and coverage, have emerged as a key tool for data selection and summarization. Many recent approaches to learn submodular functions suffer from limited expressiveness. In this work, we propose FLEXSUBNET, a family of flexible neural models for both monotone and non-monotone submodular functions. To fit a latent submodular function from (set, value) observations, FLEXSUBNET applies a concave function on modular functions in a recursive manner. We do not draw the concave function from a restricted family, but rather learn from data using a highly expressive neural network that implements a differentiable quadrature procedure. Such an expressive neural model for concave functions may be of independent interest. Next, we extend this setup to provide a novel characterization of monotone \alpha-submodular functions, a recently introduced notion of approximate submodular functions. We then use this characterization to design a novel neural model for such functions. Finally, we consider learning submodular set functions under distant supervision in the form of (perimeter-set, high-value-subset) pairs. This yields a novel subset selection method based on an order-invariant, yet greedy sampler built around the above neural set functions. Our experiments on synthetic and real data show that FLEXSUBNET outperforms several baselines.
Representation Learning with Diffusion Models

Diffusion models (DMs) have achieved state-of-the-art results for image synthesis tasks as well as density estimation. Applied in the latent space of a powerful pretrained autoencoder (LDM), their immense computational requirements can be significantly reduced without sacrificing sampling quality. However, DMs and LDMs lack a semantically meaningful representation space as the diffusion process gradually destroys information in the latent variables. We introduce a framework for learning such representations with diffusion models (LRDM). To that end, a LDM is conditioned on the representation extracted from the clean image by a separate encoder. In particular, the DM and the representation encoder are trained jointly in order to learn rich representations specific to the generative denoising process. By introducing a tractable representation prior, we can efficiently sample from the representation distribution for unconditional image synthesis without training of any additional model. We demonstrate that i) competitive image generation results can be achieved with image-parameterized LDMs, ii) LRDMs are capable of learning semantically meaningful representations, allowing for faithful image reconstructions and semantic interpolations. Our implementation is available at https://github.com/jeremiastraub/diffusion.
Freeze then Train: Towards Provable Representation Learning under Spurious Correlations and Feature Noise

The existence of spurious correlations such as image backgrounds in the training environment can make empirical risk minimization (ERM) perform badly in the test environment. To address this problem, Kirichenko et al. (2022) empirically found that the core features that are related to the outcome can still be learned well even with the presence of spurious correlations. This opens a promising strategy to first train a feature learner rather than a classifier, and then perform linear probing (last layer retraining) in the test environment. However, a theoretical understanding of when and why this approach works is lacking. In this paper, we find that core features are only learned well when their associated non-realizable noise is smaller than that of spurious features, which is not necessarily true in practice. We provide both theories and experiments to support this finding and to illustrate the importance of non-realizable noise. Moreover, we propose an algorithm called Freeze then Train (FTT), that first freezes certain salient features and then trains the rest of the features using ERM. We theoretically show that FTT preserves features that are more beneficial to test time probing. Across two commonly used spurious correlation datasets, FTT outperforms ERM, IRM, JTT and CVaR-DRO, with substantial improvement in accuracy (by 4.5%) when the feature noise is large. FTT also performs better on general distribution shift benchmarks.
Robust Image Registration with Absent Correspondences in Pre-operative and Follow-up Brain MRI Scans of Diffuse Glioma Patients

Registration of pre-operative and follow-up brain MRI scans is challenging due to the large variation of tissue appearance and missing correspondences in tumour recurrence regions caused by tumour mass effect. Although recent deep learning-based deformable registration methods have achieved remarkable success in various medical applications, most of them are not capable of registering images with pathologies. In this paper, we propose a 3-step registration pipeline for pre-operative and follow-up brain MRI scans that consists of 1) a multi-level affine registration, 2) a conditional deep Laplacian pyramid image registration network (cLapIRN) with forward-backward consistency constraint, and 3) a non-linear instance optimization method. We apply the method to the Brain Tumor Sequence Registration (BraTS-Reg) Challenge. Our method achieves accurate and robust registration of brain MRI scans with pathologies, which achieves a median absolute error of 1.64 mm and 88\% of successful registration rate in the validation set of BraTS-Reg challenge. Our method ranks 1st place in the 2022 MICCAI BraTS-Reg challenge.
Sequential Quantum Channel Discrimination

We consider the sequential quantum channel discrimination problem using adaptive and non-adaptive strategies. In this setting the number of uses of the underlying quantum channel is not fixed but a random variable that is either bounded in expectation or with high probability. We show that both types of error probabilities decrease to zero exponentially fast and, when using adaptive strategies, the rates are characterized by the measured relative entropy between two quantum channels, yielding a strictly larger region than that achievable by non-adaptive strategies. Allowing for quantum memory, we see that the optimal rates are given by the regularized channel relative entropy. Finally, we discuss achievable rates when allowing for repeated measurements via quantum instruments and conjecture that the achievable rate region is not larger than that achievable with POVMs by connecting the result to the strong converse for the quantum channel Stein's Lemma.
Pruning by Active Attention Manipulation

Filter pruning of a CNN is typically achieved by applying discrete masks on the CNN's filter weights or activation maps, post-training. Here, we present a new filter-importance-scoring concept named pruning by active attention manipulation (PAAM), that sparsifies the CNN's set of filters through a particular attention mechanism, during-training. PAAM learns analog filter scores from the filter weights by optimizing a cost function regularized by an additive term in the scores. As the filters are not independent, we use attention to dynamically learn their correlations. Moreover, by training the pruning scores of all layers simultaneously, PAAM can account for layer inter-dependencies, which is essential to finding a performant sparse sub-network. PAAM can also train and generate a pruned network from scratch in a straightforward, one-stage training process without requiring a pre-trained network. Finally, PAAM does not need layer-specific hyperparameters and pre-defined layer budgets, since it can implicitly determine the appropriate number of filters in each layer. Our experimental results on different network architectures suggest that PAAM outperforms state-of-the-art structured-pruning methods (SOTA). On CIFAR-10 dataset, without requiring a pre-trained baseline network, we obtain 1.02% and 1.19% accuracy gain and 52.3% and 54% parameters reduction, on ResNet56 and ResNet110, respectively. Similarly, on the ImageNet dataset, PAAM achieves 1.06% accuracy gain while pruning 51.1% of the parameters on ResNet50. For Cifar-10, this is better than the SOTA with a margin of 9.5% and 6.6%, respectively, and on ImageNet with a margin of 11%.
Vine copula based knockoff generation for high-dimensional controlled variable selection

Vine copulas are a flexible tool for high-dimensional dependence modeling. In this article, we discuss the generation of approximate model-X knockoffs with vine copulas. It is shown how Gaussian knockoffs can be generalized to Gaussian copula knockoffs. A convenient way to parametrize Gaussian copulas are partial correlation vines. We discuss how completion problems for partial correlation vines are related to Gaussian knockoffs. A natural generalization of partial correlation vines are vine copulas which are well suited for the generation of approximate model-X knockoffs. We discuss a specific D-vine structure which is advantageous to obtain vine copula knockoff models. In a simulation study, we demonstrate that vine copula knockoff models are effective and powerful for high-dimensional controlled variable selection.
Knowledge-Enhanced Relation Extraction Dataset

Recently, knowledge-enhanced methods leveraging auxiliary knowledge graphs have emerged in relation extraction, surpassing traditional text-based approaches. However, to our best knowledge, there is currently no public dataset available that encompasses both evidence sentences and knowledge graphs for knowledge-enhanced relation extraction. To address this gap, we introduce the Knowledge-Enhanced Relation Extraction Dataset (KERED). KERED annotates each sentence with a relational fact, and it provides knowledge context for entities through entity linking. Using our curated dataset, We compared contemporary relation extraction methods under two prevalent task settings: sentence-level and bag-level. The experimental result shows the knowledge graphs provided by KERED can support knowledge-enhanced relation extraction methods. We believe that KERED offers high-quality relation extraction datasets with corresponding knowledge graphs for evaluating the performance of knowledge-enhanced relation extraction methods. Our dataset is available at: \url{https://figshare.com/projects/KERED/134459}
Modelling and Explaining Legal Case-based Reasoners through Classifiers

This paper brings together two lines of research: factor-based models of case-based reasoning (CBR) and the logical specification of classifiers. Logical approaches to classifiers capture the connection between features and outcomes in classifier systems. Factor-based reasoning is a popular approach to reasoning by precedent in AI & Law. Horty (2011) has developed the factor-based models of precedent into a theory of precedential constraint. In this paper we combine the modal logic approach (binary-input classifier, BLC) to classifiers and their explanations given by Liu & Lorini (2021) with Horty's account of factor-based CBR, since both a classifier and CBR map sets of features to decisions or classifications. We reformulate case bases of Horty in the language of BCL, and give several representation results. Furthermore, we show how notions of CBR, e.g. reason, preference between reasons, can be analyzed by notions of classifier system.
Context-driven Visual Object Recognition based on Knowledge Graphs

Current deep learning methods for object recognition are purely data-driven and require a large number of training samples to achieve good results. Due to their sole dependence on image data, these methods tend to fail when confronted with new environments where even small deviations occur. Human perception, however, has proven to be significantly more robust to such distribution shifts. It is assumed that their ability to deal with unknown scenarios is based on extensive incorporation of contextual knowledge. Context can be based either on object co-occurrences in a scene or on memory of experience. In accordance with the human visual cortex which uses context to form different object representations for a seen image, we propose an approach that enhances deep learning methods by using external contextual knowledge encoded in a knowledge graph. Therefore, we extract different contextual views from a generic knowledge graph, transform the views into vector space and infuse it into a DNN. We conduct a series of experiments to investigate the impact of different contextual views on the learned object representations for the same image dataset. The experimental results provide evidence that the contextual views influence the image representations in the DNN differently and therefore lead to different predictions for the same images. We also show that context helps to strengthen the robustness of object recognition models for out-of-distribution images, usually occurring in transfer learning tasks or real-world scenarios.
Efficient inference in the transverse field Ising model

In this paper we introduce an approximate method to solve the quantum cavity equations for transverse field Ising models. The method relies on a projective approximation of the exact cavity distributions of imaginary time trajectories (paths). A key feature, novel in the context of similar algorithms, is the explicit separation of the classical and quantum parts of the distributions. Numerical simulations show accurate results in comparison with the sampled solution of the cavity equations, the exact diagonalization of the Hamiltonian (when possible) and other approximate inference methods in the literature. The computational complexity of this new algorithm scales linearly with the connectivity of the underlying lattice, enabling the study of highly connected networks, as the ones often encountered in quantum machine learning problems.
Strong Normalization for the Calculus of Constructions

The calculus of constructions (CC) is a core theory for dependently typed programming and higher-order constructive logic. Originally introduced in Coquand's 1985 thesis, CC has inspired 25 years of research in programming languages and type theory. Today, extensions of CC form the basis of languages like Coq and Agda. This survey reviews three proofs of CC's strong normalization property (the fact that there are no infinite reduction sequences from well-typed expressions). It highlights the similarities in the structure of the proofs while showing how their differences are motivated by the varying goals of their authors.
A Dialogue Robot System to Improve Credibility in Sightseeing Spot Recommendations

Various studies have been conducted on human-supporting robot systems. These systems have been put to practical use over the years and are now seen in our daily lives. In particular, robots communicating smoothly with people are expected to play an active role in customer service and guidance. In this case, it is essential to determine whether the customer is satisfied with the dialog robot or not. However, it is not easy to satisfy all of the customer's requests due to the diversity of the customer's speech. In this study, we developed a dialog mechanism that prevents dialog breakdowns and keeps the customer satisfied by providing multiple scenarios for the robot to take control of the dialog. We tested it in a travel destination recommendation task at a travel agency.
Image Semantic Relation Generation

Scene graphs provide structured semantic understanding beyond images. For downstream tasks, such as image retrieval, visual question answering, visual relationship detection, and even autonomous vehicle technology, scene graphs can not only distil complex image information but also correct the bias of visual models using semantic-level relations, which has broad application prospects. However, the heavy labour cost of constructing graph annotations may hinder the application of PSG in practical scenarios. Inspired by the observation that people usually identify the subject and object first and then determine the relationship between them, we proposed to decouple the scene graphs generation task into two sub-tasks: 1) an image segmentation task to pick up the qualified objects. 2) a restricted auto-regressive text generation task to generate the relation between given objects. Therefore, in this work, we introduce image semantic relation generation (ISRG), a simple but effective image-to-text model, which achieved 31 points on the OpenPSG dataset and outperforms strong baselines respectively by 16 points (ResNet-50) and 5 points (CLIP).
Real-World Chaos-Based Cryptography Using Synchronised Chua Chaotic Circuits

This work presents the hardware demonstrator of a secure encryption system based on synchronised Chua chaotic circuits. In particular, the presented encryption system comprises two Chua circuits that are synchronised using a dedicated bidirectional synchronisation line. One of them forms part of the transmitter, while the other of the receiver. Both circuits are tuned to operate in a chaotic mode. The output (chaotic) signal of the first circuit (transmitter) is digitised and then combined with the message to be encrypted, through an XOR gate. The second Chua circuit (receiver) is used for the decryption; the output chaotic signal of this circuit is similarly digitised and combined with the encrypted message to retrieve the original message. Our hardware demonstrator proves that this method can be used in order to provide extremely lightweight real-world, chaos-based cryptographic solutions.
Accelerating equilibrium spin-glass simulations using quantum annealers via generative deep learning

Adiabatic quantum computers, such as the quantum annealers commercialized by D-Wave Systems Inc., are routinely used to tackle combinatorial optimization problems. In this article, we show how to exploit them to accelerate equilibrium Markov chain Monte Carlo simulations of computationally challenging spin-glass models at low but finite temperatures. This is achieved by training generative neural networks on data produced by a D-Wave quantum annealer, and then using them to generate smart proposals for the Metropolis-Hastings algorithm. In particular, we explore hybrid schemes by combining single spin-flip and neural proposals, as well as D-Wave and classical Monte Carlo training data. The hybrid algorithm outperforms the single spin-flip Metropolis-Hastings algorithm. It is competitive with parallel tempering in terms of correlation times, with the significant benefit of a much shorter equilibration time.