Miracle-dz/UniFAR-FTUs huggingface dataset:
DAVE: A Deep Audio-Visual Embedding for Dynamic Saliency Prediction

This paper studies audio-visual deep saliency prediction. It introduces a conceptually simple and effective Deep Audio-Visual Embedding for dynamic saliency prediction dubbed ``DAVE" in conjunction with our efforts towards building an Audio-Visual Eye-tracking corpus named ``AVE". Despite existing a strong relation between auditory and visual cues for guiding gaze during perception, video saliency models only consider visual cues and neglect the auditory information that is ubiquitous in dynamic scenes. Here, we investigate the applicability of audio cues in conjunction with visual ones in predicting saliency maps using deep neural networks. To this end, the proposed model is intentionally designed to be simple. Two baseline models are developed on the same architecture which consists of an encoder-decoder. The encoder projects the input into a feature space followed by a decoder that infers saliency. We conduct an extensive analysis on different modalities and various aspects of multi-model dynamic saliency prediction. Our results suggest that (1) audio is a strong contributing cue for saliency prediction, (2) salient visible sound-source is the natural cause of the superiority of our Audio-Visual model, (3) richer feature representations for the input space leads to more powerful predictions even in absence of more sophisticated saliency decoders, and (4) Audio-Visual model improves over 53.54\% of the frames predicted by the best Visual model (our baseline). Our endeavour demonstrates that audio is an important cue that boosts dynamic video saliency prediction and helps models to approach human performance. The code is available at this https URL
Learning by stochastic serializations

Complex structures are typical in machine learning. Tailoring learning algorithms for every structure requires an effort that may be saved by defining a generic learning procedure adaptive to any complex structure. In this paper, we propose to map any complex structure onto a generic form, called serialization, over which we can apply any sequence-based density estimator. We then show how to transfer the learned density back onto the space of original structures. To expose the learning procedure to the structural particularities of the original structures, we take care that the serializations reflect accurately the structures’ properties. Enumerating all serializations is infeasible. We propose an effective way to sample representative serializations from the complete set of serializations which preserves the statistics of the complete set. Our method is competitive or better than state of the art learning algorithms that have been specifically designed for given structures. In addition, since the serialization involves sampling from a combinatorial process it provides considerable protection from overfitting, which we clearly demonstrate on a number of experiments.
Communication-Efficient Distributed Blockwise Momentum SGD with Error-Feedback

Communication overhead is a major bottleneck hampering the scalability of distributed machine learning systems. Recently, there has been a surge of interest in using gradient compression to improve the communication efficiency of distributed neural network training. Using 1-bit quantization, signSGD with majority vote achieves a 32x reduction on communication cost. However, its convergence is based on unrealistic assumptions and can diverge in practice. In this paper, we propose a general distributed compressed SGD with Nesterov's momentum. We consider two-way compression, which compresses the gradients both to and from workers. Convergence analysis on nonconvex problems for general gradient compressors is provided. By partitioning the gradient into blocks, a blockwise compressor is introduced such that each gradient block is compressed and transmitted in 1-bit format with a scaling factor, leading to a nearly 32x reduction on communication. Experimental results show that the proposed method converges as fast as full-precision distributed momentum SGD and achieves the same testing accuracy. In particular, on distributed ResNet training with 7 workers on the ImageNet, the proposed algorithm achieves the same testing accuracy as momentum SGD using full-precision gradients, but with $46\%$ less wall clock time.
Finding Task-Relevant Features for Few-Shot Learning by Category Traversal

Few-shot learning is an important area of research. Conceptually, humans are readily able to understand new concepts given just a few examples, while in more pragmatic terms, limited-example training situations are common practice. Recent effective approaches to few-shot learning employ a metric-learning framework to learn a feature similarity comparison between a query (test) example, and the few support (training) examples. However, these approaches treat each support class independently from one another, never looking at the entire task as a whole. Because of this, they are constrained to use a single set of features for all possible test-time tasks, which hinders the ability to distinguish the most relevant dimensions for the task at hand. In this work, we introduce a Category Traversal Module that can be inserted as a plug-and-play module into most metric-learning based few-shot learners. This component traverses across the entire support set at once, identifying task-relevant features based on both intra-class commonality and inter-class uniqueness in the feature space. Incorporating our module improves performance considerably (5%-10% relative) over baseline systems on both miniImageNet and tieredImageNet benchmarks, with overall performance competitive with the most recent state-of-the-art systems.
Shared Feelings: Understanding Facebook Reactions to Scholarly Articles

Research on social-media platforms has tended to rely on textual analysis to perform research tasks. While text-based approaches have significantly increased our understanding of online behavior and social dynamics, they overlook features on these platforms that have grown in prominence in the past few years: click-based responses to content. In this paper, we present a new dataset of Facebook Reactions to scholarly content. We give an overview of its structure, analyze some of the statistical trends in the data, and use it to train and test two supervised learning algorithms. Our preliminary tests suggest the presence of stratification in the number of users following pages, divisions that seem to fall in line with distinctions in the subject matter of those pages.
Additive manufacturing for high precision structural properties via feedback control

This paper discusses the possibility of making an object that precisely meets global structural requirements using additive manufacturing and feedback control. An experimental validation is presented by printing a cantilever beam with a prescribed stiffness requirement. The printing process is formalized as a model-based finite-horizon discrete control problem, where the control variables are the widths of the successive layers. Sensing is performed by making {\em in situ} intermediate stiffness measurements on the partially built part. The hypothesis that feedback control is effective in enabling the 3D-printed beam to meet precise stiffness requirements is validated experimentally.
Interpretable deep Gaussian processes

We propose interpretable deep Gaussian Processes (GPs) that combine the expressiveness of deep Neural Networks (NNs) with quantified uncertainty of deep GPs. Our approach is based on approximating deep GP as a GP, which allows explicit, analytic forms for compositions of a wide variety of kernels. Consequently, our approach admits interpretation as both NNs with specified activation functions and as a variational approximation to deep GPs. We provide general recipes for deriving the effective kernels for deep GPs of two, three, or infinitely many layers, composed of homogeneous or heterogeneous kernels. Results illustrate the expressiveness of our effective kernels through samples from the prior and inference on simulated data and demonstrate advantages of interpretability by analysis of analytic forms, drawing relations and equivalences across kernels, and a priori identification of non-pathological regimes of hyperparameter space.
Enhancing ML Robustness Using Physical-World Constraints

Recent advances in Machine Learning (ML) have demonstrated that neural networks can exceed human performance in many tasks. While generalizing well over natural inputs, neural networks are vulnerable to adversarial inputs -an input that is ``similar'' to the original input, but misclassified by the model. Existing defenses focus on Lp-norm bounded adversaries that perturb ML inputs in the digital space. In the real world, however, attackers can generate adversarial perturbations that have a large Lp-norm in the digital space. Additionally, these defenses also come at a cost to accuracy, making their applicability questionable in the real world. To defend models against such a powerful adversary, we leverage one constraint on its power: the perturbation should not change the human's perception of the physical information; the physical world places some constraints on the space of possible attacks. Two questions follow: how to extract and model these constraints? and how to design a classification paradigm that leverages these constraints to improve robustness accuracy trade-off? We observe that an ML model is typically a part of a larger system with access to different input modalities. Utilizing these modalities, we introduce invariants that limit the attacker's action space. We design a hierarchical classification paradigm that enforces these invariants at inference time. As a case study, we implement and evaluate our proposal in the context of the real-world application of road sign classification because of its applicability to autonomous driving. With access to different input modalities, such as LiDAR, camera, and location we show how to extract invariants and develop a hierarchical classifier. Our results on the KITTI and GTSRB datasets show that we can improve the robustness against physical attacks at minimal harm to accuracy.
Construction of genuine multipartite entangled states

Genuine multipartite entanglement is of great importance in quantum information, especially from the experimental point of view. Nevertheless, it is difficult to characterize genuine multipartite entangled states, because the genuine multipartite entanglement is unruly. There are scattered results on the criteria for genuine multipartite entangled states. We propose another product based on the Kronecker product in this paper. The Kronecker product is a common product in quantum information with good physical interpretation. We mainly investigate whether the proposed product of two genuine multipartite entangled states is still a genuine entangled one. We understand the entanglement of the proposed product better by characterizing the entanglement of the Kronecker product. Then we show the proposed product is a genuine multipartite entangled state in two cases. The results develop nontrivial criteria for a class of genuine multipartite entangled states. One can use such criteria to systematically construct genuine multipartite entangled states of more parties.
On Learning Over-parameterized Neural Networks: A Functional Approximation Prospective

We consider training over-parameterized two-layer neural networks with Rectified Linear Unit (ReLU) using gradient descent (GD) method. Inspired by a recent line of work, we study the evolutions of network prediction errors across GD iterations, which can be neatly described in a matrix form. When the network is sufficiently over-parameterized, these matrices individually approximate {\em an} integral operator which is determined by the feature vector distribution $\rho$ only. Consequently, GD method can be viewed as {\em approximately} applying the powers of this integral operator on the underlying/target function $f^*$ that generates the responses/labels. We show that if $f^*$ admits a low-rank approximation with respect to the eigenspaces of this integral operator, then the empirical risk decreases to this low-rank approximation error at a linear rate which is determined by $f^*$ and $\rho$ only, i.e., the rate is independent of the sample size $n$. Furthermore, if $f^*$ has zero low-rank approximation error, then, as long as the width of the neural network is $\Omega(n\log n)$, the empirical risk decreases to $\Theta(1/\sqrt{n})$. To the best of our knowledge, this is the first result showing the sufficiency of nearly-linear network over-parameterization. We provide an application of our general results to the setting where $\rho$ is the uniform distribution on the spheres and $f^*$ is a polynomial. Throughout this paper, we consider the scenario where the input dimension $d$ is fixed.
Plasmonic optical tweezers based on nanostructures: fundamentals, advances and prospects

Abstract The ability of metallic nanostructures to confine light at the sub-wavelength scale enables new perspectives and opportunities in the field of nanotechnology. Making use of this unique advantage, nano-optical trapping techniques have been developed to tackle new challenges in a wide range of areas from biology to quantum optics. In this work, starting from basic theories, we present a review of research progress in near-field optical manipulation techniques based on metallic nanostructures, with an emphasis on some of the most promising advances in molecular technology, such as the precise control of single biomolecules. We also provide an overview of possible future research directions of nanomanipulation techniques.
A retrospective analysis of mathieu and tip urethroplasty techniques for distal hypospadias repair; A 20 year experience.

OBJECTIVE We aimed to identify the changes in the application rate of two surgical techniques in distal hypospadias repair in years and compare the most popular two surgical repair techniques for distal hypospadias in terms of surgical outcomes, the factors that affect the outcomes, which were performed over a 20 year period. METHODS In this study, the records of 492 consecutive patients that had undergone an operation for distal hypospadias in the urology clinic of Ankara between May 1990 and December 2010 using either Mathieu or TIPU surgical techniques were reviewed retrospectively. The patients who had glanular, coronal, and subcoronal meatus, were accepted as distal hypospadias cases. Among the 492 examined medical records, it was revealed that 331 and 161 surgical interventions were performed by using the Mathieu urethroplasty technique (Group-1) and TIP urethroplasty technique (Group-2), respectively. Group-1 was divided into two subgroups; namely Group-1a (patients with primary hypospadias) and Group-1b (patients with previous hypospadias operation). Likewise, Group-2 was divided into two subgroups; namely group-2a and group-2b. The patients' ages, number of previously urethroplasty operations, localization of the external urethral meatus prior to the operation, chordee state, length of the newly formed urethra, whether urinary diversion was done or not, post-operative complications and data regarding the follow-up period were evaluated, and the effects of these variables on the surgical outcome were investigated via statistical analyses. The primary objective of this study is to identify the changes in the application rate of two surgical techniques in distal hypospadias repair over the a 20 year period, and the secondary objectives are to compare the most popular two surgical repair techniques for distal hypospadias in terms of surgical outcomes, and the factors affecting the outcomes. Independent samples t test and Pearson's Chisquare test was used for statistical analysis. p<0.05 was considered as statistically significant. RESULTS There were no statistically significant differences between the subgroups in terms of age, length of the neo-urethra, number of previously performed urethroplasty operations, surgical success rates, or complications (p>0.05). The concurrent utilization of the cystostomy and urethral stent was significantly more frequent in group-1 (p<0.05; Pearson's Chi-square test). It was determined that over time, TIP urethroplasty has become a more preferred technique for the repair of distal hypospadias. CONCLUSIONS Both surgical techniques have similar success rates in distal hypospadias cases. TIP urethroplasty has become the method of choice over time.
On approximating dropout noise injection

This paper examines the assumptions of the derived equivalence between dropout noise injection and $L_2$ regularisation for logistic regression with negative log loss. We show that the approximation method is based on a divergent Taylor expansion, making, subsequent work using this approximation to compare the dropout trained logistic regression model with standard regularisers unfortunately ill-founded to date. Moreover, the approximation approach is shown to be invalid using any robust constraints. We show how this finding extends to general neural network topologies that use a cross-entropy prediction layer.
A Reference Architecture for Blockchain-based Peer-to-Peer IoT Applications

The advent of Blockchain and Distributed Ledger Technologies enable IoT and smart city application developers to conceive new types of applications and solutions for identity management, trust, and data monetization. However, architecting blockchain-based IoT applications remain challenging due to the heterogeneous nature of blockchain platforms and lack of guidelines on how to interface existing components in the IoT ecosystem with the emerging Blockchain technology. This article explains the characteristics of blockchain and IoT technologies and presents a general reference architecture that can be used to develop many blockchain-based peer-to-peer IoT applications.
Semi-supervised GAN for Classification of Multispectral Imagery Acquired by UAVs

Unmanned aerial vehicles (UAV) are used in precision agriculture (PA) to enable aerial monitoring of farmlands. Intelligent methods are required to pinpoint weed infestations and make optimal choice of pesticide. UAV can fly a multispectral camera and collect data. However, the classification of multispectral images using supervised machine learning algorithms such as convolutional neural networks (CNN) requires large amount of training data. This is a common drawback in deep learning we try to circumvent making use of a semi-supervised generative adversarial networks (GAN), providing a pixel-wise classification for all the acquired multispectral images. Our algorithm consists of a generator network that provides photo-realistic images as extra training data to a multi-class classifier, acting as a discriminator and trained on small amounts of labeled data. The performance of the proposed method is evaluated on the weedNet dataset consisting of multispectral crop and weed images collected by a micro aerial vehicle (MAV). The results by the proposed semi-supervised GAN achieves high classification accuracy and demonstrates the potential of GAN-based methods for the challenging task of multispectral image classification.
Purely Combinatorial Algorithms for Approximate Directed Minimum Degree Spanning Trees

Given a directed graph $G$ on $n$ vertices with a special vertex $s$, the directed minimum degree spanning tree problem requires computing a incoming spanning tree rooted at $s$ whose maximum tree in-degree is the smallest among all such trees. The problem is known to be NP-hard, since it generalizes the Hamiltonian path problem. The best LP-based polynomial time algorithm can achieve an approximation of $\Delta^*+2$ [Bansal et al, 2009], where $\Delta^*$ denotes the optimal maximum tree in-degree. As for purely combinatorial algorithms (algorithms that do not use LP), the best approximation is $O(\Delta^*+\log n)$ [Krishnan and Raghavachari, 2001] but the running time is quasi-polynomial. In this paper, we focus on purely combinatorial algorithms and try to bridge the gap between LP-based approaches and purely combinatorial approaches. As a result, we propose a purely combinatorial polynomial time algorithm that also achieves an $O(\Delta^* + \log n)$ approximation. Then we improve this algorithm to obtain a $(1+\epsilon)\Delta^* + O(\frac{\log n}{\log\log n})$ for any constant $0<\epsilon<1$ approximation in polynomial time.
Split Network Polytopes and Network Spaces

Phylogenetics begins with reconstructing biological family trees from genetic data. Since Nature is not limited to tree-like histories, we use networks to organize our data, and have discovered new polytopes, metric spaces, and simplicial complexes that help us do so. Moreover, we show that the space of phylogenetic trees dually embeds into the Balanced Minimum Evolution polytope, and use this result to find a complex of faces within the subtour-elimination facets of the Symmetric Traveling Salesman polytope, which is shown to be dual to a quotient complex in network space.
On the Parallels between Paxos and Raft, and how to Port Optimizations

In recent years, Raft has surpassed Paxos to become the more popular consensus protocol in the industry. While many researchers have observed the similarities between the two protocols, no one has shown how Raft and Paxos are formally related to each other. In this paper, we present a formal mapping between Raft and Paxos, and use this knowledge to port a certain class of optimizations from Paxos to Raft. In particular, our porting method can automatically generate an optimized protocol specification with guaranteed correctness. As case studies, we port and evaluate two optimizations, Mencius and Paxos Quorum Lease to Raft.
Adversarial Distillation for Ordered Top-k Attacks

Deep Neural Networks (DNNs) are vulnerable to adversarial attacks, especially white-box targeted attacks. One scheme of learning attacks is to design a proper adversarial objective function that leads to the imperceptible perturbation for any test image (e.g., the Carlini-Wagner (C&W) method). Most methods address targeted attacks in the Top-1 manner. In this paper, we propose to learn ordered Top-k attacks (k>= 1) for image classification tasks, that is to enforce the Top-k predicted labels of an adversarial example to be the k (randomly) selected and ordered labels (the ground-truth label is exclusive). To this end, we present an adversarial distillation framework: First, we compute an adversarial probability distribution for any given ordered Top-k targeted labels with respect to the ground-truth of a test image. Then, we learn adversarial examples by minimizing the Kullback-Leibler (KL) divergence together with the perturbation energy penalty, similar in spirit to the network distillation method. We explore how to leverage label semantic similarities in computing the targeted distributions, leading to knowledge-oriented attacks. In experiments, we thoroughly test Top-1 and Top-5 attacks in the ImageNet-1000 validation dataset using two popular DNNs trained with clean ImageNet-1000 train dataset, ResNet-50 and DenseNet-121. For both models, our proposed adversarial distillation approach outperforms the C&W method in the Top-1 setting, as well as other baseline methods. Our approach shows significant improvement in the Top-5 setting against a strong modified C&W method.