Miracle-dz/UniFAR-FTUs huggingface dataset:
Model Fusion via Optimal Transport

Combining different models is a widely used paradigm in machine learning applications. While the most common approach is to form an ensemble of models and average their individual predictions, this approach is often rendered infeasible by given resource constraints in terms of memory and computation, which grow linearly with the number of models. We present a layer-wise model fusion algorithm for neural networks that utilizes optimal transport to (soft-) align neurons across the models before averaging their associated parameters. We show that this can successfully yield "one-shot" knowledge transfer (i.e, without requiring any retraining) between neural networks trained on heterogeneous non-i.i. d. data. In both i.i. d. and non-i.i. d. settings, we illustrate that our approach significantly outperforms vanilla averaging, as well as how it can serve as an efficient replacement for the ensemble with moderate fine-tuning, for standard convolutional networks (like VGG11), residual networks (like ResNet18), and multi-layer perceptrons on CIFAR10 and MNIST. Finally, our approach also provides a principled way to combine the parameters of neural networks with different widths, and we explore its application for model compression. Code is available under the following link https://github.com/modelfusion.
LiveGraph

The specific characteristics of graph workloads make it hard to design a one-size-fits-all graph storage system. Systems that support transactional updates use data structures with poor data locality, which limits the efficiency of analytical workloads or even simple edge scans. Other systems run graph analytics workloads efficiently, but cannot properly support transactions. This paper presents LiveGraph, a graph storage system that outperforms both the best graph transactional systems and the best solutions for real-time graph analytics on fresh data. LiveGraph achieves this by ensuring that adjacency list scans, a key operation in graph workloads, are purely sequential: they never require random accesses even in presence of concurrent transactions. Such pure-sequential operations are enabled by combining a novel graph-aware data structure, the Transactional Edge Log (TEL), with a concurrency control mechanism that leverages TEL's data layout. Our evaluation shows that LiveGraph significantly outperforms state-of-the-art (graph) database solutions on both transactional and real-time analytical workloads.
Stripe-based and attribute-aware network: a two-branch deep model for vehicle re-identification

Vehicle re-identification (Re-ID) has been attracting increasing interest in the field of computer vision due to the growing utilization of surveillance cameras in public security. However, vehicle Re-ID still suffers a similarity challenge, despite the efforts made to solve this problem. This challenge involves distinguishing different instances with nearly identical appearances. In this paper, a novel two-branch stripe-based and attribute-aware deep convolutional neural network (SAN) is proposed to learn the efficient feature embedding for a vehicle Re-ID task. The two-branch neural network, consisting of a stripe-based branch and an attribute-aware branch, can adaptively extract the discriminative features from the visual appearance of vehicles. Horizontal average pooling and dimension-reduced convolutional layers are inserted into the stripe-based branch to achieve part-level features. Meanwhile, the attribute-aware branch extracts the global feature under the supervision of vehicle attribute labels to separate the similar vehicle identities with different attribute annotations. Finally, the part-level and global features are concatenated together to form the final descriptor of the input image for vehicle Re-ID. The final descriptor can not only separate vehicles with different attributes but also distinguish vehicle identities with the same attributes. The extensive experiments on both the VehicleID and VeRi datasets show that the proposed SAN method outperforms other state-of-the-art vehicle Re-ID approaches.
Two classes of p-ary linear codes and their duals

Let $\mathbb{F}_{p^m}$ be the finite field of order $p^m$, where $p$ is an odd prime and $m$ is a positive integer. In this paper, we investigate a class of subfield codes of linear codes and obtain the weight distribution of \begin{equation*} \begin{split} \mathcal{C}_k=\left\{\left(\left( {\rm Tr}_1^m\left(ax^{p^k+1}+bx\right)+c\right)_{x \in \mathbb{F}_{p^m}}, {\rm Tr}_1^m(a)\right) : \, a,b \in \mathbb{F}_{p^m}, c \in \mathbb{F}_p\right\}, \end{split} \end{equation*} where $k$ is a nonnegative integer. Our results generalize the results of the subfield codes of the conic codes in \cite{Hengar}. Among other results, we study the punctured code of $\mathcal{C}_k$, which is defined as $$\mathcal{\bar{C}}_k=\left\{\left( {\rm Tr}_1^m\left(a x^{{p^k}+1}+bx\right)+c\right)_{x \in \mathbb{F}_{p^m}} : \, a,b \in \mathbb{F}_{p^m}, \,\,c \in \mathbb{F}_p\right\}. $$ The parameters of these linear codes are new in some cases. Some of the presented codes are optimal or almost optimal. Moreover, let $v_2(\cdot)$ denote the 2-adic order function and $v_2(0)=\infty$, the duals of $\mathcal{C}_k$ and $\mathcal{\bar{C}}_k$ are optimal with respect to the Sphere Packing bound if $p>3$, and the dual of $\mathcal{\bar{C}}_k$ is an optimal ternary linear code for the case $v_2(m)\leq v_2(k)$ if $p=3$ and $m>1$.
CMB anisotropy and BBN constraints on pre-recombination decay of dark matter to visible particles

Injection of high energy electromagnetic particles around the recombination epoch can modify the standard recombination history and therefore the CMB anisotropy power spectrum. Previous studies have put strong constraints on the amount of electromagnetic energy injection around the recombination era (redshifts z≲ 4500). However, energy injected in the form of energetic (> keV) visible standard model particles is not deposited instantaneously. The considerable delay between the time of energy injection and the time when all energy is deposited to background baryonic gas and CMB photons, together with the extraordinary precision with which the CMB anisotropies have been measured, means that CMB anisotropies are sensitive to energy that was injected much before the epoch of recombination. We show that the CMB anisotropy power spectrum is sensitive to energy injection even at z = 10000, giving stronger constraints compared to big bang nucleosynthesis and CMB spectral distortions. We derive, using Planck CMB data, the constraints on long-lived unstable particles decaying at redshifts z≲ 10000 (lifetime τX≳ 1011s) by explicitly evolving the electromagnetic cascades in the expanding Universe, thus extending previous constraints to lower particle lifetimes. We also revisit the BBN constraints and show that the delayed injection of energy is important for BBN constraints. We find that the constraints can be weaker by a factor of few to almost an order of magnitude, depending on the energy, when we relax the quasi-static or on-the-spot assumptions.
DeepSearch: Simple and Effective Blackbox Fuzzing of Deep Neural Networks

Although deep neural networks have been successful in image classification, they are prone to adversarial attacks. To generate misclassified inputs, there has emerged a wide variety of techniques, such as black- and whitebox testing of neural networks. In this paper, we present DeepSearch, a novel blackbox-fuzzing technique for image classifiers. Despite its simplicity, DeepSearch is shown to be more effective in finding adversarial examples than closely related black- and whitebox approaches. DeepSearch is additionally able to generate the most subtle adversarial examples in comparison to these approaches.
Zero-shot Dependency Parsing with Pre-trained Multilingual Sentence Representations

We investigate whether off-the-shelf deep bidirectional sentence representations (Devlin et al., 2019) trained on a massively multilingual corpus (multilingual BERT) enable the development of an unsupervised universal dependency parser. This approach only leverages a mix of monolingual corpora in many languages and does not require any translation data making it applicable to low-resource languages. In our experiments we outperform the best CoNLL 2018 language-specific systems in all of the shared task’s six truly low-resource languages while using a single system. However, we also find that (i) parsing accuracy still varies dramatically when changing the training languages and (ii) in some target languages zero-shot transfer fails under all tested conditions, raising concerns on the ‘universality’ of the whole approach.
Deep Independently Recurrent Neural Network (IndRNN)

Recurrent neural networks (RNNs) are known to be difficult to train due to the gradient vanishing and exploding problems and thus difficult to learn long-term patterns. Long short-term memory (LSTM) was developed to address these problems, but the use of hyperbolic tangent and the sigmoid activation functions results in gradient decay over layers. Consequently, construction of an efficiently trainable deep RNN is challenging. Moreover, training of LSTM is very compute-intensive as the recurrent connection using matrix product is conducted at every time step. To address these problems, this paper proposes a new type of RNNs with the recurrent connection formulated as Hadamard product, referred to as independently recurrent neural network (IndRNN), where neurons in the same layer are independent of each other and connected across layers. The gradient vanishing and exploding problems are solved in IndRNN by simply regulating the recurrent weights, and thus long-term dependencies can be learned. Moreover, an IndRNN can work with non-saturated activation functions such as ReLU and be still trained robustly. Different deeper IndRNN architectures, including the basic stacked IndRNN, residual IndRNN and densely connected IndRNN, have been investigated, all of which can be much deeper than the existing RNNs. Furthermore, IndRNN reduces the computation at each time step and can be over 10 times faster than the LSTM. The code is made publicly available at this https URL. Experimental results have shown that the proposed IndRNN is able to process very long sequences (over 5000 time steps), can be used to construct very deep networks (the 21 layers residual IndRNN and deep densely connected IndRNN used in the experiment for example). Better performances have been achieved on various tasks with IndRNNs compared with the traditional RNN and LSTM.
Graphs in which the Maxine heuristic produces a maximum independent set.

The residue of a graph is the number of zeros left after iteratively applying the Havel-Hakimi algorithm to its degree sequence. Favaron, Maheo, and Sacle showed that the residue is a lower bound on the independence number. The Maxine heuristic reduces a graph to an independent set of size $M$. It has been shown that given a graph $G$, $M$ is bounded between the independence number and the residue of a graph for any application of the Maxine heuristic. We improve upon a forbidden subgraph classification of graphs such that $M$ is equal to the independence number given by Barrus and Molnar in 2015.
Finite-dimensional Nichols algebras over dual Radford algebras

For [Formula: see text], let [Formula: see text] be the dual of the Radford algebra of dimension [Formula: see text]. We present new finite-dimensional Nichols algebras arising from the study of simple Yetter–Drinfeld modules over [Formula: see text]. Along the way, we describe the simple objects in [Formula: see text] and their projective envelopes. Then we determine those simple modules that give rise to finite-dimensional Nichols algebras for the case [Formula: see text]. There are 18 possible cases. We present by generators and relations, the corresponding Nichols algebras on five of these eighteen cases. As an application, we characterize finite-dimensional Nichols algebras over indecomposable modules for [Formula: see text] and [Formula: see text], [Formula: see text], which recovers some results of the second and third author in the former case, and of Xiong in the latter. Cualquier destino, por largo y complicado que sea, consta en realidad de un solo momento: el momento en que el hombre sabe para siempre quién es. Jorge Luis Borges
Saliency Guided Self-attention Network for Weakly-supervised Semantic Segmentation

—Weakly supervised semantic segmentation (WSSS) using only image-level labels can greatly reduce the annotation cost and therefore has attracted considerable research interest. However, its performance is still inferior to the fully supervised counterparts. To mitigate the performance gap, we propose a saliency guided self-attention network (SGAN) to address the WSSS problem. The introduced self-attention mechanism is able to capture rich and extensive contextual information but also may mis-spread attentions to unexpected regions. To enable this mechanism work effectively under weak supervision, we integrate class-agnostic saliency priors into the self-attention mechanism to prevent the attentions on discriminative parts from mis-spreading to the background. And meanwhile we utilize class-speciﬁc attention cues as an additional supervision for SGAN, which reduces the mis-spread of attentions in regions belonging to different foreground categories. The proposed approach is able to produce dense and accurate localization cues, by which the segmentation performance is boosted. Experiments on PASCAL VOC 2012 dataset show that the proposed approach outperforms all other state-of-the-art methods.
Eavesdrop the Composition Proportion of Training Labels in Federated Learning

Federated learning (FL) has recently emerged as a new form of collaborative machine learning, where a common model can be learned while keeping all the training data on local devices. Although it is designed for enhancing the data privacy, we demonstrated in this paper a new direction in inference attacks in the context of FL, where valuable information about training data can be obtained by adversaries with very limited power. In particular, we proposed three new types of attacks to exploit this vulnerability. The first type of attack, Class Sniffing, can detect whether a certain label appears in training. The other two types of attacks can determine the quantity of each label, i.e., Quantity Inference attack determines the composition proportion of the training label owned by the selected clients in a single round, while Whole Determination attack determines that of the whole training process. We evaluated our attacks on a variety of tasks and datasets with different settings, and the corresponding results showed that our attacks work well generally. Finally, we analyzed the impact of major hyper-parameters to our attacks and discussed possible defenses.
Developing New Solutions for a Reconfigurable Microstrip Patch Antenna by Inverse Artificial Neural Networks

This paper presents the use of inverse artificial neural networks (ANNs) to develop and optimize a reconfigurable 5-fingers shaped microstrip patch antenna. New solutions are produced by using three accurate prior knowledge inverse ANNs with sufficient amount of training data where the frequency information is incorporated into the structure of ANNs. The proposed antenna can operate with four modes, which are controlled by two PIN diode switches with ON/OFF states, and it resonates at multiple frequencies between 2-7 GHz. The complexity of the input/output relationship is reduced by using prior knowledge. Three independent methods of incorporating knowledge in the second step of the training process with a multilayer perceptron (MLP) in the first step are demonstrated and their results are compared to EM simulation.
On Identification Issues in Business Cycle Accounting Models

Since its introduction by Chari et al. (2018), Business Cycle Accounting (BCA) exercises have become widespread. Much attention has been devoted to the results of such exercises and to methodological departures from the baseline methodology. Little attention has been paid to identification issues within these classes of models, despite the methodology typically involving estimating relatively large scale dynamic stochastic general equilibrium models. In this paper we investigate whether such issues are of concern in the original methodology and in an extension proposed by Sustek (2011) called Monetary BCA. We resort to two types of identification tests in population. One concerns strict identification as theorized by Komuner and Ng (2011), while the other deals both with strict and weak identification as in Iskrev (2015). As to the former, when restricting the estimation to the parameters governing the latent variable's laws of motion, we find that both in the BCA and MBCA framework, all parameters fulfill the requirements for strict identification. If instead we estimate all structural parameters of the model jointly, both frameworks show strict identification failures in several parameters. These results hold for both tests. We show that restricting estimation of some deep parameters can obviate such failures. When we explore weak identification issues, we find that they affect both models. They arise from the fact that many of the estimated parameters do not have a distinct effect on the likelihood. Most importantly, we explore the extent to which these weak identification problems affect the main economic takeaways and find that the identification deficiencies are not relevant for the standard BCA model. Finally, we compute some statistics of interest to practitioners of the BCA methodology.
3D-Scene-GAN: Three-dimensional Scene Reconstruction with Generative Adversarial Networks

Three-dimensional (3D) Reconstruction is a vital and challenging research topic in advanced computer graphics and computer vision due to the intrinsic complexity and computation cost. Existing methods often produce holes, distortions and obscure parts in the reconstructed 3D models which are not adequate for real usage. The focus of this paper is to achieve high quality 3D reconstruction performance of complicated scene by adopting Generative Adversarial Network (GAN). We propose a novel workflow, namely 3D-Scene-GAN, which can iteratively improve any raw 3D reconstructed models consisting of meshes and textures. 3D-SceneGAN is a weakly semi-supervised model. It only takes real-time 2D observation images as the supervision, and doesn’t rely on prior knowledge of shape models or any referenced observations. Finally, through the qualitative and quantitative experiments, 3D-Scene-GAN shows compelling advantages over the state-of-the-art methods: balanced rank estimation (BRE) scores are improved by 30%-100% on ICL-NUIM dataset, and 36%-190% on SUN3D dataset. And the mean distance error (MDR) also outperforms other state-of-the-art methods on benchmarks.
Strategies for reducing the size of the search space in semantic genetic programming

Ao tentar resolver problemas de otimizacao, algoritmos de programacao genetica aplicam operacoes bio-inspiradas (mutacao e cruzamento, por exemplo) de forma a explorar o espaco de possiveis solucoes em busca de uma solucao satisfatoria. Normalmente tais algoritmos sao utilizados na resolucao de um problema conhecido como regressao simbolica, onde o objetivo e encontrar uma expressao matematica cuja curva correspondente se aproxime daquela induzida a partir de um conjunto de instancias de treinamento. Operadores canonicos de programacao genetica nao levam em conta aspectos semanticos, o que tende a piorar o desempenho e a robustez dos metodos que os uti- lizam. Operadores geneticos semanticos, por outro lado, agregam nocoes de semantica que permitem uma exploracao mais consistente do espaco de busca. Outra melhoria parte da exploracao de propriedades geometricas que descrevem a relacao espacial entre possiveis solucoes em um espaco semantico n-dimensional, onde n e igual ao numero de instancias de treinamento. O metodo normalmente tomado como referencia nesse contexto se chama Geometric Semantic Genetic Programming (GSGP). No entanto, em problemas onde o valor de n e alto - um cenario comum em aplicacoes reais - o processo de busca pode se tornar excessivamente complicado, uma vez que o volume do espaco semantico cresce exponencialmente em funcao do numero de dimensoes. Esta tese busca reduzir esse problema, focando na reducao da dimensionalidade do espaco de busca no contexto de programacao genetica semantica atraves de metodos de selecao de instancias. Nosso principal objetivo e projetar, implementar e validar metodos que reduzam o tamanho do espaco de busca. Mais precisamente, queremos entender ate que ponto o numero de dimensoes do espaco semantico e capaz de influenciar o processo de busca e qual o impacto da aplicacao de metodos de selecao sobre os resultados da busca realizada pelo GSGP. Alem disso, buscamos entender o impacto provocado pelo ruido sobre os metodos de programacao genetica e sobre as estrategias de selecao de instancia propostas. Duas abordagens sao consideradas: (i) aplicar metodos de selecao de instancias como uma etapa de pre-processamento, antes que as instancias de treinamento sejam fornecidas ao GSGP e (ii) incorporar a selecao de instancias ao processo evolutivo do GSGP. Um conjunto de experimentos foi realizado em um grupo de bases de dados reais e sinteticas. A analise experimental realizada indica que parte dos metodos propostos podem, de fato, melhorar aspectos relacionados a busca realizada pelo GSGP.
Preliminary filtering and lossy compression of noisy remote sensing images

Compression is a typical stage in processing of remote sensing images. Here we consider lossy compression of images that contain noise. Then, there are, at least, two approaches – to compress images without pre-filtering and with prefiltering. The second approach is shown to have some advantages but only under certain conditions. Several metrics characterizing quality of compressed images are used. Main conclusions are in good agreement in cases of using different metrics. Recommendations concerning parameter selection for lossy compression with pre-filtering are given.
Multi-Class Classification of Turkish Texts with Machine Learning Algorithms

The problem of text classification is the process of supervised assignment of text documents to one or more predefined categories or classes according to the content of the processed texts with natural language processing methods. Text classification applications are actively used in various fields such as categorization of social interactions, web pages and news texts, optimization of search engines, extracting information, and automatically processing e-mails. In this context, it is aimed to classify Turkish texts with methods based on supervised machine learning. In this context, the classification success of supervised learning models on Turkish texts was analyzed with different parameters. These models have been tested for classification of news texts on five predefined classes (economy, politics, sport, health, and technology) and the system was trained with different number of training documents and the classification process was carried out. In this context, the classification performances of Multinomial Naive Bayes, Bernoulli Naive Bayes, Support Vector Machine, K-Nearest Neighbor, and Decision Trees algorithms on Turkish news texts are compared and interpreted in the light of the results obtained with different parameters. As a result of the study, the procedure with the best classification success was the Multinomial Naive Bayes algorithm with a classification success of about 90%. These results show that the Naive Bayes probability model can be used as an effective classifier method in classifying Turkish texts compared to other methods. In this context, it is envisaged that the proposed methodology could be applied to Turkish texts on different web platforms (social networks, forums, communication networks, etc.) for different purposes.