Learned representations of source code enable various software developer tools, e.g., to detect bugs or to predict program properties.At the core of code representations often are word embeddings of identifier names in source code, because identifiers account for the majority of source code vocabulary and convey important semantic information.Unfortunately, there currently is no generally accepted way of evaluating the quality of word embeddings of identifiers, and current evaluations are biased toward specific downstream tasks.This paper presents IdBench, the first benchmark for evaluating to what extent word embeddings of identifiers represent semantic relatedness and similarity.The benchmark is based on thousands of ratings gathered by surveying 500 software developers.We use IdBench to evaluate state-of-the-art embedding techniques proposed for natural language, an embedding technique specifically designed for source code, and lexical string distance functions, as these are often used in current developer tools.Our results show that the effectiveness of embeddings varies significantly across different embedding techniques and that the best available embeddings successfully represent semantic relatedness.On the downside, no existing embedding provides a satisfactory representation of semantic similarities, e.g., because embeddings consider identifiers with opposing meanings as similar, which may lead to fatal mistakes in downstream developer tools.IdBench provides a gold standard to guide the development of novel embeddings that address the current limitations.

Un benchmark per valutare gli embedding neurali degli identificatori nel codice sorgente.


Generative adversarial nets (GANs) are widely used to learn the data sampling process and their performance may heavily depend on the loss functions, given a limited computational budget.This study revisits MMD-GAN that uses the maximum mean discrepancy (MMD) as the loss function for GAN and makes two contributions.First, we argue that the existing MMD loss function may discourage the learning of fine details in data as it attempts to contract the discriminator outputs of real data.To address this issue, we propose a repulsive loss function to actively learn the difference among the real data by simply rearranging the terms in MMD.Second, inspired by the hinge loss, we propose a bounded Gaussian kernel to stabilize the training of MMD-GAN with the repulsive loss function.The proposed methods are applied to the unsupervised image generation tasks on CIFAR-10, STL-10, CelebA, and LSUN bedroom datasets.Results show that the repulsive loss function significantly improves over the MMD loss at no additional computational cost and outperforms other representative loss functions.The proposed methods achieve an FID score of 16.21 on the CIFAR-10 dataset using a single DCGAN network and spectral normalization.

Riordinando i termini della maximum mean discrepancy si ottiene una loss molto migliore per il discriminatore delle reti generative adversarial


Deep neural networks have shown incredible performance for inference tasks in a variety of domains.Unfortunately, most current deep networks are enormous cloud-based structures that require significant storage space, which limits scaling of deep learning as a service (DLaaS) and use for on-device augmented intelligence.  This paper finds algorithms that directly use lossless compressed representations of deep feedforward networks (with synaptic weights drawn from discrete sets), to perform inference without full decompression.The basic insight that allows less rate than naive approaches is the recognition that the bipartite graph layers of feedforward networks have a kind of permutation invariance to the labeling of nodes, in terms of inferential operation and that the inference operation depends locally on the edges directly connected to it.We also provide experimental results of our approach on the MNIST dataset.

Questo articolo trova algoritmi che usano direttamente rappresentazioni compresse lossless di deep feedforward network, per eseguire l'inferenza senza decompressione completa.


Generative adversarial networks (GANs) form a generative modeling approach known for producing appealing samples, but they are notably difficult to train.One common way to tackle this issue has been to propose new formulations of the GAN objective.Yet, surprisingly few studies have looked at optimization methods designed for this adversarial training.In this work, we cast GAN optimization problems in the general variational inequality framework.Tapping into the mathematical programming literature, we counter some common misconceptions about the difficulties of saddle point optimization and propose to extend methods designed for variational inequalities to the training of GANs.We apply averaging, extrapolation and a computationally cheaper variant that we call extrapolation from the past to the stochastic gradient method (SGD) and Adam.

Abbiamo applicato le GAN al framework della disuguaglianza variazionale e importiamo tecniche da questa letteratura per ottimizzare meglio le GAN; diamo estensioni algoritmiche e testiamo empiricamente le loro prestazioni per il training delle GAN.


In order to efficiently learn with small amount of data on new tasks, meta-learning transfers knowledge learned from previous tasks to the new ones.However, a critical challenge in meta-learning is the task heterogeneity which cannot be well handled by traditional globally shared meta-learning methods.In addition, current task-specific meta-learning methods may either suffer from hand-crafted structure design or lack the capability to capture complex relations between tasks.In this paper, motivated by the way of knowledge organization in knowledge bases, we propose an automated relational meta-learning (ARML) framework that automatically extracts the cross-task relations and constructs the meta-knowledge graph.When a new task arrives, it can quickly find the most relevant structure and tailor the learned structure knowledge to the meta-learner.As a result, the proposed framework not only addresses the challenge of task heterogeneity by a learned meta-knowledge graph, but also increases the model interpretability.We conduct extensive experiments on 2D toy regression and few-shot image classification and the results demonstrate the superiority of ARML over state-of-the-art baselines.

Affrontare il problema dell'eterogeneità dei task nel meta-learning introducendo il grafo della meta-conoscenza


In this paper, a deep boosting algorithm is developed tolearn more discriminative ensemble classifier by seamlessly combining a set of base deep CNNs (base experts)with diverse capabilities, e.g., these base deep CNNs aresequentially trained to recognize a set of object classes in an easy-to-hard way according to theirlearning complexities.Our experimental results have demonstratedthat our deep boosting algorithm can significantly improve theaccuracy rates on large-scale visual recognition.

Un algoritmo di deep boosting viene sviluppato per imparare un classificatore ensemble più discriminante combinando senza soluzione di continuità un insieme di deep CNN di base.


We present a method for translating music across musical instruments and styles.This method is based on unsupervised training of a multi-domain wavenet autoencoder, with a shared encoder and a domain-independent latent space that is trained end-to-end on waveforms.Employing a diverse training dataset and large net capacity, the single encoder allows us to translate also from musical domains that were not seen during training.We evaluate our method on a dataset collected from professional musicians, and achieve convincing translations.We also study the properties of the obtained translation and demonstrate translating even from a whistle, potentially enabling the creation of instrumental music by untrained humans.

Un metodo automatico per convertire la musica tra strumenti e stili


Most existing defenses against adversarial attacks only consider robustness to L_p-bounded distortions.In reality, the specific attack is rarely known in advance and adversaries are free to modify images in ways which lie outside any fixed distortion model; for example, adversarial rotations lie outside the set of L_p-bounded distortions.In this work, we advocate measuring robustness against a much broader range of unforeseen attacks, attacks whose precise form is unknown during defense design.We propose several new attacks and a methodology for evaluating a defense against a diverse range of unforeseen distortions.First, we construct novel adversarial JPEG, Fog, Gabor, and Snow distortions to simulate more diverse adversaries.We then introduce UAR, a summary metric that measures the robustness of a defense against a given distortion.  Using UAR to assess robustness against existing and novel attacks, we perform an extensive study of adversarial robustness.We find that evaluation against existing L_p attacks yields redundant information which does not generalize to other attacks; we instead recommend evaluating against our significantly more diverse set of attacks.We further find that adversarial training against either one or multiple distortions fails to confer robustness to attacks with other distortion types.  These results underscore the need to evaluate and study robustness against unforeseen distortions.

Proponiamo diversi nuovi attacchi e una metodologia per misurare la robustezza contro gli attacchi adversarial imprevisti.


Deep neural networks (DNNs) have witnessed as a powerful approach in this year by solving long-standing Artificialintelligence (AI) supervised and unsupervised tasks exists in natural language processing, speech processing, computer vision and others.In this paper, we attempt to apply DNNs on three different cyber security use cases: Android malware classification, incident detection and fraud detection.The data set of each use case contains real known benign and malicious activities samples.These use cases are part of Cybersecurity Data Mining Competition (CDMC) 2017.The efficient network architecture for DNNs is chosen by conducting various trails of experiments for network parameters and network structures.The experiments of such chosen efficient configurations of DNNs are run up to 1000 epochs with learning rate set in the range [0.01-0.5].Experiments of DNNs performed well in comparison to the classical machine learning algorithm in all cases of experiments of cyber security use cases.This is due to the fact that DNNs implicitly extract and build better features, identifies the characteristics of the data that lead to better accuracy.The best accuracy obtained by DNNs and XGBoost on Android malware classification 0.940 and 0.741, incident detection 1.00 and 0.997, and fraud detection 0.972 and 0.916 respectively.The accuracy obtained by DNNs varies -0.05%, +0.02%, -0.01% from the top scored system in CDMC 2017 tasks.

Deep-Net: deep neural network per i casi d'uso della sicurezza informatica


In this paper, we present an approach to learn recomposable motor primitives across large-scale and diverse manipulation demonstrations.Current approaches to decomposing demonstrations into primitives often assume manually defined primitives and bypass the difficulty of discovering these primitives.On the other hand, approaches in primitive discovery put restrictive assumptions on the complexity of a primitive, which limit applicability to narrow tasks.Our approach attempts to circumvent these challenges by jointly learning both the underlying motor primitives and recomposing these primitives to form the original demonstration.Through constraints on both the parsimony of primitive decomposition and the simplicity of a given primitive, we are able to learn a diverse set of motor primitives, as well as a coherent latent representation for these primitives.We demonstrate, both qualitatively and quantitatively, that our learned primitives capture semantically meaningful aspects of a demonstration.This allows us to compose these primitives in a hierarchical reinforcement learning setup to efficiently solve robotic manipulation tasks like reaching and pushing.

Impariamo uno spazio di primitive motorie da dimostrazioni robotiche non annotate, e dimostriamo che queste primitive sono semanticamente significative e possono essere composte per nuovi task robotici.


Using modern deep learning models to make predictions on time series data from wearable sensors generally requires large amounts of labeled data.However, labeling these large datasets can be both cumbersome and costly.In this paper, we apply weak supervision to time series data, and programmatically label a dataset from sensors worn by patients with Parkinson's.We then built a LSTM model that predicts when these patients exhibit clinically relevant freezing behavior (inability to make effective forward stepping).We show that (1) when our model is trained using patient-specific data (prior sensor sessions), we come within 9% AUROC of a model trained using hand-labeled data and (2) when we assume no prior observations of subjects, our weakly supervised model matched performance with hand-labeled data.These results demonstrate that weak supervision may help reduce the need to painstakingly hand label time series training data.

Dimostriamo la fattibilità di un approccio di classificazione delle serie temporali weakly supervised per i dati dei sensori indossabili.


Learning semantic correspondence between the structured data (e.g., slot-value pairs) and associated texts is a core problem for many downstream NLP applications, e.g., data-to-text generation.Recent neural generation methods require to use large scale training data.However, the collected data-text pairs for training are usually loosely corresponded, where texts contain additional or contradicted information compare to its paired input.In this paper, we propose a local-to-global alignment (L2GA) framework to learn semantic correspondences from loosely related data-text pairs.First, a local alignment model based on multi-instance learning is applied to build the semantic correspondences within a data-text pair.Then, a global alignment model built on top of a memory guided conditional random field (CRF) layer is designed to exploit dependencies among alignments in the entire training corpus, where the memory is used to integrate the alignment clues provided by the local alignment model.Therefore, it is capable of inducing missing alignments for text spans that are not supported by its imperfect paired input.Experiments on recent restaurant dataset show that our proposed method can improve the alignment accuracy and as a by product, our method is also applicable to induce semantically equivalent training data-text pairs for neural generation models.

Proponiamo un framework di allineamento locale-globale per imparare corrispondenze semantiche da coppie di dati-testo rumorose con una weak supervision


Imitation learning algorithms provide a simple and straightforward approach for training control policies via standard supervised learning methods.By maximizing the likelihood of good actions provided by an expert demonstrator, supervised imitation learning can produce effective policies without the algorithmic complexities and optimization challenges of reinforcement learning, at the cost of requiring an expert demonstrator -- typically a person -- to provide the demonstrations.In this paper, we ask: can we use imitation learning to train effective policies without any expert demonstrations?The key observation that makes this possible is that, in the multi-task setting, trajectories that are generated by a suboptimal policy can still serve as optimal examples for other tasks.In particular, in the setting where the tasks correspond to different goals, every trajectory is a successful demonstration for the state that it actually reaches.Informed by this observation, we propose a very simple algorithm for learning behaviors without any demonstrations, user-provided reward functions, or complex reinforcement learning methods.Our method simply maximizes the likelihood of actions the agent actually took in its own previous rollouts, conditioned on the goal being the state that it actually reached.Although related variants of this approach have been proposed previously in imitation learning settings with example demonstrations, we present the first instance of this approach as a method for learning goal-reaching policies entirely from scratch.We present a theoretical result linking self-supervised imitation learning and reinforcement learning, and empirical results showing that it performs competitively with more complex reinforcement learning methods on a range of challenging goal reaching problems.

Imparare a raggiungere gli obiettivi da zero usando l'imitation learning con la riannotazione dei dati


Neural networks have recently shown excellent performance on numerous classi- fication tasks.These networks often have a large number of parameters and thus require much data to train.When the number of training data points is small, however, a network with high flexibility will quickly overfit the training data, resulting in a large model variance and a poor generalization performance.To address this problem, we propose a new ensemble learning method called InterBoost for small-sample image classification.In the training phase, InterBoost first randomly generates two complementary datasets to train two base networks of the same structure, separately, and then next two complementary datasets for further training the networks are generated through interaction (or information sharing) between the two base networks trained previously.This interactive training process continues iteratively until a stop criterion is met.In the testing phase, the outputs of the two networks are combined to obtain one final score for classification.Detailed analysis of the method is provided for an in-depth understanding of its mechanism.

Nell'articolo, abbiamo proposto un metodo di ensemble chiamato InterBoost per il training di reti neurali per la classificazione di piccoli sample. Il metodo ha migliori prestazioni di generalizzazione rispetto ad altri metodi d'insieme e riduce significativamente le varianze.


Interpreting neural networks is a crucial and challenging task in machine learning.In this paper, we develop a novel framework for detecting statistical interactions captured by a feedforward multilayer neural network by directly interpreting its learned weights.Depending on the desired interactions, our method can achieve significantly better or similar interaction detection performance compared to the state-of-the-art without searching an exponential solution space of possible interactions.We obtain this accuracy and efficiency by observing that interactions between input features are created by the non-additive effect of nonlinear activation functions, and that interacting paths are encoded in weight matrices.We demonstrate the performance of our method and the importance of discovered interactions via experimental results on both synthetic datasets and real-world application datasets.

Rileviamo le interazioni statistiche catturate da una rete neurale multilayer feedforward interpretando direttamente i suoi pesi appresi.


The neural linear model is a simple adaptive Bayesian linear regression method that has recently been used in a number of problems ranging from Bayesian optimization to reinforcement learning.Despite its apparent successes in these settings, to the best of our knowledge there has been no systematic exploration of its capabilities on simple regression tasks.In this work we characterize these on the UCI datasets, a popular benchmark for Bayesian regression models, as well as on the recently introduced ''gap'' datasets, which are better tests of out-of-distribution uncertainty.We demonstrate that the neural linear model is a simple method that shows competitive performance on these tasks.

Facciamo un benchmark del modello lineare neurale sui dataset UCI e UCI "gap".


The reproducibility of reinforcement-learning research has been highlighted as a key challenge area in the field.In this paper, we present a case study in reproducing the results of one groundbreaking algorithm, AlphaZero, a reinforcement learning system that learns how to play Go at a superhuman level given only the rules of the game.We describe Minigo, a reproduction of the AlphaZero system using publicly available Google Cloud Platform infrastructure and Google Cloud TPUs.The Minigo system includes both the central reinforcement learning loop as well as auxiliary monitoring and evaluation infrastructure.With ten days of training from scratch on 800 Cloud TPUs, Minigo can play evenly against LeelaZero and ELF OpenGo, two of the strongest publicly available Go AIs.We discuss the difficulties of scaling a reinforcement learning system and the monitoring systems required to understand the complex interplay of hyperparameter configurations.

Abbiamo riprodotto AlphaZero su Google Cloud Platform


Generative adversarial networks (GANs) train implicit generative models through solving minimax problems.Such minimax problems are known as nonconvex- nonconcave, for which the dynamics of first-order methods are not well understood.In this paper, we consider GANs in the type of the integral probability metrics (IPMs) with the generator represented by an overparametrized neural network.When the discriminator is solved to approximate optimality in each iteration, we prove that stochastic gradient descent on a regularized IPM objective converges globally to a stationary point with a sublinear rate.Moreover, we prove that when the width of the generator network is sufficiently large and the discriminator function class has enough discriminative ability, the obtained stationary point corresponds to a generator that yields a distribution that is close to the distribution of the observed data in terms of the total variation.To the best of our knowledge, we seem to first establish both the global convergence and global optimality of training GANs when the generator is parametrized by a neural network.

Stabiliamo la convergenza globale all'ottimalità per le GAN basate su IPM dove il generatore è una rete neurale iperparametrizzata.


We present network embedding algorithms that capture information about a node from the local distribution over node attributes around it, as observed over random walks following an approach similar to Skip-gram.Observations from neighborhoods of different sizes are either pooled (AE) or encoded distinctly in a multi-scale approach (MUSAE).  Capturing attribute-neighborhood relationships over multiple scales is useful for a diverse range of applications, including latent feature identification across disconnected networks with similar attributes.We prove theoretically that matrices of node-feature pointwise mutual information are implicitly factorized by the embeddings.Experiments show that our algorithms are robust, computationally efficient and outperform comparable models on social, web and citation network datasets.

Sviluppiamo efficienti procedure di inclusione di reti approssimate su più scale con proprietà dimostrabili.


Few-shot classiﬁcation aims to learn a classiﬁer to recognize unseen classes during training with limited labeled examples.While signiﬁcant progress has been made, the growing complexity of network designs, meta-learning algorithms, and differences in implementation details make a fair comparison difﬁcult.In this paper, we present1) a consistent comparative analysis of several representative few-shot classiﬁcation algorithms, with results showing that deeper backbones signiﬁcantly reduce the gap across methods including the baseline,2) a slightly modiﬁed baseline method that surprisingly achieves competitive performance when compared with the state-of-the-art on both the mini-ImageNet and the CUB datasets, and3) a new experimental setting for evaluating the cross-domain generalization ability for few-shot classiﬁcation algorithms.Our results reveal that reducing intra-class variation is an important factor when the feature backbone is shallow, but not as critical when using deeper backbones.In a realistic, cross-domain evaluation setting, we show that a baseline method with a standard ﬁne-tuning practice compares favorably against other state-of-the-art few-shot learning algorithms.

Uno studio empirico dettagliato nella classificazione few-shot che rivela le sfide nel setting di valutazione standard e mostra una nuova direzione.


Temporal logics are useful for describing dynamic system behavior, and have been successfully used as a language for goal definitions during task planning.Prior works on inferring temporal logic specifications have focused on "summarizing" the input dataset -- i.e., finding specifications that are satisfied by all plan traces belonging to the given set.In this paper, we examine the problem of inferring specifications that describe temporal differences between two sets of plan traces.We formalize the concept of providing such contrastive explanations, then present a Bayesian probabilistic model for inferring contrastive explanations as linear temporal logic specifications.We demonstrate the efficacy, scalability, and robustness of our model for inferring correct specifications across various benchmark planning domains and for a simulated air combat mission.

Presentiamo un modello di inferenza bayesiana per dedurre spiegazioni contrastive (come specifiche LTL) che descrivono come due set di plan trace differiscono.


This work tackles the problem of characterizing and understanding the decision boundaries of neural networks with piece-wise linear non-linearity activations.We use tropical geometry, a new development in the area of algebraic geometry, to provide a characterization of the decision boundaries of a simple neural network of the form (Affine, ReLU, Affine).Specifically, we show that the decision boundaries are a subset of a tropical hypersurface, which is intimately related to a polytope formed by the convex hull of two zonotopes.The generators of the zonotopes are precise functions of the neural network parameters.We utilize this geometric characterization to shed light and new perspective on three tasks.In doing so, we propose a new tropical perspective for the lottery ticket hypothesis, where we see the effect of different initializations on the tropical geometric representation of the decision boundaries.Also, we leverage this characterization as a new set of tropical regularizers, which  deal directly  with the decision boundaries of a network.We investigate the use of these regularizers  in neural network pruning (removing network parameters that do not contribute to the tropical geometric representation of the decision boundaries) and in generating adversarial input attacks (with input perturbations explicitly perturbing the decision boundaries geometry to change the network prediction of the input).

La geometria tropicale può essere sfruttata per rappresentare i confini decisionali delle reti neurali e portare alla luce intuizioni interessanti.


First-order methods such as stochastic gradient descent (SGD) are currently the standard algorithm for training deep neural networks.Second-order methods, despite their better convergence rate, are rarely used in practice due to the pro- hibitive computational cost in calculating the second-order information.In this paper, we propose a novel Gram-Gauss-Newton (GGN) algorithm to train deep neural networks for regression problems with square loss.Our method draws inspiration from the connection between neural network optimization and kernel regression of neural tangent kernel (NTK).Different from typical second-order methods that have heavy computational cost in each iteration, GGN only has minor overhead compared to first-order methods such as SGD.We also give theoretical results to show that for sufficiently wide neural networks, the convergence rate of GGN is quadratic.Furthermore, we provide convergence guarantee for mini-batch GGN algorithm, which is, to our knowledge, the first convergence result for the mini-batch version of a second-order method on overparameterized neural net- works.Preliminary experiments on regression tasks demonstrate that for training standard networks, our GGN algorithm converges much faster and achieves better performance than SGD.

Un nuovo metodo Gram-Gauss-Newton per addestrare le reti neurali, ispirato dal kernel della tangente neurale e dal metodo Gauss-Newton, con velocità di convergenza veloce sia teoricamente che sperimentalmente.


Recent pretrained sentence encoders achieve state of the art results on language understanding tasks, but does this mean they have implicit knowledge of syntactic structures?We introduce a grammatically annotated development set for the Corpus of Linguistic Acceptability (CoLA; Warstadt et al., 2018), which we use to investigate the grammatical knowledge of three pretrained encoders, including the popular OpenAI Transformer (Radford et al., 2018) and BERT (Devlin et al., 2018).We fine-tune these encoders to do acceptability classification over CoLA and compare the models’ performance on the annotated analysis set.Some phenomena, e.g. modification by adjuncts, are easy to learn for all models, while others, e.g. long-distance movement, are learned effectively only by models with strong overall performance, and others still, e.g. morphological agreement, are hardly learned by any model.

Indaghiamo la conoscenza sintattica implicita degli embedding di frasi usando un nuovo set di analisi di frasi annotate grammaticalmente con giudizi di accettabilità.


When considering simultaneously a finite number of tasks, multi-output learning enables one to account for the similarities of the tasks via appropriate regularizers.We propose a generalization of the classical setting to a continuum of tasks by using vector-valued RKHSs.

Proponiamo un'estensione dell'apprendimento multi-output ad un continuum di task utilizzando kernel valutati dall'operatore.


We analyze the joint probability distribution on the lengths of thevectors of hidden variables in different layers of a fully connecteddeep network, when the weights and biases are chosen randomly according toGaussian distributions, and the input is binary-valued. We showthat, if the activation function satisfies a minimal set ofassumptions, satisfied by all activation functions that we know thatare used in practice, then, as the width of the network gets large,the ``length process'' converges in probability to a length mapthat is determined as a simple function of the variances of therandom weights and biases, and the activation function.We also show that this convergence may fail for activation functions that violate our assumptions.

Dimostriamo che, per funzioni di attivazione che soddisfano alcune condizioni, man mano che una deep network si allarga, le lunghezze dei vettori delle variabili nascoste convergono verso una mappa di lunghezza.


Data augmentation is one of the most effective approaches for improving the accuracy of modern machine learning models, and it is also indispensable to train a deep model for meta-learning.However, most current data augmentation implementations applied in meta-learning are the same as those used in the conventional image classification.In this paper, we introduce a new data augmentation method for meta-learning, which is named as ``Task Level Data Augmentation'' (referred to Task Aug).The basic idea of Task Aug is to increase the number of image classes rather than the number of images in each class.In contrast, with a larger amount of classes, we can sample more diverse task instances during training.This allows us to train a deep network by meta-learning methods with little over-fitting.Experimental results show that our approach achieves state-of-the-art performance on miniImageNet, CIFAR-FS, and FC100 few-shot learning benchmarks.Once paper is accepted, we will provide the link to code.

Proponiamo un approccio di data augmentation per meta-learning e dimostriamo che è valido.


In this paper, we present a general framework for distilling expectations with respect to the Bayesian posterior distribution of a deep neural network, significantly extending prior work on a method known as ``Bayesian Dark Knowledge."  Our generalized framework applies to the case of classification models and takes as input the architecture of a ``teacher" network, a general posterior expectation of interest, and the architecture of a ``student" network.The distillation method performs an online compression of the selected posterior expectation using iteratively generated Monte Carlo samples from the parameter posterior of the teacher model.We further consider the problem of optimizing the student model architecture with respect to an accuracy-speed-storage trade-off.We present experimental results investigating multiple data sets, distillation targets,  teacher model architectures, and approaches to searching for student model architectures.We establish the key result that distilling into a student model with an architecture that matches the teacher, as is done in Bayesian Dark Knowledge, can lead to sub-optimal performance.Lastly, we show that student architecture search methods can identify student models with significantly improved performance.

Un framework generale per distillare le aspettative posteriori bayesiane per le deep neural network.


Variational Autoencoders (VAEs) have proven to be powerful latent variable models.How- ever, the form of the approximate posterior can limit the expressiveness of the model.Categorical distributions are flexible and useful building blocks for example in neural memory layers.We introduce the Hierarchical Discrete Variational Autoencoder (HD-VAE): a hi- erarchy of variational memory layers.The Concrete/Gumbel-Softmax relaxation allows maximizing a surrogate of the Evidence Lower Bound by stochastic gradient ascent.We show that, when using a limited number of latent variables, HD-VAE outperforms the Gaussian baseline on modelling multiple binary image datasets.Training very deep HD-VAE remains a challenge due to the relaxation bias that is induced by the use of a surrogate objective.We introduce a formal definition and conduct a preliminary theoretical and empirical study of the bias.

In questo articolo, introduciamo una gerarchia discreta di variabili latenti categoriche che addestriamo usando il rilassamento Concrete/Gumbel-Softmax e deriviamo un limite superiore per la differenza assoluta tra l'obiettivo biased e quello unbiased.


In this paper, we propose a novel technique for improving the stochastic gradient descent (SGD) method to train deep networks, which we term \emph{PowerSGD}.The proposed PowerSGD method simply raises the stochastic gradient to a certain power $\gamma\in[0,1]$ during iterations and introduces only one additional parameter, namely, the power exponent $\gamma$ (when $\gamma=1$, PowerSGD reduces to SGD).We further propose PowerSGD with momentum, which we term \emph{PowerSGDM}, and provide convergence rate analysis on both PowerSGD and PowerSGDM methods.Experiments are conducted on popular deep learning models and benchmark datasets.Empirical results show that the proposed PowerSGD and PowerSGDM obtain faster initial training speed than adaptive gradient methods,  comparable generalization ability with SGD, and improved robustness to hyper-parameter selection and vanishing gradients.PowerSGD is essentially a gradient modifier via a nonlinear transformation.As such, it is orthogonal and complementary to other techniques for accelerating gradient-based optimization.

Proponiamo una nuova classe di ottimizzatori per l'ottimizzazione accelerata non convessa tramite una trasformazione non lineare del gradiente.


We aim to build complex humanoid agents that integrate perception, motor control, and memory.In this work, we partly factor this problem into low-level motor control from proprioception and high-level coordination of the low-level skills informed by vision.We develop an architecture capable of surprisingly flexible, task-directed motor control of a relatively high-DoF humanoid body by combining pre-training of low-level motor controllers with a high-level, task-focused controller that switches among low-level sub-policies.The resulting system is able to control a physically-simulated humanoid body to solve tasks that require coupling visual perception from an unstabilized egocentric RGB camera during locomotion in the environment.Supplementary video link:  https://youtu.be/fBoir7PNxPk

Risolvere task che coinvolgono la locomozione umanoide guidata dalla visione, riutilizzando il comportamento di locomozione dai dati di motion capture.


The gap between the empirical success of deep learning and the lack of strong theoretical guarantees calls for studying simpler models.By observing that a ReLU neuron is a product of a linear function with a gate (the latter determines whether the neuron is active or not), where both share a jointly trained weight vector, we propose to decouple the two.We introduce GaLU networks — networks in which each neuron is a product of a Linear Unit, defined by a weight vector which is being trained, with a Gate, defined by a different weight vector which is not being trained.Generally speaking, given a base model and a simpler version of it, the two parameters that determine the quality of the simpler version are whether its practical performance is close enough to the base model and whether it is easier to analyze it theoretically.We show that GaLU networks perform similarly to ReLU networks on standard datasets and we initiate a study of their theoretical properties, demonstrating that they are indeed easier to analyze.We believe that further research of GaLU networks may be fruitful for the development of a theory of deep learning.

Proponiamo le reti Gated Linear Unit - un modello che si comporta in modo simile alle reti ReLU su dati reali, pur essendo molto più facile da analizzare teoricamente.


Machine learning systems often encounter Out-of-Distribution (OoD) errors when dealing with testing data coming from a different distribution from the one used for training.With their growing use in critical applications, it becomes important to develop systems that are able to accurately quantify its predictive uncertainty and screen out these anomalous inputs.However, unlike standard learning tasks, there is currently no well established guiding principle for designing architectures that can accurately quantify uncertainty.Moreover, commonly used OoD detection approaches are prone to errors and even sometimes assign higher likelihoods to OoD samples.To address these problems, we first seek to identify guiding principles for designing uncertainty-aware architectures, by proposing Neural Architecture Distribution Search (NADS).Unlike standard neural architecture search methods which seek for a single best performing architecture, NADS searches for a distribution of architectures that perform well on a given task, allowing us to identify building blocks common among all uncertainty aware architectures.With this formulation, we are able to optimize a stochastic outlier detection objective and construct an ensemble of models to perform OoD detection.We perform multiple OoD detection experiments and observe that our NADS performs favorably compared to state-of-the-art OoD detection methods.

Proponiamo un metodo di ricerca dell'architettura per identificare una distribuzione di architetture e usarla per costruire un insieme bayesiano per il rilevamento degli outlier.


Modern applications from Autonomous Vehicles to Video Surveillance generate massive amounts of image data.In this work we propose a novel image outlier detection approach (IOD for short) that leverages the cutting-edge image classifier to discover outliers without using any labeled outlier.We observe that although intuitively the confidence that a convolutional neural network (CNN) has that an image belongs to a particular class could serve as outlierness measure to each image, directly applying this confidence to detect outlier does not work well.This is because CNN often has high confidence on an outlier image that does not belong to any target class due to its generalization ability that ensures the high accuracy in classification.To solve this issue, we propose a Deep Neural Forest-based approach that harmonizes the contradictory requirements of accurately classifying images and correctly detecting the outlier images.Our experiments using several benchmark image datasets including MNIST, CIFAR-10, CIFAR-100, and SVHN demonstrate the effectiveness of our IOD approach for outlier detection, capturing more than 90% of outliers generated by injecting one image dataset into another, while still preserving the classification accuracy of the multi-class classification problem.

Un nuovo approccio che rileva i valori anomali dai dati di immagine, preservando l'accuratezza della classificazione delle immagini


This paper introduces CloudLSTM, a new branch of recurrent neural models tailored to forecasting over data streams generated by geospatial point-cloud sources.We design a Dynamic Point-cloud Convolution (D-Conv) operator as the core component of CloudLSTMs, which performs convolution directly over point-clouds and extracts local spatial features from sets of neighboring points that surround different elements of the input.This operator maintains the permutation invariance of sequence-to-sequence learning frameworks, while representing neighboring correlations at each time step -- an important aspect in spatiotemporal predictive learning.The D-Conv operator resolves the grid-structural data requirements of existing spatiotemporal forecasting models and can be easily plugged into traditional LSTM architectures with sequence-to-sequence learning and attention mechanisms.    We apply our proposed architecture to two representative, practical use cases that involve point-cloud streams, i.e. mobile service traffic forecasting and air quality indicator forecasting.Our results, obtained with real-world datasets collected in diverse scenarios for each use case, show that CloudLSTM delivers accurate long-term predictions, outperforming a variety of neural network models.

Questo articolo introduce CloudLSTM, un nuovo tipo di modelli neurali ricorrenti su misura per la predizione su flussi di dati generati da fonti geospaziali point-cloud.


Knowledge Graphs (KG), composed of entities and relations, provide a structured representation of knowledge.For easy access to statistical approaches on relational data, multiple methods to embed a KG as components of R^d have been introduced.We propose TransINT, a novel and interpretable KG embedding method that isomorphically preserves the implication ordering among relations in the embedding space.TransINT maps set of entities (tied by a relation) to continuous sets of vectors that are inclusion-ordered isomorphically to relation implications.With a novel parameter sharing scheme, TransINT enables automatic training on missing but implied facts without rule grounding.We achieve new state-of-the-art performances with signficant margins in Link Prediction and Triple Classification on FB122 dataset, with boosted performance even on test instances that cannot be inferred by logical rules.The angles between the continuous sets embedded by TransINT provide an interpretable way to mine semantic relatedness and implication rules among relations.

Proponiamo TransINT, un metodo di embedding KG nuovo e interpretabile che conserva isomorficamente l'ordine di implicazione tra le relazioni nello spazio di embedding in un modo spiegabile, robusto e geometricamente coerente.


Unsupervised domain adaptive object detection aims to learn a robust detector on the domain shift circumstance, where the training (source) domain is label-rich with bounding box annotations, while the testing (target) domain is label-agnostic and the feature distributions between training and testing domains are dissimilar or even totally different.In this paper, we propose a gradient detach based Stacked Complementary Losses (SCL) method that uses detection objective (cross entropy and smooth l1 regression) as the primary objective, and cuts in several auxiliary losses in different network stages to utilize information from the complement data (target images) that can be effective in adapting model parameters to both source and target domains.A gradient detach operation is applied between detection and context sub-networks during training to force networks to learn discriminative representations.We argue that the conventional training with primary objective mainly leverages the information from the source-domain for maximizing likelihood and ignores the complement data in shallow layers of networks, which leads to an insufficient integration within different domains.Thus, our proposed method is a more syncretic adaptation learning process.We conduct comprehensive experiments on seven datasets, the results demonstrate that our method performs favorably better than the state-of-the-art methods by a large margin.For instance, from Cityscapes to FoggyCityscapes, we achieve 37.9% mAP, outperforming the previous art Strong-Weak by 3.6%.

Introduciamo una nuova strategia di training con obiettivo complementare basata sul gradiente per la rilevazione adattiva degli oggetti nel dominio.


Convolutional neural networks (CNN) have become the most successful and popular approach in many vision-related domains.While CNNs are particularly well-suited for capturing a proper hierarchy of concepts from real-world images, they are limited to domains where data is abundant.Recent attempts have looked into mitigating this data scarcity problem by casting their original single-task problem into a new multi-task learning (MTL) problem.The main goal of this inductive transfer mechanism is to leverage domain-specific information from related tasks, in order to improve generalization on the main task.While recent results in the deep learning (DL) community have shown the promising potential of training task-specific CNNs in a soft parameter sharing framework, integrating the recent DL advances for improving knowledge sharing is still an open problem.In this paper, we propose the Deep Collaboration Network (DCNet), a novel approach for connecting task-specific CNNs in a MTL framework.We define connectivity in terms of two distinct non-linear transformation blocks.One aggregates task-specific features into global features, while the other merges back the global features with each task-specific network.Based on the observation that task relevance depends on depth, our transformation blocks use skip connections as suggested by residual network approaches, to more easily deactivate unrelated task-dependent features.To validate our approach, we employed facial landmark detection (FLD) datasets as they are readily amenable to MTL, given the number of tasks they include.Experimental results show that we can achieve up to 24.31% relative improvement in landmark failure rate over other state-of-the-art MTL approaches.We finally perform an ablation study showing that our approach effectively allows knowledge sharing, by leveraging domain-specific features at particular depths from tasks that we know are related.

Proponiamo un nuovo approccio per collegare le reti specifiche di un task in un setting di multi-task learning basato su recenti progressi delle reti con residual connection.


Zero-Shot Learning (ZSL) is a classification task where some classes referred as unseen classes have no labeled training images.Instead, we only have side information (or description) about seen and unseen classes, often in the form of semantic or descriptive attributes.Lack of training images from a set of classes restricts the use of standard classification techniques and losses, including the popular cross-entropy loss.The key step in tackling ZSL problem is bridging visual to semantic space via learning a nonlinear embedding.A well established approach is to obtain the semantic representation of the visual information and perform classification in the semantic space.In this paper, we propose a novel architecture of casting ZSL as a fully connected neural-network with cross-entropy loss to embed visual space to semantic space.During training in order to introduce unseen visual information to the network, we utilize soft-labeling based on semantic similarities between seen and unseen classes.To the best of our knowledge, such similarity based soft-labeling is not explored for cross-modal transfer and ZSL.We evaluate the proposed model on five benchmark datasets for zero-shot learning, AwA1, AwA2, aPY, SUN and CUB datasets, and show that, despite the simplicity, our approach achieves the state-of-the-art performance in Generalized-ZSL setting on all of these datasets and outperforms the state-of-the-art for some datasets.

Come usare la cross-entropy loss per zero shot learning con soft labelling su classi non viste: una soluzione semplice ed efficace che raggiunge prestazioni all'avanguardia su cinque dataset di riferimento ZSL.


In complex tasks, such as those with large combinatorial action spaces, random exploration may be too inefficient to achieve meaningful learning progress.In this work, we use a curriculum of progressively growing action spaces to accelerate learning.We assume the environment is out of our control, but that the agent may set an internal curriculum by initially restricting its action space.Our approach uses off-policy reinforcement learning to estimate optimal value functions for multiple action spaces simultaneously and efficiently transfers data,  value estimates, and state representations from restricted action spaces to the full task.We show the efficacy of our approach in proof-of-concept control tasks and on challenging large-scale StarCraft micromanagement tasks with large, multi-agent action spaces.

Far crescere progressivamente lo spazio d'azione disponibile è un ottimo curriculum per gli agenti di apprendimento


Recently, researchers have discovered that the state-of-the-art object classifiers can be fooled easily by small perturbations in the input unnoticeable to human eyes. It is known that an attacker can generate strong adversarial examples if she knows the classifier parameters. Conversely, a defender can robustify the classifier by retraining if she has the adversarial examples. The cat-and-mouse game nature of attacks and defenses raises the question of the presence of equilibria in the dynamics.In this paper, we present a neural-network based attack class to approximate a larger but intractable class of attacks, and formulate the attacker-defender interaction as a zero-sum leader-follower game.We present sensitivity-penalized optimization algorithms to find minimax solutions, which are the best worst-case defenses against whitebox attacks.Advantages of the learning-based attacks and defenses compared to gradient-based attacks and defenses are demonstrated with MNIST and CIFAR-10.

Una soluzione game-theoretic agli attacchi e alle difese adversarial.


Supervised learning with irregularly sampled time series have been a challenge to Machine Learning methods due to the obstacle of dealing with irregular time intervals.Some papers introduced recently recurrent neural network models that deals with irregularity, but most of them rely on complex mechanisms to achieve a better performance.This work propose a novel method to represent timestamps (hours or dates) as dense vectors using sinusoidal functions, called Time Embeddings.As a data input method it and can be applied to most machine learning models.The method was evaluated with two predictive tasks from MIMIC III, a dataset of irregularly sampled time series of electronic health records.Our tests showed an improvement to LSTM-based and classical machine learning models, specially with very irregular data.

Un nuovo metodo per creare descrittori densi di tempo (Time Embeddings) per far capire le strutture temporali a modelli semplici


Community detection in graphs can be solved via spectral methods or posterior inference under certain probabilistic graphical models.Focusing on random graph families such as the stochastic block model, recent research has unified both approaches and identified both statistical and computational detection thresholds in terms of the signal-to-noise ratio.By recasting community detection as a node-wise classification problem on graphs, we can also study it from a learning perspective.We present a novel family of Graph Neural Networks (GNNs) for solving community detection problems in a supervised learning setting.We show that, in a data-driven manner and without access to the underlying generative models, they can match or even surpass the performance of the belief propagation algorithm on binary and multiclass stochastic block models, which is believed to reach the computational threshold in these cases.In particular, we propose to augment GNNs with the non-backtracking operator defined on the line graph of edge adjacencies.The GNNs are achieved good performance on real-world datasets.  In addition, we perform the first analysis of the optimization landscape of using (linear) GNNs to solve community detection problems, demonstrating that under certain simplifications and assumptions, the loss value at any local minimum is close to the loss value at the global minimum/minima.

Proponiamo una nuova architettura di graph neural network basata sulla matrice di non-backtracking definita sulle adiacenze degli archi e dimostriamo la sua efficacia nei task di rilevamento della comunità sui grafi.


Residual networks (Resnets) have become a prominent architecture in deep learning.However, a comprehensive understanding of Resnets is still a topic of ongoing research.A recent view argues that Resnets perform iterative refinement of features.We attempt to further expose properties of this aspect.To this end, we study Resnets both analytically and empirically.We formalize the notion of iterative refinement in Resnets by showing that residual architectures naturally encourage features to move along the negative gradient of loss during the feedforward phase.In addition, our empirical analysis suggests that Resnets are able to perform both representation learning and iterative refinement.In general, a Resnet block tends to concentrate representation learning behavior in the first few layers while higher layers perform iterative refinement of features.Finally we observe that sharing residual layers naively leads to representation explosion and hurts generalization performance, and show that simple existing strategies can help alleviating this problem.

Le residual connection eseguono davvero un'inferenza iterativa


We develop end-to-end learned reconstructions for lensless mask-based cameras, including an experimental system for capturing aligned lensless and lensed images for training.  Various reconstruction methods are explored, on a scale from classic iterative approaches (based on the physical imaging model) to deep learned methods with many learned parameters.  In the middle ground, we present several variations of unrolled alternating direction method of multipliers (ADMM) with varying numbers of learned parameters.The network structure combines knowledge of the physical imaging model with learned parameters updated from the data, which compensate for artifacts caused by physical approximations.Our unrolled approach is 20X faster than classic methods and produces better reconstruction quality than both the classic and deep methods on our experimental system.

Miglioriamo il tempo e la qualità della ricostruzione su un imager sperimentale senza maschera utilizzando un approccio di apprendimento end-to-end che incorpora la conoscenza del modello di imaging.


Deep learning, a rebranding of deep neural network research works, has achieved a remarkable success in recent years.With multiple hidden layers, deep learning models aim at computing the hierarchical feature representations of the observational data.Meanwhile, due to its severe disadvantages in data consumption, computational resources, parameter tuning costs and the lack of result explainability, deep learning has also suffered from lots of criticism.In this paper, we will introduce a new representation learning model, namely “Sample-Ensemble Genetic Evolutionary Network” (SEGEN), which can serve as an alternative approach to deep learning models.Instead of building one single deep model, based on a set of sampled sub-instances, SEGEN adopts a genetic-evolutionary learning strategy to build a group of unit models generations by generations.The unit models incorporated in SEGEN can be either traditional machine learning models or the recent deep learning models with a much “narrower” and “shallower” architecture.The learning results of each instance at the final generation will be effectively combined from each unit model via diffusive propagation and ensemble learning strategies.From the computational perspective, SEGEN requires far less data, fewer computational resources and parameter tuning efforts, but has sound theoretic interpretability of the learning process and results.Extensive experiments have been done on several different real-world benchmark datasets, and the experimental results obtained by SEGEN have demonstrated its advantages over the state-of-the-art representation learning models.

Introduciamo un nuovo modello di representation learning, cioè "Sample-Ensemble Genetic Evolutionary Network" (SEGEN), che può servire come un approccio alternativo ai modelli di deep learning.


How can we teach artificial agents to use human language flexibly to solve problems in a real-world environment?We have one example in nature of agents being able to solve this problem: human babies eventually learn to use human language to solve problems, and they are taught with an adult human-in-the-loop.Unfortunately, current machine learning methods (e.g. from deep reinforcement learning) are too data inefficient to learn a language in this way (3).An outstanding goal is finding an algorithm with a suitable ‘language learning prior’ that allows it to learn human language, while minimizing the number of required human interactions.In this paper, we propose to learn such a prior in simulation, leveraging the increasing amount of available compute for machine learning experiments (1).We call our approach Learning to Learn to Communicate (L2C).Specifically, in L2C we train a meta-learning agent in simulation to interact with populations of pre-trained agents, each with their own distinct communication protocol.Once the meta-learning agent is able to quickly adapt to each population of agents, it can be deployed in new populations unseen during training, including populations of humans.To show the promise of the L2C framework, we conduct some preliminary experiments in a Lewis signaling game (4), where we show that agentstrained with L2C are able to learn a simple form of human language (represented by a hand-coded compositional language) in fewer iterations than randomly initialized agents.

Proponiamo di utilizzare il meta-learning per un apprendimento linguistico più efficiente, attraverso una sorta di "randomizzazione del dominio".


We study the problem of fitting task-specific learning rate schedules from the perspective of hyperparameter optimization.  This allows us to explicitly search for schedules that achieve good generalization.We describe the structure of the gradient of a validation error w.r.t.the learning rates, the hypergradient, and based on this we introduce a novel online algorithm.Our method adaptively interpolates between two recently proposed techniques (Franceschi et al., 2017; Baydin et al.,2018), featuring increased stability and faster convergence.We show empirically that the proposed technique compares favorably with baselines and related methodsin terms of final test accuracy.

MARTHE: un nuovo metodo per adattare le schedule di learning rate specifici del task dal punto di vista dell'ottimizzazione degli iperparametri


Recent years have witnessed some exciting developments in the domain of generating images from scene-based text descriptions.These approaches have primarily focused on generating images from a static text description and are limited to generating images in a single pass.They are unable to generate an image interactively based on an incrementally additive text description (something that is more intuitive and similar to the way we describe an image). We propose a method to generate an image incrementally based on a sequence of graphs of scene descriptions (scene-graphs).We propose a recurrent network architecture that preserves the image content generated in previous steps and modifies the cumulative image as per the newly provided scene information.Our model utilizes Graph Convolutional Networks (GCN) to cater to variable-sized scene graphs along with Generative Adversarial image translation networks to generate realistic multi-object images without needing any intermediate supervision during training.We experiment with Coco-Stuff dataset which has multi-object images along with annotations describing the visual scene and show that our model significantly outperforms other approaches on the same dataset in generating visually consistent images for incrementally growing scene graphs.

Generazione interattiva di immagini da grafi di scena in crescita incrementale in più step utilizzando GAN mentre si conservano i contenuti dell'immagine generata negli step precedenti


In some important computer vision domains, such as medical or hyperspectral imaging, we care about the classification of tiny objects in large images.However, most Convolutional Neural Networks (CNNs) for image classification were developed using biased datasets that contain large objects, in mostly central image positions.To assess whether classical CNN architectures work well for tiny object classification we build a comprehensive testbed containing two datasets: one derived from MNIST digits and one from histopathology images.This testbed allows controlled experiments to stress-test CNN architectures with a broad spectrum of signal-to-noise ratios.Our observations indicate that: (1) There exists a limit to signal-to-noise below which CNNs fail to generalize and that this limit is affected by dataset size - more data leading to better performances; however, the amount of training data required for the model to generalize scales rapidly with the inverse of the object-to-image ratio (2) in general, higher capacity models exhibit better generalization; (3) when knowing the approximate object sizes, adapting receptive field is beneficial; and (4) for very small signal-to-noise ratio the choice of global pooling operation affects optimization, whereas for relatively large signal-to-noise values, all tested global pooling operations exhibit similar performance.

Studiamo scenari di classificazione a basso e bassissimo rapporto segnale-rumore, dove gli oggetti che si correlano con la label di classe occupano una piccola parte dell'intera immagine (ad esempio, immagini mediche o iperspettrali).


Recent trends of incorporating attention mechanisms in vision have led researchers to reconsider the supremacy of convolutional layers as a primary building block.Beyond helping CNNs to handle long-range dependencies, Ramachandran et al. (2019) showed that attention can completely replace convolution and achieve state-of-the-art performance on vision tasks.This raises the question: do learned attention layers operate similarly to convolutional layers?This work provides evidence that attention layers can perform convolution and, indeed, they often learn to do so in practice.Specifically, we prove that a multi-head self-attention layer with sufficient number of heads is at least as expressive as any convolutional layer.Our numerical experiments then show that self-attention layers attend to pixel-grid patterns similarly to CNN layers, corroborating our analysis.Our code is publicly available.

Un layer di self-attention può eseguire la convoluzione e spesso impara a farlo nella pratica.


We introduce a “learning-based” algorithm for the low-rank decomposition problem: given an $n \times d$ matrix $A$, and a parameter $k$, compute a rank-$k$ matrix $A'$ that minimizes the approximation loss $||A- A'||_F$.The algorithm uses a training set of input matrices in order to optimize its performance.Specifically, some of the most efficient approximate algorithms for computing low-rank approximations proceed by computing a projection $SA$, where $S$ is a sparse random $m \times n$ “sketching matrix”, and then performing the singular value decomposition of $SA$.We show how to replace the random matrix $S$ with a “learned” matrix of the same sparsity to reduce the error.Our experiments show that, for multiple types of data sets, a learned sketch matrix can substantially reduce the approximation loss compared to a random matrix $S$, sometimes by one order of magnitude.We also study mixed matrices where only some of the rows are trained and the remaining ones are random, and show that matrices still offer improved performance while retaining worst-case guarantees.

Gli algoritmi basati sull'apprendimento possono migliorare le prestazioni degli algoritmi classici per il problema dell'approssimazione low-rank mantenendo la garanzia del caso peggiore.


Neural conversational models are widely used in applications like personal assistants and chat bots.These models seem to give better performance when operating on word level.However, for fusion languages like French, Russian and Polish vocabulary size sometimes become infeasible since most of the words have lots of word forms.We propose a neural network architecture for transforming normalized text into a grammatically correct one.Our model efficiently employs correspondence between normalized and target words and significantly outperforms character-level models while being 2x faster in training and 20\% faster at evaluation.We also propose a new pipeline for building conversational models: first generate a normalized answer and then transform it into a grammatically correct one using our network.The proposed pipeline gives better performance than character-level conversational models according to assessor testing.

Architettura proposta per risolvere il task di morphological agreement


This paper proposes the use of spectral element methods \citep{canuto_spectral_1988} for fast and accurate training of Neural Ordinary Differential Equations (ODE-Nets; \citealp{Chen2018NeuralOD}) for system identification.This is achieved by expressing their dynamics as a truncated series of Legendre polynomials.The series coefficients, as well as the network weights, are computed by minimizing the weighted sum of the loss function and the violation of the ODE-Net dynamics.The problem is solved by coordinate descent that alternately minimizes, with respect to the coefficients and the weights, two unconstrained sub-problems using standard backpropagation and gradient methods.The resulting optimization scheme is fully time-parallel and results in a low memory footprint.Experimental comparison to standard methods, such as backpropagation through explicit solvers and the adjoint technique \citep{Chen2018NeuralOD}, on training surrogate models of small and medium-scale dynamical systems shows that it is at least one order of magnitude faster at reaching a comparable value of the loss function.The corresponding testing MSE is one order of magnitude smaller as well, suggesting generalization capabilities increase.

Questo articolo propone l'uso di metodi di elementi spettrali per il training veloce e accurato delle Equazioni Differenziali Ordinarie Neurali per l'identificazione del sistema.


Exploration in sparse reward reinforcement learning remains an open challenge.Many state-of-the-art methods use intrinsic motivation to complement the sparse extrinsic reward signal, giving the agent more opportunities to receive feedback during exploration.Commonly these signals are added as bonus rewards, which results in a mixture policy that neither conducts exploration nor task fulfillment resolutely.In this paper, we instead learn separate intrinsic and extrinsic task policies and schedule between these different drives to accelerate exploration and stabilize learning.Moreover, we introduce a new type of intrinsic reward denoted as successor feature control (SFC), which is general and not task-specific.It takes into account statistics over complete trajectories and thus differs from previous methods that only use local information to evaluate intrinsic motivation.We evaluate our proposed scheduled intrinsic drive (SID) agent using three different environments with pure visual inputs: VizDoom, DeepMind Lab and DeepMind Control Suite.The results show a substantially improved exploration efficiency with SFC and the hierarchical usage of the intrinsic drives.A video of our experimental results can be found at https://gofile.io/?c=HpEwTd.

Un nuovo segnale di reward intrinseca basato su feature di successo e un nuovo modo di combinare la reward estrinseca e intrinseca.


Meta-Reinforcement learning approaches aim to develop learning procedures that can adapt quickly to a distribution of tasks with the help of a few examples.Developing efficient exploration strategies capable of finding the most useful samples becomes critical in such settings.Existing approaches to finding efficient exploration strategies add auxiliary objectives to promote exploration by the pre-update policy, however, this makes the adaptation using a few gradient steps difficult as the pre-update (exploration) and post-update (exploitation) policies are quite different.Instead, we propose to explicitly model a separate exploration policy for the task distribution.Having two different policies gives more flexibility in training the exploration policy and also makes adaptation to any specific task easier.We show that using self-supervised or supervised learning objectives for adaptation stabilizes the training process and also demonstrate the superior performance of our model compared to prior works in this domain.

Proponiamo di utilizzare una policy di esplorazione separata per raccogliere le traiettorie di pre-adattamento in MAML. Mostriamo anche che l'utilizzo di un obiettivo self-supervised nel ciclo interno porta ad un training più stabile e a prestazioni molto migliori.


The “Supersymmetric Artificial Neural Network” in deep learning (denoted (x; θ, bar{θ})Tw), espouses the importance of considering biological constraints in the aim of further generalizing backward propagation. Looking at the progression of ‘solution geometries’; going from SO(n) representation (such as Perceptron like models) to SU(n) representation (such as UnitaryRNNs) has guaranteed richer and richer representations in weight space of the artificial neural network, and hence better and better hypotheses were generatable.The Supersymmetric Artificial Neural Network explores a natural step forward, namely SU(m|n) representation.These supersymmetric biological brain representations (Perez et al.) can be represented by supercharge compatible special unitary notation SU(m|n), or (x; θ, bar{θ})Tw parameterized by θ, bar{θ}, which are supersymmetric directions, unlike θ seen in the typical non-supersymmetric deep learning model.Notably, Supersymmetric values can encode or represent more information than the typical deep learning model, in terms of “partner potential” signals for example.

Generalizzazione della backpropagation, usando metodi formali dalla supersimmetria.


Regularization-based continual learning approaches generally prevent catastrophic forgetting by augmenting the training loss with an auxiliary objective.However in most practical optimization scenarios with noisy data and/or gradients, it is possible that stochastic gradient descent can inadvertently change critical parameters.In this paper, we argue for the importance of regularizing optimization trajectories directly.We derive a new co-natural gradient update rule for continual learning whereby the new task gradients are preconditioned with the empirical Fisher information of previously learnt tasks.We show that using the co-natural gradient systematically reduces forgetting in continual learning.Moreover, it helps combat overfitting when learning a new task in a low resource scenario.

Regolarizzare la traiettoria di ottimizzazione con le informazioni di Fisher dei vecchi task riduce notevolmente la catastrophic forgetting


We study the problem of generating source code in a strongly typed,Java-like programming language, given a label (for example a set ofAPI calls or types) carrying a small amount of information about thecode that is desired.The generated programs are expected to respect a`"realistic" relationship between programs and labels, as exemplifiedby a corpus of labeled programs available during training.Two challenges in such *conditional program generation* are thatthe generated programs must satisfy a rich set of syntactic andsemantic constraints, and that source code contains many low-levelfeatures that impede learning. We address these problems by traininga neural generator not on code but on *program sketches*, ormodels of program syntax that abstract out names and operations thatdo not generalize across programs.During generation, we infer aposterior distribution over sketches, then concretize samples fromthis distribution into type-safe programs using combinatorialtechniques. We implement our ideas in a system for generatingAPI-heavy Java code, and show that it can often predict the entirebody of a method given just a few API calls or data types that appearin the method.

Diamo un metodo per generare programmi type-safe in un linguaggio simile a Java, data una piccola quantità di informazioni sintattiche sul codice desiderato.


We propose an approach for sequence modeling based on autoregressive normalizing flows.Each autoregressive transform, acting across time, serves as a moving reference frame for modeling higher-level dynamics.This technique provides a simple, general-purpose method for improving sequence modeling, with connections to existing and classical techniques.We demonstrate the proposed approach both with standalone models, as well as a part of larger sequential latent variable models.Results are presented on three benchmark video datasets, where flow-based dynamics improve log-likelihood performance over baseline models.

Mostriamo come i flussi autoregressivi possono essere utilizzati per migliorare i modelli di variabili latenti sequenziali.


It is well-known that many machine learning models are susceptible to adversarial attacks, in which an attacker evades a classifier by making small perturbations to inputs.This paper discusses how industrial copyright detection tools, which serve a central role on the web, are susceptible to adversarial attacks.We discuss a range of copyright detection systems, and why they are particularly vulnerable to attacks.  These vulnerabilities are especially apparent for neural network based systems.  As proof of concept, we describe a well-known music identification method and implement this system in the form of a neural net.We then attack this system using simple gradient methods.Adversarial music created this way successfully fools industrial systems, including the AudioTag copyright detector and YouTube's Content ID system.Our goal is to raise awareness of the threats posed by adversarial examples in this space and to highlight the importance of hardening copyright detection systems to attacks.

Gli adversarial example possono ingannare il sistema di rilevamento del copyright di YouTube


Equilibrium Propagation (EP) is a learning algorithm that bridges Machine Learning and Neuroscience, by computing gradients closely matching those of Backpropagation Through Time (BPTT), but with a learning rule local in space.Given an input x and associated target y, EP proceeds in two phases: in the first phase neurons evolve freely towards a first steady state; in the second phase output neurons are nudged towards y until they reach a second steady state.However, in existing implementations of EP, the learning rule is not local in time:the weight update is performed after the dynamics of the second phase have converged and requires information of the first phase that is no longer available physically.This is a major impediment to the biological plausibility of EP and its efficient hardware implementation.In this work, we propose a version of EP named Continual Equilibrium Propagation (C-EP) where neuron and synapse dynamics occur simultaneously throughout the second phase, so that the weight update becomes local in time.We prove theoretically that, provided the learning rates are sufficiently small, at each time step of the second phase the dynamics of neurons and synapses follow the gradients of the loss given by BPTT (Theorem 1).We demonstrate training with C-EP on MNIST and generalize C-EP to neural networks where neurons are connected by asymmetric connections.We show through experiments that the more the network updates follows the gradients of BPTT, the best it performs in terms of training.These results bring EP a step closer to biology while maintaining its intimate link with backpropagation.

Proponiamo una versione continua della Propagazione dell'Equilibrio, in cui la dinamica dei neuroni e delle sinapsi avviene simultaneamente durante la seconda fase, con garanzie teoriche e simulazioni numeriche.


There are two main lines of research on visual reasoning: neural module network (NMN) with explicit multi-hop reasoning through handcrafted neural modules, and monolithic network with implicit reasoning in the latent feature space.The former excels in interpretability and compositionality, while the latter usually achieves better performance due to model flexibility and parameter efficiency.  In order to bridge the gap of the two, we present Meta Module Network (MMN), a novel hybrid approach that can efficiently utilize a Meta Module to perform versatile functionalities, while preserving compositionality and interpretability through modularized design.The proposed model first parses an input question into a functional program through a Program Generator.Instead of handcrafting a task-specific network to represent each function like traditional NMN, we use Recipe Encoder to translate the functions into their corresponding recipes (specifications), which are used to dynamically instantiate the Meta Module into Instance Modules.To endow different instance modules with designated functionality, a Teacher-Student framework is proposed, where a symbolic teacher pre-executes against the scene graphs to provide guidelines for the instantiated modules (student) to follow.In a nutshell, MMN adopts the meta module to increase its parameterization efficiency, and uses recipe encoding to improve its generalization ability over NMN.Experiments conducted on the GQA benchmark demonstrates that: (1) MMN achieves significant improvement over both NMN and monolithic network baselines; (2) MMN is able to generalize to unseen but related functions.

Proponiamo una nuova Meta Module Network per risolvere alcune delle restrizioni della precedente Neural Module Network  per ottenere forti prestazioni su dataset realistici di visual reasoning.


We propose a new perspective on adversarial attacks against deep reinforcement learning agents.Our main contribution is CopyCAT, a targeted attack able to consistently lure an agent into following an outsider's policy.It is pre-computed, therefore fast inferred, and could thus be usable in a real-time scenario.We show its effectiveness on Atari 2600 games in the novel read-only setting.In the latter, the adversary cannot directly modify the agent's state -its representation of the environment- but can only attack the agent's observation -its perception of the environment.Directly modifying the agent's state would require a write-access to the agent's inner workings and we argue that this assumption is too strong in realistic settings.

Proponiamo un nuovo attacco per prendere il pieno controllo delle policy neurali in ambienti realistici.


Cold-start and efficiency issues of the Top-k recommendation are critical to large-scale recommender systems.Previous hybrid recommendation methods are effective to deal with the cold-start issues by extracting real latent factors of cold-start items(users) from side information, but they still suffer low efficiency in online recommendation caused by the expensive similarity search in real latent space.This paper presents a collaborative generated hashing (CGH) to improve the efficiency by denoting users and items as binary codes, which applies to various settings: cold-start users, cold-start items and warm-start ones.Specifically, CGH is designed to learn hash functions of users and items through the Minimum Description Length (MDL) principle; thus, it can deal with various recommendation settings.In addition, CGH initiates a new marketing strategy through mining potential users by a generative step.To reconstruct effective users, the MDL principle is used to learn compact and informative binary codes from the content data.Extensive experiments on two public datasets show the advantages for recommendations in various settings over competing baselines and analyze the feasibility of the application in marketing.

Può generare codici hash efficaci per un'efficiente raccomandazione a freddo e nel frattempo fornire una strategia di marketing fattibile.


Recent efforts to combine Representation Learning with Formal Methods, commonly known as the Neuro-Symbolic Methods, have given rise to a new trend of applying rich neural architectures to solve classical combinatorial optimization problems.In this paper, we propose a neural framework that can learn to solve the Circuit Satisfiability problem.Our framework is built upon two fundamental contributions: a rich embedding architecture that encodes the problem structure and an end-to-end differentiable training procedure that mimics Reinforcement Learning and trains the model directly toward solving the SAT problem.The experimental results show the superior out-of-sample generalization performance of our framework compared to the recently developed NeuroSAT method.

Proponiamo un framework neurale che può imparare a risolvere il problema di Circuit Satisfiability da istanze di circuito (non annotate).


Sequence generation models such as recurrent networks can be trained with a diverse set of learning algorithms.For example, maximum likelihood learning is simple and efficient, yet suffers from the exposure bias problem.Reinforcement learning like policy gradient addresses the problem but can have prohibitively poor exploration efficiency.A variety of other algorithms such as RAML, SPG, and data noising, have also been developed in different perspectives.This paper establishes a formal connection between these algorithms.We present a generalized entropy regularized policy optimization formulation, and show that the apparently divergent algorithms can all be reformulated as special instances of the framework, with the only difference being the configurations of reward function and a couple of hyperparameters.The unified interpretation offers a systematic view of the varying properties of exploration and learning efficiency.Besides, based on the framework, we present a new algorithm that dynamically interpolates among the existing algorithms for improved learning.Experiments on machine translation and text summarization demonstrate the superiority of the proposed algorithm.

Una prospettiva unificata di vari algoritmi di apprendimento per la generazione di sequenze, come MLE, RL, RAML, data noising, ecc.


We are reporting the SHINRA project, a project for structuring Wikipedia with collaborative construction scheme.The goal of the project is to create a huge and well-structured knowledge base to be used in NLP applications, such as QA, Dialogue systems and explainable NLP systems.It is created based on a scheme of ”Resource by Collaborative Contribution (RbCC)”.We conducted a shared task of structuring Wikipedia, and at the same, submitted results are used to construct a knowledge base.There are machine readable knowledge bases such as CYC, DBpedia, YAGO, Freebase Wikidata and so on, but each of them has problems to be solved.CYC has a coverage problem, and others have a coherence problem due to the fact that these are based on Wikipedia and/or created by many but inherently incoherent crowd workers.In order to solve the later problem, we started a project for structuring Wikipedia using automatic knowledge base construction shared-task.The automatic knowledge base construction shared-tasks have been popular and well studied for decades.However, these tasks are designed only to compare the performances of different systems, and to find which system ranks the best on limited test data.The results of the participated systems are not shared and the systems may be abandoned once the task is over.We believe this situation can be improved by the following changes:1. designing the shared-task to construct knowledge base rather than evaluating only limited test data2. making the outputs of all the systems open to public so that we can run ensemble learning to create the better results than the best systems3. repeating the task so that we can run the task with the larger and better training data from the output of the previous task (bootstrapping and active learning)We conducted “SHINRA2018” with the above mentioned scheme and in this paperwe report the results and the future directions of the project.The task is to extract the values of the pre-defined attributes from Wikipedia pages.We have categorized most of the entities in Japanese Wikipedia (namely 730 thousand entities) into the 200 ENE categories.Based on this data, the shared-task is to extract the values of the attributes from Wikipedia pages.We gave out the 600 training data and the participants are required to submit the attribute-values for all remaining entities of the same category type.Then 100 data out of them for each category are used to evaluate the system output in the shared-task.We conducted a preliminary ensemble learning on the outputs and found 15 F1 score improvement on a category and the average of 8 F1 score improvements on all 5 categories we tested over a strong baseline.Based on this promising results, we decided to conduct three tasks in 2019; multi-lingual categorization task (ML), extraction for the same 5 categories in Japanese with a larger training data (JP-5) and extraction for 34 new categories in Japanese (JP-34).

Introduciamo uno schema "Resource by Collaborative Construction" per creare KB, Wikipedia strutturata


Recent image super-resolution(SR) studies leverage very deep convolutional neural networks and the rich hierarchical features they offered, which leads to better reconstruction performance than conventional methods.However, the small receptive fields in the up-sampling and reconstruction process of those models stop them to take full advantage of global contextual information.This causes problems for further performance improvement.In this paper, inspired by image reconstruction principles of human visual system, we propose an image super-resolution global reasoning network (SRGRN) to effectively learn the correlations between different regions of an image, through global reasoning.Specifically, we propose global reasoning up-sampling module (GRUM) and global reasoning reconstruction block (GRRB).They construct a graph model to perform relation reasoning on regions of low resolution (LR) images.They aim to reason the interactions between different regions in the up-sampling and reconstruction process and thus leverage more contextual information to generate accurate details.Our proposed SRGRN are more robust and can handle low resolution images that are corrupted by multiple types of degradation.Extensive experiments on different benchmark data-sets show that our model outperforms other state-of-the-art methods.Also our model is lightweight and consumes less computing power, which makes it very suitable for real life deployment.

Un modello state-of-the-art basato sul ragionamento globale per la super-risoluzione delle immagini


Graph Neural Networks (GNNs) have received tremendous attention recently due to their power in handling graph data for different downstream tasks across different application domains.The key of GNN is its graph convolutional filters, and recently various kinds of filters are designed.However, there still lacks in-depth analysis on (1) Whether there exists a best filter that can perform best on all graph data; (2) Which graph properties will influence the optimal choice of graph filter; (3) How to design appropriate filter adaptive to the graph data.In this paper, we focus on addressing the above three questions.We first propose a novel assessment tool to evaluate the effectiveness of graph convolutional filters for a given graph.Using the assessment tool, we find out that there is no single filter as a `silver bullet' that perform the best on all possible graphs.In addition, different graph structure properties will influence the optimal graph convolutional filter's design choice.Based on these findings, we develop Adaptive Filter Graph Neural Network (AFGNN), a simple but powerful model that can adaptively learn task-specific filter.For a given graph, it leverages graph filter assessment as regularization and learns to combine from a set of base filters.Experiments on both synthetic and real-world benchmark datasets demonstrate that our proposed model can indeed learn an appropriate filter and perform well on graph tasks.

Proporre un framework di valutazione per analizzare e imparare il filtro convoluzionale a grafo


The advance of node pooling operations in Graph Neural Networks (GNNs) has lagged behind the feverish design of new message-passing techniques, and pooling remains an important and challenging endeavor for the design of deep architectures.In this paper, we propose a pooling operation for GNNs that leverages a differentiable unsupervised loss based on the minCut optimization objective.For each node, our method learns a soft cluster assignment vector that depends on the node features, the target inference task (e.g., a graph classification loss), and, thanks to the minCut objective, also on the connectivity structure of the graph.Graph pooling is obtained by applying the matrix of assignment vectors to the adjacency matrix and the node features.We validate the effectiveness of the proposed pooling method on a variety of supervised and unsupervised tasks.

Un nuovo pooling layer per GNN che impara come raggruppare i nodi, secondo le loro feature, la connettività del grafo e l'obiettivo del downstream task.


Knowledge graphs are structured representations of real world facts.However, they typically contain only a small subset of all possible facts.Link prediction is the task of inferring missing facts based on existing ones.We propose TuckER, a relatively simple yet powerful linear model based on Tucker decomposition of the binary tensor representation of knowledge graph triples.By using this particular decomposition, parameters are shared between relations, enabling multi-task learning.TuckER outperforms previous state-of-the-art models across several standard link prediction datasets.

Proponiamo TuckER, un modello lineare relativamente semplice ma potente per la predizione dei link nei knowledge graph, basato sulla decomposizione Tucker della rappresentazione tensoriale binaria delle triple dei knowledge graph.


With innovations in architecture design, deeper and wider neural network models deliver improved performance on a diverse variety of tasks.But the increased memory footprint of these models presents a challenge during training, when all intermediate layer activations need to be stored for back-propagation.Limited GPU memory forces practitioners to make sub-optimal choices: either train inefficiently with smaller batches of examples; or limit the architecture to have lower depth and width, and fewer layers at higher spatial resolutions.This work introduces an approximation strategy that significantly reduces a network's memory footprint during training, but has negligible effect on training performance and computational expense.During the forward pass, we replace activations with lower-precision approximations immediately after they have been used by subsequent layers, thus freeing up memory.The approximate activations are then used during the backward pass.This approach limits the accumulation of errors across the forward and backward pass---because the forward computation across the network still happens at full precision, and the approximation has a limited effect when computing gradients to a layer's input.Experiments, on CIFAR and ImageNet, show that using our approach with 8- and even 4-bit fixed-point approximations of 32-bit floating-point activations has only a minor effect on training and validation performance, while affording significant savings in memory usage.

Un algoritmo per ridurre la quantità di memoria richiesta per il training delle deep network, basato su una strategia di approssimazione.


Estimating the frequencies of elements in a data stream is a fundamental task in data analysis and machine learning.The problem is typically addressed using streaming algorithms which can process very large data using limited storage.Today's streaming algorithms, however, cannot exploit patterns in their input to improve performance.We propose a new class of algorithms that automatically learn relevant patterns in the input data and use them to improve its frequency estimates.   The proposed algorithms combine the benefits of machine learning with the formal guarantees available through algorithm theory.  We prove that our learning-based algorithms have lower estimation errors than their non-learning counterparts.  We also evaluate our algorithms on two real-world datasets and demonstrate empirically their performance gains.

Gli algoritmi di data stream possono essere migliorati utilizzando deep learning, pur mantenendo le garanzie di prestazioni.


Link prediction in simple graphs is a fundamental problem in which new links between nodes are predicted based on the observed structure of the graph.However, in many real-world applications, there is a need to model relationships among nodes which go beyond pairwise associations.For example, in a chemical reaction, relationship among the reactants and products is inherently higher-order.Additionally, there is need to represent the direction from reactants to products.Hypergraphs provide a natural way to represent such complex higher-order relationships.Even though Graph Convolutional Networks (GCN) have recently emerged as a powerful deep learning-based approach for link prediction over simple graphs, their suitability for link prediction in hypergraphs is unexplored -- we fill this gap in this paper and propose Neural Hyperlink Predictor (NHP). NHP adapts GCNs for link prediction in hypergraphs. We propose two variants of NHP --NHP-U and NHP-D -- for link prediction over undirected and directed hypergraphs, respectively.To the best of our knowledge, NHP-D is the first method for link prediction over directed hypergraphs.Through extensive experiments on multiple real-world datasets, we show NHP's effectiveness.

Noi proponiamo Neural Hyperlink Predictor (NHP). NHP adatta le graph convolutional neural network per la predizione dei link negli ipergrafi


In this paper, we present a method for adversarial decomposition of text representation.This method can be used to decompose a representation of an input sentence into several independent vectors, where each vector is responsible for a specific aspect of the input sentence.We evaluate the proposed method on two case studies: the conversion between different social registers and diachronic language change.We show that the proposed method is capable of fine-grained con- trolled change of these aspects of the input sentence.For example, our model is capable of learning a continuous (rather than categorical) representation of the style of the sentence, in line with the reality of language use.The model uses adversarial-motivational training and includes a special motivational loss, which acts opposite to the discriminator and encourages a better decomposition.Finally, we evaluate the obtained meaning embeddings on a downstream task of para- phrase detection and show that they are significantly better than embeddings of a regular autoencoder.

Un metodo che impara rappresentazioni separate per il significato e la forma di una frase


We were approached by a group of healthcare providers who are involved in the care of chronic patients looking for potential technologies to facilitate the process of reviewing patient-generated data during clinical visits.  Aiming at understanding the healthcare providers' attitudes towards reviewing patient-generated data, we (1) conducted a focus group with a mixed group of healthcare providers.Next, to gain the patients' perspectives, we (2) interviewed eight chronic patients, collected a sample of their data and designed a series of visualizations representing patient data we collected.Last, we (3) sought feedback on the visualization designs from healthcare providers who requested this exploration.We found four factors shaping patient-generated data: data & context, patient's motivation, patient's time commitment, and patient's support circle.Informed by the results of our studies, we discussed the importance of designing patient-generated visualizations for individuals by considering both patient and healthcare provider rather than designing with the purpose of generalization and provided guidelines for designing future patient-generated data visualizations.

Abbiamo esplorato i disegn di visualizzazione che possono sostenere i pazienti cronici per presentare e rivedere i loro dati sanitari con i fornitori di assistenza sanitaria durante le visite cliniche.