TASK DEFINITION: In this task, you are given a part of an article. Your task is to generate headline (title) for this text. Preferred headlines are under fifteen words.
PROBLEM: Collaborative data consist of ratings relating two distinct sets of objects: users and items. Much of the work with such data focuses on filtering: predicting unknown ratings for pairs of users and items. In this paper we focus on the problem of visualizing the information. Given all of the ratings, our task is to embed all of the users and items as points in the same Euclidean space. We would like to place users near items that they have rated (or would rate) high, and far away from those they would give low ratings. We pose this problem as a real-valued non-linear Bayesian network and employ Markov chain Monte Carlo and expectation maximization to find an embedding. We present a metric by which to judge the quality of a visualization and compare our results to Eigentaste, locally linear embedding and cooccurrence data embedding on three real-world datasets.

SOLUTION: Visualization of Collaborative Data

PROBLEM: In this paper we present the greedy step averaging(GSA) method, a parameter-free stochastic optimization algorithm for a variety of machine learning problems. As a gradient-based optimization method, GSA makes use of the information from the minimizer of a single sample’s loss function, and takes average strategy to calculate reasonable learning rate sequence. While most existing gradient-based algorithms introduce an increasing number of hyper parameters or try to make a trade-off between computational cost and convergence rate, GSA avoids the manual tuning of learning rate and brings in no more hyper parameters or extra cost. We perform exhaustive numerical experiments for logistic and softmax regression to compare our method with the other state of the art ones on 16 datasets. Results show that GSA is robust on various scenarios.

SOLUTION: Greedy Step Averaging: A parameter-free stochastic optimization method

PROBLEM: Deep Neural Networks (DNNs) are presently the state-of-the-art for image classification tasks. However, recent works have shown that these systems can be easily fooled to misidentify images by modifying the image in particular ways, often rendering them practically useless. Moreover, defense mechanisms proposed in the literature so far are mostly attack-specific and prove to be ineffective against new attacks. Indeed, recent work on universal perturbations can generate a single modification for all test images that is able to make existing networks misclassify 90% of the time. Presently, to our knowledge, no defense mechanisms are effective in preventing this. As such, the design of a general defense strategy against a wide range of attacks for Neural Networks becomes a challenging problem. In this paper, we derive inspiration from recent advances in the field of cybersecurity and multi-agent systems, and propose to use the concept of Moving Target Defense (MTD) for increasing the robustness of well-known deep networks trained on the ImageNet dataset towards such adversarial attacks. In using this technique, we formalize and exploit the notion of differential immunity of different networks to specific attacks. To classify a single test image, we pick one of the trained networks each time and then use its classification output. To ensure maximum robustness, we generate an effective strategy by formulating this interaction as a Repeated Bayesian Stackelberg Game (BSG) with a Defender (who hosts the classification networks) and Users (both Legitimate users and Attackers). As a network switching strategy, we compute a Strong Stackelberg Equilibrium that optimizes the accuracy of prediction while at the same time reduces the misclassification rate on adversarial modification of test images. We show that while our approach produces an accuracy of 92.79% for the legitimate users, attackers can only misclassify images 58% (instead of 93.7%) of the time even when they select the best attack available to them. This is at least twice as good, to sometimes even an order of magnitude better, compared the accuracy rates of the worst affected networks.

SOLUTION:
Securing Deep Neural Nets against Adversarial Attacks with Moving Target Defense