In this task, you are given a part of an article. Your task is to generate headline (title) for this text. Preferred headlines are under fifteen words.

We envision a machine learning service provider facing a continuous stream of problems with the same input domain, but with output domains that may differ. Clients present the provider with problems implicitly, by labeling a few example inputs, and then ask the provider to train models which reasonably extend their labelings to novel inputs. The provider wants to avoid constraining its users to a set of common labels, so it does not assume any particular correspondence between labels for a new task and labels for previously encountered tasks. To perform well in this setting, the provider needs a representation of the input domain which, in expectation, permits effective models for new problems to be learned efficiently from a small number of examples. While this bears a resemblance to settings considered in previous work on multitask and lifelong learning, our non-assumption of inter-task label correspondence leads to a novel algorithm: Lifelong Learner of Discriminative Representations (LLDR), which explicitly minimizes a proxy for the intra-task small-sample generalization error. We examine the relative benefits of our approach on a diverse set of real-world datasets in three significant scenarios: representation learning, multitask learning and lifelong learning.
Lifelong Learning of Discriminative Representations

In this paper we present methods for attacking and defending k-gram statistical analysis techniques that are used, for example, in network traffic analysis and covert channel detection. The main new result is our demonstration of how to use a behavior’s or process’ k-order statistics to build a stochastic process that has those same k-order stationary statistics but possesses different, deliberately designed, (k + 1)order statistics if desired. Such a model realizes a “complexification” of the process or behavior which a defender can use to monitor whether an attacker is shaping the behavior. By deliberately introducing designed (k + 1)-order behaviors, the defender can check to see if those behaviors are present in the data. We also develop constructs for source codes that respect the k-order statistics of a process while encoding covert information. One fundamental consequence of these results is that certain types of behavior analyses techniques come down to an arms race in the sense that the advantage goes to the party that has more computing resources applied to the problem. Points of view in this document are those of the authors and do not necessarily represent the official position of the sponsoring agencies or the U.S. Government. V. Crespi is with the Department of Computer Science, California State University at Los Angeles, Los Angeles CA, 90032 USA. email: vcrespi@calstatela.edu. Crespi’s work was partially supported by AFOSR Grant FA9550-07-1-0421 and by NSF Grant HRD-0932421. G. Cybenko is with the Thayer School of Engineering, Dartmouth College, Hanover NH 03755. email: gvc@dartmouth.edu. Cybenko’s work was partially supported by Air Force Research Laboratory contracts FA8750-10-1-0045, FA8750-09-1-0174, AFOSR contract FA9550-07-1-0421, U.S. Department of Homeland Security Grant 2006-CS-001-000001 and DARPA Contract HR001-06-1-0033 A. Giani is with the Department of EECS, University of California at Berkeley, Berkeley CA 94720. email: agiani@eecs.berkeley.edu. Giani’s’s work was partially supported by U.S. Department of Homeland Security Grant 2006-CS001-000001 and DARPA Contract HR001-06-1-0033 when she was a Ph.D. student at Dartmouth April 28, 2011 DRAFT ar X iv :1 10 4. 50 71 v1 [ cs .L G ] 2 7 A pr 2 01 1 CRESPI, CYBENKO, GIANI 2
Attacking and Defending Covert Channels and Behavioral Models

Common statistical practice has shown that the full power of Bayesian methods is not realized until hierarchical priors are used, as these allow for greater “robustness” and the ability to “share statistical strength.” Yet it is an ongoing challenge to provide a learning-theoretically sound formalism of such notions that: offers practical guidance concerning when and how best to utilize hierarchical models; provides insights into what makes for a good hierarchical prior; and, when the form of the prior has been chosen, can guide the choice of hyperparameter settings. We present a set of analytical tools for understanding hierarchical priors in both the online and batch learning settings. We provide regret bounds under log-loss, which show how certain hierarchical models compare, in retrospect, to the best single model in the model class. We also show how to convert a Bayesian log-loss regret bound into a Bayesian risk bound for any bounded loss, a result which may be of independent interest. Risk and regret bounds for Student’s t and hierarchical Gaussian priors allow us to formalize the concepts of “robustness” and “sharing statistical strength.” Priors for feature selection are investigated as well. Our results suggest that the learning-theoretic benefits of using hierarchical priors can often come at little cost on practical problems.

Risk and Regret of Hierarchical Bayesian Learners