TASK DEFINITION: In this task, you are given a part of an article. Your task is to generate headline (title) for this text. Preferred headlines are under fifteen words.
PROBLEM: We envision a machine learning service provider facing a continuous stream of problems with the same input domain, but with output domains that may differ. Clients present the provider with problems implicitly, by labeling a few example inputs, and then ask the provider to train models which reasonably extend their labelings to novel inputs. The provider wants to avoid constraining its users to a set of common labels, so it does not assume any particular correspondence between labels for a new task and labels for previously encountered tasks. To perform well in this setting, the provider needs a representation of the input domain which, in expectation, permits effective models for new problems to be learned efficiently from a small number of examples. While this bears a resemblance to settings considered in previous work on multitask and lifelong learning, our non-assumption of inter-task label correspondence leads to a novel algorithm: Lifelong Learner of Discriminative Representations (LLDR), which explicitly minimizes a proxy for the intra-task small-sample generalization error. We examine the relative benefits of our approach on a diverse set of real-world datasets in three significant scenarios: representation learning, multitask learning and lifelong learning.

SOLUTION: Lifelong Learning of Discriminative Representations

PROBLEM: Most past work on social network link fraud detection tries to separate genuine users from fraudsters, implicitly assuming that there is only one type of fraudulent behavior. But is this assumption true? And, in either case, what are the characteristics of such fraudulent behaviors? In this work, we set up honeypots, (“dummy” social network accounts), and buy fake followers (after careful IRB approval). We report the signs of such behaviors including oddities in local network connectivity, account attributes, and similarities and differences across fraud providers. Most valuably, we discover and characterize several types of fraud behaviors. We discuss how to leverage our insights in practice by engineering strongly performing entropy-based features and demonstrating high classification accuracy. Our contributions are (a) instrumentation: we detail our experimental setup and carefully engineered data collection process to scrape Twitter data while respecting API rate-limits, (b) observations on fraud multimodality: we analyze our honeypot fraudster ecosystem and give surprising insights into the multifaceted behaviors of these fraudster types, and (c) features: we propose novel features that give strong (>0.95 precision/recall) discriminative power on ground-truth Twitter data.

SOLUTION: The Many Faces of Link Fraud

PROBLEM: A Bayesian net (BN) is more than a succinct way to encode a probabilistic distribution; it also corresponds to a function used to answer queries. A BN can therefore be evaluated by the accuracy of the answers it returns. Many algorithms for learning BNs, however, attempt to optimize another criterion (usu­ ally likelihood, possibly augmented with a regularizing term) , which is independent of the distribution of queries that are posed. This paper takes the "performance criteria" seriously, and considers the challenge of com­ puting the BN whose performance read "accuracy over the distribution of queries" is optimal. We show that many aspects of this learning task are more difficult than the corresponding subtasks in the standard model.

SOLUTION:
Learning Bayesian Nets that Perform Well