Highlights of KDD 2019
By: Criteo Labs
Tags: Machine Learning, Data Science, AI, Kdd

The full list of talks with the corresponding slides could be found on the following link.

Applied Data Science — guest talks

Rich Caruana presented in his talk “Friends don’t let friends deploy black-box models” the importance of intelligibility and interpretability of machine learning models. Many machine learning researchers believe that if you train a deep net on enough data and it looks accurate in the test set, it is safe to deploy to production. In some context this is true, but in some specific settings, it can be extremely risky. In the study he realized in the nineties on the prediction of death for pneumonia, the most accurate algorithm was based on a Neural Net. They realized that a much simpler rule-based algorithm learned that asthma was reducing the risk of death if pneumonia occurred. Doctors confirmed that asthmatics are high risk, but it was a real pattern in the data (asthmatics notice symptoms sooner, get healthcare faster and receive more aggressive treatment). Eventually, they decided not to use the neural network in the US healthcare system, even if it performed best on test data.

Caruana motivated the usage in this context of GAMS models, due to their accuracy (comparable with neural nets in this task), but highly interpretable by domain experts. Based on the application (in this case, a decision on the treatment of ill people), he also proposed to manually edit the model based on domain experts knowledge.

Microsoft and Healthcare

Peter Lee, from Microsoft, described the potential of ML in healthcare, as well as the challenges they are facing in his talk “The Unreasonable Effectiveness, and Difficulty, of Data in Healthcare”.

Satya Nadella defined the new strategy of the company, shifting more and more to healthcare. Microsoft has many partnerships with medical institutions and hospitals, to collect data and provide analytics on top of data. It is natural to use machine learning techniques to create new innovative products in the area.

The possibilities are endless, ranging from more systems assisting radiologists to delineate tumours, computer visions system to help diagnosis of tumours, graph and knowledge extraction from medical papers (4000 new papers are published every day on PubMed!). A place where ML brings value where you do not expect: Merritt Hawkins found in a 2018 survey that 78% of doctors suffered from symptoms of burnout. A particularly stressful task are medical visits: the doctor has to take accurate notes during medical visits, maintaining empathy with the patient. Microsoft is building an assistant that takes automatically notes so that the doctor can keep eye contact with the patient, and then interprets the text to extract medical concepts. In this way, the doctor can review the notes and he is ultimately completely owner of the process. The system is there to assist him, and it learns from past corrections, reducing progressively the number of interventions that the doctor has to do.

Main challenges in the field are data collections since we miss modern standards for health data. They introduced, in a consortium on-boarding Google, IBM, Oracle and SalesForce, FHIR a standard for data models, API specs to exchange data, and a set of tools and servers to build applications with. US government is promoting FHIR as the data standard for health.

It is the first citizen in Azure, and a server is published on Github. Retailers (pharmacies) are integrating that.

He closed his talk with the message that ultimately we do not know how good is AI for prediction in medicine. Papers have often statistical methodological issues, and we miss a real perspective.

Selected papers

Revisited kd-trees

The paper describes a variation of kd-trees for nearest neighbor search with favourable probabilistic guarantees. Their method gets inspiration from Random partitioning trees, although simpler. The algorithm relies on rotating the data with random rotation and creating an ordinary axis-aligned kd-tree. The search procedure is a defeatist procedure that looks for the nearest neighbor into the insertion leaf of the query (without backtracking, and a leaf contains a fixed number of nodes). The use of multiple trees with different rotations reduces the probability of a miss (just as in random projection-based methods such as LSH). The authors also use approximate schemes to perform random rotations to reduce the computation time required for a search query. The approximate final algorithm runs on O(dlg(d) + lg(n)) with n the number of points in the database and d the dimension of the data. This is a better search query complexity compared to the original vanilla kd-trees which have complexity logarithmic in n but exponential in d. The figure below from the paper illustrates the idea behind the paper with the search space of three rotated search trees being the union of the individual search queries.

Check the reference paper here: Revisiting Kd-Trees for nearest neighbor search.

K-multiple means

The authors propose an extension to the k-means problem by relaxing the constraint of belonging to one cluster, instead they assume that the points belong to each centroid with a probability. The optimization problem seeks to find the solution minimizing the loss over the centroids and the probabilities vectors. The authors then propose an alternating optimization scheme and equivalent modelling as a constrained bipartite partitioning problem. The main motivation for the method is that it allows capturing non-convex clusters.

Optimizing impression Counts for Outdoor Advertising

This paper is an interesting one as it is the projection of retargeting to the real physical world. The authors aim to solve the problem of deploying ads on billboards in order to maximize influence or impression counts. The problem setup assumes having a set of billboards, a set of trajectories, and assumes that the influence of a billboard is a logistic function in the number of times the ad is seen by an individual driver on his journeys. In other words, the influence begins small, then increases rapidly, and then plateaus. Finding an optimal assignment that maximizes the overall influence is NP-Hard, the authors hence propose a branch and bound schema, and use a submodular estimation of the logistic influence function.

Hands-On tutorials

Deep Learning at Scale

In this session, we went through the steps of single-node deep learning model to distributed model inference and finally distributed model training and productionization. We used Keras with a Tensorflowbackend for the deep learning model. We leveraged Spark to distribute the computations across the workers and Horovod to distribute the model training. Furthermore, in order to be able to track and reproduce our numerous experiments, we used MLFlow which is an open-source platform for managing the end-to-end machine learning lifecycle. One of the main advantages of MLFlow is that it is library-agnostic. You can use it with any machine learning or deep learning models. It is even possible to mix different programming languages such as Scala, Python and R.

The slides are available here.

Concept to Code: Deep Neural Conversational System

This session showed a few deep learning algorithms for NLP. The repository with notebooks and paper references is available here.

Democratizing & Accelerating AI through Automated Machine Learning

It gave an introduction to AutoML tools in Microsoft environment. Reasons for AutoML are: it helps improving models, where AutoML tools do not need access to data, data is owned; it democratizes ML, enabling domain experts and data scientists to focus on business problems, and developers to prototype products based on ML. Finally, it accelerates the work of data scientists, that can leave hyperparameter tuning to automatic smart tools, and the ability to manage many more models than what they can do today.

Code for the tutorial is available here on Github.

“Tensorized Determinantal Point Process for Basket Completion” — our AI Lab poster session

There is a lot of exciting research happening at Criteo, we contributed to this year’s KDD with our work on Tensorized Determinantal Point Process for Recommendation. This work focuses on learning to predict the next item that should be added to an online shopping basket.

More precisely, the objective of basket completion is to suggest to a user one or more items according to items already in her cart. Early approaches involved computing a collection of rules in order to provide the recommendation, where all rules that satisfy the conditions are selected for recommendation. This is very heavy computation and not scalable for large catalogues. An alternative approach is based on determinantal point processes model co-purchase probability from item-item similarity kernel matrix and computation of determinants of the submatrices. The main contribution of this work is the generalization of the previous work on DPP for basket completion using tensorized approach enhanced by logistic regression. This new model allows us to capture ordered basket completion and we can leverage the information on the order in which items are added to a basket to improve predictive quality.