Title: Out-of-class novelty generation: an experimental foundation
Authors: Mehdi Cherti
Balázs Kégl
Akın Kazakçı
Venue: ICLR 2017
Keywords: Deep learning
Unsupervised Learning

Abstract: Recent advances in machine learning have brought the field closer to computational creativity research. From a creativity research point of view, this offers the potential to study creativity in relationship with knowledge acquisition. From a machine learning perspective, however, several aspects of creativity need to be better defined to allow the machine learning community to develop and test hypotheses in a systematic way. We propose an actionable definition of creativity as the generation of out-of-distribution novelty. We assess several  metrics designed for evaluating the quality of generative models on this new task. We also propose a new experimental setup. Inspired by the usual held-out validation, we hold out entire classes for evaluating the generative potential of models. The goal of the novelty generator is then to use training classes to build a model that can generate objects from future (hold-out) classes, unknown at training time - and thus, are novel with respect to the knowledge the model incorporates. Through extensive experiments on various types of generative models, we are able to find architectures and hyperparameter combinations which lead to out-of-distribution novelty.



Review discussion:
The definition of novelty is relative. Its hard to draw the line between novel and random garbage.
Practical definition of novelty should be task dependent. Is there a tractable and useful task where novelty is required to achieve an objective? Have you tested on any such objectives and measure the end performance?
We are grateful for this question: it allows us to highlight several points that can be a source of confusion, about the current paper but possibly beyond: regarding the possible relationship between computational creativity and machine learning.
First and foremost, we would like to underline that the aim of the current paper is _not_ to address any particular creativity task, but to propose and test measures for quantifying the quality of generative models with respect to their capacity for artificial novelty generation & creativity systems. This, we believe, is a pre-requisite for using such models in novelty generation tasks.
We believe this to be a fundamental issue for the usability of machine learning (generative) models in computational creativity - since current metrics in ML for evaluating generative models, such as the likelihood, are not adequate for measuring novelty generation capacity. This issue is discussed lengthily in the paper.
Other criteria, and possibly new experimental settings, need to be developed to test the capacity of a model to generate a broad set of useful objects, outside the scope of what it has seen.
The current paper makes a step in this direction by adapting the held-out validation set notion to a notion of “out-of-class generation”: we were able to demonstrate that the suggested criteria is able to select _models_ that can generate a wide variety of objects, some of which can “fool” a state-of-the-art classifier trained on letters (a class of objects that the generating models have never seen).
Second,  this doesn’t preclude the existence of several tasks and domains where machine learning can be useful for artificial creativity tasks.
In fact, this is the whole premise of the constructive machine learning approach (https://nips.cc/Conferences/2016/Schedule?showEvent=6231):
“Constructive machine learning describes a class of related machine learning problems where the ultimate goal of learning is not to find a good model of the data but instead to find one or more particular instances of the domain which are likely to exhibit desired properties.”
Several such tasks and domains exist. Quoting (http://www.cs.nott.ac.uk/~psztg/cml/):
“Interesting applications are in the domains of chemistry (e.g. de novo drug design), biology (e.g. gene design, metabolic path design, RNA polymer design), computer science (e.g. automatic software generation, resource network allocation), art (e.g. music or poetry composition), gaming (e.g. character or level construction), education (e.g. personalized curricula design), services (e.g. personalized travel itinerary or insurance policy composition), layout design (e.g. urban planning, furniture arrangement, advertisement composition), alimentary (e.g. generation of novel food recipes or cocktails).”
Arguably, these domains and the associated tasks exist for some time - independently of the constructive machine learning approach.
But, typically these are the kinds of concrete problems where novelty generation can be put to use (which is an ongoing research program for many).
Third,  novelty generation, beyond any notion of task, is a topic of interest for art, and more generally, for understanding human cognition. It is a legitimate research question to investigate how  machine learning (which is the flagship of AI research) can help.
Further points:
“The definition of novelty is relative.”
We fully acknowledge this fact, in this and previous work.
“It’s hard to draw the line between novel and random garbage.”
Drawing such a line is one of the explicit objectives of our research program. Creativity is not random and measures such as likelihood do not capture its essence.
Thus, we believe, developing and testing measures such as the five measures considered in our paper is an important step.
“Practical definition of novelty should be task dependent.”
We respectfully disagree. We fully acknowledge that adopting domain-dependent definitions or including domain knowledge in an artificial system have more often than not significant “practical” utility.
However, this fact should not hinder our ability or will to come up with more general and theoretical principles.
Otherwise, establishing more general principles, and eventually a more theoretical understanding of (artificial) creativity will be harder.
“Is there a tractable and useful task where novelty is required to achieve the objective?”
Yes. See e.g. the above list of tasks from the last NIPS-CML workshop.
Also, see the general research on novelty search (e.g https://goo.gl/fKzL1B, https://goo.gl/yj7Rdb). Note that, the usual novelty search literature is based on (mostly, evolutionary) optimisation models - where knowledge is missing. Since we believe knowledge (and its structure) to be an important part of the generative process, we prefer studying it with machine learning models. A deeper issue here is the following: is there anything beyond combinatorial optimisation (e.g.
such as genetic algorithms) that applied mathematics can offer for supporting & modeling creativity. We believe the hierarchical knowledge models to offer an interesting alternative to be studied in this respect.
“Have you tested on any such objectives and measure the end performance?”
No. As explained above, the current paper proposes metrics for choosing better generative models for any such creativity task that will be solved later.
Although not reported here, we are fully exploring this avenue.
On the notion of “task”.
Consider the following. a) Reducing the temperature of an engine during runtime. b) An artist playing guitar. The first is a clear-cut “task” with a trivial metric. What would be the metric for the second case? Obviously, such creative and/or open-ended activities elude metrics at this stage.
Not only a rigorous effort is needed for developing & testing metrics (as our paper attempts) for a scientific study of the notion, but also we need _new&adapted engineering principles_.
The fundamental engineering principle in machine learning (which allowed so far fruitful exploration and systematic study) is the train-test validation setup. Our work suggest a similar setup where the held-out objects are entire classes (instead of data points).
This is normal since, we do want to simulate the unknown (by held-out classes) instead of future (data points).
All of MNIST were written by people who knew and have written letters. Some samples from MNIST would be indeed expected look like letters.
Some letters do look like some MNIST digits, but there are plenty of letters (typically a w, an m or even a c) that don't, and we do generate samples of those. As demonstrated in the paper, if we optimize for in-class, then we are much less likely to obtain letters.
Nevertheless, as reported in the paper (and shown, for example, in Figure 3), other models optimizing other metrics are able to generate letters that do not look like digits.
2) We are not only generating letters, we are using the setup as a proxy to select models that generate novelty. Figure 3 should be quite convincing that most of the symbols look nothing like digits. Our proxy is working in the sense that it can separate in-class (mostly digits) samples (including the MNIST test set) from samples that contain non-digits (novelty).
3) Capturing the internal rules of the MNIST writers that enable them them drawing  MNIST symbols and then use these rules to tap into this "imagination" space in order to draw other (imaginary) symbols is precisely the mechanism we wish to demonstrate through our research program: We believe this knowledge-driven process to be what ML has to offer to novelty generation and creativity research (automated capture of knowledge from a set of referential objects, and the use of this knowledge model to go beyond this reference).
Are Parzen likelihood estimates reported anywhere? I could only find Table 3 reporting correlation of Parzen likelihoods between models.
In the following are reported all the Parzen likelihood estimates of log-likelihood for all the models we trained:
- for in-class Parzen estimates, we evaluated on digits test data (mnist) :  https://goo.gl/n68jIr
- for out-of-class Parzen estimates, we evaluated on letters test data :  https://goo.gl/eVnfev
The id column is a model identifier, the following columns are mean and standard deviation of Parzen likelihood estimates of log-likelihood.
We did not report absolute scores in the paper because, with simplicity as an objective in mind, we felt that the inter-score correlations and correlations with human judgement were sufficient to make our point.
But we can certainly put them in the paper. It would be useful if the you could explain why you feel that reporting the Parzen likelihoods would improve the paper.
This paper proposed a quantitative metric for evaluating out-of-class novelty of samples from generative models. The authors evaluated the proposed metric on over 1000 models with different hyperparameters and performed human subject study on a subset of them.
The authors mentioned difficulties in human subject studies, but did not provide details of their own setting. An "in-house" annotation tool was used but it's unclear how many subjects were involved, who they are, and how many samples were presented to each subject. I'm worried about the diversity in the subjects because there may be too few subjects who are shown too many samples and/or are experts in this field.
This paper aims at proposing a general metric for novelty but the experiments only used one setting, namely generating Arabic digits and English letters. There is insufficient evidence to prove the generality of the proposed metric.
Moreover, defining English letters as "novel" compared to Arabic digits is questionable. What if the model generates Arabic or Indian letters? Can a human who has never seen Arabic handwriting tell it from random doodle? What makes English letters more "novel" than random doodle? In my opinion these questions are best answered through large scale human subject study on tasks that has clear real world meanings.
For example, do you prefer painting A (generated) or B (painted by artist).
We thank the reviewer for his/her comments. We believe his/her criticism does not apply to our work and we feel that he/she may have misunderstood the problem our paper deals with, as the following quotes demonstrate:
“This paper proposed a quantitative metric for evaluating out-of-class novelty of samples from generative models.”
“This paper aims at proposing a general metric for novelty but the experiments only…”
The paper proposes metrics for model selection, _not_ for novelty selection.
Thus, the issues raised about universality of a novelty metric, novelty or artistic value of letters, experimental conditions with human subjects are all out of the scope. One should not confuse the objective of this work:
Selecting a good generative model for novelty generation is an open question. Current work on deep generative modeling learns a set of objects (e.g. digits or bedrooms) and generate the same _kinds_ of objects (e.g. digits or bedrooms).
Clearly, when a generative model is used in this way, it is unlikely to generate novelty.
Thus, the scientific question here is what setup would enable us to measure the novelty generation capacity of a deep generative model, where novelty is defined as above - generating objects from categories unknown to the machine.
This idea is the basis of the setup introduced in the paper: much as we, as humans, know Roman letters, the generative models we train do not.
Albeit this fact, some of the 1000 or so models we trained are able to generate objects that a separate letter-discriminator model classifies as letters. This, we believe, is a good proxy for testing the capacity of a generative model to generate unknown categories of objects, much in the same way, held-out data points enabling testing the generalization capacity of a predictive model.
Furthermore, we would like to highlight the following point: if our objective was to generate art or genuine novelty, we would have used some data sets other than roman letters or digits.
That's precisely what we _don't_ want to do, since in more complex systems and datasets, it is unclear whether the computer generates art (or novelty) or whether the object generated based on a possibly infinite combinatorics is _perceived_ as such by the humans.
In such complicated systems, the frontier of what the system creator has put in and what the machine adds is often blurred. To be able to start investigating what the machine can come up with without any value objective imposed by its creator, starting with a simple and controlled setup is only a best practice in science.
The authors proposed an way to measure the generation of out-of-distribution novelty. Their methods implied, if a model trained on MNIST digits could generate some samples are more like letters judged by anther model trained both on MNIST and letters,  the model trained on MNIST could be seen as having  the ability to generate novel samples. Some empirical experiments were reported.
The novelty is hard to define. The proposed metric is also problematic. A naive combination of MNIST and letters dataset do not represent the natural distribution of handwritten digits and letters. IT means that the model trained on the combination could not properly distinguished digits and letters. The proposed out-of-class count and out-of-class max are thus pointless. For the "novel" samples in Fig. 3,  they are clearly digits. I guess they quantize the samples to binary. If they would quantize the samples to 8 bit, the resulting images would look even more like digits.
The reviewer states:
“For the "novel" samples in Fig. 3,  they are clearly digits.”
We kindly ask the reviewer to classify the symbols on the following panel into one of the digit categories 0-9. They are from Figure 3. We de-binarized them (quantized them to 8 bits) as you suggested.
https://lh3.googleusercontent.com/-Bs646MOG89g/WFalSDVY2JI/AAAAAAAAOfk/cvGQqQQZeTw6PMWdih_LrGyYUfFW0LoTwCL0B/h88/panel2.png
Also, we kindly ask the reviewer to comment on why our brains can read the following pangrams as sentences, and not digits:
https://lh3.googleusercontent.com/-zcZv3_saVkQ/WFe294YB9pI/AAAAAAAAFGg/yoODIjOQSBkc_ZGmiG5eI7_6hCJId5uDQCL0B/h282/2016-12-19.jpg
Note that the models were never _asked_ to generate letters; they just do.
With respect to the limited knowledge the model has, how is this not novel?
The problem we are dealing with is not whether we can learn or not a supposedly natural distribution of handwritten digits and letters, from digits only.
The problem we are dealing with is this:
Suppose we ask a thousand children, who only know how to write digits, to write something different - How would you evaluate these children on their ability to come up with coherent, and even meaningful symbols?
Please replace child/children in the above two sentences with “generative models”.
First, the bad:
This paper is frustratingly written. The grammar is fine, but:
- The first four pages are completely theoretical and difficult to follow without any concrete examples. These sections would benefit greatly from a common example woven through the different aspects of the theoretical discussion.
- The ordering of the exposition is also frustrating. I found myself constantly having to refer ahead to figures and back to details that were important but seemingly presented out of order. Perhaps a reordering of some details could fix this. Recommendation: give the most naturally ordered oral presentation of the work and then order the paper similarly.
Finally, the description of the experiments is cursory, and I found myself wondering whether the details omitted were important or not.
Including experimental details in a supplementary section could help assuage these fears.
The good:
What the paper does well is to gather together past work on novelty generation and propose a unified framework in which to evaluate past and future models. This is done by repurposing existing generative model evaluation metrics for the task of evaluating novelty. The experiments are basic, but even the basic experiments go beyond previous work in this area (to this reviewer’s knowledge).
Overall I recommend the paper be accepted, but I strongly recommend rewriting some components to make it more digestible. As with other novelty papers, it would be read thoroughly by the interested few, but it is likely to fight an uphill battle against the majority of readers outside the sub-sub-field of novelty generation;
for this reason the theory should be made even more intuitive and clear and the experiments and results even more accessible.
We are grateful to the reviewer:
“Paper fights a difficult battle to defend and unify novelty generating models. It fights somewhat well.”
“the basic experiments go beyond previous work in this area”
We think the following points is especially relevant, for this kind of paper:
As with other novelty papers, [...] it is likely to fight an uphill battle against the majority of readers outside the sub-sub-field of novelty generation;
We contend that this is true for many cross-domain, multidisciplinary work. The chance to present the paper in the conference will surely improve future versions and reduce incomprehensions that typically arise in such efforts.
“for this reason the theory should be made even more intuitive and clear and the experiments and results even more accessible.”
Other reviewers have stated that the paper is well-written and easy to understand for machine learning researchers.
Nevertheless, we took into account reviewer 3’s suggestion: We made some modifications in the paper that will clarify that the readers interested into the metrics and the experiments can jump to the relevant sections.
This paper examines computational creativity from a machine learning perspective. Creativity is defined as a model's ability to generate new types of objects unseen during training. The authors argue that likelihood training and evaluation are by construction ill-suited for out-of-class generation and propose a new evaluation framework which relies on the use of held-out classes of objects to measure a model's ability to generate new and interesting object types.
I am not very familiar with the literature on computational creativity research, so I can't judge on how well this work has been put into the context of existing work. From a machine learning perspective, I find the ideas presented in this paper new, interesting and thought-provoking.
As I understand, the hypothesis is that the ability of a model to generate new and interesting types we *do not* know about correlates with its ability to generate new and interesting types we *do* know about, and the latter is a good proxy for the former. The extent to which this is true depends on the bias introduced by model selection. Just like when measuring generalization performance, one should be careful not to reuse the same held-out classes for model selection and for evaluation.
Nevertheless, I appreciate the effort that has been made to formalize the notion of computational creativity within the machine learning framework. I view it as an important first step in that direction, and I think it deserves its place at ICLR, especially given that the paper is well-written and approachable for machine learning researchers.
We are grateful to the reviewer for qualifying the ideas presented in the paper as “new, interesting and thought-provoking” and judging that our paper deserves its place at ICLR.
We also thank him or her for acknowledging the importance of our effort in formalizing creativity from a machine learning perspective.
Indeed, as the reviewer states, we strived to make the topic approachable by machine learning researchers, both for enabling a transfer of knowledge across domains and to bring a systematic and rigorous evaluation scheme for computational creativity work.

Meta-review:
role: PC
data: comment: This paper aims to present an experimental framework for selecting machine learning models that can generate novel objects. As the work is devoted to a relatively subjective area of study, it is not surprising that opinions of the work are mixed.
 
 A large section of the paper is devoted to review, and more detail could be given to the experimental framework. It is not clear whether the framework can actually be useful outside the synthetic setup described. Moreover, I worry it encourages unhealthy directions for the field. Over 1000 models were trained and evaluated. There is no form of separate held-out comparison: the framework encourages people to keep trying random stuff until the chosen measure reports success.

Decision: Reject