Venue: ICLR 2018 conference

Reviews:

final_score: rating: 3: Clear rejection
review_content: For me, this paper is such a combination of unusual (in the combinations of ideas that it presents) and cryptic (in its presentation) that I have found it exceedingly hard to evaluate fairly. For example, Section 4 is very unclear to me. Its relationship to Section 3.3 is also very unclear to me. 

Before that point in the paper, there are many concepts and ideas alluded to, some described with less clarity than others, but the overall focus is unclear to me and the relationship to the actual algorithms and implementation is also unclear to me. That relationship (conceptual motivation --> implementation) is exactly what would be needed in order to fully justify the inclusion (in an ICLR paper) of so much wide-ranging philosophical/conceptual discussion in the paper.

My educated guess is that other reviewers at ICLR may have related concerns. In its current state, it therefore does not feel like an appropriate paper for this particular conference. If the authors do feel that the content itself is indeed a good fit, then my strong recommendation would be to begin by re-writing it so that it makes complete sense to someone with a "standard" machine learning background. The current presentation just makes it very hard to assess, at least for this reviewer.

If other reviewers are more easily able to read and understand this paper, then I will be glad to defer to their assessments of it and will happily retract my own.
review_title: Unusual, perhaps creative paper, very cryptically presented-- hard to evaluate fairly for this reviewer

final_score: rating: 3: Clear rejection
review_content: The authors argue that many machine learning systems need a large amount of data and long training times.  To mend those shortcomings their proposed algorithm takes the novel approach of combining mathematical category theory and continental philosophy.  Instead of computation units, the concept of entities and a 'me' is introduced to solve reinforcement learning tasks on a cyber-physical system as well as the Atari environment. This allows for an AI that is understandable for humans at every step of the computation in comparison to the 'black box learning of an neural network. 
 
 
Positives:
	•	Novel approach towards more explainable and shorter training times/ less data
	•	Solid mathematical description in part 3.3
	•	Setup well explained
 
 
Negatives:
	•	Use of colloquial language (the first sentence of the abstract alone contains the word 'very' twice)
	•	Some paragraphs are strangely structured
	•	Incoherent abstract 
	•	Only brief and shallow motivation given (No evidence to support the claim)
	•.      Brief and therefore confusing mention of methods 
	•	No mention of results
	•	Very short in general
	•	Many grammatical errors (wrong tense use, misuse of a/an,... )
	•	Related Work is either Background or an explanation of the two test systems. While related approaches in those systems are also provided, the section is mainly used to introduce the test beds 
	•	No direct comparison between algorithm and existing methods is given. It is stated that some extra measures from other measures such as sensors are not used and that it learns to rank with a human in under a minute. However, many questions remain unanswered: But how good is this? How long do other systems need? Is this a valid point to raise? What score functions do other papers use?
	•	2.2: Title choice could have been  more descriptive of the subsection. 'Video Games' indicates a broader analysis of RL in any game but the section mainly restricts itself to the Atari Environment
	•	While many methods are mentioned they are not set in context but only enumerated. Many concepts are only named without explanation or how they fit into the picture the authors are trying to paint.
 
	•	A clear statement of the hypothesis and reason/motivation behind pursuing this approach is missing. Information is indirectly given in the third section where the point is raised that the approach was chosen in contrast to 'black box NNs'. This seems to be a very crucial point that could have been highlighted more. The achieved result are by no means comparable to the NN approaches but they are faster and explainable for a human. 
	•	Dreyfus' criticism of AI is presented as the key initiator for this idea. Ideas by other authors that utilise this criticism as their foundation are conceptually similar, they could have therefore been mentioned in the related work section.
	•	The paper fails to mention the current movement in the AI community to make AI more explainable. One of their two key advantages seems to be that they develop a more intuitive explainable system. However, this movement is completely ignored and not given a single mention. The paper, therefore,  does not set their approach in context and is not able to acknowledge related work in this area. 
	•	The section about continental based philosophy is rather confusing
	•	Instead of explaining the philosophy, analytical philosophy is described in details and continental philosophy is only described as not following analytical patterns. A clear introduction to this topic is missing.
	•	When described, it is stated that it's a mix of different German and French doctrines that are name dropped but not explained\ and leave the reader confused.
	•	Result section not well structured and results lack credibility:
	•	Long sections in the result section describe the actual algorithm. This should have been discussed before the results.
	•	Results for slot car are not convincing:
	•	Table 1 only shows the first the last and the best lap (and in most of them the human is better) 
	•	Not even an average measure is given only samples. This is very suspicious.
	•	Why the comparison with DQN and only DQN? How was this comparison initialised? Which parameters were used? Neither is the term DQN resolved as Deep Q-Network nor is any explanation given. There are many methods/method classes performing RL on the Atari Environment. The mention of only one comparison  leaves reasonable doubt about the claim that the system learns faster.   
 
SUMMARY: Reject. Even though the idea presented is a novel contribution and has potential the paper itself is highly unstructured and confusing and lacks a proper grammar check. No clear hypothesis is formed until section 3. The concept of Explainable AI which could have been a good motivation does not find any mentioning. Key concepts such as continental philosophy are not explained in a coherent way. The results are presented in a questionable way. As the idea is promising it is recommended to the authors to restructure the paper and conduct more experiments to be able to get accepted. 

review_title: learning for Atari 

final_score: rating: 2: Strong rejection
review_content: In this paper the authors address the very important challenges of current deep learning approaches, which is that these algorithms typically need an extraordinarily large number of training rounds to learn their strategies.  The authors note that in real life, this type of training will outstrip both the training and time budget of most real world problems.  The solution they propose is to take a high level approach and to learn more like humans do by creating strategies that involve relationships between entities rather than trying to build up strategies from pixels. 
The authors credit their reframing of their approach to AI to the “continental philosophers” (e.g. Heidegger) in opposition to the “analytical philosophers” such as Wittgenstein.  The authors associate current machine learning approaches with the analytic philosophers, based on propositions that are either provably true or untrue and their own approach as in opposition to these, however from my reading of this paper what the authors are saying is that if you start learning with higher level concepts (relationships between entities) rather than doing analysis on low level information such as pixels.   Starting with low level concepts makes learning very difficult at first and leads to a path where many trials are required.  Staring from higher level concepts such as relationships between entities allows learning to happen quickly and in a manner much more similar in nature to what humans actually do.
While the authors bring up many valid points, and in essence I believe that they may be correct, the flaw in this paper is that they do not provide methods for teaching computers to learn these higher level concepts.  The algorithms they present all require human knowledge to be encoded in the algorithms to identify the higher level concepts.  The true power of the deep learning approach is that it can actually learn from low level data, without humans hand crafting the higher level entities on their behalf.

While I agree with Dreyfus that understanding what is important and interesting given a situation would be an incredible boon to any AI algorithm, it remains an unsolved problem as to how to teach a computer to understand what is interesting in a scene with the same intuition that a human has.  In the first experiment the authors need to pre-define the concepts of a straight road and a curved road and identify them for the algorithm.  They also need to tell the algorithm exactly how to count the number of sections that the track has.  In the second experiment, to identify the “Me” in the game, the authors instruct the computer to recognize “me” as the things that move when the controller is activated.  While in some ways this is clever, mimicking what a child might do to see what moves in the world when it issues a command to move from its own brain and thus learning what “me” is, children take year to develop a sense of “self” and part of that is learning that a “concept of self” is an interesting and useful thing to have.  In their work the authors know, from their human intelligence, what are the important concepts in the game (again from a human perspective) and devise simple methods for the computer to learn these.  Again the problem here is that the human has to define the important concepts for the computer and define a specific strategy for the computer to learn to identify these important policies.  Data intensive deep learning algorithms are able to infer strategies without these concepts being defined for them.

This reframing does point out a different and perhaps better path for AI, but it is not entirely new and this paper does not present a method for getting from sensed data to higher level concepts.  For each of the experiments, the strategies used rely on human intuition to define policies.  In the first experiment with slot cars, a human needs to provide n laps of driving to imitate.  The authors identify the “shortest lap” and store it for the “AI” to replay.  The only “learning” is from an optimization that minimizes the difference between the AI’s lap time and the best lap time (tbest) of the human by scaling that recorded sample of the human driving.  This results is a strategy that is essentially just trying to replicate (imitate) what the human is doing will not lead to a generalizable learning strategy that could ever exceed a human example.   This is at best a very limited form of imitation learning.  The learning process for the second example is explained in even less detail.
Overall, this paper presents a different way of thinking about AI, one in which the amount of training time and training data required for learning is greatly reduced, however what is missing Is a generalizable algorithmic strategy for implementing this framework.m  

 




review_title:  A new approach to AI based on concepts from continental philosophies

Meta-review: This paper does not seem completely appropriate for ICLR.
Decision: Reject