TASK DEFINITION: You are given an original reference as well as a system generated reference. Your task is to judge the naturaleness of the system generated reference. If the utterance could have been produced by a native speaker output 1, else output 0.
PROBLEM: System Reference: may i ask near where?
Original Reference: would you like to know where this this hotel is near?.

SOLUTION: 0

PROBLEM: System Reference: what area would you like the restaurant to be in?
Original Reference: what area do you want?.

SOLUTION: 1

PROBLEM: System Reference: arabian nights restaurant, is in the are of hayes valley or noma.
Original Reference: arabian nights restaurant is in the hayes valley or noma area.

SOLUTION:
0