Patent Document ID: 9589579
Application ID: 14302096

Base Claim:
1. One or more non-transitory computer-readable storage media storing instructions that, when executed by one or more processors, cause the one or more processors to perform operations comprising: evaluating a natural language processing system with a group of input-result pairs, each input-result pair of the group of input-result pairs mapping an input to a result, the evaluating including: processing each input of the group of input-result pairs with the natural language processing system to identify a result; and determining, for each input of the group of input-result pairs, whether or not the result of the processing matches the result of the input-result pair; and in response to determining that a result of the processing does not match a corresponding result of the input-result pair, providing a user interface with an indicator for the result of the processing that does not match the corresponding result of the input-result pair to enable a user to modify the natural language processing system.

---

Claim 2:
2. The one or more non-transitory computer-readable storage media of claim 1 , wherein the natural language processing system is implemented at least in part by a knowledge base, the knowledge base including a set of concept-based units that are each paired with a response.