Patent Document ID: 8571869
Application ID: 12121329

Base Claim:
1. A method comprising: obtaining first data from a plurality of calls routed by a natural language call routing system that uses a first speech recognition model to recognize speech of the plurality of calls and a first action classification model to route calls based on the recognized speech, the first data comprising audio data from the plurality of calls, a first N-best list of word sequences generated by using the first speech recognition model to recognize at least a portion of the audio data and associated call classification data indicating how each of the plurality of calls were routed by the natural language call routing system; and modifying the first speech recognition model and the first action classification model based at least in part on the first data, including using the first N-best list of word sequences, to obtain a second speech recognition model and a second action classification model; and modifying the second speech recognition model and the second action classification model based at least in part on the first data to obtain a third speech recognition model and a third action classification model, wherein the modifying comprises modifying the second speech recognition model and the second action classification model by using the first N-best list or a second N-best list of word sequences generated by using the second speech recognition model to recognize at least the portion of the audio data.

---

Claim 6:
6. The method of claim 1 , further comprising: presenting data indicative of performance of said natural language call routing system to at least one human, said data including a number of calls misdirected by the natural language call routing system and/or confidence scores associated with operation of the natural language call routing system.