Project: usergrid
Issue USERGRID-471: Long Pauses and Errors from ES on deleteByQuery
When a single thread is doing PUTs against Usergrid, one after the other, ES can time out on deleteByQuery and block the caller. The response time from Usergrid goes from 200ms to ~60s during this time. Note that the error in ES logs indicates a 1 minute timeout. [2015-03-10 17:37:42,947][DEBUG][action.deletebyquery ] [res003sy] observer: timeout notification from cluster service. timeout setting [1m], time since start [1m] [2015-03-10 17:37:42,947][DEBUG][action.deletebyquery ] [res003sy] observer: timeout notification from cluster service. timeout setting [1m], time since start [1m] [2015-03-10 17:37:42,948][DEBUG][action.deletebyquery ] [res003sy] observer: timeout notification from cluster service. timeout setting [1m], time since start [1m] [2015-03-10 17:37:42,948][DEBUG][action.deletebyquery ] [res003sy] observer: timeout notification from cluster service. timeout setting [1m], time since start [1m] [2015-03-10 17:37:42,949][DEBUG][action.deletebyquery ] [res003sy] observer: timeout notification from cluster service. timeout setting [1m], time since start [1m] [2015-03-10 17:37:42,949][DEBUG][action.deletebyquery ] [res003sy] observer: timeout notification from cluster service. timeout setting [1m], time since start [1m] [2015-03-10 17:37:43,007][DEBUG][action.deletebyquery ] [res003sy] observer: timeout notification from cluster service. timeout setting [1m], time since start [1m] [2015-03-10 17:37:43,008][DEBUG][action.deletebyquery ] [res003sy] observer: timeout notification from cluster service. timeout setting [1m], time since start [1m] [2015-03-10 17:37:43,008][DEBUG][action.deletebyquery ] [res003sy] observer: timeout notification from cluster service. timeout setting [1m], time since start [1m] [2015-03-10 17:37:43,008][DEBUG][action.deletebyquery ] [res003sy] observer: timeout notification from cluster service. timeout setting [1m], time since start [1m] [2015-03-10 17:37:43,008][DEBUG][action.deletebyquery ] [res003sy] observer: timeout notification from cluster service. timeout setting [1m], time since start [1m] [2015-03-10 17:37:43,008][DEBUG][action.deletebyquery ] [res003sy] observer: timeout notification from cluster service. timeout setting [1m], time since start [1m] [2015-03-10 17:37:43,008][DEBUG][action.deletebyquery ] [res003sy] observer: timeout notification from cluster service. timeout setting [1m], time since start [1m] [2015-03-10 17:37:43,012][DEBUG][action.deletebyquery ] [res003sy] observer: timeout notification from cluster service. timeout setting [1m], time since start [1m] [2015-03-10 17:37:43,013][DEBUG][action.deletebyquery ] [res003sy] observer: timeout notification from cluster service. timeout setting [1m], time since start [1m] [2015-03-10 17:37:43,013][DEBUG][action.deletebyquery ] [res003sy] observer: timeout notification from cluster service. timeout setting [1m], time since start [1m] [2015-03-10 17:37:43,013][DEBUG][action.deletebyquery ] [res003sy] observer: timeout notification from cluster service. timeout setting [1m], time since start [1m] [2015-03-10 17:37:43,013][DEBUG][action.deletebyquery ] [res003sy] observer: timeout notification from cluster service. timeout setting [1m], time since start [1m] [2015-03-10 17:37:43,013][DEBUG][action.deletebyquery ] [res003sy] observer: timeout notification from cluster service. timeout setting [1m], time since start [1m] [2015-03-10 17:37:43,013][DEBUG][action.deletebyquery ] [res003sy] observer: timeout notification from cluster service. timeout setting [1m], time since start [1m] [2015-03-10 17:37:43,072][DEBUG][action.deletebyquery ] [res003sy] observer: timeout notification from cluster service. timeout setting [1m], time since start [1m] [2015-03-10 17:37:43,073][DEBUG][action.deletebyquery ] [res003sy] observer: timeout notification from cluster service. timeout setting [1m], time since start [1m] [2015-03-10 17:37:43,073][DEBUG][action.deletebyquery ] [res003sy] observer: timeout notification from cluster service. timeout setting [1m], time since start [1m] [2015-03-10 17:37:43,073][DEBUG][action.deletebyquery
Story points: 3