Only decrement the thread backlog if it is a server.