[libcommhistory] Optimize RecentContactsModel query

For a large database of calls and messages, this improves the query time
from about 2100ms to 60ms.