now assuming sane field names and reporting progress every thousand records
fara indexes, leaving out fulltext for now
api endpoints