Workaround for _.difference bug of lodash

/cc @arikon @scff