Tweak RemoveLog logic

1. Use removed_by as the foreign key for removed_by column in logs
2. Use update_columns instead of update_attributes