This should be `remove_foreign_key :user_id` which will also drop the column. MySQL does not automatically drop foreign keys.