fix delete_or_destroy on association

This fix CI..

### Problem
When having a `inverse_of`, `.find` method in the association will return the actual object if the given id match with the object in memory. In that case the `delete_or_destroy` could receive one record only, and we cannot call `flatten` if one record given.

### Solution
Only call flatten if it is an array.

related to #12359:
As on that PR I added a `inverse_of` for `clients_of_firm`, the destroy stopped working as it had this other bug.


review @rafaelfranca