Only cache attributes for ActiveRecord objects

Remove TODO; cleanup condition.