Allow to disable already enabled observer

Related: https://github.com/rails/rails/pull/2514