Why did you get rid of modelsDirty?  The purpose of that is to make sure that we don't perform a full update (with joints, etc.) unless/until we get an update packet from the server with new state.  Without it, we spend a lot of time (as revealed in profiling) performing redundant transform updates on avatars.