require square matrices in trace(). fixes #3319
allow AbstractVector in dot()