Hm, would also like to figure this out. Is it possible that it's because we're using the fake adapter: https://github.com/assaf/vanity/blob/master/lib/vanity/experiment/ab_test.rb#L460?