Note the difference between the Physical Universe and the Observable Universe. This is related to horizons. This is our causal horizon - beyond this distance something would have to travel faster than the speed of light in order to communicate with us. All observers are surrounded by such a horizon.
Horizons are okay.
Our assumption about homogeneity means that the stuff beyond the horizon is the same stuff we already know about. This assumption must be correct due to horizon overlaps and causality. The observable Universe is the one which is defined by our horizon. This is always finite. Note however, that as H approaches zero, which it must since gravity is slowing down the expansion rate of the universe, the horizon size approaches infinity and hence the physical universe and the observable universe become the same.