Vertical polarisation is when the electric field is vertical.
When people talk about polarisation they are referring to the electric field vector. References to 'vertical' and 'horizontal' are frequent since terrestrial antennas are normally vertical or horizontal relative to the ground. At an instant the top is positive voltage and the bottom is negative voltage - so the electric field produced is vertical. Vertical polarisation is produced by the open end of a rectangular waveguide whose narrow dimension is vertical and whose broad face is horizontal. Inside the waveguide think in terms of voltage between the centre lines of the the two broad faces.