The effective population size is the size for which expected heterozygosity matches observed heterozygosity. Individuals vary greatly in number of progeny. Numbers of males and females are not equal. Population size fluctuates. Subdivided population structure.
Successive generations are a random sample from the parental gene pool.
That is, indeed, the Hardy-Weinberg expectation. Understand that random samples are NOT selection. Drift occurs gradually over many generations. The founder effect results from a single small sample when a population is first founded. Inbreeding can reduce the effective size of a population.