The time it takes to get from one celestial body to another depends largely on the energy that one is willing to expend. Here  "energy" refers  to the effort put in by the launch vehicle and the sum of the manoeuvres of the rocket motors aboard the spacecraft, and the amount of propellant that is used. In space travel, everything boils down to energy. Spaceflight is the clever management of energy.
Some common solutions for transfers to the moon are 1) the Hohmann-like transfer and 2) the Free Return Transfer. The Hohmann Transfer is often referred to as the one that requires the lowest energy, but that is true only if you want the transfer to last only a few days and, in addition, if some constraints on the launch apply. Things get very complicated from there on, so I won't go into details.
The transfer duration for the Hohmann-like transfer is around 5 days. There is some variation in this duration because the moon orbit is eccentric, so its distance from the Earth varies quite a bit with time, and with it, the characteristics of the transfer orbit.
The Free Return transfer is a popular transfer for manned spacecraft. It requires more energy than the Hohmann-like transfer, but it is a lot safer, because its design is such that if the rocket engine fails at the moment you are trying to insert into the orbit around the Moon, the gravity of the Moon  will deflect the orbit exactly such that it returns to the Earth. So even with a defective propulsion system, you can still get the people back safely. The Apollo missions flew on Free Return transfers. They take around 3 days to reach the moon.