The process can take 4-8 weeks on average. However, this is assuming that all planning is done efficiently and that you're ordering your products to fill the man cave in time, or you already have them to hand.
It does also depend on the size of the space, so it's not uncommon for any home build projects like this to go into the couple of months territory in regards to time.