It depends on how you use WordPress. The more themes or plugins you install, the higher the PHP memory limit you will need.
You can start with the default limit and then increase it as needed at any time.
See this help article for more information on PHP memory limits and how WordPress works.