No, you'll have plenty of locations to collect the material and can go to your friend's world for collecting. I agree that there are respawn timers for in-game materials, and they may take a while to accumulate. But you don't have to worry about them not being available after you collect; they will respawn after some time in your world. If you're a casual player or won't be able to dedicate much time to farming then, You can take your time to mark the locations and collect them as you feel.