Whether your conference guests dial in too early or too late, it shouldn't cause any issue with the overall meeting. Guests that dial in early often simply wait in the virtual meeting room, sometimes treated to hold music provided by the software. If the guest is unusually early, some software prompts users to check the time of their meeting after a period of no activity, or gives the date and time of the next meeting, in case it was changed without the guest's knowledge.
Conference meeting guests that dial in late similarly have nothing to worry about. They'll simply be dropped into the meeting with everyone else. If you are late though, your tardy presence will be felt, as most services will announce the name or provide a notification when someone enters a meeting.