Line noise can have many sources.
Some devices can put a small drain on the phone line current and interfere with your modem transmission. If your modem is external, do the same for the cable connecting the modem to your computer. Wiggle the phone cable while you're dialed up and see if the garbage characters appear. If they do, you might need a new cable. These cables are fairly inexpensive, and you can find them at most discount department or electronics stores.