It's likely that man got magic from the elves, though not neccessarily with the elves' consent. Age of Giants Several Elf cities are founded.
Some powerful magic is forged for their defense.
Relations with the dwarves are good, and some cities carry on much trade. Then came the Elfwar, when the evil elves battled the good and were driven underground. In the Giant-Dragon Wars, when the humanoids became numerous, came the Goblin Wars. The elves forged many Elven swords in this time. Several of the great cities of the Golden Age fell to the goblins, due to so many of the elves having left in the Fractioning.