ARPANET was built using a decentralized approach with Interface Message Processors (IMPs) connecting various host computers. These IMPs served as the first routers and were responsible for transmitting data packets between nodes.