First inhabited by Arawak and later by Carib Indians , the Virgin Islands were settled by the Dutch in 1648 and then annexed by the English in 1672 .