The Nicobar Islands make up a small chain that runs across the Andaman Sea near southern Thailand .