They were interned on Flinders Island where they succumbed to European diseases , poor food and unsanitary conditions and died in such numbers that alarmed authorities moved them to Oyster Cove , where they eventually died out in 1876 .