The Bay of Pigs invasion was a CIA-backed attempt by Cuban exiles to overthrow Fidel Castro .