Test retries should be used to identify non-deterministic tests rather than masking the issues they present. They help in confirming the flakiness of a test and should prompt further investigation to resolve underlying problems.