Jim's son, James Jr., was killed in a sprint car crash at Winchester in October 1977. James Jr. had attempted to qualify for the 1977 Indy 500 earlier that year. Alongside his father, they were attempting to become the first father and son combination to qualify for the same race. However, James Jr. was too slow to make the field.