During the Second World War he served with the Royal Scots in Hong Kong, with the rank of captain. When Hong Kong was taken by the Japanese, Ford and his brother Douglas were captured and held as prisoners-of-war. Douglas was executed, and James was not released until 1945. He received the Military Cross for his service.