James served in the United States Army as an officer in the 101st Airborne Division. He resigned in protest after Black officers were stripped of their command when the 101st was sent to quell White rioting in response to the integration of the University of Mississippi by James Meredith in 1962.