Hugging the east bank of the Hudson , the train passes several historic sites , including Bannerman 's Castle , the U.S. Military Academy at West Point ( across the river ) and Frederick Vanderbilt 's 54-room summer home .