A system for registering and tracking an anatomical structure of a patient, comprising: - a camera array comprising at least two cameras rigidly coupled to a surgical microscope and adapted to acquire a first stereoscopic image dataset and a second stereoscopic image data set comprising video images taken from different viewing angles and having a first image content showing the anatomical structure and a second image content showing the anatomical structure , respectively; - a first reference structure and a second reference structure different to the first reference structure that are trackable and configured to be rigidly coupled to the anatomical structure ; and - a computer storing and/or running a program, which, when running on the computer , causes the computer to perform a method for registering and tracking the anatomical structure of the patient, comprising the steps of: - acquiring, via the camera array, the first stereoscopic image dataset; - registering said first stereoscopic image dataset to a 3D image dataset showing the anatomical structure thereby utilizing the first reference structure shown in the first stereoscopic image dataset; - acquiring, via the camera array, the second stereoscopic image dataset providing the second image content showing the anatomical structure , which differs from the first image content ; - registering the anatomical structure shown in the second stereoscopic image dataset to the second reference structure shown in the second stereoscopic image dataset, and - track the anatomical structure based on the registration of the anatomical structure to the second reference structure . - a camera array comprising at least two cameras rigidly coupled to a surgical microscope and adapted to acquire a first stereoscopic image dataset and a second stereoscopic image data set comprising video images taken from different viewing angles and having a first image content showing the anatomical structure and a second image content showing the anatomical structure , respectively; - a first reference structure and a second reference structure different to the first reference structure that are trackable and configured to be rigidly coupled to the anatomical structure ; and - a computer storing and/or running a program, which, when running on the computer , causes the computer to perform a method for registering and tracking the anatomical structure of the patient, comprising the steps of: - acquiring, via the camera array, the first stereoscopic image dataset; - registering said first stereoscopic image dataset to a 3D image dataset showing the anatomical structure thereby utilizing the first reference structure shown in the first stereoscopic image dataset; - acquiring, via the camera array, the second stereoscopic image dataset providing the second image content showing the anatomical structure , which differs from the first image content ; - registering the anatomical structure shown in the second stereoscopic image dataset to the second reference structure shown in the second stereoscopic image dataset, and - track the anatomical structure based on the registration of the anatomical structure to the second reference structure . - acquiring, via the camera array, the first stereoscopic image dataset; - registering said first stereoscopic image dataset to a 3D image dataset showing the anatomical structure thereby utilizing the first reference structure shown in the first stereoscopic image dataset; - acquiring, via the camera array, the second stereoscopic image dataset providing the second image content showing the anatomical structure , which differs from the first image content ; - registering the anatomical structure shown in the second stereoscopic image dataset to the second reference structure shown in the second stereoscopic image dataset, and - track the anatomical structure based on the registration of the anatomical structure to the second reference structure . <EOS>