A system for registering and tracking an anatomical structure of a patient, comprising: - a camera array (4, 6A, 6B) comprising at least two cameras (4, 6A, 6B) rigidly coupled to a surgical microscope and adapted to acquire a first stereoscopic image dataset and a second stereoscopic image data set comprising video images taken from different viewing angles and having a first image content showing the anatomical structure and a second image content showing the anatomical structure, respectively; - a first reference structure and a second reference structure different to the first reference structure that are trackable and configured to be rigidly coupled to the anatomical structure; and - a computer storing and/or running a program, which, when running on the computer, causes the computer to perform a method for registering and tracking the anatomical structure of the patient, comprising the steps of: - acquiring, via the camera array, the first stereoscopic image dataset; - registering said first stereoscopic image dataset to a 3D image dataset showing the anatomical structure thereby utilizing the first reference structure shown in the first stereoscopic image dataset; - acquiring, via the camera array, the second stereoscopic image dataset providing the second image content showing the anatomical structure, which differs from the first image content; - registering the anatomical structure shown in the second stereoscopic image dataset to the second reference structure shown in the second stereoscopic image dataset, and - track the anatomical structure based on the registration of the anatomical structure to the second reference structure. - a camera array (4, 6A, 6B) comprising at least two cameras (4, 6A, 6B) rigidly coupled to a surgical microscope and adapted to acquire a first stereoscopic image dataset and a second stereoscopic image data set comprising video images taken from different viewing angles and having a first image content showing the anatomical structure and a second image content showing the anatomical structure, respectively; - a first reference structure and a second reference structure different to the first reference structure that are trackable and configured to be rigidly coupled to the anatomical structure; and - a computer storing and/or running a program, which, when running on the computer, causes the computer to perform a method for registering and tracking the anatomical structure of the patient, comprising the steps of: - acquiring, via the camera array, the first stereoscopic image dataset; - registering said first stereoscopic image dataset to a 3D image dataset showing the anatomical structure thereby utilizing the first reference structure shown in the first stereoscopic image dataset; - acquiring, via the camera array, the second stereoscopic image dataset providing the second image content showing the anatomical structure, which differs from the first image content; - registering the anatomical structure shown in the second stereoscopic image dataset to the second reference structure shown in the second stereoscopic image dataset, and - track the anatomical structure based on the registration of the anatomical structure to the second reference structure. - acquiring, via the camera array, the first stereoscopic image dataset; - registering said first stereoscopic image dataset to a 3D image dataset showing the anatomical structure thereby utilizing the first reference structure shown in the first stereoscopic image dataset; - acquiring, via the camera array, the second stereoscopic image dataset providing the second image content showing the anatomical structure, which differs from the first image content; - registering the anatomical structure shown in the second stereoscopic image dataset to the second reference structure shown in the second stereoscopic image dataset, and - track the anatomical structure based on the registration of the anatomical structure to the second reference structure.