A surgical system comprising: an ophthalmic microscope configured to provide an image of a posterior segment of an eye; a control unit; a vitreoretinal surgical tool configured to be placed at a surgical site in the posterior segment of the eye; a tracking unit configured to track motion of the vitreoretinal surgical tool, while the vitreoretinal surgical tool is located in the posterior segment of the eye, the tracking unit comprising an imaging unit configured to capture more than one image of the vitreoretinal surgical tool and the posterior segment of the eye; a processing unit for processing the tracked motion of the vitreoretinal surgical tool while the vitreoretinal surgical tool is employed in a surgery in the posterior segment of the eye, wherein the processing unit is configured to receive the captured images and to process and enhance the captured images to extract the motion of the surgical tool so as to obtain temporal spatial information of the tracked motion of a distal end of the vitreoretinal surgical tool; wherein the control unit further comprises a control input unit comprising a number of control commands, wherein each of the control commands is associated with a motion pattern; wherein the control unit is configured to identify a control action by associating the control input unit and the temporal spatial information of the vitreoretinal surgical tool to identify whether the temporal spatial information is associated with a motion pattern of a control command of the control input unit and to apply a corresponding control command to the surgical system; wherein the surgical system further comprises a heads-up display configured in the ophthalmic microscope, the heads-up display configured to inform a user of a system status or to enable the user to confirm the corresponding control command. an ophthalmic microscope configured to provide an image of a posterior segment of an eye; a control unit; a vitreoretinal surgical tool configured to be placed at a surgical site in the posterior segment of the eye; a tracking unit configured to track motion of the vitreoretinal surgical tool, while the vitreoretinal surgical tool is located in the posterior segment of the eye, the tracking unit comprising an imaging unit configured to capture more than one image of the vitreoretinal surgical tool and the posterior segment of the eye; a processing unit for processing the tracked motion of the vitreoretinal surgical tool while the vitreoretinal surgical tool is employed in a surgery in the posterior segment of the eye, wherein the processing unit is configured to receive the captured images and to process and enhance the captured images to extract the motion of the surgical tool so as to obtain temporal spatial information of the tracked motion of a distal end of the vitreoretinal surgical tool; wherein the control unit further comprises a control input unit comprising a number of control commands, wherein each of the control commands is associated with a motion pattern; wherein the control unit is configured to identify a control action by associating the control input unit and the temporal spatial information of the vitreoretinal surgical tool to identify whether the temporal spatial information is associated with a motion pattern of a control command of the control input unit and to apply a corresponding control command to the surgical system; wherein the surgical system further comprises a heads-up display configured in the ophthalmic microscope, the heads-up display configured to inform a user of a system status or to enable the user to confirm the corresponding control command.