A radiotherapy system capable of monitoring a target position in real time, comprising: a radiation source; a real-time image capturing device comprising an ultrasound probe mounted with a robotic manipulator, wherein while radiation is emitted from the radiation source, the real-time image capturing device is adapted to capture an image of a targeted location in real time; a remote actuating system comprising the robotic manipulator and a visual servoing and force control system comprising two binocular color cameras (1241, 1242), wherein the robotic manipulator has multiple degrees of freedom and is equipped with a six-axis force sensor, the robotic manipulator being controlled by the visual servoing and force control system or other position servo control systems to actuate the real-time image capturing device; and an image registration system operable to register in real time an image obtained from the real-time image capturing device with a pre-stored image, wherein the image registration system is also operable to perform image segmentation, image slice interpolation and slice reconstruction, feature extraction, and image registration; a plurality of color (1122; 1132) balls of different colors and black rods (1121; 1131) which are respectively disposed on the ultrasound probe and the handheld probe, whereby the camera is adapted to capture an image of the color balls and said black rods disposed on the ultrasound probe and the camera is adapted to capture an image of said color balls and black rods provided on the handheld probe, whereby the visual servoing and force control system is operable to actuate the robotic manipulator to track and control the position, orientation and contact force of the ultrasound probe based on said color balls (1122, 1132) and black rods (1121, 1131), so that the ultrasound probe can effect the same position and orientation as the handheld probe and can be kept in position with a desirable contact force on the targeted location of patient's body. a radiation source; a real-time image capturing device comprising an ultrasound probe mounted with a robotic manipulator, wherein while radiation is emitted from the radiation source, the real-time image capturing device is adapted to capture an image of a targeted location in real time; a remote actuating system comprising the robotic manipulator and a visual servoing and force control system comprising two binocular color cameras (1241, 1242), wherein the robotic manipulator has multiple degrees of freedom and is equipped with a six-axis force sensor, the robotic manipulator being controlled by the visual servoing and force control system or other position servo control systems to actuate the real-time image capturing device; and an image registration system operable to register in real time an image obtained from the real-time image capturing device with a pre-stored image, wherein the image registration system is also operable to perform image segmentation, image slice interpolation and slice reconstruction, feature extraction, and image registration; a plurality of color (1122; 1132) balls of different colors and black rods (1121; 1131) which are respectively disposed on the ultrasound probe and the handheld probe, whereby the camera is adapted to capture an image of the color balls and said black rods disposed on the ultrasound probe and the camera is adapted to capture an image of said color balls and black rods provided on the handheld probe, whereby the visual servoing and force control system is operable to actuate the robotic manipulator to track and control the position, orientation and contact force of the ultrasound probe based on said color balls (1122, 1132) and black rods (1121, 1131), so that the ultrasound probe can effect the same position and orientation as the handheld probe and can be kept in position with a desirable contact force on the targeted location of patient's body.