A robotic system comprising: a master control device having a master frame; an image capturing system (304, 112, 1220, 1304) having an image frame whose position and orientation relative to a fixed frame is determinable through one or more image frame transforms; a tool (14, 406a, 1230, 1314) disposed within a field of view of the image capturing system (304, 112, 1220, 1304) and having a tool frame whose position and orientation relative to the fixed frame is determinable through one or more tool frame transforms; and at least one processor programmed to perform operations including: identifying pose indicating points of the tool (14, 406a, 1230, 1314) from information in one or more images captured by the image capturing system(304, 112, 1220, 1304); determining an estimated transform from a first frame to a second frame for an unknown one of the image and tool frame transforms by using the identified pose indicating points and known ones of the image and tool frame transforms, the estimated transform being determined from a chain of frame relationships between the first frame and the second frame with each known image and tool frame transform relating its corresponding frames to one another and with the identified pose indicating points relating the tool frame to the image frame; updating a master-to-tool transform by using the estimated and known ones of the image and tool frame transforms; and commanding movement of the tool (14, 406a, 1230, 1314) in response to movement of the master control device by using the updated master-to-tool transform. a master control device having a master frame; an image capturing system (304, 112, 1220, 1304) having an image frame whose position and orientation relative to a fixed frame is determinable through one or more image frame transforms; a tool (14, 406a, 1230, 1314) disposed within a field of view of the image capturing system (304, 112, 1220, 1304) and having a tool frame whose position and orientation relative to the fixed frame is determinable through one or more tool frame transforms; and at least one processor programmed to perform operations including: identifying pose indicating points of the tool (14, 406a, 1230, 1314) from information in one or more images captured by the image capturing system(304, 112, 1220, 1304); determining an estimated transform from a first frame to a second frame for an unknown one of the image and tool frame transforms by using the identified pose indicating points and known ones of the image and tool frame transforms, the estimated transform being determined from a chain of frame relationships between the first frame and the second frame with each known image and tool frame transform relating its corresponding frames to one another and with the identified pose indicating points relating the tool frame to the image frame; updating a master-to-tool transform by using the estimated and known ones of the image and tool frame transforms; and commanding movement of the tool (14, 406a, 1230, 1314) in response to movement of the master control device by using the updated master-to-tool transform. identifying pose indicating points of the tool (14, 406a, 1230, 1314) from information in one or more images captured by the image capturing system(304, 112, 1220, 1304); determining an estimated transform from a first frame to a second frame for an unknown one of the image and tool frame transforms by using the identified pose indicating points and known ones of the image and tool frame transforms, the estimated transform being determined from a chain of frame relationships between the first frame and the second frame with each known image and tool frame transform relating its corresponding frames to one another and with the identified pose indicating points relating the tool frame to the image frame; updating a master-to-tool transform by using the estimated and known ones of the image and tool frame transforms; and commanding movement of the tool (14, 406a, 1230, 1314) in response to movement of the master control device by using the updated master-to-tool transform.