## 1 Introduction  
Optical tweezers (OT) are well-suited for contact free manipulation of microscale objects of various shapes, dimensions and material compositions [1]. It uses a high numerical aperture microscope objective to tightly focus a laser beam that exerts the required amount of force to grasp (trap), move and orient microscale objects in a fluid medium. A large number of microscale objects are manipulated simultaneously by using a dynamic hologram to split the beam into multiple beams with complete individual control over the focal positions of each of the beams in three dimensions [2]. Combining these characteristics with low trapping force magnitudes (tens of picoNewtons) and the dependence of the forces on the relative refractive index of the manipulated objects with respect to the surrounding medium, the OT system is used often in many biomanipulation applications such as cell sorting, drug delivery and mechanistic study of fundamental physiological processes [3]. By automating the process of controlling the laser beams, the OT functions as an optically actuated microrobot with multiple reconfigurable end effectors (see Figure 1), or a collection of independently steerable microrobots [4]. We are particularly interested in the indirect manipulation of a large number of cells to study their collective migration behaviors in the presence of external stimuli or form precise patterns such as vascular microstructures that can be embedded within maturing engineered tissue constructs. These cells are not trapped directly by the laser beams to avoid photodamage caused by laser exposure induced local heating or photopolymerization. Instead, they are manipulated by inert end effectors, which consist of optically-trapped microspheres (beads) that are completely unaffected by the laser [5]. The trapped beads are able to easily grasp as well as release the cells due to the absence of chemical bonding or functional adhesion between the beads and the cells. Such automated micromanipulation requires accurate, real-time estimation of the positions and orientations (states) of the workspace objects. These estimated states are combined with a dynamical model of the objects, and fed to a path planner to compute collision-free object trajectories. Trajectory information is then provided to a motion controller to execute stable actions (set and change the focal positions of the laser beams) that closely follow the trajectories. Considering that the objects are sensed using a charge-coupled device (CCD) camera employing standard bright field imaging, being able to process the low contrast images to perceive what type of object is located where is important. Since some of the objects change shape dynamically, it is necessary for any perception method to track object boundaries in addition to detecting their centroid locations and motion directions. It is also important for the perception method to work in a variety of fluid media offering different background image contrasts where the objects will be manipulated. A plethora of work already exists in the context of automated optical micromanipulation, particularly for dynamics modeling, path planning and motion control. Representative examples of dynamics modeling include the use of Langevin equation to simulate the behavior of an optically-trapped microsphere [6] and a stochastic perturbation model to analyze the trapping stability of microparticles during motion [7]. For path or trajectory planning, representative examples include a partially observable Markov decision process algorithm for single object [8] and multiple object [9] transport, a rapidly-exploring random tree method for cell transport [10], and an A* algorithm for indirect manipulation of cells using optically-trapped microspheres [11].