# Collaborative magnetic agents for 3D microrobotic grasping

Franco N Piñan Basualdo, Sarthak Misra

<figure><img src="image_1.png" /></figure>

## To cite this version:

Franco N Piñan Basualdo, Sarthak Misra. Collaborative magnetic agents for 3D microrobotic grasping. Advanced Intelligent Systems, 2023, 5 (12), 10.1002/aisy.202300365. hal-04753501

HAL Id: hal-04753501

https://hal.science/hal-04753501v1

Submitted on 25 Oct 2024

HAL is a multi-disciplinary open access archive for the deposit and dissemination of scientific research documents, whether they are published or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

<figure><img src="image_2.png" /></figure>
# Collaborative magnetic agents for 3D microrobotic grasping

Franco N. Piñan Basualdo* Sarthak Misra

Dr. F. N. Piñan Basualdo and Prof. S. Misra

Surgical Robotics Laboratory, Department of Biomechanical Engineering
University of Twente

7522NB, Enschede, The Netherlands

Email Address: f.n.pinanbasualdo@utwente.nl

Prof. S. Misra

Surgical Robotics Laboratory, Department of Biomedical Engineering,
University of Groningen and University Medical Centre Groningen

9713 GZ Groningen, The Netherlands.

Keywords: Microrobotics; Magnetic actuation; Collaborative robotics; Robotic grasping

Untethered microrobotic grasping has enabled the precise manipulation of small components in difficult-to-reach environments. However, the fabrication of deformable grasping robots at small scales requires advanced materials and fabrication technologies. Alternatively, multiple microrobots can be used to surround and grasp objects. By collaborating, microrobots can perform complex tasks that they would not be capable of executing individually. The complexity of controlling multiple untethered microrobots has limited the applications of collaborative microrobots to two-dimensional operations. To fill this gap, this work proposes a collaborative microrobotic system for three-dimensional grasping operations. The system is based on the control of two magnetic agents and the exploitation of the magnetic interactions between them to grasp, manipulate, and assemble passive components. We have developed a custom-made closed-loop controller to automatically stabilize the system, achieving pose control of a grasped passive component (of 2 mm in size) with a precision of 300 µm in position and 10° in orientation. The main advantage of the proposed system is its capability to reconfigure the magnetic agents for other operations. The collaborative assembly of a microgear and subsequent reconfiguration to power the assembled mechanism are demonstrated.

# 1 Introduction

By incorporating robotic systems, modern industries have been able to increase throughput and precision, leading to higher efficiency and lower costs [1]. In particular, robotic grasping [2, 3] has enabled the automation of pick-and-place and assembly operations. These operations are typically performed by robotic arms equipped with application-dependent end effectors, such as grippers, or magnetic clamps. However, with the miniaturization of technological components [4], there has been a growing demand for technological solutions that can perform similar tasks at smaller scales. The development of micromanipulation solutions has become an active area of research, with many potential applications in fields such as medicine and biology [5, 6]. The proposed micromanipulation solutions can be classified into two alternative approaches: Tethered micromanipulators and untethered microrobots.

Tethered micromanipulators have become increasingly important in modern manufacturing and scientific research. These devices are typically composed of a macroscopic arm equipped with a miniature end effector, such as a gripper or a needle. The end effectors are designed to perform precise movements, which can be actuated by different means, including piezoelectric [7], magnetic [8], optical [9], or pneumatic [10] actuators. These manipulators are ideal for performing fast and accurate pick-and-place operations [11] or for delicate assembly tasks [12]. Nevertheless, micromanipulators are unable to work effectively in confined environments. This is primarily due to the large size of the manipulator compared to the end effector, which can limit its ability to access small areas.

Microrobots have emerged as a promising technology to perform operations in difficult-to-reach or enclosed locations [13]. Unlike tethered micromanipulators, microrobots are small mobile robotic agents that can move freely while remotely actuated. Acoustic [14, 15, 16] and optical [17, 18, 19] effects have been proposed as microrobotic actuation alternatives. However, the most common actuation methodology is magnetic actuation [20, 21, 22] since it allows for fast and precise control, is biocompatible, and has a higher penetration depth than the other methods [23]. Moreover, magnetically actuated microrobots can be designed to perform various robotic operations, such as grasping and transporting passive objects [24].
One option to remotely manipulate passive objects is the use of robotic microgrippers. Robotic microgrippers can be passive and rely on their geometry to displace objects [17]. However, the ability to externally actuate a microgripper [25, 26] enables a more robust grasping of micro-objects. One of the most promising types of externally actuated microgrippers is based on flexible magnetic materials [24], which can be designed for a desired response to an external magnetic field. Microgrippers based on flexible magnetic materials have shown promising results in pick-and-place [27] and assembly [28, 29] applications. Nevertheless, the main disadvantages of robotic microgrippers are their complex fabrication (which normally involves special materials and technological processes), and their limited capability to execute operations beyond grasping.

On the other hand, collaborative grasping using multiple microrobots has proven to be a promising alternative to robotic microgrippers [30]. Collaborative grasping relies on the control of multiple agents to surround and grasp passive objects. For instance, optically actuated bubbles [31] and microbeads [18] have been used for in-plane grasping of passive particles. Alternatively, magnetic collectives have been employed for 2D actuation [32] and grasping [33, 34, 35], taking advantage of the intrinsic inter-agent interactions. The advantages of collaborative grasping include the relative simplicity of each agent (compared to microgrippers) and the reconfigurability of the group of particles. The reconfigurability of the group enables the performance of other operations besides grasping, like changing the formation to navigate through confined environments [35]. Nevertheless, due to the complexity of controlling multiple agents simultaneously [36, 37], the solutions proposed in the literature have been so far limited to 2D operations. To fill this gap, a novel 3D collaborative grasping system based on magnetic actuation is proposed in this work.

The proposed 3D collaborative grasping consists of the use of an electromagnetic actuation system, to control two spherical magnetic agents for grasping operations. As demonstrated in a prior study [38], the system can independently control two magnetic agents as long as they are sufficiently apart (normally, more than 5 diameters). On the other hand, when the agents are close to each other, the magnetic interaction between them becomes significant and the independent control becomes more difficult. In this work, however, we exploit this interaction for the grasping of passive objects. Indeed, when two magnetic agents are placed on opposite sides of a passive component, an external field can be used to generate attraction between them and grasp the component. Then, the magnetic field and magnetic gradients can be used to rotate and displace the ensemble. This system offers a fabrication-free alternative for untethered magnetic manipulation of diverse passive components in 3D space, opening new possibilities for micromanipulation and micro-assembly applications. Furthermore, the pair of magnetic agents can be easily reconfigured for other purposes, such as powering micromachines, making this system a promising solution for a wide range of microscale applications.

## 2 Collaborative magnetic control

The actuation system proposed in this work comprises nine iron-core coils arranged in the configuration depicted in Figure 1. This system is capable of generating spatially selective magnetic actuation, enabling the simultaneous control of two magnetic agents. Additionally, the system can control the interactions between neighboring agents by simultaneously controlling the magnetic field and gradient. This capability enables the exploitation of magnetic gradient and dipole-dipole forces for the collaborative grasping and displacement of passive objects.

### 2.1 Magnetic gradient and dipole-dipole forces

The proposed collaborative grasping technique is based on the forces experienced by magnetic agents in a magnetic field. Modeling a magnetic agent as a point dipole, the gradient force ($F_{grad}$) experienced by the agent in a magnetic field can be expressed as [39]:

$$F_{grad} = \nabla (m \cdot B), \qquad (1)$$
<figure><img src="image_3.png" /><figcaption>Figure 1: Electromagnetic actuation system and magnetic forces. A) The geometric arrangement of electromagnetic coils in the proposed actuation system. B) Example pick and place operation where two soft magnetic agents are used to grasp a passive cube. C) Free-body diagram showing the magnetic gradient (F<sub>grad</sub>) and dipole-dipole (F<sub>dip-dip</sub>) forces experienced by soft magnetic agents in an inhomogeneous magnetic field (B). The gradient force pulls magnetic agents towards stronger magnetic fields, while the dipole-dipole interaction generates attraction between them. See video S1 for example.</figcaption></figure>

where **m** is the magnetic dipole moment and **B** is the magnetic field. For a soft magnetic agent in the linear regime, we can assume **m** = K<sub>mag</sub> **B**, where K<sub>mag</sub> is a constant depending on the agent size and material. Thus, we can rewrite Equation (1) as:

$$ \mathbf{F}_{\text{grad}} = K_{\text{mag}} \nabla |\mathbf{B}|^2 = 2K_{\text{mag}} |\mathbf{B}| \nabla |\mathbf{B}|. \quad (2) $$

For paramagnetic agents (K<sub>mag</sub> > 0), the force is directed towards regions of stronger magnetic field. Some examples of the magnetic fields and gradients that the employed electromagnetic system can generate are shown in Figure 2A.

Apart from the external field, magnetic agents also interact with each other. The interaction force between two equal soft magnetic agents in a homogeneous magnetic field can be modeled as a dipole-dipole interaction. The interaction force (F<sub>dip-dip</sub>), schematized in Figure 2B, can be expressed as [39]:

$$ \begin{aligned} \mathbf{F}_{\text{dip-dip}} &= \frac{3\mu_0 K_{\text{mag}}^2 |\mathbf{B}|^2}{4\pi |\mathbf{r}|^4} \left[ 2\hat{\mathbf{B}} (\hat{\mathbf{B}} \cdot \hat{\mathbf{r}}) + \hat{\mathbf{r}} \left(1 - 5 (\hat{\mathbf{B}} \cdot \hat{\mathbf{r}})^2\right) \right] \\ &= \frac{3\mu_0 K_{\text{mag}}^2 |\mathbf{B}|^2}{4\pi |\mathbf{r}|^4} \left[ \hat{\mathbf{r}} \left( \frac{-3 \cos(2\theta) - 1}{2} \right) - \hat{\mathbf{n}} \sin(2\theta) \right], \end{aligned} \quad (3) $$

where μ<sub>0</sub> is the vacuum magnetic permeability, **r** is the vector between both agents (and **r** the unit vector), θ is the angle between **r** and **B**, and **n** is the unit vector normal to **r** (in the **B**-**r** plane). For |θ| < 54.7°, the radial force is attractive (F<sub>r</sub> < 0), which can be exploited as a grasping force. The normal force (F<sub>n</sub>), on the other hand, tends to align the pair of agents with the external field, which can be exploited for orientation control. Moreover, dipole-dipole interactions become dominant over spatially selective gradient forces for short distances (< 2.5 diameters) between the magnetic agents, as shown in Figure 2C.

The previously described magnetic gradient and dipole-dipole forces can be used to displace the magnetic agents and the passive object as a whole. When the passive object remains grasped, the motion of the ensemble can be modeled as a rigid body's translation and rotation. The translation dynamics are governed only by gradient forces since dipole-dipole forces cancel out. For a low Reynolds number (in our
<figure><img src="image_4.png" /><figcaption>Figure 2: Magnetic gradient and dipole-dipole forces. A) Examples of possible magnetic fields (B) and gradients ($\nabla|B|^2$) that can be generated with the used electromagnetic system. B) Magnetic interaction between magnetic dipoles (m) induced on soft-magnetic agents in a homogeneous field (B). Radial (F<sub>r</sub>) and normal (F<sub>n</sub>) force for a pair of 1 mm agents, at 2 mm from each other in a 50 mT field. C) Comparison between the use of dipole-dipole interactions and spatially selective gradient forces for grasping and rotating. The red arrows represent the dipole-dipole (left) or gradient (right) forces experienced by the agents. Dipole-dipole forces become dominant at small distances between the magnetic agents.</figcaption></figure>
<figure><img src="image_5.png" /><figcaption>Figure 3: Schematic of the developed control system. The tracking subsystem is in charge of getting the ensemble position **x** from the cameras' frames. The position controller is in charge of finding the necessary magnetic gradient $\nabla|\mathbf{B}|_d^2$ to drive the ensemble to the target position **T**. The grasping and orientation controller is in charge of finding the required magnetic field $\mathbf{B}_d$ to maintain grasping and to rotate the ensemble to the target orientation $\hat{\mathbf{R}}$. The inverse mapping is in charge of finding the coils' currents $\mathbf{I}$ that would generate said gradient and field.</figcaption></figure>

experimental conditions, $Re \approx 10^{-3}$), the translation dynamics of the ensemble can be written as:

$$m \ddot{\mathbf{x}} = 2 \mathbf{F}_{\text{grad}} + \mathbf{F}_{\text{grav}} + \mathbf{F}_{\text{drag}} = 2 K_{\text{mag}} \nabla |\mathbf{B}|^2 + \mathbf{W} - C_d \dot{\mathbf{x}}, \quad (4)$$

where $m$ is the total mass, $\mathbf{W}$ is the effective weight (weight minus buoyancy force), and $C_d$ is the total drag coefficient acting on the ensemble.

In addition to displacement, the pair of magnetic agents can also rotate the passive object. The torque resulting from magnetic gradient forces is negligible in the grasped state since during displacement each agent experiences a similar gradient force. Therefore, the rotation dynamics can be modeled by considering the normal component of the dipole-dipole interaction in Equation (3). The applied torque is always perpendicular to **r** (the vector between both magnetic agents), which limits the orientation control to two degrees of freedom. The (one-dimensional) rotation dynamics of the ensemble can be written as:

$$I_{\text{rot}} \ddot{\theta} = - \frac{3\mu_0 K_{\text{mag}}^2 |\mathbf{B}|^2}{4\pi |\mathbf{r}|^3} \sin(2\theta) - C_{\text{rot}} \dot{\theta}, \quad (5)$$

where $I_{\text{rot}}$ is the moment of inertia of the ensemble, $C_{\text{rot}}$ is the rotational drag coefficient, and $\theta$ is the angle between $\mathbf{B}$ and $\mathbf{r}$ (shown in Figure 2B). The result of Equation (5) is to align the magnetic agents with the magnetic field and stabilizing at $\theta = 0^\circ$ or $\theta = 180^\circ$. In conclusion, the magnetic field and its gradient can be used to control 6 degrees of freedom (3 for position, 2 orientations, and the grasping force).

## 2.2 Control system

To exploit the previously introduced phenomena for collaborative grasping, the simultaneous control of both magnetic agents is necessary. Two distinct approaches have been suggested for the independent control of identical agents. The first approach relies on spatially selective control and is effective only if the agents are sufficiently distant from one another [40]. The second method relies on inter-agent interactions and can be used for formation control when the agents are close to each other [41]. In our case, a typical grasping operation includes an initial approach, where the agents are apart from each other and must be independently controlled, and a grasping state, where the agents are close to each other and their formation
must be controlled. In this work, however, we only discuss the developed controller for when the magnetic agents are grasping the passive object.

The developed controller comprises four subsystems: tracking, position controller, orientation and grasping controller, and inverse field mapping. Figure 3 displays a schematic of the controller and its subsystems. The magnetic agents are tracked through stereo-visual feedback. The closed-loop position controller computes the magnetic gradient necessary to move the ensemble (modeled by Equation (4)) to a target position (T) as:

$$ \nabla|\mathbf{B}|_d^2 = K_p \mathbf{e} + K_I \int_0^t \mathbf{e} dt - \frac{\mathbf{W}_{\text{est}}}{K_{\text{mag}}}, \qquad (6) $$

where $\mathbf{e} = \mathbf{T} - \mathbf{x}$ is the error, $\mathbf{W}_{\text{est}}$ is the estimated weight, and $K_p$ and $K_I$ are the proportional and integral gains, respectively. The open-loop orientation and grasping controller computes the required magnetic field to keep a grasping force and rotate the ensemble (modeled in Equation (5)) to a target orientation $\hat{\mathbf{R}}$ as:

$$ \mathbf{B}_d = B_{\text{grasp}} \hat{\mathbf{R}}, \qquad (7) $$

where $B_{\text{grasp}}$ is the required magnetic field magnitude to maintain a sufficient grasping force. Finally, the inverse mapping subsystem computes the necessary currents to generate the magnetic field and gradient defined by the previous subsystems. Although there is no analytic solution for the currents, modern computational algorithms enable the numerical solution of non-linear systems in real-time [38, 42]. For more details on the inverse field mapping, see Section 5.

# 3 Experimental validation

The proposed methodology and control strategies are validated experimentally by performing grasping, manipulation, and assembly operations with varied passive objects. The operations are performed in a nine-coil magnetic actuation system [38] capable of spatially selective magnetic actuation. The developed field mapping inversion is used to find the limits of the system in two cases, independent gradient control in two points, and field and gradient control in one point. The results, shown in Figure S1, exhibit a resolution of 10 T$^2$/m$^2$ in imposing different gradients at two different points. If only the minimum magnetic field magnitude is imposed, the limits of the map inversion in field and gradient are 70 mT and 0.1 T$^2$/m, respectively. The utilized soft magnetic agents are two 1 mm steel spheres, with a measured $K_{\text{mag}} = 1.4$ mN m/T$^2$ (Figure S2). All the experiments are performed in silicone oil M1000 to dampen the motion. Due to this slippery media, the manipulated passive objects have grooves to accommodate the agents and enable grasping. In our experimental conditions (summarized in Table S1), the estimated maximum pulling force the two magnetic agents can exert is 500 µN, and the maximum grasping force (Equation (3)) is 150 µN (for a distance between agents of 2.5 mm).

## 3.1 Controlled manipulation and assembly of non-magnetic object

First, we validate experimentally the pose (position and orientation) control of grasped passive objects proposed in Section 2.2. The passive object is a 2 mm cube with grooves to accommodate the magnetic agents. The desired magnetic field magnitude is fixed at $B_{\text{grasp}} = 30$ mT. The experimental pose control results are shown in Figure 4. The passive object is displaced (peak velocity of 2.3 mm/s) and rotated (peak velocity of 35 °/s) without losing grasp. The closed-loop position control achieved a steady-state precision of approximately 300 µm, with a rising time and settling time of approximately 10 s and 20 s, respectively. The position control also exhibited some overshoot (< 1 mm), which could be reduced by adding a derivative action. On the other hand, the open-loop orientation control exhibited different steady-state errors, but always less than 10°. In this case, the settling time is approximately 2 s, making the orientation response much quicker than the translation one. Both the position and orientation responses are in good agreement (Pearson correlation coefficient > 0.8 for the position and 0.98 for the orientation) with the dynamic models in Equation (4) and Equation (5), as shown in Figure S4.
<figure><img src="image_6.png" /><figcaption>Figure 4: Experimental collaborative grasping and manipulation results. The magnetic agents are 1 mm stainless steel spheres and the passive object is a 2 mm 3D printed cube. A) Closed-loop 3D position control with constant orientation. B) Open-loop orientation control with a constant target position. C) Snapshots of the pose control experiment. The cube has been highlighted in the top view to increase clarity. See Video S2 for the recording.</figcaption></figure>

Next, the potential of the proposed grasping and manipulation strategy to assemble passive objects is demonstrated. A typical assembly operation involves four steps. First, the magnetic agents approach a passive object from opposite sides using the independent control algorithm. Second, a magnetic field is imposed to generate attraction between the agents and grasp the passive object. Third, the previously described control strategy is used to automatically drive the passive object to its desired position. Finally, the object is released by first turning off the magnetic field, and then switching back to the independent control algorithm. This procedure can be repeated to assemble multiple objects sequentially. We validated this procedure through two experiments, as shown in Figure 5. First, the stacking of three passive cubes on top of each other shows a sequential assembly. Second, placing a beam on top of two pillars shows the possibility of controlling the pose of the passive object. As explained Section 2, one of the orientations of the passive object always remains uncontrolled. However, by exploiting the mechanical interaction between the beam and one of the pillars, the uncontrolled orientation is constrained to follow the position of the magnetic agents.
<figure><img src="image_7.png" /><figcaption>Figure 5: Experimental collaborative grasping and assembly results. The magnetic agents are 1 mm stainless steel spheres and the passive objects are 2 mm 3D printed cubes. A) The procedure consisted of four steps, approach, grasping, translation, and release. The solid red arrows represent the motion of the magnetic agents and the dashed green arrow represents the motion of the ensemble. B) Snapshots of the grasping and stacking of three cubes experiment. See Video S3 for the recording. C) Snapshots of the grasping and stacking of a beam on top of two cubes experiment. The passive objects (cubes and beam) have been highlighted in the top view to increase clarity. See Video S4 for the recording.</figcaption></figure>
<figure><img src="image_8.png" /><figcaption>Figure 6: Experimental collaborative assembly and actuation of a micromachine. The magnetic agents are 1 mm stainless steel spheres and the passive objects are plexiglass gears. A) The procedure consisted of four steps: loading the magnetic agents, assembly, reconfiguration, and actuation. The solid red arrows represent the motion of the magnetic agents and the dashed green arrow represents the motion of the ensemble. B) Snapshots of the micromachine assembly and actuation experiment. The gears have been highlighted in the top view to increase clarity. See Video S5 for the recording.</figcaption></figure>

### 3.2 Assembly and actuation of a micromechanism

One of the main advantages of collaborative grasping compared to grippers is the ability to reconfigure the agents for different tasks. We demonstrated experimentally this advantage by assembling and powering a two-gear mechanism, as shown in Figure 6. For this, the same 1 mm soft magnetic agents and two Plexiglas gears are used. One gear is designed to be levitated by making it lighter and by including holes to accommodate the magnetic agents. The other gear is designed for rotational actuation and has a single groove to accommodate both magnetic agents as a doublet (contact between the agents). As explained in Supplementary Material Section S2 and shown in Figure S3, when the agents are in contact with each other, their magnetization increases significantly, which gives place to higher magnetic torques. In this case, the assembly and actuation procedure consisted of four steps. First, the magnetic agents are loaded into their respective holes in the first gear. To prevent the agents from interacting with each other, they are levitated at different heights above the holes and then the field is turned off. The agents fall under the effect of gravity without interacting with each other since their magnetization disappears without the external field. Second, the previously described manipulation strategy is used to lift the gear and mount it into its shaft. Third, the magnetic agents are removed from the first gear, let collapse into each other (forming a doublet), and loaded into the second gear. Finally, a rotating field is used to actuate the two-gear mechanism. Notice that although in theory a doublet could be separated into two independent agents by imposing a magnetic field perpendicular to them (leading to a repulsive interaction), experimentally this can be challenging to achieve. As shown in Figure S3, when the agents are in contact, the interaction force between them is attractive, even for field angles larger than 54.7°. Therefore, if the field direction is not perfectly perpendicular to the doublet, the latter will turn and align with the field instead of separating.
<figure><img src="image_9.png" /><figcaption>Figure 7: Experimental stirring and mixing of dissolved blocks. The magnetic agents are 1 mm stainless steel spheres and the blocks are temperature-controlled soluble gelatine blocks. A) The experiment consisted of three steps: loading different color blocks in a chamber, increasing the temperature to dissolve them, and stirring the resulting aqueous solution. B) Snapshots of the stirring experiment. The gelatine blocks have been highlighted in the first frame of the top view to increase clarity. See Video S6 for the recording.</figcaption></figure>

## 3.3 Stirring and mixing of viscous solution

Besides grasping and assembling rigid passive components, collaborative magnetic microagents can actuate soft or liquid components. We use the magnetic agents to move soft gelatine blocks into a chamber where they dissolve, and to stir the resulting solution, as shown in Figure 7. For this, we use soluble gelatine blocks of different colors that dissolve at temperatures above 37°C [43]. The blocks are first loaded into a reaction chamber by pushing them with the magnetic agents. Then, the workspace is heated externally with a heat gun to dissolve the blocks into a viscous aqueous solution. Finally, the magnetic agents are used to stir the resulting liquid until obtaining a homogeneous solution (which takes approximately 2 minutes). Two stirring techniques were investigated: rotating the doublet by imposing a horizontal rotating magnetic field (30 mT field rotating at 45°/s) and displacing the doublet by imposing a rotating magnetic gradient (0.1 T²/m field rotating at 30°/s). Qualitatively, rotating the magnetic field helped to mix the liquid locally, and rotating the gradient helped to stir the entire solution.

# 4 Discussion

In this work, we propose a novel microrobotic 3D collaborative grasping method based on the simultaneous control of two magnetic agents. The system exploits magnetic gradient forces and magnetic dipole-dipole interactions between the magnetic agents to grasp, manipulate, and assemble passive components. For this, we used an electromagnetic system to generate and shape the magnetic field and magnetic gradient in the workspace. The non-linear dynamics of the system require the implementation of a closed-loop controller consisting of visual tracking, force and orientation control, and numerical inverse field mapping. This controller achieved pose control of a grasped passive component (of 2 mm in size) with a precision of
300 µm in positions and 10° in orientation.

The capability to reconfigure the magnetic agents is exploited for the assembly of 3D structures and micromachines. Indeed, collaborative magnetic agents possess the ability to release a held object and subsequently grasp another one, which enables the assembly of complex structures sequentially. Moreover, the magnetic agents can be reconfigured and put in contact with each other to create a doublet. Such a doublet can exert four times higher force and torque than a single agent, which has been exploited for the actuation of microgears and the stirring of viscous solutions. These capabilities could lead to new methods for cargo delivery, assembly, and actuation in microfluidic environments.

The proposed system offers several advantages over other micromanipulation and micro-assembly methods. Unlike tethered micromanipulators, our untethered system can function in difficult-to-reach or completely enclosed environments. On the other hand, unlike mobile microgrippers, our collaborative grasping system does not rely on advanced materials or fabrication techniques. Additionally, the use of multiple agents increases the versatility of the system, allowing for the manipulation of diverse passive components and the reconfiguration of the agents for other purposes.

In this work the agent size is chosen to be 1 mm to have comparable magnetic gradient and dipole-dipole interaction strengths in our experimental conditions (see Supplementary Material Section S4 and Figure S6). Nevertheless, the proposed collaborative grasping technique could be applied at smaller scales by miniaturizing the magnetic actuation system. Moreover, at smaller scales viscous drag becomes dominant, which would enable the performance of grasping operations in water or biofluids [44]. The potential miniaturization of the system and the biocompatibility of magnetic actuation [23] opens the door to the use of collaborative grasping for biomedical applications. For example, collaborative grasping could be employed for the manipulation of cell spheroids, which have a size of approximately 450 µm in size [45]. Moreover, these mechanically soft spheroids could deform upon grasping, which would eliminate the need for grooves.

The proposed approach presents several further development directions. First, the magnetic agents themselves could be optimized to improve grasping. For example, the surface of the magnetic agents could be patterned to increase friction and enable the grasping of slippery objects. Alternatively, soft magnetic agents could deform and adapt to the grasped object, enabling the grasping of smaller or round passive objects. Second, the tracking system could be replaced by an alternative imaging technique, such as ultrasound, to enable control in opaque environments. Finally, an increment in the number of magnetic agents could lead to more reliable grasping operations. However, the independent control of a higher number of magnetic agents would require a proportional increment in the number of electromagnetic coils. Alternatively, hundreds of particles can be controlled simultaneously (although not independently) by exploiting swarming behaviors of magnetic matter [46, 47].

In conclusion, the proposed 3D collaborative grasping system based on two magnetic agents offers a promising approach to micromanipulation and micro-assembly, with potential applications in various fields, including biomedicine and micromechanics. Moreover, the system could be further miniaturized for its application in biology.

# 5 Materials and methods

## 5.1 Hardware

**Electromagnetic system:** The actuation system employed in this work has been described in a previous work [38]. The system consists of 9 Vacoflux-core coils. All the coils are placed at 30 mm from a common center, 8 of them in the corners of a cube and the 9th one at the bottom. The system is capable of generating magnetic fields of up to 100 mT and gradients of up to 0.2 T²/m.

**Imaging system:** The imaging system consists of two Grasshopper 3 cameras imaging the system from the side and the top. The cameras operate at a resolution of 2048 × 2048 pixels (corresponding to a pixel size of approximately 10 µm), and an acquisition frequency of 10 Hz.

**Magnetic agents:** The utilized magnetic agents are 1 mm AISI 420C stainless steel spheres. The
magnetization curve of these agents can be seen in Figure S2. These agents present a negligible magnetic remanence and, in the linear regime, we estimate $K_{mag} = 1.4$ mN m/T².

**Passive particles:** The passive cubes and beam (Figures 4 and 5) where fabricated using the FORM2 SLA printer with White V4 resin. The passive gears (Figure 6) were fabricated by laser-cutting plexiglass in a Trotec laser engraver. The gears' axles are 0.6 mm copper wires.

**Others:** All the experiments were performed in M1000 silicon oil, with a viscosity of 1 Pa s.

## 5.2 Software

**Tracking algorithm:** For stereo visual feedback, we use two cameras, one that captures the system from the top and another from the side. We then utilized an image analysis algorithm based on the Hough Circles transformation to identify the position of the magnetic agents in each view. Finally, the 3D positions are determined using a triangulation algorithm. In the grasped state, the position of the passive object is defined as the mid-point between both magnetic agents, and its orientation as the normalized position of one agent with respect to the other. Since the magnetic agents can become occluded in one or both views while manipulating passive objects, we developed an observer that can predict the agents' positions when tracking is lost.

**Direct model:** The direct model computes the resulting gradient and field for a set of currents. Since the coils are operated in the linear (unsaturated) regime, the magnetic field at any given point in space can be calculated as the sum of the fields generated by each coil at that point. This can be expressed as:

$$ \mathbf{B} = \tilde{\mathbf{B}} \mathbf{I}, \qquad (8) $$

where $\mathbf{I}$ is a column vector with the currents circulating through each of the 9 coils, and $\tilde{\mathbf{B}}$ is a 3 × 9 matrix characteristic of the electromagnetic system. Each column of $\tilde{\mathbf{B}}$ represents the magnetic field generated by each coil and depends on the considered point. The $\tilde{\mathbf{B}}$ matrix is obtained through a calibration process, which consists of measuring the magnetic field generated by each coil at different positions [38]. On the other hand, magnetic gradients are non-linear with respect to the currents. We can use block notation to write:

$$ \nabla |\mathbf{B}|^2 = 2 \mathbf{J}_B^T \mathbf{B} = 2 \begin{pmatrix} \mathbf{I}^T & 0 & 0 \\ 0 & \mathbf{I}^T & 0 \\ 0 & 0 & \mathbf{I}^T \end{pmatrix} \begin{pmatrix} \tilde{\mathbf{B}}_x^T \\ \tilde{\mathbf{B}}_y^T \\ \tilde{\mathbf{B}}_z^T \end{pmatrix} \tilde{\mathbf{B}} \mathbf{I}, \qquad (9) $$

where $\mathbf{J}_B$ is the Jacobian of the magnetic field in that position, and $\tilde{\mathbf{B}}_x, \tilde{\mathbf{B}}_y$ and $\tilde{\mathbf{B}}_z$ are the first derivatives of $\tilde{\mathbf{B}}$ with respect to $x, y$ and $z$, respectively.

**Inverse model:** Due to the nonlinearity of Equation (9), the model inversion process is carried out using a numerical optimization algorithm. Since in our case, the only requirement is to keep the object grasped while displacing and rotating, we impose the magnetic field direction and only a minimum magnitude. Indeed, allowing the magnitude to vary can improve the response of the system, as mentioned in [20]. In that case, the model inversion objective is to find a set of currents that minimizes the total power under the non-linear constraints:

$$ \left\{ \begin{array}{l} \mathbf{F}_{\text{grad}} = 2K_{\text{mag}} \begin{pmatrix} \mathbf{I}^T & 0 & 0 \\ 0 & \mathbf{I}^T & 0 \\ 0 & 0 & \mathbf{I}^T \end{pmatrix} \begin{pmatrix} \tilde{\mathbf{B}}_x^T \\ \tilde{\mathbf{B}}_y^T \\ \tilde{\mathbf{B}}_z^T \end{pmatrix} \tilde{\mathbf{B}} \mathbf{I} \\ (\hat{\mathbf{R}} \cdot \tilde{\mathbf{B}} \mathbf{I}) = |\tilde{\mathbf{B}} \mathbf{I}| > B_{\text{grasp}}. \end{array} \right. \qquad (10) $$

To achieve this, we utilized a sequential quadratic programming (SQP) algorithm [48]. The SQP algorithm is a sequential quadratic programming method that solves optimization problems by iteratively approximating the constraints and minimizing the objective function using a quadratic model. This algorithm is particularly well-suited for solving problems with nonlinear constraints, making it an ideal choice for solving inverse mapping in our control system. The implementation of the SQP algorithm in MATLAB allowed us to easily incorporate it into our control system and solve the optimization problem in real-time. We
tested this algorithm for different sets of positions, desired gradients, and desired fields, and we found that if the field magnitude is below 70 mT and the gradient magnitude is below 0.1 T²/m, the system finds a solution 99.7% of the time.

**Supporting Information**

Supporting Information is available from the Wiley Online Library or from the author.

**Acknowledgements**

This work was supported in part by the European Commission under the Horizon Europe program under Grant #101070066 (RÉGO), and in part by the European Research Council (ERC) through the European Union's Horizon 2020 Research and Innovation Program under Grant #866494 (MAESTRO).

**References**

[1] J. Wallén, *The history of the industrial robot*, Linköping University Electronic Press, **2008**.

[2] A. Bicchi, V. Kumar, In *2000 IEEE International Conference on Robotics and Automation*, volume 1. IEEE, **2000** 348–353.

[3] D. Prattichizzo, J. C. Trinkle, *Springer Handbook of Robotics* **2016**, 955–988.

[4] P. S. Peercy, *Nature* **2000**, *406*, 6799–4023.

[5] B. J. Nelson, I. K. Kaliakatsos, J. J. Abbott, *Annual Review of Biomedical Engineering* **2010**, *12* 55.

[6] M. Sitti, H. Ceylan, W. Hu, J. Giltinan, M. Turan, S. Yim, E. Diller, *Proceedings of the IEEE* **2015**, *103*, 205.

[7] S. Nah, Z. Zhong, *Sensors and Actuators A: Physical* **2007**, *133*, 1–218.

[8] O. Fuchiwaki, A. Ito, D. Misaki, H. Aoyama, In *2008 IEEE International Conference on Robotics and Automation*. IEEE, **2008** 893–898.

[9] B. Ahmad, A. Barbot, G. Ulliac, A. Bolopion, *IEEE/ASME Transactions on Mechatronics* **2022**.

[10] A. Alogla, F. Amalou, C. Balmer, P. Scanlan, W. Shu, R. Reuben, *Sensors and Actuators A: Physical* **2015**, *236* 394.

[11] M. Leveziel, W. Haouas, G. J. Laurent, M. Gauthier, R. Dahmouche, *Science Robotics* **2022**, *7*, 69–72.

[12] D. Hériban, M. Gauthier, In *2008 IEEE/RSJ International Conference on Intelligent Robots and Systems*. IEEE, **2008** 4042–4047.

[13] E. Diller, M. Sitti, *Foundations and Trends in Robotics* **2013**, *2*, 3–143.

[14] M. Kaynak, P. Dirix, M. S. Sakar, *Advanced Science* **2020**, *7*, 202001120.

[15] A. Aghakhani, A. Pena-Francesch, U. Bozuyuk, H. Cetin, P. Wrede, M. Sitti, *Science Advances* **2022**, *8*, eabm5126.

[16] S. Mohanty, A. Paul, P. M. Matos, J. Zhang, J. Sikorski, S. Misra, *Small* **2022**, *18*, 971–989.

[17] E. Gerena, F. Legendre, A. Molawade, Y. Vitry, S. Régnier, S. Haliyo, *Micromachines* **2019**, *10*, 10067.

[18] Q. M. Ta, C. C. Cheah, *IEEE/ASME Transactions on Mechatronics* **2020**, *25*, 447–461.

[19] F. N. P. Basualdo, A. Bolopion, M. Gauthier, P. Lambert, *Science Robotics* **2021**, *6*, 52.
[20] M. P. Kummer, J. J. Abbott, B. E. Kratochvil, R. Borer, A. Sengul, B. J. Nelson, *IEEE Transactions on Robotics* **2010**, 26, 6 1006.

[21] B. Yigit, Y. Alapan, M. Sitti, *Advanced Science* **2019**, 6, 6 1801837.

[22] A. Ramos-Sebastian, S.-J. Gwak, S. H. Kim, *Advanced Science* **2022**, 9, 7 2103863.

[23] M. Sitti, D. S. Wiersma, *Advanced Materials* **2020**, 32, 20 1906766.

[24] T. Xu, J. Zhang, M. Salehizadeh, O. Onaizah, E. Diller, *Science Robotics* **2019**, 4, 29 eaav4494.

[25] J.-C. Kuo, H.-W. Huang, S.-W. Tung, Y.-J. Yang, *Sensors and Actuators A: Physical* **2014**, 211 121.

[26] B. Ahmad, A. Barbot, G. Ulliac, A. Bolopion, *IEEE Robotics and Automation Letters* **2023**, 8, 3 1675.

[27] J. Zhang, O. Onaizah, K. Middleton, L. You, E. Diller, *IEEE Robotics and Automation Letters* **2017**, 2, 2 835.

[28] E. Diller, M. Sitti, *Advanced Functional Materials* **2014**, 24, 28 4397.

[29] S. E. Chung, X. Dong, M. Sitti, *Lab on a Chip* **2015**, 15, 7 1667.

[30] M. A. Rahman, A. T. Ohta, *IEEE Open Journal of Nanotechnology* **2021**, 2 8.

[31] M. A. Rahman, J. Cheng, Z. Wang, A. T. Ohta, *Scientific Reports* **2017**, 7, 1 3278.

[32] F. N. Piñan Basualdo, G. Gardi, W. Wang, S. O. Demir, A. Bolopion, M. Gauthier, P. Lambert, M. Sitti, *IEEE Robotics and Automation Letters* **2022**, 7, 2 2156.

[33] N. A. Torres, D. O. Popa, In *2015 IEEE International Conference on Automation Science and Engineering (CASE)*. IEEE, **2015** 1608–1613.

[34] Y. Kantaros, B. V. Johnson, S. Chowdhury, D. J. Cappelleri, M. M. Zavlanos, *IEEE Transactions on Robotics* **2018**, 34, 6 1472.

[35] X. Dong, M. Sitti, *The International Journal of Robotics Research* **2020**, 39, 5 617.

[36] S. Chowdhury, W. Jing, D. J. Cappelleri, *Journal of Micro-Bio Robotics* **2015**, 10 1.

[37] L. Yang, L. Zhang, *Annual Review of Control, Robotics, and Autonomous Systems* **2021**, 4 509.

[38] F. Ongaro, S. Pane, S. Scheggi, S. Misra, *IEEE Transactions on Robotics* **2018**, 35, 1 174.

[39] D. J. Griffiths, Introduction to electrodynamics, **2013**.

[40] A. Denasi, S. Misra, *IEEE Robotics and Automation Letters* **2017**, 3, 1 218.

[41] M. Salehizadeh, E. Diller, *The International Journal of Robotics Research* **2020**, 39, 12 1377.

[42] F. N. Piñan Basualdo, O. Stephan, A. Bolopion, M. Gauthier, P. Lambert, *IEEE/ASME Transactions on Mechatronics* **2023**, 29, 1 64.

[43] C. M. Heunis, Z. Wang, G. de Vente, S. Misra, V. K. Venkiteswaran, *Macromolecular Bioscience* **2023**, 2200559.

[44] Z. Wu, Y. Zhang, N. Ai, H. Chen, W. Ge, Q. Xu, *Advanced Intelligent Systems* **2022**, 4, 7 2100266.

[45] X. Peng, Ž. Janićijević, S. Lemm, M. Laube, J. Pietzsch, M. Bachmann, L. Baraban, *Biotechnology Journal* **2023**, 18, 6 2200365.

[46] D. Jin, L. Zhang, *Accounts of Chemical Research* **2021**, 55, 1 98.

[47] Z. Xu, Q. Xu, *ACS Nano* **2022**, 16, 9 13728.

[48] M. Biggs, *Towards Global Optimization* **1975**.