Don't force the issue. Let them have their taste, then supply them with as wide a variety of foods as possible. Lovebirds do tend to thin down quite a bit during the weaning process. Pay careful attention to this as you don't want the bird to get too thin. I find lovebirds are just not big piggies when they wean the way budgies are.
Lovebirds tend to eat a little all day long rather than pig out.
There is nothing wrong with offering a little formula just to be sure, but if your bird has been going four days on just a tad of formula, you can be pretty sure they are weaned. It's always better to go a little long with handfeeding than to wean too quickly. You'll know pretty quickly if there is a change in the bird's appearance or vitality. A new owner might not notice the subtle signs that you would notice. This is when the bird wants to handfeed again after being off formula for a few days, and can occur when they go through the stress of being in a strange, new home.