Without abolishing VAT, what would be a way to ensure that it is properly collected by the buyer's country without putting undue stress on small sellers?

One thing I can think of is that you could target the payment processors. They already have personal data about the buyer, so they should know where they are located. Most transactions are going to be done by a credit card processing company, or something like PayPal. In either case, the EU could deal with those sorts of companies directly.

In that case, merchants could sell their goods without worrying about tax at all. You buy something at a listed $100, you have 22% local tax, the credit card company charges the buyer $122, seller doesn't have to worry about it. Now, the seller could program some option that could estimate VAT based on geolocation with some disclaimer that the customer will be charged for VAT by the payment processor and are responsible for their own charges. But that would be a courtesy of the seller, the buyer should be aware of their own VAT.

Some things are excluded from VAT, but they are the exception, and in that case, the seller could jump through some small hoops to register with the payment processors to allow those goods to exclude VAT.

There are still other methods of collecting payment, like bitcoin, like money orders, whatever. In those cases, the law should require the buyer to self-assess. In reality the self-assessment would be rarely followed, and honestly nobody would care if it were small purchases. But the self-assessment rules would mean that anyone seriously gaming the system by avoiding compliant payment processing options could be punished, while still both avoiding an less enforceable situation like trying to punish a foreign company for failure to charge local tax, or limiting the payment options that foreign companies can provide.

This could also solve problems with collecting VAT purchases made from foreign entities (instead of just companies from other EU states) and could even be extended further (You are a resident of Luxembourg, you travel and shop in France, you pay with a credit card, you are charged less VAT on the purchase rather than having to claim the difference when you return.)

In short, compel large payment processors to verify the home of buyers when the account is created, compel them to have some checks to avoid fraud. Allow sellers to not charge VAT if the buyer is either using an approved payment processor, or if the buyer is from another country. If you are accepting payment by cash or money order from someone local, then you must charge VAT at the local rate. But if you are accepting a credit card through an approved processor, whether the buyer is local or foreign, you charge the rate pre-tax, and the processor is required to A) add the appropriate VAT, and B) verify that the buyer is not lying about their home country. If you accept a money order from a foreign country, you do not charge VAT, and the buyer self-assesses.

Alternatively, you could say that all purchases that do not go through an approved processor have the local VAT added to them, and the buyer is compelled to self-asses for the difference, it's just a matter of how much you are willing to inconvenience non-EU foreign countries.

But I don't think the seller should be required to ensure that the buyer abides by their local tax rules. I do think that's a task that can be handled by large banks or payment processors.