Oh! That's a good question!

tl;dr: Nobody knows :(

It's an open question in physics. It's essentially the https://en.wikipedia.org/wiki/Measurement_problem or at least a huge part of it.

It's a weird rule in Quantum Mechanics, that is solved by hand waving. In most experiments it's obvious what is the system and what are the measurement devices. Nobody likes the rule, but no one know how to avoid it (and nobody knows what it really means).

There is a small group of people trying to solve it, and the rest of us just ignore the problem. We have been using QM for a century, and no one is sure that we don't need another century to fix the rule. So "Shut up and calculate!" meanwhile.

It's difficult to answer without adding a few guesses here and there, so take the rest of my comment with a grain of salt.

<may include some guesses>

If it's an experiment made on purpose, people is careful to avoid anything in between, so everyone can use hand waving and agree what is the system and what are the measurement devices.

There are some experiments trying to use quantum systems as big as possible, for example the one described in the article. We expect that it's possible to use even bigger things, but it get's difficult to keep them cold and isolated.

I'm not sure if there is an experiment showing that a small "brick" can act as an observer when the "brick" is very small. The idea is that the particles inside the brick are so many that it's impossible to track them and also all the quantum path are so different that there is no interference between them.

Nobody is trying to measure that, because every time the system is not isolated enough (for example because you expected a total vacuum, but some molecule of air entered the system anyway) the quantum effects are no visible. Does it count as noise or a measurement by a rogue device?

It's difficult to make experiments with particles of different size to understand where is the threshold. It's easier to make experiments with gas at different temperatures.

<tangent>

At high temperatures gas (like air) act as a classic gas, and follow the rules we would expect if they are made of a huge lot of small balls.

At low temperatures, the gas become weird. (Assuming you pick a gas that does not condensate and is a liquid.) They don't follow the rules of a classic gas. For example if you fix the temperature, the volume and the amount of gas inside a recipient, then there are some quantum effect that make the pressure of the gas higher or lower than the equivalent classic gas.

It's quite easy to measure, and there are many quantum mechanics effects that are posible to see with this method, and it's easy to regulate the temperature to get a "classic" gas or a "quantum" gas. The interesting part is that there is no threshold. The gas changes smoothly between "quantum" and "classic" when you increase the temperature, and you get a behavior that is in between.

What is really happening is that the gas is always a quantum gas. But a hot quantum gas has an equation that is very very very similar to the equation of a classic gas. So if you are only going to use hot gas in your experiment, you can ignore the quantum effects and use the classic gas equations that are much simpler and almost as good as the difficult quantum gas equations in your experiment.

</tangent>

Back to your question... Nobody is sure, but the guess is that the difference between observed and observer is just a simplification. There is probably not true difference. It's just a trick to simplify the calculations. If you avoid some weird corner case, it work really well. Let's hope we can know the correct answer some day.

</may include some guesses>