Nothing "real" exists exactly the way we perceive it. The world we experience is one where the amount of detail has been compressed by an extreme factor. We don't perceive the quantum fields of the surfaces we touch, we don't perceive individual photons hitting our eyes. The "real" world contains so much information that our brains have no way to process it in detail.

Instead, our brains will create a model of the world that is optimized for utility (in a Darwinian sense). In fact, this happens at different levels. Some centers of the brain (those that regulate basic functions as well as sensory input) will have access to different information to what reaches our consciousness.

A "purple" flower doesn't emit "purple" photons, but rather a combination of photons that our sensory system aggregate to "purple". Another mix of the same types of photons, but at different relative number may look "brown", "gray", "white", etc. In other words, all such qualia are some kind of illusion, or maybe more accurately, aggregated information.

The same is true for most categories of concepts. Physical objects, surfaces, fluids, living objects, etc. Including what we think of as "agents".

"Agents", whether they are the tiger we imagine in the forest, another human or indeed ourselves, and whether real or imagined, are also way too complex for us to comprehend in detail.

What we're interested in (or rather, what has Darwinian utility) when dealing with "Agents", as opposed to non-"Agents", is that "Agents" tend to respond to stimuli in more sophisticated ways than non-"Agents". If we look big and scary, a wolf may decide not to eat us or a robber from robbing us, but it will not stop a house fire from burning us.

As a consequence, there are actions we may take to improve the odds that the "agent" types around us behave in ways that are beneficial to us. Just like we're not able to comprehend something as simple as "real" photons behind the color "purple", our brain is not able to calculate EXACTLY what we can do to reach that goal, however.

Instead we (most of us) seem to have a simplified framework more or less built in to find something approximate to optimal ways to modify the environment in ways that affect the behavior of the "agents" around us. "Free will" or "consciousness" or not perfect representations of the ways the brains of "agents" work. But the concepts are simple enough that our brains can reason about them (at some level), while predictive enough to have significant utility and also to be hard to "hack" for an agent that may have conflicting interests.

Roughly speaking, the utility of the concept "free will" is that it allows us to communicate to others a set of behaviors that are likely to be met with retribution, while mostly limiting such threats to situations where the threat itself is likely to prevent the unwanted behavior from happening in the first place.

Actually having to dish out punishment is usually not high utility behavior. It can be both costly and expensive. Having to do it, can mean one (or more) out of three things:

1) The punishment isn't harsh enough

2) The adversary does not see the threat as credible. A thief may think they will not be caught, or a state leader thinks the ally of a country will not help defend them.

3) The adversary is not able to understand the threat, or in some other way incapable of taking the threat into account when acting.

In the case of the last one, one can have a rule that punishes people from killing other people, for instance. If the punishment is severe enough, and the likelihood of getting caught is close to 100%, most people will stop killing each other. However, SOME people will not be able to take the information into account for some reason or another. It could be that they go into a rage that make them ignore any kind of upcoming punishment, there may be some kind of mental disease, they may not have enough intelligence to understand the threat, or avoiding killing them could require some skill they do not possess (maybe they were not able to tell the gun was real, and indeed loaded).

If we were able to isolate all cases that fall under type 3, we might call those cases acts should not be punished, in other words, they did not have the "free will" to avoid them.

But the distinction may be hard to identify. Making too many exceptions will often both increase the number of people who do not see a threat of punishment as credible AND also increase the number of people that would be able to understand what the rule was.

So, basically:

- "Free will" is a simplification of the kind of process that causes threat of punishment to prevent us to do some set of actions that will be met by retribution. If legal systems (or similar informal retribution threats) are set up properly, those who actually have "free will" about the type of action, will really be punished for it.

- Those who DO get punished will very often NOT have "free will", in the sense that they are unresponsive to stimuli about that type of action. When people DO end up in prison (and assuming that punishment is strict enough and the police is able to catch most criminals), will be those NOT responsive to the threat of punishment, for some reason or another.

- This may still provide close to optimal utility for the set of people who decide what to punish. The cost of punishing those who are for some reason not able to avoid doing the "crime" is lower to that group than the benefit gained by the threat causing many others to NOT commit the "crime", when they otherwise would have.

- Indeed, since we are not capable of a fully detailed analysis of the "real" things happening inside the brains of "agents", we have no choice but using a simplified model. Some concept like "Free Will" is probably needed to lay out consequences of actions. Also some people have more of an ability to understand the details and limitations of such models, so simpler models have the advantage that it is easier to communicate. (To children, people with limited cognitive abilities or immigrants with poor cultural understanding.)

At this point, I could start writing about how empathy comes into this, how empathy is modulated by a feeling of kinship or shard identity as well as a perception of being threatened by someone, but that would make this post quite long :)