I don't think it's that complex. I always thought it was inevitable that once everything we did happened via a computer, gathering it all together would be an irresistible attraction to man+dog. It was always going to accelerate, and it was always going to go into overshoot.

And while it's painted in forum's like this as all negative, the reality is it's not. In fact there have been some pretty big positives. Crime levels have been dropping, while prosecutions have gone up. Criminologists have put that down to better surveillance. Cameras everywhere have meant the odds of getting caught for a given offense have gone up, so more people have been getting caught on their first offence. Amusingly, it's just just the criminals who have been on the receiving end of this - it's the law enforcers too. They used to be able to hide their overreaches behind the solidarity of the blue line. Now misbehaviour is very likely caught on a camera that's not behind the line, and the results have been on the world's headlines.

I know the naturally privacy conscious feel like this is a personal invasion, I feel like that too on occasions. Their problem is that to date it's most likely been a net positive. Open societies with less criminality and corruption do better, and that seems to have been the net result so far. While like you I doubt many people have analysed in these terms, what they have done is evaluated how life is now vs before, and it seems OK. In simple terms, we can have more shiny things. Therefore the simply don't care some privacy guru seems a privacy apocalypse around the corner. So far the reverse has been true, and wot do these supposed experts know anyway?

Still, I feel decidedly uncomfortable with the Assistance and Access bill, and its friends. My problem is not that they allow enough information to be mined - it's that they don't allow enough. Specifically, while they allow the government agencies to gather information on its citizens, the bills go out of their way to ensure the citizens have no idea how much their government is spying on them, or who is being spied upon, or why, or at whose behest. Without a little sunlight to sterilise things, it's inevitable malfeasance will grow in the dark. We already have numerous examples of petty privacy theft - police spying on their ex's, taxi's chasing credit card owners and the like. It's just a question of time before one politician plunders another's emails and polling data, or one of these systems gets hacked and mined for identity data for years before anybody becomes the wiser.

The sad bit is, it is only after they have blatant examples staring them in the face that voters will conclude this overreach has downsides too. So we will have to suffer some real pain before there is a correction. The good news is the public education has already started. The Huawei bans are an example of that. It was only after Huawei won the contract to run the Danish telephone system, and later the Danish discovered their private data leaking across the world that politicians and intelligence agencies woke up to the fact the same data gathering capabilities they were exploiting could also be used against them.

They reacted by banning Huawei. One day they will figure out that isn't going to fix the problem. We've since learnt Juniper was hacked for years, apparently by the Chinese who could watch all data flowing through their gear. And that's just the foreign players. In Russia we have private criminal gangs gathering data on a scale well beyond their Law Enforcement Agencies, and using it to shake down institutions like banks. The only cure I can see for this is a transparency on a scale we haven't seen to date. We have to put the days of blindly trusting government bureaucratic caesars to determine what data is collected, who can see it and what it is used for behind us. That information is too dangerous to be kept in the dark - it must be open to all citizens to inspect.