>They're broadcasting their data. If you request data in a specific way (with a given user agent, or via a proxy), are you violating their copyright?

>If you hit a paywall and use the built in web tools to delete the offending divs or disable javascript for that site, are you violating their copyright? Do sites have a right to require you represent the data they send in a particular way?

Generally, yes, doubly so if you persist after being "asked" to stop. You can also be found offending under the Computer Fraud and Abuse Act. Read the case law on this if you want to get pissed off.

There have been several cases on these matters, and the overwhelming majority of them have confirmed that such acts are illegal. Due to the structure of copyright law, the small quantity of victories are hard to generalize (fair use is an affirmative defense), and basically only happened because the judges didn't want to take the heat for shutting down Google.

A web page cannot be accessed without infringing copyright except insofar as the user displaying the page is entitled to either an explicit or implicit license. Without valid licensing, the copy of the web page that exists in your system's RAM has been repeatedly ruled to be infringing.

Using proxies and masking user agents are big no-nos and usually function as good evidence that infringement was willful, which means an entitlement to treble damages, plus the animosity of the court.

The long and short of it is that if you continue to attempt to access a site after you know they don't want you to access it, you have violated the CFAA and could go to federal prison for doing so. You've almost certainly also violated the Copyright Act. Even if you don't know they don't want you to access it, it's certainly doable to build a case along these lines.

Aaron Swartz was being prosecuted for exactly this. He had set up a computer to download publicly-funded research papers from a service that MIT had a subscription to. His crime was downloading content in a way that the service didn't like; as far as I know, there was no service disruption as a result of Swartz's activity (and even if there were, that should entail civil, not criminal, penalties). We all know how that turned out.

You can make anyone in the U.S. a felon-in-waiting by a) knowing they visit your site or otherwise access a computer system under your control; b) deciding you don't want them to visit anymore; and c) making some indication of this, explicit or not, through some channel that a judge will decide should've been good enough to inform the accessor that his access was no longer wanted. Banning their IP probably works just fine, and surely setting up a paywall that takes explicit action to circumvent would as well.

Our laws are not made for the digital era.

Big tech companies like the status quo because it gives them carte blanche to rip off and trounce smaller competitors, while judges give them the benefit of the doubt (Perfect 10 v. Amazon). The big guys use fancy, $1,000/hr law firms to intimidate people who would make their data portable, and if that doesn't scare you off, they proceed with the lawsuit, ask their friends in the government to proceed with the prosecution, and take both your financial and physical independence. And yes, this does happen. It happens daily.

The "network effect" and "chicken-and-egg syndrome" discussed with relation to online services exists solely because we've legislated them into place through these laws. The digital world is valuable because massive amounts of data can be replicated quickly and exactly. There is no reason that "network effects" would ever be a thing here (people would use tools that read data sources and products would compete based on the features of the data viewer, not based on who had which data), except that we've made it a thing so that it's easier for big companies to kill competitors.

I'm not a lawyer; this reflects my layman's understanding.