You don't even need to make "EU user data" as a whole stay in the EU.

Assuming the hypothetical company question is actually affected (e.g. a company with no EU presence for the most part don't need to care):

You need to make sure privacy is respected. If you want to be able to tie your data back to the identity of an individual user, then yes, you potentially have more work. If you don't need to do that, often simply stripping personally identifiable details will be sufficient.

This means stripping names and addresses, phone numbers, e-mail addresses, and possibly - depending on context - IP addresses, but not much more.

Some filtering of your logs prior to aggregation, or adjusting database queries accordingly will get you far.

But the thing US companies with a EU presence should be more concerned about is that the ruling opens the door for people to raise claims with national data protection authorities to try to get a ruling that storing the data in Europe is insufficient.

E.g. if a US company stores data in Europe, but the hosting is in the name of the US company, then what is stopping US authorities from compelling the transfer of the data to the US?

It is at least possible to envision the possibility that US companies with sufficient exposure to the EU may end up needing a subsidiary in Europe to be responsible for the hosting and put in place processes to ensure that the EU based subsidiary maintains control of access to the data, to ensure that the US parent can't just come in and grab personally identifiable information.

It would seem to me that regardless of how this plays out, it is a good argument for thinking about privacy as part of technical architecture:

Consider what is personally identifiable information (in a legal sense; e.g. in Europe your e-mail address will generally be considered to identify you, and so be covered), what is private but possibly not personally identifiable in a legal sense (e.g. your photos) vs. what is not affected by privacy (e.g. fully anonymous data, such as performance metrics for requests made to non-private urls), and design your systems to segregate them from the outset where possible.

E.g. don't tie data back to users if you don't have to; when you have to, consider if you can anonymize it (e.g. do you need to know that record X refers to user Y that is john.smith@example.com, or is it sufficient to be able to tell that record X refers to a user Y where "Y" is a unique id that does not link back to your user data but is consistent within the given report, for example), or hash it in ways that makes the data defacto anyonymous to anyone without access to the personally identifiable data; normalize your data sets, and store links from personally identifiable data to public data separate from the public data.

In most systems I've worked on, if you start looking at this, the end result would generally be a system that is easier to secure in a meaningful way too: You end up with clearly delineated datasets with very different security requirements, and the datasets that needs to be most thoroughly secured end up being vastly smaller than if it's all smudged together in a single database and littered all over your log files etc.

I used to work on an e-mail platform where we did this around 2000. In effect it was out of naivety: We thought people actually cared about data security and that keeping such data separately was just what sensible people would do.

As a result, when users registered, most of the registration information we regularly moved to an offline system (as in: no network), where it was encrypted, burned to CDs, which were moved into a bank security box together with our backups once a day.

It just seemed to make sense, and it was not such a huge imposition or cost. It took an old PC sitting in the corner, and one guy making a short trip once a day.

It wasn't a perfect split: Since we were running an e-mail service, of course we had the e-mail addresses and peoples e-mails to deal with in an online system. But while being a pain for reporting, it taught us to always question why a certain piece of information was needed. Did we really need the addresses or phone numbers people had given for this report? Important enough to get the user data out of the security deposit box to take an extract and merge? Really?

It turned out most of the time people did not really need most of the data, or only needed e.g. a way to indicate that something belonged to the same user as something else, but may not have needed to know who that user actually was.