To me, this is the ultimate nightmare scenario, as I cannot see a good solution to this problem (though I might be missing something obvious... please let me know!)
The problem with this solution is that it could be very, very expensive with thousands of users, given all the data they've stored in the contract.
The problem with this solution is that EVERY SINGLE CALL into the contract will have to pay extra gas for this indirection, which seems costly. Plus, there still has to be some contract that holds the storage data, and that contract may still be at risk for containing errors.
That solution just passes the problem to another system, so this also can't lower costs (since I'd just be able to apply the same optimization in my Dapp, whatever it is.)
Additional note: I'm not asking the question "how does the app decide who gets to make the modification, and how do we make sure the users are OK with this?" That's a separate problem that is also very important, but this question concerns the more fundamental question of "How do I make the contract code upgradable in a cost-effective manner, for a dapp with lots of storage?"
so I think the solution is probably two fold. first, you want to separate data and logic as much as possible. write a contract for storing your data simply, with a simple, solid api. test it thoroughly. Maybe you'll use the eris stack for the testing.
Then use a name reg contract (or what Eris calls a Doug), to be the unchangeable contract address of the dapp, and store the address of the contract that does the logic. The logic contract talks to the storage contract (which may also be pointed to by doug), and may be updated by whoever has permission to change the doug.
You simply have to accept the extra fees for the calls as the cost of better programming. Eris will add a series of opcodes to make this process easier, and help with testing, and maybe ethereum will adopt too, but in the end if ethereum is going to be successful, bugs are going to be expensive.
If you are familiar with DOUG, this is something he is well set up to help out with. DOUG can track contracts which server a purpose as well as permissions. There are two equally valid ways to do this with DOUG.
First off when structuring your smart contract suites keep your user data and things in a contract dedicated to just managing your database of user data. Keep your logic in other contracts which manipulate these databases through basic api's. If you find a bug in your logic, use DOUG to replace the broken logic component.
The two ways to do this is to give permissions to a contract to access the database. or have the contract serve a "named" role so when a contract is looking for the contract which serves a particular role it asks DOUG for it.
(PS if you are not familiar with DOUG I have written a lot about it as a smart contract design philosophy I'm sure you can find some stuff or stop by and I'll be happy to answer questions. Including a DOUG really makes a lot of things simpler in smart contract suites.)
Thanks work2heat and dennismckinnon, those were really helpful responses. Clearly, creating a contract that only manages data (which is a clearer framing of the solution than I had offered) seems to be the most sensible and agreed-upon approach, now that I read your answers... I will just need to bear the additional cost of the call.
Also, I will be spending some time with Eris and DOUGs to see if I want to incorporate that for higher-level management of my dapp.
This is a tough one. I have come across this some times, but did not find any be-all-end-all solution. That being said, I don't think any contract that is expected to hold a lot of data should be without a separate "controller" contract. Bugs are not the only issues, you might just want to modify it at some point (or have to), and with a fixed interface you're basically screwed. Yes - it does cost more, but by trying to cut costs you could end up with something much worse - some hacky emergency solution that is both more bulky and more expensive to run, or data that is essentially locked down (been there...). Contract to contract calls aren't that expensive, and any non trivial system will have to do a lot of them. There is definitely room for optimization in the actual controller code, though.
Also btw. I think one of the best things to do when managing large amount of data is to think about if the data can be structured in some way - like can it be divided up into groups, or maybe even made into something like a tree. If you keep a lot of un-structured data in one single contract (say a massive list of key value pairs), you will also run into issues like when you want to remove chunks of it (not just single entries). This applies to copying too. Storage is expensive, and because each block has a max gas limit, you'll only be able to do a small number of adds/removals per block, which could be a problem. Having structured it into multiple contracts, for example, makes it a lot easier to remove large chunks of it by just removing a reference or two...
> Storage is expensive, and because each block has a max gas limit, you'll only be able to do a small number of adds/removals per block, which could be a problem.
Darn, a new problem that hadn't even occurred to me yet. Thanks for the protip on "sharding" data across contracts, that may apply in my use case.
Maybe for some things you can just aim at not having that happen. Particularly for simple contracts, or contracts that go through use, and are then suicided/reset. So if a flaw is detected, they are simply replaced after the use. Edit: maybe there is a condition where users can agree to join to change to some other contract.
Not-having-breakage happen, well reducing the odds is a matter of 1) keeping things simple, 2)walk-through review 3) static tests(properties of code, perhaps adding tested declarations) 4) fuzz tests and specific cases for review. (not sure how to best hook in more advanced fuzzers, like the one that recently made jpegs.)
Now, the risk that people creating DApps with less review/testing are more successful because they get there earlier. People might anticipate that too. Of course, if the 'consumer end' is smart, they dont take unnecessary risks on that side. Partly the point of smart contracts is that the users will 'trust, but verify'* and have 'informational networks' to a degree that they can figure out what contracts are good to use.(reputation systems made for/with ethereum being part of said networks)
All good points, Jasper... that's why I'm working hard to get my final contracts for launch done ASAP so there's lots and lots and lots of room for testing.