Title: EIP_4444__Bound_Historical_Data_in_Execution_Clien
Category: EIPs

Author: matt
Date: 2021-11-10T14:17:27.190Z

Discussion thread for EIP-4444.

Author: yoavw
Date: 2021-11-10T20:32:38.231Z

A link to the EIP itself?  (I don’t see it in the github repo)

Author: matt
Date: 2021-11-10T21:24:05.775Z

Was hoping to get it merged quickly and update with eips.ethereum.org link, but it didn’t happen. I’ve updated it just now with the PR link.

Author: mkalinin
Date: 2021-11-11T10:42:32.666Z

It worth say when the specification of this EIP comes into effect. An important thing that this change depends on the Merge. This is because before the Merge historical block headers are essential for sync and bootstrapping process. Verifying the PoW seal of all blocks in the chain is the only way to prove the chain is valid. Network upgrade to PoS shifts this paradigm and makes historical data of EL chain not a requisite for node bootstrapping process.
I don’t think that this spec should be so prescriptive, i.e. saying MUST NOT wrt serving ancient data. The reason is that sync process of some clients may depend on the historic data and they will need a time to be prepared for such a big change. For instance, Erigon executes all blocks since genesis and doesn’t used any state downloading techniques to get in sync with the network; they will probably want to serve ancient blocks even if other clients stop doing it.
IMO, the purpose of this EIP in the context of the Merge is to send a clear signal to infra, users, and clients that they should consider that the invariant of storing the history is going to be broken in some future. If we don’t want this for the Merge then it could be prescriptive but it worth considering that it will take a lot of time for network participants to get prepared for such a change.

Author: djrtwo
Date: 2021-11-11T14:53:38.684Z

mkalinin:

I don’t think that this spec should be so prescriptive, i.e. saying MUST NOT wrt serving ancient data.


I disagree here. If we don’t make this prescriptive then we will have a gradual degradation in historic block and receipt sync while users/clients still rely on it, until it just becomes broken and a bunch of users are confused and upset.
Making this a MUST NOT serve makes this a harder EIP to implement because it will require preparing dapps (especially receipts) and users for this breaking change, but then it will be completed in a clean way. If instead it’s a SHOULD NOT or MAY NOT, we create a path for dapps to slowly become broken on some indefinite timeline (because many will just continue to rely on the functionality for as long as it seems to work).



 mkalinin:

For instance, Erigon executes all blocks since genesis and doesn’t used any state downloading techniques to get in sync with the network; they will probably want to serve ancient blocks even if other clients stop doing it.


iiuc, Erigon has had a torrent block downloader in production for quite a while (it is way faster) and is expecting this breaking change at some point.



 mkalinin:

IMO, the purpose of this EIP in the context of the Merge is to send a clear signal to infra, users, and clients that they should consider that the invariant of storing the history is going to be broken in some future


I agree with this, but to do so effectively, I think the strategy is to specify the EIP as it will be in it’s final form and begin communicating about it now rather than actually introducing the breaking change simultaneously with the Merge. Imo, this is going to take 12+ months to properly communicate and execute on the dapp/community side, but this EIP can use the Merge and weak subjectivity in it’s rationale to bound it to this shift to PoS even though it wont be fully implemented at the point of the shift.

Author: gcolvin
Date: 2021-11-12T02:26:27.245Z

Preserving the history of Ethereum is fundamental

Yes.

We suggest that a specialized “full sync” client is built. The client is a shim that pieces together different releases of execution engines and can import historical blocks to validate the entire Ethereum chain from genesis and generate all other historical data.

You don’t say who would build or maintain this client.  And it’s not clear to me how the shimming would work.  Existing clients go to some effort to efficiently manage the differences between releases, given most of the code hasn’t changed.

There are censorship/availability risks if there is a lack of incentives to keep historical data.
…
there is a risk that more dapps will rely on centralized services for acquiring historical data.

Yes.  And I don’t think mitigating these risks is “out-of-scope for this proposal”.

Author: matt
Date: 2021-11-12T04:55:09.961Z

gcolvin:

it’s not clear to me how the shimming would work


Let’s say there are upgrades X, Y, and Z. During X, clients support X. During Y, clients support Y and so on. After each fork the code to run the fork is removed (save for the transition period of the fork). So if you want to validate the entire chain, you would start the client version that supports X and import all the X blocks. Then it would shutdown and you would run the client version that supports Y with the same data directory and again import all the blocks. Proceed with this and eventually you’ll be executing the tip of the chain with everything validated. The two caveats are i) after the merge, you’ll also need to get the beacon history and drive clients with that and ii) you may need to run glue code between client versions if there are breaking changes to the state / history storage.



 gcolvin:

And I don’t think mitigating these risks is “out-of-scope for this proposal”.


I think there are pretty straightforward solutions to replicated storage of static data that we can tap into for this (e.g. IPFS, torrent, data mirrors, etc).
–
Copying from the PR thread:

@djrtwo: Note, that discussions with @karalabe have pointed to making this a MUST because otherwise users will still rely on it while quality of this feature degrades until it is unusable. A MUST, instead, forces users and dapps to actually “upgrade” how they utilize this at the point of this shipping, forcing everyone’s hand so it doesn’t silently get worse and worse.


@djrtwo: If we go that path, then this spec should specify that devp2p does return errors on requests outside of the specified range of epochs/time

I feel like SHOULD is adequate level of force for clients to do this? Geth continue to make up a large portion of the network and if they stop serving the data, it’s going to mostly be unavailable (and users will mostly be forced to adapt). That said, I don’t have a strong argument for one or the other. The one caveat is that most devp2p messages wouldn’t know how to distinguish requests for “non-existent data” and “expired data” because they are by hash. Only GetBlockHeaders is done via block number and could therefore return an error.
If we were to go with SHOULD and GetBlockHeaders returns an empty response instead of error, I think the main fail case will be clients that try to header sync the old way will be confused since they don’t get any headers back. This seems acceptable and avoids an new wire protocol version.
–

@axic:  Actually back in April 2020 or so, within the Ipsilon team we thought about making a proposal to just hardcode given hashes for given blocks in an EIP. The idea would be to hardcode the hashes for past hard forks.


However then regenesis was proposed, which in practice does the same, but programatically.


Since regenesis as a concept is delayed ever so often, and given this EIP, would such a proposal make sense now?

In my mind, regenesis tackles a slightly different problem (and when I say regenesis, I generally mean this version). This EIP is about removing the need to store historical data whereas regenesis is a mechanism primarily aimed at reducing the state data. Regenesis, as far as I understand it, does not prescribe that clients should discontinue holding historical headers / bodies / etc.
I am curious to understand better what you mean about hard coding past fork blocks. I think you’re referring to a type of weak subjectivity checkpoint?
–
Generally a question that keeps coming up is how to deal with the difference between non-existent and expired data. Geth has set the precedence in v1.10.0 by turning the txlookup index to prune indexes older that 1 year by default. This means that if a user calls eth_getTransactionByHash with a valid tx hash from Byzantium, it will return an empty response.
Is the going to be acceptable behavior other data like blocks? And is acceptable over the wire? Seems like we’re leaning towards “yes”.

Author: MicahZoltu
Date: 2021-11-12T12:07:48.548Z

djrtwo:

I disagree here. If we don’t make this prescriptive then we will have a gradual degradation in historic block and receipt sync while users/clients still rely on it, until it just becomes broken and a bunch of users are confused and upset.
Making this a MUST NOT serve makes this a harder EIP to implement because it will require preparing dapps (especially receipts) and users for this breaking change, but then it will be completed in a clean way. If instead it’s a SHOULD NOT or MAY NOT, we create a path for dapps to slowly become broken on some indefinite timeline (because many will just continue to rely on the functionality for as long as it seems to work).


My understanding of RFC 2119 is that what you are describing is exactly what SHOULD is used for.  SHOULD is a way for a specification to say “this behavior is what is best for the ecosystem/user, but it isn’t something that is strictly enforced and if you don’t follow it nothing is going to outright break”.

Author: gcolvin
Date: 2021-11-12T22:25:43.064Z

matt:

So if you want to validate the entire chain, you would start the client version that supports X and import all the X blocks. Then it would shutdown and you would run the client version that supports Y with the same data directory and again import all the blocks. Proceed with this and eventually you’ll be executing the tip of the chain with everything validated


That’s what I feared  It means all of these clients have to be maintained indefinitely.

Author: gcolvin
Date: 2021-11-12T22:45:50.134Z

matt:




 gcolvin:

And I don’t think mitigating these risks is “out-of-scope for this proposal”.


I think there are pretty straightforward solutions to replicated storage of static data that we can tap into for this (e.g. IPFS, torrent, data mirrors, etc).


Then I’d like a bit more discussion of what the solutions and who is responsible for fixing what is going to get broken.

Author: MicahZoltu
Date: 2021-11-13T11:37:29.611Z

A. This is only necessary if one wants to validate the entire blockchain from genesis, which I argue is an uncommon operation at best, and I suspect eventually will simply be something that no one does.
B. The old clients don’t have to be maintained, they only need to continue to exist.  No updates need to be applied to them.

Author: jpitts
Date: 2021-11-15T03:08:46.321Z

Preserving the history of Ethereum is fundamental and we believe there are various out-of-band ways to achieve this.

It should be stated what preserving the history of Ethereum is fundamental to, and how important state history preservation is relative to other properties of the protocol.
And if it is as fundamental as stated, why do the authors not propose an alternative, sustainable mechanism for it?
Adding this: Not that the current situation is indefinitely sustainable, but the current requirement sufficiently preserves and provides state history. The burden put on network users is heavy and growing, but there needs a realistic plan for how to maintain this widely-used aspect of the Ethereum network.

Author: gcolvin
Date: 2021-11-15T05:49:47.734Z

Yes.  The Ethereum blockchain is, fundamentally, an immutable record of transactions – value transfers and  valid computations. It seems to me that there should be some standard protocol for that, however the history is stored.

Author: mkalinin
Date: 2021-11-15T09:37:33.419Z

djrtwo:

Making this a MUST NOT serve makes this a harder EIP to implement because it will require preparing dapps (especially receipts) and users for this breaking change


Additionally, if we use MUST NOT then the peer that breaks this requirement SHOULD be disconnected and penalised.

Author: tjayrush
Date: 2021-11-24T17:27:03.782Z

I agree completely that “preserving this widely-used aspect of the Ethereum network” is of utmost importance. But why does it have to be a capability that the node maintains?
It seems to me that maintaining this ability in the node is exactly the problem. If the data is immutable, the entire state and the entire history of the chain can be written once to some content-addressable store (such as IPFS) and as long as someone preserves that data, anyone can get it. AN Ethereum node would not even have to be involved. All one would need is the hash of where the immutable data is stored.
Fresh data can be written as an ‘addendum’, so there would have to be some sort of published manifest of the original hash and the periodic ongoing hashes. I would argue that the hash of the manifest should be part of the chain, but, short of that, the community would have to maintain it (perhaps by publishing the hash to a smart contract).
My point is that because the data is immutable, and because we have content-address storage to store it in, there’s literally no need to continue to provide the ability to regenerate this data from genesis. The only outcome of regenerating from genesis would be to arrive at the same IPFS hash as you already have.
On top of that, there’s no reason the clients have to maintain this capability, and the entire purpose of this EIP is to remove that requirement. This might possibly open a whole new area of innovation related to providing access to this historical data – which I think would allow for amazingly more interesting dApps than we currently have (because of the need for a node to get to it).
Furthermore, if the historical data is chunked into manageable pieces, and it was properly indexed by chunk (with a bloom filter in front of each chunk) each individual user could easily download and pin only that portion of the database that they are interested in. Thereby distributing this historical data throughout the community as a natural by-product of using it. (See TrueBlocks).

Author: anett
Date: 2021-12-01T12:33:50.581Z

I agree that people are uploading lot of stuff on blockchain especially with the rise of NFTs but also not optimized token contracts which are causing state bloat. But did anyone think about other examples and use cases of blockchain? What if people uploaded their important documents like birth certificates on blockchain as the mission of blockchain is ledger which stores information on-chain forever. Suddenly those people won’t be able to access their documents because some devs thought it’s a good idea to delete blockchain state after some time… Another great example is NFTs especially NFTs that were made before ERC-721 ie 2017 and older like CryptoPunks. Those will be gone for ever.
From developer perspective, I’m sure that there are many data that are not important and doesn’t need to be stored.
Probably better idea would be to store data on full nodes and have light nodes or think about different ideas how to make infrastructure the most efficient without having to delete and loose data.
Don’t get me wrong, I’m just trying to think realistically from non-core-dev perspective and I’m against this EIP.

Author: MicahZoltu
Date: 2021-12-02T04:34:16.773Z

Ethereum was never designed to be a permanent data storage system.  Something like FileCoin is much better suited for long term data storage, and they have incentives built into the protocol to ensure that the cost of long term storage is paid for by those seeking it.
Also, this EIP removes history but not state.  State expiry is also an active area of research, but out of scope for this thread.

Author: niceblue
Date: 2021-12-07T18:37:42.887Z

I have been an Ethereum watcher and dapp developer for years, and have admired all the EIPs that have come through. However, this EIP is deeply troubling. I think this would be an extremely negative EIP to implement. Here’s why:
Ethereum was touted as the system to build “unstoppable” apps over the years, I loved it. With this EIP, these “unstoppable” apps will simply, well, stop (at least, their UXs/UIs will). It forces substantial and necessary adoption of some a.n.other (unknown and uncertain) protocol entirely outside of the Ethereum system. Pulling the “promise” of data persistence on old apps will be disastrous for the long term reputation of dapp development on Ethereum.
For those who say Ethereum was never meant to store data permanently, that is simply not true. It was! It’s specced that way and therefore used that way. And with this EIP, it will no longer. This is a truly fundamental change to (and destruction of) the Ethereum value proposition. Providing canonical transaction history tightly coupled with canonical transaction generation is CRITICAL to Ethereum’s value proposition. Offloading this entirely outside of Etheruem’s control gives away (and destroys) the future utility of Ether. Why? Because the whole point of canonical transaction generation is that you also have canonical transaction history.
With EIP 4444, it is possible to lose entire chunks of past transaction history. Forever. As in gone. No one knows who sent (or did) what to who or when.
There’s a reason why JP Morgan, HSBC, etc, any of these long storied banks are still around. It’s because you can rely on them having, somewhere inside their big walled offices, a transaction history going back over 100 years. This builds TRUST (yes, centralized trust). But that’s why (other) people come back to them (even if blockchain ppl don’t). The old history may be hand written in log books, sure not convenient, but it’s there.
Now imagine Ethereum was just such an organisation. You go to Ethereum in 12 years time and you ask (in code), what transaction happened on this account 10 years ago? The reply: oh go to Graph/Bittorrent/IPFS/a.n. other, we don’t keep that. You try numerous of these organisations, by some stroke of bad luck, they messed up on your particular EOA/contract (or their tech dies), and it’s gone. Would you trust the Ethereum system, simply because it moved to cool stateless consensus, and therefore decided they didn’t need to include anything as boring as past history anymore? I wouldn’t.
What this EIP fails to realise is that the value of canonical decentralised transactions in almost all real world use cases isn’t canonical decentralised  transactions, but canonical history of decentralised transactions. The blockchain that does this will win. Ethereum does both today, but with this EIP, it won’t any more. That’s what I would call the broken promise of Ethereum if this EIP happens, and without a simultaneous EIP that ensures canonical transaction history at the Ethereum protocol level. Perhaps some compromise can be made on time horizon. “Forever” is not enforceable, but say 20 years (for example) is, and good enough for most real world use cases (but even then not good enough for academic records, for example). Remember, the well known banks most of us also use can keep (centralised) canonical records for over a century, and universities can keep records for multiple centuries.

Author: Lilaaffe
Date: 2021-12-07T21:15:14.421Z

Arweave fixes your problem. Period.

Author: niceblue
Date: 2021-12-07T21:22:03.175Z

It is a.n.other protocol. canonical transaction history is absolutely critical to Ethereum. Why entrust that outside? If Arweave figures out Smartweave properly, let’s see where Ether ends up. More power to Arweave

Author: aronchick
Date: 2022-02-10T20:24:16.614Z

Hi! I co-lead ResDev for Protocol Labs, we’re more than happy to help make available prove-ably immutable history of ETH forever on Filecoin & IPFS, and would not require any protocol changes. Please don’t hesitate to let me know if anyone would like help doing this!
Thanks!

Author: poojaranjan
Date: 2022-03-21T14:12:58.641Z

PEEPanEIP-4444: Bound Historical Data in Execution Clients with @ralexstokes ️

Author: Neurone
Date: 2022-06-08T02:33:20.589Z

My two cents. I disagree with this EIP, both in general and about the contents.
It doesn’t add any real advantage to clients, I can already prune txs if I’d like to.
It removes a feature without offering any solution; everything is “out-of-scope” or “just use other p2p network for that”.
We can already use centralized servers or CANs (IPFS, Swarm, etc.) to store data, even Ethereum related (i.e., TrueBlocks indexes), but having that mandatory is something that makes no sense for me.
Ethereum is a p2p network, using the network to share data and reach consensus should be what it’s used for. And all the Ethereum ecosystem should be consistent without relying on others p2p networks.
Furthermore, storage is a commodity by now, and it will be even more in the future. Four hundred GB (and counting) of old data is really nothing to fear of, I can buy an HDD to store 4 TB of data for $30, and it will last for another 20 years of historical txs. Storage technologies grow and improve really faster than Ethereum’s txs.
In my opinion, the right path is continuing to improve light client, spreading more nodes among constrained devices and hardware.
The wrong path is transforming forcibly all full nodes into light clients.

Author: ytrezq
Date: 2022-09-03T23:29:40.962Z

Hi, your are saying this, but TheGraph just censored Tornado Cash.
The underlying problem is things like databased shared over ipfs or Google Bigquery wouldn’t allows us to have a realtime service as it would take a day to download updates. We also need very old data in order to let a user withdraw funds deposited several years ago.
This isn’t that much a problem currently as this just mean we will need our own home (not cloud) hosted node with parity_tracing, along OpenEthereum Fatdb like for getting smart contract storage range at past blocks. Such thing is possible because full anciant data is broadcasted and updated in real time but show why it is important to have node/rpc be able to broadcast full chain history over the p2p network.
If third party service are made required, then well, Ethereum, will be decentralized at the currency level like Bitcoin while Dapps like casinos or yield farming, will be fully permissionned have to register with authorities by paying an army of lawayers in order to be allowed to run which means, DeFi won’t be that much different than Fintechs on the tradionnal banking system which rent their computing hardware.
I think keeping history on the p2p definitely worth the reduced transactions per seconds outputs or if we decide to behave like swift or MasterCard or Visa for being able to run as fast as them.

Author: ytrezq
Date: 2022-09-03T23:34:33.519Z

No, their point is large database reduce the transaction per second speed. But I think this is far fetched as Visa and Mastercard are in my country required to record data of all transactions of the past 10 years for law enforcement, and this doesn’t prevent them to run.

Author: ytrezq
Date: 2022-09-03T23:41:51.463Z

Would this be in real time for each blocks like the current p2p network, or would there be daily updates pushes ?

Author: ytrezq
Date: 2022-09-03T23:43:31.634Z

Hi, your are saying this, but TheGraph just censored Tornado Cash.
The underlying problem is things like databased shared over ipfs or Google Bigquery wouldn’t allows us to have a realtime service as it would take a day to download updates. We also need very old data in order to let a user withdraw funds deposited several years ago.
This isn’t that much a problem currently as this just mean we will need our own home (not cloud) hosted node with parity_tracing, along OpenEthereum Fatdb like for getting smart contract storage range at past blocks. Such thing is possible because full anciant data is broadcasted and updated in real time but show why it is important to have node/rpc be able to broadcast full chain history over the p2p network.
If third party service are made required, then well, Ethereum, will be decentralized at the currency level like Bitcoin while Dapps like casinos or yield farming, will be fully permissionned have to register with authorities by paying an army of lawayers in order to be allowed to run which means, DeFi won’t be that much different than Fintechs on the tradionnal banking system which rent their computing hardware.
I think keeping history on the p2p definitely worth the reduced transactions per seconds outputs or if we decide to behave like swift or MasterCard or Visa for being able to run as fast as them.

Author: ytrezq
Date: 2022-09-03T23:46:19.459Z

I propose something which is done by the Cloud industry which bill money to keep data. Deposit Ethers on smart contracts : at each blocks, a very tiny fee is removed. When the smart contract values drops to 0, it’s code/storage is SUICIDED and it’s relevant transactions deleted from history.
That way : what is needed is kept while what is forgotten is destroyed. This is also means more efficient than the proposal since stuff can be destroyed before 1 year.
Please also notice that destroying what is unused is also how the human memory works and things always fit in the size of a human skull.

Author: gcolvin
Date: 2022-09-04T03:54:28.223Z

ytrezq Deposit Ethers on smart contracts : at each blocks, a very tiny fee is removed

Or maybe, stake the ether and use the proceeds to pay for the data storage in perpetuity.

Author: chaals
Date: 2023-06-19T12:37:03.672Z

Would this imply setting the decay fee in relation to the staking return? (How does it relate to the magical 32 Eth stake number?)

Author: chaals
Date: 2023-06-19T12:38:41.399Z

gcolvin:


there is a risk that more dapps will rely on centralized services for acquiring historical data.

Yes. And I don’t think mitigating these risks is “out-of-scope for this proposal”.


Ditto. I think it’s important to address it explicitly as part of any proposal, beyond “some other tool will solve this”. For largely the same reasons advanced for requiring processing clients to drop data after a given time.

Author: tonytony32
Date: 2023-11-21T11:00:25.273Z

There is a way to preserve historical state data, it can be based on economic incentives, and also in an altruistic way simultaneously (!), should it happen to be the popular solution.
So pruning data according to this EIP does not necessarily mean that it is lost, but only that the process of verifiability is changed elsewhere.

Author: tonytony32
Date: 2023-11-21T11:12:01.844Z

If the data continues to grow, the rewards per data become smaller with eth limited supply, and historical data is at a disadvantage.
Keep data alive for a non-defined period of time is a promise. You need specific offchain systems to keep it alive, and those systems need to work on an altruistic and incetivized fashion.

Author: qizhou
Date: 2023-12-01T07:01:20.312Z

The rewards per data in ETH being smaller may not imply that the historical data is at risk (e.g., under replicated) - if more ETH is staked to pay toward storage, then the price goes higher.  Further, the storage cost also decreases constantly over time.

Author: tonytony32
Date: 2023-12-01T17:43:43.801Z

qizhou:

if more ETH is staked to pay toward storage, then the price goes higher.


The assumption that ETH raises its price mainly because of storage costs is risky. Consensus and data storage are two (very) different tasks, markets and economies that uncorrelated contributes to the long-term good of the project. Unless the motivation is to grab the potential profits of the storage market, because the consensus market is not profitable enough.
Also, storage costs decreasing its value over time is one of my favourite assumptions. Moore’s Law always worked until it didn’t.

Author: yorickdowne
Date: 2023-12-02T15:32:43.627Z

I’d like to see this land in early 2025, which means a solid design needs to be in place by early-mid 2024. Is that even remotely feasible?
Solo stakers on 2TB disks have until maybe late 2025 - optimistically sometime 2026 - before they run out of space, and early-mid 2025 before they get nervous. It’d be amazing if they weren’t forced to upgrade to 4TB.

Author: gcolvin
Date: 2024-02-16T06:45:14.272Z

4TB SSDs start at $280.
https://www.newegg.com/p/pl?N=100011693%20600545605&Order=1

Author: peersky
Date: 2024-02-22T17:47:52.752Z

My suggestion is to change wording
Clients SHOULD NOT serve headers <...> -> Clients MAY NOT serve headers <...>

Rationale:
MAY formulation is less assertive on node behaviour while does not effect of this EIP.
Ability to use uniform access interfaces such as eth eth_getLogs and eth_call to read both historical, latest and global index data shown great from developer and user experience allowing us to build non permissioned database of records. This is a great functionality which we should not lose in attempt to optimise.
While nodes MAY drop the historical data, there are reasons to assume that not everyone would be willing to do so because (i) older data has intrinsic value (ii) other nodes pruning data are effectively reducing supply of such (iii)  data availability of nodes can be ultimately solved by having a tightly and correctly architected financial incentives model. Some discussion in that regard in research forum: Economics for P2P (RPC) data marketplace - Economics - Ethereum Research
(iv) Users who want to maintain full or extended historic experience can negotiate with the Node availability beyond the minimal agreed on.  (v) the epoch timeframe is a subjective matter and ideally should be considered to be as short as it can be to maintain consensus and to keep consensus participation storage requirements low while keeping data availability as optional feature

Author: nuliknol
Date: 2025-03-17T16:32:53.915Z

peersky:

Ability to use uniform access interfaces such as eth eth_getLogs and eth_call to read both historical, latest and global index data shown great from developer and user experience allowing us to build non permissioned database of records. This is a great functionality which we should not lose in attempt to optimise.


This is just a symptom of a larger problem: computing parallelism. Ethereum is a distributed computer, so it needs to have an architecture of a distributed computer, not a Von Neumann architecture just replicated in parallel. If you give a parallel task to a bunch of computers designed to run serial processes (even if they have some communication between them) you are going to have serious issues with tasks that share state.  If you want to fly you have to design a plane, not a bycicle with wings. Right now you might fix the issue by deleting some not-so-used data from the database (a temporary fix), but the core problem (i.e. the EVM is not natively paralell) will be popping up in some other stuff and eventually Ethereum will scale just proportional to the capacity current hardware.
To design a truly distributed computer (which won’t have issues with scaling), you need to use a computing model designed for parallelism natively. There is a page in wikipedia called “Model of computation” and if you scroll down, there is a section called “Concurrent models”. What the Ethereum team has to do, is to rewrite the whole EVM using some of the models listed in that section. If somebody would ask me , I would implement Kahn’s Processing Network, it is very simple and very reliable model.
Obviously this is a huge change (architectural change) but this is the only right solution. Otherwise, you are going to be patching , patching, patching and keep patching stuff forever and you will never achieve scaling. The data will be growing like popcorn , and  you will have to keep buying and buying SSD disk. But if you implement truly parallel computing architecture, every node will have a portion of the data and it could scale to hundred thousand of transactions per second efortlessly.

Author: TimDaub
Date: 2025-03-18T16:54:47.270Z

I think your argument is a lot of hand-wavying for removing an invariant that has been in Ethereum for roughly a decade. We app developers depend on these interfaces to exist and I don’t want them to be gone.
You joined this forum a day ago. I don’t think you have Ethereum’s best interests in mind and your argument is strange and convoluted at best for arguing for such a narrow change to the protocol

Author: TimDaub
Date: 2025-03-18T16:58:15.475Z

MicahZoltu:

Ethereum was never designed to be a permanent data storage system. Something like FileCoin is much better suited for long term data storage, and they have incentives built into the protocol to ensure that the cost of long term storage is paid for by those seeking it.


You have no authority to say that Ethereum wasn’t designed for permanent data storage. How can you know what all designers of Ethereum were thinking? That invariant has existed for 10 years and app developers have taken advantage. I don’t know why you even bring up Filecoin, it clearly isn’t useful at all. Should rollups also have used Filecoin over Ethereum calldata? Strange argument

Author: tonytony32
Date: 2025-03-18T18:00:25.396Z

Ethereum was not designed for permanent data storage, from the early days of the World Computer (Ethereum, Swarm, Whisper) it was clear that Ethereum blockchain was not supposed to store such data.
Blobs do not provide true on-chain storage compared to calldata, as blob data is only available for 18 days. This necessitates external archiving solutions to preserve blob data. Those solutions require permissionless upload and download, information neutrality, and plausible deniability. There is only a handful of storage solutions who provide those.

Author: tonytony32
Date: 2025-03-18T18:03:06.810Z

Filecoin falls short in addressing plausible deniability challenges, as its nodes are fully aware of the data they store, making it easier to target and track operators. Additionally, the network’s high hardware requirements limit node distribution, hindering its ability to scale in a truly decentralized manner.

Author: tamas6
Date: 2025-03-19T18:21:23.475Z

It’s still relevant despite its flaws, I wonder what would they say about these

Author: peersky
Date: 2025-03-20T07:20:47.368Z

IMHO best way of solving this is accepting that standard should say MAY NOT store.
getLogs interface has nothing wrong with it from client (dApp dev) perspective.
In fact, as dApp developer, I would prefer just having an RPC endpoint that I can arrange with that it will return low-latency event data for pre-arranged contracts/events trough getLogs instead of having to manage separate indexer.