Title: ERC_7512__Onchain_Audit_Representation
Category: ERCs

Author: rmeissner
Date: 2023-09-05T20:40:40.145Z

Discussion for Add EIP: Onchain Representation for Audits by rmeissner · Pull Request #7652 · ethereum/EIPs · GitHub
The purpose of the ERC is to propose a standard to represent audits onchain. Any discussion related to this can be facilitated here.

Author: Dexaran
Date: 2023-09-06T15:07:37.860Z

The idea of having on-chain audits is useful. However the implementation proposed in this ERC is overcomplicated significantly.
The goal of having on-chain audit registry is to allow contracts to verify if a contract is secure or nobody ever reviewed it (the fact that it was reviewed and assigned a “secure” label does not automatically mean that it is really secure however).
Taking this into account lets review audit properties:

Auditor - Do we need it at all?

name - well, there are better ways of recognizing auditors. For example just keep a track of auditors addresses in an open registry and let contracts match the issuer of the on-chain report against one of the auditors publicly available addresses.
uri - it doesn’t tell contract anything. A human can follow the link and read whatever is written there but not the contract.
authors - it doesn’t tell contract anything.


Audit

auditor - ok
contract - ok, chainID and address are useful.
issuedAt - may be
ercs - why is it important at all? I don’t think that a contract may want to conduct this type of standard recognition via audit reports instead of ERC-165 for example. Also, USDT is not compliant with ERC-20 spec, do you expect some auditor like OpenZeppelin or CertiK to say “USDT is not a ERC-20 token”?
auditHash - ok
auditUri - ok



Auditor Verification
Why would someone need such a complex system of auditor verification if that same task can be accomplished in a much easier way - just let auditors submit audit reports from a publicly known address and match addresses to the name of the auditing company.
Like 0x111111 is OpenZeppelin
0x222222 is CertiK
0x333333 is Callisto Audits
etc.
You don’t need such a complex structure and the whole load of processes for signing / verifying if it can be done in a way that would allow even a technically inexperienced user to verify who is the auditor.
The current ERC does not have any mentions of the findings of an audits.
This is the most crucial part honestly. There can be multiple audit reports for one contract and if at least one indicates a problem with the contract - it is more important than all other reports that do not indicate any problems with this exact contract.
If you have 3 auditors who have reviewed one contract, two of them found nothing and the third found a critical vulnerability - it’s much more logical to indicate that “the contract might have a critical vulnerability” rather than resort to an assumption “if there is at least one audit report that doesn’t indicate any problems then the contract is most likely safe”.
I think that a system that does not allow for findings specification and independent audits submissions for multiple different auditors - will not work or even worse it will deceive users into thinking that some contract is secure while in fact there are problems with it.

Author: Dexaran
Date: 2023-09-06T15:32:25.020Z

I would propose an alternative structure for on-chain audits.
Create a “registry” contract that will allow anyone (or a select group of addresses) to issue an “audit report” for another address. This “audit report” should act as a Soulbound Token with configurable properties.
I have proposed this type of NFTs in the past (it is easy to turn into SBT by simply removing transferring features):



ERC CallistoNFT standard Tokens


    ERC - CallistoNFT 
Preamble
EIP: <to be assigned>
Title: Non-fungible Container Token Standard
Author: Dexaran <dexaran@ethereumclassic.org / dexaran820@gmail.com>
Type: Standard
Category: ERC
Status: Draft
Created: 2022-22-02

Simple Summary 
An interface for non-fungible tokens and minor behavior logic standardisation. 
Abstract 
This standard allows for the implementation of a standard API for non-fungible tokens (henceforth referred to as “NFTs”) within smart contracts. This standard introdu…
  

The SBT must contain the following properties:

Issuer - the address of the auditor or an auditing company
Critical findings: number
High severity findings: number
Medium severity findings: number
Low severity findings: number
Audit hash
Audit report link
Chain ID

It is possible to leave severity assignment to auditors I think.
In this way it would be possible to ask one contract (registry) and get a list of audits if there are multiple. At the same time if there is already one audit report that says “everything is fine with the contract” but in fact the contract has security problems - there will be a way for other auditors to submit reports that point out security problems of the contract.

Author: rmeissner
Date: 2023-09-06T19:05:19.642Z

Dexaran:

Create a “registry” contract that will allow anyone


The purpose of the ERC is not to define the registry, but rather a format in which audits can be represented on chain.



 Dexaran:

type of standard recognition via audit reports instead of ERC-165


The security implications are quite different if a contract can self proclaim what they support and if you get an external party “verify” that they follow a standard. Also not all ERCs are interface standards.



 Dexaran:

do you expect some auditor like OpenZeppelin or CertiK to say “USDT is not a ERC-20 token”


Yes that would be the correct way and is actually critical for implementations building on top of such standard. Some tokens that claim to be ERC-20 compatible, but because of different behavior of their transfer function require contracts building on top to implement special handling just for these contracts.



 Dexaran:

Auditor - Do we need it at all?


That is a good question and it should be considered. The authors fields was meant to provide a indicator which auditor was actually auditing the contracts (as there are differences within audit companies), but there are alternative ways to represent this.



 Dexaran:

The current ERC does not have any mentions of the findings of an audits.


This has been discussed with some auditors before and it would be indeed very helpful, but there are some challenges on how to align on the definition for the severity. To not make the ERC more complicated leaving it out would be a first step.
I agree that leaving this classification up to the auditors is also a solution. I would rely on the impact of the auditors for this.



 Dexaran:

just let auditors submit audit reports from a publicly known address and match addresses to the name of the auditing company.


As the ERC aims to only create a representation of an audit and not define how it is handled on chain, the definition of the verification scheme make it possible to use them independently of specific chain allowing a verification of the representation offchain.



 Dexaran:


Medium severity findings: number
Low severity findings: number



How useful are these? Normally medium and low severity findings are not security critical and the usage of the auditors of these states might differ.



 Dexaran:

This “audit report” should act as a Soulbound Token with configurable properties.


In my opinion this would be an application of an onchain audit representation. I.e. auditors would create the representation, sign it and then anyone could submit it to create a SBT based on it that can be used onchain.

Author: PatrickAlphaC
Date: 2023-09-11T04:48:54.266Z

I’ve thought a decent bit about this, and every time, I conclude: “What would this accomplish?”
At the end of the day, audits (henceforth, “security reviews”) are for the protocols, not the community. Having a security review hosted on-chain suggests, “You can trust this code because there is a security review here.”
However, the security review was paid for, scoped by, and conducted by the protocol. It’s meant for the protocol. In the context of feedback for the protocol.
At the same time, if a project wanted to show off its audits for people to learn and grow from, that seems fine. But I’m just nervous that this is a standard for projects to say “Look, we are safe. You can see our on-chain security review” when the security review wasn’t done with the community in mind.
My main question is, “Why would we even care to have this standard?”

Author: lukasschor
Date: 2023-09-15T01:01:55.637Z

Not sure security reviews are in practice used „for the protocol“. Pretty much every landing page features security reviews as a signal to the community that there has been some measures being taken and that there is a baseline of security focus in the project. Yet the connection of these PDFs on a landing page and the actual smart contract code is very loose. So this ERC is at least an incremental improvement by bringing security reviews closer to the actual code they were covering.
Ideally it‘s also going to be the basis of much more significant improvements such as reputation systems being built on top of this standard or new incentive mechanisms where it‘s actually not the protocol team scoping and paying for the security review.

Author: chendatony31
Date: 2023-09-19T13:58:32.729Z

I have discussed this issue with some security companies before, and the solution is similar to @Dexaran’s. Security companies can issue audit SBT to their audited contracts. which means that each security company or person has their own corresponding SBT contract. We can verify whether the contract belongs to this company by using the url metadata of the contract and the /.well-know/contract.json file of the official company domain name
As a wallet (or an explorer), you can enumerate these contracts to display which people/companies have audited the contracts that your user is interacting with and whether they are relatively secure.

Author: thezluf
Date: 2023-09-20T05:35:54.410Z

As @rmeissner mentioned, this ERC focuses on standardizing what auditors should sign, rather than defining the registry. The goal is to ensure consistent verification across the ecosystem.

Author: blackbigswan
Date: 2023-09-20T10:38:59.256Z

PatrickAlphaC:

I’ve thought a decent bit about this, and every time, I conclude: “What would this accomplish?”


I totally get where you’re coming from as I share same sentiment. That said, there’s a solid reason to standardize the data format for posting audits on-chain, and it aligns with the SEC’s recent thinking. While the SEC is primarily concerned with reliable financial information, you could argue that audits indirectly fall under this umbrella. Take a look at this document:
F5bvlrbWgAEVC0m630×900 169 KB
Notice: […] in practice the source code implemented on blockchains are in machine-readable format, may not conform to public descriptions of the code
So, creating a standardized, public, and immutable registry for audits of a given protocol makes sense. But a standard is only as good as the infrastructure built around it (gh2source2bytecode compare, documentation2implementation compare, does audit audit actual bytecode or github etc.). That’s mostly going to be off-chain though. Personally I call it “GenslerProtocol”. So yeah, this EIP does have value, especially since there’s at least one clear use case that would benefit from a robust data format.

Author: lefuncq
Date: 2023-09-20T12:40:51.412Z

Many people suggest going with a registry instead of the current proposed implementation. That’s precisely what we’ve been working on for the past couple of years at Trustblock.
Compared to what we’ve built, the current proposal has its strengths and weaknesses.
The main problems I see so far:

Expect protocols to add this to their codebase, which is far from trivial
Synchronicity? Auditors have to submit audits upon each request, which is very limiting in terms of usage
Handling upgrades
Findings are missing but still valuable information for protocols to act upon.

Author: rmeissner
Date: 2023-09-20T13:50:07.067Z

lefuncq:

Expect protocols to add this to their codebase, which is far from trivial


I would be interested how this problem is differently handled by the 2 approaches.
The current ERC proposes a way that could be used in any registry and without any onchain interaction necessary.
The challenge we see with registries is that it is still a very centralized approach (which has its benefits). Therefore the ERC aims to create a building block to make audit information available and verifiable onchain, not to build such registries itself.


Synchronicity? Auditors have to submit audits upon each request, which is very limiting in terms of usage


Overhead wise the ERC aims to keep it minimal. Many auditors are already signing their audits as part of the process and the additional overhead to create and sign the onchain representation should be quite low (would be interesting where you see this becoming a blocker).
When it comes to publishing the representation and pushing it onchain, then this can be done by anyone. Auditors could upload this side-by-side with their audit files (similar how checksums are published) and anyone could make use of these. This way the overhead to interact with any 3rd party is not required.

Author: Angler
Date: 2023-09-20T14:24:21.961Z

lefuncq:

Many people suggest going with a registry instead of the current proposed implementation.


The benefit of the representation over a registry IMO is that we can have many different registries that utilize the audit representation. This has multiple benefits.

No single entity is in control over the audit representations
The user can select from different registries one that matches his needs. E.g. the user might select a registry for smart contracts that are audited only by specific auditors.
The registry can combine the representation with other criteria

Author: thebensams
Date: 2023-09-20T15:11:47.352Z

How does this work for upgradable contracts?

Author: lefuncq
Date: 2023-09-20T15:48:07.454Z

I would be interested how this problem is differently handled by the 2 approaches.

In our case, we have one registry per chain, and these registries contain auditors’ wallet addresses that are allowed to publish audits. We whitelist wallets for now, but the best would be to have governance voting auditors on and off.

Therefore the ERC aims to create a building block to make audit information available and verifiable onchain, not to build such registries itself

I agree with you it’s clearly out of scope for this EIP, but I think it’d still be interesting to actually work together on an audit registry ERC that we could also integrate on Trustblock. The main reason for a registry is that it will create many more use cases for audits on-chain, according to our research through these couple of years, than if the verification has to be made synchronously with the auditor. With the data being directly accessible, protocols can be used for various composability purposes. Moreover, storing on-chain also guarantees the immutability of audits.

The challenge we see with registries is that it is still a very centralized approach (which has its benefits).

Audits are delivered by a single entity, so they will always be centralized by nature, right? However, if you meant that registries would be controlled by a single entity (like us for example), then I think it depends on the registry functioning, for example in our case only auditors can publish audits so we don’t control the data.
Overall

Per my knowledge and seeing how many auditors work, I have not seen many auditors sign their audits (not sure exactly what you meant there) nor keep track of contract addresses they have audited. So, if they want to support this, they must change how they do their business. I believe specifying exactly which contract addresses audited is the right thing, but that isn’t the reality right now

If they want to upload old reports, they must find the exact addresses they audited. When they do new ones, they have to ask for deployed contract addresses after an audit, verify them, and add them to the report
Preparing audit representation and signing. Depending on the tooling used, it can be automated or simplified. Still, it is an additional step
Display their wallet addresses or public keys so that others can verify audits
So they should have the right incentive behind it. I know it is not exactly the scope of this EIP, but that’s the practical aspect of it.


If protocols want to support this and accept only, let’s say, tokens with audits into their system, they have to:

Pre-select auditors they trust. It means that they have to get their wallet addresses/public keys to verify the signature belongs to the auditor they trust
Implement verifier either on-chain or off-chain

Author: lefuncq
Date: 2023-09-20T16:05:31.859Z

I agree with you. The way you would implement registries is very interesting and different from the way we implemented them in our system so far.
In our case, we have one registry per chain, and protocols can preselect which authorized auditors they want to get audits’ data from.
Another significant advantage in favor of registries is that if the audits were to be stored on-chain, we could make them immutable, which is super helpful to balance trust relations further between users, auditors & protocols.

Author: srw
Date: 2023-09-20T17:10:41.857Z

We just published our opinion at ERC-7512: A Solution to the Centralization of Security Audits Data?.
The TLDR is that solves the issue with sites such as CoinMarketCap, Etherscan, CoinGecko, etc because they don’t publish accurate data about auditors even when you follow all their forms. Hence, they are a point of centralization that filters real information. On the other hand, we think that this problem should not be limited to audits and a way to add metadata in general could be interesting.

Author: Dexaran
Date: 2023-09-21T17:02:01.975Z

The main problem with this ERC is that it only allows for “positive” audits that say the contract is secure.
I own a security auditing company (https://audits.callisto.network/), I’m an auditor myself and in many cases it is really important to say “Warning! This contract is not secure” while others are saying it is secure.
It is not secure to pretend “if at least one company said that the contract is secure = we label it as secure” .
Also audit records must not be immutable by any means. Imagine a contract was audited, then a vulnerability is discovered but there is already a signed audit report that states that the contract is fine. Anything can happen or be discovered AFTER the audit report so this needs to be accounted.

Author: JakubLipinski
Date: 2023-09-22T19:38:02.838Z

Just for the sake of reference: I tried (and failed) to create an on-chain registry of smart contract audits in 2008 as a part of SolidStamp service. Some of my learnings and ideas are still available at:
https://medium.com/@SolidStamp/solidstamp-putting-skin-in-the-blockchain-game-6c061cd33c6
https://medium.com/@SolidStamp/solidstamp-a-flight-recorder-for-the-ethereum-ecosystem-7a6247947733
Good luck this time.

Author: Hans
Date: 2023-09-23T11:19:28.714Z

After all, the proposal seems to be to see if a protocol was audited.
We can achieve that more simply, in fact:

Auditors (personal or firm, whatever) create NFT to represent the audit information. Additional information like the issued date and a report link can be included in ERC721Metadata.
Maintain an address registry of auditors on-chain.

What else do we need?

Author: rmeissner
Date: 2023-09-23T11:41:33.040Z

You need an agreement of the auditors on this metadata and how to issue that nft.
What you described is not the purpose of the nft the purpose is to create a basis that allows you to create such nfts. I.e. could you say that you have a nft contract where you can submit an erc-7512 object which will mint the nft with some basic information (like auditor). This then can be used in different protocol to perform security check (or other logic based on the audits).

Author: rube-de
Date: 2023-09-25T09:38:24.849Z

Shouldn’t this support also erc-1271? As audit company it would make sense to have a smart account, instead of an EOA.

Author: rmeissner
Date: 2023-09-25T09:40:12.812Z

This is considered and mentioned in the ERC under “SignatureTypes” (see here)

Author: eawosika
Date: 2023-12-15T22:17:09.481Z

I recently wrote a deep dive on ERC-7512 for anyone interested in a comprehensive overview of the cases for and against the proposal (thanks to everyone who contributed to the discussion thread–found many of the insights useful during my research): ERC-7512: Bolstering Smart Contract Security With Onchain Audits

Author: RexShinka
Date: 2024-02-14T14:55:55.018Z

Another possible initial step, rather than doing audits (which are complex) do DeFiSafety protocol reviews.  These are fixed format, have a pass/fail and are designed for reading by the general public.  This could be implemented quickly with audits brought in later.  An example report is at : Liquity - detailed report | DeFiSafety

Author: elliot
Date: 2024-02-26T06:13:35.542Z

While we’re here, it seems like we might as well provide the bytecode hash as part of the onchain representation so that users of these systems can verify that the deployed code matches the code that was audited.
This could be represented as an array of structs containing the address of the contract and its corresponding bytecode hash.

Author: eawosika
Date: 2024-04-01T04:24:27.903Z

2077 Research’s analysis of ERC-7512 suggested extending ERC-7512 to aggregating information about other security measures (e.g., bug bounties and formal verification) vs. providing only audit data to investors doing due diligence on protocols.
The article even mentions DeFi Safety’s protocol reviews as an example of how this idea can be implemented, with the caveat that the product has switched to a subscription model vs. as being publicly accessible (which was the case circa 2022 IIRC). Here’s the relevant quote from the article’s conclusion:

ERC-7512 moves us a step closer to building trust in on-chain applications and may inspire more efforts to standardize other aspects of the security review process. For instance, given projects now increasingly adopt a “defense-in-depth” approach to protocol security—bug bounties, formal verification, audit contests, incident monitoring, and more—a system that aggregates (verifiable) information about a project’s various security measures in the same location (as opposed to this information being fragmented across multiple websites and dashboards) would do wonders for investors, users, business development (BD) teams  conducting due diligence for DeFi protocols.  (DeFi Safety used to have a similar service but recently switched to a revenue model.).

Author: peersky
Date: 2024-06-03T21:33:11.479Z

@cylon56 I think this ERC should rely on  address.codehash instead and/or on smart-contract based signatures (dependency on ERC-1271).
That would be much more generic solution, allowing re-deployment of code without requiring audited signatures.
Rationale
1. Byte code signing could simplify on-chain verification bytecode security:
Contract deployer does not need to supply any audit data at all. Existing contracts can be easily incorporated to this ecosystem by signing their code.
All signatures for all auditors can be stored in same registry that simply maps byte code hash to signer addresses mapping.
2. Verified state management is better done off trough Factory contract
While use of address.codehash obviously limits auditor responsibility to the state only, It also breaks down audit to pieces, and leaves state verification to be a mission on it’s own.
For example  in diagram below we can see that stateful audit can be implemented in form of Trusted Factory. Such design would require less Bridge Operator actions and hence eventually would likely to surpass current form of this ERC.
Screenshot 2024-06-03 at 23.19.111492×1366 140 KB
Even already deployed code contracts could be enlisted by auditors manually adding signature for such contracts or having their own logic, paving way for ERC1271.
NB: in diagram above I assume calldata signed by auditor for factory can be factory-specific instructions, not a particular constructor arguments.
3. More secure
Compilers are the point of failure, signing compiled byte code is intrinsically more solid
4. Revokable
Since signatures are issued for a bytecode, not a contract, audits registry becomes more powerful as in case of disclosed vulnerability, auditors may just use nonce like system to invalidate their signatures.

Author: peersky
Date: 2024-06-04T21:05:05.819Z

Also, the trusted source makes total sense to implement via EAS

Author: peersky
Date: 2024-07-16T17:27:12.312Z

To be not a empty-worded, here is my reference implementation for how I envision this working with byte-code level:

github.com




GitHub - peeramid-labs/eds: Ethereum Distribution System 
Ethereum Distribution System