Title: Jello_Paper_as_Canonical_EVM_Spec
Category: Magicians

Author: expede
Date: 2019-01-10T21:43:48.397Z

Hello all 
During Devcon 4, there was some discussion about moving the canonical EVM spec to the Jello Paper. I think that this is a fantastic idea. Very few people seem happy with the Yellow Paper as written, and the ambiguous state of maintenance. I am partway through an EVM implementation myself, and have found the Jello Paper to be far superior to the Yellow and Beige Papers. Some of the reasons include:

Much clearer
Fewer edge cases
Took the time to formalize the semantics
Executable (hooray for K!)
Can generate various EVMs (via K backends)
Very readable (not hard to get used to K’s output)

I will be submitting a Meta EIP shortly, but would love to hear other people’s thoughts!

Related EM Posts

Discussion about the technical spec of the EVM
Yellow Paper Maintainership
Highlighting the other standards & specs that make up Ethereum

Author: boris
Date: 2019-01-11T19:52:36.366Z

Does anyone know if the Constantinople changes have been added to the Yellow Paper or Jello Paper? @jpitts @Arachnid @ehildenb
I just filed a PR to set 1014 and 1052 to Accepted – I’m not even sure if they went through Last Call (not, as far as I can tell, they just sat in Draft). I’m happy to help maintain the EIPs repo if there aren’t enough hands on deck for this basic admin stuff.
EDIT: @axic beat me to creating the same PR. I noticed this back in December, should’ve done it then.

Author: ehildenb
Date: 2019-01-11T19:55:29.272Z

We are current as of 23 days ago: LINK REDACTED FOR SOME REASON
The Jello Paper is unfortunately not automatically updated on PRs into KEVM, just occasionally manually (though the derivation process itself is 100% automatic). It’s on my todo list to have the CI server update the Jello Paper automatically on merges to master.

Author: gcolvin
Date: 2019-01-12T06:57:58.783Z

I brought this up on the AllCoreDevs channel and there was the usual criticism, but more support for the Yellow Paper than I expected.  People liked that there were formal specs, but wanted there to be one that they can read as well.
The math in the Yellow Paper is not that advanced,  (anyone with a year of so of undergraduate study can handle it) but it’s badly presented. And the English could be much better, as in the Beige Paper.  But with effort people who need to read it can, and without learning another language first.  And with effort it could be much more readable.
Compared to technical English and basic math very few people can read formal specification languages, and that terribly compromises many purposes of having a spec.  For one important example, a spec that an engineer can use to write a client matters.  For that purpose, for most engineers, the Yellow Paper falls short, but a K spec would be nearly useless.
To be truly useful I think our K spec would need to use a carefully designed presentation syntax, teach the reader the K they need to know, and be sure the embedding English is complete and correct without the K.  Because in the end a K spec is a reference implementation in a language that hardly anybody knows.  And reference implementations have well-known strengths and weaknesses as specs.
Still, it depends on who does the work, and how the community cares to put it to use.  A number of people volunteered to get to work on a better Yellow Paper, including how best to embrace our K and Lem formalizations.  Whether they do the work remains to be seen.

Author: gcolvin
Date: 2019-01-12T08:19:23.217Z

PS.
For what it’s worth.

The Yellow Paper specifies the entire client in about 2600 lines of LaTex.  About half of that is the VM.
The Jello Paper specifies the VM in about 2400 lines of K.
py-evm implements the VM in about 2900 lines of Python.
aleth implements the VM in about 2400 lines of C++.

Author: grosu
Date: 2019-01-12T16:26:24.939Z

Thanks, @expede, for starting this discussion!  Your list of reasons is great!  I would like to add one more, which in my view is in fact the most important one in the long term (disclaimer: I am a researcher in formal methods, specification and verification, as well as PL design — so I may not see things as clearly as I should):

Usable as is as input to a program verifier that can formally specify and verify smart contracts at the EVM bytecode level.  That is, the gap between what you verify and what you execute is zero.
That was the main reason we came up with the K semantics of EVM first place (we’ve already done that with other languages, such as C, Java, JavaScript, Python, etc., in the past).

Author: grosu
Date: 2019-01-12T16:36:42.932Z

@gcolvin I share your thoughts about  formal specs being harder to read/grasp by non-experts.  And I am genuinely interested in doing something about it.  It would be really nice to have a tool that takes a formal semantics (say in K for starters) and generate human-readable documentation from it.  Note that this is research, though!  That is, I’d need one or two people to work on such a project for 6 months or so.  My company does not have any resources allocated to do this, so we either need some funding or some enthusiastic volunteers.  Such a tool would have the benefit that we can then keep using it as EVM itself evolves, not to mention that it can be used for other VMs and languages as well (WASM, IELE, maybe even higher level languages).
Anyway, I am in for this.  Our main concern so far was really to make sure that the KEVM semantics is adequate, especially because as I said above, we use it for formal verification of smart contracts.  Serving as documentation for EVM was not our original intention.  But it makes a lot of sense to pick a reference model that is rigorous and that is used as input to tools, because it avoids the problem of having to keep things in synch.  Ah, and additional advantage is that different uses of it with different tools sometimes reveal quite subtle bugs in the semantics/model, which once fixed, they are then fixed for all tools.

Author: grosu
Date: 2019-01-12T16:40:44.104Z

Thanks for these statistics, @gcolvin, very useful to know!  Again, the K semantics can be used for many purposes as is.  Execution is only one use.  Symbolic execution, model checking, deductive verification (verification of smart contracts), and hopefully test-case generation soon, are other possible uses.  In my view this is the main advantage of the K approach, not necessarily compactness.  I would go for these advantages even if the K semantics of the EVM were 4x larger than an implementation.  But again, I am biased here.  Heavily.

Author: boris
Date: 2019-01-12T18:27:34.565Z

Is there a tracking Github issue for this? (auto-update of Jello Paper by CI Server) Please paste it here or create it, because then it can be bountied!

Author: gcolvin
Date: 2019-01-12T18:27:52.282Z

LatTex is fairly verbose, so the Yellow Paper is already half the line count of the three implementations, and even more compact once printed.  And readable by a vastly larger audience.  And I mean vastly.  That counts a lot for me.
But in the end we will continue to have the Yellow Paper for so long as we continue to have editors, and it will continue to be used as a standard for so long as people find it useful.  We will also have implementations in various languages, including the Python reference implementation, the K and Lem specifications, and various production clients.  Some of those, especially the Python and K work, may also prove useful as standards.
In a consensus network they all must agree.  Any disagreement indicates a defect in at least one of them.  Which one, if any, to call canonical or normative is to some extent a matter of convenience.  But whose convenience, and for what purposes?
PS.  The K spec being no larger than programs that implement it I consider a victory.

Author: gcolvin
Date: 2019-01-12T19:12:27.667Z

So we have a small number of specifications and reference implementations. Primarily

Python research implementation and specification
EIPs slated for a release
KEVM reference implementation and specification
Yellow Paper

Each serves different purposes at difference points in the workflow. I don’t see a need (or even a way) to bless one of them as “the standard.”

Author: gcolvin
Date: 2019-01-12T19:18:07.436Z

Note.  It’s a good thing that K is code.  Mostly.
Strip-Les-specs-cest-du-code-650-finalenglish650×613 266 KB

Author: ehildenb
Date: 2019-01-12T22:30:47.803Z

Hmmmmm, seems my posts are being hidden because I’m including links to repository stuffs? That seems insane.
Anyway, @boris, I’ve made an issue about adding a CD job to update the Jello Paper on updates to master. It’s issue 239 on KEVM repository, which I can’t link to because that would be “overly promotional”. A little bit insane, because we’re literally talking about that repository.

Author: boris
Date: 2019-01-13T02:29:33.845Z

Have to stop the spammers some how!
After spending time coming back to the forum, your account gets upgraded. I don’t have admin powers to do more right now, but just PM me if it’s a problem.
Thanks for filing the issue. Here’s the link for reference: https://github.com/kframework/evm-semantics/issues/293

Author: gcolvin
Date: 2019-01-14T20:40:58.378Z

@ehildenb Could summarize what we haven’t been able to read?

Author: ehildenb
Date: 2019-01-14T20:53:19.334Z

The post that remains hidden says:
Great to see other people championing this! 
Feel free to open issues against the KEVM repository (at LINK REDACTED FOR SOME REASON) if you find places where it could be clearer or could use more explanation. The Jello Paper is derived from that repository directly.

Author: gcolvin
Date: 2019-01-15T04:48:01.262Z

Thanks @ehildenb  To be clear, @grosu and @expede, I’m a big fan of K-EVM and the Jello Paper.  I’m also a big fan of the work to maintain and improve the Yellow Paper.  But I’m becoming convinced that given our anarchy there is nobody to declare one spec normative, only community consensus that certain specs are useful.   And given the computational consensus network we are building there is no need for a normative spec; the network itself is normative.  To mangle Heinlein, “The blockchain is a harsh mistress.”

Author: expede
Date: 2019-01-16T00:30:27.827Z

Hmm, I’m not sure why LOC would matter here. @gcolvin am I missing something here? IMO a spec should be complete and clear, without regard for length (either short or long)
Anyhow, here’s some information in a table





kLOC
Language
Style
Executable
Complete
Days Since Update




Yellow
~1.3
LaTeX
Prose & Equations

Mostly
37


Beige
1.3
LaTeX
Prose


249


Jello
2.4
K
Declarative


7


Trinity
2.9
Python
Imperative


0


Aleth
2.4
C++
Imperative


0

Author: expede
Date: 2019-01-16T00:46:55.651Z

To be clear, @grosu and @expede, I’m a big fan of K-EVM and the Jello Paper.

 

there is no need for a normative spec

 Thinking out loud
Interesting. Shouldn’t the semantics of the execution engine be strongly guaranteed by the platform, lest a network split occur from differing state? AFAICT Ethereum is intended to have a logically centralized VM. Such an event would get healed, and is absolutely possible today, but also would cause some general thrash in the system (uncle rate), which should be best avoided, no?

Author: expede
Date: 2019-01-16T06:04:33.594Z

TL;DR an impassioned defense of the Jello Paper’s notation
I’m trying to balance facilitating discussion and being opinionated  Take the following with a grain of salt, and opposing viewpoints are very strongly encouraged!
While almost everyone that I speak to is immediately positive about moving to the JP, I feel like there’s two points of resistance:

No one seems to love the YP’s notation as-is, but some have already invested in learning it
Don’t want to invest in learning a new notation / assume K is for math geniuses only

K doesn’t read like the PLT literature (aside from BNF). As such, there’s no special advantage on first reading of the JP vs YP, even for people with such a background (omitting those who already know K). From an informal sample, K appears to be much easier to grok for programmers than judgements or the lambda calculus. Further, being that it’s executable (and formalized), the JP has no option but to be extremely clear about all details, which is what you want out of a spec.
The YP’s notation also takes some getting used to. I personally find its syntax to be unnecessarily dense (ex. “which μ subscript is which again?”). Also some details are unclear, so I found myself repeatedly referring to concrete implementations (mainly Parity and Trinity) while writing my implementation based on the YP. It’s a a small sample, but I hear this from others as well. Admittedly this could be corrected in the notation and prose, but this paper is almost 5 years old and still reads this way

Author: expede
Date: 2019-01-18T22:10:13.723Z

I’m just going to vaguely gesture at this thread as more reason to have a formally verified spec. Explicit invariant are also very important. We can go pretty far enshrining invariants in the system




Immutables, invariants, and upgradability EIPs


    One of the critical meta-questions raised by Remediations for EIP-1283 reentrancy bug and the delay of the Constantinople upgrade is: Precisely what on Ethereum is immutable and what behavior should be considered invariant? 
Since irregular state transitions are outside the scope of this conversation, for sake of argument let’s all agree that code and data (storage) are immutable. 
However, we’re left with the challenge that EVM semantics can and do change during a hard fork, the most germane ex…
  

I personally would love if the community would take a first-principles approach on Ethereum, with only explicit invariants. The current system is loose enough to make spec change conversations difficult.

Author: jpitts
Date: 2019-01-18T22:18:36.914Z

I was starting to think about semantic versioning.
One notion that I arrived at is that the protocol should itself have semantic version, and that its component parts, i.e. the EVM, consensus, RLP, JSON-RPC, each should have semantic versions. Within their various contexts, the versioning changes would have different meaning.


Semantic Versioning


Semantic Versioning 2.0.0
Semantic Versioning spec and website





Ethereum protocol would increment MAJOR when the update to any of its components leads to a fork.
EVM would increment MAJOR only if the update leads to incompatibility or security vulnerability (given contract development norms). MAJOR version increments in EVM is similar to a new series in microprocessors. But MINOR version increments in the EVM would lead to a MAJOR increment at the protocol level.

Author: expede
Date: 2019-01-18T22:25:55.447Z

jpitts:

I was starting to think about semantic versioning.


 I very much want semantic versioning!

It’s currently difficult to track the various names and EIPs that should be applied in a certain order
Clearer for users (“oh, this implements Ethereum 4.0.0, so something is not backwards compatible”)
Distinctions between bug fixes and semantic changes generally useful

Author: gcolvin
Date: 2019-01-19T13:30:12.266Z

@expede  I simply found LOC interesting.  First because I was surprised that the C++ version was no longer than the others, despite being a low-level language bummed for performance. It was shorter when I started work on it, and  I’d have expected Python to be shorter still.  And second because the K version was no longer than the rest.  Formal specs used to be a lot longer than the code they formalized.

Author: gcolvin
Date: 2019-01-19T14:16:10.848Z

expede:

While almost everyone that I speak to is immediately positive about moving to the JP, I feel like there’s two points of resistance:

No one seems to love the YP’s notation as-is, but some have already invested in learning it
Don’t want to invest in learning a new notation / assume K is for math geniuses only



You talk to different people than I do.  In that thread of core dev discussion I shared I think only Everett was unconditionally positive about dropping the Yellow Paper.
This is an old discussion, with a chapter on it in The Mythical Man Month.  We are in the usual bind.  Most all engineers and mathematicians can read English and ordinary math.  They cannot, without prior study, read formal notations like K.  And they usually haven’t time or desire for such study. (One way out is an English spec and a formal appendix.  Another is to make the spec a tutorial on the formal notation.)
Regardless, as long as people want to maintain and improve the Yellow Paper there will be one.  And if people want to complete a Jello Paper there will be one.  And keeping them in sync with each other, the various clients, and the network itself will be a pain.  It’s pain with only one spec too.

Author: gcolvin
Date: 2019-01-19T14:31:05.136Z

expede:

The YP’s notation also takes some getting used to. I personally find its syntax to be unnecessarily dense […] this could be corrected in the notation and prose, but this paper is almost 5 years old and still reads this way


Until last year the copyright prevented anyone but Gav from changing it.  Yoichi put a lot of work into it, (and on formalizing the EVM) but then decided to stop working on crypto.  So only recently are others taking up the task.

Author: expede
Date: 2019-01-21T03:49:23.445Z

Thanks for taking the time to explore this! 

Copyright Issues



 gcolvin:

Until last year the copyright prevented anyone but Gav from changing it. Yoichi put a lot of work into it, (and on formalizing the EVM) but then decided to stop working on crypto. So only recently are others taking up the task.


Ah, excellent context — thanks! That’s a pretty horrendous situation for a community project to be in  The copyright for such a foundational component of this project absolutely should have been released much earlier, but that’s neither here nor there.
It sounds like there’s a solve now, though, so that’s great 

Unnamed Sources



 gcolvin:

You talk to different people than I do. In that thread of core dev discussion I shared I think only Everett was unconditionally positive about dropping the Yellow Paper.





 gcolvin:

I’m a big fan of K-EVM and the Jello Paper. I’m also a big fan of the work to maintain and improve the Yellow Paper.


My original source was the controversial semi(?) closed discussion in Prague, where it was brought up by someone else (I can’t remember who for the life of me). The Twitter Thread has a number of supporters, as well as some of the folks in this thread (including you   ).
I agree that there is no strong consensus on this change!
I really enjoyed the IETF’s “rough consensus and running code” concept that came up at the EM Council of Prague. In the Jello Paper we have a more complete & working spec, and support from a number of people in the know. The question becomes if this is a “over my dead body”, and why if so?

The Yellow Paper is Dead, Long Live the Yellow Paper



 gcolvin:

as long as people want to maintain and improve the Yellow Paper there will be one


Wonderful! Let people use whichever notation they like, and let a thousand flowers bloom! But even then, why not make the canonical spec for EIPs the formally verified executable one? People can still translate concepts into other notations, or have then generated directly by the K Framework. This was the intent of creating the Beige Paper by hand (since the Yellow Paper was too obtuse for most people), but the canonical spec was still assumed to be the YP.
One such setup could look like this:
                     Jello Paper
                   (canonical spec)
                         |
                         |
      +------------------+---------------------+
      |                  |                     |
      |                  |                     |
Yellow Paper         Beige Paper       BNF & Proof Trees
(historical)        (accessible)        (mathematicians)


The Comprehensibility of K



 gcolvin:

Most all engineers and mathematicians can read English and ordinary math. They cannot, without prior study, read formal notations like K


I actually strongly disagree on needing formal training to read K 
I’m not sure that I’d call K a “formal notation”. It is very much a programming language, and generates both executable code and documentation (conceptually in whatever notation you would like). This reads in a way not unlike modern Javascript’s arrow functions with destructuring, and consuming a K-based spec requires no additional study in PLT or specialized mathematical knowledge than any other programming language. Presumably you would like the writers of the spec in any notation to have some relevant background, though they may or may not be experts in K (I certainly am not one!)
For reference on how much more familiar K is than operational semantics, see the illustrative examples below
Example of K (from Jello Paper)
    syntax UnStackOp ::= "POP"
    rule <k> POP W => . ... </k>

    syntax StackOp ::= DUP ( Int ) | SWAP ( Int )
    rule <k> DUP(N)  WS:WordStack => #setStack ((WS [ N -Int 1 ]) : WS)                      ... </k>
    rule <k> SWAP(N) (W0 : WS)    => #setStack ((WS [ N -Int 1 ]) : (WS [ N -Int 1 := W0 ])) ... </k>

    syntax PushOp ::= PUSH ( Int , Int )
    rule <k> PUSH(_, W) => W ~> #push ... </k>

Formal Notation
This is what a common formal notation looks like:
Big-step-operational-semantics-for-expression-evaluation839×440 64.2 KB

Call For “Over My Dead Bodies” 
Correct me if I’m wrong, but you [Greg] are not an “over my dead body”. I would love to hear directly from people that are dead set against this idea, and why that’s the case for them! If you could point the people that you’re talking to at this thread, that would be very helpful!

Author: gcolvin
Date: 2019-01-21T20:14:03.304Z

expede:

why not make the canonical spec for EIPs the formally verified executable one


The questions I keep bumping into are, “Who has the authority to make it canonical?”  and, “Do we really need one spec to be canonical?”
I’d say the current consensus is that the Yellow Paper specifies the Ethereum protocol, and that where it doesn’t that is a defect that needs fixing.  No other specification that I know of is complete enough to even attempt to do that job.  When there are other specs that can then we can ask about their relative status.



 expede:

I actually strongly disagree on needing formal training to read K


Ok, but I can follow the VM part of the Yellow Paper well enough to write code, and can just barely follow the KEVM.  And yes, the notation you quote is unreadable too (though it’s not the same as the K), but the math in the Yellow Paper is much easier than your example, and could be made even easier.
It looks to me like different specs are useful in different ways to different people (and programs).  Different enough that there are people who can use some but cannot use the others.

Author: jpitts
Date: 2019-01-21T22:14:58.798Z

To continue the semver sub-thread here, @axic pointed out EIP-178 for versioning the EVM:



Sermantic versioning for the protocol, with release candidates

A similar proposal was made back here: Assigning version numbers to EVM changes · Issue #178 · ethereum/EIPs · GitHub
It comes with some differences:

only considers EVM,
because of this goes well with EVM Version Identifier · Issue #154 · ethereum/EIPs · GitHub,
only uses major/minor in order to save space

Author: hjort
Date: 2019-01-23T10:08:39.723Z

Another is to make the spec a tutorial on the formal notation.

I very much support this idea. Having an executable specification will help with verification efforts and proving invariants. The YP does a good job of being self-contained, being a tutorial in its own notation. I don’t see why the JP couldn’t do the same thing – making itself usable to anyone with a year or so of undergraduate study.

Author: grosu
Date: 2019-01-23T13:13:34.777Z

That you for all the comments above about K-EVM, both the positive and the negative ones!  Of course, as creators and everyday users of KEVM it is hard for us at RV (http://runtimeverification.com) to see how it fails short, which is why it is so important to have feedback like here.  Here is another thing that may be useful to consider:

At my knowledge, there are three other companies besides RV using KEVM in their smart contract execution or auditing.  In particular, we have 2-3 smart contracts under “formal auditing” at any given moment.  So there is a huge incentive to keep it updated and current!  This may not happen with a paper model/semantics, because that is not directly used.  Additionally, there are several formal tools based on it: an interpreter (which btw way, will be soon much faster, because we transitioned from OCAML to LLVM as target language we generate from K), a bounded symbolic state-space explorer, and a program verifier.  And tens of thousands of tests for each of these that we execute in our CI.  These tend to find bugs quickly in the spec, for the benefit of all.  Only one tool, say an execution engine, can only find a particular subset of bugs but not all.  For example, what if your spec happens to be non-deterministic?  You may not find that with the interpreter but can find that with the state exploration tool (model checker).

If I understand it well, the main criticism is the lack of readability.  I must confess that we put very little to almost no effort into making it readable!  Our main focus was to make it precise and have all the tools work with it.  This is good in my view, because we can address the readability part.  It may only be a matter of having a good technical writer spend 2 months on it and turn it into a book/tutorial in literate programming style.  Or two writers, a technical and a non-technical.  We could do this if there is interest.

Author: dteiml
Date: 2019-01-24T09:30:41.685Z

Hello, Dominik here from Gnosis, one of the companies that uses K-EVM internally. I recently completed the K onboarding, thought I’d share my experience:

There is a pretty good K tutorial available. The only prerequisite that I remember is lambda calculus, but looking back, it is totally possible to skip 1_lambda and 3_lambda++ and complete only 2_imp and 4_imp++ to grasp K.
I have read through the entire K-EVM specification. I disagree with @grosu that it is not readable.  If you have any experience with the functional paradigm in Haskell or even Javascript, it is very readable, and even if one doesn’t, it is very easy to get into.
What makes K readable in my experience is its explicitness - everything is defined in configuration, syntax and rules - as such the language definition is self-contained, which is super powerful. I personally use K-EVM as the canonical specification of the EVM - if I am unsure about the semantics of an opcode/gas calculation/whatever, I refer to that definition, because of its completeness and simplicity.
I have also used K for formal verification. This, imho, is its weak point when it comes to usability. The kprove tool is quite difficult to use and I found I had to contact RV folks to get help iteratively. In our discussions, this is one thing they are working on, as well as many other things like producing easily verifiable proofs etc. (as they switch from Java to Haskell backend), so I have no doubt that this will improve  shortly. But specification of EVM, rather than its programs, is the main topic of this discussion, and for that, nothing beats K

Talking about tutorials/onboarding, I made a Formal verification workshop not long ago. I would be happy to expand this to a full-fledge course (working with RV) if there is demand from the community!
Best,

Author: boris
Date: 2019-01-24T17:29:47.771Z

dteiml:

I made a Formal verification workshop not long ago. I would be happy to expand this to a full-fledge course (working with RV) if there is demand from the community!


Maybe we can schedule a workshop for Council of Paris? Or at least, that week, maybe during ETHCC or after? This is the kind of not-quite-council stuff that I am happy to help volunteer around if there is interest.
When you say “I found I had to contact RV folks” – this is the sort of thing where public questions (e.g. github issues or forum threads) would be super helpful. Many times, multiple people have the same question, and either through search or from following along with a discussion, can get answers and learn.

Author: dteiml
Date: 2019-01-28T21:35:34.384Z

boris:

Maybe we can schedule a workshop for Council of Paris? Or at least, that week, maybe during ETHCC or after? This is the kind of not-quite-council stuff that I am happy to help volunteer around if there is interest.


Totally agree. A workshop at Council of Paris would probably be the easiest option, who can I get in touch with?  I could give an introduction into K, K-EVM and deductive program verification using K-EVM!

Author: gcolvin
Date: 2019-01-28T21:57:09.025Z

An aside: Isn’t the Council of Paris just one day?  Seems it might not be worth squeezing in many workshops.  Better to spend most of our time on the important issues affecting the Fellowship as a whole.

Author: boris
Date: 2019-01-28T22:55:29.132Z

@dteiml that would be me  PM me / I’ll reach out and we can see about organizing.
I’m thinking a few events post ETHCC with smaller groups.

Author: pmackay1982
Date: 2019-01-29T16:21:22.239Z

Boris - good morning from very cold Urbana, IL.  Please include myself and @ehildenb in this discussion about a K workshop. Thanks!

Author: gcolvin
Date: 2019-01-30T00:25:05.856Z

gcolvin:

The questions I keep bumping into are, “Who has the authority to make it canonical?” and, “Do we really need one spec to be canonical?”


Since I like arguing with myself, one authoritative group are the core developers, who more than anyone actually have to use the spec.
My own ideal would be in three parts:

a cleaned-up, Beige-tinted Yellow Paper,
a readable, executable K spec, and
an associated test suite.

The whole package would be one canonical specification, each part of which serves as a check on the others.  Note that the test suite should not be generated by the K, or else it will not serve as an independent check. Never go to sea with two chronometers, take one or three. – Fred Brooks, The Mythical Man Month
But most of all I’d like to see the Yellow, Jello, and Beige people work together towards what is best.  I think @jpitts is working towards that goal.  And some money for @expede, @grosu and some others would help  ;->

Author: fubuloubu
Date: 2019-01-30T23:56:03.673Z

gcolvin:

The questions I keep bumping into are, “Who has the authority to make it canonical?” and, “Do we really need one spec to be canonical?”


We all do! So let’s do it! We need something that describes the spec in the best fashion possible.



 gcolvin:

Note that the test suite should not be generated by the K, or else it will not serve as an independent check.


Here here! Triplex redundancy is the strongest architecture you can get for the least amount of complexity.



 gcolvin:

And some money… would help


Without a doubt!

I’d like to underscore that a canonical spec and test suite (that is maintained and updated with every EIP as it hits mainnet) is a great coordinating factor for all those who wish to use Ethereum and the EVM (compiler devs, tool makers, client devs, etc.) It is the one thing I believe led to the success of all those who built clients in the past year (ask @pipermerriam about Trinity and how the test suite helped)
We REALLY need to keep this documentation coordinated and as up to date as possible. As a piece of core infrastructure it is critical to our success. Funding it is a no brainer!
Also, there should be an Ethereum 2.0 branch for all of this. Tracing the history alone would be worth it.

Author: hmijail
Date: 2019-01-31T00:53:23.592Z

gcolvin:

Never go to sea with two chronometers, take one or three.


But, does this really work when we’re talking about software whose goal is to reach consensus?
You either have one chronometer, or else you have differences in consensus, therefore forks, therefore multiple different blockchains.
Code is law at the end of the day. The way I see it, the only choice is whether it’s formal code or informal code. Everything else (papers, docs, whatever) has to play catch up to the existing code up to the point of being bug-compatible.
To me it’s amazing to say the least that informal code has brought us so far…

Author: expede
Date: 2019-01-31T01:06:00.404Z

gcolvin:

Since I like arguing with myself, one authoritative group are the core developers, who more than anyone actually have to use the spec.


 Yup, agreed! It’s the same people who are able to accept opcodes and consensus changes. Nothing is stopping others from running other versions of the spec, but the major implementations should agree.



 gcolvin:

But most of all I’d like to see the Yellow, Jello, and Beige people work together towards what is best. I think @jpitts is working towards that goal. And some money for @expede, @grosu and some others would help ;->


 Thanks for the shout out! Yes, we would love to devote more time to this work, but attention requires funding.



 fubuloubu:

We REALLY need to keep this documentation coordinated and as up to date as possible. As a piece of core infrastructure it is critical to our success. Funding it is a no brainer!


 Could not agree more! Yes, I am involved in some of this work, but I would like to see this regardless of who does the work. We need a solid footing to build this extremely important part of the world’s financial system & highly available censorship resistant open service worth tens of billions of dollars.



 gcolvin:

My own ideal would be in three parts:





 fubuloubu:

Here here! Triplex redundancy is the strongest architecture you can get for the least amount of complexity.


These quotes are about the test suite (I agree), bit to maybe expand on why the Jello Paper as the top-level canonical spec rather than competing specs…
TL;DR
Specs and implementations are different beasts. You don’t want multiple sources of truth in your spec. A spec is more abstract and need to have as little ambiguity as possible. Redundancy on implementations are a really great practice, but that’s a different layer in this process.
Rationale
Client Implementations
Yes, TMR is extremely useful for mission critical implementations. Each implementation in the TMR gate is based on the same high level spec: they need to achieve some common goal. We have something like this built into Ethereum itself: the different clients have to reach consensus about state transitions, and are parallel implementations of the same underlying spec.
TMR is great when you’re interacting with empirical data and concrete implementations: chronometers, flight system computers, deployed blockchain clients, and so on. These are also automated systems, with very easy ways to distinguish the degree of disagreement (results are outside of some error threshold).
To be clear: I fully support doing TMR for the actual EVM software. Super great idea, +1, 5/5 stars, would recommend 
System Specification
Specifications should be a priori and universal. (This is part of the push for a formalized spec). If you have multiple specs, have an unspoken, fuzzy, abstract spec somewhere in the collective consciousness of which the others are simulacra. The versions in everyone’s head may or may not agree.
Multiple competing top-level (canonical) specs aren’t just inconvenient, they’re often actively confusing. For instance, if a feature is added to the Blue Paper after going through the full EIP process, but the Red and Purple Paper maintainers haven’t got around to adding it yet — is the feature available for use? What if the Red Paper is no longer being actively maintained? What if they all differ on parts of certain details in their description of the same feature? Which is the “right” one to base your concrete implementation off of? Should you have to read 3 versions of a tech spec and understand them to the point that you can compare every detail between them?
Since client implementers are doing this by hand, it’s a laborious process, and the likelihood of missing a subtlety between m/n specs is very high. With one canonical, formalized spec, you have a single  source of truth.



 fubuloubu:

I’d like to underscore that a canonical spec and test suite (that is maintained and updated with every EIP as it hits mainnet) is a great coordinating factor for all those who wish to use Ethereum and the EVM (compiler devs, tool makers, client devs, etc.) It is the one thing I believe led to the success of all those who built clients in the past year (ask @pipermerriam about Trinity and how the test suite helped)


Exactly! 



 gcolvin:

Note that the test suite should not be generated by the K, or else it will not serve as an independent check.


Yes, multiple test suites would be awesome (1x K-generated + 1x done by hand, + 1x other)

Author: gcolvin
Date: 2019-01-31T19:44:40.620Z

I’m not really calling for TMR per se, just the principle.  I’m trying to reconcile the fact that people disagree (some strongly) as to whether a better Yellow Paper or a brand-new K spec or something else would be best, and there are strong arguments on all sides.



 expede:

Multiple competing top-level (canonical) specs aren’t just inconvenient, they’re often actively confusing. For instance, if a feature is added to the Blue Paper after going through the full EIP process, but the Red and Purple Paper maintainers haven’t got around to adding it yet — is the feature available for use? What if […]? What if […]? Which is the “right” one?


In my suggestion the specs aren’t competing, they are reinforcing, and maintained by the same team.
A feature is available for use as soon as hits the mainnet.  It will need to be implemented before that, but currently EIPs aren’t merged with the Yellow Paper until after that, because we are moving too fast and haven’t the resources to do better.  Once the Yellow Paper is updated future implementations can refer to it.  Until then you have to read the EIP and the Yellow Paper.  Tests get written partly before and partly after an EIP hits the mainnet, depending on how quickly we are moving.
Having two top-level notations for a spec is not unusual. For instance, in the WebAssembly spec we find:

The prose and formal rules are equivalent, so that understanding of the formal notation is not required to read this specification. The formalism offers a more concise description in notation that is used widely in programming languages semantics and is readily amenable to mathematical proof.

This discussion goes back decades in our craft.  Probably all the way back to Mesopotamia or further in other crafts. Forgive me for a long quotation from Brooks’ classic Mythical Man Month

English, or any other human language, is not naturally a precision instrument for such definitions. Therefore the manual writer must strain himself and his language to achieve the precision needed. An attractive alternative is to use a formal notation for such definitions. After all, precision is the stock in trade, the raison d’etre of formal notations.
Let us examine the merits and weaknesses of formal definitions. As noted, formal definitions are precise. They tend to be complete; gaps show more conspicuously, so they are filled sooner. What they lack is comprehensibility. With English prose one can show structural principles, delineate structure in stages or levels, and give examples. One can readily mark exceptions and emphasize contrasts. Most important, one can explain why. The formal definitions put forward so far have inspired wonder at their elegance and confidence in their precision. But they have demanded prose explanations to make their content easy to learn and teach. For these reasons, I think we will see future specifications to consist of both a formal definition and a prose definition.
An ancient adage warns, “Never go to sea with two chronometers; take one or three.” The same thing clearly applies to prose and formal definitions. If one has both, one must be the standard, and the other must be a derivative description, clearly labeled as such. Either can be the primary standard.

Brooks takes the same lesson from the chronometers as you do @expede.  I’m adding tests to get three co-equal chronometers.  But this only works if the work is coordinated.  Note that not so long ago Yoichi was meeting with Gavin weekly to better understand and edit the Yellow Paper, authored a Lem spec of the EVM, (and specs of other Ethereum components) and was team lead for a group including formalists and testing.  I don’t know if the Foundation is still supporting that group.
If we go with Brooks’ take on things I have my preferences, but all this is just opinions until there is a working group.  For most people it shouldn’t matter which spec is primary.  They will use the one that works for the purpose at hand, on the promise that they are equivalent.  It’s the working group’s job to keep that promise.

Author: fubuloubu
Date: 2019-02-01T03:59:19.196Z

gcolvin:

I’m adding tests to get three co-equal chronometers.


Yeah, this is what I basically meant by TMR. The three (written spec, K spec, and JSON test suite) work in unison, and if there’s ever a disagreement usually 2 of them would disagree with the third (easier to debug!)
They should absolutely be maintained in the same repo by the same maintainers: each proposed feature or fix (EIP) corresponds to a PR against the 3 items together. Tested together, deployed together, merged together. There should be a branch corresponding to each relevant network (at least mainnet and testnet/staging)… I think we can see the heirarchy.
The most important thing to remember is that unlike a lot of the client libraries, this repo would be vastly referenced, so strong commit message practices and the like should be encouraged.

Author: gcolvin
Date: 2019-02-03T01:26:59.461Z

I hate to disagree with someone who agrees with me, @fubuloubu (at least until @expede convinces me otherwise  ) but single PRs against one repo seem unworkable, given that there are currently three repos, maintained by different teams working on different schedules.  I think I sketched out the current process above.  At best they can be synced up at some point after a release.  We don’t operate on three-year release cycles with a well-compensated documentation team.

Author: fubuloubu
Date: 2019-02-03T03:08:42.319Z

That is… Fair!
For the record, this is exactly how a well-organized, well-compensated engineering team would handle documentation for a high-security, complex system that is safety-/mission-critical.

Author: gcolvin
Date: 2019-02-03T03:39:33.448Z

fubuloubu:

this is exactly how


Not sure which this is that. Bryant, but of course we don’t have any such teams.  And it’s time to go dance to some reggae.  Later.

Author: fubuloubu
Date: 2019-02-03T03:54:00.672Z

Good documentation, basically

Author: fubuloubu
Date: 2019-02-03T03:55:34.922Z

You have to pay people for good documentation, because no one wants to write it and maintain it.

Author: gcolvin
Date: 2019-02-03T16:34:18.775Z

I like to write good documentation, and I’m not alone.  The pros at it love to do it.  And lots of people like to get paid for whatever value they create, because they need food, shelter, and medicine.

Author: pdaian
Date: 2019-02-04T17:01:09.306Z

Obviously I have some interest in seeing the Jello Paper adopted, so take my $.02 with this in mind.

A feature is available for use as soon as hits the mainnet. It will need to be implemented before that, but currently EIPs aren’t merged with the Yellow Paper until after that, because we are moving too fast and haven’t the resources to do better. Once the Yellow Paper is updated future implementations can refer to it. Until then you have to read the EIP and the Yellow Paper. Tests get written partly before and partly after an EIP hits the mainnet, depending on how quickly we are moving.

I think this is emblematic of the problems with the Yellow Paper.  Few people understand it, fewer want to touch it, and at the end of the day it trails production because its utility is somewhat limited to being translational.  I think the Jello Paper provides a lot more value in the release cycle that makes it worth updating before releasing code to mainnet.
For example, the Constantinople reentrancy probably would have been caught through a Jello Paper update, assuming KEVM had a re-entrancy model (it currently does not but AFAIK one is being actively investigated); a lot of existing contracts in integration tests likely would have failed symbolic re-entrancy safety checks under the new EIP changes.

For most people it shouldn’t matter which spec is primary. They will use the one that works for the purpose at hand, on the promise that they are equivalent. It’s the working group’s job to keep that promise.

I think what the RV and K folks want to see from the Ethereum community is commitment to long-term maintenance of the Jello Paper and use in documentation / communications material.  The Jello Paper and KEVM should be independent of RV Inc., UIUC, or any of its other stakeholders such as smart contract auditors who are actively using it.
A commitment to maintaining the spec would be just as useful as funding for R&D if managed appropriately, and could be even more useful for ensuring the independence and antifragility of the document.
I totally agree with the Hydra approach… I don’t think there should necessarily be one official specification, but the Yellow Paper has unfortunately grown to sort of inhabit that space, and some official guidance that this is not the case could be helpful.
It’s also worth noting that K specs can contain e.g. LaTeX, so you can have a branch of the KEVM repository or a flag on that project which also generates either the beige(/yellow?) paper, the Jello paper, or a combination of the two where both notations are presented in-line (to help serve as translational if you need to use one of the specs for some reason but only want to understand/make reference to the other).
At the end of the day, my own personal goals would be:

Seeing commitment to maintaining formal specs actively.
Updating specs at appropriate points in the release cycle (before mainnet, testnet, etc).
Making all specs involved more readable; merging the Yellow and Beige paper could fall here.
Ensuring some spec exists in every high level category of specification (purely mathematical/on-paper, executable, test-generating, usable in verification, etc.).  If multiple goals can be hit optimally by a single repo as KEVM attempts, that’s great, if not, other specs can be merged in or included.
Avoiding excessive redundancy.  I think for example KEVM/the Jello Paper provides some clear value over the Yellow Paper in its ability to be used for proofs.  The yellow/beige papers could probably be somehow consolidated, though someone of course needs to take ownership for that.

Author: gcolvin
Date: 2019-02-04T17:56:13.010Z

pdaian:

I think this is emblematic of the problems with the Yellow Paper. Few people understand it, fewer want to touch it, and at the end of the day it trails production because its utility is somewhat limited to being translational. I think the Jello Paper provides a lot more value in the release cycle that makes it worth updating before releasing code to mainnet.


There are people maintaining the Yellow Paper, and that situation is improving since Gavin released the copyright.  I think the issue of whether we can get the specs ahead of the code is independent of the form of the specs.

Author: gcolvin
Date: 2019-02-04T18:07:30.326Z

pdaian:

I think what the RV and K folks want to see from the Ethereum community is commitment to long-term maintenance of the Jello Paper and use in documentation / communications material.


I just want to see long-term commitment to a good specification, and a good process.  And I’m seeing lots of good ideas here.

Author: grosu
Date: 2019-02-04T21:42:01.287Z

We are on the same page, @gcolvin.  Cannot speak for everybody, but I can speak for the RV, UIUC and K teams: we would be glad to take the task of properly documenting the KEVM seriously.  You and @expede and anybody else who is interested are very welcome to help, even if only with suggestions for notation and working.  I know it, in particular, that you dislike the XML-like notation for semantic cells in K.  In fact, K is very flexible wrt syntax; you can literally choose whatever syntax you want for any symbol.  Moreover, as Philip pointed out, we can also generate Latex from K definitions, and you can pick whatever Latex macro you wish for any symbol.  See, for example, the bubbles in this paper: http://fsl.cs.illinois.edu/FSL/papers/2013/rosu-serbanuta-2013-k/rosu-serbanuta-2013-k-public.pdf.  For example, this is how the program configuration looks like in a C-like language:
SIMPLE-Configuration444×384 42.2 KB

Author: grosu
Date: 2019-02-04T21:43:22.379Z

Here is how the semantics of variable lookup looks like:
SIMPLE-Lookup247×104 6.96 KB

Author: grosu
Date: 2019-02-04T21:44:02.666Z

And here is something more complicated, like exceptions in the same imperative language:
SIMPLE-Exceptions493×560 45.4 KB
The point is that this can be easily changed and we would be very glad to work with you and anybody interested to come up with the most appropriate notation for the KEVM.  Not everybody needs to see the textual definition, many can only read the PDF generated out of it, following whatever notation we all agree upon.

Author: expede
Date: 2019-02-05T22:58:56.959Z

I think that this is a major advantage for K (and the Jello Paper). We can generate classic semantics (BMF, judgements, &c), a custom backend for YP-style idioms, and many more. Rather than duplicating a bunch of work N times to support N docs (in one or more repos — I really don’t think that’s the issue), we can have one canonical spec in K that generates whatever notation we want. It still leaves open the ability to write other specs by hand.
I think that this conversation does need to distinguish between concerns of notation and TMR or n/m specs. For example, I strongly agree with the former, and hugely disagree with the later

Author: tjayrush
Date: 2019-02-08T16:32:37.572Z

I’m just going to throw something out there having been previously un-involved in this conversation.
To me, it seems obvious that the yellow paper is past its useful lifetime. I’ve read it many times, and I can agree that it is dense. It’s nearly impossible to update as well. (As is evidenced by the fact that it doesn’t keep up with the code).  To the extent that the Jello Paper can be more easily updated, and that it can, over time, become the generating document for all sorts of things (Latex specs, test cases, formal verification), it seems to me an obvious choice.
At this point, instead of asking if the Jello Paper should be the formal spec, I think we should all simply say “The Jello Paper is the official spec.”  So: “The Jello Paper is the official spec.”
I would go further to propose that the magicians make a concrete statement (or “sense of the fellowship”) at its next meeting to the same effect. The stated purpose of the magicians is to guide the technical direction of the ecosystem. This seems like a clear win.

Author: gcolvin
Date: 2019-02-08T20:07:56.639Z

tjayrush:

I would go further to propose that the magicians make a concrete statement (or “sense of the fellowship”) at its next meeting


It’s not that easy in this decentralized community.  The current Yellow Paper editors operate independently of the Magicians, so such a statement wouldn’t necessarily carry much weight, even if you could get a consensus behind it.   And the Yellow Paper editors operate independently of the EIP process, so the usual course of forming a Ring and getting to consensus on an EIP to improve the Yellow Paper isn’t there.  The Yellow Paper being the current “official spec” that creates a bit of an impasse.  @jpitts is trying to start up a Specification working group to pull together interested people, so I hope he drops by and explains where that is going.

Author: tjayrush
Date: 2019-02-08T20:21:25.042Z

Why do “the current yellow paper editors operate independently of the Magicians”? Why do “the current yellow paper editors operate independently of the EIP process”? Why don’t the future Jello paper editors work with the Magicians and the EIP process?
Open question to anyone who can answer: are there known issues with the K Framework that allow for the possibility of an ill-defined spec? There is clearly the possibility of an ill-defined spec in the current methodology, so I would think we would be trying to move towards more rigor.

Author: boris
Date: 2019-02-08T22:01:16.880Z

I think we’re just going to start filing issues, specifying new EIPs in K, and doing the work. Not too complicated 
If people want to maintain the Yellow Paper — great! It’s all time and effort.
Tightening the EIP process for Core EIPs to make it clear when the specs need to be done is also a good process. Especially if they can assist in testing process and so on.

Author: jpitts
Date: 2019-02-09T00:12:54.070Z

tjayrush:

Why do “the current yellow paper editors operate independently of the Magicians”? Why do “the current yellow paper editors operate independently of the EIP process”? Why don’t the future Jello paper editors work with the Magicians and the EIP process?


Reasons the YP isn’t a Magician’s Ring yet:

We can’t assume by default that YP or other important artifacts of Ethereum belong to any group
AFAICT, Yellow Paper editors aren’t organized enough to be called operating
The YP is not an EIP, although it should be accumulating EIPs

I predict that YP, JP, and “protocol tests” will be loosely maintained by a single group, each sub-project being somewhat independent, and that this single group will eventually become a Ring

Author: gcolvin
Date: 2019-02-09T01:48:51.530Z

The Yellow Paper predates the Magicians by years. It didn’t get updated well for quite a while because of copyright hassles.  Those are settled, more people are working on it–with Nick Savers in the lead–and it’s keeping up with the EIPs better.  I’d say they are as organized as most anything else around here 

github.com/ethereum/yellowpaper








Maintainer(s) for this repo



        opened 03:25AM - 10 Jan 19 UTC


          closed 05:20PM - 05 Feb 19 UTC




          bmann
        







Hey all, trying to figure out who maintains this repo, and is reviewing PRs and …other items on a regular basis. 

From last three commits @acoglio, @tjade273, @nicksavers — do any of you consider yourself maintainers?

Do you need help? Would you be open to adding more maintainers?

Author: gcolvin
Date: 2019-02-09T02:01:58.552Z

@grosu
On notation, I think you know that getting rid of the pseudo-XML would remove about 90% of my distaste for the syntax.  And a complete K spec of Ethereum would be incredibly useful regardless of the fate of the Yellow Paper.  Funding is another matter, and you are not the only team lacking it.

Author: boris
Date: 2019-02-09T17:06:58.542Z

Yes. I’m not disagreeing. I just want to stop talking and continue doing. There is no need to talk about one vs the other. People can just do the work.

Author: ehildenb
Date: 2019-02-09T20:04:53.336Z

Exactly my sentiment  People will vote with their time and code.
I’ll continue to help maintain KEVM, people are welcome to contribute PRs.
With KIELE already existing, and KWASM around the corner, I think having a common spec language will yield increasing returns, in the sense that once you learn K once you can read all the specs that are developed in K.

Author: ehildenb
Date: 2019-02-09T20:09:19.198Z

Having a nicer presentation has been discussed many times, funding and/or devs would help turn discussions into action

Author: gcolvin
Date: 2019-02-11T08:50:19.679Z

<die> xml </die>

Believe me, we are all looking for money for K.  Sort of a love/hate relationship for me, but at my age I’m used to those.
@grosu @ehildenb @expede @boris

Author: grosu
Date: 2019-02-14T15:22:52.170Z

In case the bubble notation above is too colorful, then here is another style for displaying/presenting K semantics, using a more mathematical notation. See, for example, Denis Bogdanas’ thesis on the Java semantics, specifically the Appendix: https://fmse.info.uaic.ro/publication/a-complete-semantics-of-java/.  For example, here is a snapshot:
Capture780×144 10.7 KB
It is really easy to implement such variations; all it takes is to implement the K Latex macros differently in k.sty.

Author: gcolvin
Date: 2019-02-14T21:37:57.186Z

grosu:

In case the bubble notation above is too colorful, then here is another style for displaying/presenting K semantics, using a more mathematical notation


These are all better, depending on the audience.  What are the chances of generating (or in some way automatically aligning with) something like the ordinary math of the Yellow Paper from a K specification?  The math itself is no big deal: ‘and’, ‘or’, ‘not’, ‘there exists’, ‘for all’, ‘if and only iff’, ‘is an element of’ … It’s the presentation of the math that hurts, and the presentation could easily be much better.  And as bad as it is I can follow the Yellow Paper’s description of the EVM much more easily than I can KEVM.   (And yes, that’s because I’m too lazy to learn much K until I really have to, whereas I’ve been dealing with bad math notation for over five decades 
That is, I want the power of K, but I don’t want to give up the familiarity of ordinary math to myself and most any other engineer or scientist.
Failing that, I’d go for a syntax closer to either Python or Solidity, as those are languages most used by the community for describing algorithms and interfaces.

Author: gcolvin
Date: 2019-02-14T21:42:44.752Z

expede:

I think that this conversation does need to distinguish between concerns of notation and TMR or n/m specs. For example, I strongly agree with the former, and hugely disagree with the late


Sorry… what are TMR and n/m?

Author: expede
Date: 2019-02-14T21:58:11.829Z

Correct me if I’m wrong @grosu (you’re more of an expert on K), but it’s totally possible to generate any arbitrary notation, but there needs to be such a generator backend. I’m not sure if one exist for the YP’s notation as of yet, but we could always write one

Author: expede
Date: 2019-02-14T22:03:16.368Z

TMR is “triple modular redundancy” (AKA the chronometer example from The Mythical Man Month)
n/m (read “en of em”) is the generalization of the above. TMR is 2/3, but you can have other ratios depending on the tolerances in your system (ex. 7/9)

Author: gcolvin
Date: 2019-02-15T05:00:27.949Z

Ah yes.  Yet another TLA