GHC Reading Guide- Exploring entrances and mental models to the source code -Takenobu T.
Rev. 0.01.1

WIPNOTE:- This is not an official document by the ghc development team.
- Please refer to the official documents in detail.
- Don't forget “semantics”. It's very important.
- This is written for ghc 9.0.

Contents

Introduction
1. Compiler- Compilation pipeline- Each pipeline stages- Intermediate language syntax- Call graph
2. Runtime system

3. Core libraries
Appendix

References

Introduction

Introduction

Official resources are here

GHC source repository :https://gitlab.haskell.org/ghc/ghc

The GHC Commentary (for developers) :https://gitlab.haskell.org/ghc/ghc/-/wikis/commentary

GHC Documentation (for users) :* master HEAD
* latest major release* version specified

The User's Guide

https://ghc.gitlab.haskell.org/ghc/doc/
https://downloads.haskell.org/~ghc/latest/docs/html/
https://downloads.haskell.org/~ghc/9.0.1/docs/html/

Core Libraries

GHC API

Introduction

The GHC = Compiler + Runtime System (RTS) + Core Libraries

Haskellsource (.hs)

GHC compilerobject (.o)

RuntimeSystem(libHsRts.o)Core Libraries(GHC.Base, ...)

LinkerExecutablebinary

including the RTS(* static link case)Introduction

Each division is located in the GHC source tree
GHC source repository :https://gitlab.haskell.org/ghc/ghc

compiler/
rts/
libraries/... compiler sources... runtime system sources... core library sources

ghc/
includes/... compiler main... include files

testsuite/
nofib/... test suites... performance tests

mk/hadrian/... build system... hadrian build systemdocs/... documents::

1. Compiler1. Compiler

Compilation pipeline

1. compiler

The GHC compiler

Haskell language

GHC compiler

Assembly language(native or llvm)

1. compiler

GHC compiler comprises pipeline stages
GHC compiler

Haskell language

ParserRenamer
Type checker
Desugarer
Core to Core
Core to STG
STG to Cmm
Cmm to Asm

Assembly language(native or llvm)

1. compiler

Pipeline stages process with intermediate languages
GHC compiler

Haskell language

Intermediate languages

ParserRenamer

Haskell abstract syntax(HsSyn)Type checker
Desugarer
Core to Core

Core language

Core to STGSTG language

STG to Cmm

Cmm language

Cmm to Asm

Assembly language(native or llvm)

1. compiler

Each code is located in
GHC compiler

Parser[compiler/GHC/Parser]

Renamer[compiler/GHC/Rename]

Type checker[compiler/GHC/Tc]

Desugarer[compiler/GHC/HsToCore]

Core to Core

[compiler/GHC/Core/Opt]

Core to STG[compiler/GHC/CoreToStg]

STG to Cmm[compiler/GHC/StgToCmm]

Cmm to Asm[compiler/GHC/CmmTo{Asm, Llvm}]

1. Compiler

Each pipeline stages

1. compiler

ParserHsSyn

Haskell source(Haskell Abstract Syntax)
FunBindf x = inc x

Parserfx

HsApp(reader)inc

Abstracted- Parsing a Haskell source file
- Checking user syntax errors- etc.

x1. compiler

fRenamer

HsSyn

HsSyn

FunBind

FunBindx

Renamer

HsApp
incx

f_1

x_2

HsApp
inc_3

Unique named- Resolving all of the identifiers- Rearranging infix expressions- Checking user scope errors- Building global environments- etc.

x_2

1. compiler

f_1

Type checker

HsSyn

HsSyn

FunBind

FunBindx_2

Type checker

HsApp
inc_3

x_2

- Resolving/Inferring types- Decorating AST with types- Checking user type errors- etc.

f_1

x_2@ Int -> Int@ IntHsApp
inc_3

x_2

@ Int -> Int@ Int

Explicitly typed

1. compiler

Desugarer

HsSynCore language

FunBindf_1

x_2@ Int -> Int

@ Int

Bind

Desugarer

HsApp(HsToCore)
inc_3

x_2

@ Int -> Int@ Int- Desugaring HsSyn to Core
- Checking pattern-match overlap- etc.

Lam
App

Squeezed into λ calculus

1. compiler

Core to Core

Core language

Core language

BindBindCore to Core

LamAppOptimized- Optimizing Core (simplifier, ...)- Checking typechecker's result with Lint
- Tidying Core to Core- etc.

1. compiler

Core to Stg

Core language

STG language
StgBindBind

Core to STG

StgRhsClosure
StgCaseregisterstack- Transforming Core to STG- etc.

heap

Operationally mapped1. compiler

STG to Cmm

STG language

Cmm language

StgBind
StgRhsClosure
StgCaseregisterstack

STG to Cmm(code gen)heap

- Optimizing STG- Transforming STG to Cmm- etc.

Instruction blocked1. compiler

Cmm to Assembly

Cmm language

Assembly/LLVM language

Cmm to Asm(native code gen)

add $0x2, %rbx
jmpq *0x0(%rbp):Machine coded- Optimizing Cmm- Generating Asm- etc.

1. Compiler

Intermediate language syntax

1. compiler

HsSyn (Haskell abstract syntax)[compiler/GHC/Hs/Decls.hs][compiler/GHC/Hs/Expr.hs]

data HsDecl p= TyClD ...
| InstD ...
| DerivD ...
| ValD ...
| SigD ...
| KindSigD ...
| DefD ...
| ForD ...
| WarningD ...
| AnnD ...
| RuleD ...
| SpliceD ...
| DocD ...
| RoleAnnotD ...
| XHsDecl ...

data HsExpr p= HsVar ...
| HsUnboundVar ...
| HsConLikeOut ...
| HsRecFld ...
| HsOverLabel ...
| HsIPVar ...
| HsOverLit ...
| HsLit ...
| HsLam ...
| HsLamCase ...
| HsApp ...
| HsAppType ...
| OpApp...
| NegApp...
| HsPar...
| SectionL ...
| SectionR ...
| ExplicitTuple| ExplicitSum| HsCase...
| HsIf...
| HsMultiIf ...
| HsLet...
| HsDo...
| ExplicitList| RecordCon| RecordUpd| ExprWithTySig| ArithSeq:-- Type or Class Declaration-- Instance declaration-- Deriving declaration-- Value declaration-- Signature declaration-- Standalone kind signature-- 'default' declaration-- Foreign declaration-- Warning declaration-- Annotation declaration-- Rule declaration-- Splice declaration-- Documentation comment declaration-- Role annotation declaration[compiler/GHC/Hs/Binds.hs]
data HsBindLR idL idR= FunBind ...
-- Function-like Binding| PatBind ...
-- Pattern Binding| VarBind ...
-- Variable Binding| AbsBinds ...
-- Abstraction Bindings| PatSynBind ...
-- Patterns Synonym Binding| XHsBindsLR ...

An abstract syntax corresponding to Haskell user source.

1. compiler

Core language[compiler/GHC/Core.hs]

type CoreProgram = [CoreBind]
type CoreBndr = Var
type CoreExpr = Expr CoreBndr
type CoreArg = Arg CoreBndr
type CoreBind = Bind CoreBndr
type CoreAlt = Alt CoreBndr
data Expr b= Var Id| Lit Literal| App (Expr b) (Arg b)| Lam b (Expr b)| Let (Bind b) (Expr b)| Case (Expr b) b Type [Alt b]| Cast (Expr b) Coercion| Tick (Tickish Id) (Expr b)| Type Type| Coercion Coercion-- Variable-- Literal-- Application-- Lambda abstraction-- Variable binding-- Pattern match-- Cast-- Internal note-- Type-- Coercion

A tiny explicitly-typed functional language.
Only ten data constructors based on System FC.

1. compiler

STG language[compiler/GHC/Stg/Syntax.hs]
data GenStgTopBinding pass
= StgTopLifted (GenStgBinding pass) | StgTopStringLit Id ByteString
data GenStgBinding pass
= StgNonRec (BinderP pass) (GenStgRhs pass) | StgRec [(BinderP pass, GenStgRhs pass)]
data GenStgRhs pass
= StgRhsClosure (XRhsClosure pass) CostCentreStack !UpdateFlag [BinderP pass] (GenStgExpr pass)| StgRhsCon
CostCentreStack DataCon [StgArg]

data GenStgExpr pass
= StgAppId [StgArg]| StgLit
Literal| StgConApp
DataCon [StgArg] [Type]| StgOpApp
StgOp [StgArg] Type| StgLam(NonEmpty (BinderP pass)) StgExpr| StgCase(GenStgExpr pass) (BinderP pass) AltType [GenStgAlt pass]| StgLet(XLet pass) (GenStgBinding pass) (GenStgExprpass)| StgLetNoEscape (XLetNoEscape pass) (GenStgBinding pass) (GenStgExpr pass)| StgTick(Tickish Id) (GenStgExpr pass)
A very small purely-functional language with the abstract machine (STG-machine) semantics.

1. compiler

Cmm language[compiler/GHC/Cmm.hs]
type CmmProgram = [CmmGroup]
type CmmGroup = GenCmmGroup CmmStatics CmmTopInfo CmmGraph
type CmmGraph = GenCmmGraph CmmNode[compiler/GHC/Cmm/Node.hs]
data CmmNode e x where
CmmEntry ...
CmmComment ...
CmmTick ...
CmmUnwind ...
CmmAssign :: !CmmReg -> !CmmExpr -> CmmNode O O
CmmStore ...
CmmUnsafeForeignCall ...
CmmBranch ...
CmmCondBranch ...
CmmSwitch ...
CmmCall ...
CmmForeignCall ...

-- Entry-- Comment-- Tick annotation-- Unwind pseudo-instruction-- Assign to register-- Assign to memory location-- An unsafe foreign call-- Goto another block-- Conditional branch-- Switch-- A native call or tail call-- A safe foreign call[compiler/GHC/Cmm/Expr.hs]
data CmmExpr= CmmLit
CmmLit| CmmLoad!CmmExpr !CmmType| CmmReg!CmmReg| CmmMachOp
MachOp [CmmExpr]| CmmStackSlot Area {-# UNPACK #-} !Int| CmmRegOff!CmmReg Int-- Literal-- Read memory location-- Contents of register-- Machine operation (+, -, *, etc.)

A low-level imperative language with an explicit stack.

1. Compiler

Call graph1. compiler

An example of a call graph
main
main'
oneShot
compileFile
runPipeline
runPipeline'
pipeLoop
runHookedPhase
runPhasehscIncrementalCompile
hscIncrementalFrontend
hsc_typecheckhscParse'
ParserparseModule
tcRnModule'
Renamer andtcRnModule
tcRnModuleTcRnM
Type checker
tcRnSrcDecls
tc_rn_src_decls
rnTopSrcDecls
tcTopSrcDecls
finishhscDesugar'
Desugarer
deSugarhscSimplify'
Core to Core
core2core
hscGenHardCode
corePrepPgm
Core to STG
myCoreToStg
doCodeGen
STG to Cmm
cmmToRawCmm
codeOutput
outputAsm
Cmm to Asm
nativeCodeGen[ghc/Main.hs][ghc/Main.hs][compiler/GHC/Driver/Pipeline.hs][compiler/GHC/Driver/Pipeline.hs][compiler/GHC/Driver/Pipeline.hs][compiler/GHC/Driver/Pipeline.hs][compiler/GHC/Driver/Pipeline.hs][compiler/GHC/Driver/Pipeline.hs][compiler/GHC/Driver/Pipeline.hs][compiler/GHC/Driver/Main.hs][compiler/GHC/Driver/Main.hs][compiler/GHC/Driver/Main.hs][compiler/GHC/Driver/Main.hs][compiler/GHC/Parser.y][compiler/GHC/Driver/Main.hs][compiler/GHC/Tc/Module.hs][compiler/GHC/Tc/Module.hs][compiler/GHC/Tc/Module.hs][compiler/GHC/Tc/Module.hs][compiler/GHC/Tc/Module.hs][compiler/GHC/Tc/Module.hs][compiler/GHC/Driver/Main.hs][compiler/GHC/Driver/Main.hs][compiler/GHC/HsToCore.hs][compiler/GHC/Driver/Main.hs][compiler/GHC/Core/Opt/Pipeline.hs][compiler/GHC/Driver/Main.hs][compiler/GHC/CoreToStg/Prep.hs][compiler/GHC/Driver/Main.hs][compiler/GHC/Driver/Main.hs][compiler/GHC/Cmm/Info.hs][compiler/GHC/Driver/CodeOutput.hs][compiler/GHC/Driver/CodeOutput.hs][compiler/GHC/CmmToAsm.hs]

Appendix

Appendix

Dump intermediate languages
Dump parser output:$ ghc -ddump-parsed$ ghc -ddump-parsed-ast
Dump renamer output:$ ghc -ddump-rn$ ghc -ddump-rn-ast
Dump type-checker output:$ ghc -ddump-tc$ ghc -ddump-tc-ast:

Dump Core:$ ghc -ddump-ds-preopt$ ghc -ddump-ds$ ghc -ddump-simpl$ ghc -ddump-prep:
Dump STG:$ ghc -ddump-stg-final:
Dump Cmm:$ ghc -ddump-cmm$ ghc -ddump-opt-cmm:
Dump asm/llvm:$ ghc -ddump-asm$ ghc -ddump-llvm

Each intermediate language can be dumped using ghc’s flags.
See the user's guide in detail :https://downloads.haskell.org/~ghc/latest/docs/html/users_guide/debugging.html#dumping-out-compiler-intermediate-structures

Appendix

Additional useful flags for dumps
Use ghc’s flags when you need more detailed information :-fprint-explicit-kinds-fprint-explicit-coercions-fprint-typechecker-elaboration-fprint-explicit-runtime-reps:: Print out kind applications: Print out details of coercions: Print out extra gubbins the type-checker inserts: Don't simplify away RuntimeRep arguments

See the user's guide in detail :https://downloads.haskell.org/~ghc/latest/docs/html/users_guide/using.html#verbosity-options

Use ghc’s flags when you want to suppress some information :-dsuppress-module-prefixes-dsuppress-coercions-dsuppress-uniques-dsuppress-type-applications:: Suppress the printing of module qualification prefixes: Suppress the printing of coercions: Suppress the printing of uniques: Suppress type applications

See the user's guide in detail :https://downloads.haskell.org/~ghc/latest/docs/html/users_guide/debugging.html#suppressing-unwanted-information

Appendix

Patches are good entrances to dive into GHC
Merge Requests (Pull requests) :https://gitlab.haskell.org/ghc/ghc/-/merge_requests

The merge-requests page is a mine of practical codes.

ReferencesReferences

References

Overview:
The Architecture of Open Source Applications: The Glasgow Haskell Compiler
https://www.aosabook.org/en/ghc.html
GHC Commentary: The Compiler
https://gitlab.haskell.org/ghc/ghc/-/wikis/commentary/compiler
Compiling one module: GHC.Driver.Main
https://gitlab.haskell.org/ghc/ghc/-/wikis/commentary/compiler/hsc-main
A Haskell Compiler
https://www.scs.stanford.edu/11au-cs240h/notes/ghc-slides.html
Dive into GHC
https://www.stephendiehl.com/posts/ghc_01.html
Write a GHC extension in 30 minutes
https://www.youtube.com/watch?v=bhhE2DxbrJM
The GHC Commentary
https://gitlab.haskell.org/ghc/ghc/-/wikis/commentary
The GHC reading list
https://gitlab.haskell.org/ghc/ghc/-/wikis/reading-list

References

References

Parser:The Parser
https://gitlab.haskell.org/ghc/ghc/-/wikis/commentary/compiler/parser
Syntactic ambiguity resolution in the GHC parser
https://blog.shaynefletcher.org/2020/04/syntactic-ambiguity-resolution-in-ghc.htmlThe HsSyn types
https://gitlab.haskell.org/ghc/ghc/-/wikis/commentary/compiler/hs-syn-type

Renamer:The renamer
https://gitlab.haskell.org/ghc/ghc/-/wikis/commentary/compiler/renamer
The Name type
https://gitlab.haskell.org/ghc/ghc/-/wikis/commentary/compiler/name-type

Type checker:
The GHC Commentary: Checking Types
https://gitlab.haskell.org/ghc/ghc/-/wikis/commentary/compiler/type-checker
The GHC reading list: Types and type inference
https://gitlab.haskell.org/ghc/ghc/-/wikis/reading-list#types-and-type-inferenceReferences

References

Desugarer, Core:
Into the Core - Squeezing Haskell into Nine Constructors
https://www.youtube.com/watch?v=uR_VzYxvbxg
https://www.erlang-factory.com/static/upload/media/1488806820775921euc2016intothecoresimonpeytonjones.pdf
The Core type
https://gitlab.haskell.org/ghc/ghc/-/wikis/commentary/compiler/core-syn-type
Core-to-Core optimization pipeline
https://gitlab.haskell.org/ghc/ghc/-/wikis/commentary/compiler/core-to-core-pipeline

System FC, as implemented in GHC
https://gitlab.haskell.org/ghc/ghc/blob/master/docs/core-spec/core-spec.pdf
The GHC reading list: Optimisations
https://gitlab.haskell.org/ghc/ghc/-/wikis/reading-list#optimisations
Haskell to Core: Understanding Haskell Features Through Their Desugaring
https://serokell.io/blog/haskell-to-coreReferences

References

STG, Code generator:
Implementing lazy functional languages on stock hardware: the Spineless Tagless G-machine Version 2.5
https://www.microsoft.com/en-us/research/wp-content/uploads/1992/04/spineless-tagless-gmachine.pdf
Making a Fast Curry: Push/Enter vs. Eval/Apply for Higher-order Languages
https://www.microsoft.com/en-us/research/wp-content/uploads/2016/07/eval-apply.pdf
Faster Laziness Using Dynamic Pointer Tagging
https://simonmar.github.io/bib/papers/ptr-tagging.pdf
The STG syntax data types
https://gitlab.haskell.org/ghc/ghc/-/wikis/commentary/compiler/stg-syn-type
I know kung fu: learning STG by example
https://gitlab.haskell.org/ghc/ghc/-/wikis/commentary/compiler/generated-code
Overview of GHC's code generator
https://gitlab.haskell.org/ghc/ghc/-/wikis/commentary/compiler/code-gen/overview
GHC illustrated
https://takenobu-hs.github.io/downloads/haskell_ghc_illustrated.pdf

References

ReferencesCmm:
I know kung fu: learning STG by example
https://gitlab.haskell.org/ghc/ghc/-/wikis/commentary/compiler/generated-code
Cmm syntax
https://gitlab.haskell.org/ghc/ghc/-/wikis/commentary/compiler/cmm-syntax
cmm type [outdated]
https://gitlab.haskell.org/ghc/ghc/-/wikis/commentary/compiler/cmm-type
The C-- Language Specification Version 2.0
https://www.cs.tufts.edu/~nr/c--/extern/man2.pdf
Understanding the RealWorld
https://www.well-typed.com/blog/95/

Native/LLVM code generator:
Native Code Generator (NCG) [outdated]
https://gitlab.haskell.org/ghc/ghc/-/wikis/commentary/compiler/backends/ncg
The LLVM backend
https://gitlab.haskell.org/ghc/ghc/-/wikis/commentary/compiler/backends/llvm
Low Level Virtual Machine for Glasgow Haskell Compiler
https://llvm.org/pubs/2009-10-TereiThesis.pdf

Happy haskelling!

Here is the slide: https://github.com/takenobu-hs/haskell-ghc-reading-guide