Repository: openllb/hlb
Pull request #286 (2022-02-05 05:28:56)

Decouple debugger from TUI
## Features
- Clean separation of concerns: `codegen.Debugger` doesn't know about stdin, stdout or input steering. `debug.TUIFrontend` only needs to deal with user interactions.
- Adds TUI as the first frontend for debugger
- Solves each step as its yielded (non-fs types return noop solve requests that solve with no error)
- Creates possibility of remote attach / multiple tui attaching / DAP attaching to debugger
- Full test suite

## Implementation notes
- Introduces interface `ast.StopNode` which are nodes that debugger should potentially halt on. These are: `*ast.FuncSignature`, `*ast.CallStmt` and `*ast.CallExpr`. `(ast.StopNode).Subject()` returns the node that should be the subject of a breakpoint, since we shouldn't highlight the entire command which may be multi-line.
- Breakpoint no longer `fs` or `option::run` builtin, but rather keyword that is ignored by checker & codegen. The debugger parses these to create breakpoints.
- Scopes now have levels (categorization), to make it easy to access nodes of a specific scope, or to compare apples to apples. For example, with scope levels you can compare two `*codegen.State` to see if they live inside the same block scope. Or you can quickly access its parent `*ast.Module`.
- `shell-on-error` removed in favor of the debugger being able to catch `*errdefs.SolveError` as a regular runtime exception. Users can choose to reverse step from there, exec into the exception state as many times as they like. In the future, when debug attach is available, consider additional TUI commands to exec into the same process, or start multiple processes.
- `(*RunInfo).ControlDebugger` is a new callback that gives programmatic access to the `codegen.Debugger` interface. See `codegen/debugger_tests.go` to see how this can be used.
- `debug.ParseLinespec` based on delve linespec: https://github.com/go-delve/delve/blob/master/Documentation/cli/locspec.md

## Design
- When codegen executes and a debugger is attached, codegen will yield to give a chance for the debugger to decide to halt. Debugger may choose to halt at the start of the program, based on movement mode selected by the user (e.g. next, step, continue, etc), or based on source-defined breakpoints or breakpoints created by the user.
- Every time codegen yields to debugger, the debugger also takes a snapshot of the context, scope, node, return register value, etc. Older snapshots are not used unless it is instructed to make reverse movements (`rev step`, `rev next`, `rev continue`, `restart`). Instead of trying to run codegen backwards, the debugger continues to block on the last yield and simulates reversal by playing back from its snapshots. To the user, it looks and feels like codegen running backwards.
- Note this snapshotting, and reverse execution is identical in design to the old implementation.
- Control of the debugger is based on one `sync.Mutex`. It is locked when the debugger is created and only unlocked when waiting for instruction. When a client instructs how to continue, the mutex is locked again. `(codegen.Debugger).GetState` is blocked until the debugger is available to accept instructions, leading to a clean interface in the TUI (and DAP server later on).
- When exceptions are hit (like a diagnostic error), a "snapshot" is created for it so users can step back and forth like any other, or issue other commands to inspect the state.

### How `errdefs.SolveError` is handled
- BuildKit's `errdefs.SolveError` are returned after codegen is complete and the solve request is sent off to BuildKit.
- When captured, codegen's error handler will yield to the debugger with that error, creating a new snapshot.
- Since `errdefs.SolveError` is a snapshot just like any other yield from codegen, the `codegen.Debugger` interface only has one `Exec` method. Internally it distinguishes based on the snapshot's `Err` containing an `errdefs.SolveError` to choose between `ExecWithFS` and `ExecWithSolveErr`, but clients of the debugger need not worry about the difference.

hinshun: Superseded by #286 and #288

Patch:
diff --git a/builtin/builtin.go b/builtin/builtin.go
index 0943f9d..962a724 100644
--- a/builtin/builtin.go
+++ b/builtin/builtin.go
@@ -28,7 +28,7 @@ func initSources() (err error) {
 	Module, err = parser.Parse(ctx, &parser.NamedReader{
 		Reader: strings.NewReader(Reference),
 		Value:  "<builtin>",
-	})
+	}, filebuffer.WithEphemeral())
 	if err != nil {
 		return errors.Wrapf(err, "failed to initialize filebuffer for builtins")
 	}
diff --git a/builtin/lookup.go b/builtin/lookup.go
index e7f77ca..f3fff58 100644
--- a/builtin/lookup.go
+++ b/builtin/lookup.go
@@ -205,12 +205,6 @@ var (
 						},
 						Effects: []*ast.Field{},
 					},
-					"breakpoint": {
-						Params: []*ast.Field{
-							ast.NewField(ast.String, "command", true),
-						},
-						Effects: []*ast.Field{},
-					},
 				},
 			},
 			"option::copy": {
@@ -539,12 +533,6 @@ var (
 							ast.NewField(ast.Filesystem, "target", false),
 						},
 					},
-					"breakpoint": {
-						Params: []*ast.Field{
-							ast.NewField(ast.String, "command", true),
-						},
-						Effects: []*ast.Field{},
-					},
 				},
 			},
 			"option::secret": {
@@ -916,17 +904,6 @@ option::run secret(string localPath, string mountPoint)
 # @return an option to mount an additional filesystem.
 option::run mount(fs input, string mountPoint) binds (fs target)
 
-# Sets a debugger breakpoint just before the &#34;run&#34; command.
-#
-# @param command optional command to invoke for the debugger &#34;exec&#34; command,
-# followed by optional arguments
-# If no value is given, it will execute /bin/sh.
-# If exactly one value is given, the specified command will be invoked.
-# If more than one value is given, values after the first one are treated as
-# arguments for the command.
-# @return an option to set a breakpoint
-option::run breakpoint(variadic string command)
-
 # Sets the target directory to mount the SSH agent socket. By default, it is
 # mounted to &#34;/run/buildkit/ssh_agent.${N}&#34;, where N is the index of the 
 # socket. If $SSH_AUTH_SOCK is not set, it will set SSH_AUTH_SOCK to the
@@ -1397,16 +1374,5 @@ option::template stringField(string name, string value)
 # @return a pipeline that returns when all its targets have finished.
 pipeline stage(variadic pipeline pipelines)
 
-# Sets a static breakpoint in the debugger.
-#
-# @param command optional command to invoke for the debugger &#34;exec&#34; command at
-# this breakpoint, followed by optional arguments
-# If no value is given, it will execute /bin/sh.
-# If exactly one value is given, the specified command will be invoked.
-# If more than one value is given, values after the first one are treated as
-# arguments for the command.
-# @return the filesystem after the breakpoint
-fs breakpoint(variadic string command)
-
 `
 )
diff --git a/checker/builtin.go b/checker/builtin.go
index 9873e3e..342c346 100644
--- a/checker/builtin.go
+++ b/checker/builtin.go
@@ -15,7 +15,7 @@ const (
 // NewBuiltinScope returns a new scope containing synthetic FuncDecl Objects for
 // builtins.
 func NewBuiltinScope(builtins builtin.BuiltinLookup) *ast.Scope {
-	scope := ast.NewScope(nil, nil)
+	scope := ast.NewScope(nil, ast.BuiltinScope, builtin.Module)
 	ast.Match(builtin.Module, ast.MatchOpts{},
 		func(fd *ast.FuncDecl) {
 			obj := scope.Lookup(fd.Sig.Name.Text)
diff --git a/checker/checker.go b/checker/checker.go
index 809bd2e..d4ca447 100644
--- a/checker/checker.go
+++ b/checker/checker.go
@@ -53,7 +53,7 @@ func (c *checker) SemanticPass(mod *ast.Module) error {
 	//
 	// A module scope is a child of the global scope where builtin functions are
 	// defined.
-	mod.Scope = ast.NewScope(mod, GlobalScope)
+	mod.Scope = ast.NewScope(GlobalScope, ast.ModuleScope, mod)
 
 	// (1) Build lexical scopes and memoize semantic data into the CST.
 	ast.Match(mod, ast.MatchOpts{},
@@ -74,7 +74,7 @@ func (c *checker) SemanticPass(mod *ast.Module) error {
 			}
 
 			// Create a lexical scope for this function.
-			fd.Scope = ast.NewScope(fd, mod.Scope)
+			fd.Scope = ast.NewScope(mod.Scope, ast.FunctionScope, fd)
 
 			if fd.Sig.Params != nil {
 				// Create entries for call parameters to the function. Later at code
@@ -380,6 +380,9 @@ func (c *checker) checkType(node ast.Node, kset *ast.KindSet, actual ast.Kind, o
 }
 
 func (c *checker) checkCallStmt(scope *ast.Scope, kset *ast.KindSet, call *ast.CallStmt) error {
+	if call.Breakpoint() {
+		return nil
+	}
 	signature, err := c.checkCall(scope, kset, call.Name, call.Args, call.WithClause)
 	if err != nil {
 		return err
@@ -393,6 +396,9 @@ func (c *checker) checkCallStmt(scope *ast.Scope, kset *ast.KindSet, call *ast.C
 }
 
 func (c *checker) checkCallExpr(scope *ast.Scope, kset *ast.KindSet, call *ast.CallExpr) error {
+	if call.Breakpoint() {
+		return nil
+	}
 	signature, err := c.checkCall(scope, kset, call.Name, call.Args(), nil)
 	if err != nil {
 		return err
diff --git a/cmd/hlb/command/langserver.go b/cmd/hlb/command/langserver.go
index 2a45657..2bcf18e 100644
--- a/cmd/hlb/command/langserver.go
+++ b/cmd/hlb/command/langserver.go
@@ -11,7 +11,7 @@ import (
 
 var langserverCommand = &cli.Command{
 	Name:  "langserver",
-	Usage: "run hlp lsp language server",
+	Usage: "run hlb language server over stdio",
 	Flags: []cli.Flag{
 		&cli.StringFlag{
 			Name:  "logfile",
diff --git a/cmd/hlb/command/run.go b/cmd/hlb/command/run.go
index b2a44c9..54579ab 100644
--- a/cmd/hlb/command/run.go
+++ b/cmd/hlb/command/run.go
@@ -1,23 +1,21 @@
 package command
 
 import (
-	"bufio"
 	"context"
 	"errors"
 	"fmt"
 	"io"
 	"os"
 	"strings"
-	"sync"
 
 	"github.com/mattn/go-isatty"
 	"github.com/moby/buildkit/client"
-	gateway "github.com/moby/buildkit/frontend/gateway/client"
 	solvererrdefs "github.com/moby/buildkit/solver/errdefs"
 	specs "github.com/opencontainers/image-spec/specs-go/v1"
 	"github.com/openllb/hlb"
 	"github.com/openllb/hlb/builtin"
 	"github.com/openllb/hlb/codegen"
+	"github.com/openllb/hlb/codegen/debug"
 	"github.com/openllb/hlb/diagnostic"
 	"github.com/openllb/hlb/errdefs"
 	"github.com/openllb/hlb/local"
@@ -27,6 +25,7 @@ import (
 	"github.com/openllb/hlb/solver"
 	cli "github.com/urfave/cli/v2"
 	"github.com/xlab/treeprint"
+	"golang.org/x/sync/errgroup"
 )
 
 var (
@@ -46,11 +45,7 @@ var runCommand = &cli.Command{
 		},
 		&cli.BoolFlag{
 			Name:  "debug",
-			Usage: "jump into a source level debugger for hlb",
-		},
-		&cli.BoolFlag{
-			Name:  "shell-on-error",
-			Usage: "execute an interactive shell in the build container when an error occurs",
+			Usage: "attach a debugger",
 		},
 		&cli.BoolFlag{
 			Name:  "tree",
@@ -83,70 +78,43 @@ var runCommand = &cli.Command{
 			return err
 		}
 
-		ri := RunInfo{
-			DisplayErrorOnce: &sync.Once{},
-			Tree:             c.Bool("tree"),
-			Targets:          c.StringSlice("target"),
-			LLB:              c.Bool("llb"),
-			Backtrace:        c.Bool("backtrace"),
-			LogOutput:        c.String("log-output"),
-			ErrOutput:        os.Stderr,
-			Output:           os.Stdout,
-			DefaultPlatform:  c.String("platform"),
-		}
-
-		var inputSteerer *steer.InputSteerer
-		if c.Bool("debug") {
-			pr, pw := io.Pipe()
-			r := bufio.NewReader(pr)
-			inputSteerer = steer.NewInputSteerer(os.Stdin, pw)
-
-			ri.Debugger = codegen.NewDebugger(cln, os.Stderr, inputSteerer, r)
-		}
-
-		if c.Bool("shell-on-error") {
-			if inputSteerer == nil {
-				inputSteerer = steer.NewInputSteerer(os.Stdin)
-			}
-
-			ri.SolveErrorHandler = func(ctx context.Context, c gateway.Client, err error) {
-				var se *solvererrdefs.SolveError
-				if !errors.As(err, &se) {
-					return
-				}
-
-				ri.DisplayErrorOnce.Do(func() {
-					DisplayError(ctx, ri, err)
-				})
-
-				pr, pw := io.Pipe()
-				defer pw.Close()
-
-				inputSteerer.Push(pw)
-				defer inputSteerer.Pop()
+		return Run(ctx, cln, rc, RunInfo{
+			Tree:            c.Bool("tree"),
+			Targets:         c.StringSlice("target"),
+			LLB:             c.Bool("llb"),
+			Backtrace:       c.Bool("backtrace"),
+			LogOutput:       c.String("log-output"),
+			DefaultPlatform: c.String("platform"),
+			Debug:           c.Bool("debug"),
+			ControlDebugger: ControlDebuggerTUI(os.Stdin, os.Stdout, os.Stderr),
+		})
+	},
+}
 
-				if err := codegen.ExecWithSolveErr(ctx, c, se, pr, ri.Output, nil); err != nil {
-					fmt.Fprintf(ri.ErrOutput, "failed to exec debug shell after error: %s\n", err)
-				}
-			}
-		}
+type ControlDebugger func(context.Context, codegen.Debugger) error
 
-		return Run(ctx, cln, rc, ri)
-	},
+func ControlDebuggerTUI(stdin io.Reader, stdout, stderr io.Writer) ControlDebugger {
+	return func(ctx context.Context, dbgr codegen.Debugger) error {
+		pr, pw := io.Pipe()
+		is := steer.NewInputSteerer(stdin, pw)
+		return debug.TUIFrontend(ctx, dbgr, is, pr, stdout, stderr)
+	}
 }
 
 type RunInfo struct {
-	Debugger          codegen.Debugger
-	SolveErrorHandler func(context.Context, gateway.Client, error)
-	DisplayErrorOnce  *sync.Once
-	Tree              bool
-	Backtrace         bool
-	Targets           []string
-	LLB               bool
-	LogOutput         string
-	ErrOutput         solver.Console
-	Output            io.Writer
-	DefaultPlatform   string // format: osname/osarch
+	Tree            bool
+	Backtrace       bool
+	Targets         []string
+	LLB             bool
+	LogOutput       string
+	DefaultPlatform string // format: osname/osarch
+
+	Stdin  io.Reader
+	Stderr io.Writer
+	Stdout io.Writer
+
+	Debug           bool
+	ControlDebugger ControlDebugger
 
 	// override defaults sources as necessary
 	Environ []string
@@ -159,8 +127,14 @@ func Run(ctx context.Context, cln *client.Client, rc io.ReadCloser, info RunInfo
 	if len(info.Targets) == 0 {
 		info.Targets = []string{"default"}
 	}
-	if info.Output == nil {
-		info.Output = os.Stdout
+	if info.Stdin == nil {
+		info.Stdin = os.Stdin
+	}
+	if info.Stdout == nil {
+		info.Stdout = os.Stdout
+	}
+	if info.Stderr == nil {
+		info.Stderr = os.Stderr
 	}
 
 	ctx = local.WithEnviron(ctx, info.Environ)
@@ -178,28 +152,32 @@ func Run(ctx context.Context, cln *client.Client, rc io.ReadCloser, info RunInfo
 		ctx = codegen.WithDefaultPlatform(ctx, specs.Platform{OS: platformParts[0], Architecture: platformParts[1]})
 	}
 
-	var progressOpts []solver.ProgressOption
+	var (
+		progressOpts []solver.ProgressOption
+		con          solver.Console
+	)
 	if info.LogOutput == "" || info.LogOutput == "auto" {
-		// assume plain output, will upgrade if we detect tty
+		// Assume plain output, will upgrade if we detect tty.
 		info.LogOutput = "plain"
-		if fdAble, ok := info.Output.(interface{ Fd() uintptr }); ok {
-			if isatty.IsTerminal(fdAble.Fd()) {
-				info.LogOutput = "tty"
-			}
+
+		var ok bool
+		con, ok = info.Stderr.(solver.Console)
+		if ok && isatty.IsTerminal(con.Fd()) {
+			info.LogOutput = "tty"
 		}
 	}
 
 	// Always force plain output in debug mode so the prompts are displayed
 	// correctly
-	if info.Debugger != nil {
+	if info.Debug {
 		info.LogOutput = "plain"
 	}
 
 	switch info.LogOutput {
 	case "tty":
-		progressOpts = append(progressOpts, solver.WithLogOutput(info.ErrOutput, solver.LogOutputTTY))
+		progressOpts = append(progressOpts, solver.WithLogOutputTTY(con))
 	case "plain":
-		progressOpts = append(progressOpts, solver.WithLogOutput(info.ErrOutput, solver.LogOutputPlain))
+		progressOpts = append(progressOpts, solver.WithLogOutputPlain(info.Stderr))
 	default:
 		return fmt.Errorf("unrecognized log-output %q", info.LogOutput)
 	}
@@ -208,6 +186,7 @@ func Run(ctx context.Context, cln *client.Client, rc io.ReadCloser, info RunInfo
 	if err != nil {
 		return err
 	}
+
 	// store Progress in context in case we need to synchronize output later
 	ctx = codegen.WithProgress(ctx, p)
 	ctx = codegen.WithMultiWriter(ctx, p.MultiWriter())
@@ -217,18 +196,7 @@ func Run(ctx context.Context, cln *client.Client, rc io.ReadCloser, info RunInfo
 		if err == nil {
 			return
 		}
-		info.DisplayErrorOnce.Do(func() {
-			DisplayError(ctx, info, err)
-		})
-
-		numErrs := 1
-		backtrace := diagnostic.Backtrace(ctx, err)
-		if len(backtrace) == 0 {
-			// Handle diagnostic errors.
-			spans := diagnostic.Spans(err)
-			numErrs = len(spans)
-		}
-
+		numErrs := displayError(ctx, info.Stderr, err, info.Backtrace)
 		err = errdefs.WithAbort(err, numErrs)
 	}()
 
@@ -244,22 +212,26 @@ func Run(ctx context.Context, cln *client.Client, rc io.ReadCloser, info RunInfo
 
 	ctx = codegen.WithImageResolver(ctx, codegen.NewCachedImageResolver(cln))
 
-	var opts []codegen.CodeGenOption
-	if info.Debugger != nil {
-		opts = append(opts, codegen.WithDebugger(info.Debugger))
-	}
+	g, ctx := errgroup.WithContext(ctx)
 
-	var solveOpts []solver.SolveOption
-	if info.SolveErrorHandler != nil {
-		solveOpts = append(solveOpts, solver.WithEvaluate, solver.WithErrorHandler(info.SolveErrorHandler))
-		opts = append(opts, codegen.WithExtraSolveOpts(solveOpts))
+	var dbgr codegen.Debugger
+	if info.Debug {
+		dbgr = codegen.NewDebugger(cln)
+		ctx = codegen.WithDebugger(ctx, dbgr)
+		ctx = codegen.WithGlobalSolveOpts(ctx, solver.WithEvaluate)
+
+		if info.ControlDebugger != nil {
+			g.Go(func() error {
+				return info.ControlDebugger(ctx, dbgr)
+			})
+		}
 	}
 
-	solveReq, err := hlb.Compile(ctx, cln, mod, targets, opts...)
+	solveReq, err := hlb.Compile(ctx, cln, mod, targets)
 	if err != nil {
 		perr := p.Wait()
 		// Ignore early exits from the debugger.
-		if err == codegen.ErrDebugExit {
+		if errors.Is(err, codegen.ErrDebugExit) {
 			return perr
 		}
 		return err
@@ -287,53 +259,19 @@ func Run(ctx context.Context, cln *client.Client, rc io.ReadCloser, info RunInfo
 		return nil
 	}
 
-	defer p.Wait()
-	return solveReq.Solve(ctx, cln, p.MultiWriter(), solveOpts...)
-}
-
-func DisplayError(ctx context.Context, info RunInfo, err error) {
-	// Handle backtrace.
-	backtrace := diagnostic.Backtrace(ctx, err)
-	if len(backtrace) > 0 {
-		color := diagnostic.Color(ctx)
-		fmt.Fprintf(info.ErrOutput, color.Sprintf(
-			"%s: %s\n",
-			color.Bold(color.Red("error")),
-			color.Bold(diagnostic.Cause(err)),
-		))
-	}
-	for i, span := range backtrace {
-		if !info.Backtrace && i != len(backtrace)-1 {
-			if i == 0 {
-				color := diagnostic.Color(ctx)
-				frame := "frame"
-				if len(backtrace) > 2 {
-					frame = "frames"
-				}
-				fmt.Fprintf(info.ErrOutput, color.Sprintf(color.Cyan(" ⫶ %d %s hidden ⫶\n"), len(backtrace)-1, frame))
-			}
-			continue
+	g.Go(func() error {
+		defer p.Wait()
+		if dbgr != nil {
+			defer dbgr.Close()
 		}
+		return solveReq.Solve(ctx, cln, p.MultiWriter())
+	})
 
-		pretty := span.Pretty(ctx, diagnostic.WithNumContext(2))
-		lines := strings.Split(pretty, "\n")
-		for j, line := range lines {
-			if j == 0 {
-				lines[j] = fmt.Sprintf(" %d: %s", i+1, line)
-			} else {
-				lines[j] = fmt.Sprintf("    %s", line)
-			}
-		}
-		fmt.Fprintf(info.ErrOutput, "%s\n", strings.Join(lines, "\n"))
-	}
-
-	if len(backtrace) == 0 {
-		// Handle diagnostic errors.
-		spans := diagnostic.Spans(err)
-		for _, span := range spans {
-			fmt.Fprintf(info.ErrOutput, "%s\n", span.Pretty(ctx))
-		}
+	err = g.Wait()
+	if errors.Is(err, codegen.ErrDebugExit) {
+		return nil
 	}
+	return err
 }
 
 func ModuleReadCloser(args []string) (io.ReadCloser, error) {
@@ -354,3 +292,17 @@ func ModuleReadCloser(args []string) (io.ReadCloser, error) {
 
 	return os.Open(args[0])
 }
+
+func displayError(ctx context.Context, w io.Writer, err error, printBacktrace bool) (numErrs int) {
+	spans := diagnostic.SourcesToSpans(ctx, solvererrdefs.Sources(err), err)
+	if len(spans) > 0 {
+		diagnostic.DisplayError(ctx, w, spans, err, printBacktrace)
+		return 1
+	}
+
+	// Handle diagnostic errors.
+	for _, span := range diagnostic.Spans(err) {
+		fmt.Fprintln(w, span.Pretty(ctx))
+	}
+	return len(spans)
+}
diff --git a/codegen/builtin.go b/codegen/builtin.go
index 422348b..1c85f4b 100644
--- a/codegen/builtin.go
+++ b/codegen/builtin.go
@@ -39,7 +39,6 @@ var (
 			"downloadTarball":       DownloadTarball{},
 			"downloadOCITarball":    DownloadOCITarball{},
 			"downloadDockerTarball": DownloadDockerTarball{},
-			"breakpoint":            SetBreakpoint{},
 		},
 		ast.String: {
 			"format":    Format{},
@@ -90,7 +89,6 @@ var (
 			"forward":        Forward{},
 			"secret":         Secret{},
 			"mount":          Mount{},
-			"breakpoint":     RunBreakpoint{},
 		},
 		"option::ssh": {
 			"target":     MountTarget{},
diff --git a/codegen/builtin_option.go b/codegen/builtin_option.go
index e64395b..cb2e80c 100644
--- a/codegen/builtin_option.go
+++ b/codegen/builtin_option.go
@@ -440,17 +440,6 @@ func (ru RunUser) Call(ctx context.Context, cln *client.Client, val Value, opts
 	return NewValue(ctx, append(retOpts, llbutil.WithUser(name)))
 }
 
-type RunBreakpoint struct{}
-
-func (rb RunBreakpoint) Call(ctx context.Context, cln *client.Client, val Value, opts Option, command ...string) (Value, error) {
-	retOpts, err := val.Option()
-	if err != nil {
-		return nil, err
-	}
-
-	return NewValue(ctx, append(retOpts, breakpointCommand(command)))
-}
-
 type IgnoreCache struct{}
 
 func (ig IgnoreCache) Call(ctx context.Context, cln *client.Client, val Value, opts Option) (Value, error) {
diff --git a/codegen/codegen.go b/codegen/codegen.go
index f05406f..81facf1 100644
--- a/codegen/codegen.go
+++ b/codegen/codegen.go
@@ -21,51 +21,29 @@ import (
 )
 
 type CodeGen struct {
-	Debug          Debugger
-	cln            *client.Client
-	resolver       Resolver
-	extraSolveOpts []solver.SolveOption
+	cln      *client.Client
+	resolver Resolver
+	dbgr     *debugger
 }
 
-type CodeGenOption func(*CodeGen) error
-
-func WithDebugger(dbgr Debugger) CodeGenOption {
-	return func(i *CodeGen) error {
-		i.Debug = dbgr
-		return nil
-	}
-}
-
-func WithExtraSolveOpts(extraOpts []solver.SolveOption) CodeGenOption {
-	return func(i *CodeGen) error {
-		i.extraSolveOpts = append(i.extraSolveOpts, extraOpts...)
-		return nil
-	}
-}
-
-func New(cln *client.Client, resolver Resolver, opts ...CodeGenOption) (*CodeGen, error) {
-	cg := &CodeGen{
-		Debug:    NewNoopDebugger(),
+func New(cln *client.Client, resolver Resolver) *CodeGen {
+	return &CodeGen{
 		cln:      cln,
 		resolver: resolver,
 	}
-	for _, opt := range opts {
-		err := opt(cg)
-		if err != nil {
-			return cg, err
-		}
-	}
-
-	return cg, nil
 }
 
 type Target struct {
 	Name string
 }
 
-func (cg *CodeGen) Generate(ctx context.Context, mod *ast.Module, targets []Target) (solver.Request, error) {
-	var requests []solver.Request
+func (cg *CodeGen) Generate(ctx context.Context, mod *ast.Module, targets []Target) (result solver.Request, err error) {
+	if GetDebugger(ctx) != nil {
+		cg.dbgr = GetDebugger(ctx).(*debugger)
+		ctx = WithGlobalSolveOpts(ctx, solver.WithErrorHandler(cg.errorHandler))
+	}
 
+	var requests []solver.Request
 	for i, target := range targets {
 		_, ok := mod.Scope.Objects[target.Name]
 		if !ok {
@@ -74,9 +52,11 @@ func (cg *CodeGen) Generate(ctx context.Context, mod *ast.Module, targets []Targ
 
 		// Yield before compiling anything.
 		ret := NewRegister(ctx)
-		err := cg.Debug(ctx, mod.Scope, mod, ret, nil)
-		if err != nil {
-			return nil, err
+		if cg.dbgr != nil {
+			err := cg.dbgr.yield(ctx, mod.Scope, mod, ret.Value(), nil, nil)
+			if err != nil {
+				return nil, err
+			}
 		}
 
 		// Build expression for target.
@@ -85,7 +65,7 @@ func (cg *CodeGen) Generate(ctx context.Context, mod *ast.Module, targets []Targ
 		ie.Pos.Line = i
 
 		// Every target has a return register.
-		err = cg.EmitIdentExpr(ctx, mod.Scope, ie, ie.Ident, nil, nil, nil, ret)
+		err := cg.EmitIdentExpr(ctx, mod.Scope, ie, ie.Ident, nil, nil, nil, ret)
 		if err != nil {
 			return nil, err
 		}
@@ -111,6 +91,15 @@ func (cg *CodeGen) EmitExpr(ctx context.Context, scope *ast.Scope, expr *ast.Exp
 		return cg.EmitBasicLit(ctx, scope, expr.BasicLit, ret)
 	case expr.CallExpr != nil:
 		return ret.SetAsync(func(v Value) (Value, error) {
+			if expr.CallExpr.Breakpoint() {
+				var err error
+				if cg.dbgr != nil {
+					ctx = WithFrame(ctx, Frame{expr.CallExpr.Name})
+					err = cg.dbgr.yield(ctx, scope, expr.CallExpr, v, nil, nil)
+				}
+				return v, err
+			}
+
 			err := cg.lookupCall(ctx, scope, expr.CallExpr.Name.Ident)
 			if err != nil {
 				return nil, err
@@ -257,14 +246,6 @@ func (cg *CodeGen) EmitRawHeredoc(ctx context.Context, scope *ast.Scope, heredoc
 }
 
 func (cg *CodeGen) EmitCallExpr(ctx context.Context, scope *ast.Scope, call *ast.CallExpr, ret Register) error {
-	ctx = WithFrame(ctx, Frame{call.Name})
-
-	// Yield before executing call expression.
-	err := cg.Debug(ctx, scope, call.Name, ret, nil)
-	if err != nil {
-		return err
-	}
-
 	args, err := cg.Evaluate(ctx, scope, nil, call.Signature, call.Args()...)
 	if err != nil {
 		return err
@@ -273,6 +254,16 @@ func (cg *CodeGen) EmitCallExpr(ctx context.Context, scope *ast.Scope, call *ast
 		ctx = WithArg(ctx, i, arg)
 	}
 
+	ctx = WithFrame(ctx, Frame{call.Name})
+
+	// Yield before executing call expression.
+	if cg.dbgr != nil {
+		err := cg.dbgr.yield(ctx, scope, call, ret.Value(), nil, nil)
+		if err != nil {
+			return err
+		}
+	}
+
 	return cg.EmitIdentExpr(ctx, scope, call.Name, call.Name.Ident, args, nil, nil, ret)
 }
 
@@ -390,7 +381,7 @@ func (cg *CodeGen) EmitImport(ctx context.Context, mod *ast.Module, id *ast.Impo
 	return
 }
 
-func (cg *CodeGen) EmitBuiltinDecl(ctx context.Context, scope *ast.Scope, bd *ast.BuiltinDecl, args []Value, opts Option, b *ast.Binding, v Value) (Value, error) {
+func (cg *CodeGen) EmitBuiltinDecl(ctx context.Context, scope *ast.Scope, bd *ast.BuiltinDecl, args []Value, opts Option, b *ast.Binding, val Value) (Value, error) {
 	var callable interface{}
 	if ReturnType(ctx) != ast.None {
 		callable = Callables[ReturnType(ctx)][bd.Name]
@@ -412,16 +403,12 @@ func (cg *CodeGen) EmitBuiltinDecl(ctx context.Context, scope *ast.Scope, bd *as
 		ctx = WithBinding(ctx, b)
 	}
 
-	for _, opt := range cg.extraSolveOpts {
-		opts = append(opts, opt)
-	}
-
 	var (
 		c   = reflect.ValueOf(callable).MethodByName("Call")
 		ins = []reflect.Value{
 			reflect.ValueOf(ctx),
 			reflect.ValueOf(cg.cln),
-			reflect.ValueOf(v),
+			reflect.ValueOf(val),
 			reflect.ValueOf(opts),
 		}
 	)
@@ -443,22 +430,22 @@ func (cg *CodeGen) EmitBuiltinDecl(ctx context.Context, scope *ast.Scope, bd *as
 			param = c.Type().In(i)
 			arg   = args[i-len(PrototypeIn)]
 		)
-		v, err := arg.Reflect(param)
+		rval, err := arg.Reflect(param)
 		if err != nil {
 			return nil, err
 		}
-		ins = append(ins, v)
+		ins = append(ins, rval)
 	}
 
 	// Reflect variadic arguments.
 	if c.Type().IsVariadic() {
 		for i := numIn - len(PrototypeIn); i < len(args); i++ {
 			param := c.Type().In(numIn).Elem()
-			v, err := args[i].Reflect(param)
+			rval, err := args[i].Reflect(param)
 			if err != nil {
 				return nil, err
 			}
-			ins = append(ins, v)
+			ins = append(ins, rval)
 		}
 	}
 
@@ -470,12 +457,23 @@ func (cg *CodeGen) EmitBuiltinDecl(ctx context.Context, scope *ast.Scope, bd *as
 			err = ProgramCounter(ctx).WithError(err)
 		}
 
-		return nil, WithBacktraceError(ctx, err)
+		err = WithBacktraceError(ctx, err)
+		if cg.dbgr != nil {
+			derr := cg.dbgr.yield(ctx, scope, ProgramCounter(ctx), val, nil, err)
+			if derr != nil {
+				return nil, derr
+			}
+		}
+		return nil, err
 	}
 	return outs[0].Interface().(Value), nil
 }
 
 func (cg *CodeGen) EmitFuncDecl(ctx context.Context, fd *ast.FuncDecl, args []Value, b *ast.Binding, ret Register) error {
+	if fd.Body == nil {
+		return nil
+	}
+
 	ctx = WithProgramCounter(ctx, fd.Sig.Name)
 
 	params := fd.Sig.Params.Fields()
@@ -487,7 +485,7 @@ func (cg *CodeGen) EmitFuncDecl(ctx context.Context, fd *ast.FuncDecl, args []Va
 		return errdefs.WithInternalErrorf(ProgramCounter(ctx), "`%s` expected %d args, got %d", name, len(params), len(args))
 	}
 
-	scope := ast.NewScope(fd, fd.Scope)
+	scope := ast.NewScope(fd.Body.Scope, ast.ArgsScope, fd)
 	for i, param := range params {
 		if param.Modifier != nil {
 			continue
@@ -501,10 +499,14 @@ func (cg *CodeGen) EmitFuncDecl(ctx context.Context, fd *ast.FuncDecl, args []Va
 		})
 	}
 
-	// Yield before executing a function.
-	err := cg.Debug(ctx, scope, fd.Sig.Name, ret, nil)
-	if err != nil {
-		return err
+	if cg.dbgr != nil {
+		// The frame for the function signature is only kept for this yield so don't
+		// assign it to ctx. Once the debugger steps after the function signature, we
+		// don't want it as part of the backtrace.
+		err := cg.dbgr.yield(WithFrame(ctx, Frame{fd.Sig.Name}), scope, fd.Sig, ret.Value(), nil, nil)
+		if err != nil {
+			return err
+		}
 	}
 
 	return cg.EmitBlock(ctx, scope, fd.Body, b, ret)
@@ -527,7 +529,7 @@ func (cg *CodeGen) lookupCall(ctx context.Context, scope *ast.Scope, lookup *ast
 			break
 		}
 
-		mod := scope.Root().Node.(*ast.Module)
+		mod := scope.ByLevel(ast.ModuleScope).Node.(*ast.Module)
 		imod, _, err := cg.EmitImport(ctx, mod, n)
 		if err != nil {
 			return err
@@ -547,6 +549,7 @@ func (cg *CodeGen) EmitBlock(ctx context.Context, scope *ast.Scope, block *ast.B
 	if block == nil {
 		return nil
 	}
+	scope = ast.NewScope(scope, ast.BlockScope, block)
 
 	ctx = WithReturnType(ctx, block.Kind())
 	for _, stmt := range block.Stmts() {
@@ -555,6 +558,15 @@ func (cg *CodeGen) EmitBlock(ctx context.Context, scope *ast.Scope, block *ast.B
 		switch {
 		case stmt.Call != nil:
 			err = ret.SetAsync(func(v Value) (Value, error) {
+				if stmt.Call.Breakpoint() {
+					var err error
+					if cg.dbgr != nil {
+						ctx = WithFrame(ctx, Frame{stmt.Call.Name})
+						err = cg.dbgr.yield(ctx, scope, stmt.Call, v, nil, nil)
+					}
+					return v, err
+				}
+
 				err := cg.lookupCall(ctx, scope, stmt.Call.Name.Ident)
 				if err != nil {
 					return nil, err
@@ -579,8 +591,6 @@ func (cg *CodeGen) EmitBlock(ctx context.Context, scope *ast.Scope, block *ast.B
 }
 
 func (cg *CodeGen) EmitCallStmt(ctx context.Context, scope *ast.Scope, call *ast.CallStmt, b *ast.Binding, ret Register) error {
-	ctx = WithFrame(ctx, Frame{call.Name})
-
 	args, err := cg.Evaluate(ctx, scope, nil, call.Signature, call.Args...)
 	if err != nil {
 		return err
@@ -596,8 +606,15 @@ func (cg *CodeGen) EmitCallStmt(ctx context.Context, scope *ast.Scope, call *ast
 			ast.Kind(fmt.Sprintf("%s::%s", ast.Option, call.Name)),
 		}
 
+		// If with clause is a call expr, still wrap the scope as if it was a single
+		// element option block.
+		escope := scope
+		if call.WithClause.Expr.CallExpr != nil {
+			escope = ast.NewScope(scope, ast.BlockScope, call.WithClause.Expr.CallExpr)
+		}
+
 		// WithClause provides option expressions access to the binding.
-		values, err := cg.Evaluate(ctx, scope, b, kinds, call.WithClause.Expr)
+		values, err := cg.Evaluate(ctx, escope, b, kinds, call.WithClause.Expr)
 		if err != nil {
 			return err
 		}
@@ -608,27 +625,15 @@ func (cg *CodeGen) EmitCallStmt(ctx context.Context, scope *ast.Scope, call *ast
 		}
 	}
 
+	ctx = WithFrame(ctx, Frame{call.Name})
+
 	// Yield before executing the next call statement.
-	if call.Breakpoint(ReturnType(ctx)) {
-		var command []string
-		for _, arg := range args {
-			if arg.Kind() != ast.String {
-				return errors.New("breakpoint args must be strings")
-			}
-			argStr, err := arg.String()
-			if err != nil {
-				return err
-			}
-			command = append(command, argStr)
-		}
-		if len(command) != 0 {
-			opts = append(opts, breakpointCommand(command))
+	if cg.dbgr != nil {
+		err := cg.dbgr.yield(ctx, scope, call, ret.Value(), opts, nil)
+		if err != nil {
+			return err
 		}
 	}
-	err = cg.Debug(ctx, scope, call.Name, ret, opts)
-	if err != nil {
-		return err
-	}
 
 	// Pass the binding if this is the matching CallStmt.
 	var binding *ast.Binding
@@ -639,8 +644,6 @@ func (cg *CodeGen) EmitCallStmt(ctx context.Context, scope *ast.Scope, call *ast
 	return cg.EmitIdentExpr(ctx, scope, call.Name, call.Name.Ident, args, opts, binding, ret)
 }
 
-type breakpointCommand []string
-
 func (cg *CodeGen) Evaluate(ctx context.Context, scope *ast.Scope, b *ast.Binding, kinds []ast.Kind, exprs ...*ast.Expr) (values []Value, err error) {
 	if len(kinds) != len(exprs) {
 		return nil, errdefs.WithInternalErrorf(ProgramCounter(ctx), "expected %d kinds but got %d", len(exprs), len(kinds))
diff --git a/codegen/context.go b/codegen/context.go
index 361e84b..fc415d4 100644
--- a/codegen/context.go
+++ b/codegen/context.go
@@ -12,6 +12,7 @@ import (
 	"github.com/moby/buildkit/solver/errdefs"
 	"github.com/moby/buildkit/solver/pb"
 	specs "github.com/opencontainers/image-spec/specs-go/v1"
+	"github.com/openllb/hlb/diagnostic"
 	"github.com/openllb/hlb/parser/ast"
 	"github.com/openllb/hlb/pkg/filebuffer"
 	"github.com/openllb/hlb/pkg/llbutil"
@@ -21,17 +22,19 @@ import (
 )
 
 type (
-	programCounterKey struct{}
-	returnTypeKey     struct{}
-	argKey            struct{ n int }
-	bindingKey        struct{}
-	sessionIDKey      struct{}
-	multiwriterKey    struct{}
-	imageResolverKey  struct{}
-	backtraceKey      struct{}
-	progressKey       struct{}
-	platformKey       struct{}
-	dockerAPIKey      struct{}
+	programCounterKey  struct{}
+	returnTypeKey      struct{}
+	argKey             struct{ n int }
+	bindingKey         struct{}
+	sessionIDKey       struct{}
+	multiwriterKey     struct{}
+	imageResolverKey   struct{}
+	backtraceKey       struct{}
+	progressKey        struct{}
+	platformKey        struct{}
+	dockerAPIKey       struct{}
+	debuggerKey        struct{}
+	globalSolveOptsKey struct{}
 )
 
 func WithProgramCounter(ctx context.Context, node ast.Node) context.Context {
@@ -129,14 +132,9 @@ func WithFrame(ctx context.Context, frame Frame) context.Context {
 	return context.WithValue(ctx, backtraceKey{}, frames)
 }
 
-func Backtrace(ctx context.Context) []Frame {
-	frames, _ := ctx.Value(backtraceKey{}).([]Frame)
-	return frames
-}
-
-func WithBacktraceError(ctx context.Context, err error) error {
-	for _, frame := range Backtrace(ctx) {
-		err = errdefs.WithSource(err, errdefs.Source{
+func FramesToSources(frames []Frame) (sources []*errdefs.Source) {
+	for _, frame := range frames {
+		sources = append(sources, &errdefs.Source{
 			Info: &pb.SourceInfo{
 				Filename: frame.Position().Filename,
 			},
@@ -146,7 +144,22 @@ func WithBacktraceError(ctx context.Context, err error) error {
 			}},
 		})
 	}
+	return
+}
+
+func FramesToSpans(ctx context.Context, frames []Frame) []*diagnostic.SpanError {
+	return diagnostic.SourcesToSpans(ctx, FramesToSources(frames), nil)
+}
+
+func Backtrace(ctx context.Context) []Frame {
+	frames, _ := ctx.Value(backtraceKey{}).([]Frame)
+	return frames
+}
 
+func WithBacktraceError(ctx context.Context, err error) error {
+	for _, source := range FramesToSources(Backtrace(ctx)) {
+		err = errdefs.WithSource(err, *source)
+	}
 	return errors.WithStack(err)
 }
 
@@ -179,7 +192,6 @@ func SourceMap(ctx context.Context) (opts []llb.ConstraintsOpt) {
 			End:   llbutil.PositionFromLexer(node.End()),
 		}}))
 	}
-
 	return
 }
 
@@ -221,3 +233,21 @@ func DockerAPI(ctx context.Context) DockerAPIClient {
 	}
 	return d
 }
+
+func WithDebugger(ctx context.Context, dbgr Debugger) context.Context {
+	return context.WithValue(ctx, debuggerKey{}, dbgr)
+}
+
+func GetDebugger(ctx context.Context) Debugger {
+	dbgr, _ := ctx.Value(debuggerKey{}).(Debugger)
+	return dbgr
+}
+
+func WithGlobalSolveOpts(ctx context.Context, opts ...solver.SolveOption) context.Context {
+	return context.WithValue(ctx, globalSolveOptsKey{}, append(GlobalSolveOpts(ctx), opts...))
+}
+
+func GlobalSolveOpts(ctx context.Context) []solver.SolveOption {
+	opts, _ := ctx.Value(globalSolveOptsKey{}).([]solver.SolveOption)
+	return opts
+}
diff --git a/codegen/debug.go b/codegen/debug.go
deleted file mode 100644
index 5241aea..0000000
--- a/codegen/debug.go
+++ /dev/null
@@ -1,874 +0,0 @@
-package codegen
-
-import (
-	"bufio"
-	"context"
-	"fmt"
-	"io"
-	"os"
-	"os/exec"
-	"strings"
-
-	"github.com/docker/buildx/util/progress"
-	shellquote "github.com/kballard/go-shellquote"
-	"github.com/moby/buildkit/client"
-	"github.com/moby/buildkit/client/llb"
-	gateway "github.com/moby/buildkit/frontend/gateway/client"
-	solvererrdefs "github.com/moby/buildkit/solver/errdefs"
-	"github.com/moby/buildkit/solver/pb"
-	digest "github.com/opencontainers/go-digest"
-	specs "github.com/opencontainers/image-spec/specs-go/v1"
-	"github.com/openllb/hlb/diagnostic"
-	"github.com/openllb/hlb/errdefs"
-	"github.com/openllb/hlb/parser"
-	"github.com/openllb/hlb/parser/ast"
-	"github.com/openllb/hlb/pkg/llbutil"
-	"github.com/openllb/hlb/pkg/steer"
-	"github.com/openllb/hlb/solver"
-	"github.com/pkg/errors"
-	"golang.org/x/crypto/ssh/terminal"
-	"golang.org/x/sync/errgroup"
-)
-
-var (
-	ErrDebugExit = errors.Errorf("exiting debugger")
-)
-
-type Debugger func(ctx context.Context, scope *ast.Scope, node ast.Node, ret Register, opts Option) error
-
-func NewNoopDebugger() Debugger {
-	return func(ctx context.Context, _ *ast.Scope, _ ast.Node, _ Register, _ Option) error {
-		return nil
-	}
-}
-
-type snapshot struct {
-	scope *ast.Scope
-	node  ast.Node
-	val   Value
-	opts  Option
-}
-
-func (s snapshot) fs() (Filesystem, error) {
-	if s.val == nil {
-		return Filesystem{}, errors.New("no value available")
-	}
-	return s.val.Filesystem()
-}
-
-func NewDebugger(c *client.Client, w io.Writer, inputSteerer *steer.InputSteerer, promptReader *bufio.Reader) Debugger {
-	var (
-		fd                *ast.FuncDecl
-		next              *ast.FuncDecl
-		history           []*snapshot
-		historyIndex      = -1
-		reverseStep       bool
-		cont              bool
-		staticBreakpoints []*Breakpoint
-		breakpoints       []*Breakpoint
-	)
-
-	var previousCommand string
-
-	return func(ctx context.Context, scope *ast.Scope, node ast.Node, ret Register, opts Option) error {
-		// Store a snapshot of the current debug step so we can backtrack.
-		historyIndex++
-		history = append(history, &snapshot{scope, node, ret.Value(), opts})
-
-		debug := func(s *snapshot) error {
-			showList := true
-
-			// Keep track of whether we're in global scope or a lexical scope.
-			switch n := node.(type) {
-			case *ast.Module:
-				staticBreakpoints = FindStaticBreakpoints(ctx, n)
-				breakpoints = staticBreakpoints
-
-				// Don't print source code on the first debug section.
-				showList = false
-			default:
-				fd = scope.Node.(*ast.FuncDecl)
-				if AtBreakpoint(node, fd, breakpoints, opts) {
-					cont = false
-				}
-			}
-
-			if showList && !cont {
-				PrintList(ctx, w, s.node)
-			}
-
-			if next != nil {
-				// If next is not in the same function scope, skip over it.
-				if next != fd {
-					return nil
-				}
-				next = nil
-			}
-
-			// Continue until we find a breakpoint or end of program.
-			if cont {
-				return nil
-			}
-
-			for {
-				err := Progress(ctx).Sync()
-				if err != nil {
-					return err
-				}
-				fmt.Fprint(w, "(hlb) ")
-
-				command, err := promptReader.ReadString('\n')
-				if err != nil {
-					return err
-				}
-
-				command = strings.Replace(command, "\n", "", -1)
-
-				if command == "" && previousCommand == "" {
-					continue
-				} else if command == "" {
-					command = previousCommand
-				}
-				previousCommand = command
-
-				args, err := shellquote.Split(command)
-				if err != nil {
-					return err
-				}
-
-				switch args[0] {
-				case "break", "b":
-					var bp *Breakpoint
-
-					if len(args) == 1 {
-						switch n := s.node.(type) {
-						case *ast.FuncDecl:
-							bp = &Breakpoint{
-								Func: n,
-							}
-						case *ast.CallStmt:
-							if n.Name.Ident.Text == "breakpoint" {
-								fmt.Fprintf(w, "%s cannot break at breakpoint\n", parser.FormatPos(n.Pos))
-								continue
-							}
-
-							bp = &Breakpoint{
-								Func: fd,
-								Call: n,
-							}
-						default:
-							fmt.Fprintln(w, "cannot break here")
-							continue
-						}
-					} else {
-						fmt.Fprintln(w, "unimplemented")
-						continue
-					}
-					breakpoints = append(breakpoints, bp)
-				case "breakpoints":
-					for i, bp := range breakpoints {
-						pos := bp.Func.Pos
-						if bp.Call != nil {
-							pos = bp.Call.Pos
-						}
-
-						msg := fmt.Sprintf("Breakpoint %d for %s%s %s",
-							i,
-							bp.Func.Sig.Name,
-							bp.Func.Sig.Params,
-							parser.FormatPos(pos))
-
-						if bp.Call != nil {
-							bp.Call.Terminate = nil
-							msg = fmt.Sprintf("%s %s", msg, bp.Call)
-						}
-
-						fmt.Fprintln(w, msg)
-					}
-				case "clear":
-					if len(args) == 0 {
-						breakpoints = append([]*Breakpoint{}, staticBreakpoints...)
-					} else {
-						fmt.Fprintln(w, "unimplemented")
-						continue
-					}
-				case "continue", "c":
-					cont = true
-					return nil
-				case "dir":
-					fs, err := s.fs()
-					if err != nil {
-						fmt.Fprintln(w, "current step is not in a fs scope")
-						continue
-					}
-
-					dir, err := fs.State.GetDir(ctx)
-					if err != nil {
-						fmt.Fprintln(w, "err:", err)
-						continue
-					}
-
-					fmt.Fprintf(w, "Working directory %q\n", dir)
-				case "dot":
-					fs, err := s.fs()
-					if err != nil {
-						fmt.Fprintln(w, "current step is not in a fs scope")
-						continue
-					}
-
-					var sh string
-					if len(args) == 2 {
-						sh = args[1]
-					}
-
-					err = printGraph(ctx, fs.State, fs.Platform, sh)
-					if err != nil {
-						fmt.Fprintln(w, "err:", err)
-					}
-					continue
-				case "env":
-					fs, err := s.fs()
-					if err != nil {
-						fmt.Fprintln(w, "current step is not in a fs scope")
-						continue
-					}
-
-					env, err := fs.State.Env(ctx)
-					if err != nil {
-						fmt.Fprintln(w, "err:", err)
-						continue
-					}
-
-					fmt.Fprintln(w, "Environment ", env)
-				case "exec":
-					fs, err := s.fs()
-					if err != nil {
-						fmt.Fprintln(w, "current step is not in a fs scope")
-						continue
-					}
-
-					execPR, execPW := io.Pipe()
-					inputSteerer.Push(execPW)
-
-					func() {
-						defer inputSteerer.Pop()
-
-						err = ExecWithFS(ctx, c, fs, execPR, w, opts, args[1:]...)
-						if err != nil {
-							fmt.Fprintln(w, "err:", err)
-							return
-						}
-					}()
-				case "exit", "quit", "q":
-					return ErrDebugExit
-				case "funcs":
-					for _, obj := range s.scope.Defined() {
-						switch obj.Node.(type) {
-						case *ast.FuncDecl, *ast.BindClause:
-							fmt.Fprintln(w, obj.Ident.String())
-						}
-					}
-				case "help":
-					fmt.Fprintln(w, "# Inspect")
-					fmt.Fprintln(w, "help - shows this help message")
-					fmt.Fprintln(w, "list - show source code")
-					fmt.Fprintln(w, "print - print evaluate an expression")
-					fmt.Fprintln(w, "funcs - print list of functions")
-					fmt.Fprintln(w, "locals - print local variables")
-					fmt.Fprintln(w, "types - print list of types")
-					fmt.Fprintln(w, "whatis - print type of an expression")
-					fmt.Fprintln(w, "# Movement")
-					fmt.Fprintln(w, "exit - exit the debugger")
-					fmt.Fprintln(w, "break [ <symbol> | <linespec> ] - sets a breakpoint")
-					fmt.Fprintln(w, "breakpoints - print out info for active breakpoints")
-					fmt.Fprintln(w, "clear [ <breakpoint-index> ] - deletes breakpoint")
-					fmt.Fprintln(w, "continue - run until breakpoint or program termination")
-					fmt.Fprintln(w, "next - step over to next source line")
-					fmt.Fprintln(w, "step - single step through program")
-					fmt.Fprintln(w, "stepout - step out of current function")
-					fmt.Fprintln(w, "reverse-step - single step backwards through program")
-					fmt.Fprintln(w, "restart - restart program from the start")
-					fmt.Fprintln(w, "# Filesystem")
-					fmt.Fprintln(w, "dir - print working directory")
-					fmt.Fprintln(w, "env - print environment")
-					fmt.Fprintln(w, "network - print network mode")
-					fmt.Fprintln(w, "security - print security mode")
-				case "list", "l":
-					if showList {
-						PrintList(ctx, w, s.node)
-					} else {
-						fmt.Fprintln(w, "Program has not started yet")
-					}
-				case "locals":
-					if fd != nil {
-						for _, arg := range fd.Sig.Params.Fields() {
-							obj := s.scope.Lookup(arg.Name.Text)
-							if obj == nil {
-								fmt.Fprintln(w, "err:", errors.WithStack(errdefs.WithUndefinedIdent(arg, nil)))
-								continue
-							}
-							fmt.Fprintf(w, "%s %s = %#v\n", arg.Type, arg.Name, obj.Data)
-						}
-					}
-				case "next", "n":
-					next = fd
-					return nil
-				case "network":
-					fs, err := s.fs()
-					if err != nil {
-						fmt.Fprintln(w, "current step is not in a fs scope")
-						continue
-					}
-
-					network, err := fs.State.GetNetwork(ctx)
-					if err != nil {
-						fmt.Fprintln(w, "err:", err)
-						continue
-					}
-
-					fmt.Fprintln(w, "Network", network.String())
-				case "print":
-					fmt.Fprintln(w, "print")
-				case "restart", "r":
-					reverseStep = true
-					historyIndex = 1
-					return nil
-				case "reverse-step", "rs":
-					if historyIndex == 0 {
-						fmt.Fprintln(w, "Already at the start of the program")
-					} else {
-						reverseStep = true
-						return nil
-					}
-				case "security":
-					fs, err := s.fs()
-					if err != nil {
-						fmt.Fprintln(w, "current step is not in a fs scope")
-						continue
-					}
-
-					security, err := fs.State.GetSecurity(ctx)
-					if err != nil {
-						fmt.Fprintln(w, "err:", err)
-						continue
-					}
-
-					fmt.Fprintln(w, "Security", security.String())
-				case "step", "s":
-					return nil
-				case "stepout":
-					fmt.Fprintln(w, "unimplemented")
-				case "types":
-					for _, kind := range []string{"string", "int", "bool", "fs", "option"} {
-						fmt.Fprintln(w, kind)
-					}
-				case "whatis":
-					fmt.Fprintln(w, "unimplemented")
-				default:
-					fmt.Fprintln(w, "unrecognized command", command)
-				}
-			}
-		}
-
-		err := debug(history[historyIndex])
-		if err != nil {
-			return err
-		}
-
-		if reverseStep {
-			historyIndex--
-			reverseStep = false
-
-			for historyIndex < len(history) {
-				err = debug(history[historyIndex])
-				if err != nil {
-					return err
-				}
-
-				if reverseStep {
-					historyIndex--
-					reverseStep = false
-				} else {
-					historyIndex++
-				}
-			}
-
-			historyIndex--
-		}
-
-		return nil
-	}
-}
-
-func PrintList(ctx context.Context, w io.Writer, node ast.Node) {
-	err := node.WithError(nil, node.Spanf(diagnostic.Primary, ""))
-	for _, span := range diagnostic.Spans(err) {
-		fmt.Fprintln(w, span.Pretty(ctx, diagnostic.WithNumContext(3)))
-	}
-}
-
-type Breakpoint struct {
-	Func *ast.FuncDecl
-	Call *ast.CallStmt
-}
-
-func FindStaticBreakpoints(ctx context.Context, mod *ast.Module) []*Breakpoint {
-	var breakpoints []*Breakpoint
-
-	ast.Match(mod, ast.MatchOpts{},
-		func(fd *ast.FuncDecl, call *ast.CallStmt) {
-			if fd.Kind() == "option::run" {
-				return
-			}
-			if !call.Breakpoint(ReturnType(ctx)) {
-				return
-			}
-			bp := &Breakpoint{
-				Func: fd,
-				Call: call,
-			}
-			breakpoints = append(breakpoints, bp)
-		},
-	)
-
-	return breakpoints
-}
-
-func AtBreakpoint(node ast.Node, fd *ast.FuncDecl, breakpoints []*Breakpoint, opts Option) bool {
-	for _, opt := range opts {
-		if _, hasBreakpointCommand := opt.(breakpointCommand); hasBreakpointCommand {
-			return true
-		}
-	}
-	for _, bp := range breakpoints {
-		if node == fd.Sig.Name {
-			if bp.Call == nil && bp.Func == fd {
-				return true
-			}
-		} else if bp.Call != nil && bp.Call.Name == node {
-			return true
-		}
-	}
-	return false
-}
-
-func printGraph(ctx context.Context, st llb.State, p specs.Platform, sh string) error {
-	def, err := st.Marshal(ctx, llb.Platform(p))
-	if err != nil {
-		return err
-	}
-
-	ops, err := loadLLB(def)
-	if err != nil {
-		return err
-	}
-
-	r, w := io.Pipe()
-	defer r.Close()
-
-	go func() {
-		defer w.Close()
-		writeDot(ops, w)
-	}()
-
-	if sh == "" {
-		_, err = io.Copy(os.Stderr, r)
-		return err
-	}
-
-	cmd := exec.CommandContext(ctx, "sh", "-c", sh)
-	cmd.Stdin = r
-	cmd.Stdout = os.Stdout
-	cmd.Stderr = os.Stderr
-
-	return cmd.Run()
-}
-
-type llbOp struct {
-	Op         pb.Op
-	Digest     digest.Digest
-	OpMetadata pb.OpMetadata
-}
-
-func loadLLB(def *llb.Definition) ([]llbOp, error) {
-	var ops []llbOp
-	for _, dt := range def.Def {
-		var op pb.Op
-		if err := (&op).Unmarshal(dt); err != nil {
-			return nil, err
-		}
-		dgst := digest.FromBytes(dt)
-		ent := llbOp{Op: op, Digest: dgst, OpMetadata: def.Metadata[dgst]}
-		ops = append(ops, ent)
-	}
-	return ops, nil
-}
-
-func writeDot(ops []llbOp, w io.Writer) {
-	fmt.Fprintln(w, "digraph {")
-	defer fmt.Fprintln(w, "}")
-	for _, op := range ops {
-		name, shape := attr(op.Digest, op.Op)
-		fmt.Fprintf(w, "  %q [label=%q shape=%q];\n", op.Digest, name, shape)
-	}
-	for _, op := range ops {
-		for i, inp := range op.Op.Inputs {
-			label := ""
-			if eo, ok := op.Op.Op.(*pb.Op_Exec); ok {
-				for _, m := range eo.Exec.Mounts {
-					if int(m.Input) == i && m.Dest != "/" {
-						label = m.Dest
-					}
-				}
-			}
-			fmt.Fprintf(w, "  %q -> %q [label=%q];\n", inp.Digest, op.Digest, label)
-		}
-	}
-}
-
-func attr(dgst digest.Digest, op pb.Op) (string, string) {
-	switch op := op.Op.(type) {
-	case *pb.Op_Source:
-		return op.Source.Identifier, "ellipse"
-	case *pb.Op_Exec:
-		return strings.Join(op.Exec.Meta.Args, " "), "box"
-	case *pb.Op_Build:
-		return "build", "box3d"
-	case *pb.Op_File:
-		names := []string{}
-
-		for _, action := range op.File.Actions {
-			var name string
-
-			switch act := action.Action.(type) {
-			case *pb.FileAction_Copy:
-				name = fmt.Sprintf("copy{src=%s, dest=%s}", act.Copy.Src, act.Copy.Dest)
-			case *pb.FileAction_Mkfile:
-				name = fmt.Sprintf("mkfile{path=%s}", act.Mkfile.Path)
-			case *pb.FileAction_Mkdir:
-				name = fmt.Sprintf("mkdir{path=%s}", act.Mkdir.Path)
-			case *pb.FileAction_Rm:
-				name = fmt.Sprintf("rm{path=%s}", act.Rm.Path)
-			}
-
-			names = append(names, name)
-		}
-		return strings.Join(names, ","), "note"
-	default:
-		return dgst.String(), "plaintext"
-	}
-}
-
-func ExecWithFS(ctx context.Context, cln *client.Client, fs Filesystem, r io.ReadCloser, w io.Writer, opts Option, args ...string) error {
-	var (
-		securityMode pb.SecurityMode
-		netMode      pb.NetMode
-		extraHosts   []*pb.HostIP
-		secrets      []llbutil.SecretOption
-		ssh          []llbutil.SSHOption
-	)
-
-	cwd := "/"
-	if fs.Image.Config.WorkingDir != "" {
-		cwd = fs.Image.Config.WorkingDir
-	}
-
-	env := make([]string, len(fs.Image.Config.Env))
-	copy(env, fs.Image.Config.Env)
-
-	user := fs.Image.Config.User
-
-	mounts := []*llbutil.MountRunOption{
-		{
-			Source: fs.State,
-			Target: "/",
-		},
-	}
-
-	for _, opt := range opts {
-		switch o := opt.(type) {
-		case llbutil.ReadonlyRootFSOption:
-			mounts[0].Opts = append(mounts[0].Opts, llbutil.WithReadonlyMount())
-		case *llbutil.MountRunOption:
-			mounts = append(mounts, o)
-		case llbutil.UserOption:
-			user = o.User
-		case llbutil.DirOption:
-			cwd = o.Dir
-		case llbutil.EnvOption:
-			env = append(env, o.Name+"="+o.Value)
-		case llbutil.SecurityOption:
-			securityMode = o.SecurityMode
-		case llbutil.NetworkOption:
-			netMode = o.NetMode
-		case llbutil.HostOption:
-			extraHosts = append(extraHosts, &pb.HostIP{
-				Host: o.Host,
-				IP:   o.IP.String(),
-			})
-		case llbutil.SecretOption:
-			secrets = append(secrets, o)
-		case llbutil.SSHOption:
-			ssh = append(ssh, o)
-		case llbutil.SessionOption:
-			fs.SessionOpts = append(fs.SessionOpts, o)
-		case breakpointCommand:
-			if len(args) == 0 {
-				args = []string(o)
-			}
-		}
-	}
-
-	if len(args) == 0 {
-		args = []string{"/bin/sh"}
-	}
-
-	s, err := llbutil.NewSession(ctx, fs.SessionOpts...)
-	if err != nil {
-		return err
-	}
-
-	g, ctx := errgroup.WithContext(ctx)
-
-	g.Go(func() error {
-		return s.Run(ctx, cln.Dialer())
-	})
-
-	g.Go(func() error {
-		var pw progress.Writer
-
-		mw := MultiWriter(ctx)
-		if mw != nil {
-			pw = mw.WithPrefix("", false)
-		}
-
-		return solver.Build(ctx, cln, s, pw, func(ctx context.Context, c gateway.Client) (res *gateway.Result, err error) {
-			ctrReq := gateway.NewContainerRequest{
-				NetMode:    netMode,
-				ExtraHosts: extraHosts,
-			}
-			for _, mount := range mounts {
-				gatewayMount := gateway.Mount{
-					Dest:      mount.Target,
-					MountType: pb.MountType_BIND,
-				}
-
-				for _, opt := range mount.Opts {
-					switch o := opt.(type) {
-					case llbutil.ReadonlyMountOption:
-						gatewayMount.Readonly = true
-					case llbutil.SourcePathMountOption:
-						gatewayMount.Selector = o.Path
-					case llbutil.CacheMountOption:
-						gatewayMount.MountType = pb.MountType_CACHE
-						gatewayMount.CacheOpt = &pb.CacheOpt{
-							ID:      o.ID,
-							Sharing: pb.CacheSharingOpt(o.Sharing),
-						}
-						switch o.Sharing {
-						case llb.CacheMountShared:
-							gatewayMount.CacheOpt.Sharing = pb.CacheSharingOpt_SHARED
-						case llb.CacheMountPrivate:
-							gatewayMount.CacheOpt.Sharing = pb.CacheSharingOpt_PRIVATE
-						case llb.CacheMountLocked:
-							gatewayMount.CacheOpt.Sharing = pb.CacheSharingOpt_LOCKED
-						default:
-							return nil, errors.Errorf("unrecognized cache sharing mode %v", o.Sharing)
-						}
-					case llbutil.TmpfsMountOption:
-						gatewayMount.MountType = pb.MountType_TMPFS
-					}
-				}
-
-				if gatewayMount.MountType == pb.MountType_BIND {
-					var def *llb.Definition
-					def, err = mount.Source.Marshal(ctx, llb.Platform(fs.Platform))
-					if err != nil {
-						return
-					}
-
-					res, err = c.Solve(ctx, gateway.SolveRequest{
-						Definition: def.ToPB(),
-					})
-					if err != nil {
-						return
-					}
-					gatewayMount.Ref = res.Ref
-				}
-
-				ctrReq.Mounts = append(ctrReq.Mounts, gatewayMount)
-
-			}
-
-			for _, secret := range secrets {
-				secretMount := gateway.Mount{
-					Dest:      secret.Dest,
-					MountType: pb.MountType_SECRET,
-					SecretOpt: &pb.SecretOpt{},
-				}
-				for _, opt := range secret.Opts {
-					switch o := opt.(type) {
-					case llbutil.IDOption:
-						secretMount.SecretOpt.ID = string(o)
-					case llbutil.UID:
-						secretMount.SecretOpt.Uid = uint32(o)
-					case llbutil.GID:
-						secretMount.SecretOpt.Gid = uint32(o)
-					case llbutil.Chmod:
-						secretMount.SecretOpt.Mode = uint32(o)
-					}
-				}
-				ctrReq.Mounts = append(ctrReq.Mounts, secretMount)
-			}
-
-			for i, sshOpt := range ssh {
-				sshMount := gateway.Mount{
-					Dest:      sshOpt.Dest,
-					MountType: pb.MountType_SSH,
-					SSHOpt:    &pb.SSHOpt{},
-				}
-				for _, opt := range sshOpt.Opts {
-					switch o := opt.(type) {
-					case llbutil.IDOption:
-						sshMount.SSHOpt.ID = string(o)
-					case llbutil.UID:
-						sshMount.SSHOpt.Uid = uint32(o)
-					case llbutil.GID:
-						sshMount.SSHOpt.Gid = uint32(o)
-					case llbutil.Chmod:
-						sshMount.SSHOpt.Mode = uint32(o)
-					}
-				}
-				if sshMount.Dest == "" {
-					sshMount.Dest = fmt.Sprintf("/run/buildkit/ssh_agent.%d", i)
-				}
-				if i == 0 {
-					env = append(env, "SSH_AUTH_SOCK="+sshMount.Dest)
-				}
-				ctrReq.Mounts = append(ctrReq.Mounts, sshMount)
-			}
-
-			ctr, err := c.NewContainer(ctx, ctrReq)
-			if err != nil {
-				return
-			}
-			defer ctr.Release(ctx)
-
-			err = Progress(ctx).Sync()
-			if err != nil {
-				return
-			}
-
-			startReq := gateway.StartRequest{
-				Args:         args,
-				Cwd:          cwd,
-				User:         user,
-				Env:          env,
-				Tty:          true,
-				Stdin:        r,
-				Stdout:       NopWriteCloser(w),
-				SecurityMode: securityMode,
-			}
-
-			proc, err := ctr.Start(ctx, startReq)
-			if err != nil {
-				return
-			}
-
-			oldState, err := terminal.MakeRaw(int(os.Stdin.Fd()))
-			if err == nil {
-				defer terminal.Restore(int(os.Stdin.Fd()), oldState)
-
-				cleanup := addResizeHandler(ctx, proc)
-				defer cleanup()
-			}
-
-			return res, proc.Wait()
-		}, fs.SolveOpts...)
-	})
-
-	return g.Wait()
-}
-
-func ExecWithSolveErr(ctx context.Context, c gateway.Client, se *solvererrdefs.SolveError, r io.ReadCloser, w io.Writer, env []string, args ...string) error {
-	op := se.Op
-	solveExec, ok := op.Op.(*pb.Op_Exec)
-	if !ok {
-		return nil
-	}
-
-	exec := solveExec.Exec
-
-	if len(args) == 0 {
-		args = []string{"/bin/sh"}
-	}
-
-	var mounts []gateway.Mount
-	for i, mnt := range exec.Mounts {
-		mounts = append(mounts, gateway.Mount{
-			Selector:  mnt.Selector,
-			Dest:      mnt.Dest,
-			ResultID:  se.Solve.MountIDs[i],
-			Readonly:  mnt.Readonly,
-			MountType: mnt.MountType,
-			CacheOpt:  mnt.CacheOpt,
-			SecretOpt: mnt.SecretOpt,
-			SSHOpt:    mnt.SSHOpt,
-		})
-	}
-
-	ctr, err := c.NewContainer(ctx, gateway.NewContainerRequest{
-		Mounts:      mounts,
-		NetMode:     exec.Network,
-		ExtraHosts:  exec.Meta.ExtraHosts,
-		Platform:    op.Platform,
-		Constraints: op.Constraints,
-	})
-	if err != nil {
-		return err
-	}
-	defer ctr.Release(ctx)
-
-	err = Progress(ctx).Sync()
-	if err != nil {
-		return err
-	}
-
-	startReq := gateway.StartRequest{
-		Args:         args,
-		Cwd:          exec.Meta.Cwd,
-		User:         exec.Meta.User,
-		Env:          append(exec.Meta.Env, env...),
-		Tty:          true,
-		Stdin:        r,
-		Stdout:       NopWriteCloser(w),
-		SecurityMode: exec.Security,
-	}
-
-	proc, err := ctr.Start(ctx, startReq)
-	if err != nil {
-		return err
-	}
-
-	oldState, err := terminal.MakeRaw(int(os.Stdin.Fd()))
-	if err == nil {
-		defer terminal.Restore(int(os.Stdin.Fd()), oldState)
-
-		cleanup := addResizeHandler(ctx, proc)
-		defer cleanup()
-	}
-
-	return proc.Wait()
-}
-
-func NopWriteCloser(w io.Writer) io.WriteCloser {
-	return &nopWriteCloser{w}
-}
-
-type nopWriteCloser struct {
-	io.Writer
-}
-
-func (w *nopWriteCloser) Close() error {
-	return nil
-}
diff --git a/codegen/debug/linespec.go b/codegen/debug/linespec.go
new file mode 100644
index 0000000..3f94b4c
--- /dev/null
+++ b/codegen/debug/linespec.go
@@ -0,0 +1,176 @@
+package debug
+
+import (
+	"context"
+	"errors"
+	"fmt"
+	"os"
+	"strconv"
+	"strings"
+
+	"github.com/openllb/hlb/checker"
+	"github.com/openllb/hlb/linter"
+	"github.com/openllb/hlb/parser"
+	"github.com/openllb/hlb/parser/ast"
+)
+
+// ParseLinespec returns an ast.StopNode that matches one of the location
+// specifiers supported by linespec.
+//
+// `<line>` Specifies the line *line* in the current file.
+//
+// `+<offset>` Specifies the line *offset* lines after the current one.
+//
+// `-<offset>` Specifies the line *offset* lines before the current one.
+//
+// `<function>[:<line>] Specifies the line *line* inside *function* in the
+// current file.
+//
+// `<filename>:<line>` Specifies the line *line* in *filename*, *filename*
+// can be the partial path to a file or even just the base name as long as the
+// expression remains unambiguous.
+//
+// `<filename>:<function>[:<line>] Specifies the line *line* inside *function*
+// in *filename*, *filename* can be the partial path toa file or eve njust the
+// base name as long as the expression remains unambiguous.
+//
+// `/<regex>/` Specifies the location of all functions matching *regex*.
+func ParseLinespec(ctx context.Context, scope *ast.Scope, node ast.Node, linespec string) (ast.StopNode, error) {
+	parts := strings.Split(linespec, ":")
+
+	var (
+		root ast.Node
+		line int
+		err  error
+	)
+	switch len(parts) {
+	case 1: // Either `<line>`, `+<offset>`, `-<offset>`, `<function>` or `/<regex>/`
+		mod := scope.ByLevel(ast.ModuleScope).Node.(*ast.Module)
+		if strings.HasPrefix(parts[0], "+") {
+			offset, err := strconv.Atoi(parts[0][1:])
+			if err != nil {
+				return nil, fmt.Errorf("offset is not an int: %w", err)
+			}
+			root = mod
+			line = node.Position().Line + offset
+		} else if strings.HasPrefix(parts[0], "-") {
+			offset, err := strconv.Atoi(parts[0][1:])
+			if err != nil {
+				return nil, fmt.Errorf("offset is not an int: %w", err)
+			}
+			root = mod
+			line = node.Position().Line - offset
+		} else if strings.HasPrefix(parts[0], "/") && strings.HasSuffix(parts[0], "/") {
+			return nil, fmt.Errorf("regex is unimplemented")
+		} else {
+			line, err = strconv.Atoi(parts[0])
+			if err == nil {
+				root = mod
+			} else {
+				fd, err := findFunction(mod, parts[0])
+				if err != nil {
+					return nil, err
+				}
+				root = fd
+				line = fd.Position().Line
+			}
+		}
+	case 2: // Either `<filename>:<line>` or `<function>:<line>` or `<filename>:<function>
+		mod, err := findModule(ctx, parts[0])
+		if err == nil { // <filename>:<line> or <filename>:<function>
+			root = mod
+			line, err = strconv.Atoi(parts[1])
+			if err != nil { // <filename>:<function>
+				fd, err := findFunction(mod, parts[1])
+				if err != nil {
+					return nil, err
+				}
+				root = fd
+				line = fd.Pos.Line
+			}
+		} else if errors.Is(err, os.ErrNotExist) { // <function>:<line>
+			mod := scope.ByLevel(ast.ModuleScope).Node.(*ast.Module)
+			fd, err := findFunction(mod, parts[0])
+			if err != nil {
+				return nil, err
+			}
+			root = fd
+
+			line, err = strconv.Atoi(parts[1])
+			if err != nil {
+				return nil, fmt.Errorf("line is not an int: %w", err)
+			}
+
+			line = fd.Pos.Line + line
+		} else {
+			return nil, err
+		}
+	case 3: // <filename>:<function>:<line>
+		filename, function := parts[0], parts[1]
+		line, err = strconv.Atoi(parts[2])
+		if err != nil {
+			return nil, fmt.Errorf("line is not an int: %w", err)
+		}
+
+		mod, err := findModule(ctx, filename)
+		if err != nil {
+			return nil, err
+		}
+
+		fd, err := findFunction(mod, function)
+		if err != nil {
+			return nil, err
+		}
+		root = fd
+		line = fd.Pos.Line + line
+	default:
+		return nil, fmt.Errorf("invalid linespec %s", linespec)
+	}
+
+	node = ast.Find(root, line, 0, ast.StopNodeFilter)
+	if node == nil {
+		return nil, fmt.Errorf("%s:%d is not a valid line to breakpoint", root.Position().Filename, line)
+	}
+	return node.(ast.StopNode), nil
+}
+
+func findModule(ctx context.Context, filename string) (*ast.Module, error) {
+	mod := ast.Modules(ctx).Get(filename)
+	if mod == nil {
+		f, err := os.Open(filename)
+		if err != nil {
+			return nil, err
+		}
+		defer f.Close()
+
+		mod, err = parser.Parse(ctx, f)
+		if err != nil {
+			return nil, err
+		}
+
+		err = checker.SemanticPass(mod)
+		if err != nil {
+			return nil, err
+		}
+
+		_ = linter.Lint(ctx, mod)
+
+		err = checker.Check(mod)
+		if err != nil {
+			return nil, err
+		}
+	}
+	return mod, nil
+}
+
+func findFunction(mod *ast.Module, function string) (*ast.FuncDecl, error) {
+	obj := mod.Scope.Lookup(function)
+	if obj == nil {
+		return nil, fmt.Errorf("no function %s in %s", function, mod.Pos.Filename)
+	}
+	fd, ok := obj.Node.(*ast.FuncDecl)
+	if !ok {
+		return nil, fmt.Errorf("%s is not a function but %T", function, obj.Node)
+	}
+	return fd, nil
+}
diff --git a/codegen/debug/tui.go b/codegen/debug/tui.go
new file mode 100644
index 0000000..0e34d2a
--- /dev/null
+++ b/codegen/debug/tui.go
@@ -0,0 +1,565 @@
+package debug
+
+import (
+	"bytes"
+	"context"
+	"errors"
+	"fmt"
+	"io"
+	"strconv"
+	"strings"
+
+	"github.com/chzyer/readline"
+	shellquote "github.com/kballard/go-shellquote"
+	"github.com/moby/buildkit/solver/errdefs"
+	"github.com/openllb/hlb/codegen"
+	"github.com/openllb/hlb/diagnostic"
+	"github.com/openllb/hlb/parser/ast"
+	"github.com/openllb/hlb/pkg/steer"
+)
+
+func TUIFrontend(ctx context.Context, dbgr codegen.Debugger, is *steer.InputSteerer, stdin io.ReadCloser, stdout, stderr io.Writer) error {
+	s, serr := dbgr.GetState()
+	if serr != nil {
+		return serr
+	}
+	defer dbgr.Terminate()
+
+	l, err := readline.NewEx(&readline.Config{
+		Prompt: "(hlb) ",
+		Stdin:  stdin,
+		Stdout: stdout,
+		Stderr: stderr,
+	})
+	if err != nil {
+		return err
+	}
+	defer l.Close()
+
+	firstPrompt, firstExec := true, true
+	color := diagnostic.Color(ctx)
+
+	var prevCommand string
+	for {
+		if serr != nil {
+			return serr
+		}
+		p := codegen.Progress(ctx)
+		if p != nil {
+			err := p.Sync()
+			if err != nil {
+				return err
+			}
+		}
+
+		if s.Err != nil {
+			spans := diagnostic.SourcesToSpans(s.Ctx, errdefs.Sources(s.Err), s.Err)
+			if len(spans) > 0 {
+				diagnostic.DisplayError(s.Ctx, stdout, spans, s.Err, true)
+			} else {
+				for _, span := range diagnostic.Spans(s.Err) {
+					fmt.Fprintln(stdout, span.Pretty(ctx))
+				}
+			}
+
+			_, err = s.Value.Filesystem()
+			if err == nil && firstExec {
+				firstExec = false
+				fmt.Fprintln(stdout, color.Sprintf("%s %s %s",
+					color.Green("Type"),
+					color.Yellow("exec"),
+					color.Green("to start a process in the failed state"),
+				))
+			}
+		}
+
+		stop, ok := s.Node.(ast.StopNode)
+		if s.Err == nil && ok {
+			err = handleList(stdout, s, stop, nil)
+			if err != nil {
+				printError(stderr, s, err)
+			}
+		}
+
+		if firstPrompt {
+			firstPrompt = false
+			fmt.Fprintln(stdout, color.Sprintf("%s %s %s",
+				color.Green("Type"),
+				color.Yellow("help"),
+				color.Green("for a list of commands"),
+			))
+		}
+
+	prompt:
+		line, err := l.Readline()
+		if err != nil {
+			if errors.Is(err, readline.ErrInterrupt) || errors.Is(err, io.EOF) {
+				return codegen.ErrDebugExit
+			}
+			return err
+		}
+
+		line = strings.TrimSpace(line)
+		if line == "" {
+			if prevCommand == "" {
+				continue
+			}
+			line = prevCommand
+		} else {
+			prevCommand = line
+		}
+
+		args, err := shellquote.Split(line)
+		if err != nil {
+			return err
+		}
+		cmd, args := args[0], args[1:]
+		direction := codegen.ForwardDirection
+
+	execute:
+		switch cmd {
+		case "args":
+			err = handleArgs(stdout, s)
+			if err != nil {
+				printError(stderr, s, err)
+			}
+			goto prompt
+		case "backtrace", "bt":
+			err = handleBacktrace(stdout, s, dbgr)
+			if err != nil {
+				printError(stderr, s, err)
+			}
+			goto prompt
+		case "break", "b":
+			err = handleBreak(stdout, s, dbgr, stop, args)
+			if err != nil {
+				printError(stderr, s, err)
+			}
+			goto prompt
+		case "breakpoints", "bp":
+			for _, bp := range dbgr.Breakpoints() {
+				bp.Print(s.Ctx, stdout, false)
+			}
+			goto prompt
+		case "clear":
+			err := handleClear(stdout, s, dbgr, args)
+			if err != nil {
+				printError(stderr, s, err)
+			}
+			goto prompt
+		case "clearall":
+			err = handleClearAll(stdout, s, dbgr)
+			if err != nil {
+				printError(stderr, s, err)
+			}
+			goto prompt
+		case "continue", "c":
+			s, serr = dbgr.Continue(direction)
+		case "environ":
+			err = handleEnviron(stdout, s)
+			if err != nil {
+				printError(stderr, s, err)
+			}
+			goto prompt
+		case "exec":
+			err = l.Close()
+			if err != nil {
+				return err
+			}
+
+			err = handleExec(ctx, dbgr, is, stdout, stderr, args...)
+			if err != nil {
+				printError(stderr, s, err)
+			}
+
+			l, err = readline.NewEx(&readline.Config{
+				Prompt: "(hlb) ",
+				Stdin:  stdin,
+				Stdout: stdout,
+				Stderr: stderr,
+			})
+			if err != nil {
+				return err
+			}
+			goto prompt
+		case "exit":
+			return nil
+		case "funcs":
+			handleFuncs(stdout, s)
+			goto prompt
+		case "help":
+			handleHelp(ctx, stdout)
+			goto prompt
+		case "list", "ls":
+			err = handleList(stdout, s, stop, args)
+			if err != nil {
+				printError(stderr, s, err)
+			}
+			goto prompt
+		case "network":
+			err = handleNetwork(stdout, s)
+			if err != nil {
+				printError(stderr, s, err)
+			}
+			goto prompt
+		case "next", "n":
+			s, serr = dbgr.Next(direction)
+		case "pwd":
+			err = handlePwd(stdout, s)
+			if err != nil {
+				printError(stderr, s, err)
+			}
+			goto prompt
+		case "restart":
+			s, serr = dbgr.Restart()
+		case "rev", "r":
+			if len(args) == 0 {
+				printError(stderr, s, requiredArgs("rev", 1))
+				goto prompt
+			}
+			cmd, args = args[0], args[1:]
+			direction = codegen.BackwardDirection
+			goto execute
+		case "security":
+			err = handleSecurity(stdout, s)
+			if err != nil {
+				printError(stderr, s, err)
+			}
+			goto prompt
+		case "step", "s":
+			s, serr = dbgr.Step(direction)
+		case "stepout":
+			s, serr = dbgr.StepOut(direction)
+		default:
+			fmt.Fprintf(stdout, color.Sprintf("%s %s\n", color.Red("Unrecognized command"), color.Yellow(cmd)))
+			goto prompt
+		}
+	}
+
+	return nil
+}
+
+func printError(w io.Writer, s *codegen.State, err error) {
+	color := diagnostic.Color(s.Ctx)
+	fmt.Fprintln(w, color.Sprintf("%s: %s", color.Red("Command failed"), err.Error()))
+}
+
+func requiredArgs(command string, i int) error {
+	msg := fmt.Sprintf("%s requires exactly %d arg", command, i)
+	if i > 1 {
+		msg = msg + "s"
+	}
+	return errors.New(msg)
+}
+
+func handleArgs(w io.Writer, s *codegen.State) error {
+	scope := s.Scope.ByLevel(ast.ArgsScope)
+	if scope == nil {
+		return errors.New("no args")
+	}
+	for _, obj := range scope.Locals() {
+		var value string
+		val, err := codegen.NewValue(s.Ctx, obj.Data)
+		if err != nil {
+			value = fmt.Sprintf("<%s>", obj.Kind)
+		} else if obj.Kind == ast.String {
+			value, _ = val.String()
+			value = strconv.Quote(value)
+		} else {
+			value = fmt.Sprintf("<%s>", obj.Kind)
+		}
+		fmt.Fprintf(w, "%s = %s\n", obj.Ident, value)
+	}
+	return nil
+}
+
+func handleBacktrace(w io.Writer, s *codegen.State, dbgr codegen.Debugger) error {
+	frames, err := dbgr.Backtrace()
+	if err != nil {
+		return err
+	}
+
+	if len(frames) == 0 {
+		return errors.New("cannot backtrace on program start")
+	}
+
+	spans := codegen.FramesToSpans(s.Ctx, frames)
+	diagnostic.DisplayError(s.Ctx, w, spans, nil, true)
+	return nil
+}
+
+func handleBreak(w io.Writer, s *codegen.State, dbgr codegen.Debugger, stop ast.StopNode, args []string) error {
+	// If no args, then create break on current line.
+	if len(args) == 0 {
+		if stop == nil {
+			return errors.New("cannot break on program start")
+		}
+
+		bp, err := dbgr.CreateBreakpoint(&codegen.Breakpoint{Node: stop.Subject()})
+		if err != nil {
+			return err
+		}
+		bp.Print(s.Ctx, w, true)
+		return nil
+	}
+
+	if len(args) > 1 {
+		return errors.New("requires only 0 or 1 arg")
+	}
+
+	var err error
+	linespec := args[0]
+	stop, err = ParseLinespec(s.Ctx, s.Scope, s.Node, linespec)
+	if err != nil {
+		return err
+	}
+
+	bp, err := dbgr.CreateBreakpoint(&codegen.Breakpoint{Node: stop.Subject()})
+	if err != nil {
+		return err
+	}
+	bp.Print(s.Ctx, w, false)
+	return nil
+}
+
+func handleClear(w io.Writer, s *codegen.State, dbgr codegen.Debugger, args []string) error {
+	if len(args) == 0 {
+		return requiredArgs("clear", 1)
+	}
+
+	i, err := strconv.Atoi(args[0])
+	if err != nil {
+		return err
+	}
+
+	bps := dbgr.Breakpoints()
+	if i > len(bps) {
+		return fmt.Errorf("no breakpoint with id %d", i)
+	}
+
+	// Write to output to buffer first, because print will have the wrong index
+	// if we clear it first.
+	buf := new(bytes.Buffer)
+	color := diagnostic.Color(s.Ctx)
+	fmt.Fprintf(buf, color.Sprintf(color.Green("Cleared ")))
+	bps[i].Print(s.Ctx, buf, true)
+
+	err = dbgr.ClearBreakpoint(bps[i])
+	if err != nil {
+		return err
+	}
+
+	fmt.Fprintf(w, buf.String())
+	return nil
+}
+
+func handleClearAll(w io.Writer, s *codegen.State, dbgr codegen.Debugger) error {
+	for _, bp := range dbgr.Breakpoints() {
+		if bp.SourceDefined {
+			continue
+		}
+
+		// Write to output to buffer first, because print will have the wrong index
+		// if we clear it first.
+		buf := new(bytes.Buffer)
+		color := diagnostic.Color(s.Ctx)
+		fmt.Fprintf(buf, color.Sprintf(color.Green("Cleared ")))
+		bp.Print(s.Ctx, buf, true)
+
+		err := dbgr.ClearBreakpoint(bp)
+		if err != nil {
+			return err
+		}
+		fmt.Fprintf(w, buf.String())
+	}
+	return nil
+}
+
+func handleEnviron(w io.Writer, s *codegen.State) error {
+	fs, err := s.Value.Filesystem()
+	if err != nil {
+		return err
+	}
+
+	envs, err := fs.State.Env(s.Ctx)
+	if err != nil {
+		return err
+	}
+
+	for _, env := range envs {
+		fmt.Fprintln(w, env)
+	}
+	return nil
+}
+
+func handleFuncs(w io.Writer, s *codegen.State) {
+	color := diagnostic.Color(s.Ctx)
+	scope := s.Scope.ByLevel(ast.ModuleScope)
+	for _, obj := range scope.Objects {
+		fd, ok := obj.Node.(*ast.FuncDecl)
+		if !ok {
+			continue
+		}
+
+		fmt.Fprintf(w, color.Sprintf(
+			"%s at ",
+			color.Yellow(fd.Sig.Name),
+		))
+		err := fd.Sig.Name.WithError(nil, fd.Sig.Name.Spanf(diagnostic.Primary, ""))
+		for _, span := range diagnostic.Spans(err) {
+			fmt.Fprintln(w, span.Pretty(s.Ctx, diagnostic.WithNumContext(1)))
+		}
+	}
+}
+
+func handleHelp(ctx context.Context, w io.Writer) {
+	printSection(ctx, w, "Running the program")
+	printCommand(ctx, w, "continue", "c", nil, "run until breakpoint or program termination")
+	printCommand(ctx, w, "next", "n", nil, "step over to next source line")
+	printCommand(ctx, w, "step", "s", nil, "single step through program")
+	printCommand(ctx, w, "stepout", "", nil, "step out of current function")
+	printCommand(ctx, w, "rev", "r", []string{"movement"}, "reverses execution of program for movement specified")
+	printCommand(ctx, w, "restart", "", nil, "restart program from the start")
+	fmt.Println("")
+
+	printSection(ctx, w, "Manipulating breakpoints")
+	printCommand(ctx, w, "break", "b", []string{"symbol | linespec"}, "sets a breakpoint")
+	printCommand(ctx, w, "breakpoints", "bp", nil, "prints out active breakpoints")
+	printCommand(ctx, w, "clear", "", []string{"breakpoint-index"}, "deletes breakpoint")
+	printCommand(ctx, w, "clearall", "", nil, "deletes all breakpoints")
+	fmt.Println("")
+
+	printSection(ctx, w, "Viewing program variables and functions")
+	printCommand(ctx, w, "args", "", nil, "print function arguments")
+	printCommand(ctx, w, "funcs", "", nil, "print functions in this module")
+	fmt.Println("")
+
+	printSection(ctx, w, "Viewing the call stack and selecting frames")
+	printCommand(ctx, w, "backtrace", "bt", nil, "prints backtrace at this step")
+	fmt.Println("")
+
+	printSection(ctx, w, "Filesystem only commands")
+	printCommand(ctx, w, "pwd", "", nil, "print working directory at this step")
+	printCommand(ctx, w, "environ", "", nil, "print environment at this step")
+	printCommand(ctx, w, "network", "", nil, "print network mode at this step")
+	printCommand(ctx, w, "security", "", nil, "print security mode at this step")
+	fmt.Println("")
+
+	printSection(ctx, w, "Other commands")
+	printCommand(ctx, w, "help", "", nil, "prints this help message")
+	printCommand(ctx, w, "list", "ls", nil, "prints source code at this step")
+	printCommand(ctx, w, "exit", "quit", nil, "exits the debugger")
+}
+
+func printSection(ctx context.Context, w io.Writer, section string) {
+	fmt.Fprintln(w, diagnostic.Color(ctx).Blue(fmt.Sprintf("# %s", section)))
+}
+
+func printCommand(ctx context.Context, w io.Writer, command, alias string, args []string, help string) {
+	color := diagnostic.Color(ctx)
+	command = color.Sprintf(color.Green(command))
+	if alias != "" {
+		command = color.Sprintf("%s %s", command,
+			color.Faint(color.Green(fmt.Sprintf("(alias: %s)", alias))),
+		)
+	}
+	for i, arg := range args {
+		args[i] = color.Sprintf(color.Yellow(fmt.Sprintf("<%s>", arg)))
+	}
+	if len(args) > 0 {
+		command = color.Sprintf("%s %s", command, strings.Join(args, " "))
+	}
+	fmt.Fprintf(w, "    %s - %s\n", command, help)
+}
+
+func handleList(w io.Writer, s *codegen.State, stop ast.StopNode, args []string) error {
+	if stop == nil {
+		return errors.New("cannot list on program start")
+	} else if len(args) > 1 {
+		return errors.New("requires only 0 or 1 arg")
+	}
+
+	if s.Err != nil {
+		spans := diagnostic.SourcesToSpans(s.Ctx, errdefs.Sources(s.Err), s.Err)
+		if len(spans) > 0 {
+			diagnostic.DisplayError(s.Ctx, w, spans, s.Err, true)
+		} else {
+			for _, span := range diagnostic.Spans(s.Err) {
+				fmt.Fprintf(w, "%s\n", span.Pretty(s.Ctx))
+			}
+		}
+		return nil
+	}
+
+	numContext := 4
+	if len(args) == 1 {
+		var err error
+		numContext, err = strconv.Atoi(args[0])
+		if err != nil {
+			return err
+		}
+	}
+
+	err := stop.Subject().WithError(nil, stop.Subject().Spanf(diagnostic.Primary, ""))
+	for _, span := range diagnostic.Spans(err) {
+		fmt.Fprintln(w, span.Pretty(s.Ctx, diagnostic.WithNumContext(numContext)))
+	}
+	return nil
+}
+
+func handleNetwork(w io.Writer, s *codegen.State) error {
+	fs, err := s.Value.Filesystem()
+	if err != nil {
+		return err
+	}
+	network, err := fs.State.GetNetwork(s.Ctx)
+	if err != nil {
+		return err
+	}
+	fmt.Fprintf(w, "Network mode %q\n", network)
+	return nil
+}
+
+func handlePwd(w io.Writer, s *codegen.State) error {
+	fs, err := s.Value.Filesystem()
+	if err != nil {
+		return err
+	}
+	pwd, err := fs.State.GetDir(s.Ctx)
+	if err != nil {
+		return err
+	}
+	fmt.Fprintf(w, "Current working directory %q\n", pwd)
+	return nil
+}
+
+func handleSecurity(w io.Writer, s *codegen.State) error {
+	fs, err := s.Value.Filesystem()
+	if err != nil {
+		return err
+	}
+	security, err := fs.State.GetSecurity(s.Ctx)
+	if err != nil {
+		return err
+	}
+	fmt.Fprintf(w, "Security mode %q\n", security)
+	return nil
+}
+
+func handleExec(ctx context.Context, d codegen.Debugger, is *steer.InputSteerer, stdout, stderr io.Writer, args ...string) error {
+	pr, pw := io.Pipe()
+	is.Push(pw)
+	defer is.Pop()
+
+	if len(args) == 0 {
+		args = []string{"/bin/sh"}
+	}
+
+	color := diagnostic.Color(ctx)
+	fmt.Fprintln(stdout, color.Sprintf("%s %q",
+		color.Green("Starting process"),
+		color.Bold(strings.Join(args, " ")),
+	))
+
+	return d.Exec(ctx, pr, stdout, stderr, args...)
+}
diff --git a/codegen/debugger.go b/codegen/debugger.go
new file mode 100644
index 0000000..3b3f729
--- /dev/null
+++ b/codegen/debugger.go
@@ -0,0 +1,544 @@
+package codegen
+
+import (
+	"context"
+	"fmt"
+	"io"
+	"strings"
+	"sync"
+
+	"github.com/moby/buildkit/client"
+	"github.com/moby/buildkit/solver/errdefs"
+	"github.com/openllb/hlb/diagnostic"
+	"github.com/openllb/hlb/parser/ast"
+	"github.com/pkg/errors"
+)
+
+var (
+	// ErrDebugExit is a special error to early exit from a program.
+	ErrDebugExit = errors.Errorf("exiting debugger")
+)
+
+// Debugger is a source-level debugger that provides controls over the program
+// flow and introspection of state.
+type Debugger interface {
+	io.Closer
+
+	// GetState returns the current debugger state.
+	// This call blocks until the debugger has stopped for any reason.
+	GetState() (*State, error)
+
+	// Restart moves the program back to the start.
+	Restart() (*State, error)
+
+	// Continue resumes execution.
+	Continue(Direction) (*State, error)
+
+	// Next continues to the next source line, not entering function calls.
+	Next(Direction) (*State, error)
+
+	// Step continues to the next source line, entering function calls.
+	Step(Direction) (*State, error)
+
+	// StepOut continues to the next source line outside the current function.
+	StepOut(Direction) (*State, error)
+
+	// Backtrace returns all the frames in the current call stack.
+	Backtrace() ([]Frame, error)
+
+	// Breakpoints gets all breakpoints.
+	Breakpoints() []*Breakpoint
+
+	// CreateBreakpoint creates a new breakpoint.
+	CreateBreakpoint(bp *Breakpoint) (*Breakpoint, error)
+
+	// ClearBreakpoint deletes a breakpoint.
+	ClearBreakpoint(bp *Breakpoint) error
+
+	// Terminate sends a signal to end the debugging session.
+	Terminate()
+
+	// Exec starts a process in the current debugging state.
+	Exec(ctx context.Context, stdin io.ReadCloser, stdout, stderr io.Writer, args ...string) error
+}
+
+// DebugMode is a mode of the debugger that affects control flow.
+type DebugMode int
+
+const (
+	DebugNone = iota
+	DebugStartStop
+	DebugContinue
+	DebugNext
+	DebugRestart
+	DebugStep
+	DebugStepOut
+	DebugTerminate
+)
+
+// Direction is the direction of execution.
+type Direction int8
+
+const (
+	// NoneDirection is not a valid direction.
+	NoneDirection Direction = iota
+
+	// ForwardDirection executes the target normally.
+	ForwardDirection
+
+	// BackwardDirection executes the target in reverse.
+	BackwardDirection
+)
+
+// State is a snapshot of the application state when the debugger has halted
+// for any reason.
+type State struct {
+	Ctx        context.Context
+	Scope      *ast.Scope
+	Node       ast.Node
+	Value      Value
+	Options    Option
+	StopReason string
+	Err        error
+}
+
+type debugger struct {
+	cln *client.Client
+	err error
+	mu  sync.Mutex
+
+	cursor    *State
+	direction Direction
+	mode      DebugMode
+	control   chan DebugMode
+
+	done chan struct{}
+	wg   sync.WaitGroup
+
+	recording      []*State
+	recordingIndex int
+
+	loadedSourceDefinedBreakpoints bool
+	sourceDefinedBreakpoints       []*Breakpoint
+	breakpoints                    []*Breakpoint
+	breakpointIDs                  map[string]struct{}
+}
+
+// DebuggerOption is optional configuration for the debugger.
+type DebuggerOption func(*debugger)
+
+// WithInitialMode overrides the initial debug mode that the debugger starts
+// with.
+func WithInitialMode(mode DebugMode) DebuggerOption {
+	return func(d *debugger) {
+		d.mode = mode
+	}
+}
+
+// NewDebugger returns a headless debugger.
+func NewDebugger(cln *client.Client, opts ...DebuggerOption) Debugger {
+	dbgr := &debugger{
+		cln:           cln,
+		mode:          DebugStartStop,
+		done:          make(chan struct{}),
+		control:       make(chan DebugMode),
+		breakpointIDs: make(map[string]struct{}),
+	}
+	for _, opt := range opts {
+		opt(dbgr)
+	}
+	dbgr.mu.Lock()
+	return dbgr
+}
+
+func (d *debugger) GetState() (*State, error) {
+	d.mu.Lock()
+	defer d.mu.Unlock()
+	if d.err != nil {
+		return nil, d.err
+	}
+	return d.recording[d.recordingIndex], nil
+}
+
+func (d *debugger) Restart() (*State, error) {
+	d.sendControl(DebugRestart, ForwardDirection)
+	return d.GetState()
+}
+
+func (d *debugger) Continue(direction Direction) (*State, error) {
+	d.sendControl(DebugContinue, direction)
+	return d.GetState()
+}
+
+func (d *debugger) Next(direction Direction) (*State, error) {
+	d.sendControl(DebugNext, direction)
+	return d.GetState()
+}
+
+func (d *debugger) Step(direction Direction) (*State, error) {
+	d.sendControl(DebugStep, direction)
+	return d.GetState()
+}
+
+func (d *debugger) StepOut(direction Direction) (*State, error) {
+	d.sendControl(DebugStepOut, direction)
+	return d.GetState()
+}
+
+func (d *debugger) Backtrace() ([]Frame, error) {
+	s, err := d.GetState()
+	if err != nil {
+		return nil, err
+	}
+	return Backtrace(s.Ctx), nil
+}
+
+func (d *debugger) Breakpoints() []*Breakpoint {
+	return d.breakpoints
+}
+
+func (d *debugger) CreateBreakpoint(bp *Breakpoint) (*Breakpoint, error) {
+	if _, ok := d.breakpointIDs[bp.ID()]; ok {
+		return bp, fmt.Errorf("breakpoint already exists at %s", bp.ID())
+	}
+	bp.dbgr = d
+	d.breakpoints = append(d.breakpoints, bp)
+	d.breakpointIDs[bp.ID()] = struct{}{}
+	return bp, nil
+}
+
+func (d *debugger) ClearBreakpoint(bp *Breakpoint) error {
+	if _, ok := d.breakpointIDs[bp.ID()]; !ok {
+		return fmt.Errorf("breakpoint at %s does not exist", bp.ID())
+	}
+	if bp.SourceDefined {
+		return fmt.Errorf("cannot clear source defined breakpoint at %s", bp.ID())
+	}
+	for i, candidate := range d.breakpoints {
+		if candidate == bp {
+			d.breakpoints = append(d.breakpoints[:i], d.breakpoints[i+1:]...)
+			break
+		}
+	}
+	delete(d.breakpointIDs, bp.ID())
+	return nil
+}
+
+func (d *debugger) Terminate() {
+	// Set debugger error so that next yield it exits early.
+	d.err = ErrDebugExit
+	d.sendControl(DebugTerminate, NoneDirection)
+}
+
+func (d *debugger) Exec(ctx context.Context, stdin io.ReadCloser, stdout, stderr io.Writer, args ...string) error {
+	s, err := d.GetState()
+	if err != nil {
+		return err
+	}
+
+	var se *errdefs.SolveError
+	if errors.As(s.Err, &se) {
+		var ge *gatewayError
+		if !errors.As(s.Err, &ge) {
+			return errors.Wrap(s.Err, "solve error without gateway")
+		}
+
+		// If the gateway context is canceled, also cancel the user's context.
+		ctx, cancel := context.WithCancel(ctx)
+		go func() {
+			<-ge.Done()
+			cancel()
+		}()
+
+		return ExecWithSolveErr(ctx, ge.Client, se, stdin, stdout, stderr, args...)
+	}
+
+	fs, err := s.Value.Filesystem()
+	if err != nil {
+		return err
+	}
+
+	return ExecWithFS(ctx, d.cln, fs, s.Options, stdin, stdout, stderr, args...)
+}
+
+func (d *debugger) sendControl(control DebugMode, direction Direction) {
+	// Prevent control being sent in parallel.
+	d.mu.Lock()
+	select {
+	case <-d.done:
+		// If debugger is closed, then release lock for other clients to exit
+		// gracefully.
+		d.mu.Unlock()
+		return
+	default:
+	}
+
+	// Otherwise send control, note that when multiple clients send control on
+	// the same state it will queue.
+	//
+	// There's no need to unlock at the end of sendControl because the debugger
+	// will unlock when it has reached a new halting condition or it is closed.
+	d.wg.Add(1)
+	defer d.wg.Done()
+	d.direction = direction
+	d.control <- control
+}
+
+func (d *debugger) Close() error {
+	// Set the debugger exit err.
+	d.err = ErrDebugExit
+	// Cancel incoming control signals.
+	close(d.done)
+	// Wait for clients to exit gracefully.
+	d.wg.Wait()
+	// Close control signal channel.
+	close(d.control)
+	// Allow clients to acquire lock to receive the exit err.
+	d.mu.Unlock()
+	return nil
+}
+
+func (d *debugger) yield(ctx context.Context, scope *ast.Scope, node ast.Node, val Value, opts Option, yieldErr error) error {
+	// If debugger has an error, continue to exit.
+	if d.err != nil {
+		return d.err
+	}
+
+	if yieldErr == nil && d.cln != nil {
+		req, err := val.Request()
+		if err != nil {
+			return ProgramCounter(ctx).WithError(err)
+		}
+
+		err = req.Solve(ctx, d.cln, MultiWriter(ctx))
+		if err != nil {
+			yieldErr = ProgramCounter(ctx).WithError(err)
+		}
+	}
+
+	// Record codegen state in order to support rewinding in playback.
+	d.recording = append(d.recording, &State{ctx, scope, node, val, opts, "", yieldErr})
+	for d.recordingIndex < len(d.recording) {
+		state := d.recording[d.recordingIndex]
+		err := d.playback(state)
+		if err != nil {
+			return err
+		}
+		switch d.direction {
+		case ForwardDirection:
+			d.recordingIndex++
+		case BackwardDirection:
+			if d.recordingIndex > 0 {
+				d.recordingIndex--
+			}
+		}
+	}
+
+	last := d.recording[len(d.recording)-1]
+	if last.Err != nil {
+		d.err = ErrDebugExit
+		return d.err
+	}
+	return nil
+}
+
+func (d *debugger) playback(s *State) error {
+	mod, ok := s.Node.(*ast.Module)
+	if ok && !d.loadedSourceDefinedBreakpoints {
+		// Load source defined breakpoints.
+		d.findSourceDefinedBreakpoints(mod)
+		d.loadedSourceDefinedBreakpoints = true
+		d.breakpoints = d.sourceDefinedBreakpoints
+	}
+
+	s.StopReason = d.stopReason(s)
+	if s.StopReason == "" {
+		return nil
+	}
+	d.cursor = nil
+
+	d.mu.Unlock()
+	d.mode = <-d.control
+	switch d.mode {
+	case DebugContinue, DebugStep:
+	case DebugNext, DebugStepOut:
+		d.cursor = s
+	case DebugRestart:
+		d.recordingIndex = -1
+		d.direction = ForwardDirection
+	case DebugTerminate:
+		return ErrDebugExit
+	default:
+		return fmt.Errorf("unrecognized mode: %d", d.mode)
+	}
+	return nil
+}
+
+func (d *debugger) stopReason(s *State) string {
+	if s.Err != nil {
+		return "exception"
+	}
+
+	switch d.mode {
+	case DebugStartStop, DebugRestart, DebugStep:
+		return "step"
+	case DebugNext:
+		// Reverse next back into program start.
+		if _, ok := s.Node.(*ast.Module); ok {
+			return "entry"
+		}
+		// Next from program start.
+		if _, ok := d.cursor.Node.(*ast.Module); ok {
+			return "step"
+		}
+		// Skip over steps that are not a stop node.
+		stop, ok := s.Node.(ast.StopNode)
+		if !ok {
+			return ""
+		}
+		// Skip over steps in a deeper frames than the cursor.
+		if len(Backtrace(s.Ctx)) > len(Backtrace(d.cursor.Ctx)) {
+			return ""
+		}
+		cursorBlockScope := d.cursor.Scope.ByLevel(ast.BlockScope)
+		if cursorBlockScope != nil {
+			// Skip over steps in a deeper scope.
+			stateBlockScope := s.Scope.ByLevel(ast.BlockScope)
+			if stateBlockScope != nil && stateBlockScope.Depth() > cursorBlockScope.Depth() {
+				return ""
+			}
+
+			// If there is a parent node in the same line as the current step, skip over
+			// this step.
+			// If n == nil, then we have left the cursorBlockScope and we should step.
+			n := ast.Find(cursorBlockScope.Node, s.Node.Position().Line, 0, ast.StopNodeFilter)
+			if n != nil && n.(ast.StopNode).Subject() != stop.Subject() {
+				return ""
+			}
+		}
+		return "step"
+	case DebugStepOut:
+		// Skip over steps in the same or deeper frames than the cursor.
+		if len(Backtrace(s.Ctx)) >= len(Backtrace(d.cursor.Ctx)) {
+			return ""
+		}
+		return "step"
+	case DebugContinue:
+		// Skip over steps that are not a stop node.
+		stop, ok := s.Node.(ast.StopNode)
+		if !ok {
+			// Except if we're reverse continue back to module.
+			if _, ok = s.Node.(*ast.Module); ok {
+				return "entry"
+			}
+			return ""
+		}
+
+		// Break if the stop node is one of the breakpoints.
+		for _, bp := range d.breakpoints {
+			if bp.Position().Filename == stop.Position().Filename &&
+				ast.IsPositionWithinNode(
+					stop.Subject(),
+					bp.Position().Line,
+					bp.Position().Column,
+				) {
+				return "breakpoint"
+			}
+		}
+	}
+	return ""
+}
+
+func (d *debugger) findSourceDefinedBreakpoints(mod *ast.Module) {
+	ast.Match(mod, ast.MatchOpts{},
+		func(block *ast.BlockStmt, call *ast.CallStmt) {
+			// If the surrounding block is an option block, skip checking because even
+			// if there is a breakpoint it is for its parent call stmt.
+			if strings.HasPrefix(string(block.Kind()), string(ast.Option)) {
+				return
+			}
+
+			if !call.Breakpoint() {
+				with := call.WithClause
+				if with == nil {
+					return
+				}
+
+				// Breakpoints can be either a call expr or within a func lit.
+				if with.Expr.CallExpr == nil || !with.Expr.CallExpr.Breakpoint() {
+					if with.Expr.FuncLit == nil {
+						return
+					}
+
+					breakpoint := false
+					for _, stmt := range with.Expr.FuncLit.Body.Stmts() {
+						if stmt.Call != nil && stmt.Call.Breakpoint() {
+							breakpoint = true
+							break
+						}
+					}
+
+					// Return if no breakpoint was found.
+					if !breakpoint {
+						return
+					}
+				}
+			}
+
+			bp := &Breakpoint{
+				Node:          call.Name,
+				SourceDefined: true,
+				dbgr:          d,
+			}
+			// If there is already a breakpoint there, then skip.
+			_, ok := d.breakpointIDs[bp.ID()]
+			if ok {
+				return
+			}
+			d.breakpointIDs[bp.ID()] = struct{}{}
+			d.sourceDefinedBreakpoints = append(d.sourceDefinedBreakpoints, bp)
+		},
+	)
+}
+
+// Breakpoint is an intentional stopping point in a program, put in place
+// by the source or debugger.
+type Breakpoint struct {
+	ast.Node
+
+	// Disabled is true if the breakpoint should not halt the program.
+	Disabled bool
+
+	// SourceDefined is true if the breakpoint is defined by the source.
+	SourceDefined bool
+
+	dbgr *debugger
+}
+
+func (bp *Breakpoint) ID() string {
+	return diagnostic.FormatPos(bp.Position())
+}
+
+func (bp *Breakpoint) Print(ctx context.Context, w io.Writer, cleared bool) {
+	index := 0
+	for i, breakpoint := range bp.dbgr.breakpoints {
+		if bp == breakpoint {
+			index = i
+			break
+		}
+	}
+
+	color := diagnostic.Color(ctx)
+	name := fmt.Sprintf("Breakpoint %d", index+1)
+	if cleared {
+		name = color.Sprintf(color.StrikeThrough(name))
+	}
+	fmt.Fprintf(w, color.Sprintf(
+		"%s at ",
+		color.Yellow(name),
+	))
+	err := bp.WithError(nil, bp.Spanf(diagnostic.Primary, ""))
+	for _, span := range diagnostic.Spans(err) {
+		fmt.Fprintln(w, span.Pretty(ctx, diagnostic.WithNumContext(1)))
+	}
+}
diff --git a/codegen/error_handler.go b/codegen/error_handler.go
new file mode 100644
index 0000000..58babcb
--- /dev/null
+++ b/codegen/error_handler.go
@@ -0,0 +1,34 @@
+package codegen
+
+import (
+	"context"
+
+	gateway "github.com/moby/buildkit/frontend/gateway/client"
+)
+
+type gatewayError struct {
+	context.Context
+	Client gateway.Client
+	err    error
+}
+
+func withGatewayError(ctx context.Context, c gateway.Client, err error) *gatewayError {
+	return &gatewayError{Context: ctx, Client: c, err: err}
+}
+
+func (e *gatewayError) Unwrap() error {
+	return e.err
+}
+
+func (e *gatewayError) Error() string {
+	return e.err.Error()
+}
+
+func (cg *CodeGen) errorHandler(ctx context.Context, c gateway.Client, gerr error) error {
+	if cg.dbgr == nil {
+		return gerr
+	}
+
+	s := cg.dbgr.recording[cg.dbgr.recordingIndex-1]
+	return cg.dbgr.yield(s.Ctx, s.Scope, s.Node, s.Value, s.Options, withGatewayError(ctx, c, gerr))
+}
diff --git a/codegen/gateway_exec.go b/codegen/gateway_exec.go
new file mode 100644
index 0000000..9893f0a
--- /dev/null
+++ b/codegen/gateway_exec.go
@@ -0,0 +1,325 @@
+package codegen
+
+import (
+	"context"
+	"fmt"
+	"io"
+	"os"
+
+	"github.com/docker/buildx/util/progress"
+	"github.com/moby/buildkit/client"
+	"github.com/moby/buildkit/client/llb"
+	gateway "github.com/moby/buildkit/frontend/gateway/client"
+	solvererrdefs "github.com/moby/buildkit/solver/errdefs"
+	"github.com/moby/buildkit/solver/pb"
+	"github.com/openllb/hlb/pkg/llbutil"
+	"github.com/openllb/hlb/solver"
+	"github.com/pkg/errors"
+	"golang.org/x/crypto/ssh/terminal"
+	"golang.org/x/sync/errgroup"
+)
+
+func ExecWithFS(ctx context.Context, cln *client.Client, fs Filesystem, opts Option, stdin io.Reader, stdout, stderr io.Writer, args ...string) error {
+	var (
+		securityMode pb.SecurityMode
+		netMode      pb.NetMode
+		extraHosts   []*pb.HostIP
+		secrets      []llbutil.SecretOption
+		ssh          []llbutil.SSHOption
+	)
+
+	cwd := "/"
+	if fs.Image.Config.WorkingDir != "" {
+		cwd = fs.Image.Config.WorkingDir
+	}
+
+	env := make([]string, len(fs.Image.Config.Env))
+	copy(env, fs.Image.Config.Env)
+
+	user := fs.Image.Config.User
+
+	mounts := []*llbutil.MountRunOption{
+		{
+			Source: fs.State,
+			Target: "/",
+		},
+	}
+
+	for _, opt := range opts {
+		switch o := opt.(type) {
+		case llbutil.ReadonlyRootFSOption:
+			mounts[0].Opts = append(mounts[0].Opts, llbutil.WithReadonlyMount())
+		case *llbutil.MountRunOption:
+			mounts = append(mounts, o)
+		case llbutil.UserOption:
+			user = o.User
+		case llbutil.DirOption:
+			cwd = o.Dir
+		case llbutil.EnvOption:
+			env = append(env, o.Name+"="+o.Value)
+		case llbutil.SecurityOption:
+			securityMode = o.SecurityMode
+		case llbutil.NetworkOption:
+			netMode = o.NetMode
+		case llbutil.HostOption:
+			extraHosts = append(extraHosts, &pb.HostIP{
+				Host: o.Host,
+				IP:   o.IP.String(),
+			})
+		case llbutil.SecretOption:
+			secrets = append(secrets, o)
+		case llbutil.SSHOption:
+			ssh = append(ssh, o)
+		case llbutil.SessionOption:
+			fs.SessionOpts = append(fs.SessionOpts, o)
+		}
+	}
+
+	s, err := llbutil.NewSession(ctx, fs.SessionOpts...)
+	if err != nil {
+		return err
+	}
+
+	g, ctx := errgroup.WithContext(ctx)
+
+	g.Go(func() error {
+		return s.Run(ctx, cln.Dialer())
+	})
+
+	g.Go(func() error {
+		var pw progress.Writer
+		mw := MultiWriter(ctx)
+		if mw != nil {
+			pw = mw.WithPrefix("", false)
+		}
+
+		return solver.Build(ctx, cln, s, pw, func(ctx context.Context, c gateway.Client) (res *gateway.Result, err error) {
+			ctrReq := gateway.NewContainerRequest{
+				NetMode:    netMode,
+				ExtraHosts: extraHosts,
+			}
+			for _, mount := range mounts {
+				gatewayMount := gateway.Mount{
+					Dest:      mount.Target,
+					MountType: pb.MountType_BIND,
+				}
+
+				for _, opt := range mount.Opts {
+					switch o := opt.(type) {
+					case llbutil.ReadonlyMountOption:
+						gatewayMount.Readonly = true
+					case llbutil.SourcePathMountOption:
+						gatewayMount.Selector = o.Path
+					case llbutil.CacheMountOption:
+						gatewayMount.MountType = pb.MountType_CACHE
+						gatewayMount.CacheOpt = &pb.CacheOpt{
+							ID:      o.ID,
+							Sharing: pb.CacheSharingOpt(o.Sharing),
+						}
+						switch o.Sharing {
+						case llb.CacheMountShared:
+							gatewayMount.CacheOpt.Sharing = pb.CacheSharingOpt_SHARED
+						case llb.CacheMountPrivate:
+							gatewayMount.CacheOpt.Sharing = pb.CacheSharingOpt_PRIVATE
+						case llb.CacheMountLocked:
+							gatewayMount.CacheOpt.Sharing = pb.CacheSharingOpt_LOCKED
+						default:
+							return nil, errors.Errorf("unrecognized cache sharing mode %v", o.Sharing)
+						}
+					case llbutil.TmpfsMountOption:
+						gatewayMount.MountType = pb.MountType_TMPFS
+					}
+				}
+
+				if gatewayMount.MountType == pb.MountType_BIND {
+					var def *llb.Definition
+					def, err = mount.Source.Marshal(ctx, llb.Platform(fs.Platform))
+					if err != nil {
+						return
+					}
+
+					res, err = c.Solve(ctx, gateway.SolveRequest{
+						Definition: def.ToPB(),
+					})
+					if err != nil {
+						return
+					}
+					gatewayMount.Ref = res.Ref
+				}
+
+				ctrReq.Mounts = append(ctrReq.Mounts, gatewayMount)
+			}
+
+			for _, secret := range secrets {
+				secretMount := gateway.Mount{
+					Dest:      secret.Dest,
+					MountType: pb.MountType_SECRET,
+					SecretOpt: &pb.SecretOpt{},
+				}
+				for _, opt := range secret.Opts {
+					switch o := opt.(type) {
+					case llbutil.IDOption:
+						secretMount.SecretOpt.ID = string(o)
+					case llbutil.UID:
+						secretMount.SecretOpt.Uid = uint32(o)
+					case llbutil.GID:
+						secretMount.SecretOpt.Gid = uint32(o)
+					case llbutil.Chmod:
+						secretMount.SecretOpt.Mode = uint32(o)
+					}
+				}
+				ctrReq.Mounts = append(ctrReq.Mounts, secretMount)
+			}
+
+			for i, sshOpt := range ssh {
+				sshMount := gateway.Mount{
+					Dest:      sshOpt.Dest,
+					MountType: pb.MountType_SSH,
+					SSHOpt:    &pb.SSHOpt{},
+				}
+				for _, opt := range sshOpt.Opts {
+					switch o := opt.(type) {
+					case llbutil.IDOption:
+						sshMount.SSHOpt.ID = string(o)
+					case llbutil.UID:
+						sshMount.SSHOpt.Uid = uint32(o)
+					case llbutil.GID:
+						sshMount.SSHOpt.Gid = uint32(o)
+					case llbutil.Chmod:
+						sshMount.SSHOpt.Mode = uint32(o)
+					}
+				}
+				if sshMount.Dest == "" {
+					sshMount.Dest = fmt.Sprintf("/run/buildkit/ssh_agent.%d", i)
+				}
+				if i == 0 {
+					env = append(env, "SSH_AUTH_SOCK="+sshMount.Dest)
+				}
+				ctrReq.Mounts = append(ctrReq.Mounts, sshMount)
+			}
+
+			ctr, err := c.NewContainer(ctx, ctrReq)
+			if err != nil {
+				return
+			}
+			defer ctr.Release(ctx)
+
+			p := Progress(ctx)
+			if p != nil {
+				err = p.Sync()
+				if err != nil {
+					return
+				}
+			}
+
+			startReq := gateway.StartRequest{
+				Args:         args,
+				Cwd:          cwd,
+				User:         user,
+				Env:          env,
+				Tty:          true,
+				Stdin:        io.NopCloser(stdin),
+				Stdout:       NopWriteCloser(stdout),
+				Stderr:       NopWriteCloser(stderr),
+				SecurityMode: securityMode,
+			}
+
+			proc, err := ctr.Start(ctx, startReq)
+			if err != nil {
+				return
+			}
+
+			oldState, err := terminal.MakeRaw(int(os.Stdin.Fd()))
+			if err == nil {
+				defer terminal.Restore(int(os.Stdin.Fd()), oldState)
+
+				cleanup := addResizeHandler(ctx, proc)
+				defer cleanup()
+			}
+
+			return res, proc.Wait()
+		}, fs.SolveOpts...)
+	})
+
+	return g.Wait()
+}
+
+func ExecWithSolveErr(ctx context.Context, c gateway.Client, se *solvererrdefs.SolveError, stdin io.ReadCloser, stdout, stderr io.Writer, args ...string) error {
+	op := se.Op
+	solveExec, ok := op.Op.(*pb.Op_Exec)
+	if !ok {
+		return nil
+	}
+
+	exec := solveExec.Exec
+
+	var mounts []gateway.Mount
+	for i, mnt := range exec.Mounts {
+		mounts = append(mounts, gateway.Mount{
+			Selector:  mnt.Selector,
+			Dest:      mnt.Dest,
+			ResultID:  se.Solve.MountIDs[i],
+			Readonly:  mnt.Readonly,
+			MountType: mnt.MountType,
+			CacheOpt:  mnt.CacheOpt,
+			SecretOpt: mnt.SecretOpt,
+			SSHOpt:    mnt.SSHOpt,
+		})
+	}
+
+	ctr, err := c.NewContainer(ctx, gateway.NewContainerRequest{
+		Mounts:      mounts,
+		NetMode:     exec.Network,
+		ExtraHosts:  exec.Meta.ExtraHosts,
+		Platform:    op.Platform,
+		Constraints: op.Constraints,
+	})
+	if err != nil {
+		return err
+	}
+	defer ctr.Release(ctx)
+
+	err = Progress(ctx).Sync()
+	if err != nil {
+		return err
+	}
+
+	startReq := gateway.StartRequest{
+		Args:         args,
+		Cwd:          exec.Meta.Cwd,
+		User:         exec.Meta.User,
+		Env:          exec.Meta.Env,
+		Tty:          true,
+		Stdin:        io.NopCloser(stdin),
+		Stdout:       NopWriteCloser(stdout),
+		Stderr:       NopWriteCloser(stderr),
+		SecurityMode: exec.Security,
+	}
+
+	proc, err := ctr.Start(ctx, startReq)
+	if err != nil {
+		return err
+	}
+
+	oldState, err := terminal.MakeRaw(int(os.Stdin.Fd()))
+	if err == nil {
+		defer terminal.Restore(int(os.Stdin.Fd()), oldState)
+
+		cleanup := addResizeHandler(ctx, proc)
+		defer cleanup()
+	}
+
+	return proc.Wait()
+}
+
+func NopWriteCloser(w io.Writer) io.WriteCloser {
+	return &nopWriteCloser{w}
+}
+
+type nopWriteCloser struct {
+	io.Writer
+}
+
+func (w *nopWriteCloser) Close() error {
+	return nil
+}
diff --git a/codegen/debug_unix.go b/codegen/gateway_exec_unix.go
similarity index 100%
rename from codegen/debug_unix.go
rename to codegen/gateway_exec_unix.go
diff --git a/codegen/debug_windows.go b/codegen/gateway_exec_windows.go
similarity index 100%
rename from codegen/debug_windows.go
rename to codegen/gateway_exec_windows.go
diff --git a/codegen/value.go b/codegen/value.go
index 0a088a4..3745718 100644
--- a/codegen/value.go
+++ b/codegen/value.go
@@ -114,6 +114,7 @@ func NewValue(ctx context.Context, iface interface{}) (Value, error) {
 	case Value:
 		return v, nil
 	case Filesystem:
+		v.SolveOpts = append(v.SolveOpts, GlobalSolveOpts(ctx)...)
 		return &fsValue{&nilValue{}, v}, validateState(v.State)
 	case llb.State:
 		zero := ZeroValue(ctx)
diff --git a/diagnostic/error.go b/diagnostic/error.go
index e3d1046..89cec98 100644
--- a/diagnostic/error.go
+++ b/diagnostic/error.go
@@ -3,6 +3,8 @@ package diagnostic
 import (
 	"context"
 	"errors"
+	"fmt"
+	"io"
 	"strings"
 
 	"github.com/moby/buildkit/solver/errdefs"
@@ -44,8 +46,47 @@ func Spans(err error) (spans []*SpanError) {
 	return
 }
 
-func Backtrace(ctx context.Context, err error) (spans []*SpanError) {
-	srcs := errdefs.Sources(err)
+func DisplayError(ctx context.Context, w io.Writer, spans []*SpanError, err error, printBacktrace bool) {
+	if len(spans) == 0 {
+		return
+	}
+
+	color := Color(ctx)
+	if err != nil {
+		fmt.Fprintf(w, color.Sprintf(
+			"%s: %s\n",
+			color.Bold(color.Red("error")),
+			color.Bold(Cause(err)),
+		))
+	}
+
+	for i, span := range spans {
+		if !printBacktrace && i != len(spans)-1 {
+			if i == 0 {
+				color := Color(ctx)
+				frame := "frame"
+				if len(spans) > 2 {
+					frame = "frames"
+				}
+				fmt.Fprintf(w, color.Sprintf(color.Cyan(" ⫶ %d %s hidden ⫶\n"), len(spans)-1, frame))
+			}
+			continue
+		}
+
+		pretty := span.Pretty(ctx, WithNumContext(2))
+		lines := strings.Split(pretty, "\n")
+		for j, line := range lines {
+			if j == 0 {
+				lines[j] = fmt.Sprintf(" %d: %s", i+1, line)
+			} else {
+				lines[j] = fmt.Sprintf("    %s", line)
+			}
+		}
+		fmt.Fprintf(w, "%s\n", strings.Join(lines, "\n"))
+	}
+}
+
+func SourcesToSpans(ctx context.Context, srcs []*errdefs.Source, err error) (spans []*SpanError) {
 	for i, src := range srcs {
 		fb := filebuffer.Buffers(ctx).Get(src.Info.Filename)
 		if fb != nil {
@@ -71,8 +112,9 @@ func Backtrace(ctx context.Context, err error) (spans []*SpanError) {
 				msg = Cause(err)
 			}
 
-			start := fb.PositionFromProto(src.Ranges[0].Start)
-			end := fb.PositionFromProto(src.Ranges[0].End)
+			loc := src.Ranges[0]
+			start := fb.Position(int(loc.Start.Line), int(loc.Start.Character))
+			end := fb.Position(int(loc.End.Line), int(loc.End.Character))
 			se := WithError(nil, start, end, Spanf(Primary, start, end, msg))
 
 			var span *SpanError
@@ -85,5 +127,8 @@ func Backtrace(ctx context.Context, err error) (spans []*SpanError) {
 }
 
 func Cause(err error) string {
+	if err == nil {
+		return ""
+	}
 	return strings.TrimPrefix(perrors.Cause(err).Error(), "rpc error: code = Unknown desc = ")
 }
diff --git a/docs/reference.md b/docs/reference.md
index 34dfa41..686dc7c 100644
--- a/docs/reference.md
+++ b/docs/reference.md
@@ -1,18 +1,4 @@
 ## <span class='hlb-type'>fs</span> functions
-### <span class='hlb-type'>fs</span> <span class='hlb-name'>breakpoint</span>(<span class='hlb-type'>string</span> <span class='hlb-variable'>command</span>)
-
-!!! info "<span class='hlb-type'>string</span> <span class='hlb-variable'>command</span>"
-	
-
-
-
-	#!hlb
-	fs default() {
-		breakpoint "command"
-	}
-
-
-
 ### <span class='hlb-type'>fs</span> <span class='hlb-name'>cmd</span>(<span class='hlb-type'>string</span> <span class='hlb-variable'>args</span>)
 
 !!! info "<span class='hlb-type'>string</span> <span class='hlb-variable'>args</span>"
@@ -588,7 +574,6 @@
 	#!hlb
 	fs default() {
 		run "arg" with option {
-			breakpoint "command"
 			dir "path"
 			env "key" "value"
 			forward "src" "dest"
@@ -606,13 +591,6 @@
 	}
 
 
-#### <span class='hlb-type'>option::run</span> <span class='hlb-name'>breakpoint</span>(<span class='hlb-type'>string</span> <span class='hlb-variable'>command</span>)
-
-!!! info "<span class='hlb-type'>string</span> <span class='hlb-variable'>command</span>"
-	
-
-
-
 #### <span class='hlb-type'>option::run</span> <span class='hlb-name'>dir</span>(<span class='hlb-type'>string</span> <span class='hlb-variable'>path</span>)
 
 !!! info "<span class='hlb-type'>string</span> <span class='hlb-variable'>path</span>"
diff --git a/go.mod b/go.mod
index ac49c8a..0391b4c 100644
--- a/go.mod
+++ b/go.mod
@@ -4,11 +4,12 @@ go 1.16
 
 require (
 	github.com/alecthomas/participle v1.0.0-alpha2
+	github.com/chzyer/readline v0.0.0-20180603132655-2972be24d48e
 	github.com/containerd/console v1.0.3
 	github.com/containerd/containerd v1.6.0-beta.3
 	github.com/creachadair/jrpc2 v0.26.1
 	github.com/creack/pty v1.1.11
-	github.com/docker/buildx v0.7.1-0.20220204032525-595285736c66
+	github.com/docker/buildx v0.7.1-0.20220207185201-60a025b22768
 	github.com/docker/cli v20.10.11+incompatible
 	github.com/docker/distribution v2.7.1+incompatible
 	github.com/docker/docker v20.10.7+incompatible
@@ -31,13 +32,14 @@ require (
 	github.com/xlab/treeprint v1.0.0
 	golang.org/x/crypto v0.0.0-20211202192323-5770296d904e
 	golang.org/x/sync v0.0.0-20210220032951-036812b2e83c
-	golang.org/x/sys v0.0.0-20211123173158-ef496fb156ab
+	golang.org/x/sys v0.0.0-20220209214540-3681064d5158
 	google.golang.org/grpc v1.42.0
 )
 
 replace (
 	github.com/docker/cli => github.com/docker/cli v20.10.3-0.20210702143511-f782d1355eff+incompatible
 	github.com/docker/docker => github.com/docker/docker v20.10.3-0.20211216190657-088afc99e4bf+incompatible
+	github.com/peterh/liner => github.com/hinshun/liner v1.2.3-0.20220210021123-9c016f2646f8
 	github.com/sourcegraph/go-lsp => github.com/radeksimko/go-lsp v0.0.0-20200223162147-9f2c54f29c9f
 	go.opentelemetry.io/contrib/instrumentation/net/http/httptrace/otelhttptrace => github.com/tonistiigi/opentelemetry-go-contrib/instrumentation/net/http/httptrace/otelhttptrace v0.0.0-20211026174723-2f82a1e0c997
 	go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp => github.com/tonistiigi/opentelemetry-go-contrib/instrumentation/net/http/otelhttp v0.0.0-20211026174723-2f82a1e0c997
diff --git a/go.sum b/go.sum
index b707c32..6e4cac0 100644
--- a/go.sum
+++ b/go.sum
@@ -259,8 +259,11 @@ github.com/cespare/xxhash/v2 v2.1.1 h1:6MnRN8NT7+YBpUIWxHtefFZOKTAPgGjpQSxqLNn0+
 github.com/cespare/xxhash/v2 v2.1.1/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs=
 github.com/checkpoint-restore/go-criu/v4 v4.1.0/go.mod h1:xUQBLp4RLc5zJtWY++yjOoMoB5lihDt7fai+75m+rGw=
 github.com/checkpoint-restore/go-criu/v5 v5.0.0/go.mod h1:cfwC0EG7HMUenopBsUf9d89JlCLQIfgVcNsNN0t6T2M=
+github.com/chzyer/logex v1.1.10 h1:Swpa1K6QvQznwJRcfTfQJmTE72DqScAa40E+fbHEXEE=
 github.com/chzyer/logex v1.1.10/go.mod h1:+Ywpsq7O8HXn0nuIou7OrIPyXbp3wmkHB+jjWRnGsAI=
+github.com/chzyer/readline v0.0.0-20180603132655-2972be24d48e h1:fY5BOSpyZCqRo5OhCuC+XN+r/bBCmeuuJtjz+bCNIf8=
 github.com/chzyer/readline v0.0.0-20180603132655-2972be24d48e/go.mod h1:nSuG5e5PlCu98SY8svDHJxuZscDgtXS6KTTbou5AhLI=
+github.com/chzyer/test v0.0.0-20180213035817-a1ea475d72b1 h1:q763qf9huN11kDQavWsoZXJNW3xEE4JJyHa5Q25/sd8=
 github.com/chzyer/test v0.0.0-20180213035817-a1ea475d72b1/go.mod h1:Q3SI9o4m/ZMnBNeIyt5eFwwo7qiLfzFZmjNmxjkiQlU=
 github.com/cilium/ebpf v0.0.0-20200110133405-4032b1d8aae3/go.mod h1:MA5e5Lr8slmEg9bt0VpxxWqJlO4iwu3FBdHUzV7wQVg=
 github.com/cilium/ebpf v0.0.0-20200702112145-1c8d4c9ef775/go.mod h1:7cR51M8ViRLIdUjrmSXlK9pkrsDlLHbO8jiB8X8JnOc=
@@ -448,8 +451,8 @@ github.com/dimchansky/utfbom v1.1.0/go.mod h1:rO41eb7gLfo8SF1jd9F8HplJm1Fewwi4mQ
 github.com/dimchansky/utfbom v1.1.1/go.mod h1:SxdoEBH5qIqFocHMyGOXVAybYJdr71b1Q/j0mACtrfE=
 github.com/distribution/distribution/v3 v3.0.0-20210316161203-a01c71e2477e/go.mod h1:xpWTC2KnJMiDLkoawhsPQcXjvwATEBcbq0xevG2YR9M=
 github.com/dnaeon/go-vcr v1.0.1/go.mod h1:aBB1+wY4s93YsC3HHjMBMrwTj2R9FHDzUr9KyGc8n1E=
-github.com/docker/buildx v0.7.1-0.20220204032525-595285736c66 h1:/TTMjaV7clfa1EIQTDGrhd0/AoYQweSsfUf194BWXdY=
-github.com/docker/buildx v0.7.1-0.20220204032525-595285736c66/go.mod h1:7nCeQ7ysmY5Im/vm5mk4fF3ZyOxaGKD920tJqT8b8vg=
+github.com/docker/buildx v0.7.1-0.20220207185201-60a025b22768 h1:JqsaZFzU5FiqJowo+h35kEZhfQv/ducQFVL2AOI3XjY=
+github.com/docker/buildx v0.7.1-0.20220207185201-60a025b22768/go.mod h1:7nCeQ7ysmY5Im/vm5mk4fF3ZyOxaGKD920tJqT8b8vg=
 github.com/docker/cli v20.10.3-0.20210702143511-f782d1355eff+incompatible h1:CaaxCD/l9Dxogu6lxf7AQautlv3sHULrasPadayp0fM=
 github.com/docker/cli v20.10.3-0.20210702143511-f782d1355eff+incompatible/go.mod h1:JLrzqnKDaYBop7H2jaqPtU4hHvMKP+vjCwu2uszcLI8=
 github.com/docker/cli-docs-tool v0.3.0/go.mod h1:rgW5KKdNpLMBIuH4WQ/1RNh38nH+/Ay5jgL4P0ZMPpY=
@@ -1724,8 +1727,9 @@ golang.org/x/sys v0.0.0-20210809222454-d867a43fc93e/go.mod h1:oPkhp1MJrh7nUepCBc
 golang.org/x/sys v0.0.0-20210820121016-41cdb8703e55/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
 golang.org/x/sys v0.0.0-20210831042530-f4d43177bf5e/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
 golang.org/x/sys v0.0.0-20211025201205-69cdffdb9359/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
-golang.org/x/sys v0.0.0-20211123173158-ef496fb156ab h1:rfJ1bsoJQQIAoAxTxB7bme+vHrNkRw8CqfsYh9w54cw=
 golang.org/x/sys v0.0.0-20211123173158-ef496fb156ab/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
+golang.org/x/sys v0.0.0-20220209214540-3681064d5158 h1:rm+CHSpPEEW2IsXUib1ThaHIjuBVZjxNgSKmBLFfD4c=
+golang.org/x/sys v0.0.0-20220209214540-3681064d5158/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
 golang.org/x/term v0.0.0-20201117132131-f5c789dd3221/go.mod h1:Nr5EML6q2oocZ2LXRh80K7BxOlk5/8JxuGnuhpl+muw=
 golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
 golang.org/x/term v0.0.0-20210220032956-6a3ed077a48d h1:SZxvLBoTP5yHO3Frd4z4vrF+DBX9vMVanchswa69toE=
diff --git a/hlb.go b/hlb.go
index 5a219aa..76fba6a 100644
--- a/hlb.go
+++ b/hlb.go
@@ -16,7 +16,7 @@ import (
 	"github.com/openllb/hlb/solver"
 )
 
-func Compile(ctx context.Context, cln *client.Client, mod *ast.Module, targets []codegen.Target, opts ...codegen.CodeGenOption) (solver.Request, error) {
+func Compile(ctx context.Context, cln *client.Client, mod *ast.Module, targets []codegen.Target) (solver.Request, error) {
 	err := checker.SemanticPass(mod)
 	if err != nil {
 		return nil, err
@@ -39,11 +39,7 @@ func Compile(ctx context.Context, cln *client.Client, mod *ast.Module, targets [
 		return nil, err
 	}
 
-	cg, err := codegen.New(cln, resolver, opts...)
-	if err != nil {
-		return nil, err
-	}
-
+	cg := codegen.New(cln, resolver)
 	ctx = codegen.WithSessionID(ctx, identity.NewID())
 	return cg.Generate(ctx, mod, targets)
 }
diff --git a/langserver/server.go b/langserver/server.go
index 5db2564..14ad1c9 100644
--- a/langserver/server.go
+++ b/langserver/server.go
@@ -553,12 +553,7 @@ func (ls *LangServer) textDocumentDefinitionHandler(ctx context.Context, params
 				return
 			}
 
-			cg, err := codegen.New(ls.cln, ls.resolver)
-			if err != nil {
-				log.Printf("failed to create codegen: %s", err)
-				return
-			}
-
+			cg := codegen.New(ls.cln, ls.resolver)
 			ctx = codegen.WithProgramCounter(ctx, id.Expr)
 			imod, filename, err := cg.EmitImport(ctx, td.Module, id)
 			if err != nil {
diff --git a/language/builtin.hlb b/language/builtin.hlb
index b063c12..712b678 100644
--- a/language/builtin.hlb
+++ b/language/builtin.hlb
@@ -223,17 +223,6 @@ option::run secret(string localPath, string mountPoint)
 # @return an option to mount an additional filesystem.
 option::run mount(fs input, string mountPoint) binds (fs target)
 
-# Sets a debugger breakpoint just before the "run" command.
-#
-# @param command optional command to invoke for the debugger "exec" command,
-# followed by optional arguments
-# If no value is given, it will execute /bin/sh.
-# If exactly one value is given, the specified command will be invoked.
-# If more than one value is given, values after the first one are treated as
-# arguments for the command.
-# @return an option to set a breakpoint
-option::run breakpoint(variadic string command)
-
 # Sets the target directory to mount the SSH agent socket. By default, it is
 # mounted to "/run/buildkit/ssh_agent.${N}", where N is the index of the 
 # socket. If $SSH_AUTH_SOCK is not set, it will set SSH_AUTH_SOCK to the
@@ -703,14 +692,3 @@ option::template stringField(string name, string value)
 # @param pipelines the targets to run in parallel.
 # @return a pipeline that returns when all its targets have finished.
 pipeline stage(variadic pipeline pipelines)
-
-# Sets a static breakpoint in the debugger.
-#
-# @param command optional command to invoke for the debugger "exec" command at
-# this breakpoint, followed by optional arguments
-# If no value is given, it will execute /bin/sh.
-# If exactly one value is given, the specified command will be invoked.
-# If more than one value is given, values after the first one are treated as
-# arguments for the command.
-# @return the filesystem after the breakpoint
-fs breakpoint(variadic string command)
diff --git a/module/resolve.go b/module/resolve.go
index d326733..7bc95a3 100644
--- a/module/resolve.go
+++ b/module/resolve.go
@@ -313,13 +313,8 @@ type resolveGraphInfo struct {
 
 // ResolveGraph traverses the import graph of a given module.
 func ResolveGraph(ctx context.Context, cln *client.Client, resolver codegen.Resolver, mod *ast.Module, visitor Visitor) error {
-	cg, err := codegen.New(cln, resolver)
-	if err != nil {
-		return err
-	}
-
 	info := &resolveGraphInfo{
-		cg:      cg,
+		cg:      codegen.New(cln, resolver),
 		visitor: visitor,
 	}
 	return resolveGraph(ctx, info, mod)
diff --git a/parser/ast/ast.go b/parser/ast/ast.go
index 8de77c9..2763a97 100644
--- a/parser/ast/ast.go
+++ b/parser/ast/ast.go
@@ -103,6 +103,11 @@ type Node interface {
 	Spanf(t diagnostic.Type, format string, a ...interface{}) diagnostic.Option
 }
 
+type StopNode interface {
+	Node
+	Subject() Node
+}
+
 type Mixin struct {
 	Pos    lexer.Position
 	EndPos lexer.Position
@@ -128,7 +133,7 @@ func (m Mixin) Spanf(t diagnostic.Type, format string, a ...interface{}) diagnos
 type Kind string
 
 const (
-	None       Kind = ""
+	None       Kind = "none"
 	String     Kind = "string"
 	Int        Kind = "int"
 	Bool       Kind = "bool"
@@ -278,6 +283,10 @@ func (fs *FuncSignature) Kind() Kind {
 	return fs.Type.Kind
 }
 
+func (fs *FuncSignature) Subject() Node {
+	return fs.Name
+}
+
 // Type represents an object type.
 type Type struct {
 	Mixin
@@ -459,10 +468,11 @@ func NewCallStmt(name string, args []*Expr, with *WithClause, binds *BindClause)
 	}
 }
 
-func (cs *CallStmt) Breakpoint(returnType Kind) bool {
-	if returnType == Kind(fmt.Sprintf("%s::%s", Option, "run")) {
-		return false
-	}
+func (cs *CallStmt) Subject() Node {
+	return cs.Name
+}
+
+func (cs *CallStmt) Breakpoint() bool {
 	if cs.Name == nil || cs.Name.Ident == nil {
 		return false
 	}
@@ -832,6 +842,17 @@ type CallExpr struct {
 	List      *ExprList  `parser:"@@?"`
 }
 
+func (ce *CallExpr) Subject() Node {
+	return ce.Name
+}
+
+func (ce *CallExpr) Breakpoint() bool {
+	if ce.Name == nil || ce.Name.Ident == nil {
+		return false
+	}
+	return ce.Name.Ident.Text == "breakpoint"
+}
+
 func (ce *CallExpr) Args() []*Expr {
 	var args []*Expr
 	if ce.List != nil {
diff --git a/parser/ast/scope.go b/parser/ast/scope.go
index f14386c..9939ca6 100644
--- a/parser/ast/scope.go
+++ b/parser/ast/scope.go
@@ -1,28 +1,56 @@
 package ast
 
 import (
-	"sort"
-
 	"github.com/openllb/hlb/diagnostic"
 )
 
+type ScopeLevel string
+
+var (
+	BuiltinScope  ScopeLevel = "Builtins"
+	ModuleScope   ScopeLevel = "Module"
+	FunctionScope ScopeLevel = "Function"
+	BlockScope    ScopeLevel = "Block"
+	ArgsScope     ScopeLevel = "Arguments"
+)
+
 // Scope maintains the set of named language entities declared in the scope
 // and a link to the immediately surrounding (outer) scope.
 type Scope struct {
-	Node    Node
+	Node
+	Level   ScopeLevel
 	Outer   *Scope
 	Objects map[string]*Object
 }
 
 // NewScope creates a new scope linking to an outer scope.
-func NewScope(node Node, outer *Scope) *Scope {
+func NewScope(outer *Scope, level ScopeLevel, node Node) *Scope {
 	return &Scope{
 		Node:    node,
+		Level:   level,
 		Outer:   outer,
 		Objects: make(map[string]*Object),
 	}
 }
 
+func (s *Scope) Depth() int {
+	depth := 1
+	if s.Outer != nil {
+		depth += s.Outer.Depth()
+	}
+	return depth
+}
+
+func (s *Scope) ByLevel(level ScopeLevel) *Scope {
+	if s.Level == level {
+		return s
+	}
+	if s.Outer != nil {
+		return s.Outer.ByLevel(level)
+	}
+	return nil
+}
+
 // Lookup returns the object with the given name if it is
 // found in scope, otherwise it returns nil.
 func (s *Scope) Lookup(name string) *Object {
@@ -59,28 +87,11 @@ func (s *Scope) Insert(obj *Object) {
 	s.Objects[obj.Ident.Text] = obj
 }
 
-// Root returns the outer-most scope.
-func (s *Scope) Root() *Scope {
-	// Outer-most scope is the one right before a global scope of all the
-	// builtins, which doesn't have a node defined.
-	if s.Outer.Node == nil {
-		return s
-	}
-	return s.Outer.Root()
-}
-
-// Defined returns all objects with the given kind.
-func (s *Scope) Defined() []*Object {
+func (s *Scope) Locals() []*Object {
 	var objs []*Object
-	if s.Outer != nil {
-		objs = s.Outer.Defined()
-	}
 	for _, obj := range s.Objects {
 		objs = append(objs, obj)
 	}
-	sort.SliceStable(objs, func(i, j int) bool {
-		return objs[i].Ident.Text < objs[j].Ident.Text
-	})
 	return objs
 }
 
diff --git a/parser/ast/walk.go b/parser/ast/walk.go
index 6514137..7f281d3 100644
--- a/parser/ast/walk.go
+++ b/parser/ast/walk.go
@@ -424,6 +424,12 @@ func Find(root Node, line, column int, filter func(Node) bool) Node {
 	return f.match
 }
 
+// StopNodeFilter returns true if Node n is a StopNode.
+func StopNodeFilter(n Node) bool {
+	_, ok := n.(StopNode)
+	return ok
+}
+
 // IsPositionWithinNode returns true if a line column is within a node's position.
 func IsPositionWithinNode(node Node, line, column int) bool {
 	return IsIntersect(node.Position(), node.End(), line, column)
diff --git a/parser/parse.go b/parser/parse.go
index 4db42aa..9239f9d 100644
--- a/parser/parse.go
+++ b/parser/parse.go
@@ -12,7 +12,7 @@ import (
 	"golang.org/x/sync/errgroup"
 )
 
-func Parse(ctx context.Context, r io.Reader) (*ast.Module, error) {
+func Parse(ctx context.Context, r io.Reader, opts ...filebuffer.Option) (*ast.Module, error) {
 	name := lexer.NameOfReader(r)
 	if name == "" {
 		name = "<stdin>"
@@ -22,7 +22,7 @@ func Parse(ctx context.Context, r io.Reader) (*ast.Module, error) {
 	mod := &ast.Module{}
 	defer AssignDocStrings(mod)
 
-	fb := filebuffer.New(name)
+	fb := filebuffer.New(name, opts...)
 	r = io.TeeReader(r, fb)
 	defer func() {
 		if mod.Pos.Filename != "" {
diff --git a/pkg/filebuffer/filebuffer.go b/pkg/filebuffer/filebuffer.go
index 5286a3c..8fc665b 100644
--- a/pkg/filebuffer/filebuffer.go
+++ b/pkg/filebuffer/filebuffer.go
@@ -10,7 +10,6 @@ import (
 
 	"github.com/alecthomas/participle/lexer"
 	"github.com/moby/buildkit/client/llb"
-	"github.com/moby/buildkit/solver/pb"
 )
 
 type buffersKey struct{}
@@ -51,31 +50,45 @@ func (b *BufferLookup) Set(filename string, fb *FileBuffer) {
 }
 
 type FileBuffer struct {
-	filename  string
-	buf       bytes.Buffer
-	offset    int
-	offsets   []int
-	sourceMap *llb.SourceMap
-	mu        sync.Mutex
+	filename string
+	buf      bytes.Buffer
+	offset   int
+	offsets  []int
+	mu       sync.Mutex
+	onDisk   bool
 }
 
-func New(filename string) *FileBuffer {
-	return &FileBuffer{
+type Option func(*FileBuffer)
+
+func WithEphemeral() Option {
+	return func(fb *FileBuffer) {
+		fb.onDisk = false
+	}
+}
+
+func New(filename string, opts ...Option) *FileBuffer {
+	fb := &FileBuffer{
 		filename: filename,
+		onDisk:   true,
 	}
+	for _, opt := range opts {
+		opt(fb)
+	}
+	return fb
 }
 
 func (fb *FileBuffer) Filename() string {
 	return fb.filename
 }
 
+func (fb *FileBuffer) OnDisk() bool {
+	return fb.onDisk
+}
+
 func (fb *FileBuffer) SourceMap() *llb.SourceMap {
 	fb.mu.Lock()
 	defer fb.mu.Unlock()
-	if fb.sourceMap == nil {
-		fb.sourceMap = llb.NewSourceMap(nil, fb.filename, fb.buf.Bytes())
-	}
-	return fb.sourceMap
+	return llb.NewSourceMap(nil, fb.filename, fb.buf.Bytes())
 }
 
 func (fb *FileBuffer) Len() int {
@@ -101,8 +114,7 @@ func (fb *FileBuffer) Write(p []byte) (n int, err error) {
 	return n, err
 }
 
-func (fb *FileBuffer) PositionFromProto(pos pb.Position) lexer.Position {
-	line, column := int(pos.Line), int(pos.Character)
+func (fb *FileBuffer) Position(line, column int) lexer.Position {
 	var offset int
 	if line-2 < 0 {
 		offset = column - 1
diff --git a/solver/progress.go b/solver/progress.go
index ea40fa3..6c45b0c 100644
--- a/solver/progress.go
+++ b/solver/progress.go
@@ -25,24 +25,33 @@ type Console interface {
 	Name() string
 }
 
-type ProgressOption func(*ProgressInfo) error
+type ProgressOption func(*progressInfo) error
 
-type ProgressInfo struct {
-	Console   Console
-	LogOutput LogOutput
+type progressInfo struct {
+	writer    io.Writer
+	console   Console
+	logOutput logOutput
 }
 
-type LogOutput int
+type logOutput int
 
 const (
-	LogOutputTTY LogOutput = iota
-	LogOutputPlain
+	logOutputTTY logOutput = iota
+	logOutputPlain
 )
 
-func WithLogOutput(con Console, logOutput LogOutput) ProgressOption {
-	return func(info *ProgressInfo) error {
-		info.Console = con
-		info.LogOutput = logOutput
+func WithLogOutputPlain(w io.Writer) ProgressOption {
+	return func(info *progressInfo) error {
+		info.writer = w
+		info.logOutput = logOutputPlain
+		return nil
+	}
+}
+
+func WithLogOutputTTY(con Console) ProgressOption {
+	return func(info *progressInfo) error {
+		info.console = con
+		info.logOutput = logOutputTTY
 		return nil
 	}
 }
@@ -59,27 +68,25 @@ type Progress interface {
 // NewProgress returns a Progress that presents all the progress on multiple
 // solves to the terminal stdout.
 func NewProgress(ctx context.Context, opts ...ProgressOption) (Progress, error) {
-	info := &ProgressInfo{
-		Console: os.Stderr,
-	}
+	info := progressInfo{console: os.Stderr}
 	for _, opt := range opts {
-		err := opt(info)
+		err := opt(&info)
 		if err != nil {
 			return nil, err
 		}
 	}
 
 	var mode string
-	switch info.LogOutput {
-	case LogOutputTTY:
+	switch info.logOutput {
+	case logOutputTTY:
 		mode = "tty"
-	case LogOutputPlain:
+	case logOutputPlain:
 		mode = "plain"
 	default:
-		return nil, errors.Errorf("unknown log output %q", info.LogOutput)
+		return nil, errors.Errorf("unknown log output %q", info.logOutput)
 	}
 
-	spp := newSyncProgressPrinter(info.Console, mode)
+	spp := newSyncProgressPrinter(info.writer, info.console, mode)
 	p := &progressUI{
 		origCtx: ctx,
 		spp:     spp,
@@ -139,6 +146,7 @@ func (p *progressUI) waitNoLock() error {
 type syncProgressPrinter struct {
 	mu     sync.Mutex
 	p      *progress.Printer
+	w      io.Writer
 	out    console.File
 	cancel func()
 	mode   string
@@ -147,8 +155,9 @@ type syncProgressPrinter struct {
 
 var _ progress.Writer = (*syncProgressPrinter)(nil)
 
-func newSyncProgressPrinter(out console.File, mode string) *syncProgressPrinter {
+func newSyncProgressPrinter(w io.Writer, out console.File, mode string) *syncProgressPrinter {
 	spp := &syncProgressPrinter{
+		w:    w,
 		out:  out,
 		mode: mode,
 	}
@@ -164,7 +173,7 @@ func (spp *syncProgressPrinter) reset() {
 	defer spp.mu.Unlock()
 	spp.cancel = cancel
 	spp.done = make(chan struct{})
-	spp.p = progress.NewPrinter(pctx, spp.out, spp.mode)
+	spp.p = progress.NewPrinter(pctx, spp.w, spp.out, spp.mode)
 }
 
 func (spp *syncProgressPrinter) Write(s *client.SolveStatus) {
diff --git a/solver/solve.go b/solver/solve.go
index a4bc047..33aa68a 100644
--- a/solver/solve.go
+++ b/solver/solve.go
@@ -21,7 +21,6 @@ type SolveCallback func(ctx context.Context, resp *client.SolveResponse) error
 
 type SolveInfo struct {
 	Evaluate              bool
-	ErrorHandler          func(context.Context, gateway.Client, error)
 	OutputMoby            bool
 	OutputDockerRef       string
 	OutputPushImage       string
@@ -30,6 +29,7 @@ type SolveInfo struct {
 	OutputLocalOCITarball bool
 	Callbacks             []SolveCallback `json:"-"`
 	ImageSpec             *ImageSpec
+	ErrorHandler          ErrorHandler
 	Entitlements          []entitlements.Entitlement
 }
 
@@ -125,7 +125,9 @@ func WithEvaluate(info *SolveInfo) error {
 	return nil
 }
 
-func WithErrorHandler(errorHandler func(context.Context, gateway.Client, error)) SolveOption {
+type ErrorHandler func(context.Context, gateway.Client, error) error
+
+func WithErrorHandler(errorHandler ErrorHandler) SolveOption {
 	return func(info *SolveInfo) error {
 		info.ErrorHandler = errorHandler
 		return nil
@@ -148,7 +150,7 @@ func Solve(ctx context.Context, c *client.Client, s *session.Session, pw progres
 		})
 		if err != nil {
 			if info.ErrorHandler != nil {
-				info.ErrorHandler(ctx, c, err)
+				return nil, info.ErrorHandler(ctx, c, err)
 			}
 			return nil, err
 		}


Test patch:
diff --git a/codegen/codegen_test.go b/codegen/codegen_test.go
index eab7470..a1a9e87 100644
--- a/codegen/codegen_test.go
+++ b/codegen/codegen_test.go
@@ -975,9 +975,7 @@ func TestCodeGen(t *testing.T) {
 				targets = append(targets, codegen.Target{Name: target})
 			}
 
-			cg, err := codegen.New(nil, nil)
-			require.NoError(t, err, tc.name)
-
+			cg := codegen.New(nil, nil)
 			ctx = codegen.WithSessionID(ctx, identity.NewID())
 			request, err := cg.Generate(ctx, mod, targets)
 			require.NoError(t, err, tc.name)
@@ -1045,9 +1043,7 @@ func TestCodegenError(t *testing.T) {
 				targets = append(targets, codegen.Target{Name: target})
 			}
 
-			cg, err := codegen.New(nil, nil)
-			require.NoError(t, err, tc.name)
-
+			cg := codegen.New(nil, nil)
 			ctx = codegen.WithSessionID(ctx, identity.NewID())
 			_, err = cg.Generate(ctx, mod, targets)
 			var expected error
@@ -1305,12 +1301,8 @@ func TestCodeGenImport(t *testing.T) {
 				expected := tc.fn(mod)
 				validateError(t, ctx, expected, actual, tc.name)
 			} else {
-				cg, err := codegen.New(nil, nil)
-				require.NoError(t, err, tc.name)
-
-				ctx = codegen.WithSessionID(ctx, identity.NewID())
-				targets := []codegen.Target{{Name: "default"}}
-				_, err = cg.Generate(ctx, mod, targets)
+				cg := codegen.New(nil, nil)
+				_, err = cg.Generate(ctx, mod, []codegen.Target{{Name: "default"}})
 				validateError(t, ctx, nil, err, tc.name)
 			}
 		})
diff --git a/codegen/debug/linespec_test.go b/codegen/debug/linespec_test.go
new file mode 100644
index 0000000..8451998
--- /dev/null
+++ b/codegen/debug/linespec_test.go
@@ -0,0 +1,250 @@
+package debug
+
+import (
+	"context"
+	"strings"
+	"testing"
+
+	"github.com/lithammer/dedent"
+	"github.com/openllb/hlb/builtin"
+	"github.com/openllb/hlb/checker"
+	"github.com/openllb/hlb/parser"
+	"github.com/openllb/hlb/parser/ast"
+	"github.com/openllb/hlb/pkg/filebuffer"
+	"github.com/stretchr/testify/require"
+)
+
+func cleanup(value string) string {
+	return strings.TrimSpace(dedent.Dedent(value)) + "\n"
+}
+
+func TestParseLinespec(t *testing.T) {
+	type testCase struct {
+		name     string
+		linespec string
+		files    map[string]string
+		start    func(ctx context.Context) ast.Node
+		end      func(ctx context.Context) ast.Node
+	}
+
+	for _, tc := range []testCase{{
+		"line match",
+		"3",
+		map[string]string{
+			"build.hlb": `
+			fs default() {
+				image "alpine"
+				run "echo hello"
+			}
+			`,
+		},
+		nil,
+		func(ctx context.Context) ast.Node {
+			mod := ast.Modules(ctx).Get("build.hlb")
+			return ast.Search(mod, `run "echo hello"`)
+		},
+	}, {
+		"line no match",
+		"4",
+		map[string]string{
+			"build.hlb": `
+			fs default() {
+				image "alpine"
+				run "echo hello"
+			}
+			`,
+		},
+		nil,
+		func(ctx context.Context) ast.Node {
+			return nil
+		},
+	}, {
+		"+offset match",
+		"+1",
+		map[string]string{
+			"build.hlb": `
+			fs default() {
+				image "alpine"
+				run "echo hello"
+			}
+			`,
+		},
+		func(ctx context.Context) ast.Node {
+			mod := ast.Modules(ctx).Get("build.hlb")
+			return ast.Search(mod, `image "alpine"`)
+		},
+		func(ctx context.Context) ast.Node {
+			mod := ast.Modules(ctx).Get("build.hlb")
+			return ast.Search(mod, `run "echo hello"`)
+		},
+	}, {
+		"-offset match",
+		"-1",
+		map[string]string{
+			"build.hlb": `
+			fs default() {
+				image "alpine"
+				run "echo hello"
+			}
+			`,
+		},
+		func(ctx context.Context) ast.Node {
+			mod := ast.Modules(ctx).Get("build.hlb")
+			return ast.Search(mod, `run "echo hello"`)
+		},
+		func(ctx context.Context) ast.Node {
+			mod := ast.Modules(ctx).Get("build.hlb")
+			return ast.Search(mod, `image "alpine"`)
+		},
+	}, {
+		"function match",
+		"default",
+		map[string]string{
+			"build.hlb": `
+			fs default() {
+				image "alpine"
+				run "echo hello"
+			}
+			`,
+		},
+		nil,
+		func(ctx context.Context) ast.Node {
+			mod := ast.Modules(ctx).Get("build.hlb")
+			return ast.Search(mod, `fs default()`)
+		},
+	}, {
+		"function:line match",
+		"bar:2",
+		map[string]string{
+			"build.hlb": `
+			fs default()
+			fs bar() {
+				image "alpine"
+				run "echo hello"
+			}
+			`,
+		},
+		nil,
+		func(ctx context.Context) ast.Node {
+			mod := ast.Modules(ctx).Get("build.hlb")
+			return ast.Search(mod, `run "echo hello"`)
+		},
+	}, {
+		"filename:line match",
+		"build.hlb:3",
+		map[string]string{
+			"build.hlb": `
+			fs default()
+			fs bar() {
+				image "alpine"
+				run "echo hello"
+			}
+			`,
+		},
+		nil,
+		func(ctx context.Context) ast.Node {
+			mod := ast.Modules(ctx).Get("build.hlb")
+			return ast.Search(mod, `image "alpine"`)
+		},
+	}, {
+		"filename:line another file match",
+		"other.hlb:2",
+		map[string]string{
+			"build.hlb": `
+			fs default()
+			`,
+			"other.hlb": `
+			fs build() {
+				image "alpine"
+			}
+			`,
+		},
+		nil,
+		func(ctx context.Context) ast.Node {
+			mod := ast.Modules(ctx).Get("other.hlb")
+			return ast.Search(mod, `image "alpine"`)
+		},
+	}, {
+		"filename:function match",
+		"other.hlb:build",
+		map[string]string{
+			"build.hlb": `
+			fs default()
+			`,
+			"other.hlb": `
+			fs build() {
+				image "alpine"
+			}
+			`,
+		},
+		nil,
+		func(ctx context.Context) ast.Node {
+			mod := ast.Modules(ctx).Get("other.hlb")
+			return ast.Search(mod, `fs build()`)
+		},
+	}, {
+		"filename:function:line match",
+		"other.hlb:build:1",
+		map[string]string{
+			"build.hlb": `
+			fs default()
+			`,
+			"other.hlb": `
+			fs build() {
+				image "alpine"
+			}
+			`,
+		},
+		nil,
+		func(ctx context.Context) ast.Node {
+			mod := ast.Modules(ctx).Get("other.hlb")
+			return ast.Search(mod, `image "alpine"`)
+		},
+	}} {
+		tc := tc
+		t.Run(tc.name, func(t *testing.T) {
+			ctx := filebuffer.WithBuffers(context.Background(), builtin.Buffers())
+			ctx = ast.WithModules(ctx, ast.NewModules())
+			for filename, content := range tc.files {
+				r := &parser.NamedReader{
+					Reader: strings.NewReader(cleanup(content)),
+					Value:  filename,
+				}
+				mod, err := parser.Parse(ctx, r)
+				require.NoError(t, err)
+
+				err = checker.SemanticPass(mod)
+				require.NoError(t, err)
+
+				err = checker.Check(mod)
+				require.NoError(t, err)
+			}
+
+			var (
+				mod   *ast.Module
+				start ast.Node
+			)
+			if tc.start != nil {
+				start = tc.start(ctx)
+				require.NotNil(t, start)
+				mod = ast.Modules(ctx).Get(start.Position().Filename)
+			} else {
+				mod = ast.Modules(ctx).Get("build.hlb")
+			}
+			require.NotNil(t, mod)
+
+			actual, err := ParseLinespec(ctx, mod.Scope, start, tc.linespec)
+
+			end := tc.end(ctx)
+			if end == nil {
+				require.Error(t, err)
+			} else {
+				expected, ok := end.(ast.StopNode)
+				require.True(t, ok)
+
+				require.NoError(t, err)
+				require.Equal(t, expected, actual)
+			}
+		})
+	}
+}
diff --git a/codegen/debugger_test.go b/codegen/debugger_test.go
new file mode 100644
index 0000000..c0c1ca9
--- /dev/null
+++ b/codegen/debugger_test.go
@@ -0,0 +1,557 @@
+package codegen
+
+import (
+	"bytes"
+	"context"
+	"fmt"
+	"strings"
+	"testing"
+	"time"
+
+	"github.com/alecthomas/participle/lexer"
+	"github.com/lithammer/dedent"
+	"github.com/openllb/hlb/builtin"
+	"github.com/openllb/hlb/checker"
+	"github.com/openllb/hlb/diagnostic"
+	"github.com/openllb/hlb/parser"
+	"github.com/openllb/hlb/parser/ast"
+	"github.com/openllb/hlb/pkg/filebuffer"
+	"github.com/stretchr/testify/require"
+)
+
+func TestDebuggerEarlyExit(t *testing.T) {
+	t.Parallel()
+
+	input := `
+	fs default() {
+		image "alpine"
+	}
+	`
+
+	testDebugger(t, input, func(t *testing.T, d Debugger, mod *ast.Module) {
+		// Early exit.
+		d.Terminate()
+		s, err := d.GetState()
+		require.Nil(t, s)
+		require.ErrorIs(t, err, ErrDebugExit)
+	})
+}
+
+type movement func(d Debugger) (*State, error)
+
+func restart(d Debugger) (*State, error) { return d.Restart() }
+
+func cont(direction Direction) movement {
+	return func(d Debugger) (*State, error) {
+		return d.Continue(direction)
+	}
+}
+
+func next(direction Direction) movement {
+	return func(d Debugger) (*State, error) {
+		return d.Next(direction)
+	}
+}
+
+func step(direction Direction) movement {
+	return func(d Debugger) (*State, error) {
+		return d.Step(direction)
+	}
+}
+
+func stepout(direction Direction) movement {
+	return func(d Debugger) (*State, error) {
+		return d.StepOut(direction)
+	}
+}
+
+func TestDebuggerMovement(t *testing.T) {
+	t.Parallel()
+
+	input := `
+	fs default() {
+		build
+	}
+
+	fs src() {
+		local "."
+		breakpoint
+	}
+
+	fs _build() {
+		image "alpine"
+		run "echo foo" with option {
+			mount src "/in"
+			mount scratch "/in/dist"
+		}
+
+		run "echo bar" with breakpoint
+		run "echo build > /out/build" with option {
+			mount scratch "/out" as build
+		}
+	}
+	`
+
+	testDebugger(t, input, func(t *testing.T, d Debugger, mod *ast.Module) {
+		line1 := ast.Search(mod, `fs default()`)
+		line2 := ast.Search(mod, `build`)
+		line5 := ast.Search(mod, `fs src()`)
+		line6 := ast.Search(mod, `local "."`)
+		line7 := ast.Search(mod, `breakpoint`)
+		line10 := ast.Search(mod, `fs _build()`)
+		line11 := ast.Search(mod, `image "alpine"`)
+		line12 := ast.Search(mod, `run "echo foo" with`)
+		line13src := ast.Search(mod, `src`, ast.WithSkip(1))
+		line13 := ast.Search(mod, `mount src "/in"`)
+		line14 := ast.Search(mod, `mount scratch "/in/dist"`)
+		line14scratch := ast.Search(mod, `scratch`)
+		line16bp := ast.Search(mod, `breakpoint`, ast.WithSkip(1))
+		line16 := ast.Search(mod, `run "echo bar" with`)
+		line17 := ast.Search(mod, `run "echo build > /out/build" with`)
+		line18scratch := ast.Search(mod, `scratch`, ast.WithSkip(1))
+		line18 := ast.Search(mod, `mount scratch "/out" as build`)
+
+		type testCase struct {
+			movement movement
+			expected ast.Node
+		}
+
+		for i, tc := range []testCase{{
+			step(ForwardDirection), line1,
+		}, { // movement_1
+			step(ForwardDirection), line2,
+		}, { // movement_2
+			step(ForwardDirection), line10,
+		}, { // movement_3
+			step(ForwardDirection), line11,
+		}, { // movement_4
+			step(ForwardDirection), line13src,
+		}, { // movement_5
+			step(ForwardDirection), line5,
+		}, { // movement_6
+			step(ForwardDirection), line6,
+		}, { // movement_7
+			step(ForwardDirection), line7,
+		}, { // movement_8
+			step(ForwardDirection), line13,
+		}, { // movement_9
+			step(ForwardDirection), line14scratch,
+		}, { // movement_10
+			step(ForwardDirection), line14,
+		}, { // movement_11
+			step(ForwardDirection), line12,
+		}, { // movement_12
+			step(ForwardDirection), line16bp,
+		}, { // movement_13
+			step(ForwardDirection), line16,
+		}, { // movement_14
+			step(ForwardDirection), line18scratch,
+		}, { // movement_15
+			step(ForwardDirection), line18,
+		}, { // movement_16
+			step(ForwardDirection), line17,
+		}, { // movement_17
+			restart, nil,
+		}, { // movement_18
+			cont(ForwardDirection), line7,
+		}, { // movement_19
+			cont(ForwardDirection), line16,
+		}, { // movement_20
+			cont(BackwardDirection), line7,
+		}, { // movement_21
+			cont(BackwardDirection), nil,
+		}, { // movement_22
+			cont(ForwardDirection), line7,
+		}, { // movement_23
+			step(BackwardDirection), line6,
+		}, { // movement_24
+			stepout(ForwardDirection), line13,
+		}, { // movement_25
+			next(ForwardDirection), line14,
+		}, { // movement_26
+			next(ForwardDirection), line12,
+		}, { // movement_27
+			next(ForwardDirection), line16,
+		}, { // movement_28
+			next(ForwardDirection), line17,
+		}, { // movement_29
+			next(BackwardDirection), line16,
+		}, { // movement_30
+			next(BackwardDirection), line12,
+		}, { // movement_31
+			step(BackwardDirection), line14,
+		}, { // movement_32
+			next(BackwardDirection), line13,
+		}, { // movement_33
+			next(BackwardDirection), line11,
+		}, { // movement_34
+			cont(ForwardDirection), line7,
+		}, { // movement_35
+			stepout(BackwardDirection), line13src,
+		}, { // movement_36
+			restart, nil,
+		}} {
+			tc := tc
+			name := fmt.Sprintf("movement_%d", i)
+			t.Run(name, func(t *testing.T) {
+				actual, err := tc.movement(d)
+				require.NoError(t, err)
+				require.NotNil(t, actual.Node)
+
+				if tc.expected != nil {
+					logState(t, actual, name)
+					requireSameNode(t, tc.expected, actual.Node)
+				} else {
+					t.Logf("At program start")
+					require.IsType(t, &ast.Module{}, actual.Node)
+				}
+			})
+		}
+	})
+}
+
+func TestDebuggerBacktrace(t *testing.T) {
+	t.Parallel()
+
+	input := `
+	fs default() {
+		foo
+	}
+
+	fs foo() {
+		image "alpine"
+		bar
+	}
+
+	fs bar() {
+		env "key" "value"
+	}
+	`
+
+	testDebugger(t, input, func(t *testing.T, d Debugger, mod *ast.Module) {
+		line1 := ast.Search(mod, `fs default()`)
+		line2 := ast.Search(mod, `foo`)
+		line4 := ast.Search(mod, `fs foo()`)
+		line5 := ast.Search(mod, `image "alpine"`)
+		line6 := ast.Search(mod, `bar`)
+		line8 := ast.Search(mod, `fs bar()`)
+		line9 := ast.Search(mod, `env "key" "value"`)
+
+		type testCase struct {
+			movement movement
+			expected []ast.Node
+		}
+
+		for i, tc := range []testCase{{
+			func(d Debugger) (*State, error) {
+				return d.GetState()
+			},
+			[]ast.Node{},
+		}, {
+			step(ForwardDirection),
+			[]ast.Node{line1},
+		}, {
+			step(ForwardDirection),
+			[]ast.Node{line2},
+		}, {
+			step(ForwardDirection),
+			[]ast.Node{line2, line4},
+		}, {
+			step(ForwardDirection),
+			[]ast.Node{line2, line5},
+		}, {
+			step(ForwardDirection),
+			[]ast.Node{line2, line6},
+		}, {
+			step(ForwardDirection),
+			[]ast.Node{line2, line6, line8},
+		}, {
+			step(ForwardDirection),
+			[]ast.Node{line2, line6, line9},
+		}} {
+			tc := tc
+			name := fmt.Sprintf("backtrace_%d", i)
+			t.Run(name, func(t *testing.T) {
+				s, err := tc.movement(d)
+				require.NoError(t, err)
+				require.NotNil(t, s.Node)
+
+				if _, ok := s.Node.(*ast.Module); ok {
+					t.Logf("At program start")
+				} else {
+					logState(t, s, name)
+				}
+
+				frames, err := d.Backtrace()
+				require.NoError(t, err)
+
+				expected := new(bytes.Buffer)
+				expectedFrames := make([]Frame, len(tc.expected))
+				for i, n := range tc.expected {
+					stop, ok := n.(ast.StopNode)
+					if ok {
+						n = stop.Subject()
+					}
+					expectedFrames[i] = Frame{n}
+				}
+				spans := FramesToSpans(s.Ctx, expectedFrames)
+				diagnostic.DisplayError(s.Ctx, expected, spans, nil, true)
+				t.Logf("Expected Backtrace:\n%s", expected)
+
+				actual := new(bytes.Buffer)
+				spans = FramesToSpans(s.Ctx, frames)
+				diagnostic.DisplayError(s.Ctx, actual, spans, nil, true)
+				t.Logf("Actual Backtrace:\n%s", actual)
+
+				require.Equal(t, expected.String(), actual.String())
+			})
+		}
+	})
+}
+
+func TestDebuggerBreakpoint(t *testing.T) {
+	t.Parallel()
+
+	input := `
+	fs default() {
+		bar
+	}
+
+	fs bar() {
+		image "alpine"
+		run "echo foo" with option {
+			mount scratch "/in"
+		}
+	}
+	`
+
+	testDebugger(t, input, func(t *testing.T, d Debugger, mod *ast.Module) {
+		line2 := ast.Find(mod, 2, 0, nil).(*ast.Stmt).Call
+		line5 := ast.Search(mod, `fs bar()`).(ast.StopNode)
+		line8 := ast.Search(mod, `mount scratch "/in"`).(ast.StopNode)
+
+		s, err := d.GetState()
+		require.NoError(t, err)
+
+		bp, err := d.CreateBreakpoint(&Breakpoint{
+			Node: line5.Subject(),
+		})
+		require.NoError(t, err)
+
+		buf := new(bytes.Buffer)
+		bp.Print(s.Ctx, buf, false)
+		t.Logf("\n%s", buf)
+
+		_, err = d.CreateBreakpoint(&Breakpoint{
+			Node: line5.Subject(),
+		})
+		require.Error(t, err)
+
+		s, err = d.Continue(ForwardDirection)
+		require.NoError(t, err)
+		require.Equal(t, line5, s.Node)
+		logState(t, s, "line5")
+
+		_, err = d.Restart()
+		require.NoError(t, err)
+
+		bp2, err := d.CreateBreakpoint(&Breakpoint{
+			Node: line2.Subject(),
+		})
+		require.NoError(t, err)
+
+		buf = new(bytes.Buffer)
+		bp2.Print(s.Ctx, buf, false)
+		t.Logf("\n%s", buf)
+
+		s, err = d.Continue(ForwardDirection)
+		require.NoError(t, err)
+		require.Equal(t, line2, s.Node)
+		logState(t, s, "line2")
+
+		s, err = d.Continue(ForwardDirection)
+		require.NoError(t, err)
+		require.Equal(t, line5, s.Node)
+		logState(t, s, "line5")
+
+		_, err = d.Restart()
+		require.NoError(t, err)
+
+		err = d.ClearBreakpoint(bp2)
+		require.NoError(t, err)
+		t.Logf("Cleared breakpoint 2")
+
+		s, err = d.Continue(ForwardDirection)
+		require.NoError(t, err)
+		require.Equal(t, line5, s.Node)
+		logState(t, s, "line5")
+
+		bp3, err := d.CreateBreakpoint(&Breakpoint{
+			Node: line8.Subject(),
+		})
+		require.NoError(t, err)
+
+		buf = new(bytes.Buffer)
+		bp3.Print(s.Ctx, buf, false)
+		t.Logf("\n%s", buf)
+
+		s, err = d.Continue(ForwardDirection)
+		require.NoError(t, err)
+		require.Equal(t, line8, s.Node)
+		logState(t, s, "line8")
+	})
+}
+
+func TestDebuggerSourceDefinedBreakpoint(t *testing.T) {
+	t.Parallel()
+
+	input := `
+	fs default() {
+		breakpoint
+		image "alpine"
+		run "echo hello" with breakpoint
+		run "echo world" with option {
+			breakpoint
+			mount fs {
+				breakpoint
+			} "/in"
+		}
+	}
+	`
+
+	testDebugger(t, input, func(t *testing.T, d Debugger, mod *ast.Module) {
+		bps := d.Breakpoints()
+		require.Len(t, bps, 4)
+		// breakpoint
+		require.Equal(t, "2:2", formatPos(bps[0].Position()))
+		// run "echo hello"
+		require.Equal(t, "4:2", formatPos(bps[1].Position()))
+		// run "echo world"
+		require.Equal(t, "5:2", formatPos(bps[2].Position()))
+		// mount fs { breakpoint }
+		require.Equal(t, "8:4", formatPos(bps[3].Position()))
+
+		err := d.ClearBreakpoint(bps[0])
+		require.Error(t, err)
+		err = d.ClearBreakpoint(bps[1])
+		require.Error(t, err)
+		err = d.ClearBreakpoint(bps[2])
+		require.Error(t, err)
+		err = d.ClearBreakpoint(bps[3])
+		require.Error(t, err)
+
+		// Continue to 1st breakpoint.
+		s, err := d.Continue(ForwardDirection)
+		require.NoError(t, err)
+		requireSameNode(t, ast.Search(mod, "breakpoint"), s.Node)
+		logState(t, s, "1st")
+
+		// Continue to 2nd breakpoint.
+		s, err = d.Continue(ForwardDirection)
+		require.NoError(t, err)
+		requireSameNode(t, ast.Search(mod, `run "echo hello" with`), s.Node)
+		logState(t, s, "2nd")
+
+		// Continue to 3rd breakpoint. Note that with option block is evaluated
+		// before the statement, so the breakpoint inside `mount fs { ... }` is hit
+		// first.
+		s, err = d.Continue(ForwardDirection)
+		require.NoError(t, err)
+		requireSameNode(t, ast.Search(mod, "breakpoint", ast.WithSkip(3)), s.Node)
+		logState(t, s, "3rd")
+
+		// Continue to 4th breakpoint.
+		s, err = d.Continue(ForwardDirection)
+		require.NoError(t, err)
+		requireSameNode(t, ast.Search(mod, `run "echo world" with`), s.Node)
+		logState(t, s, "4th")
+
+		// Final continue should exit program.
+		s, err = d.Continue(ForwardDirection)
+		require.Nil(t, s)
+		require.ErrorIs(t, err, ErrDebugExit)
+	})
+}
+
+func logState(t *testing.T, s *State, msg string) {
+	stop, ok := s.Node.(ast.StopNode)
+	require.True(t, ok)
+
+	// Print debugger location to test logger.
+	buf := new(bytes.Buffer)
+	err := stop.Subject().WithError(nil, stop.Subject().Spanf(diagnostic.Primary, msg))
+	for _, span := range diagnostic.Spans(err) {
+		fmt.Fprintln(buf, span.Pretty(s.Ctx, diagnostic.WithNumContext(1)))
+	}
+	t.Logf("\n%s", buf)
+}
+
+func requireSameNode(t *testing.T, expected, actual ast.Node) {
+	require.Equal(t, expected.Position().Line, actual.Position().Line)
+	require.Equal(t, expected.Position().Column, actual.Position().Column)
+	require.Equal(t, strings.TrimSpace(expected.String()), strings.TrimSpace(actual.String()))
+}
+
+func formatPos(pos lexer.Position) string {
+	return fmt.Sprintf("%d:%d", pos.Line, pos.Column)
+}
+
+func cleanup(value string) string {
+	return strings.TrimSpace(dedent.Dedent(value)) + "\n"
+}
+
+type procedure func(*testing.T, Debugger, *ast.Module)
+
+func testDebugger(t *testing.T, input string, p procedure) {
+	d := NewDebugger(nil)
+	ctx := WithDebugger(context.Background(), d)
+	ctx = filebuffer.WithBuffers(ctx, builtin.Buffers())
+	ctx = ast.WithModules(ctx, ast.NewModules())
+
+	done := make(chan struct{})
+	go func() {
+		defer close(done)
+		defer func() {
+			err := d.Close()
+			require.NoError(t, err)
+		}()
+		generate(t, ctx, input)
+	}()
+
+	s, err := d.GetState()
+	require.NoError(t, err)
+
+	mod, ok := s.Node.(*ast.Module)
+	require.True(t, ok)
+
+	p(t, d, mod)
+
+	d.Terminate()
+
+	select {
+	case <-time.After(3 * time.Second):
+		t.Fatal("codegen should exit cleanly")
+	case <-done:
+	}
+}
+
+func generate(t *testing.T, ctx context.Context, input string) {
+	r := &parser.NamedReader{
+		Reader: strings.NewReader(cleanup(input)),
+		Value:  "build.hlb",
+	}
+	mod, err := parser.Parse(ctx, r)
+	require.NoError(t, err)
+
+	err = checker.SemanticPass(mod)
+	require.NoError(t, err)
+
+	err = checker.Check(mod)
+	require.NoError(t, err)
+
+	cg := New(nil, nil)
+	_, err = cg.Generate(ctx, mod, []Target{{"default"}})
+	if err != nil {
+		require.ErrorIs(t, err, ErrDebugExit)
+	}
+}
diff --git a/parser/ast/unparse_test.go b/parser/ast/unparse_test.go
index 0d2b794..9fdb31f 100644
--- a/parser/ast/unparse_test.go
+++ b/parser/ast/unparse_test.go
@@ -1,7 +1,6 @@
 package ast
 
 import (
-	"fmt"
 	"strings"
 	"testing"
 
@@ -16,7 +15,7 @@ type testCase struct {
 }
 
 func cleanup(value string) string {
-	return fmt.Sprintf("%s\n", strings.TrimSpace(dedent.Dedent(value)))
+	return strings.TrimSpace(dedent.Dedent(value)) + "\n"
 }
 
 func TestUnparse(t *testing.T) {
diff --git a/parser/ast/walk_test.go b/parser/ast/walk_test.go
index 6e1a9ea..f65be6a 100644
--- a/parser/ast/walk_test.go
+++ b/parser/ast/walk_test.go
@@ -135,11 +135,26 @@ func TestFind(t *testing.T) {
 		}
 		`,
 		1, 0,
-		func(n Node) bool {
-			_, ok := n.(*FuncSignature)
-			return ok
-		},
+		StopNodeFilter,
 		"1:1", "1:13", // fs default()
+	}, {
+		"",
+		`
+		fs default() {
+			breakpoint
+			image "alpine"
+			run "echo hello" with breakpoint
+			run "echo world" with option {
+				breakpoint
+				mount fs {
+					breakpoint
+				} "/in"
+			}
+		}
+		`,
+		5, 0,
+		StopNodeFilter,
+		"5:2", "11:1", // run "echo world" with option { ... }
 	}} {
 		tc := tc
 		t.Run(tc.name, func(t *testing.T) {
diff --git a/solver/progress_test.go b/solver/progress_test.go
index 4a40a1c..514177f 100644
--- a/solver/progress_test.go
+++ b/solver/progress_test.go
@@ -54,9 +54,9 @@ func TestProgress(t *testing.T) {
 				var opts []ProgressOption
 				switch mode {
 				case "tty":
-					opts = append(opts, WithLogOutput(pts, LogOutputTTY))
+					opts = append(opts, WithLogOutputTTY(pts))
 				case "plain":
-					opts = append(opts, WithLogOutput(pts, LogOutputPlain))
+					opts = append(opts, WithLogOutputPlain(pts))
 				}
 
 				p, err := NewProgress(ctx, opts...)