**Note** Programmers which come to rely on the internal API for a use-case which they find realistic and useful are encouraged to open a request for improvement on the bug tracker.

## 22.15.1 Internal variables and CAML_INTERALS

Since OCaml 4.04, it is possible to get access to every part of the internal runtime API by defining the `CAML_INTERALS` macro before loading caml header files. If this macro is not defined, parts of the internal runtime API are hidden.

If you are using internal C variables, do not redefine them by hand. You should import those variables by including the corresponding header files. The representation of those variables has already changed once in OCaml 4.10, and is still under evolution. If your code relies on such internal and brittle properties, it will be broken at some point in time.

For instance, rather than redefining `caml_young_limit`:

```c
extern int caml_young_limit;
```

which breaks in OCaml ≥ 4.10, you should include the `minor_gc` header:

```c
#include <caml/minor_gc.h>
```

## 22.15.2 OCaml version macros

Finally, if including the right headers is not enough, or if you need to support version older than OCaml 4.04, the header file `caml/version.h` should help you to define your own compatibility layer. This file provides few macros defining the current OCaml version. In particular, the `OCAML_VERSION` macro describes the current version, its format is `MmmPP`. For example, if you need some specific handling for versions older than 4.10.0, you could write

```c
#include <caml/version.h>
#if OCAML_VERSION >= 41000
...
#else
...
#endif
```
## 23.12 Treatment of effects

The Lambda optimisers classify expressions in order to determine whether an expression:

* does not need to be evaluated at all; and/or

* may be duplicated.

This is done by forming judgements on the *effects* and the *coefficients* that might be performed were the expression to be executed. Effects talk about how the expression might affect the world; coefficients talk about how the world might affect the expression.

Effects are classified as follows:

**No effects:**

The expression does not change the observable state of the world. For example, it must not write to any mutable storage, call arbitrary external functions or change control flow (e.g. by raising an exception). Note that allocation is *not* classed as having “no effects” (see below).

* It is assumed in the compiler that expressions with no effects, whose results are not used, may be eliminated. (This typically happens where the expression in question is the defining expression of a *let*; in such cases the *let*-expression will be eliminated.) It is further assumed that such expressions with no effects may be duplicated (and thus possibly executed more than once).

* Exceptions arising from allocation points, for example “out of memory” or exceptions propagated from finalizers or signal handlers, are treated as “effects out of the ether” and thus ignored for our determination here of effectfulness. The same goes for floating point operations that may cause hardware traps on some platforms.

**Only generative effects:**

The expression does not change the observable state of the world save for possibly affecting the state of the garbage collector by performing an allocation. Expressions that only have generative effects and whose results are unused may be eliminated by the compiler. However, unlike expressions with “no effects”, such expressions will never be eligible for duplication.

**Arbitrary effects:**

All other expressions.

There is a single classification for coefficients:

**No coefficients:**

The expression does not observe the effects (in the sense described above) of other expressions. For example, it must not read from any mutable storage or call arbitrary external functions.

It is assumed in the compiler that, subject to data dependencies, expressions with neither effects nor coefficients may be reordered with respect to other expressions.
# Chapter 23

## Optimisation with Flambda

### 23.1 Overview

Flambda is the term used to describe a series of optimisation passes provided by the native code compilers as of OCaml 4.03.

Flambda aims to make it easier to write idiomatic OCaml code without incurring performance penalties.

To use the Flambda optimisers it is necessary to pass the `-flambda` option to the OCaml `configure` script. (There is no support for a single compiler that can operate in both Flambda and non-Flambda modes.) Code compiled with Flambda cannot be linked into the same program as code compiled without Flambda. Attempting to do this will result in a compiler error.

Whether or not a particular `ocamlopt` uses Flambda may be determined by invoking it with the `-config` option and looking for any line starting with "flambda:". If such a line is present and says "true", then Flambda is supported, otherwise it is not.

Flambda provides full optimisation across different compilation units, so long as the `.cmx` files for the dependencies of the unit currently being compiled are available. (A compilation unit corresponds to a single `.ml` source file.) However it does not yet act entirely as a whole-program compiler: for example, elimination of dead code across a complete set of compilation units is not supported.

Optimisation with Flambda is not currently supported when generating bytecode.

Flambda should not in general affect the semantics of existing programs. Two exceptions to this rule are: possible elimination of pure code that is being benchmarked (see section 23.14) and changes in behaviour of code using unsafe operations (see section 23.15).

Flambda does not yet optimise array or string bounds checks. Neither does it take hints for optimisation from any assertions written by the user in the code.

Consult the *Glossary* at the end of this chapter for definitions of technical terms used below.

### 23.2 Command-line flags

The Flambda optimisers provide a variety of command-line flags that may be used to control their behaviour. Detailed descriptions of each flag are given in the referenced sections. Those sections also describe any arguments which the particular flags take.

Commonly-used options:
## 23.13 Compilation of statically-allocated modules

Compilation of modules that are able to be statically allocated (for example, the module corresponding to an entire compilation unit, as opposed to a first class module dependent on values computed at runtime) initially follows the strategy used for bytecode. A sequence of `let`-bindings, which may be interspersed with arbitrary effects, surrounds a record creation that becomes the module block. The Lambda-specific transformation follows: these bindings are lifted to toplevel symbols, as described above.

## 23.14 Inhibition of optimisation

Especially when writing benchmarking suites that run non-side-effecting algorithms in loops, it may be found that the optimiser entirely elides the code being benchmarked. This behaviour can be prevented by using the `Sys.opaque_identity` function (which indeed behaves as a normal OCaml function and does not possess any “magic” semantics). The documentation of the `Sys` module should be consulted for further details.

## 23.15 Use of unsafe operations

The behaviour of the Lambda simplification pass means that certain unsafe operations, which may without Lambda or when using previous versions of the compiler be safe, must not be used. This specifically refers to functions found in the `Obj` module.

In particular, it is forbidden to change any value (for example using `Obj.set_field` or `Obj.set_tag`) that is not mutable. (Values returned from C stubs are always treated as mutable.) The compiler will emit warning 59 if it detects such a write—but it cannot warn in all cases. Here is an example of code that will trigger the warning:

```hml
let f x =
    let a = 42, x in
    (Obj.magic a : int ref) := 1;
    fst a
```

The reason this is unsafe is because the simplification pass believes that `fst a` holds the value 42; and indeed it must, unless type soundness has been broken via unsafe operations.

If it must be the case that code has to be written that triggers warning 59, but the code is known to actually be correct (for some definition of correct), then `Sys.opaque_identity` may be used to wrap the value before unsafe operations are performed upon it. Great care must be taken when doing this to ensure that the opacity is added at the correct place. It must be emphasised that this use of `Sys.opaque_identity` is only for **exceptional** cases. It should not be used in normal code or to try to guide the optimiser.

As an example, this code will return the integer 1:

```hml
let f x =
    let a = Sys.opaque_identity (42, x) in
    (Obj.magic a : int ref) := 1;
    fst a
```
-02 Perform more optimisation than usual. Compilation times may be lengthened. (This flag is an abbreviation for a certain set of parameters described in section 23.5.)

-03 Perform even more optimisation than usual, possibly including unrolling of recursive functions. Compilation times may be significantly lengthened.

**-Oclassic**

Make inlining decisions at the point of definition of a function rather than at the call site(s). This mirrors the behaviour of OCaml compilers not using Flambda. Compared to compilation using the new Flambda inlining heuristics (for example at -02) it produces smaller .cmx files, shorter compilation times and code that probably runs rather slower. When using -Oclassic, only the following options described in this section are relevant: -inlining-report and -inline. If any other of the options described in this section are used, the behaviour is undefined and may cause an error in future versions of the compiler.

**-inlining-report**

Emit .inlining files (one per round of optimisation) showing all of the inliner's decisions.

Less commonly-used options:

**-remove-unused-arguments**

Remove unused function arguments even when the argument is not specialised. This may have a small performance penalty. See section 23.10.3.

**-unbox-closures**

Pass free variables via specialised arguments rather than closures (an optimisation for reducing allocation). See section 23.9.3. This may have a small performance penalty.

Advanced options, only needed for detailed tuning:

**-inline**

The behaviour depends on whether -Oclassic is used.

* When not in -Oclassic mode, -inline limits the total size of functions considered for inlining during any speculative inlining search. (See section 23.3.10.) Note that this parameter does not control the assessment as to whether any particular function may be inlined. Raising it to excessive amounts will not necessarily cause more functions to be inlined.

* When in -Oclassic mode, -inline behaves as in previous versions of the compiler: it is the maximum size of function to be considered for inlining. See section 23.3.2.

**-inline-t LEVEL**

The equivalent of -inline but used when speculative inlining starts at toplevel. See section 23.3.10. Not used in -Oclassic mode.

**-inline-branch-factor**

Controls how the inliner assesses whether a code path is likely to be hot or cold. See section 23.3.9.
However the following code will still return 42:

```h
let f x =
    let a = 42, x in
    Sys.opaque_identity (Obj.magic a : int ref) := 1;
    fst a
```

High levels of inlining performed by Flambda may expose bugs in code thought previously to be correct. Take care, for example, not to add type annotations that claim some mutable value is always immediate if it might be possible for an unsafe operation to update it to a boxed value.

## 23.16 Glossary

The following terminology is used in this chapter of the manual.

### Call site

See *direct call site* and *indirect call site* below.

### Closed function

A function whose body has no free variables except its parameters and any to which are bound other functions within the same (possibly mutually-recursive) declaration.

### Closure

The runtime representation of a function. This includes pointers to the code of the function together with the values of any variables that are used in the body of the function but actually defined outside of the function, in the enclosing scope. The values of such variables, collectively known as the *environment*, are required because the function may be invoked from a place where the original bindings of such variables are no longer in scope. A group of possibly mutually-recursive functions defined using *let rec* all share a single closure. (Note to developers: in the Flambda source code a *closure* always corresponds to a single function; a *set of closures* refers to a group of such.)

### Closure variable

A member of the environment held within the closure of a given function.

### Constant

Some entity (typically an expression) the value of which is known by the compiler at compile time. Constantness may be explicit from the source code or inferred by the Flambda optimisers.

### Constant closure

A closure that is statically allocated in an object file. It is almost always the case that the environment portion of such a closure is empty.

### Defining expression

The expression `e` in `let x = e in e'`.

### Direct call site

A place in a program's code where a function is called and it is known at compile time which function it will always be.
-inline-alloc-cost, -inline-branch-cost, -inline-call-cost

Controls how the inliner assesses the runtime performance penalties associated with various operations. See section 23.3.9.

-inline-indirect-cost, -inline-prim-cost

Likewise.

-inline-lifting-benefit

Controls inlining of functors at toplevel. See section 23.3.9.

-inline-max-depth

The maximum depth of any speculative inlining search. See section 23.3.10.

-inline-max-unroll

The maximum depth of any unrolling of recursive functions during any speculative inlining search. See section 23.3.10.

-no-unbox-free-vars-of-closures

Do not unbox closure variables. See section 23.9.1.

-no-unbox-specialised-args

Do not unbox arguments to which functions have been specialised. See section 23.9.2.

-rounds

How many rounds of optimisation to perform. See section 23.2.1.

-unbox-closures-factor

Scaling factor for benefit calculation when using -unbox-closures. See section 23.9.3.

Notes

* The set of command line flags relating to optimisation should typically be specified to be the same across an entire project. Lambda does not currently record the requested flags in the `.cmx` files. As such, inlining of functions from previously-compiled units will subject their code to the optimisation parameters of the unit currently being compiled, rather than those specified when they were previously compiled. It is hoped to rectify this deficiency in the future.

* Lambda-specific flags do not affect linking with the exception of affecting the optimisation of code in the startup file (containing generated functions such as currying helpers). Typically such optimisation will not be significant, so eliding such flags at link time might be reasonable.

* Lambda-specific flags are silently accepted even when the `-flambda` option was not provided to the `configure` script. (There is no means provided to change this behaviour.) This is intended to make it more straightforward to run benchmarks with and without the Lambda optimisers in effect.

* Some of the Lambda flags may be subject to change in future releases.
**Indirect call site**

A place in a program's code where a function is called but is not known to be a *direct call site*.

**Program**

A collection of *symbol bindings* forming the definition of a single compilation unit (i.e. `.cmx` file).

**Specialised argument**

An argument to a function that is known to always hold a particular value at runtime. These are introduced by the inliner when specialising recursive functions; and the `unbox-closures` pass. (See section 23.4.)

**Symbol**

A name referencing a particular place in an object file or executable image. At that particular place will be some constant value. Symbols may be examined using operating system-specific tools (for example `objdump` on Linux).

**Symbol binding**

Analogous to a `let`-expression but working at the level of symbols defined in the object file. The address of a symbol is fixed, but it may be bound to both constant and non-constant expressions.

**TLEVEL**

An expression in the current program which is not enclosed within any function declaration.

**Variable**

A named entity to which some OCaml value is bound by a `let` expression, pattern-matching construction, or similar.
### 23.2.1 Specification of optimisation parameters by round

Lambda operates in *rounds*: one round consists of a certain sequence of transformations that may then be repeated in order to achieve more satisfactory results. The number of rounds can be set manually using the `-rounds` parameter (although this is not necessary when using predefined optimisation levels such as with `-02` and `-03`). For high optimisation the number of rounds might be set at 3 or 4.

Command-line flags that may apply per round, for example those with `-cost` in the name, accept arguments of the form:

$$n \mid round=n[,...]$$

* If the first form is used, with a single integer specified, the value will apply to all rounds.

* If the second form is used, zero-based *round* integers specify values which are to be used only for those rounds.

The flags `-0classic`, `-02` and `-03` are applied before all other flags, meaning that certain parameters may be overridden without having to specify every parameter usually invoked by the given optimisation level.

## 23.3 Inlining

Inlining refers to the copying of the code of a function to a place where the function is called. The code of the function will be surrounded by bindings of its parameters to the corresponding arguments.

The aims of inlining are:

* to reduce the runtime overhead caused by function calls (including setting up for such calls and returning afterwards);

* to reduce instruction cache misses by expressing frequently-taken paths through the program using fewer machine instructions; and

* to reduce the amount of allocation (especially of closures).

These goals are often reached not just by inlining itself but also by other optimisations that the compiler is able to perform as a result of inlining.

When a recursive call to a function (within the definition of that function or another in the same mutually-recursive group) is inlined, the procedure is also known as *unrolling*. This is somewhat akin to loop peeling. For example, given the following code:

```h
let rec fact x =
    if x = 0 then
        1
    else
        x * fact (x - 1)
```

```h
let n = fact 4
```
Chapter 24

Fuzzing with afl-fuzz

24.1 Overview

American fuzzy lop (“afl-fuzz”) is a fuzzer, a tool for testing software by providing randomly-generated
inputs, searching for those inputs which cause the program to crash.

Unlike most fuzzers, afl-fuzz observes the internal behaviour of the program being tested, and adjusts the test cases it generates to trigger unexplored execution paths. As a result, test cases generated by afl-fuzz cover more of the possible behaviours of the tested program than other fuzzers.

This requires that programs to be tested are instrumented to communicate with afl-fuzz. The
native-code compiler "ocamopt" can generate such instrumentation, allowing afl-fuzz to be used
against programs written in OCaml.

For more information on afl-fuzz, see the website at http://lcamtuf.coredump.cx/afl/

**24.2 Generating instrumentation**

The instrumentation that afl-fuzz requires is not generated by default, and must be explicitly
enabled, by passing the `-afl-instrument` option to `ocamopt`.

To fuzz a large system without modifying build tools, OCaml's `configure` script also accepts the
`afl-instrument` option. If OCaml is configured with `afl-instrument`, then all programs compiled
by `ocamopt` will be instrumented.

24.2.1 Advanced options

In rare cases, it is useful to control the amount of instrumentation generated. By passing the
`-afl-inst-ratio N` argument to `ocamopt` with N less than 100, instrumentation can be generated
for only N% of branches. (See the afl-fuzz documentation on the parameter `AFL_INSTALL_ratio` for
the precise effect of this).

24.3 Example

As an example, we fuzz-test the following program, `readline.ml`:
unrolling once at the call site `fact 4` produces (with the body of `fact` unchanged):

```python
let n =
    if 4 == 0 then
        1
    else
        4 * fact (4 - 1)
```

This simplifies to:

```python
let n = 4 * fact 3
```

Flambda provides significantly enhanced inlining capabilities relative to previous versions of the compiler.

### 23.3.1 Aside: when inlining is performed

Inlining is performed together with all of the other Flambda optimisation passes, that is to say, after closure conversion. This has three particular advantages over a potentially more straightforward implementation prior to closure conversion:

* It permits higher-order inlining, for example when a non-inlinable function always returns the same function yet with different environments of definition. Not all such cases are supported yet, but it is intended that such support will be improved in future.

* It is easier to integrate with cross-module optimisation, since imported information about other modules is already in the correct intermediate language.

* It becomes more straightforward to optimise closure allocations since the layout of closures does not have to be estimated in any way: it is known. Similarly, it becomes more straightforward to control which variables end up in which closures, helping to avoid closure bloat.

### 23.3.2 Classic inlining heuristic

In `-Oclassic` mode the behaviour of the Flambda inliner mimics previous versions of the compiler. (Code may still be subject to further optimisations not performed by previous versions of the compiler: functors may be inlined, constants are lifted and unused code is eliminated all as described elsewhere in this chapter. See sections 23.3.5, 23.8.1 and 23.10. At the definition site of a function, the body of the function is measured. It will then be marked as eligible for inlining (and hence inlined at every direct call site) if:

* the measured size (in unspecified units) is smaller than that of a function call plus the argument of the `-inline` command-line flag; and

* the function is not recursive.

Non-Lambda versions of the compiler cannot inline functions that contain a definition of another function. However `-Oclassic` does permit this. Further, non-Lambda versions also cannot inline functions that are only themselves exposed as a result of a previous pass of inlining, but again this is permitted by `-Oclassic`. For example:
let _ =
let s = read_line () in
match Array.to_list (Array.init (String.length s) (String.get s)) with
['s'; 'e'; 'c'; 'r'; 'e'; 't'; ' '; 'c'; 'o'; 'd'; 'e'] -> failwith "uh oh"
| _ -> ()

There is a single input (the string "secret code") which causes this program to crash, but finding
it by blind random search is infeasible.

Instead, we compile with afl-fuzz instrumentation enabled:

ocamlopt -afl-instrument readline.ml -o readline

Next, we run the program under afl-fuzz:

mkdir input
echo asdf > input/testcase
mkdir output
afl-fuzz -m none -i input -o output ./readline

By inspecting instrumentation output, the fuzzer finds the crashing input quickly.

Note: To fuzz-test an OCaml program with afl-fuzz, passing the option `-m none` is required to disable afl-fuzz's default 50MB virtual memory limit.
```python
module M : sig
    val i : int
end = struct
    let f x =
        let g y = x + y in
        g
    let h = f 3
    let i = h 4 (* h is correctly discovered to be g and inlined *)
end
```

All of this contrasts with the normal Flambda mode, that is to say without `-Oclassic`, where:

* the inlining decision is made at the **call site**; and

* recursive functions can be handled, by **specialisation** (see below).

The Flambda mode is described in the next section.

### 23.3.3 Overview of “Flambda” inlining heuristics

The Flambda inlining heuristics, used whenever the compiler is configured for Flambda and `-Oclassic` was not specified, make inlining decisions at call sites. This helps in situations where the context is important. For example:

```python
let f b x =
    if b then
        x
    else
        ... big expression ...

let g x = f true x
```

In this case, we would like to inline `f` into `g`, because a conditional jump can be eliminated and the code size should reduce. If the inlining decision has been made after the declaration of `f` without seeing the use, its size would have probably made it ineligible for inlining; but at the call site, its final size can be known. Further, this function should probably not be inlined systematically: if `b` is unknown, or indeed `false`, there is little benefit to trade off against a large increase in code size. In the existing non-Flambda inliner this isn't a great problem because chains of inlining were cut off fairly quickly. However it has led to excessive use of overly-large inlining parameters such as `-inline 10000`.

In more detail, at each call site the following procedure is followed:

* Determine whether it is clear that inlining would be beneficial without, for the moment, doing any inlining within the function itself. (The exact assessment of *benefit* is described below.) If so, the function is inlined.

* If inlining the function is not clearly beneficial, then inlining will be performed *speculatively* inside the function itself. The search for speculative inlining possibilities is controlled by two parameters: the *inlining threshold* and the *inlining depth*. (These are described in more detail below.)
# Chapter 25

## Runtime tracing with runtime events

This chapter describes the runtime events tracing system which enables continuous extraction of performance information from the OCaml runtime with very low overhead. The system and interfaces are low-level and tightly coupled to the runtime implementation, it is intended for end-users to rely on tooling to consume and visualise data of interest.

Data emitted includes:

* Event times of garbage collector and runtime phases

* Minor and major heap sizings and utilization

* Allocation and promotion rates between heaps

### 25.1 Overview

There are three main classes of events emitted by the runtime events system:

**Spans** Events spanning over a duration in time. For example, the runtime events tracing system emits a span event that starts when a minor collection begins in the OCaml garbage collector and ends when the collection is completed. Spans can contain other spans, e.g other span events may be emitted that begin after a minor collection has begun and end before it does.

**Lifecycle events** Events that occur at a moment in time. For example, when a domain terminates, a corresponding lifecycle event is emitted.

**Counters** Events that include a measurement of some quantity of interest. For example, the number of words promoted from the minor to the major heap during the last minor garbage collection is emitted as a counter event.

The runtime events tracing system is designed to be used in different contexts:

**Self monitoring** OCaml programs and libraries can install their own callbacks to listen for runtime events and react to them programmatically, for example, to export events to disk or over the network.
- If such speculation shows that performing some inlining inside the function would be beneficial, then such inlining is performed and the resulting function inlined at the original call site.

- Otherwise, nothing happens.

Inlining within recursive functions of calls to other functions in the same mutually-recursive group is kept in check by an *unrolling depth*, described below. This ensures that functions are not unrolled to excess. (Unrolling is only enabled if `-O3` optimisation level is selected and/or the `-inline-max-unroll` flag is passed with an argument greater than zero.)

### 23.3.4 Handling of specific language constructs

### 23.3.5 Functors

There is nothing particular about functors that inhibits inlining compared to normal functions. To the inliner, these both look the same, except that functors are marked as such.

Applications of functors at toplevel are biased in favour of inlining. (This bias may be adjusted: see the documentation for `-inline-lifting-benefit` below.)

Applications of functors not at toplevel, for example in a local module inside some other expression, are treated by the inliner identically to normal function calls.

### 23.3.6 First-class modules

The inliner will be able to consider inlining a call to a function in a first class module if it knows which particular function is going to be called. The presence of the first-class module record that wraps the set of functions in the module does not per se inhibit inlining.

### 23.3.7 Objects

Method calls to objects are not at present inlined by `Lambda`.

### 23.3.8 Inlining reports

If the `-inlining-report` option is provided to the compiler then a file will be emitted corresponding to each round of optimisation. For the `OCaml` source file `basename.ml` the files are named `basename_round.inlining.org`, with `round` a zero-based integer. Inside the files, which are formatted as "org mode", will be found English prose describing the decisions that the inliner took.

### 23.3.9 Assessment of inlining benefit

Inlining typically results in an increase in code size, which if left unchecked, may not only lead to grossly large executables and excessive compilation times but also a decrease in performance due to worse locality. As such, the `Lambda` inliner trades off the change in code size against the expected runtime performance benefit, with the benefit being computed based on the number of operations that the compiler observes may be removed as a result of inlining.

For example given the following code:
**External monitoring** An external process can consume the runtime events of an OCaml program whose runtime tracing system has been enabled by setting the corresponding environment variable.

The runtime events tracing system logs events to a *ring buffer*. Consequently, old events are being overwritten by new events. Consumers can either continuously consume events or choose to only do so in response to some circumstance, e.g if a particular query or operation takes longer than expected to complete.

## 25.2 Architecture

The runtime tracing system conceptually consists of two parts: 1) the probes which emit events and 2) the events transport that ingests and transports these events.

### 25.2.1 Probes

Probes collect events from the runtime system. These are further split in to two sets: 1) probes that are always available and 2) probes that are only available in the instrumented runtime. Probes in the instrumented runtime are primarily of interest to developers of the OCaml runtime and garbage collector and, at present, only consist of major heap allocation size counter events.

The full set of events emitted by probes and their documentation can be found in section ??.

### 25.2.2 Events transport

The events transport part of the system ingests events emitted by the probes and makes them available to consumers.

### 25.2.3 Ring buffers

Events are transported using a data structure known as a *ring buffer*. This data structure consists of two pointers into a linear backing array, the tail pointer points to a location where new events can be written and the head pointer points to the oldest event in the buffer that can be read. When insufficient space is available in the backing array to write new events, the head pointer is advanced and the oldest events are overwritten by new ones.

The ring buffer implementation used in runtime events can be written by at most one producer at a time but can be read simultaneously by multiple consumers without coordination from the producer. There is a unique ring buffer for every running domain and, on domain termination, ring buffers may be re-used for newly spawned domains. The ring buffers themselves are stored in a memory-mapped file with the processes identifier as the name and the extension `.events`, this enables them to be read from outside the main OCaml process. See `Runtime_events[??]` for more information.

### 25.2.4 Consumption APIs

The runtime event tracing system provides both OCaml and C APIs which are cursor-based and polling-driven. The high-level process for consuming events is as follows:
```hml
let f b x =
  if b then
    x
  else
    ... big expression ...
```

```hml
let g x = f true x
```

it would be observed that inlining of `f` would remove:

* one direct call;

* one conditional branch.

Formally, an estimate of runtime performance benefit is computed by first summing the cost of the operations that are known to be removed as a result of the inlining and subsequent simplification of the inlined body. The individual costs for the various kinds of operations may be adjusted using the various `-inline-...-cost` flags as follows. Costs are specified as integers. All of these flags accept a single argument describing such integers using the conventions detailed in section 23.2.1.

**-inline-alloc-cost**

The cost of an allocation.

**-inline-branch-cost**

The cost of a branch.

**-inline-call-cost**

The cost of a direct function call.

**-inline-indirect-cost**

The cost of an indirect function call.

**-inline-prim-cost**

The cost of a *primitive*. Primitives encompass operations including arithmetic and memory access.

(Default values are described in section 23.5 below.)

The initial benefit value is then scaled by a factor that attempts to compensate for the fact that the current point in the code, if under some number of conditional branches, may be cold. (Lambda does not currently compute hot and cold paths.) The factor—the estimated probability that the inliner really is on a *hot path*—is calculated as $\frac{1}{(1+f)^d}$, where f is set by `-inline-branch-factor` and d is the nesting depth of branches at the current point. As the inliner descends into more deeply-nested branches, the benefit of inlining thus lessens.

The resulting benefit value is known as the *estimated benefit*.

The change in code size is also estimated: morally speaking it should be the change in machine code size, but since that is not available to the inliner, an approximation is used.

If the estimated benefit exceeds the increase in code size then the inlined version of the function will be kept. Otherwise the function will not be inlined.

Applications of functors at toplevel will be given an additional benefit (which may be controlled by the `-inline-lifting-benefit` flag) to bias inlining in such situations towards keeping the inlined version.
1. A cursor is created via `Runtime_events.create_cursor` for either the current process or an external process (specified by a path and PID).

2. `Runtime_events SabITS` is called to register a callback function to receive the events.

3. The cursor is polled via `Runtime_events.read_poll` using the callbacks created in the previous step. For each matching event in the ring buffers, the provided callback functions are called.

## 25.3 Usage

### 25.3.1 With OCaml APIs

We start with a simple example that prints the name, begin and end times of events emitted by the
runtime event tracing system:

```ocaml
let runtime_begin _ ts phase =
    Printfprintf "Begin\t%s\t%Ld\n"
        (Runtime_events runtime_phase_name phase)
        (Runtime_events Timestamp.to_int64 ts)
```

```ocaml
let runtime_end _ ts phase =
    Printfprintf "End\t%s\t%Ld\n"
        (Runtime_events runtime_phase_name phase)
        (Runtime_events Timestamp.to_int64 ts)
```

```ocaml
let () =
    Runtime_events.start ();

let cursor = Runtime_events.create_cursor None in
let callbacks = Runtime_events SabITs.create ~runtime_begin ~runtime_end ()
in
while true do
    let list_ref = ref [] in (* for later fake GC work *)
    for _ = 1 to 100 do
        (* here we do some fake GC work *)
        list_ref := [];
        for _ = 1 to 10 do
            list_ref := (Sys.opaque_identity(ref 42)) :: !list_ref
        done;
        Gc.full_major ();
    done;
    ignore(Runner_events.read_poll cursor callbacks None);
    Unix.sleep 1
    done
```
### 23.3.10 Control of speculation

As described above, there are three parameters that restrict the search for inlining opportunities during speculation:

* the inlining threshold;

* the inlining depth;

* the unrolling depth.

These parameters are ultimately bounded by the arguments provided to the corresponding command-line flags (or their default values):

* `-inline` (or, if the call site that triggered speculation is at toplevel, `-inline-t LEVEL`);

* `-inline-max-depth`;

* `-inline-max-unroll`.

**Note in particular** that `-inline` does not have the meaning that it has in the previous compiler or in `-Oclassic` mode. In both of those situations `-inline` was effectively some kind of basic assessment of inlining benefit. However in Lambda inlining mode it corresponds to a constraint on the search; the assessment of benefit is independent, as described above.

When speculation starts the inlining threshold starts at the value set by `-inline` (or `-inline-tLEVEL` if appropriate, see above). Upon making a speculative inlining decision the threshold is reduced by the code size of the function being inlined. If the threshold becomes exhausted, at or below zero, no further speculation will be performed.

The inlining depth starts at zero and is increased by one every time the inliner descends into another function. It is then decreased by one every time the inliner leaves such function. If the depth exceeds the value set by `-inline-max-depth` then speculation stops. This parameter is intended as a general backstop for situations where the inlining threshold does not control the search sufficiently.

The unrolling depth applies to calls within the same mutually-recursive group of functions. Each time an inlining of such a call is performed the depth is incremented by one when examining the resulting body. If the depth reaches the limit set by `-inline-max-unroll` then speculation stops.

## 23.4 Specialisation

The inliner may discover a call site to a recursive function where something is known about the arguments: for example, they may be equal to some other variables currently in scope. In this situation it may be beneficial to *specialise* the function to those arguments. This is done by copying the declaration of the function (and any others involved in any same mutually-recursive declaration) and noting the extra information about the arguments. The arguments augmented by this information are known as *specialised arguments*. In order to try to ensure that specialisation is not performed uselessly, arguments are only specialised if it can be shown that they are *invariant*: in other words, during the execution of the recursive function(s) themselves, the arguments never change.

Unless overridden by an attribute (see below), specialisation of a function will not be attempted if:
The next step is to compile and link the program with the `runtime_events` library. This can be done as follows:

```
ocamlopt -I +runtime_events -I +unix unix.cmxa runtime_events.cmxa example.ml -o example
```

When using the `dune` build system, this example can be built as follows:

```
hti
(name example)
(modules example)
(libraries unix runtime_events))
```

Running the compiled binary of the example gives an output similar to:

```
Begin   explicit_gc_full_major 24086187297852
Begin   stwLeader              24086187298594
Begin   minor                 24086187299404
Begin   minor_global_roots      24086187299807
End     minor_global_roots      24086187331461
Begin   minor_remembered_set   24086187331631
Begin   minor_finalizers_oldify 24086187544312
End     minor_finalizers_oldify 24086187544704
Begin   minor_remembered_setPromote 24086187544879
End     minor_remembered_setPromote 24086187606414
End     minor_remembered_set      24086187606584
Begin   minor_finalizers_admin  24086187606854
End     minor_finalizers_admin  24086187607152
Begin   minor_local_roots       24086187607329
Begin   minor_local_rootsPromote 24086187609699
End     minor_local_rootsPromote 24086187610539
End     minor_local_roots        24086187610709
End     minor                 24086187611746
Begin   minor_clear           24086187612238
End     minor_clear           24086187612580
End     stwLeader              24086187613209
...
```

This is an example of self-monitoring, where a program explicitly starts listening to runtime events and monitors itself.

For external monitoring, a program does not need to be aware of the existence of runtime events. Runtime events can be controlled via the environment variable `OCAML_RUNTIME_events_START` which, when set, will cause the runtime tracing system to be started at program initialization.

We could remove `Runtime_events.start()`; from the previous example and, instead, call the program as below to produce the same result:

```
OCAML_RUNTIME_events_START=1 ./example
```
* the compiler is in `-Oclassic` mode;

* the function is not obviously recursive;

* the function is not closed.

The compiler can prove invariance of function arguments across multiple functions within a recursive group (although this has some limitations, as shown by the example below).

It should be noted that the *unboxing of closures* pass (see below) can introduce specialised arguments on non-recursive functions. (No other place in the compiler currently does this.)

**Example: the well-known `List.iter` function** This function might be written like so:

```python
let rec iter f l =
    match l with
    | [] -> ()
    | h :: t ->
        f h;
        iter f t
```

and used like this:

```python
let print_int x =
    print_endline (Int.to_string x)
```

```python
let run xs =
    iter print_int (List.rev xs)
```

The argument `f` to `iter` is invariant so the function may be specialised:

```python
let run xs =
    let rec iter' f l =
        (* The compiler knows: f holds the same value as foo throughout iter'. *)
        match l with
        | [] -> ()
        | h :: t ->
            f h;
            iter' f t
        in
        iter' print_int (List.rev xs)
```

The compiler notes down that for the function `iter'`, the argument `f` is specialised to the constant closure `print_int`. This means that the body of `iter'` may be simplified:

```python
let run xs =
    let rec iter' f l =
        (* The compiler knows: f holds the same value as foo throughout iter'. *)
        match l with
        | [] -> ()
```
### 25.3.2 Environment variables

Environment variables can be used to control different aspects of the runtime event tracing system. The following environment variables are available:

* OCAML_RUNTIME_events_START if set will cause the runtime events system to be started as part of the OCaml runtime initialization.

* OCAML_RUNTIME_events_DIR sets the directory where the `.events` files containing the runtime event tracing system's ring buffers will be located. If not present the program's working directory will be used.

* OCAML_RUNTIME_events_PRESERVE if set will make the OCaml runtime preserve the runtime events ring buffer files past the termination of the OCaml program. This can be useful for monitoring very short running programs. If not set, the `.events` files of the OCaml program will be deleted at program termination.

The size of the runtime events ring buffers can be configured via OCAMLRUNPARAM, see section 15.2 for more information.

### 25.3.3 Building with the instrumented runtime

To receive events that are only available in the instrumented runtime, the OCaml program needs to be compiled and linked against the instrumented runtime. For our example program from earlier, this is achieved as follows:

```
ocamlopt -runtime-variant i -I +runtime_events -I +unix unix.cmxa runtime_events.cmxa example.
```

And for dune:

```
(excutable
(name example)
(modules example)
 flags "-runtime-variant=i")
(libraries unix runtime_events))
```

### 25.3.4 With tooling

Programmatic access to events is intended primarily for writers of observability libraries and tooling that end-users use. The flexible API enables use of the performance data from runtime events for logging and monitoring purposes.

In this section we cover several utilities in the `runtime_events_tools` package which provide simple ways of extracting and summarising data from runtime events. The trace utility in particular produces similar data to the previous 'eventlog' instrumentation system available in OCaml 4.12 to 4.14.

First, install `runtime_events_tools` in an OCaml 5.0+ opam switch:

```
opam install runtime_events_tools
```
```h
| h :: t ->
  print_int h; (* this is now a direct call *)
  iter' f t

in
iter' print_int (List.rev xs)
```

The call to `print_int` can indeed be inlined:

```h
let run xs =
let rec iter' f l =
(* The compiler knows: f holds the same value as foo throughout iter'. *)
match l with
| [] -> ()
| h :: t ->
  print_endline (Int.to_string h);
  iter' f t

in
iter' print_int (List.rev xs)
```

The unused specialised argument `f` may now be removed, leaving:

```h
let run xs =
let rec iter' l =
match l with
| [] -> ()
| h :: t ->
  print_endline (Int.to_string h);
  iter' t

in
iter' (List.rev xs)
```

**Aside on invariant parameters.** The compiler cannot currently detect invariance in cases such as the following.

```h
let rec iter_swap f g l =
match l with
| [] -> ()
| 0 :: t ->
  iter_swap g f l
| h :: t ->
  f h;
  iter_swap f g t
```

### 23.4.1 Assessment of specialisation benefit

The benefit of specialisation is assessed in a similar way as for inlining. Specialised argument information may mean that the body of the function being specialised can be simplified: the removed operations are accumulated into a benefit. This, together with the size of the duplicated (specialised) function declaration, is then assessed against the size of the call to the original function.
This should install the olly tool in your path. You can now generate runtime traces for programs compiled with OCaml 5.0+ using the trace subcommand:

olly trace trace.json 'your_program.exe .. args ..'

Runtime tracing data will be generated in the json Trace Event Format to trace.json. This can then be loaded into the Chrome tracing viewer or into Perfetto to visualize the collected trace.

### 25.3.5 Measuring GC latency

The olly utility also includes a latency subcommand which consumes runtime events data and on program completion emits a parseable histogram summary of pause durations. It can be run as follows:

olly latency 'your_program.exe .. args ..'

This should produce an output similar to the following:

GC latency profile:

#[Mean (ms): 2.46, Stddev (ms): 3.87]

#[Min (ms): 0.01, max (ms): 9.17]

<table><thead><tr><td>Percentile</td><td>Latency (ms)</td></tr></thead><tbody><tr><td>25.0000</td><td>0.01</td></tr><tr><td>50.0000</td><td>0.23</td></tr><tr><td>60.0000</td><td>0.23</td></tr><tr><td>70.0000</td><td>0.45</td></tr><tr><td>75.0000</td><td>0.45</td></tr><tr><td>80.0000</td><td>0.45</td></tr><tr><td>85.0000</td><td>0.45</td></tr><tr><td>90.0000</td><td>9.17</td></tr><tr><td>95.0000</td><td>9.17</td></tr><tr><td>96.0000</td><td>9.17</td></tr><tr><td>97.0000</td><td>9.17</td></tr><tr><td>98.0000</td><td>9.17</td></tr><tr><td>99.0000</td><td>9.17</td></tr><tr><td>99.9000</td><td>9.17</td></tr><tr><td>99.9900</td><td>9.17</td></tr><tr><td>99.9990</td><td>9.17</td></tr><tr><td>99.9999</td><td>9.17</td></tr><tr><td>100.0000</td><td>9.17</td></tr></tbody></table>
## 23.5 Default settings of parameters

The default settings (when not using `-Oclassic`) are for one round of optimisation using the following parameters.

<table><thead><tr><td>Parameter</td><td>Setting</td></tr></thead><tbody><tr><td>-inline</td><td>10</td></tr><tr><td>-inline-branch-factor</td><td>0.1</td></tr><tr><td>-inline-alloc-cost</td><td>7</td></tr><tr><td>-inline-branch-cost</td><td>5</td></tr><tr><td>-inline-call-cost</td><td>5</td></tr><tr><td>-inline-indirect-cost</td><td>4</td></tr><tr><td>-inline-prim-cost</td><td>3</td></tr><tr><td>-inline-lifting-benefit</td><td>1300</td></tr><tr><td>-inline-tplevel</td><td>160</td></tr><tr><td>-inline-max-depth</td><td>1</td></tr><tr><td>-inline-max-unroll</td><td>0</td></tr><tr><td>-unbox-closures-factor</td><td>10</td></tr></tbody></table>

### 23.5.1 Settings at -O2 optimisation level

When `-O2` is specified two rounds of optimisation are performed. The first round uses the default parameters (see above). The second uses the following parameters.

<table><thead><tr><td>Parameter</td><td>Setting</td></tr></thead><tbody><tr><td>-inline</td><td>25</td></tr><tr><td>-inline-branch-factor</td><td>Same as default</td></tr><tr><td>-inline-alloc-cost</td><td>Double the default</td></tr><tr><td>-inline-branch-cost</td><td>Double the default</td></tr><tr><td>-inline-call-cost</td><td>Double the default</td></tr><tr><td>-inline-indirect-cost</td><td>Double the default</td></tr><tr><td>-inline-prim-cost</td><td>Double the default</td></tr><tr><td>-inline-lifting-benefit</td><td>Same as default</td></tr><tr><td>-inline-tplevel</td><td>400</td></tr><tr><td>-inline-max-depth</td><td>2</td></tr><tr><td>-inline-max-unroll</td><td>Same as default</td></tr><tr><td>-unbox-closures-factor</td><td>Same as default</td></tr></tbody></table>

### 23.5.2 Settings at -O3 optimisation level

When `-O3` is specified three rounds of optimisation are performed. The first two rounds are as for `-O2`. The third round uses the following parameters.
Chapter 26

# The “Tail Modulo Constructor”
program transformation

(Introduced in OCaml 4.14)

Note: this feature is considered experimental, and its interface may evolve, with user feedback, in the next few releases of the language.

Consider this natural implementation of the `List.map` function:

``` Ocaml
let rec map f l =
match l with
| [] -> []
| x :: xs ->
  let y = f x in
  y :: map f xs
```

A well-known limitation of this implementation is that the recursive call, `map f xs`, is not in *tail* position. The runtime needs to remember to continue with `y :: r` after the call returns a value `r`, therefore this function consumes some amount of call-stack space on each recursive call. The stack usage of `map f li` is proportional to the length of `li`. This is a correctness issue for large lists on systems configured with limited stack space – the dreaded `Stack_overflow` exception.

``` OCaml
# let with_stack_limit stack_limit f =
  let old_gcSettings = Gc.get () in
  Gc.set { old_gcSettings with stack_limit };
  Fun.protect ~finally: (fun () -> Gc.set old_gcSettings) f
;;
val with_stack_limit : int -> (unit -> 'a) -> 'a = <fun>

# with_stack_limit 20_000 (fun () ->
  List.length (map Fun.id (List.init 1_000_000 Fun.id))
);;

Stack overflow during evaluation (looping recursion?).
```

In this implementation of `map`, the recursive call happens in a position that is not a *tail* position in the program, but within a datatype constructor application that is itself in *tail* position. We
<table><thead><tr><td>Parameter</td><td>Setting</td></tr></thead><tbody><tr><td>-inline</td><td>50</td></tr><tr><td>-inline-branch-factor</td><td>Same as default</td></tr><tr><td>-inline-alloc-cost</td><td>Triple the default</td></tr><tr><td>-inline-branch-cost</td><td>Triple the default</td></tr><tr><td>-inline-call-cost</td><td>Triple the default</td></tr><tr><td>-inline-indirect-cost</td><td>Triple the default</td></tr><tr><td>-inline-prim-cost</td><td>Triple the default</td></tr><tr><td>-inline-lifting-benefit</td><td>Same as default</td></tr><tr><td>-inline-topleft</td><td>800</td></tr><tr><td>-inline-max-depth</td><td>3</td></tr><tr><td>-inline-max-unroll</td><td>1</td></tr><tr><td>-unbox-closures-factor</td><td>Same as default</td></tr></tbody></table>

## 23.6 Manual control of inlining and specialisation

Should the inliner prove recalcitrant and refuse to inline a particular function, or if the observed inlining decisions are not to the programmer's satisfaction for some other reason, inlining behaviour can be dictated by the programmer directly in the source code. One example where this might be appropriate is when the programmer, but not the compiler, knows that a particular function call is on a cold code path. It might be desirable to prevent inlining of the function so that the code size along the hot path is kept smaller, so as to increase locality.

The inliner is directed using attributes. For non-recursive functions (and one-step unrolling of recursive functions, although `@unroll` is more clear for this purpose) the following are supported:

**@@inline always or @@inline never**

Attached to a *declaration* of a function or functor, these direct the inliner to either always or never inline, irrespective of the size/benefit calculation. (If the function is recursive then the body is substituted and no special action is taken for the recursive call site(s).) `@@inline` with no argument is equivalent to `@@inline always`.

**@inlined always or @inlined never**

Attached to a function *application*, these direct the inliner likewise. These attributes at call sites override any other attribute that may be present on the corresponding declaration. `@inlined` with no argument is equivalent to `@inlined always`. `@@inlined hint` is equivalent to `@@inline always` except that it will not trigger warning 55 if the function application cannot be inlined.

For recursive functions the relevant attributes are:

**@@specialise always or @@specialise never**

Attached to a declaration of a function or functor, this directs the inliner to either always or never specialise the function so long as it has appropriate contextual knowledge, irrespective of the size/benefit calculation. `@@specialise` with no argument is equivalent to `@@specialise always`.
say that those positions, that are composed of tail positions and constructor applications, are *tail modulo constructor* (TMC) positions – we sometimes write `tail modulo cons` for brevity.

It is possible to rewrite programs such that tail modulo cons positions become tail positions; after this transformation, the implementation of `map` above becomes *tail-recursive*, in the sense that it only consumes a constant amount of stack space. The OCaml compiler implements this transformation on demand, using the `[@tail_mod_cons]` or `[@ocaml.tail_mod_cons]` attribute on the function to transform.

```ocaml
let[@tail_mod_cons] rec map f l =
match l with
| [] -> []
| x :: xs ->
    let y = f x in
    y :: map f xs
# List.length (map Fun.id (List.init 1_000_000 Fun.id));
- : int = 1000000
```

This transformation only improves calls in tail-modulo-cons position, it does not improve recursive calls that do not fit in this fragment:

(* does *not* work: addition is not a data constructor *)

```ocaml
let[@tail_mod_cons] rec length l =
match l with
| [] -> 0
| _ :: xs -> 1 + length xs
```

Warning 71 [unused-tmc-attribute]: This function is marked `@tail_mod_cons` but is never applied in TMC position.

It is of course possible to use the `[@tail_mod_cons]` transformation on functions that contain some recursive calls in tail-modulo-cons position, and some calls in other, arbitrary positions. Only the tail calls and tail-modulo-cons calls will happen in constant stack space.

**General design** This feature is provided as an explicit program transformation, not an implicit optimization. It is annotation-driven: the user is expected to express their intent by adding annotations in the program using attributes, and will be asked to do so in any ambiguous situation.

We expect it to be used mostly by advanced OCaml users needing to get some guarantees on the stack-consumption behavior of their programs. Our recommendation is to use the `[@tailcall]` annotation on all callsites that should not consume any stack space. `[@tail_mod_cons]` extends the set of functions on which calls can be annotated to be tail calls, helping establish stack-consumption guarantees in more cases.

**Performance** A standard approach to get a tail-recursive version of `List.map` is to use an accumulator to collect output elements, and reverse it at the end of the traversal.

```ocaml
let rec map f l = map_aux f [] l
and map_aux f acc l =
match l with
```
**@specialised always or @specialised never**

Attached to a function application, this directs the inliner likewise. This attribute at a call site overrides any other attribute that may be present on the corresponding declaration. (Note that the function will still only be specialised if there exist one or more invariant parameters whose values are known.) `@specialised` with no argument is equivalent to `@specialised always`.

**@unrolled n**

This attribute is attached to a function application and always takes an integer argument. Each time the inliner sees the attribute it behaves as follows:

* If `n` is zero or less, nothing happens.

* Otherwise the function being called is substituted at the call site with its body having been rewritten such that any recursive calls to that function or any others in the same *mutually-recursive group* are annotated with the attribute `unrolled(n - 1)`. Inlining may continue on that body.

As such, `n` behaves as the “maximum depth of unrolling”.

A compiler warning will be emitted if it was found impossible to obey an annotation from an `@inlined` or `@specialised` attribute.

### Example showing correct placement of attributes

```python
module F (M : sig type t end) = struct
    let[@inline never] bar x =
        x * 3

    let foo x =
        (bar[@inlined]) (42 + x)
end[@inline never]

module X = F[@inlined] (struct type t = int end)
```

## 23.7 Simplification

Simplification, which is run in conjunction with inlining, propagates information (known as *approximations*) about which variables hold what values at runtime. Certain relationships between variables and symbols are also tracked: for example, some variable may be known to always hold the same value as some other variable; or perhaps some variable may be known to always hold the value pointed to by some symbol.

The propagation can help to eliminate allocations in cases such as:

```python
let f x y =
    ...
let p = x, y in
    ...
    (fst p) ... (snd p) ...
```
```h
| [] -> List.rev acc
| x :: xs ->
  let y = f x in
  map_aux f (y :: acc) xs
```

This version is tail-recursive, but it is measurably slower than the simple, non-tail-recursive version. In contrast, the tail-mod-cons transformation provides an implementation that has comparable performance to the original version, even on small inputs.

**Evaluation order** Beware that the tail-modulo-cons transformation has an effect on evaluation order: the constructor argument that is transformed into tail-position will always be evaluated last. Consider the following example:

```h
type 'a two_headed_list =
  | Nil
  | Consnoc of 'a * 'a two_headed_list * 'a
```

```h
let[@tail_mod_cons] rec map f = function
  | Nil -> Nil
  | Consnoc (front, body, rear) ->
    Consnoc (f front, map f body, f rear)
```

Due to the `[@tail_mod_cons]` transformation, the calls to `f front` and `f rear` will be evaluated before `map f body`. In particular, this is likely to be different from the evaluation order of the unannotated version. (The evaluation order of constructor arguments is unspecified in OCaml, but many implementations typically use left-to-right or right-to-left.)

This effect on evaluation order is one of the reasons why the tail-modulo-cons transformation has to be explicitly requested by the user, instead of being applied as an automatic optimization.

**Why tail-modulo-cons?** Other program transformations, in particular a transformation to continuation-passing style (CPS), can make all functions tail-recursive, instead of targeting only a small fragment. Some reasons to provide builtin support for the less-general tail-mod-cons are as follows:

* The tail-mod-cons transformation preserves the performance of the original, non-tail-recursive version, while a continuation-passing-style transformation incurs a measurable constant-factor overhead.

* The tail-mod-cons transformation cannot be expressed as a source-to-source transformation of OCaml programs, as it relies on mutable state in type-unsafe ways. In contrast, continuation-passing-style versions can be written by hand, possibly using a convenient monadic notation.

**Note: OCaml call stack size** In OCaml 4.x and earlier, bytecode programs respect the `stack_limit` runtime parameter configuration (as set using `Gc.set` in the example above), or the `l` setting of the `OCAMLRUNPARAM` variable. Native programs ignore these settings and only respect the operating system native stack limit, as set by `ulimit` on Unix systems. Most operating systems run with a relatively low stack size limit by default, so stack overflow on non-tail-recursive functions are a common programming bug.
The projections from p may be replaced by uses of the variables x and y, potentially meaning that p becomes unused.

The propagation performed by the simplification pass is also important for discovering which functions flow to indirect call sites. This can enable the transformation of such call sites into direct call sites, which makes them eligible for an inlining transformation.

Note that no information is propagated about the contents of strings, even in safe-string mode, because it cannot yet be guaranteed that they are immutable throughout a given program.

## 23.8 Other code motion transformations

### 23.8.1 Lifting of constants

Expressions found to be constant will be lifted to symbol bindings—that is to say, they will be statically allocated in the object file—when they evaluate to boxed values. Such constants may be straightforward numeric constants, such as the floating-point number 42.0, or more complicated values such as constant closures.

Lifting of constants to toplevel reduces allocation at runtime.

The compiler aims to share constants lifted to toplevel such that there are no duplicate definitions. However if .cmx files are hidden from the compiler then maximal sharing may not be possible.

**Notes about float arrays** The following language semantics apply specifically to constant float arrays. (By “constant float array” is meant an array consisting entirely of floating point numbers that are known at compile time. A common case is a literal such as `[| 42.0; 43.0; |].`

* Constant float arrays at the toplevel are mutable and never shared. (That is to say, for each such definition there is a distinct symbol in the data section of the object file pointing at the array.)

* Constant float arrays not at toplevel are mutable and are created each time the expression is evaluated. This can be thought of as an operation that takes an immutable array (which in the source code has no associated name; let us call it the *initialising array*) and duplicates it into a fresh mutable array.

- If the array is of size four or less, the expression will create a fresh block and write the values into it one by one. There is no reference to the initialising array as a whole.

- Otherwise, the initialising array is lifted out and subject to the normal constant sharing procedure; creation of the array consists of bulk copying the initialising array into a fresh value on the OCaml heap.

### 23.8.2 Lifting of toplevel let bindings

T LEVEL let-expressions may be lifted to symbol bindings to ensure that the corresponding bound variables are not captured by closures. If the defining expression of a given binding is found to be constant, it is bound as such (the technical term is a *let-symbol* binding).

Otherwise, the symbol is bound to a (statically-allocated) *preallocated block* containing one field. At runtime, the defining expression will be evaluated and the first field of the block filled with the
Starting from OCaml 5.0, native code does not use the native system stack for OCaml function
calls anymore, so it is not affected by the operating system native stack size; both native and
bytecode programs respect the OCaml runtime's own limit. The runtime limit is set to a much
higher default than most operating system native stacks, with a limit of at least 512MiB, so stack
overflow should be much less common in practice. There is still a stack limit by default, as it
remains useful to quickly catch bugs with looping non-tail-recursive functions. Without a stack
limit, one has to wait for the whole memory to be consumed by the stack for the program to crash,
which can take a long time and make the system unresponsive.

This means that the `tail modulo constructor` transformation is less important on OCaml 5:
it does improve performance noticeably in some cases, but it is not necessary for basic correctness
for most use-cases.

## 26.1 Disambiguation

It may happen that several arguments of a constructor are recursive calls to a tail-modulo-cons
function. The transformation can only turn one of these calls into a tail call. The compiler will not
make an implicit choice, but ask the user to provide an explicit disambiguation.

Consider this type of syntactic expressions (assuming some pre-existing type `var` of expression variables):

`type var (* some pre-existing type of variables *)`

``` OCaml
type exp =
  | Var of var
  | Let of binding * exp
and binding = var * exp
```

Consider a `map` function on variables. The direct definition has two recursive calls inside
arguments of the `Let` constructor, so it gets rejected as ambiguous.

``` OCaml
let[@tail_mod_cons] rec map_vars f exp =
  match exp with
  | Var v -> Var (f v)
  | Let ((v, def), body) ->
    Let ((f v, map_vars f def), map_vars f body)
```

```
Error: [@tail_mod_cons]: this constructor application may be TMC-transformed
       in several different ways. Please disambiguate by adding an explicit
       [@tailcall] attribute to the call that should be made tail-recursive,
       or a [@tailcall false] attribute on calls that should not be
       transformed.

This call could be annotated.
This call could be annotated.
```

To disambiguate, the user should add a `@tailcall` attribute to the recursive call that should
be transformed to tail position:

``` OCaml
let[@tail_mod_cons] rec map_vars f exp =
  match exp with
```
resulting value. This *initialise-symbol* binding causes one extra indirection but ensures, by virtue of the symbol's address being known at compile time, that uses of the value are not captured by closures.

It should be noted that the blocks corresponding to *initialise-symbol* bindings are kept alive forever, by virtue of them occurring in a static table of GC roots within the object file. This extended lifetime of expressions may on occasion be surprising. If it is desired to create some non-constant value (for example when writing GC tests) that does not have this extended lifetime, then it may be created and used inside a function, with the application point of that function (perhaps at toplevel)—or indeed the function declaration itself—marked as to never be inlined. This technique prevents lifting of the definition of the value in question (assuming of course that it is not constant).

## 23.9 Unboxing transformations

The transformations in this section relate to the splitting apart of *boxed* (that is to say, non-immediate) values. They are largely intended to reduce allocation, which tends to result in a runtime performance profile with lower variance and smaller tails.

### 23.9.1 Unboxing of closure variables

This transformation is enabled unless `-no-unbox-free-vars-of-closures` is provided.

Variables that appear in closure environments may themselves be *boxed* values. As such, they may be split into further closure variables, each of which corresponds to some projection from the original closure variable(s). This transformation is called *unboxing of closure variables* or *unboxing of free variables of closures*. It is only applied when there is reasonable certainty that there are no uses of the boxed free variable itself within the corresponding function bodies.

**Example:** In the following code, the compiler observes that the closure returned from the function *f* contains a variable *pair* (free in the body of *f*) that may be split into two separate variables.

```hsharp
let f x0 x1 =
    let pair = x0, x1 in
   FORMATensis
    printf "foo\n";
fun y ->
    fst pair + snd pair + y
```

After some simplification one obtains:

```hsharp
let f x0 x1 =
    let pair_0 = x0 in
    let pair_1 = x1 in
    printf "foo\n";
fun y ->
    pair_0 + pair_1 + y
```

and then:
```h
| Var v -> Var (f v)
| Let ((v, def), body) ->
  Let ((f v, map_vars f def), (map_vars[@tailcall]) f body)
```

Be aware that the resulting function is *not* tail-recursive, the recursive call on `def` will consume stack space. However, expression trees tend to be right-leaning (lots of `Let` in sequence, rather than nested inside each other), so putting the call on `body` in tail position is an interesting improvement over the naive definition: it gives bounded stack space consumption if we assume a bound on the nesting depth of `Let` constructs.

One would also get an error when using conflicting annotations, asking for two of the constructor arguments to be put in tail position:

```h
let[@tail_mod_cons] rec map_vars f exp =
  match exp with
  | Var v -> Var (f v)
  | Let ((v, def), body) ->
    Let ((f v, (map_vars[@tailcall]) f def), (map_vars[@tailcall]) f body)
```

```
Error: [@tail_mod_cons]: this constructor application may be TMC-transformed
       in several different ways. Only one of the arguments may become a TMC
       call, but several arguments contain calls that are explicitly marked
       as tail-recursive. Please fix the conflict by reviewing and fixing the
       conflicting annotations.
```

```
This call is explicitly annotated.
```

```
This call is explicitly annotated.
```

## 26.2 Danger: getting out of tail-mod-cons

Due to the nature of the tail-mod-cons transformation (see Section 26.3 for a presentation of transformation):

* Calls from a tail-mod-cons function to another tail-mod-cons function declared in the same recursive-binding group are transformed into tail calls, as soon as they occur in tail position or tail-modulo-cons position in the source function.

* Calls from a function *not* annotated tail-mod-cons to a tail-mod-cons function or, conversely, from a tail-mod-cons function to a non-tail-mod-cons function are transformed into *non-tail* calls, even if they syntactically appear in tail position in the source program.

The fact that calls in tail position in the source program may become non-tail calls if they go from a tail-mod-cons to a non-tail-mod-cons function is surprising, and the transformation will warn about them.

For example:

```h
let[@tail_mod_cons] rec flatten = function
| [] -> []
| xs :: xss ->
  let rec append_flatten xs xss =
    match xs with
```
```h
let f x0 x1 =
    Printfprintf "foo\n";
fun y ->
    x0 + x1 + y
```

The allocation of the pair has been eliminated.

This transformation does not operate if it would cause the closure to contain more than twice as many closure variables as it did beforehand.

### 23.9.2 Unboxing of specialised arguments

This transformation is enabled unless `-no-unbox-specialised-args` is provided.

It may become the case during compilation that one or more invariant arguments to a function become specialised to a particular value. When such values are themselves boxed the corresponding specialised arguments may be split into more specialised arguments corresponding to the projections out of the boxed value that occur within the function body. This transformation is called *unboxing of specialised arguments*. It is only applied when there is reasonable certainty that the boxed argument itself is unused within the function.

If the function in question is involved in a recursive group then unboxing of specialised arguments may be immediately replicated across the group based on the dataflow between invariant arguments.

**Example:** Having been given the following code, the compiler will inline `loop` into `f`, and then observe `inv` being invariant and always the pair formed by adding 42 and 43 to the argument `x` of the function `f`.

```h
let rec loop inv xs =
    match xs with
    | [] -> fst inv + snd inv
    | x::xs -> x + loop2 xs inv
and loop2 ys inv =
    match ys with
    | [] -> 4
    | y::ys -> y - loop inv ys
```

```h
let f x =
    Printfprintf "%d\n" (loop (x + 42, x + 43) [1; 2; 3])
```

Since the functions have sufficiently few arguments, more specialised arguments will be added. After some simplification one obtains:

```h
let f x =
    let rec loop' xs inv_0 inv_1 =
        match xs with
        | [] -> inv_0 + inv_1
        | x::xs -> x + loop2' xs inv_0 inv_1
and loop2' ys inv_0 inv_1 =
    match ys with
```
```
| [] -> flatten xss
---
| x :: xs -> x :: append_flatten xs xss
---
in append_flatten xs xss
```

Warning 71 [unused-tmc-attribute]: This function is marked @tail_mod_cons but is never applied in TMC position.

Warning 72 [tmc-breaks-tailcall]: This call is in tail-modulo-cons position in a TMC function, but the function called is not itself specialized for TMC, so the call will not be transformed into a tail call.

Please either mark the called function with the [@tail_mod_cons] attribute, or mark this call with the [@tailcall false] attribute to make its non-tailness explicit.

Here the `append_flatten` helper is not annotated with `[@tail_mod_cons]`, so the calls `append_flatten xs xss` and `flatten xss` will not be tail calls. The correct fix here is to annotate `append_flatten` to be `tail-mod-cons`.

```hl60
let[@tail_mod_cons] rec flatten = function
| [] -> []
| xs :: xss ->
  let[@tail_mod_cons] rec append_flatten xs xss =
    match xs with
    | [] -> flatten xss
    | x :: xs -> x :: append_flatten xs xss
  in append_flatten xs xss
```

The same warning occurs when `append_flatten` is a non-tail-mod-cons function of the same recursive group; using the `tail-mod-cons` transformation is a property of individual functions, not whole recursive groups.

```hhl60
let[@tail_mod_cons] rec flatten = function
| [] -> []
| xs :: xss -> append_flatten xs xss
```

```hhl60
and append_flatten xs xss =
  match xs with
  | [] -> flatten xss
  | x :: xs -> x :: append_flatten xs xss
```

Warning 71 [unused-tmc-attribute]: This function is marked @tail_mod_cons but is never applied in TMC position.

Warning 72 [tmc-breaks-tailcall]: This call is in tail-modulo-cons position in a TMC function, but the function called is not itself specialized for TMC, so the call will not be transformed into a tail call.

Please either mark the called function with the [@tail_mod_cons] attribute, or mark this call with the [@tailcall false] attribute to make its non-tailness explicit.
```h
| [] -> 4
| y::ys -> y - loop' ys inv_0 inv_1
in
Printf printf "%d\n" (loop' [1; 2; 3] (x + 42) (x + 43))
```

The allocation of the pair within `f` has been removed. (Since the two closures for `loop'` and `loop2'` are constant they will also be lifted to toplevel with no runtime allocation penalty. This would also happen without having run the transformation to unbox specialise arguments.)

The transformation to unbox specialised arguments never introduces extra allocation.

The transformation will not unbox arguments if it would result in the original function having sufficiently many arguments so as to inhibit tail-call optimisation.

The transformation is implemented by creating a wrapper function that accepts the original arguments. Meanwhile, the original function is renamed and extra arguments are added corresponding to the unboxed specialised arguments; this new function is called from the wrapper. The wrapper will then be inlined at direct call sites. Indeed, all call sites will be direct unless `-unbox-closures` is being used, since they will have been generated by the compiler when originally specialising the function. (In the case of `-unbox-closures` other functions may appear with specialised arguments; in this case there may be indirect calls and these will incur a small penalty owing to having to bounce through the wrapper. The technique of *direct call surrogates* used for `-unbox-closures` is not used by the transformation to unbox specialised arguments.)

### 23.9.3 Unboxing of closures

This transformation is *not* enabled by default. It may be enabled using the `-unbox-closures` flag.

The transformation replaces closure variables by specialised arguments. The aim is to cause more closures to become closed. It is particularly applicable, as a means of reducing allocation, where the function concerned cannot be inlined or specialised. For example, some non-recursive function might be too large to inline; or some recursive function might offer no opportunities for specialisation perhaps because its only argument is one of type `unit`.

At present there may be a small penalty in terms of actual runtime performance when this transformation is enabled, although more stable performance may be obtained due to reduced allocation. It is recommended that developers experiment to determine whether the option is beneficial for their code. (It is expected that in the future it will be possible for the performance degradation to be removed.)

**Simple example:** In the following code (which might typically occur when `g` is too large to inline) the value of `x` would usually be communicated to the application of the `+` function via the closure of `g`.

```h
let f x =
let g y =
  x + y
in
(g [@inlined never]) 42
```
Again, the fix is to specialize `append_flatten` as well:

```hkl
let[@tail_mod_cons] rec flatten = function
| [] -> []
| xs :: xss -> append_flatten xs xss
```

```hkl
and[@tail_mod_cons] append_flatten xs xss =
    match xs with
    | [] -> flatten xss
    | x :: xs -> x :: append_flatten xs xss
```

Non-recursive functions can also be annotated `[@tail_mod_cons]`; this is typically useful for local bindings to recursive functions.

Incorrect version:

```hkl
let[@tail_mod_cons] rec map_vars f exp =
    let self exp = map_vars f exp in
    match exp with
    | Var v -> Var (f v)
    | Let ((v, def), body) ->
        Let ((f v, self def), (self[@tailcall]) body)
```

```
Warning 51 [wrong-tailcall-expectation]: expected tailcall
```

```
Warning 51 [wrong-tailcall-expectation]: expected tailcall
```

```
Warning 71 [unused-tmc-attribute]: This function is marked @tail_mod_cons
but is never applied in TMC position.
```

Recommended fix:

```hkl
let[@tail_mod_cons] rec map_vars f exp =
    let[@tail_mod_cons] self exp = map_vars f exp in
    match exp with
    | Var v -> Var (f v)
    | Let ((v, def), body) ->
        Let ((f v, self def), (self[@tailcall]) body)
```

In other cases, there is either no benefit in making the called function `tail-mod-cons`, or it is not possible: for example, it is a function parameter (the transformation only works with direct calls to known functions).

For example, consider a substitution function on binary trees:

```hkl
type 'a tree = Leaf of 'a | Node of 'a tree * 'a tree
```

```hkl
let[@tail_mod_cons] rec bind (f : 'a -> 'a tree) (t : 'a tree) : 'a tree =
    match t with
    | Leaf v -> f v
    | Node (left, right) ->
        Node (bind f left, (bind[@tailcall]) f right)
```

```
Warning 72 [tmc-breaks-tailcall]: This call
```
Unboxing of the closure causes the value for `x` inside `g` to be passed as an argument to `g` rather than through its closure. This means that the closure of `g` becomes constant and may be lifted to toplevel, eliminating the runtime allocation.

The transformation is implemented by adding a new wrapper function in the manner of that used when unboxing specialised arguments. The closure variables are still free in the wrapper, but the intention is that when the wrapper is inlined at direct call sites, the relevant values are passed directly to the main function via the new specialised arguments.

Adding such a wrapper will penalise indirect calls to the function (which might exist in arbitrary places; remember that this transformation is not for example applied only on functions the compiler has produced as a result of specialisation) since such calls will bounce through the wrapper. To mitigate this, if a function is small enough when weighed up against the number of free variables being removed, it will be duplicated by the transformation to obtain two versions: the original (used for indirect calls, since we can do no better) and the wrapper/rewritten function pair as described in the previous paragraph. The wrapper/rewritten function pair will only be used at direct call sites of the function. (The wrapper in this case is known as a *direct call surrogate*, since it takes the place of another function—the unchanged version used for indirect calls—at direct call sites.)

The `-unbox-closures-factor` command line flag, which takes an integer, may be used to adjust the point at which a function is deemed large enough to be ineligible for duplication. The benefit of duplication is scaled by the integer before being evaluated against the size.

**Harder example:** In the following code, there are two closure variables that would typically cause closure allocations. One is called `fv` and occurs inside the function `baz`; the other is called `z` and occurs inside the function `bar`. In this toy (yet sophisticated) example we again use an attribute to simulate the typical situation where the first argument of `baz` is too large to inline.

```hsharp
let foo c =
    let rec bar zs fv =
        match zs with
        | [] -> []
        | z::zs ->
            let rec baz f = function
              | [] -> []
              | a::l -> let r = fv + ((f [@inlined never]) a) in r :: baz f l
            in
              (map2 (fun y -> z + y) [z; 2; 3; 4]) @ bar zs fv
        in
    Printf printf "%d" (List.length (bar [1; 2; 3; 4] c))
```

The code resulting from applying `-O3 -unbox-closures` to this code passes the free variables via function arguments in order to eliminate all closure allocation in this example (aside from any that might be performed inside `printf`).
is in tail-modulo-cons position in a TMC function,
but the function called is not itself specialized for TMC,
so the call will not be transformed into a tail call.
Please either mark the called function with the [@tail_mod_cons]
attribute, or mark this call with the [@tailcall false] attribute
to make its non-tailness explicit.

Here f is a function parameter, not a direct call, and the current implementation is strictly first-order, it does not support tail-mod-cons arguments. In this case, the user should indicate that they realize this call to f v is not, in fact, in tail position, by using (f[@tailcall false]) v.

type 'a tree = Leaf of 'a | Node of 'a tree * 'a tree

let[@tail_mod_cons] rec bind (f : 'a -> 'a tree) (t : 'a tree) : 'a tree =
match t with
| Leaf v -> (f[@tailcall false]) v
| Node (left, right) ->
  Node (bind f left, (bind[@tailcall]) f right)

26.3 Details on the transformation

To use this advanced feature, it helps to be aware that the function transformation produces a
specialized function in destination-passing-style.

Recall our `map` example:

let rec map f l =
  match l with
  | [] -> []
  | x :: xs ->
      let y = f x in
      y :: map f xs

Below is a description of the transformed program in pseudo-OCaml notation: some operations
are not expressible in OCaml source code. (The transformation in fact happens on the Lambda
intermediate representation of the OCaml compiler.)

let rec map f l =
  match l with
  | [] -> []
  | x :: xs ->
      let y = f x in
      let dst = y ::{mutable} Hole in
      map_dps f xs dst 1;
      dst

and map_dps f l dst idx =
match l with
| [] -> dst.index <- []
## 23.10 Removal of unused code and values

### 23.10.1 Removal of redundant let expressions

The simplification pass removes unused `let` bindings so long as their corresponding defining expressions have “no effects”. See the section “Treatment of effects” below for the precise definition of this term.

### 23.10.2 Removal of redundant program constructs

This transformation is analogous to the removal of `let`-expressions whose defining expressions have no effects. It operates instead on symbol bindings, removing those that have no effects.

### 23.10.3 Removal of unused arguments

This transformation is only enabled by default for specialised arguments. It may be enabled for all arguments using the `-remove-unused-arguments` flag.

The pass analyses functions to determine which arguments are unused. Removal is effected by creating a wrapper function, which will be inlined at every direct call site, that accepts the original arguments and then discards the unused ones before calling the original function. As a consequence, this transformation may be detrimental if the original function is usually indirectly called, since such calls will now bounce through the wrapper. (The technique of *direct call surrogates* used to reduce this penalty during unboxing of closure variables (see above) does not yet apply to the pass that removes unused arguments.)

### 23.10.4 Removal of unused closure variables

This transformation performs an analysis across the whole compilation unit to determine whether there exist closure variables that are never used. Such closure variables are then eliminated. (Note that this has to be a whole-unit analysis because a projection of a closure variable from some particular closure may have propagated to an arbitrary location within the code due to inlining.)

## 23.11 Other code transformations

### 23.11.1 Transformation of non-escaping references into mutable variables

Flambda performs a simple analysis analogous to that performed elsewhere in the compiler that can transform `refs` into mutable variables that may then be held in registers (or on the stack as appropriate) rather than being allocated on the OCaml heap. This only happens so long as the reference concerned can be shown to not escape from its defining scope.

### 23.11.2 Substitution of closure variables for specialised arguments

This transformation discovers closure variables that are known to be equal to specialised arguments. Such closure variables are replaced by the specialised arguments; the closure variables may then be removed by the “removal of unused closure variables” pass (see below).
```h
| x :: xs ->
  let y = f x in
  let dst' = y ::{mutable} Hole in
  dst_idx <- dst';
  map_dps f xs dst' 1
```

The source version of `map` gets transformed into two functions, a *direct-style* version that is also called `map`, and a *destination-passing-style* version (DPS) called `map_dps`. The destination-passing-style version does not return a result directly, instead it writes it into a memory location specified by two additional function parameters, `dst` (a memory block) and `i` (a position within the memory block).

The source call `y :: map f xs` gets transformed into the creation of a mutable block `y ::{mutable} Hole`, whose second parameter is an un-initialized `hole`. The block is then passed to `map_dps` as a destination parameter (with offset 1).

Notice that `map` does not call itself recursively, it calls `map_dps`. Then, `map_dps` calls itself recursively, in a tail-recursive way.

The call from `map` to `map_dps` is *not* a tail call (this is something that we could improve in the future); but this call happens only once when invoking `map f l`, with all list elements after the first one processed in constant stack by `map_dps`.

This explains the “getting out of tail-mod-cons” subtleties. Consider our previous example involving mutual recursion between `flatten` and `append_flatten`.

```h
let[@tail_mod_cons] rec flatten l =
match l with
| [] -> []
| xs :: xss ->
    append_flatten xs xss
```

The call to `append_flatten`, which syntactically appears in tail position, gets transformed differently depending on whether the function has a destination-passing-style version available, that is, whether it is itself annotated `[@tail_mod_cons]`:

```h
(* if append_flatten_dps exists *)
and flatten_dps l dst i =
match l with
| [] -> dst.i <- []
| xs :: xss ->
    append_flatten_dps xs xss dst i

(* if append_flatten_dps does not exist *)
and rec flatten_dps l dst i =
match l with
| [] -> dst.i <- []
| xs :: xss ->
    dst.i <- append_flatten xs xss
```

If `append_flatten` does not have a destination-passing-style version, the call gets transformed to a non-tail call.
## 23.12 Treatment of effects

The Lambda optimisers classify expressions in order to determine whether an expression:

* does not need to be evaluated at all; and/or

* may be duplicated.

This is done by forming judgements on the *effects* and the *coefficients* that might be performed were the expression to be executed. Effects talk about how the expression might affect the world; coefficients talk about how the world might affect the expression.

Effects are classified as follows:

**No effects:**

The expression does not change the observable state of the world. For example, it must not write to any mutable storage, call arbitrary external functions or change control flow (e.g. by raising an exception). Note that allocation is *not* classed as having “no effects” (see below).

* It is assumed in the compiler that expressions with no effects, whose results are not used, may be eliminated. (This typically happens where the expression in question is the defining expression of a *let*; in such cases the *let*-expression will be eliminated.) It is further assumed that such expressions with no effects may be duplicated (and thus possibly executed more than once).

* Exceptions arising from allocation points, for example "out of memory" or exceptions propagated from finalizers or signal handlers, are treated as "effects out of the ether" and thus ignored for our determination here of effectfulness. The same goes for floating point operations that may cause hardware traps on some platforms.

**Only generative effects:**

The expression does not change the observable state of the world save for possibly affecting the state of the garbage collector by performing an allocation. Expressions that only have generative effects and whose results are unused may be eliminated by the compiler. However, unlike expressions with "no effects", such expressions will never be eligible for duplication.

**Arbitrary effects:**

All other expressions.

There is a single classification for coefficients:

**No coefficients:**

The expression does not observe the effects (in the sense described above) of other expressions.

For example, it must not read from any mutable storage or call arbitrary external functions.

It is assumed in the compiler that, subject to data dependencies, expressions with neither effects nor coefficients may be reordered with respect to other expressions.
## 26.4 Current limitations

**Purely syntactic criterion** Just like tail calls in general, the notion of tail-modulo-constructor position is purely syntactic; some simple refactoring will move calls out of tail-modulo-constructor position.

(* works as expected *)

```hkl
let[@tail_mod_cons] rec map f li =
  match li with
  | [] -> []
  | x :: xs ->
    let y = f x in
    y ::
      (* this call is in TMC position *)
      map f xs
```

(* not optimizable anymore *)

```hkl
let[@tail_mod_cons] rec map f li =
  match li with
  | [] -> []
  | x :: xs ->
    let y = f x in
    let ys =
      (* this call is not in TMC position anymore *)
      map f xs in
    y :: ys
```

Warning 71 [unused-tmc-attribute]: This function is marked @tail_mod_cons but is never applied in TMC position.

**Local, first-order transformation** When a function gets transformed with tail-mod-cons, two definitions are generated, one providing a direct-style interface and one providing the destination-passing-style version. However, not all calls to this function in tail-modulo-cons position will use the destination-passing-style version and become tail calls:

* The transformation is local: only tail-mod-cons calls to `foo` within the same compilation unit as `foo` become tail calls.

* The transformation is first-order: only direct calls to known tail-mod-cons functions become tail calls when in tail-mod-cons position, never calls to function parameters.

Consider the call `Option.map foo x` for example: even if `foo` is called in tail-mod-cons position within the definition of `Option.map`, that call will never become a tail call. (This would be the case even if the call to `Option.map` was inside the `Option` module.)

In general this limitation is not a problem for recursive functions: the first call from an outside module or a higher-order function will consume stack space, but further recursive calls in tail-mod-cons position will get optimized. For example, if `List.map` is defined as a tail-mod-cons function, calls from outside the `List` module will not become tail calls when in tail positions, but the recursive
## 23.13 Compilation of statically-allocated modules

Compilation of modules that are able to be statically allocated (for example, the module corresponding to an entire compilation unit, as opposed to a first class module dependent on values computed at runtime) initially follows the strategy used for bytecode. A sequence of `let`-bindings, which may be interspersed with arbitrary effects, surrounds a record creation that becomes the module block. The Lambda-specific transformation follows: these bindings are lifted to toplevel symbols, as described above.

## 23.14 Inhibition of optimisation

Especially when writing benchmarking suites that run non-side-effecting algorithms in loops, it may be found that the optimiser entirely elides the code being benchmarked. This behaviour can be prevented by using the `Sys.opaque_identity` function (which indeed behaves as a normal OCaml function and does not possess any “magic” semantics). The documentation of the `Sys` module should be consulted for further details.

## 23.15 Use of unsafe operations

The behaviour of the Lambda simplification pass means that certain unsafe operations, which may without Lambda or when using previous versions of the compiler be safe, must not be used. This specifically refers to functions found in the `Obj` module.

In particular, it is forbidden to change any value (for example using `Obj.set_field` or `Obj.set_tag`) that is not mutable. (Values returned from C stubs are always treated as mutable.) The compiler will emit warning 59 if it detects such a write—but it cannot warn in all cases. Here is an example of code that will trigger the warning:

```hml
let f x =
    let a = 42, x in
    (Obj.magic a : int ref) := 1;
    fst a
```

The reason this is unsafe is because the simplification pass believes that `fst a` holds the value `42`; and indeed it must, unless type soundness has been broken via unsafe operations.

If it must be the case that code has to be written that triggers warning 59, but the code is known to actually be correct (for some definition of correct), then `Sys.opaque_identity` may be used to wrap the value before unsafe operations are performed upon it. Great care must be taken when doing this to ensure that the opacity is added at the correct place. It must be emphasised that this use of `Sys.opaque_identity` is only for **exceptional** cases. It should not be used in normal code or to try to guide the optimiser.

As an example, this code will return the integer 1:

```hml
let f x =
    let a = Sys.opaque_identity (42, x) in
    (Obj.magic a : int ref) := 1;
    fst a
```
calls within the definition of `List.map` are in tail-modulo-cons positions and do become tail calls: processing the first element of the list will consume stack space, but all further elements are handled in constant space.

These limitations may be an issue in more complex situations where mutual recursion happens between functions, with some functions not annotated tail-mod-cons, or defined across different modules, or called indirectly, for example through function parameters.

### Non-exact calls to tupled functions

OCaml performs an implicit optimization for “tupled” functions, which take a single parameter that is a tuple: `let f (x, y, z) = ...`. Direct calls to these functions with a tuple literal argument (like `f (a, b, c)`) will call the “tupled” function by passing the parameters directly, instead of building a tuple of them. Other calls, either indirect calls or calls passing a more complex tuple value (like `let t = (a, b, c) in f t`) are compiled as “inexact” calls that go through a wrapper.

The `[@tail_mod_cons]` transformation supports tupled functions, but will only optimize “exact” calls in tail position; direct calls to something other than a tuple literal will not become tail calls. The user can manually unpack a tuple to force a call to be “exact”: `let (x, y, z) = t in f (x, y, z)`. If there is any doubt as to whether a call can be tail-mod-cons-optimized or not, one can use the `[@tailcall]` attribute on the called function, which will warn if the transformation is not possible.

``` Ocaml
let rec map (f, l) =
  match l with
  | [] -> []
  | x :: xs ->
      let y = f x in
      let args = (f, xs) in
      (* this inexact call cannot be tail-optimized, so a warning will be raised *)
      y :: (map[@tailcall]) args

Warning 51 [wrong-tailcall-expectation]: expected tailcall
```
However the following code will still return 42:

```h
let f x =
    let a = 42, x in
    Sys.opaque_identity (Obj.magic a : int ref) := 1;
    fst a
```

High levels of inlining performed by `Lambda` may expose bugs in code thought previously to be correct. Take care, for example, not to add type annotations that claim some mutable value is always immediate if it might be possible for an unsafe operation to update it to a boxed value.

## 23.16 Glossary

The following terminology is used in this chapter of the manual.

**Call site**

See *direct call site* and *indirect call site* below.

**Closed function**

A function whose body has no free variables except its parameters and any to which are bound other functions within the same (possibly mutually-recursive) declaration.

**Closure**

The runtime representation of a function. This includes pointers to the code of the function together with the values of any variables that are used in the body of the function but actually defined outside of the function, in the enclosing scope. The values of such variables, collectively known as the *environment*, are required because the function may be invoked from a place where the original bindings of such variables are no longer in scope. A group of possibly mutually-recursive functions defined using `let rec` all share a single closure. (Note to developers: in the `Lambda` source code a *closure* always corresponds to a single function; a *set of closures* refers to a group of such.)

**Closure variable**

A member of the environment held within the closure of a given function.

**Constant**

Some entity (typically an expression) the value of which is known by the compiler at compile time. Constantness may be explicit from the source code or inferred by the `Lambda` optimisers.

**Constant closure**

A closure that is statically allocated in an object file. It is almost always the case that the environment portion of such a closure is empty.

**Defining expression**

The expression `e` in `let x = e in e'`.

**Direct call site**

A place in a program's code where a function is called and it is known at compile time which function it will always be.
**Indirect call site**

A place in a program's code where a function is called but is not known to be a direct call site.

**Program**

A collection of *symbol bindings* forming the definition of a single compilation unit (i.e. `.cmx` file).

**Specialised argument**

An argument to a function that is known to always hold a particular value at runtime. These are introduced by the inliner when specialising recursive functions; and the `unbox-closures` pass. (See section 23.4.)

**Symbol**

A name referencing a particular place in an object file or executable image. At that particular place will be some constant value. Symbols may be examined using operating system-specific tools (for example `objdump` on Linux).

**Symbol binding**

Analogous to a `let`-expression but working at the level of symbols defined in the object file. The address of a symbol is fixed, but it may be bound to both constant and non-constant expressions.

**TLEVEL**

An expression in the current program which is not enclosed within any function declaration.

**Variable**

A named entity to which some OCaml value is bound by a `let` expression, pattern-matching construction, or similar.
# Part IV

## The OCaml library
# Chapter 24

## Fuzzing with afl-fuzz

### 24.1 Overview

American fuzzy lop ("afl-fuzz") is a fuzzer, a tool for testing software by providing randomly-generated inputs, searching for those inputs which cause the program to crash.

Unlike most fuzzers, afl-fuzz observes the internal behaviour of the program being tested, and adjusts the test cases it generates to trigger unexplored execution paths. As a result, test cases generated by afl-fuzz cover more of the possible behaviours of the tested program than other fuzzers.

This requires that programs to be tested are instrumented to communicate with afl-fuzz. The native-code compiler "ocamopt" can generate such instrumentation, allowing afl-fuzz to be used against programs written in OCaml.

For more information on afl-fuzz, see the website at http://lcamtuf.coredump.cx/afl/

### 24.2 Generating instrumentation

The instrumentation that afl-fuzz requires is not generated by default, and must be explicitly enabled, by passing the `-afl-instrument` option to `ocamopt`.

To fuzz a large system without modifying build tools, OCaml's `configure` script also accepts the `afl-instrument` option. If OCaml is configured with `afl-instrument`, then all programs compiled by `ocamopt` will be instrumented.

#### 24.2.1 Advanced options

In rare cases, it is useful to control the amount of instrumentation generated. By passing the `-afl-inst-ratio N` argument to `ocamopt` with N less than 100, instrumentation can be generated for only N% of branches. (See the afl-fuzz documentation on the parameter `AFL_INSTALL ratio` for the precise effect of this).

### 24.3 Example

As an example, we fuzz-test the following program, `readline.ml`:
<figure><img src="image_5.png" /></figure>
```perl
let _ =
    let s = read_line () in
    match Array.to_list (Array.init (String.length s) (String.get s)) with
        ['s'; 'e'; 'c'; 'r'; 'e'; 't'; ' '; 'c'; 'o'; 'd'; 'e'] -> failwith "uh oh"
    | _ -> ()
```

There is a single input (the string "secret code") which causes this program to crash, but finding it by blind random search is infeasible.

Instead, we compile with afl-fuzz instrumentation enabled:

```bash
ocamlopt -afl-instrument readline.ml -o readline
```

Next, we run the program under afl-fuzz:

```bash
mkdir input
echo asdf > input/testcase
mkdir output
afl-fuzz -m none -i input -o output ./readline
```

By inspecting instrumentation output, the fuzzer finds the crashing input quickly.

Note: To fuzz-test an OCaml program with afl-fuzz, passing the option `-m none` is required to disable afl-fuzz's default 50MB virtual memory limit.
# Chapter 27

## The core library

This chapter describes the OCaml core library, which is composed of declarations for built-in types and exceptions, plus the module `Stdlib` that provides basic operations on these built-in types. The `Stdlib` module is special in two ways:

* It is automatically linked with the user's object code files by the `ocamlc` command (chapter 13).

* It is automatically "opened" when a compilation starts, or when the toplevel system is launched. Hence, it is possible to use unqualified identifiers to refer to the functions provided by the `Stdlib` module, without adding a `open Stdlib` directive.

### Conventions

The declarations of the built-in types and the components of module `Stdlib` are printed one by one in typewriter font, followed by a short comment. All library modules and the components they provide are indexed at the end of this report.

### 27.1 Built-in types and predefined exceptions

The following built-in types and predefined exceptions are always defined in the compilation environment, but are not part of any module. As a consequence, they can only be referred by their short names.

#### Built-in types

**type int**

The type of integer numbers.

**type char**

The type of characters.

**type bytes**

The type of (writable) byte sequences.
# Chapter 25

## Runtime tracing with runtime events

This chapter describes the runtime events tracing system which enables continuous extraction of performance information from the OCaml runtime with very low overhead. The system and interfaces are low-level and tightly coupled to the runtime implementation, it is intended for end-users to rely on tooling to consume and visualise data of interest.

Data emitted includes:

* Event times of garbage collector and runtime phases

* Minor and major heap sizings and utilization

* Allocation and promotion rates between heaps

Additional events can be declared and consumed, providing higher-level monitoring capabilities to OCaml libraries. They are referred to as *custom events*.

### 25.1 Overview

There are three main classes of events emitted by the runtime events system:

**Spans** Events spanning over a duration in time. For example, the runtime events tracing system emits a span event that starts when a minor collection begins in the OCaml garbage collector and ends when the collection is completed. Spans can contain other spans, e.g other span events may be emitted that begin after a minor collection has begun and end before it does.

**Lifecycle events** Events that occur at a moment in time. For example, when a domain terminates, a corresponding lifecycle event is emitted.

**Counters** Events that include a measurement of some quantity of interest. For example, the number of words promoted from the minor to the major heap during the last minor garbage collection is emitted as a counter event.

The runtime events tracing system is designed to be used in different contexts:

**Self monitoring** OCaml programs and libraries can install their own callbacks to listen for runtime events and react to them programmatically, for example, to export events to disk or over the network.
type string

The type of (read-only) character strings.

type float

The type of floating-point numbers.

type bool = false | true

The type of booleans (truth values).

type unit = ()

The type of the unit value.

type exn

The type of exception values.

type 'a array

The type of arrays whose elements have type 'a.

type 'a list = [] | :: of 'a * 'a list

The type of lists whose elements have type 'a.

type 'a option = None | Some of 'a

The type of optional values of type 'a.

type int32

The type of signed 32-bit integers. Literals for 32-bit integers are suffixed by l. See the `Int32[28.27]` module.

type int64

The type of signed 64-bit integers. Literals for 64-bit integers are suffixed by L. See the `Int64[28.28]` module.

type nativeint

The type of signed, platform-native integers (32 bits on 32-bit processors, 64 bits on 64-bit processors). Literals for native integers are suffixed by n. See the `Nativeint[28.37]` module.

type ('a, 'b, 'c, 'd, 'e, 'f) format6

The type of format strings. 'a is the type of the parameters of the format, 'f is the result type for the `printf`-style functions, 'b is the type of the first argument given to `%a` and `%t` printing functions (see module `Format[28.43]'), 'c is the result type of these functions, and also the type of the argument transmitted to the first argument of `kprintf`-style functions, 'd is the result type for the `scanf`-style functions (see module `Scanf[28.47]`), and 'e is the type of the receiver function for the `scanf`-style functions.

type 'a lazy_t

This type is used to implement the `Lazy[28.29]` module. It should not be used directly.
**External monitoring** An external process can consume the runtime events of an OCaml program whose runtime tracing system has been enabled by setting the corresponding environment variable.

The runtime events tracing system logs events to a *ring buffer*. Consequently, old events are being overwritten by new events. Consumers can either continuously consume events or choose to only do so in response to some circumstance, e.g if a particular query or operation takes longer than expected to complete.

## 25.2 Architecture

The runtime tracing system conceptually consists of two parts: 1) the event sources which emit events and 2) the events transport that ingests and transports these events.

### 25.2.1 Event sources

Event sources are event-emitting points. There are a number of such sources in the OCaml runtime system. These are further split into two sets: 1) event sources that are always available and 2) event sources that are only available in the instrumented runtime. Event sources in the instrumented runtime are primarily of interest to developers of the OCaml runtime and garbage collector and, at present, only consist of major heap allocation size counter events.

The full set of events emitted by event sources and their documentation can be found in section 33.1.

### 25.2.2 Events transport

The events transport part of the system ingests events emitted by the event sources and makes them available to consumers.

### 25.2.3 Ring buffers

Events are transported using a data structure known as a *ring buffer*. This data structure consists of two pointers into a linear backing array, the tail pointer points to a location where new events can be written and the head pointer points to the oldest event in the buffer that can be read. When insufficient space is available in the backing array to write new events, the head pointer is advanced and the oldest events are overwritten by new ones.

The ring buffer implementation used in runtime events can be written by at most one producer at a time but can be read simultaneously by multiple consumers without coordination from the producer. There is a unique ring buffer for every running domain and, on domain termination, ring buffers may be re-used for newly spawned domains. Ring buffers are only allocated (including for the main domain) when runtime events are enabled. The ring buffers themselves are stored in a memory-mapped file with the processes identifier as the name and the extension `.events`, this enables them to be read from outside the main OCaml process. See `Runtime_events` [33.1] for more information.
## Predefined exceptions

**exception Match_failure of (string * int * int)**

Exception raised when none of the cases of a pattern-matching apply. The arguments are the location of the `match` keyword in the source code (file name, line number, column number).

**exception Assert_failure of (string * int * int)**

Exception raised when an assertion fails. The arguments are the location of the `assert` keyword in the source code (file name, line number, column number).

**exception Invalid_argument of string**

Exception raised by library functions to signal that the given arguments do not make sense. The string gives some information to the programmer. As a general rule, this exception should not be caught, it denotes a programming error and the code should be modified not to trigger it.

**exception Failure of string**

Exception raised by library functions to signal that they are undefined on the given arguments. The string is meant to give some information to the programmer; you must *not* pattern match on the string literal because it may change in future versions (use `Failure _` instead).

**exception Not_found**

Exception raised by search functions when the desired object could not be found.

**exception Out_of_memory**

Exception raised by the garbage collector when there is insufficient memory to complete the computation. (Not reliable for allocations on the minor heap.)

**exception Stack_overflow**

Exception raised by the bytecode interpreter when the evaluation stack reaches its maximal size. This often indicates infinite or excessively deep recursion in the user's program. Before 4.10, it was not fully implemented by the native-code compiler.

**exception Sys_error of string**

Exception raised by the input/output functions to report an operating system error. The string is meant to give some information to the programmer; you must *not* pattern match on the string literal because it may change in future versions (use `Sys_error _` instead).

**exception End_of_file**

Exception raised by input functions to signal that the end of file has been reached.

**exception Division_by_zero**

Exception raised by integer division and remainder operations when their second argument is zero.
### 25.2.4 Consumption APIs

The runtime event tracing system provides both OCaml and C APIs which are cursor-based and polling-driven. The high-level process for consuming events is as follows:

1. A cursor is created via `Runtime_events.create_cursor` for either the current process or an external process (specified by a path and PID).

2. `Runtime_events.Callbacks.create` is called to register a callback function to receive the events.

3. The cursor is polled via `Runtime_events.read_poll` using the callbacks created in the previous step. For each matching event in the ring buffers, the provided callback functions are called. In addition to the emitted events, callbacks are given the emitter domain's ID and the emission timestamp.

## 25.3 Usage

### 25.3.1 With OCaml APIs

We start with a simple example that prints the name, begin and end times of events emitted by the runtime event tracing system:

```ocaml
let runtime_begin _ ts phase =
    Printf printf "Begin\t%s\t%Ld\n"
    (Runtime_eventsaggolta phase)
    (Runtime_events獲取.Timestamp.to_int64 ts)
```

```ocaml
let runtime_end _ ts phase =
    Printf printf "End\t%s\t%Ld\n"
    (Runtime_events runtime_phase_name phase)
    (Runtime_events獲取.Timestamp.to_int64 ts)
```

``` Ocaml
let () =
    Runtime_events. start ();

let cursor = Runtime_events.create_cursor None in
let callbacks = Runtime_events.Callbacks.create ~runtime_begin ~runtime_end ()
in
while true do
    let list_ref = ref [] in (* for later fake GC work *)
    for _ = 1 to 100 do
        (* here we do some fake GC work *)
        list_ref := [];
        for _ = 1 to 10 do
            list_ref := (Sys.opaque_identity(ref 42)) :: !list_ref
done;
Gc.full_major ();
```
**exception Sys blocked io**

A special case of Sys_error raised when no I/O is possible on a non-blocking I/O channel.

**exception Undefined_recursive_module of (string * int * int)**

Exception raised when an ill-founded recursive module definition is evaluated. (See section 12.2.) The arguments are the location of the definition in the source code (file name, line number, column number).

## 27.2 Module Stdlib: The OCaml Standard library.

This module is automatically opened at the beginning of each compilation. All components of this module can therefore be referred by their short name, without prefixing them by Stdlib.

In particular, it provides the basic operations over the built-in types (numbers, booleans, byte sequences, strings, exceptions, references, lists, arrays, input-output channels, ...) and the standard library modules [27.2].

### Exceptions

**val raise : exn -> 'a**

Raise the given exception value

**val raise_notrace : exn -> 'a**

A faster version raise which does not record the backtrace.

**Since: 4.02**

**val invalid_arg : string -> 'a**

Raise exception Invalid_argument with the given string.

**val failwith : string -> 'a**

Raise exception Failure with the given string.

**exception Exit**

The Exit exception is not raised by any library function. It is provided for use in your programs.

**exception Match_failure of (string * int * int)**

Exception raised when none of the cases of a pattern-matching apply. The arguments are the location of the match keyword in the source code (file name, line number, column number).

**exception Assert_failure of (string * int * int)**

Exception raised when an assertion fails. The arguments are the location of the assert keyword in the source code (file name, line number, column number).
```c
done;
ignore(Runner_events.read_poll cursor callbacks None);
Unix.sleep 1

done
```

The next step is to compile and link the program with the `runtime_events` library. This can be done as follows:

```
ocamlopt -I +runtime_events -I +unix unix.cmxa runtime_events.cmxa example.ml -o example
```

When using the `dune` build system, this example can be built as follows:

```
hti
(name example)
(modules example)
(libraries unix runtime_events))
```

Running the compiled binary of the example gives an output similar to:

```
Begin   explicit_gc_full_major 24086187297852
Begin   stwLeader              24086187298594
Begin   minor                 24086187299404
Begin   minor_global_root      24086187299807
End     minor_global_root      24086187331461
Begin   minor_remembered_set   24086187331631
Begin   minor_finalizers_oldify 24086187544312
End     minor_finalizers_oldify 24086187544704
Begin   minor_remembered_setPromote 24086187544879
End     minor_remembered_setPromote 24086187606414
End     minor_remembered_set      24086187606584
Begin   minor_finalizers_admin  24086187606854
End     minor_finalizers_admin  24086187607152
Begin   minor_local_root       24086187607329
Begin   minor_local_rootPromote  24086187609699
End     minor_local_rootPromote  24086187610539
End     minor_local_root        24086187610709
End     minor                 24086187611746
Begin   minor_clear           24086187612238
End     minor_clear           24086187612580
End     stwLeader              24086187613209
...
```

This is an example of self-monitoring, where a program explicitly starts listening to runtime events and monitors itself.
**exception Invalid_argument of string**

Exception raised by library functions to signal that the given arguments do not make sense. The string gives some information to the programmer. As a general rule, this exception should not be caught, it denotes a programming error and the code should be modified not to trigger it.

**exception Failure of string**

Exception raised by library functions to signal that they are undefined on the given arguments. The string is meant to give some information to the programmer; you must not pattern match on the string literal because it may change in future versions (use `Failure _` instead).

**exception Not_found**

Exception raised by search functions when the desired object could not be found.

**exception Out_of_memory**

Exception raised by the garbage collector when there is insufficient memory to complete the computation. (Not reliable for allocations on the minor heap.)

**exception Stack_overflow**

Exception raised by the bytecode interpreter when the evaluation stack reaches its maximal size. This often indicates infinite or excessively deep recursion in the user's program.

Before 4.10, it was not fully implemented by the native-code compiler.

**exception Sys_error of string**

Exception raised by the input/output functions to report an operating system error. The string is meant to give some information to the programmer; you must not pattern match on the string literal because it may change in future versions (use `Sys_error _` instead).

**exception End_of_file**

Exception raised by input functions to signal that the end of file has been reached.

**exception Division_by_zero**

Exception raised by integer division and remainder operations when their second argument is zero.

**exception Sys_blocks_io**

A special case of `Sys_error` raised when no I/O is possible on a non-blocking I/O channel.

**exception Undefined_recursive_module of (string * int * int)**

Exception raised when an ill-founded recursive module definition is evaluated. The arguments are the location of the definition in the source code (file name, line number, column number).
For external monitoring, a program does not need to be aware of the existence of runtime events. Runtime events can be controlled via the environment variable `OCAML_RUNTIME EVENTS START` which, when set, will cause the runtime tracing system to be started at program initialization.

We could remove `Runtime_events.start()`; from the previous example and, instead, call the program as below to produce the same result:

`OCAML_RUNTIME EVENTS START=1 ./example`

### 25.3.2 Environment variables

Environment variables can be used to control different aspects of the runtime event tracing system. The following environment variables are available:

* `OCAML_RUNTIME EVENTS START` if set will cause the runtime events system to be started as part of the OCaml runtime initialization.

* `OCAML_RUNTIME EVENTS_DIR` sets the directory where the `.events` files containing the runtime event tracing system's ring buffers will be located. If not present the program's working directory will be used.

* `OCAML_RUNTIME EVENTS_PRESERVE` if set will make the OCaml runtime preserve the runtime events ring buffer files past the termination of the OCaml program. This can be useful for monitoring very short running programs. If not set, the `.events` files of the OCaml program will be deleted at program termination.

The size of the runtime events ring buffers can be configured via `OCAMLRUNPARAM`, see section 15.2 for more information.

### 25.3.3 Building with the instrumented runtime

Some events are only emitted by the instrumented runtime. To receive them, the OCaml program needs to be compiled and linked against the instrumented runtime. For our example program from earlier, this is achieved as follows:

```
ocamlopt -runtime-variant i -I +runtime_events -I +unix unix.cmxa runtime_events.cmxa example
o example
```

And for dune:

```
(excutable
(name example)
(modules example)
 flags "-runtime-variant=i")
(libraries unix runtime_events))
```
Comparisons

val (=) : 'a -> 'a -> bool

e1 = e2 tests for structural equality of e1 and e2. Mutable structures (e.g. references and arrays) are equal if and only if their current contents are structurally equal, even if the two mutable objects are not the same physical object. Equality between functional values raises Invalid_argument. Equality between cyclic data structures may not terminate.
Left-associative operator, see Ocaml_operators[28.60] for more information.

val (<>) : 'a -> 'a -> bool

Negation of (=)[27.2]. Left-associative operator, see Ocaml_operators[28.60] for more information.

val (<) : 'a -> 'a -> bool

See (>=)[27.2]. Left-associative operator, see Ocaml_operators[28.60] for more information.

val (>) : 'a -> 'a -> bool

See (>=)[27.2]. Left-associative operator, see Ocaml_operators[28.60] for more information.

val (<=) : 'a -> 'a -> bool

See (>=)[27.2]. Left-associative operator, see Ocaml_operators[28.60] for more information.

val (>=) : 'a -> 'a -> bool

Structural ordering functions. These functions coincide with the usual orderings over integers, characters, strings, byte sequences and floating-point numbers, and extend them to a total ordering over all types. The ordering is compatible with ( = ). As in the case of ( = ), mutable structures are compared by contents. Comparison between functional values raises Invalid_argument. Comparison between cyclic structures may not terminate.
Left-associative operator, see Ocaml_operators[28.60] for more information.

val compare : 'a -> 'a -> int

compare x y returns 0 if x is equal to y, a negative integer if x is less than y, and a positive integer if x is greater than y. The ordering implemented by compare is compatible with the comparison predicates =, < and > defined above, with one difference on the treatment of the float value nan[27.2]. Namely, the comparison predicates treat nan as different from any other float value, including itself; while compare treats nan as equal to itself and less than any other float value. This treatment of nan ensures that compare defines a total ordering relation.

compare applied to functional values may raise Invalid_argument. compare applied to cyclic structures may not terminate.

The compare function can be used as the comparison function required by the Set.Make[28.49] and Map.Make[28.33] functors, as well as the List.sort[28.31] and Array.sort[28.2] functions.

val min : 'a -> 'a -> 'a
### 25.3.4 With tooling

Programmatic access to events is intended primarily for writers of observability libraries and tooling for end-users. The flexible API enables use of the performance data from runtime events for logging and monitoring purposes.

In this section we cover several utilities in the `runtime_events_tools` package which provide simple ways of extracting and summarising data from runtime events. The trace utility in particular produces similar data to the previous 'eventlog' instrumentation system available in OCaml 4.12 to 4.14.

First, install `runtime_events_tools` in an OCaml 5.0+ opam switch:

```
opam install runtime_events_tools
```

This should install the olly tool in your path. You can now generate runtime traces for programs compiled with OCaml 5.0+ using the trace subcommand:

```
olly trace trace.json 'your_program.exe .. args ..'
```

Runtime tracing data will be generated in the json Trace Event Format to trace.json. This can then be loaded into the Chrome tracing viewer or into Perfetto to visualize the collected trace.

### 25.3.5 Measuring GC latency

The olly utility also includes a latency subcommand which consumes runtime events data and on program completion emits a parseable histogram summary of pause durations. It can be run as follows:

```
olly latency 'your_program.exe .. args ..'
```

This should produce an output similar to the following:

```
GC latency profile:
#[Mean (ms): 2.46, Stddev (ms): 3.87]
#[Min (ms): 0.01, max (ms): 9.17]
```

<table><thead><tr><th>Percentile</th><th>Latency (ms)</th></tr></thead><tbody><tr><td>25.0000</td><td>0.01</td></tr><tr><td>50.0000</td><td>0.23</td></tr><tr><td>60.0000</td><td>0.23</td></tr><tr><td>70.0000</td><td>0.45</td></tr><tr><td>75.0000</td><td>0.45</td></tr><tr><td>80.0000</td><td>0.45</td></tr><tr><td>85.0000</td><td>0.45</td></tr><tr><td>90.0000</td><td>9.17</td></tr><tr><td>95.0000</td><td>9.17</td></tr><tr><td>96.0000</td><td>9.17</td></tr><tr><td>97.0000</td><td>9.17</td></tr></tbody></table>
Return the smaller of the two arguments. The result is unspecified if one of the arguments contains the float value nan.

val max : 'a -> 'a -> 'a

Return the greater of the two arguments. The result is unspecified if one of the arguments contains the float value nan.

val (==) : 'a -> 'a -> bool

e1 == e2 tests for physical equality of e1 and e2. On mutable types such as references, arrays, byte sequences, records with mutable fields and objects with mutable instance variables, e1 == e2 is true if and only if physical modification of e1 also affects e2. On non-mutable types, the behavior of ( == ) is implementation-dependent; however, it is guaranteed that e1 == e2 implies compare e1 e2 = 0. Left-associative operator, see Ocaml_operators[28.60] for more information.

val (!=) : 'a -> 'a -> bool

Negation of (==)[27.2]. Left-associative operator, see Ocaml_operators[28.60] for more information.

Boolean operations

val not : bool -> bool

The boolean negation.

val (&&) : bool -> bool -> bool

The boolean 'and'. Evaluation is sequential, left-to-right: in e1 && e2, e1 is evaluated first, and if it returns false, e2 is not evaluated at all. Right-associative operator, see Ocaml_operators[28.60] for more information.

val (||) : bool -> bool -> bool

The boolean 'or'. Evaluation is sequential, left-to-right: in e1 || e2, e1 is evaluated first, and if it returns true, e2 is not evaluated at all. Right-associative operator, see Ocaml_operators[28.60] for more information.

Debugging

val __LOC__ : string

__LOC__ returns the location at which this expression appears in the file currently being parsed by the compiler, with the standard error format of OCaml: "File %S, line %d, characters %d-%d".

Since: 4.02

val __FILE__ : string
<table><tr><td>98.0000</td><td>9.17</td></tr><tr><td>99.0000</td><td>9.17</td></tr><tr><td>99.9000</td><td>9.17</td></tr><tr><td>99.9900</td><td>9.17</td></tr><tr><td>99.9990</td><td>9.17</td></tr><tr><td>99.9999</td><td>9.17</td></tr><tr><td>100.0000</td><td>9.17</td></tr></table>

# 25.4 Custom events

## 25.4.1 Overview

The runtime events system supports defining custom events. Identified by a globally unique name, they are emitted with payloads of built-in (`unit`, `int`, `span`) and user-defined types. To understand the manipulation of custom events, it is useful to know how they are transported and stored: their representation consists of a name string (in fact, an index into an array of all custom names) and an arbitrary byte sequence. Custom event types can be *registered* by providing encoding and decoding functions to and from a byte sequence, via the function `Runtime_events.Type.register`.

Defining a new custom event (whether its payload is of a built-in type or a custom type) is done via another registration function, `Runtime_events.User.register`. This function records the association between the custom event's name, its type, and a *tag*. The *tag* is then used when emitting or consuming custom events; it acts as a shorthand for the event name.

Once an event is registered, values for that event are emitted using `Runtime_events.User.write`.

To summarize, for a user to emit and consume custom events using a custom type they need to:

1. register the custom type

2. extend the `Runtime_events.User.tag` variant with a new constructor

3. register the new custom event, binding together the custom event name, the new tag, and the new custom type

4. emit instances of the event using `Runtime_events.User.write`

5. indicate a callback which should receive all events of the custom type using `Runtime_events Calls backs.add_user_event` to register it inside a `Runtime_events.Calls backs.t` object to be used when polling (see section 25.3.1 above).

6. (Optional, if there are distinct events registered with the same custom type:) In that callback, pattern-match on the event tag in order to discriminate between different event tags of the same custom type.

Note that if the emitter and the consumer are different programs, both must perform steps 1 to 3 to register custom events and custom event types (if any). Note that the tag values need not be the same in both programs; the only values that should match are the names.

Unregistered events for user-defined types are currently ignored. As a fallback, unregistered, custom events of a built-in type are available, but are all tagged `UNK` (unknown).

There are thus three cases for the consumer when an event is received:
__FILE__ returns the name of the file currently being parsed by the compiler.

Since: 4.02

val __LINE__ : int

__LINE__ returns the line number at which this expression appears in the file currently being parsed by the compiler.

Since: 4.02

val __MODULE__ : string

__MODULE__ returns the module name of the file being parsed by the compiler.

Since: 4.02

val __POS__ : string * int * int * int

__POS__ returns a tuple (file, lnum, cnum, enum), corresponding to the location at which this expression appears in the file currently being parsed by the compiler. file is the current filename, lnum the line number, cnum the character position in the line and enum the last character position in the line.

Since: 4.02

val __FUNCTION__ : string

__FUNCTION__ returns the name of the current function or method, including any enclosing modules or classes.

Since: 4.12

val __LOC_OF__ : 'a -> string * 'a

__LOC_OF__ expr returns a pair (loc, expr) where loc is the location of expr in the file currently being parsed by the compiler, with the standard error format of OCaml: "File %S, line %d, characters %d-%d".

Since: 4.02

val __LINE_OF__ : 'a -> int * 'a

__LINE_OF__ expr returns a pair (line, expr), where line is the line number at which the expression expr appears in the file currently being parsed by the compiler.

Since: 4.02

val __POS_OF__ : 'a -> (string * int * int * int) * 'a

__POS_OF__ expr returns a pair (loc, expr), where loc is a tuple (file, lnum, cnum, enum) corresponding to the location at which the expression expr appears in the file currently being parsed by the compiler. file is the current filename, lnum the line number, cnum the character position in the line and enum the last character position in the line.

Since: 4.02
* event is registered: payload and tag are available.

* event is not registered and has a built-in event type (unit, int, span): only the payload is available.

* event is not registered and has a custom event type: event is dropped.

Note that event consumers bind callbacks to event types, so they can work as generic listeners interpreting payloads coming from events that were not registered. Because this only works for events of built-in types, it can be useful to emit events in pairs: an event of a custom type with an event of a built-in type, enabling the design of a specialized consumer while staying compatible with generic tracing tools.

### 25.4.2 Tracing custom events: an example

The following program uses the `Runtime_events.User` and `Runtime_events.Type` modules to declare two custom events providing `span` and `int` values. The `tag` extensible variant is extended with `CustomSpan` and `CustomInt`.

```python
type Runtime_events.User.tag += CustomSpan | CustomInt

let count_span =
    Runtime_events.User.register "count.span" CustomSpan
    Runtime_events.Type.span

let count_value =
    Runtime_events.User.register "count.value" CustomInt
    Runtime_events.Type.int

let count () =
    Runtime_events.User.write count_span Begin;
    for i = 1 to 5 do
        Runtime_events.User.write count_value i
    done;
    Runtime_events.User.write count_span End

let () =
    Runtime_events.start ();
    for _ = 1 to 3 do
        count ()
    done
```

On the consumer side, one can use the provided event tag and type to match on the relevant events. For the sake of completeness, we assume that the consumer is a different program, and repeat the definition of the new tag constructors, as well as the registering of the custom events.

```python
type Runtime_events.User.tag += CustomSpan | CustomInt
```
## Composition operators

`val (|>) : 'a -> ('a -> 'b) -> 'b`

Reverse-application operator: `x |> f |> g` is exactly equivalent to `g (f (x))`. Left-associative operator, see `Ocaml_operators[28.60]` for more information.

**Since:** 4.01

`val (@@) : ('a -> 'b) -> 'a -> 'b`

Application operator: `g @@ f @@ x` is exactly equivalent to `g (f (x))`. Right-associative operator, see `Ocaml_operators[28.60]` for more information.

**Since:** 4.01

## Integer arithmetic

Integers are `Sys.int_size` bits wide. All operations are taken modulo $2^{\text{Sys.int_size}}$. They do not fail on overflow.

`val (~-) : int -> int`

Unary negation. You can also write `- e` instead of `~ e`. Unary operator, see `Ocaml_operators[28.60]` for more information.

`val (+-) : int -> int`

Unary addition. You can also write `+ e` instead of `~+ e`. Unary operator, see `Ocaml_operators[28.60]` for more information.

**Since:** 3.12

`val succ : int -> int`

`succ x` is `x + 1`.

`val pred : int -> int`

`pred x` is `x - 1`.

`val (+) : int -> int -> int`

Integer addition. Left-associative operator, see `Ocaml_operators[28.60]` for more information.

`val (-) : int -> int -> int`

Integer subtraction. Left-associative operator, see `Ocaml_operators[28.60]` for more information.

`val (* ) : int -> int -> int`

Integer multiplication. Left-associative operator, see `Ocaml_operators[28.60]` for more information.

`val (/) : int -> int -> int`
```pascal
let count_span =
    Runtime_events.User.register "count.span" CustomSpan
    Runtime_events.Type.span
```

```pascal
let count_value =
    Runtime_events.User.register "count.value" CustomInt
    Runtime_events.Type.int
```

```pascal
let span_event_handler domain_id ts event value =
    (* we're only interested in our CustomSpan event *)
match Runtime_events.User.tag event, value with
| CustomSpan, Runtime_events.Type.Begin -> Printfprintf "> count begin\n"
| CustomSpan, End -> Printfprintf "< count end\n"
| _ -> ()
```

```pascal
let int_event_handler domain_id ts event value =
    (* we're only interested in our CustomInt event *)
match Runtime_events.User.tag event with
| CustomInt -> Printfprintf "| count %d\n" value
| _ -> ()
```

```pascal
let () =
    let open Runtime_events in
    let cursor = create_cursor None in
    let callbacks =
        Callbacks.create ()
    |> Callbacks.add_user_event Type.span span_event_handler
    |> Callbacks.add_user_event Type.int int_event_handler
    in
    for _ = 0 to 100 do
        ignore(read_poll cursor callbacks None)
    done
```

Giving the following output:

```
> count begin
| count 1
| count 2
| count 3
| count 4
| count 5
< count end
> count begin
| count 1
| count 2
```
Integer division. Integer division rounds the real quotient of its arguments towards zero. More precisely, if `x >= 0` and `y > 0`, `x / y` is the greatest integer less than or equal to the real quotient of `x` by `y`. Moreover, `(-x) / y = x / (-y) = -(-x) / y`. Left-associative operator, see `Ocaml_operators` [28.60] for more information.

**Raises** `Division_by_zero` if the second argument is 0.

`val (mod) : int -> int -> int`

Integer remainder. If `y` is not zero, the result of `x mod y` satisfies the following properties: `x = (x / y) * y + x mod y` and `abs(x mod y) <= abs(y) - 1`. If `y = 0`, `x mod y` raises `Division_by_zero`. Note that `x mod y` is negative only if `x < 0`. Left-associative operator, see `Ocaml_operators` [28.60] for more information.

**Raises** `Division_by_zero` if `y` is zero.

`val abs : int -> int`

`abs x` is the absolute value of `x`. On `min_int` this is `min_int` itself and thus remains negative.

`val max_int : int`

The greatest representable integer.

`val min_int : int`

The smallest representable integer.

## Bitwise operations

`val (land) : int -> int -> int`

Bitwise logical and. Left-associative operator, see `Ocaml_operators` [28.60] for more information.

`val (lor) : int -> int -> int`

Bitwise logical or. Left-associative operator, see `Ocaml_operators` [28.60] for more information.

`val (lxor) : int -> int -> int`

Bitwise logical exclusive or. Left-associative operator, see `Ocaml_operators` [28.60] for more information.

`val lnot : int -> int`

Bitwise logical negation.

`val (lsl) : int -> int -> int`

`n lsl m` shifts `n` to the left by `m` bits. The result is unspecified if `m < 0` or `m > Sys.int_size`. Right-associative operator, see `Ocaml_operators` [28.60] for more information.

`val (lsr) : int -> int -> int`
[...]
n lsr m shifts n to the right by m bits. This is a logical shift: zeroes are inserted regardless of the sign of n. The result is unspecified if m < 0 or m > Sys.int_size. Right-associative operator, see Ocaml_operators[28.60] for more information.

val (asr) : int -> int -> int

n asr m shifts n to the right by m bits. This is an arithmetic shift: the sign bit of n is replicated. The result is unspecified if m < 0 or m > Sys.int_size. Right-associative operator, see Ocaml_operators[28.60] for more information.

## Floating-point arithmetic

OCaml's floating-point numbers follow the IEEE 754 standard, using double precision (64 bits) numbers. Floating-point operations never raise an exception on overflow, underflow, division by zero, etc. Instead, special IEEE numbers are returned as appropriate, such as infinity for 1.0 / 0.0, neg_infinity for -1.0 / 0.0, and nan ('not a number') for 0.0 / 0.0. These special numbers then propagate through floating-point computations as expected: for instance, 1.0 / infinity is 0.0, basic arithmetic operations (+, -, *, /) with nan as an argument return nan, ...

val (~-.) : float -> float

Unary negation. You can also write -. e instead of ~-. e. Unary operator, see Ocaml_operators[28.60] for more information.

val (~+.) : float -> float

Unary addition. You can also write +. e instead of ~+. e. Unary operator, see Ocaml_operators[28.60] for more information.

Since: 3.12

val (+.) : float -> float -> float

Floating-point addition. Left-associative operator, see Ocaml_operators[28.60] for more information.

val (-.) : float -> float -> float

Floating-point subtraction. Left-associative operator, see Ocaml_operators[28.60] for more information.

val ( * . ) : float -> float -> float

Floating-point multiplication. Left-associative operator, see Ocaml_operators[28.60] for more information.

val ( / . ) : float -> float -> float

Floating-point division. Left-associative operator, see Ocaml_operators[28.60] for more information.

val ( ** ) : float -> float -> float
Chapter 26

The “Tail Modulo Constructor”
program transformation

(Introduced in OCaml 4.14)

Note: this feature is considered experimental, and its interface may evolve, with user feedback, in the next few releases of the language.

Consider this natural implementation of the `List.map` function:

``` Ocaml
let rec map f l =
match l with
| [] -> []
| x :: xs ->
  let y = f x in
  y :: map f xs
```

A well-known limitation of this implementation is that the recursive call, `map f xs`, is not in *tail* position. The runtime needs to remember to continue with `y :: r` after the call returns a value `r`, therefore this function consumes some amount of call-stack space on each recursive call. The stack usage of `map f li` is proportional to the length of `li`. This is a correctness issue for large lists on systems configured with limited stack space – the dreaded `Stack_overflow` exception.

``` OCaml
# let with_stack_limit stack_limit f =
  let old_gcSettings = Gc.get () in
  Gc.set { old_gcSettings with stack_limit };
  Fun.protect ~finally: (fun () -> Gc.set old_gcSettings) f
;;
val with_stack_limit : int -> (unit -> 'a) -> 'a = <fun>

# with_stack_limit 20_000 (fun () ->
  List.length (map Fun.id (List.init 1_000_000 Fun.id))
);;

Stack overflow during evaluation (looping recursion?).
```

In this implementation of `map`, the recursive call happens in a position that is not a *tail* position in the program, but within a datatype constructor application that is itself in *tail* position. We
Exponentiation. Right-associative operator, see `Ocaml_operators` [28.60] for more information.

`val sqrt : float -> float`
Square root.

`val exp : float -> float`
Exponential.

`val log : float -> float`
Natural logarithm.

`val log10 : float -> float`
Base 10 logarithm.

`val expm1 : float -> float`
`expm1 x` computes `exp x - 1.0`, giving numerically-accurate results even if `x` is close to 0.0.

Since: 3.12

`val log1p : float -> float`
`log1p x` computes `log(1.0 + x)` (natural logarithm), giving numerically-accurate results even if `x` is close to 0.0.

Since: 3.12

`val cos : float -> float`
Cosine. Argument is in radians.

`val sin : float -> float`
Sine. Argument is in radians.

`val tan : float -> float`
Tangent. Argument is in radians.

`val acos : float -> float`
Arc cosine. The argument must fall within the range [-1.0, 1.0]. Result is in radians and is between 0.0 and pi.

`val asin : float -> float`
Arc sine. The argument must fall within the range [-1.0, 1.0]. Result is in radians and is between -pi/2 and pi/2.

`val atan : float -> float`
Arc tangent. Result is in radians and is between -pi/2 and pi/2.
say that those positions, that are composed of tail positions and constructor applications, are *tail modulo constructor* (TMC) positions – we sometimes write `tail modulo cons` for brevity.

It is possible to rewrite programs such that tail modulo cons positions become tail positions; after this transformation, the implementation of `map` above becomes *tail-recursive*, in the sense that it only consumes a constant amount of stack space. The OCaml compiler implements this transformation on demand, using the `[@tail_mod_cons]` or `[@ocaml.tail_mod_cons]` attribute on the function to transform.

```ocaml
let[@tail_mod_cons] rec map f l =
match l with
| [] -> []
| x :: xs ->
    let y = f x in
    y :: map f xs

# List.length (map Fun.id (List.init 1_000_000 Fun.id));
- : int = 1000000
```

This transformation only improves calls in tail-modulo-cons position, it does not improve recursive calls that do not fit in this fragment:

(* does *not* work: addition is not a data constructor *)

```ocaml
let[@tail_mod_cons] rec length l =
match l with
| [] -> 0
| _ :: xs -> 1 + length xs

Warning 71 [unused-tmc-attribute]: This function is marked @tail_mod_cons
but is never applied in TMC position.
```

It is of course possible to use the `[@tail_mod_cons]` transformation on functions that contain some recursive calls in tail-modulo-cons position, and some calls in other, arbitrary positions. Only the tail calls and tail-modulo-cons calls will happen in constant stack space.

**General design** This feature is provided as an explicit program transformation, not an implicit optimization. It is annotation-driven: the user is expected to express their intent by adding annotations in the program using attributes, and will be asked to do so in any ambiguous situation.

We expect it to be used mostly by advanced OCaml users needing to get some guarantees on the stack-consumption behavior of their programs. Our recommendation is to use the `[@tailcall]` annotation on all callsites that should not consume any stack space. `[@tail_mod_cons]` extends the set of functions on which calls can be annotated to be tail calls, helping establish stack-consumption guarantees in more cases.

**Performance** A standard approach to get a tail-recursive version of `List.map` is to use an accumulator to collect output elements, and reverse it at the end of the traversal.

```ocaml
let rec map f l = map_aux f [] l
and map_aux f acc l =
match l with
```
### val atan2 : float -> float -> float

atan2 y x returns the arc tangent of y / x. The signs of x and y are used to determine the quadrant of the result. Result is in radians and is between -pi and pi.

### val hypot : float -> float -> float

hypot x y returns sqrt(x * x + y * y), that is, the length of the hypotenuse of a right-angled triangle with sides of length x and y, or, equivalently, the distance of the point (x,y) to origin. If one of x or y is infinite, returns infinity even if the other is nan.

**Since:** 4.00

### val cosh : float -> float

Hyperbolic cosine. Argument is in radians.

### val sinh : float -> float

Hyperbolic sine. Argument is in radians.

### val tanh : float -> float

Hyperbolic tangent. Argument is in radians.

### val acosh : float -> float

Hyperbolic arc cosine. The argument must fall within the range [1.0, inf]. Result is in radians and is between 0.0 and inf.

**Since:** 4.13

### val asinh : float -> float

Hyperbolic arc sine. The argument and result range over the entire real line. Result is in radians.

**Since:** 4.13

### val atanh : float -> float

Hyperbolic arc tangent. The argument must fall within the range [-1.0, 1.0]. Result is in radians and ranges over the entire real line.

**Since:** 4.13

### val ceil : float -> float

Round above to an integer value. ceil f returns the least integer value greater than or equal to f. The result is returned as a float.

### val floor : float -> float

Round below to an integer value. floor f returns the greatest integer value less than or equal to f. The result is returned as a float.

### val abs_float : float -> float
```hml
| [] -> List.rev acc
| x :: xs ->
  let y = f x in
  map_aux f (y :: acc) xs
```

This version is tail-recursive, but it is measurably slower than the simple, non-tail-recursive version. In contrast, the tail-mod-cons transformation provides an implementation that has comparable performance to the original version, even on small inputs.

**Evaluation order** Beware that the tail-modulo-cons transformation has an effect on evaluation order: the constructor argument that is transformed into tail-position will always be evaluated last. Consider the following example:

```hml
type 'a two_headed_list =
| Nil
| Consnoc of 'a * 'a two_headed_list * 'a
```

```hml
let[@tail_mod_cons] rec map f = function
| Nil -> Nil
| Consnoc (front, body, rear) ->
  Consnoc (f front, map f body, f rear)
```

Due to the `[@tail_mod_cons]` transformation, the calls to `f front` and `f rear` will be evaluated before `map f body`. In particular, this is likely to be different from the evaluation order of the unannotated version. (The evaluation order of constructor arguments is unspecified in OCaml, but many implementations typically use left-to-right or right-to-left.)

This effect on evaluation order is one of the reasons why the tail-modulo-cons transformation has to be explicitly requested by the user, instead of being applied as an automatic optimization.

**Why tail-modulo-cons?** Other program transformations, in particular a transformation to continuation-passing style (CPS), can make all functions tail-recursive, instead of targeting only a small fragment. Some reasons to provide builtin support for the less-general tail-mod-cons are as follows:

* The tail-mod-cons transformation preserves the performance of the original, non-tail-recursive version, while a continuation-passing-style transformation incurs a measurable constant-factor overhead.

* The tail-mod-cons transformation cannot be expressed as a source-to-source transformation of OCaml programs, as it relies on mutable state in type-unsafe ways. In contrast, continuation-passing-style versions can be written by hand, possibly using a convenient monadic notation.

**Note: OCaml call stack size** In OCaml 4.x and earlier, bytecode programs respect the `stack_limit` runtime parameter configuration (as set using `Gc.set` in the example above), or the `l` setting of the `OCAMLRUNPARAM` variable. Native programs ignore these settings and only respect the operating system native stack limit, as set by `ulimit` on Unix systems. Most operating systems run with a relatively low stack size limit by default, so stack overflow on non-tail-recursive functions are a common programming bug.
`abs_float` `f` returns the absolute value of `f`.

**val copysign : float -> float -> float**

`copysign` `x` `y` returns a `float` whose absolute value is that of `x` and whose sign is that of `y`. If `x` is `nan`, returns `nan`. If `y` is `nan`, returns either `x` or `-`. `x`, but it is not specified which.

**Since: 4.00**

**val mod_float : float -> float -> float**

`mod_float` `a` `b` returns the remainder of `a` with respect to `b`. The returned value is `a` `-`. `n` `-`. `b`, where `n` is the quotient `a` `/`. `b` rounded towards zero to an integer.

**val frexp : float -> float * int**

`frexp` `f` returns the pair of the significant and the exponent of `f`. When `f` is zero, the significant `x` and the exponent `n` of `f` are equal to zero. When `f` is non-zero, they are defined by `f = x * 2 ** n` and `0.5 <= x < 1.0`.

**val ldexp : float -> int -> float**

`ldexp` `x` `n` returns `x` `-`. `2 ** n`.

**val modf : float -> float * float**

`modf` `f` returns the pair of the fractional and integral part of `f`.

**val float : int -> float**

Same as `float_of_int` [27.2].

**val float_of_int : int -> float**

Convert an integer to floating-point.

**val truncate : float -> int**

Same as `int_of_float` [27.2].

**val int_of_float : float -> int**

Truncate the given floating-point number to an integer. The result is unspecified if the argument is `nan` or falls outside the range of representable integers.

**val infinity : float**

Positive infinity.

**val neg_infinity : float**

Negative infinity.

**val nan : float**
Starting from OCaml 5.0, native code does not use the native system stack for OCaml function
calls anymore, so it is not affected by the operating system native stack size; both native and
bytecode programs respect the OCaml runtime's own limit. The runtime limit is set to a much
higher default than most operating system native stacks, with a limit of at least 512MiB, so stack
overflow should be much less common in practice. There is still a stack limit by default, as it
remains useful to quickly catch bugs with looping non-tail-recursive functions. Without a stack
limit, one has to wait for the whole memory to be consumed by the stack for the program to crash,
which can take a long time and make the system unresponsive.

This means that the `tail modulo constructor` transformation is less important on OCaml 5:
it does improve performance noticeably in some cases, but it is not necessary for basic correctness
for most use-cases.

## 26.1 Disambiguation

It may happen that several arguments of a constructor are recursive calls to a tail-modulo-cons
function. The transformation can only turn one of these calls into a tail call. The compiler will not
make an implicit choice, but ask the user to provide an explicit disambiguation.

Consider this type of syntactic expressions (assuming some pre-existing type `var` of expression variables):

`type var (* some pre-existing type of variables *)`

``` OCaml
type exp =
    | Var of var
    | Let of binding * exp
and binding = var * exp
```

Consider a `map` function on variables. The direct definition has two recursive calls inside
arguments of the `Let` constructor, so it gets rejected as ambiguous.

``` OCaml
let[@tail_mod_cons] rec map_vars f exp =
    match exp with
    | Var v -> Var (f v)
    | Let ((v, def), body) ->
        Let ((f v, map_vars f def), map_vars f body)
```

```
Error: [@tail_mod_cons]: this constructor application may be TMC-transformed
       in several different ways. Please disambiguate by adding an explicit
       [@tailcall] attribute to the call that should be made tail-recursive,
       or a [@tailcall false] attribute on calls that should not be
       transformed.

This call could be annotated.
This call could be annotated.
```

To disambiguate, the user should add a `@tailcall` attribute to the recursive call that should
be transformed to tail position:

``` OCaml
let[@tail_mod_cons] rec map_vars f exp =
    match exp with
```
A special floating-point value denoting the result of an undefined operation such as 0.0 /.
0.0. Stands for 'not a number'. Any floating-point operation with nan as argument returns
nan as result, unless otherwise specified in IEEE 754 standard. As for floating-point
comparisons, =, <, <=, > and >= return false and <> returns true if one or both of their
arguments is nan.

nan is a quiet NaN since 5.1; it was a signaling NaN before.

val max_float : float

The largest positive finite value of type `float`.

val min_float : float

The smallest positive, non-zero, non-denormalized value of type `float`.

val epsilon_float : float

The difference between 1.0 and the smallest exactly representable floating-point number greater than 1.0.

type fpclass =

| FP_normal
Normal number, none of the below

| FP_subnormal
Number very close to 0.0, has reduced precision

| FP_zero
Number is 0.0 or -0.0

| FP_infinite
Number is positive or negative infinity

| FP_nan
Not a number: result of an undefined operation

The five classes of floating-point numbers, as determined by the `classify_float` [27.2]
function.

val classify_float : float -> fpclass

Return the class of the given floating-point number: normal, subnormal, zero, infinite, or not a number.

## String operations

More string operations are provided in module `String` [27.2].

val (`~`) : string -> string -> string

String concatenation. Right-associative operator, see `Ocaml_operators` [28.60] for more
information.

**Raises** `Invalid_argument` if the result is longer than `Sys.max_string_length` [28.55] bytes.
```hkl
| Var v -> Var (f v)
| Let ((v, def), body) ->
  Let ((f v, map_vars f def), (map_vars[@tailcall]) f body)
```

Be aware that the resulting function is *not* tail-recursive, the recursive call on `def` will consume stack space. However, expression trees tend to be right-leaning (lots of `Let` in sequence, rather than nested inside each other), so putting the call on `body` in tail position is an interesting improvement over the naive definition: it gives bounded stack space consumption if we assume a bound on the nesting depth of `Let` constructs.

One would also get an error when using conflicting annotations, asking for two of the constructor arguments to be put in tail position:

```hkl
let[@tail_mod_cons] rec map_vars f exp =
  match exp with
  | Var v -> Var (f v)
  | Let ((v, def), body) ->
    Let ((f v, (map_vars[@tailcall]) f def), (map_vars[@tailcall]) f body)
```

**Error:** `[@tail_mod_cons]: this constructor application may be TMC-transformed in several different ways. Only one of the arguments may become a TMC call, but several arguments contain calls that are explicitly marked as tail-recursive. Please fix the conflict by reviewing and fixing the conflicting annotations.`

*This call is explicitly annotated.*

*This call is explicitly annotated.*

## 26.2 Danger: getting out of tail-mod-cons

Due to the nature of the tail-mod-cons transformation (see Section 26.3 for a presentation of transformation):

* Calls from a tail-mod-cons function to another tail-mod-cons function declared in the same recursive-binding group are transformed into tail calls, as soon as they occur in tail position or tail-modulo-cons position in the source function.

* Calls from a function *not* annotated tail-mod-cons to a tail-mod-cons function or, conversely, from a tail-mod-cons function to a non-tail-mod-cons function are transformed into *non-tail* calls, even if they syntactically appear in tail position in the source program.

The fact that calls in tail position in the source program may become non-tail calls if they go from a tail-mod-cons to a non-tail-mod-cons function is surprising, and the transformation will warn about them.

For example:

```hkl
let[@tail_mod_cons] rec flatten = function
| [] -> []
| xs :: xss ->
  let rec append_flatten xs xss =
    match xs with
```
## Character operations

More character operations are provided in module `Char` [27.2].

`val int_of_char : char -> int`

Return the ASCII code of the argument.

`val char_of_int : int -> char`

Return the character with the given ASCII code.

**Raises** `Invalid_argument` if the argument is outside the range 0-255.

## Unit operations

`val ignore : 'a -> unit`

Discard the value of its argument and return `()`. For instance, `ignore(f x)` discards the result of the side-effecting function `f`. It is equivalent to `f x; ()`, except that the latter may generate a compiler warning; writing `ignore(f x)` instead avoids the warning.

## String conversion functions

`val string_of_bool : bool -> string`

Return the string representation of a boolean. As the returned values may be shared, the user should not modify them directly.

`val bool_of_string_opt : string -> bool option`

Convert the given string to a boolean.

Return `None` if the string is not `"true"` or `"false"`.

**Since:** 4.05

`val bool_of_string : string -> bool`

Same as `bool_of_string_opt` [27.2], but raise `Invalid_argument` `"bool_of_string"` instead of returning `None`.

`val string_of_int : int -> string`

Return the string representation of an integer, in decimal.

`val int_of_string_opt : string -> int option`

Convert the given string to an integer. The string is read in decimal (by default, or if the string begins with `0u`), in hexadecimal (if it begins with `0x` or `0X`), in octal (if it begins with `0o` or `0O`), or in binary (if it begins with `0b` or `0B`).

The `0u` prefix reads the input as an unsigned integer in the range `[0, 2*max_int+1]`. If the input exceeds `max_int` [27.2] it is converted to the signed integer `min_int + input - max_int - 1`.
```
| [] -> flatten xss
---
| x :: xs -> x :: append_flatten xs xss
---
in append_flatten xs xss
```

Warning 71 [unused-tmc-attribute]: This function is marked @tail_mod_cons but is never applied in TMC position.

Warning 72 [tmc-breaks-tailcall]: This call is in tail-modulo-cons position in a TMC function, but the function called is not itself specialized for TMC, so the call will not be transformed into a tail call.
Please either mark the called function with the [@tail_mod_cons] attribute, or mark this call with the [@tailcall false] attribute to make its non-tailness explicit.

Here the `append_flatten` helper is not annotated with `@tail_mod_cons], so the calls `append_flatten xs xss` and `flatten xss` will not be tail calls. The correct fix here is to annotate `append_flatten` to be tail-mod-cons.

```hl60
let[@tail_mod_cons] rec flatten = function
| [] -> []
| xs :: xss ->
  let[@tail_mod_cons] rec append_flatten xs xss =
    match xs with
    | [] -> flatten xss
    | x :: xs -> x :: append_flatten xs xss
  in append_flatten xs xss
```

The same warning occurs when `append_flatten` is a non-tail-mod-cons function of the same recursive group; using the tail-mod-cons transformation is a property of individual functions, not whole recursive groups.

```hhl60
let[@tail_mod_cons] rec flatten = function
| [] -> []
| xs :: xss -> append_flatten xs xss
```

```hhl60
and append_flatten xs xss =
  match xs with
  | [] -> flatten xss
  | x :: xs -> x :: append_flatten xs xss
```

Warning 71 [unused-tmc-attribute]: This function is marked @tail_mod_cons but is never applied in TMC position.

Warning 72 [tmc-breaks-tailcall]: This call is in tail-modulo-cons position in a TMC function, but the function called is not itself specialized for TMC, so the call will not be transformed into a tail call.
Please either mark the called function with the [@tail_mod_cons] attribute, or mark this call with the [@tailcall false] attribute to make its non-tailness explicit.

Again, the fix is to specialize `append_flatten` as well:
The `_` (underscore) character can appear anywhere in the string and is ignored.

Return `None` if the given string is not a valid representation of an integer, or if the integer represented exceeds the range of integers representable in type `int`.

**Since:** 4.05

**val int_of_string : string -> int**

Same as `int_of_string_opt[27.2]`, but raise `Failure "int_of_string"` instead of returning `None`.

**val string_of_float : float -> string**

Return a string representation of a floating-point number.

This conversion can involve a loss of precision. For greater control over the manner in which the number is printed, see `f`[27.2].

**val float_of_string_opt : string -> float option**

Convert the given string to a float. The string is read in decimal (by default) or in hexadecimal (marked by `0x` or `0X`).

The format of decimal floating-point numbers is `[-]ddddd (e|E) [+-]dd`, where `d` stands for a decimal digit.

The format of hexadecimal floating-point numbers is `[-]0(x|X) hhhhh (p|P) [+-]dd`, where `h` stands for an hexadecimal digit and `d` for a decimal digit.

In both cases, at least one of the integer and fractional parts must be given; the exponent part is optional.

The `_` (underscore) character can appear anywhere in the string and is ignored.

Depending on the execution platforms, other representations of floating-point numbers can be accepted, but should not be relied upon.

Return `None` if the given string is not a valid representation of a float.

**Since:** 4.05

**val float_of_string : string -> float**

Same as `float_of_string_opt[27.2]`, but raise `Failure "float_of_string"` instead of returning `None`.

## Pair operations

**val fst : 'a * 'b -> 'a**

Return the first component of a pair.

**val snd : 'a * 'b -> 'b**

Return the second component of a pair.
let[@tail_mod_cons] rec flatten = function
| [] -> []
| xs :: xss -> append_flatten xs xss

and[@tail_mod_cons] append_flatten xs xss =
match xs with
| [] -> flatten xss
| x :: xs -> x :: append_flatten xs xss

Non-recursive functions can also be annotated [@tail_mod_cons]; this is typically useful for
local bindings to recursive functions.

Incorrect version:

let[@tail_mod_cons] rec map_vars f exp =
let self exp = map_vars f exp in
match exp with
| Var v -> Var (f v)
| Let ((v, def), body) ->
Let ((f v, self def), (self[@tailcall]) body)

Warning 51 [wrong-tailcall-expectation]: expected tailcall

Warning 51 [wrong-tailcall-expectation]: expected tailcall

Warning 71 [unused-tmc-attribute]: This function is marked @tail_mod_cons
but is never applied in TMC position.

Recommended fix:

let[@tail_mod_cons] rec map_vars f exp =
let[@tail_mod_cons] self exp = map_vars f exp in
match exp with
| Var v -> Var (f v)
| Let ((v, def), body) ->
Let ((f v, self def), (self[@tailcall]) body)

In other cases, there is either no benefit in making the called function tail-mod-cons, or it is not
possible: for example, it is a function parameter (the transformation only works with direct calls to
known functions).

For example, consider a substitution function on binary trees:

type 'a tree = Leaf of 'a | Node of 'a tree * 'a tree

let[@tail_mod_cons] rec bind (f : 'a -> 'a tree) (t : 'a tree) : 'a tree =
match t with
| Leaf v -> f v
| Node (left, right) ->
  Node (bind f left, (bind[@tailcall]) f right)

Warning 72 [tmc-breaks-tailcall]: This call is in tail-modulo-cons position
in a TMC function, but the function called is not itself specialized for
## List operations

More list operations are provided in module `List` [27.2].

`val (@) : 'a list -> 'a list -> 'a list`

10 @ 11 appends 11 to 10. Same function as `List.append` [28.31]. Right-associative operator, see `Ocaml_operators` [28.60] for more information.

**Since:** 5.1 this function is tail-recursive.

## Input/output

Note: all input/output functions can raise `Sys_error` when the system calls they invoke fail.

`type in_channel`

The type of input channel.

`type out_channel`

The type of output channel.

`val stdin : in_channel`

The standard input for the process.

`val stdout : out_channel`

The standard output for the process.

`val stderr : out_channel`

The standard error output for the process.

## Output functions on standard output

`val print_char : char -> unit`

Print a character on standard output.

`val print_string : string -> unit`

Print a string on standard output.

`val print_bytes : bytes -> unit`

Print a byte sequence on standard output.

**Since:** 4.02

`val print_int : int -> unit`

Print an integer, in decimal, on standard output.

`val print_float : float -> unit`
TMC, so the call will not be transformed into a tail call.
Please either mark the called function with the [@tail_mod_cons] attribute,
or mark this call with the [@tailcall false] attribute to make its
non-tailness explicit.

Here f is a function parameter, not a direct call, and the current implementation is strictly first-order, it does not support tail-mod-cons arguments. In this case, the user should indicate that they realize this call to f v is not, in fact, in tail position, by using (f[@tailcall false]) v.

type 'a tree = Leaf of 'a | Node of 'a tree * 'a tree

let[@tail_mod_cons] rec bind (f : 'a -> 'a tree) (t : 'a tree) : 'a tree =
match t with
| Leaf v -> (f[@tailcall false]) v
| Node (left, right) ->
  Node (bind f left, (bind[@tailcall]) f right)

26.3 Details on the transformation

To use this advanced feature, it helps to be aware that the function transformation produces a
specialized function in destination-passing-style.

Recall our *map* example:

let rec map f l =
  match l with
  | [] -> []
  | x :: xs ->
     let y = f x in
     y :: map f xs

Below is a description of the transformed program in pseudo-OCaml notation: some operations
are not expressible in OCaml source code. (The transformation in fact happens on the Lambda
intermediate representation of the OCaml compiler.)

let rec map f l =
  match l with
  | [] -> []
  | x :: xs ->
     let y = f x in
     let dst = y ::{mutable} Hole in
     map_dps f xs dst 1;
     dst

and map_dps f l dst idx =
match l with
| [] -> dst_idx <- []
| x :: xs ->
let y = f x in
Print a floating-point number, in decimal, on standard output.

The conversion of the number to a string uses `string_of_float[27.2]` and can involve a loss of precision.

`val print_endline : string -> unit`

Print a string, followed by a newline character, on standard output and flush standard output.

`val print_newline : unit -> unit`

Print a newline character on standard output, and flush standard output. This can be used to simulate line buffering of standard output.

## Output functions on standard error

`val prerr_char : char -> unit`

Print a character on standard error.

`val prerr_string : string -> unit`

Print a string on standard error.

`val prerr_bytes : bytes -> unit`

Print a byte sequence on standard error.

**Since: 4.02**

`val prerr_int : int -> unit`

Print an integer, in decimal, on standard error.

`val prerr_float : float -> unit`

Print a floating-point number, in decimal, on standard error.

The conversion of the number to a string uses `string_of_float[27.2]` and can involve a loss of precision.

`val prerr_endline : string -> unit`

Print a string, followed by a newline character on standard error and flush standard error.

`val prerr_newline : unit -> unit`

Print a newline character on standard error, and flush standard error.
```hml
let dst' = y ::{mutable} Hole in
  dst_idx <- dst';
  map_dps f xs dst' 1
```

The source version of `map` gets transformed into two functions, a *direct-style* version that is also called `map`, and a *destination-passing-style* version (DPS) called `map_dps`. The destination-passing-style version does not return a result directly, instead it writes it into a memory location specified by two additional function parameters, `dst` (a memory block) and `i` (a position within the memory block).

The source call `y :: map f xs` gets transformed into the creation of a mutable block `y ::{mutable} Hole`, whose second parameter is an uninitialized `hole`. The block is then passed to `map_dps` as a destination parameter (with offset 1).

Notice that `map` does not call itself recursively, it calls `map_dps`. Then, `map_dps` calls itself recursively, in a tail-recursive way.

The call from `map` to `map_dps` is *not* a tail call (this is something that we could improve in the future); but this call happens only once when invoking `map f l`, with all list elements after the first one processed in constant stack by `map_dps`.

This explains the “getting out of tail-mod-cons” subtleties. Consider our previous example involving mutual recursion between `flatten` and `append_flatten`.

```hml
let[@tail_mod_cons] rec flatten l =
  match l with
  | [] -> []
  | xs :: xss ->
    append_flatten xs xss
```

The call to `append_flatten`, which syntactically appears in tail position, gets transformed differently depending on whether the function has a destination-passing-style version available, that is, whether it is itself annotated `[@tail_mod_cons]`:

```hml
(* if append_flatten_dps exists *)
and flatten_dps l dst i =
  match l with
  | [] -> dst.i <- []
  | xs :: xss ->
    append_flatten_dps xs xss dst i
```

```hml
(* if append_flatten_dps does not exist *)
and rec flatten_dps l dst i =
  match l with
  | [] -> dst.i <- []
  | xs :: xss ->
    dst.i <- append_flatten xs xss
```

If `append_flatten` does not have a destination-passing-style version, the call gets transformed to a non-tail call.
## Input functions on standard input

**val read_line: unit -> string**

Flush standard output, then read characters from standard input until a newline character is encountered.

Return the string of all characters read, without the newline character at the end.

**Raises** `End_of_file` if the end of the file is reached at the beginning of line.

**val read_int_opt: unit -> int option**

Flush standard output, then read one line from standard input and convert it to an integer.

Return `None` if the line read is not a valid representation of an integer.

**Since:** 4.05

**val read_int: unit -> int**

Same as `read_int_opt[27.2]`, but raise `Failure "int_of_string"` instead of returning `None`.

**val read_float_opt: unit -> float option**

Flush standard output, then read one line from standard input and convert it to a floating-point number.

Return `None` if the line read is not a valid representation of a floating-point number.

**Since:** 4.05

**val read_float: unit -> float**

Same as `read_float_opt[27.2]`, but raise `Failure "float_of_string"` instead of returning `None`.

## General output functions

**type open_flag =**

| Open_rdry
open for reading.

| Open_wronly
open for writing.

| Open_append
open for appending: always write at end of file.

| Open_creat
create the file if it does not exist.

| Open_trunc
empty the file if it already exists.
## 26.4 Current limitations

**Purely syntactic criterion** Just like tail calls in general, the notion of tail-modulo-constructor position is purely syntactic; some simple refactoring will move calls out of tail-modulo-constructor position.

(* works as expected *)

```hkl
let[@tail_mod_cons] rec map f li =
    match li with
    | [] -> []
    | x :: xs ->
        let y = f x in
        y ::
            (* this call is in TMC position *)
            map f xs
```

(* not optimizable anymore *)

```hkl
let[@tail_mod_cons] rec map f li =
    match li with
    | [] -> []
    | x :: xs ->
        let y = f x in
        let ys =
            (* this call is not in TMC position anymore *)
            map f xs in
        y :: ys
```

Warning 71 [unused-tmc-attribute]: This function is marked @tail_mod_cons but is never applied in TMC position.

**Local, first-order transformation** When a function gets transformed with tail-mod-cons, two definitions are generated, one providing a direct-style interface and one providing the destination-passing-style version. However, not all calls to this function in tail-modulo-cons position will use the destination-passing-style version and become tail calls:

* The transformation is local: only tail-mod-cons calls to `foo` within the same compilation unit as `foo` become tail calls.

* The transformation is first-order: only direct calls to known tail-mod-cons functions become tail calls when in tail-mod-cons position, never calls to function parameters.

Consider the call `Option.map foo x` for example: even if `foo` is called in tail-mod-cons position within the definition of `Option.map`, that call will never become a tail call. (This would be the case even if the call to `Option.map` was inside the `Option` module.)

In general this limitation is not a problem for recursive functions: the first call from an outside module or a higher-order function will consume stack space, but further recursive calls in tail-mod-cons position will get optimized. For example, if `List.map` is defined as a tail-mod-cons function, calls from outside the `List` module will not become tail calls when in tail positions, but the recursive
| Open_excl

fail if Open_creates and the file already exists.

| Open_binary

open in binary mode (no conversion).

| Open_text

open in text mode (may perform conversions).

| Open_nonblock

open in non-blocking mode.

Opening modes for `open_out_gen[27.2]` and `open_in_gen[27.2]`.

val open_out : string -> out_channel

Open the named file for writing, and return a new output channel on that file, positioned at the beginning of the file. The file is truncated to zero length if it already exists. It is created if it does not already exists.

val open_out_bin : string -> out_channel

Same as `open_out[27.2]`, but the file is opened in binary mode, so that no translation takes place during writes. On operating systems that do not distinguish between text mode and binary mode, this function behaves like `open_out[27.2]`.

val open_out_gen : open_flag list -> int -> string -> out_channel

`open_out_gen` mode `perm filename` opens the named file for writing, as described above. The extra argument `mode` specifies the opening mode. The extra argument `perm` specifies the file permissions, in case the file must be created. `open_out[27.2]` and `open_out_bin[27.2]` are special cases of this function.

val flush : out_channel -> unit

Flush the buffer associated with the given output channel, performing all pending writes on that channel. Interactive programs must be careful about flushing standard output and standard error at the right time.

val flush_all : unit -> unit

Flush all open output channels; ignore errors.

val output_char : out_channel -> char -> unit

Write the character on the given output channel.

val output_string : out_channel -> string -> unit

Write the string on the given output channel.

val output_bytes : out_channel -> bytes -> unit
calls within the definition of `List.map` are in tail-modulo-cons positions and do become tail calls: processing the first element of the list will consume stack space, but all further elements are handled in constant space.

These limitations may be an issue in more complex situations where mutual recursion happens between functions, with some functions not annotated tail-mod-cons, or defined across different modules, or called indirectly, for example through function parameters.

**Non-exact calls to tupled functions** OCaml performs an implicit optimization for “tupled” functions, which take a single parameter that is a tuple: `let f (x, y, z) = ...`. Direct calls to these functions with a tuple literal argument (like `f (a, b, c)`) will call the “tupled” function by passing the parameters directly, instead of building a tuple of them. Other calls, either indirect calls or calls passing a more complex tuple value (like `let t = (a, b, c) in f t`) are compiled as “inexact” calls that go through a wrapper.

The `[@tail_mod_cons]` transformation supports tupled functions, but will only optimize “exact” calls in tail position; direct calls to something other than a tuple literal will not become tail calls. The user can manually unpack a tuple to force a call to be “exact”: `let (x, y, z) = t in f (x, y, z)`. If there is any doubt as to whether a call can be tail-mod-cons-optimized or not, one can use the `[@tailcall]` attribute on the called function, which will warn if the transformation is not possible.

```ocaml
let rec map (f, l) =
  match l with
  | [] -> []
  | x :: xs ->
     let y = f x in
     let args = (f, xs) in
       (* this inexact call cannot be tail-optimized, so a warning will be raised *)
     y :: (map[@tailcall]) args
```

Warning 51 [wrong-tailcall-expectation]: expected tailcall
Write the byte sequence on the given output channel.

**Since: 4.02**

`val output : out_channel -> bytes -> int -> int -> unit`

`output oc buf pos len` writes `len` characters from byte sequence `buf`, starting at offset `pos`, to the given output channel `oc`.

**Raises** `Invalid_argument` if `pos` and `len` do not designate a valid range of `buf`.

`val output_substring : out_channel -> string -> int -> int -> unit`

Same as `output` but take a string as argument instead of a byte sequence.

**Since: 4.02**

`val output_byte : out_channel -> int -> unit`

Write one 8-bit integer (as the single character with that code) on the given output channel. The given integer is taken modulo 256.

`val output_binary_int : out_channel -> int -> unit`

Write one integer in binary format (4 bytes, big-endian) on the given output channel. The given integer is taken modulo $2^{32}$. The only reliable way to read it back is through the `input_binary_int[27.2]` function. The format is compatible across all machines for a given version of OCaml.

`val output_value : out_channel -> 'a -> unit`

Write the representation of a structured value of any type to a channel. Circularities and sharing inside the value are detected and preserved. The object can be read back, by the function `input_value[27.2]`. See the description of module ` Marshal[27.2]` for more information. `output_value[27.2]` is equivalent to ` Marshal.to_channel[28.34] ` with an empty list of flags.

`val seek_out : out_channel -> int -> unit`

`seek_out chan pos` sets the current writing position to `pos` for channel `chan`. This works only for regular files. On files of other kinds (such as terminals, pipes and sockets), the behavior is unspecified.

`val pos_out : out_channel -> int`

Return the current writing position for the given channel. Does not work on channels opened with the `Open_append` flag (returns unspecified results). For files opened in text mode under Windows, the returned position is approximate (owing to end-of-line conversion); in particular, saving the current position with `pos_out`, then going back to this position using `seek_out` will not work. For this programming idiom to work reliably and portably, the file must be opened in binary mode.

`val out_channel_length : out_channel -> int`
Return the size (number of characters) of the regular file on which the given channel is opened. If the channel is opened on a file that is not a regular file, the result is meaningless.

val `close_out`: `out_channel` -> `unit`

Close the given channel, flushing all buffered write operations. Output functions raise a `Sys_error` exception when they are applied to a closed output channel, except `close_out` and `flush`, which do nothing when applied to an already closed channel. Note that `close_out` may raise `Sys_error` if the operating system signals an error when flushing or closing.

val `close_out_noerr`: `out_channel` -> `unit`

Same as `close_out`, but ignore all errors.

val `set_binary_mode_out`: `out_channel` -> `bool` -> `unit`

`set_binary_mode_out` oc true sets the channel oc to binary mode: no translations take place during output. `set_binary_mode_out` oc false sets the channel oc to text mode: depending on the operating system, some translations may take place during output. For instance, under Windows, end-of-lines will be translated from \n to \r\n. This function has no effect under operating systems that do not distinguish between text mode and binary mode.

## General input functions

val `open_in`: `string` -> `in_channel`

Open the named file for reading, and return a new input channel on that file, positioned at the beginning of the file.

val `open_in_bin`: `string` -> `in_channel`

Same as `open_in`[27.2], but the file is opened in binary mode, so that no translation takes place during reads. On operating systems that do not distinguish between text mode and binary mode, this function behaves like `open_in`[27.2].

val `open_in_gen`: `open_flag` list -> `int` -> `string` -> `in_channel`

`open_in_gen` mode perm filename opens the named file for reading, as described above. The extra arguments mode and perm specify the opening mode and file permissions. `open_in`[27.2] and `open_in_bin`[27.2] are special cases of this function.

val `input_char`: `in_channel` -> `char`

Read one character from the given input channel.

Raises `End_of_file` if there are no more characters to read.

val `input_line`: `in_channel` -> `string`
# Chapter 27

## Runtime detection of data races with ThreadSanitizer

### 27.1 Overview and usage

OCaml, since version 5.0, allows shared-memory parallelism and thus mutation of data shared between multiple threads. This creates the possibility of data races, i.e., unordered accesses to the same memory location with at least one of them being a write. In OCaml, data races are easy to introduce, and the behaviour of programs with data races can be unintuitive — the observed behaviours cannot be explained by simply interleaving operations from different concurrent threads. More information about data races and their consequences can be found in section 9.4 and Chapter 10.

To help detect data races, OCaml supports ThreadSanitizer (TSan), a dynamic data race detector that has been successfully used in languages such as C/C++, Swift, etc. TSan support for OCaml is available since OCaml 5.2.

To use TSan, you must configure the compiler with `--enable-tsan`. You can also install an `opam` switch with the TSan feature enabled as follows:

```
opam switch create <YOUR-SWITCH-NAME-HERE> ocaml-option-tsan
```

TSan support for OCaml is currently available for the x86_64 architecture, on FreeBSD, Linux and macOS, for the arm64 architecture on Linux and macOS, and for the POWER, riscv and s390x architectures on Linux.

Building OCaml with TSan support requires GCC or Clang. Minimal supported versions are GCC 11 and Clang 14. Note that TSan data race reports with GCC 11 are known to result in poor stack trace reporting (no line numbers), which is fixed in GCC 12.

A TSan-enabled compiler differs from a regular compiler in the following way: all programs compiled by `ocamlopt` are instrumented with calls to the TSan runtime, and TSan will detect data races encountered during execution.

For instance, consider the following program:

```ocaml
let a = ref 0 and b = ref 0

let d1 () =
```
Read characters from the given input channel, until a newline character is encountered.

Return the string of all characters read, without the newline character at the end.

**Raises** `End_of_file` if the end of the file is reached at the beginning of line.

`val input : in_channel -> bytes -> int -> int -> int`

`input ic buf pos len` reads up to `len` characters from the given channel `ic`, storing them in byte sequence `buf`, starting at character number `pos`. It returns the actual number of characters read, between 0 and `len` (inclusive). A return value of 0 means that the end of file was reached. A return value between 0 and `len` exclusive means that not all requested `len` characters were read, either because no more characters were available at that time, or because the implementation found it convenient to do a partial read; `input` must be called again to read the remaining characters, if desired. (See also `really_input` [27.2] for reading exactly `len` characters.) Exception `Invalid_argument "input"` is raised if `pos` and `len` do not designate a valid range of `buf`.

`val really_input : in_channel -> bytes -> int -> int -> unit`

`really_input ic buf pos len` reads `len` characters from channel `ic`, storing them in byte sequence `buf`, starting at character number `pos`.

**Raises**

* `End_of_file` if the end of file is reached before `len` characters have been read.

* `Invalid_argument` if `pos` and `len` do not designate a valid range of `buf`.

`val really_input_string : in_channel -> int -> string`

`really_input_string ic len` reads `len` characters from channel `ic` and returns them in a new string.

**Since:** 4.02

**Raises** `End_of_file` if the end of file is reached before `len` characters have been read.

`val input_byte : in_channel -> int`

Same as `input_char` [27.2], but return the 8-bit integer representing the character.

**Raises** `End_of_file` if the end of file was reached.

`val input_binary_int : in_channel -> int`

Read an integer encoded in binary format (4 bytes, big-endian) from the given input channel.
See `output_binary_int` [27.2].

**Raises** `End_of_file` if the end of file was reached while reading the integer.

`val input_value : in_channel -> 'a`

Read the representation of a structured value, as produced by `output_value` [27.2], and
return the corresponding value. This function is identical to ` Marshal.from_channel` [28.34];
see the description of module ` Marshal` [27.2] for more information, in particular concerning
the lack of type safety.
a := 1;

!b

let d2 () =

b := 1;

!a

let () =

let h = Domain spawn d2 in

let r1 = d1 () in

let r2 = Domain.join h in

assert (not (r1 = 0 && r2 = 0))

This program has data races. The memory locations a and b are read and written concurrently
by multiple domains d1 and d2. a and b are "non-atomic" locations according to the memory model
(see Chapter 10), and there is no synchronization between accesses to them. Hence, there are two
data races here corresponding to the two memory locations a and b.

When you compile and run this program with ocamlopt, you may observe data race reports on
the standard error, such as:

====================

WARNING: ThreadSanitizer: data race (pid=3808831)

Write of size 8 at 0x8febe0 by thread T1 (mutexes: write M90):

#0 camlSimpleRace.d2_274 simpleRace.ml:8 (simpleRace.exe+0x420a72)

#1 camlDomain.body_706 stdlib/domain.ml:211 (simpleRace.exe+0x440f2f)

#2 caml_start_program <null> (simpleRace.exe+0x47cf37)

#3 caml_callback_exn runtime/callback.c:197 (simpleRace.exe+0x445f7b)

#4 domain_thread_func runtime/domain.c:1167 (simpleRace.exe+0x44a113)

Previous read of size 8 at 0x8febe0 by main thread (mutexes: write M86):

#0 camlSimpleRace.d1_271 simpleRace.ml:5 (simpleRace.exe+0x420a22)

#1 camlSimpleRace.entry simpleRace.ml:13 (simpleRace.exe+0x420d16)

#2 caml_program <null> (simpleRace.exe+0x41ffb9)

#3 caml_start_program <null> (simpleRace.exe+0x47cf37)

[...]

WARNING: ThreadSanitizer: data race (pid=3808831)

Read of size 8 at 0x8febf0 by thread T1 (mutexes: write M90):

#0 camlSimpleRace.d2_274 simpleRace.ml:9 (simpleRace.exe+0x420a92)

#1 camlDomain.body_706 stdlib/domain.ml:211 (simpleRace.exe+0x440f2f)

#2 caml_start_program <null> (simpleRace.exe+0x47cf37)

#3 caml_callback_exn runtime/callback.c:197 (simpleRace.exe+0x445f7b)

#4 domain_thread_func runtime/domain.c:1167 (simpleRace.exe+0x44a113)

Previous write of size 8 at 0x8febf0 by main thread (mutexes: write M86):

#0 camlSimpleRace.d1_271 simpleRace.ml:4 (simpleRace.exe+0x420a01)
val seek_in : in_channel -> int -> unit

seek_in chan pos sets the current reading position to pos for channel chan. This works
only for regular files. On files of other kinds, the behavior is unspecified.

val pos_in : in_channel -> int

Return the current reading position for the given channel. For files opened in text mode under Windows, the returned position is approximate (owing to end-of-line conversion); in particular, saving the current position with `pos_in`, then going back to this position using `seek_in` will not work. For this programming idiom to work reliably and portably, the file must be opened in binary mode.

val in_channel_length : in_channel -> int

Return the size (number of characters) of the regular file on which the given channel is opened. If the channel is opened on a file that is not a regular file, the result is meaningless. The returned size does not take into account the end-of-line translations that can be performed when reading from a channel opened in text mode.

val close_in : in_channel -> unit

Close the given channel. Input functions raise a `Sys_error` exception when they are applied to a closed input channel, except `close_in`, which does nothing when applied to an already closed channel.

val close_in_noerr : in_channel -> unit

Same as `close_in`, but ignore all errors.

val set_binary_mode_in : in_channel -> bool -> unit

`set_binary_mode_in` `ic` true sets the channel `ic` to binary mode: no translations take place during input. `set_binary_mode_out` `ic` false sets the channel `ic` to text mode: depending on the operating system, some translations may take place during input. For instance, under Windows, end-of-lines will be translated from `\r\n` to `\n`. This function has no effect under operating systems that do not distinguish between text mode and binary mode.

## Operations on large files

module LargeFile :

sig

val seek_out : out_channel -> int64 -> unit

val pos_out : out_channel -> int64

val out_channel_length : out_channel -> int64

val seek_in : in_channel -> int64 -> unit

val pos_in : in_channel -> int64

val in_channel_length : in_channel -> int64
#1 camlSimpleRace.entry simpleRace.ml:13 (simpleRace.exe+0x420d16)
#2 caml_program <null> (simpleRace.exe+0x41ffb9)
#3 caml_start_program <null> (simpleRace.exe+0x47cf37)
[...]

====================

ThreadSanitizer: reported 2 warnings

For each detected data race, TSan reports the location of the conflicting accesses, their nature (read, write, atomic read, etc.), and the associated stack trace.

If you run the above program several times, the output may vary: sometimes TSan will report two data races, sometimes one, and sometimes none. This is due to the combination of two factors:

* First, TSan reports only the data races encountered during execution, i.e., conflicting, unordered memory accesses that are effectively performed.

* In addition, in this program, depending on executions, there may be no such memory accesses: if `d1` returns before `d2` has finished spawning, then all memory accesses originating from `d1` may happen-before the ones originating from `d2`, since spawning a domain involves inter-thread synchronization. In that case, the accesses are considered to be ordered and no data race is reported.

This example illustrates the fact that data races can sometimes be hidden by unrelated synchronizing operations.

## 27.2 Performance implications

TSan instrumentation imposes a non-negligible cost at runtime. Empirically, this cost has been observed to cause a slowdown which can range from 2x to 7x. One of the main factors of high slowdowns is frequent access to mutable data. In contrast, the initialising writes to and reads from immutable memory locations are not instrumented. TSan also allocates very large amounts of virtual memory, although it uses only a fraction of it. The memory consumption is increased by a factor between 4 and 7.

## 27.3 False negatives and false positives

As illustrated by the previous example, TSan will only report the data races encountered during execution. Another important caveat is that TSan remembers only a finite number of memory accesses per memory location. At the time of writing, this number is 4. Data races involving a forgotten access will not be detected. Lastly, the documentation of TSan states that there is a tiny probability to miss a race if two threads access the same location at the same time. TSan may overlook data races only in these three specific cases.

For data races between two memory accesses made from OCaml code, TSan does not produce false positives; that is, TSan will not emit spurious reports.

When mixing OCaml and C code, through the use of C primitives, the very notion of false positive becomes less clear, as it involves two memory models – OCaml and C11. However, TSan should
end

Operations on large files. This sub-module provides 64-bit variants of the channel functions that manipulate file positions and file sizes. By representing positions and sizes by 64-bit integers (type `int64`) instead of regular integers (type `int`), these alternate functions allow operating on files whose sizes are greater than `max_int`.

## References

```python
type 'a ref =
{ mutable contents : 'a ;
}
```

The type of references (mutable indirection cells) containing a value of type `'a`.

```python
val ref : 'a -> 'a ref
```

Return a fresh reference containing the given value.

```python
val (!) : 'a ref -> 'a
```

`!r` returns the current contents of reference `r`. Equivalent to `fun r -> r contents`. Unary operator, see `Ocaml_operators`[28.60] for more information.

```python
val (:=) : 'a ref -> 'a -> unit
```

`r := a` stores the value of `a` in reference `r`. Equivalent to `fun r v -> r contents <- v`. Right-associative operator, see `Ocaml_operators`[28.60] for more information.

```python
val incr : int ref -> unit
```

Increment the integer contained in the given reference. Equivalent to `fun r -> r := succ !r`.

```python
val decr : int ref -> unit
```

Decrement the integer contained in the given reference. Equivalent to `fun r -> r := pred !r`.

## Result type

```python
type ('a, 'b) result =
| Ok of 'a
| Error of 'b
```

Since: 4.03