ocaml/opam-repository — pull request #1922 by diml (2014-04-22 09:07:36+00:00)
Title: core 111.11.00-rc1

# 111.11.00

## async

- Updated the sound.ml example

## async_extra

- Made `Log` more fair with respect to other Async jobs, by working on
fixed-length groups of incoming log messages.

Previously, `Log` had processed everything available. The change
gives other Async jobs more of a chance to run.

## async_kernel

- Added `Clock.run_at_intervals`, which runs a job at regular
intervals.

## async_unix

- Added `val Scheduler.yield : unit -> unit Deferred.t`, which becomes
determined after the current cycle finishes.
- Improved the behavior of the scheduler's thread pool when
`Thread.create` raises.

With this improvement, when the thread pool is unable to create a
thread, it presses on with the threads it has rather than raise.
Subsequent attempts to add work to the thread pool will cause the
pool to attempt to create the thread, as long as enough time has
passed since the most recent thread-creation failure.

Before this change, the thread pool wouldn't handle a
`Thread.create` exception, and the exception would get raised to
whatever code happened to be calling the `Thread_pool` function that
tried to create a thread, e.g. `Thread_pool.add_work`. This caused
`In_thread.run` to unexpectedly raise, and in turn
`In_thread.syscall` to unexpectedly raise, leading to:

```
"Fd.syscall_in_thread bug -- should be impossible"
```

Also, changed `should be impossible` text to `please report`, since
there may be other lurking rare exceptions that `In_thread.syscall`
can raise, and we'd like to hear about them.

We rely on thread-pool-stuck detection to report problems where the
inability to create threads causes the inability of the thread pool
to make progress. A tweak was needed to make that work -- now the
thread-pool-stuck warning is based on whether the thread pool has
unstarted work, rather than on whether the thread pool has an
"available thread". The latter would no longer work, since it is
now possible for the thread pool to have unstarted work and to
appear to have an available thread, i.e. `num_threads <
max_num_threads`.

## core

- Change some `Bigstring` functions to retry on `EINTR` rather than
raise.

The following functions (and their unsafe versions) were affected:

* `read`
* `really_read`
* `really_recv`
* `really_write`
* `really_send_no_sigpipe`

Some other `Bigstring` functions, like `input` and `output`, already
retried on `EINTR`, so this change has precedent.

All of the affected stubs raise `Bigstring.IOError` on failure,
rather than `Unix_error`, which means the normal method for retrying
on `EINTR` doesn't work. In particular `Async.Reader` didn't retry
them, even though it was supposed to.

Additionally, the documentation for the following functions was
corrected to say that they raise =Unix_error= rather than =IOError=:

* `pread_assume_fd_is_nonblocking`
* `pwrite_assume_fd_is_nonblocking`
- Eliminated global binary-to-decimal tables computed at startup for
converting `Date` and `Of_day` to string.

Used an efficient implementation of division by 10, rather than the
`sprintf` tables in `time_internal.ml`. This result in much less
allocation at startup and it is a bit faster:

* before:

| Name | Time/Run | mWd/Run | Percentage |
|----------------|----------|---------|------------|
| Date.to_string | 69.39ns | 3.00w | 100.00% |

- after:

| Name | Time/Run | mWd/Run | Percentage |
|----------------|----------|---------|------------|
| Date.to_string | 53.38ns | 3.00w | 100.00% |
- Fixed `Time.Zone` tests so that they are deterministic.
- Added `Iobuf.to_string_hum`, which produces a readable, multi-line
representation of an iobuf, intended for debugging.
- Fixed brittle unit tests of `Command`.

## core_extended

- For `Flang`, added ordering to fields, and added `abs`, `min`, and
`max` to the language.
- Removed `Loggers` module.

## core_kernel

- Added to `String` functions for substring search and replace, based
on the KMP algorithm.

Here are some benchmarks, comparing `Re2` for a fixed pattern,
Mark's kmp from extended_string, and this implementation ("needle").

The pattern is the usual `abacabadabacabae...`. The text looks
similar, with the pattern occurring at the very end.

For =Re2= and =Needle= search benchmarks, the pattern is
preprocessed in advance, outside of the benchmark.

FWIW: I've also tried searches with pattern size = 32767, but =Re2=
blows up, saying:

```
re2/dfa.cc:447: DFA out of memory: prog size 32771 mem 2664898
```

| Name | Time/Run | mWd/Run | mjWd/Run | Prom/Run | Percentage |
|-------------------------------|-----------------|---------------|-------------|----------|------------|
| create_needle_15 | 102.56ns | 21.00w | | | |
| re2_compile_15 | 6_261.48ns | | 3.00w | | 0.01% |
| create_needle_1023 | 13_870.48ns | 5.00w | 1_024.01w | | 0.03% |
| re2_compile_1023 | 107_533.32ns | | 3.03w | | 0.24% |
| create_needle_8191 | 90_107.02ns | 5.00w | 8_192.01w | | 0.20% |
| re2_compile_8191 | 1_059_873.47ns | | 3.28w | 0.28w | 2.37% |
| create_needle_524287 | 6_430_623.96ns | 5.00w | 524_288.09w | | 14.35% |
| re2_compile_524287 | 44_799_605.83ns | | 3.77w | 0.77w | 100.00% |
| needle_search_15_95 | 349.65ns | 4.00w | | | |
| re2_search_15_95 | 483.11ns | | | | |
| mshinwell_search_15_95 | 1_151.38ns | 781.01w | | | |
| needle_search_15_815 | 2_838.85ns | 4.00w | | | |
| re2_search_15_815 | 3_293.06ns | | | | |
| mshinwell_search_15_815 | 8_360.57ns | 5_821.07w | 0.55w | 0.55w | 0.02% |
| needle_search_15_2415 | 8_395.84ns | 4.00w | | | 0.02% |
| re2_search_15_2415 | 9_594.14ns | | | | 0.02% |
| mshinwell_search_15_2415 | 24_602.09ns | 17_021.16w | 1.62w | 1.62w | 0.05% |
| needle_search_1023_6143 | 14_825.50ns | 4.00w | | | 0.03% |
| re2_search_1023_6143 | 40_926.59ns | | | | 0.09% |
| mshinwell_search_1023_6143 | 81_930.46ns | 49_149.66w | 1_025.65w | 1.65w | 0.18% |
| needle_search_1023_52223 | 126_465.96ns | 4.00w | | | 0.28% |
| re2_search_1023_52223 | 365_359.98ns | | | | 0.82% |
| mshinwell_search_1023_52223 | 527_323.73ns | 371_715.39w | 1_033.17w | 9.17w | 1.18% |
| needle_search_1023_154623 | 377_539.53ns | 4.00w | | | 0.84% |
| re2_search_1023_154623 | 1_001_251.93ns | | | | 2.23% |
| mshinwell_search_1023_154623 | 1_499_835.01ns | 1_088_518.15w | 1_033.19w | 9.19w | 3.35% |
| needle_search_8191_49151 | 115_223.31ns | 4.00w | | | 0.26% |
| re2_search_8191_49151 | 559_487.38ns | | | | 1.25% |
| mshinwell_search_8191_49151 | 653_981.19ns | 393_219.50w | 8_201.01w | 9.01w | 1.46% |
| needle_search_8191_417791 | 976_725.24ns | 4.00w | | | 2.18% |
| re2_search_8191_417791 | 4_713_965.69ns | | | | 10.52% |
| mshinwell_search_8191_417791 | 4_224_417.93ns | 2_973_709.32w | 8_202.37w | 10.37w | 9.43% |
| needle_search_8191_1236991 | 2_912_863.78ns | 4.00w | | | 6.50% |
| re2_search_8191_1236991 | 14_039_230.59ns | | | | 31.34% |
| mshinwell_search_8191_1236991 | 11_997_713.73ns | 8_708_130.87w | 8_202.47w | 10.47w | 26.78% |
- Added to `Set` functions for converting to and from a `Map.t`.

```ocaml
val to_map : ('key, 'cmp) t -> f:('key -> 'data) -> ('key, 'data, 'cmp) Map.t
val of_map_keys : ('key, _, 'cmp) Map.t -> ('key, 'cmp) t
```

This required adding some additional type trickery to
`Core_set_intf` to indicate that the comparator for a given module
may or may not be fixed.
- Added an optional `iter` parameter to `Container.Make`.

A direct implementation of `iter` is often more efficient than
defining `iter` in terms of `fold`, and in these cases, the results
of `Container.Make` that are defined in terms of `iter` will be more
efficient also.
- Added `Int.pow` (and for other integer types), for bounds-checked
integer exponentiation.

## ocaml_plugin

- Added a tag to exceptions coming from the toplevel execution of
plugins so that we do not confuse them with exceptions coming from
the library.

Also, added a function to check a plugin without executing it. And
captured the common pattern of checking the compilation of a plugin
in a `Command.t` offered in the library.

## sexplib

- Added error locations to `Macro`-expansion errors.