Implement read streaming on HTTPSFutures

Summary: See D7723.

Test Plan:
  <?php

  require_once 'scripts/__init_script__.php';

  $future1 = new HTTPSFuture('https://www.google.com/');
  $future2 = new HTTPSFuture('https://www.facebook.com/');

  $wait = 0.01; // In seconds. This is very small for demo purposes, a value
                 // like "1" is probably better in practice.

  $futures = array('google' => $future1, 'facebook' => $future2);
  foreach (Futures($futures)->setUpdateInterval($wait) as $key => $future) {

    // This might be a periodic update, in which case $future will be null.
    // It might also be a future exiting, in which case $future will not be null.
    // In either case, let's read any data we can first.
    foreach ($futures as $key => $future) {
      $new_data = $future->read();
      // Throw the buffered data away after we read it.
      $future->discardBuffers();

      $len = strlen($new_data);
      echo "Got {$len} bytes from {$key}.\n";
      if (strlen($new_data)) {
        // Do something interesting with the data.
      }
    }

    // Now, check if a future exited.
    if ($future !== null) {
      // This future has exited. We should sleep for a bit and then restart it,
      // or whatever.
      echo "Future {$key} exited!\n";
    }
  }

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D7724
Add updateEnv() to ExecFuture for adjusting environmental variables

Summary: Ref T4195. I have a use case where I need to tweak one variable that something else has already set with setEnv().

Test Plan: Used this in next diff.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4195

Differential Revision: https://secure.phabricator.com/D7762
Dramatically improve subprocess I/O for large buffers

Summary:
Ref T4189. When we `write()` a large buffer to an `ExecFuture`, the `fwrite()` loop currently looks like this:

  $bytes_written = fwrite($socket, $buffer);
  $buffer = substr($buffer, $bytes_written);

This is normally fine, but `substr()` is approximately `O(N)` in the size of the new string, since it has to allocate and copy it.

Instead, add `PhutilRope`, which stores a string as a list of small buffers. This allows us to remove bytes from the beginning of the string very cheaply.

In particular, this can occur when you `git push` a very large repository. If we read off the network faster than we write to `git receive-pack`, we end up with a very large internal buffer which is expensive and slow to write through.

Test Plan:
I ran this test script before and after the changes:

  <?php

  require_once 'scripts/__init_script__.php';

  $large = str_repeat('x', 1024 * 1024 * $argv[1]);

  echo "OK...\n";

  $future = new ExecFuture('cat');
  $future->write($large);
  $future->resolvex();

  echo "OK.\n";

A `32MB` write took `16s` before the change and `400ms` afterward. Generally, cost is close to `O(N)` now and was close to `O(N^2)` before, in the size of the buffer.

Reviewers: btrahan, zeeg

Reviewed By: btrahan

CC: aran, FacebookPOC

Maniphest Tasks: T4189

Differential Revision: https://secure.phabricator.com/D7768
Make PhutilChannel::waitForAny more reasonable about detecting unselectable channels

Summary:
Ref T4189. Currently, channel `getWriteSockets()` methods return only //blocked// write sockets, and `waitForAny()` degrades to a busy wait if any channels are missing read and write sockets, assuming the channel is not a socket-based channel.

This means that if you have an unblocked, writable, socket-based channel and a blocked, readable socket-based channel, we always degrade to busy wait immediately.

An example of this occurs during `git push`:

  - We have one R/W channel through SSH to the `git push` on the user's computer.
  - We have one R/W channel through ExecFuture to the `git receive-pack` on the remote host.
  - When the client finishes writing, we close the "R" half of the SSH channel. This leaves it writable, socket-based, and unblocked for most of the push.

Instead, make `getWriteSockets()` methods return //all// write sockets, even if they're not blocked, and then ignore them if `isWriteBufferEmpty()` says we don't need to wait for them.

This is generally cleaner, and fixes several issues:

  # The `ssh-exec` process no longer sits in a busy wait between the SSH read channel closing and the operation finishing.
  # The stdin/stdout/stderr log in the SSH stuff can be simplified (next diff).

Test Plan: Ran `git push` of the Wine repository, saw nothing we own spend an unreasonable amount of time doing CPU stuff. Added a bunch of logging to confirm things are working reasonably under the hood.

Reviewers: btrahan, zeeg

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4189

Differential Revision: https://secure.phabricator.com/D7769
Add a maximum read buffer size to ExecFuture

Summary:
Ref T4241. Currently, `ExecFuture` may read an arbitrarily large amount of data into memory during execution. In T4241, this arises specifically with `git-upload-pack`, which emits about as much data as the size of the repository. Currently, the subprocess emits this data far faster than we can send it over the network, so we end up with a huge in-process buffer containing the entire repository.

Instead, we want to throttle `git-upload-pack` until the network can catch up. This is the first step in letting that happen.

Test Plan: Added a unit test.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4241

Differential Revision: https://secure.phabricator.com/D7772
Use PhutilRope for Channel write buffers and make write buffer details public

Summary:
Ref T4241. Two changes:

  - Use PhutilRope instead of plain string for the Channel write buffers. This makes it more consistent, and should improve performance in some cases.
  - Expose information about buffer sizes as `public`. We need this in a future diff to be able to throttle reads when we're blocked on writes, since we have to be able to test if a write channel is blocked or not.

Test Plan: Ran `arc unit`. Ran `git` operations over SSH.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4241

Differential Revision: https://secure.phabricator.com/D7773
Add a read buffer size to PhutilChannel

Summary: Ref T4241. See D7772. Just like `ExecFuture`, allow the internal channel buffer to be limited in size so when we're plugging a fast pipe into a slow pipe we don't end up with all of the data in memory (subject to actually using these features).

Test Plan: Added unit tests.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4241

Differential Revision: https://secure.phabricator.com/D7774
Allow PhutilChannel to select only write sockets, and expose read buffer status

Summary: Ref T4241. Ref T4206. Small API changes to fix T4241 (next diff). See previous diffs for context.

Test Plan: See next revision.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4206, T4241

Differential Revision: https://secure.phabricator.com/D7775
Make all "protected" in ExecFuture into "private"

Summary: This class is final, these are just vestigal from long, long ago.

Test Plan: Unit tests.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D7779
Added a SIGABRT handler that will write a stack trace to /tmp/phabricator_backtrace_[PID]

Summary:
Stack traces are useful for debugging things.  This adds a
signal handler that registers SIGABRT and SIGHUP. In both cases, a
backtrace is written to /tmp/phabricator_backtrace_<PID>.  If SIGABRT
is caught, we terminate the process with extreme prejudice.

Note, however, that any php script that wishes to actually use this
functionality must declare(ticks=1) (or some other value) in the main
script; i.e., setting this value in an included script doesn't appear
to impact the main script.

Test Plan:
<?php
declare(ticks=1);
require_once ('__init_script__.php');
posix_kill(posix_getpid(), SIGABRT);
sleep (1000000);
?>

for both SIGABRT and SIGHUP, did the same with kill -ABRT and kill -HUP.
Observed that it worked as intended.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley

CC: Korvin, epriestley, aran

Differential Revision: https://secure.phabricator.com/D7797