mozilla/r2d2b2g — pull request #663 by bkase (2013-06-27 10:01:34+00:00)
Title: ADB in-process

### Overview
This pull-request brings ADB in-process. Sources from ADB were taken from AOSP and using a modified Makefile from the Ubuntu adb package (on *nixes) ADB is build as a shared library. This shared library is loaded in-process via js-ctypes. All communication to native code is currently implemented within EventedChromeWorkers or ECWs (see `addon/lib/adb-pure/evented-chrome-worker.js`), an abstraction on top of the ChromeWorker API. Additionally, a lot of code interacts with the ctypes Instantiator (see `addon/lib/adb-pure/ctypes-instantiator`), an abstraction over ctypes declarations.

### File/Directory structure
```
addon/
lib/
adb-pure/ -- directory for all of the new adb code
adb-pure.js -- top level module for new adb implementation, `_startAdbInBackground` is where adb in-process code starts

evented-chrome-worker.js -- abstraction on top of ChromeWorker API (all *thread*.js files are
EventedChromeWorkers or ECWs)
worker-console.js -- a console implementation for workers

ctypes-instantiator.js -- abstraction on top of ctypes library declare functionality
ctypes-bridge-builder.js -- automatically builds bridges between function pointers and native
function calls, used to trick Windows into working
adb-types.js -- a bunch of type definitions used for js-ctypes (for Windows)

adb-client.js -- abstract base-object for various client-side interactions with adb
adb-command-runner.js -- runs generic adb commands, implements AdbClient
adb-device-tracker.js -- tracks device changes live, implements AdbClient
adb-file-transfer.js -- transfers files to device (adb push), implements AdbClient

adb-server-thread.js -- this ECW is responsible for the main server loop in adb. Before the
server starts looping, this thread spawns several other threads. These
thread spawning calls are hijacked with JavaScript callbacks (see
`spawnIOfn` and `spawnDfn`)
adb-device-poll-thread.js -- this ECW simply calls the device-polling thread on OSX. On
Linux, due to a bug (see Bugs), this does nothing. On Windows,
the device-polling-thread is passed a reference to a structure
that bridges to USB driver calls
adb-io-thread.js -- this ECW is responsible for any I/O related activities clientside, for
now, either reading or writing to/from a file descriptor. It might be
worth making a pool of these ECWs.
adb-io-thread-spawner.js -- this ECW executes either the input or output device threads
related to communicating with a specific device. A new pair of
input and output device threads are spawned (i.e. spawnIOfn is
called from adb-device-poll-thread) whenever a device is
connected and the threads exit when a device is disconnected.
low-level/
android-tools/
adb-bin/
all adb native sources -- explaining all of these files would be out of the scope of
this pull-request comment. This might need to be explained in
person/over video chat if necessary.
*/ -- dependencies for the native code
Makefile -- the Makefile for the native code
test/
test-adb-pure.js -- the main test cases for the adb implementation. Start the tests with a
device unplugged. Plug in a FFOS device when the console asks.

test-evented-chrome-worker.js -- tests for evented-chrome-worker
evented-chrome-worker-worker.js -- a helper worker for evented-chrome-worker tests

build-and-test-adb.sh -- a simple script to rebuild native code and execute
only `test-adb-pure.js`
```

### Bugs as of time of pull
* The cleanup code needs to be async due to coordination between web workers, but even with a `setTimeout 0`, nothing on the event queue runs that is enqueued during the execution of Simulator.unload. Therefore cleaning up ADB on the simulator currently is broken
* Cleaning up the input and output device threads currently relies on a race condition. See `TODO` on line 296 of `transport.cpp`.
Since the USB reading is not backed by a file descriptor outside of linux, a `select`-variant cannot be used here to exit safely.
Also, it looks like Web Workers are backed by `PRThread`s. If there is an equivalent to `pthread_kill` in `PRThread`s, there would be a safe way to exit.
* Sometimes the `cfx test` fails exiting on Windows saying that temporary files could not be removed. However, the firefox process is always killed assuming `cfx test` is not `ctrl-C`ed before it terminates.
* On OSX, very rarely, the Simulator will fail to catch a replug after an unplug -> replug -> unplug. This seems like is should be solvable
* On Linux, USB reading and writing relies on APIs that sometimes hang (wat), so this code also relies on `pthread_kill`. I decided to revert this from a web worker back to a pthread. If `PRThread`s have something analogous to `pthread_kill`, this should be able to go back to a web worker.

### Miscellaneous
* Look over the JavaScript code first. I still have to clean-up a lot of native code.
* I will also try to knock-out TODOs while this is being reviewed
* The Windows code is built with Visual Studios, but the project files aren't all in this pull-request yet, so if you're testing this code use OSX or Linux for now
* See https://github.com/bkase/r2d2b2g/tree/adb-js-raw for a squashed-less history of this pull-request (this was my history while developing).

Sorry for the huge pull-request. Good luck!

@ochameau
@mykmelez