NonSFI mode: Make tests/common/register_set.h PIC-friendly

For x86-32 obtain address via GOT instead.

This is a necessary preparation to be able to compile via 
pnacl-clang for Non-SFI NaCl.
ARM version is already position-independent.

BUG= https://code.google.com/p/chromium/issues/detail?id=408879

Review URL: https://codereview.chromium.org/546043004

git-svn-id: svn://svn.chromium.org/native_client/trunk/src/native_client@13751 fcba33aa-ac0c-11dd-b9e7-8d5594d729c2
NonSFI mode: Add libcrt_platform for native/mixed linking.

For ARM link failure happens trying to link exception_test.o because
of missing __aeabi_read_tp symbol, this solves the problem.

$ ./scons -j20 --mode=dbg-host,nacl_irt_test bitcode=1 nonsfi_nacl=1 \
  pnacl_generate_pexe=0 run_exception_test_irt \
  use_newlib_nonsfi_loader=0 platform=arm

BUG= https://code.google.com/p/chromium/issues/detail?id=408879
BUG= https://code.google.com/p/nativeclient/issues/detail?id=3951

Review URL: https://codereview.chromium.org/581463004

git-svn-id: svn://svn.chromium.org/native_client/trunk/src/native_client@13752 fcba33aa-ac0c-11dd-b9e7-8d5594d729c2
Remove i686-hosted Linux build and start switching to new directory layout

Switch the default host for the toolchain on Linux to x86_64, and stop
building the i686 version.

Also stop using the host_x86_{32,64} directories for host binaries and
instead put everything (including the driver scripts) into bin/
(This also reverts r13694, removing the pnacl-llc driver script, as that
was the only script that had a name clash with an LLVM binary. It was
only added as a proxy to the pnacl-llc binary anyway).

This moves us closer to a layout that is more direct-to-nacl-friendly
and more like existing gnu-style toolchains.
Also update the README file which describes the layout. It reflects the
intended eventual state (including the placement of the PNaCl target
libraries and translator libraries which will move in a separate CL), but
does not yet include direct-to-nacl-related items.

R=jvoung@chromium.org, noelallen@chromium.org
BUG= https://code.google.com/p/nativeclient/issues/detail?id=3946

Review URL: https://codereview.chromium.org/580983002

git-svn-id: svn://svn.chromium.org/native_client/trunk/src/native_client@13753 fcba33aa-ac0c-11dd-b9e7-8d5594d729c2
export_header.py: Mangle include guards in exported headers

Some code (e.g. in the IRT) compiled by newlib compilers #includes
headers directly from native_client/src/trusted/service_runtime/include
e.g. to use the definitions of NACL_ABI_-prefixed errno values.
However if some other header causes the exported version of errno.h in the
toolchain to be included first, its include guard will prevent the direct
version from src/trusted from acutally being included.

BUG= https://code.google.com/p/nativeclient/issues/detail?id=3910
R=mseaborn@chromium.org

Review URL: https://codereview.chromium.org/577973003

git-svn-id: svn://svn.chromium.org/native_client/trunk/src/native_client@13754 fcba33aa-ac0c-11dd-b9e7-8d5594d729c2
Turning on goma on nacl bots.

BUG=https://code.google.com/p/nativeclient/issues/detail?id=3933
BUG=https://code.google.com/p/chromium/issues/detail?id=390764
TEST=trybots
R=ncbray@chromium.org, mseaborn@chromium.org

Review URL: https://codereview.chromium.org/580433002

git-svn-id: svn://svn.chromium.org/native_client/trunk/src/native_client@13755 fcba33aa-ac0c-11dd-b9e7-8d5594d729c2
Implemented the _exit() syscall in MinSFI.

The syscall allows to immediatelly terminate execution of the
sandbox. It is implemented with the C setjmp/longjmp exception
mechanism, and will jump back to the MinsfiInvokeSandbox function.

Calling exit() now becomes the only legal way of terminating
the sandbox, and returning from the entry function is considered
an error. This allows us to conform to the standard signature of
_start() which returns void.

BUG= https://code.google.com/p/chromium/issues/detail?id=397177
TEST= ./scons platform=x86_64 bitcode=1 minsfi=1 minsfi_tests
NOTRY=true

Review URL: https://codereview.chromium.org/569043002

git-svn-id: svn://svn.chromium.org/native_client/trunk/src/native_client@13756 fcba33aa-ac0c-11dd-b9e7-8d5594d729c2
Disable goma on win7-32 bots because it doesn't suport that platform.

BUG=https://code.google.com/p/nativeclient/issues/detail?id=3933
BUG=https://code.google.com/p/chromium/issues/detail?id=390764
TEST=None
R=ncbray@chromium.org

Review URL: https://codereview.chromium.org/585633003

git-svn-id: svn://svn.chromium.org/native_client/trunk/src/native_client@13757 fcba33aa-ac0c-11dd-b9e7-8d5594d729c2
Cleanup: Use NaClCopyInFromUser() in one case in mmap()

Using NaClCopyInFromUser() is preferred over NaClUserToSysAddrRange()
(+ a volatile memory access) where possible, because it takes care of
any necessary locking.

It looks like this case was missed when the NaClCopyIn/Out functions
were introduced.

Also remove an unnecessary check for "offp == 0".  This is left over
from when mmap() was converted from taking a 32-bit offset arg (by
value) to taking a 64-bit offset arg (by pointer).  It was added as a
sanity check against accidentally forgetting to convert call sites of
the mmap syscall.  The conversion was done long ago, so this check is
no longer needed.

BUG=none
TEST=run_mmap_test and others

Review URL: https://codereview.chromium.org/570653002

git-svn-id: svn://svn.chromium.org/native_client/trunk/src/native_client@13758 fcba33aa-ac0c-11dd-b9e7-8d5594d729c2
Update revision for PNaCl r13739->r13753

Pull the following PNaCl changes into NaCl:
  r13745: (jvoung@google.com) PNaCl: Update Subzero revision in pnacl/COMPONENT_REVISIONS
    | e58178a: (wala@chromium.org) Subzero: Factor our commonalities between mov-like instructions.
    | c8e8781: (jvoung@chromium.org) Convert lit test llvm-mc -arch arguments to full -triple.
    | f820da5: (jvoung@chromium.org) Emit .local before .comm for bss to make llvm-mc happy.
    | d4799f4: (wala@chromium.org) Subzero: Align spill locations to natural alignment.
    | 9dbe38e: (wala@chromium.org) Subzero: Start a list of SIMD improvement ideas.
    | c330274: (wala@chromium.org) Subzero: Randomly insert nops.
    | 4d79fe5: (stichnot@chromium.org) Subzero: Fix the simple register allocation for -Om1.
    | 420e8bf: (stichnot@chromium.org) COmmit
    | 6170e75: (stichnot@chromium.org) Revert "COmmit"
    | bfb03e5: (stichnot@chromium.org) Subzero: Fixes for Hello World and bisection debugging.
    | ac9c943: (stichnot@chromium.org) Subzero: Add a check-lit target for faster smoke testing.
    | ef8cf0e: (stichnot@chromium.org) Subzero: Fix some legalization issues involving immediates.
    | 14c3f41: (stichnot@chromium.org) Subzero: Fix the link command for Trusty.
    | 89d7956: (stichnot@chromium.org) Subzero: Fix address mode optimization involving phi temporaries.
    | d6064a1: (kschimpf@google.com) Start processing function blocks.
    | ce58163: (jvoung@chromium.org) Add pnacl-freeze to the tests_lit/lit.cfg. Also, unsigned vs signed.
    | b17f61d: (jvoung@chromium.org) Align function starts to target-specific bundle alignment.
    | dddc306: (jvoung@chromium.org) Convert lit tests to check disassembled assembly.
    | c0fdc27: (kschimpf@google.com) Add cast instructions to subzero's pnacl bitcode translator.
    | 16178a1: (stichnot@chromium.org) Subzero: Remove the need for a separate NaCl SDK installation.
    | 94539ce: (stichnot@chromium.org) Subzero: Rename -external to -externalize to match llc.
    | 71ba822: (kschimpf@google.com) Add vector insert/extract instructions to Subzero bitcode reader.
    | cabfa30: (stichnot@chromium.org) Subzero: Render constants in dump() to be more like LLVM.
    | dd7b846: (stichnot@chromium.org) Subzero: Make sure register preferences obey register class constraints.
    | 523ef72: (stichnot@chromium.org) Subzero: Work around another llvm-mc parser bug for relocatable symbols.
    | 1d6f0e4: (kschimpf@google.com) Add select instruction to Subzero bitcode reader.
    | dd30c81: (stichnot@chromium.org) Subzero: Fix sext/zext lowering with i1 source operands.
    | 83f9f0c: (kschimpf@google.com) Add icmp and fcmp instructions to Subzero bitcode reader.
    | c836acb: (kschimpf@google.com) Add branch instructions to Subzero bitcode reader.
    | b63cd88: (stichnot@chromium.org) Subzero: Use cvttss2si and similar instead of cvtss2si for fp->int casts.
    | 3ef786f: (stichnot@chromium.org) Subzero: Be more strict about i1 calculations.
    | 7076c88: (stichnot@chromium.org) Subzero: Move python scripts into a common pydir.
    | f12355e: (kschimpf@google.com) Add constants block to PNaCl bitcode reader.
    | 72a8f8d: (stichnot@chromium.org) Subzero: Make sure alloca with align=0 is handled correctly.
    | 10ea698: (stichnot@chromium.org) Subzero: Add a script that builds a hybrid Subzero/llc native executable.
    | 742d72d: (kschimpf@google.com) Add alloca instruction to Subzero bitcode reader.
    | 6d4f564: (stichnot@chromium.org) Subzero: The cross tests should use the actual Subzero runtime.
    | a262333: (jvoung@chromium.org) List Subzero's local optlevel flags after LLVM's cxxflags (precedence).
    | e5a5be7: (stichnot@chromium.org) Subzero: Fix Phi lowering.
    | 41689df: (kschimpf@google.com) Add load and store instructions to Subzero bitcode reader.
    | 64dcde7: (kschimpf@google.com) Fix bug introduced by CL 561883002.
    | c132b76: (kschimpf@google.com) Fix symbol table handling in functions.
    | 65d8d53: (stichnot@chromium.org) Subzero: Print defaults as appropriate in python script help text.
    | 4766156: (kschimpf@google.com) Add phi instruction to Subzero bitcode reader.
    | 44d53e1: (jvoung@chromium.org) Mark setjmp as "returns twice" and turn off SimpleCoalescing when called.
    | 5ee234a: (kschimpf@google.com) Allow ability to name unnamed global addresses in Subzero.
    | 1eb3a55: (jvoung@chromium.org) Switch to llvm::findFirstSet instead of ffs since mingw doesn't have ffs.
  r13749: (jvoung@google.com) PNaCl: Update LLVM revision in pnacl/COMPONENT_REVISIONS
    | 41b16f5: (jvoung@chromium.org) Ensure that x86 prefixes are bundled-locked to instruction (in asm mode).
    | f33570d: (kschimpf@google.com) Allow errors in PNaCl bitcode readers to be directed to any raw_ostream.
  r13752: (uekawa@chromium.org) NonSFI mode: Add libcrt_platform for native/mixed linking.
  r13753: (dschuff@chromium.org) Remove i686-hosted Linux build and start switching to new directory layout

BUG= https://code.google.com/p/chromium/issues/detail?id=408879
BUG= https://code.google.com/p/nativeclient/issues/detail?id=3929
BUG= https://code.google.com/p/nativeclient/issues/detail?id=3946
R= uekawa@chromium.org, kschimpf@google.com, jvoung@chromium.org, dschuff@chromium.org, jvoung@google.com, noelallen@chromium.org
TEST=git try
NOTRY=true
(Please LGTM this change and tick the "commit" box)

Review URL: https://codereview.chromium.org/558913004

git-svn-id: svn://svn.chromium.org/native_client/trunk/src/native_client@13759 fcba33aa-ac0c-11dd-b9e7-8d5594d729c2
untrusted.gypi: Factor out duplicated "--gomadir" option

This can be moved to "common_args".

I tested this by doing the following:

cd ..
./native_client/build/gyp_nacl native_client/build/all.gyp \
    --generator-output=out-gyp -D use_goma=1 -D gomadir=MYGOMADIR

I checked that the makefiles in out-gyp pass "--gomadir=MYGOMADIR"
before and after the change.

I also checked that there was no change when omitting "-D gomadir=..."
and when passing "-D use_goma=0".

BUG=none
TEST=see above

Review URL: https://codereview.chromium.org/583583006

git-svn-id: svn://svn.chromium.org/native_client/trunk/src/native_client@13760 fcba33aa-ac0c-11dd-b9e7-8d5594d729c2
Fix mmap() with length > 0xffff0000 to behave consistently on 32/64-bit

Before this change, mmap() with such a length would succeed on x86-32
(this was a corner-case where creating a zero-length mapping was
allowed) but fail on x86-64 with ENOMEM.

This adds tests for two cases:
 * mmap() with length == 0
 * mmap() with length == 0xffffffff

BUG= https://code.google.com/p/nativeclient/issues/detail?id=3917
TEST= run_mmap_test

Review URL: https://codereview.chromium.org/561203003

git-svn-id: svn://svn.chromium.org/native_client/trunk/src/native_client@13761 fcba33aa-ac0c-11dd-b9e7-8d5594d729c2
Support file path metadata for the main nexe.

Instead of passing file token data to the plugin for the main nexe, Chromium is
changing to fetch file path metadata on the browser and pass it down to the
plugin initially. This removes an unnecessary IPC round trip and simplifies the
main nexe loading codepath.

This change allows an embedder to supply a NaClDesc that already has origin
information and is marked safe-for-mmap, instead of clearing that information
and fetching metadata based on file tokens.

As far as I can tell, the existing call to NaClDescMarkUnsafeForMmap()
is unnecessary since I don't see how that desc could have been marked
safe for mmap.

This is needed to support a Chromium-side change to simplify validation behavior
for the main nexe:
  https://codereview.chromium.org/572973002

This has been manually tested with that change.

BUG=chromium:394130
R=mseaborn@chromium.org

Review URL: https://codereview.chromium.org/556743004

git-svn-id: svn://svn.chromium.org/native_client/trunk/src/native_client@13762 fcba33aa-ac0c-11dd-b9e7-8d5594d729c2
Add CreateNaClDescWithFilePathMetadata fn.

This change adds a method to create a NaClDesc for a NaClHandle which has
file path information without requiring the emebedder to do some of the work
preparing the internals of the NaClDesc.

WithFilePathMetadata has been added to the end of this method name since ncbray
suggested that we may want to create validated NaClDescs using different
metadata types.

BUG=chromium:394130
R=mseaborn@chromium.org

Review URL: https://codereview.chromium.org/556613002

git-svn-id: svn://svn.chromium.org/native_client/trunk/src/native_client@13763 fcba33aa-ac0c-11dd-b9e7-8d5594d729c2
Switch PNaCl lib, include, and translator directories for new layout

This makes the directory layout more like a standard gnu-style toolchain and more suitable for support of multiple architectures, which will be used in direct-to-nacl mode. For now, since the biased bitcode builds will have to be around until direct-to-nacl mode can replace them, they are included in the same scheme as if they were their own architectures.

For now the directories used by the translator (the "native libs" and translator nexes) have a similar layout to before except that they all move into the translator/ directory.

The changes to the bitcode lib builds should also make it easier to reuse them  for direct-to-nacl mode if desired.

R=jvoung@chromium.org, noelallen@chromium.org
BUG= https://code.google.com/p/nativeclient/issues/detail?id=3946

Review URL: https://codereview.chromium.org/583783002

git-svn-id: svn://svn.chromium.org/native_client/trunk/src/native_client@13764 fcba33aa-ac0c-11dd-b9e7-8d5594d729c2
Pass naclsdk_validate=0 to scons runs run by main waterfall bots

r13764 changed the directory layout used by the PNaCl toolchain; SCons
checks for the presence of headers in this known location when it starts
up, to be sure the NaCl compiler is present, and that revision updated
it to the new location. This means the main waterfall bots would be
broken until the toolchain rolls. So disable them for now. This
can be reverted after the next PNaCl toolchain roll.

R=jvoung@chromium.org
BUG=red bots

Review URL: https://codereview.chromium.org/591623002

git-svn-id: svn://svn.chromium.org/native_client/trunk/src/native_client@13765 fcba33aa-ac0c-11dd-b9e7-8d5594d729c2
Update .DEPS.git

git-svn-id: svn://svn.chromium.org/native_client/trunk/src/native_client@13766 fcba33aa-ac0c-11dd-b9e7-8d5594d729c2
Update .DEPS.git

git-svn-id: svn://svn.chromium.org/native_client/trunk/src/native_client@13767 fcba33aa-ac0c-11dd-b9e7-8d5594d729c2
Allowed decrementing program break with sbrk() in MinSFI

Even though the POSIX standard does not define the behaviour of sbrk()
when its 'increment' argument is negative, dlmalloc does free memory
by calling sbrk() in that way. This patch changes sbrk() so that it
does not trap but rather adjusts the untrusted program break accordingly.

BUG= https://code.google.com/p/chromium/issues/detail?id=397177
TEST= ./scons platform=x86_64 bitcode=1 minsfi=1 minsfi_tests
NOTRY=true

Review URL: https://codereview.chromium.org/585443004

git-svn-id: svn://svn.chromium.org/native_client/trunk/src/native_client@13768 fcba33aa-ac0c-11dd-b9e7-8d5594d729c2
Added tests for rename functions in irt_ext.

BUG= https://code.google.com/p/nativeclient/issues/detail?id=3918
TEST= trybots

Review URL: https://codereview.chromium.org/548683004

git-svn-id: svn://svn.chromium.org/native_client/trunk/src/native_client@13769 fcba33aa-ac0c-11dd-b9e7-8d5594d729c2
Added tests for unlink functions irt_ext.

R=mcgrathr@chromium.org
BUG= https://code.google.com/p/nativeclient/issues/detail?id=3918
TEST= trybots

Review URL: https://codereview.chromium.org/572723002

git-svn-id: svn://svn.chromium.org/native_client/trunk/src/native_client@13770 fcba33aa-ac0c-11dd-b9e7-8d5594d729c2