Include python-file using the real path

When layout_config points to a symbolic link, Zuul attempted to load the
python function from directory of the symbolic link instead of the
parent of the link target.  That causes the scheduled to halt.

* Use realpath on layout_config path to make sure we will load from the
  proper place.
* Update documentation of python-file to mention the include is relative
  to the directory holding layout_config.

Change-Id: I33e221dd7c323423dbf781b53c06333dab2c7d29
Doc: ref links for "approval" requirement

The pipeline.require.approval logic is referenced at two other places
but lacks a link.  Add a label to easily browse to the approval
description.

Change-Id: Iba2e5a1db4023cd61444b2b8de53ffddfc03b16f
Document usage of yappi profiler

ff4eeec introduced support for yappi, a python profiler, which is used
when generating stack dumps (via SIGUSR2).  The commit forgot to update
the related documentation.

Change-Id: Iee1d680cf7b2df4c702666aaf778991c696c3274
Document that the "Stream Events" permission is needed.

Since Gerrit 2.7 the stream-events is only available to users
that are granted the "Stream Events" permission.

Change-Id: I38960b3647145043a31e5c948141831a84643753
Remove "properly" from error message for clarity

The tool is not checking to see that values of the variables are
"proper", it's only checking that they are set.

Removing the word "Proper" from the error message makes this clearer.

Change-Id: Ieea3565a05c8eeb6f67fd140a42197a059288f33
Merge "Doc: ref links for "approval" requirement"
Merge "Document that the "Stream Events" permission is needed."
Merge "Remove "properly" from error message for clarity"
Set gearman timeout to 300

In practice we are seeing that geard can occasionally get disrupted
and then temporarily backlogged enough that it exceeds the 30 second
timeout for submitting a job.  To make Zuul less fragile in this case,
increase the timeouts for any requests submitted to gearman.

Change-Id: I12741bb259c1a78fa2446d764318f84df34bac67
Merge "Include python-file using the real path"
Merger: ensure_cloned() now looks for '.git'

merger.Repo() would not clone a directory if the target path already
exists, even though it might not have been cloned.  Adjust the test to
look whether '.git' exist (which can be either a directory or a file in
the case of submodules).

I came accross this issue when using Zuul cloner to clone two
repositories, one having the other as a submodule.

Given two repositories:
- mediawiki/extension
- mediawiki (having 'mediawiki/extension' has a submodule to /extension)

We run: `zuul-cloner mediawiki mediawiki/extension`

It first clone the mediawiki repository with a placeholder directory for
the submodule. The directory structure is:

    .
    ├── .git/
    └── mediawiki/
	└── extension/

When cloning mediawiki/extension, ensure_cloned() would skip it since
the directory exists.  Looking for subpath '.git', we trick it in
cloning the repository resulting in:

    .
    ├── .git/
    │   └── modules/
    │       └── extension/
    └── mediawiki/
	└── extension/
	    └── .git   # note: a file, not a directory

My use case is cloning a mediawiki/core branch which has several
extensions registered as submodules that I am not going to initialize.

Change-Id: I4fe67a1db31adc2e23e8baf57528fce76f282986
Update merge status after merge:merge is submitted

When preparing a reference, we set the merge state to PENDING before
having emitted the merge:merge.  If any exception occurs when submitting
the merge:merge job, the buildset is left PENDING and is never retried
because prepareRef early exit in such case.

Move the merge_state change after the job has been submitted. An
exception would let the state as is (ie NEW) and thus indicate it should
be retried.

prepareRef happens before processing jobs and we do not have a way to
pause the scheduler.  So I overrided the merger.submitJob() to throw an
exception, the scheduler keep processing until the event fail.  From
there we can assert the merge is still pending (BuildSet.NEW) then
resume the submitJob() method to clear the queue properly.

I ran the test with the change, and it fails properly (there is still a
change stuck in queue).

Closes-Bug: #1358517
Change-Id: I4d91a15aaae878ed231d50ab5f4f7a65f0d0e830
wmf: soften requirements

WebOb and paramiko have a minimal version which is enforced by OpenStack
but is most probably not needed.

ordereddict is for python 2.6, the feature is included in python 2.7.
soften PrettyTable. Precise only has 0.5, only impacts Zuul client.

Change-Id: I1aa1e3ae496d68d25dd64e8b21d2057c473342f4
Merge branch 'dependencies' into master

Adjust requirements.txt to point to the tarballs of:
- PrettyTable
- six
- Babel
- pytz

Change-Id: Ie1bc27fbf8959f8cf0ac9cc4fee1de4b19344d8f
wmf: use tarballs from dependencies branch

Change requirements.txt to point to the four tarballs provided in our
dependencies branch:

- PrettyTable
- six
- Babel
- pytz

Change-Id: Id5b4e55cb524650293252d4d8a47ba78ac1e111c
merge https://review.openstack.org/#/c/123437/3/

cc27e853 Merger: ensure_cloned() now looks for '.git'
merge https://review.openstack.org/#/c/128921/4/

3bfe4dff Update merge status after merge:merge is submitted