clone: drop connectivity check for local clones

Commit 0433ad1 (clone: run check_everything_connected,
2013-03-25) added the same connectivity check to clone that
we use for fetching. The intent was to provide enough safety
checks that "git clone git://..." could be counted on to
detect bit errors and other repo corruption, and not
silently propagate them to the clone.

For local clones, this turns out to be a bad idea, for two
reasons:

  1. Local clones use hard linking (or even shared object
     stores), and so complete far more quickly. The time
     spent on the connectivity check is therefore
     proportionally much more painful.

  2. Local clones do not actually meet our safety guarantee
     anyway. The connectivity check makes sure we have all
     of the objects we claim to, but it does not check for
     bit errors. We will notice bit errors in commits and
     trees, but we do not load blob objects at all. Whereas
     over the pack transport, we actually recompute the sha1
     of each object in the incoming packfile; bit errors
     change the sha1 of the object, which is then caught by
     the connectivity check.

This patch drops the connectivity check in the local case.
Note that we have to revert the changes from 0433ad1 to
t5710, as we no longer notice the corruption during clone.

We could go a step further and provide a "verify even local
clones" option, but it is probably not worthwhile. You can
already spell that as "cd foo.git && git fsck && git clone ."
or as "git clone --no-local foo.git".

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
git-remote-mediawiki: introduction of Git::Mediawiki.pm

We would want to allow the user to preview what he has edited locally
before pushing it out (and thus creating a non-removable revision in
the mediawiki's history).

This patch introduces a new perl package in which we will be able to
share code between that new tool and the remote helper:
git-remote-mediawiki.perl.

A perl package offers the best way to handle such case: Each script
can select what should be imported in its namespace.  The package
namespacing limits the use of side effects in the shared code.

An alternate solution is to concatenate a "toolset" file with each
*.perl when 'make'-ing the project. In that scheme, everything is
imported in the script's namespace. Plus, files should be renamed in
order to chain to Git's toplevel makefile. Hence, this solution is not
acceptable.

Signed-off-by: Benoit Person <benoit.person@ensimag.fr>
Signed-off-by: Matthieu Moy <matthieu.moy@grenoble-inp.fr>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
wrap-for-bin: make bin-wrappers chainable

For now, bin-wrappers overwrites GITPERLLIB. If we want to chain to
those scripts and define GITPERLLIB before, our changes will be
discarded.

This patch makes the bin-wrappers prepend their modifications to
GITPERLLIB rather than redefining it. It also unset GITPERLLIB in the
test-suite to prevent broken $GITPERLLIB in the user's configuration
from interfering with the testsuite.

The codes using GIT_TEMPLATE_DIR and GIT_TEXTDOMAINDIR handle only one
path in each of this variable so this new behavior would be useless on
those variables.

Signed-off-by: Benoit Person <benoit.person@ensimag.fr>
Signed-off-by: Matthieu Moy <matthieu.moy@grenoble-inp.fr>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
git-remote-mediawiki: add a git bin-wrapper for developement

The introduction of the Git::Mediawiki package makes it impossible to test,
without installation, git-remote-mediawiki and git-mw.

Using a git bin-wrapper enables us to define proper $GITPERLLIB to force the
use of the developement version of the Git::Mediawiki package, bypassing its
installed version if any.

An alternate solution was to 'install' all the files required at each build
but it pollutes the toplevel with untracked files.

Signed-off-by: Benoit Person <benoit.person@ensimag.fr>
Signed-off-by: Matthieu Moy <matthieu.moy@grenoble-inp.fr>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
git-remote-mediawiki: update tests to run with the new bin-wrapper

Until now, if git-remote-mediawiki was not installed, the test suite
copied it to the toplevel directory. This solution pollutes the
directory with untracked files. Plus, we would need to copy the new
git-mw.perl file to test it too.

Signed-off-by: Benoit Person <benoit.person@ensimag.fr>
Signed-off-by: Matthieu Moy <matthieu.moy@grenoble-inp.fr>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
git-remote-mediawiki: factoring code between git-remote-mediawiki and Git::Mediawiki

For now, Git::Mediawiki contains nothing.

This first patch moves some of git-remote-mediawiki.perl's factorisable code
into Git::Mediawiki. In the same time, it removes the side effects of that code
and renames the fucntions and constants moved to expose a better API.

Signed-off-by: Benoit Person <benoit.person@ensimag.fr>
Signed-off-by: Matthieu Moy <matthieu.moy@grenoble-inp.fr>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
git-remote-mediawiki: add git-mw command

For now, git-remote-mediawiki is only a remote-helper. This patch adds a new
toolset script in which we will be able to build new tools for
git-remote-mediawiki.

This toolset uses a subcommand-mechanism to launch the proper action. For now
only the 'help' subcommand is implemented. It also provides some generic code
for the verbose and help command line options.

Signed-off-by: Benoit Person <benoit.person@ensimag.fr>
Signed-off-by: Matthieu Moy <matthieu.moy@grenoble-inp.fr>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
git-remote-mediawiki: add preview subcommand into git mw

In the current state, a user of git-remote-mediawiki can edit the markup text
locally, but has to push to the remote wiki to see how the page is rendererd.
Add a new 'git mw preview' command that allows rendering the markup text on
the remote wiki without actually pushing any change on the wiki.

This uses Mediawiki's API to render the markup and inserts it in an actual
HTML page from the wiki so that CSS can be rendered properly. Most links
should work when the page exists on the remote.

Signed-off-by: Benoit Person <benoit.person@ensimag.fr>
Signed-off-by: Matthieu Moy <matthieu.moy@grenoble-inp.fr>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
remote.c: avoid O(m*n) behavior in match_push_refs

When pushing using a matching refspec or a pattern refspec, each ref
in the local repository must be paired with a ref advertised by the
remote server.  This is accomplished by using the refspec to transform
the name of the local ref into the name it should have in the remote
repository, and then performing a linear search through the list of
remote refs to see if the remote ref was advertised by the remote
system.

Each of these lookups has O(n) complexity and makes match_push_refs()
be an O(m*n) operation, where m is the number of local refs and n is
the number of remote refs.  If there are many refs 100,000+, then this
ref matching can take a significant amount of time.  Let's prepare an
index of the remote refs to allow searching in O(log n) time and
reduce the complexity of match_push_refs() to O(m log n).

We prepare the index lazily so that it is only created when necessary.
So, there should be no impact when _not_ using a matching or pattern
refspec, i.e. when pushing using only explicit refspecs.

Dry-run push of a repository with 121,913 local and remote refs:

        before     after
real    1m40.582s  0m0.804s
user    1m39.914s  0m0.515s
sys     0m0.125s   0m0.106s

The creation of the index has overhead.  So, if there are very few
local refs, then it could take longer to create the index than it
would have taken to just perform n linear lookups into the remote
ref space.  Using the index should provide some improvement when
the number of local refs is roughly greater than the log of the
number of remote refs (i.e. m >= log n).  The pathological case is
when there is a single local ref and very many remote refs.

Dry-run push of a repository with 121,913 remote refs and a single
local ref:

        before    after
real    0m0.525s  0m0.566s
user    0m0.243s  0m0.279s
sys     0m0.075s  0m0.099s

Using an index takes 41 ms longer, or roughly 7.8% longer.

Jeff King measured a no-op push of a single ref into a remote repo
with 370,000 refs:

        before    after
real    0m1.087s  0m1.156s
user    0m1.344s  0m1.412s
sys     0m0.288s  0m0.284s

Using an index takes 69 ms longer, or roughly 6.3% longer.

None of the measurements above required transferring any objects to
the remote repository.  If the push required transferring objects and
updating the refs in the remote repository, the impact of preparing
the search index would be even smaller.

A similar operation is performed in the reverse direction when pruning
using a matching or pattern refspec.  Let's avoid O(m*n) behavior in
the same way by lazily preparing an index on the local refs.

Signed-off-by: Brandon Casey <drafnel@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
t9902: fix 'test A == B' to use = operator

The == operator as an alias to = is not POSIX.  This doesn't actually
matter for the execution of the script, because it only runs when the
shell is bash.  However, it trips up test-lint, so it's nicer to use
the standard form.

Signed-off-by: Thomas Rast <trast@inf.ethz.ch>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
test-lint: detect 'export FOO=bar'

Some shells do not understand the one-line construct, and instead need

  FOO=bar &&
  export FOO

Detect this in the test-lint target.

Signed-off-by: Thomas Rast <trast@inf.ethz.ch>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Merge branch 'jc/triangle-push-fixup' into jch

Earlier remote.pushdefault (and per-branch branch.*.pushremote)
were introduced as an additional mechanism to choose what
repository to push into when "git push" did not say it from the
command line, to help people who push to a repository that is
different from where they fetch from.  This attempts to finish that
topic by teaching the default mechanism to choose branch in the
remote repository to be updated by such a push.

The 'current', 'matching' and 'nothing' modes (specified by the
push.default configuration variable) extend to such a "triangular"
workflow naturally, but 'upstream' and 'simple' have to be updated.

. 'upstream' is about pushing back to update the branch in the
remote repository that the current branch fetches from and
integrates with, it errors out in a triangular workflow.

. 'simple' is meant to help new people by avoiding mistakes, and
will be the safe default in Git 2.0.  In a non-triangular
workflow, it will continue to act as a cross between 'upstream'
and 'current' in that it pushes to the current branch's
@{upstream} only when it is set to the same name as the current
branch (e.g. your 'master' forks from the 'master' from the
central repository).  In a triangular workflow, this series
tentatively defines it as the same as 'current', but we may have
to tighten it to avoid surprises in some way.

* jc/triangle-push-fixup:
  t/t5528-push-default: test pushdefault workflows
  t/t5528-push-default: generalize test_push_*
  push: change `simple` to accommodate triangular workflows
  config doc: rewrite push.default section
  t/t5528-push-default: remove redundant test_config lines
Merge branch 'jx/clean-interactive' (early part) into jch

* 'jx/clean-interactive' (early part):
  git-clean: refactor git-clean into two phases
  write_name{_quoted_relative,}(): remove redundant parameters
  quote_path_relative(): remove redundant parameter
  quote.c: substitute path_relative with relative_path
  path.c: refactor relative_path(), not only strip prefix
  test: add test cases for relative_path
Merge branch 'rr/rebase-checkout-reflog' into jch

Invocations of "git checkout" used internally by "git rebase" were
counted as "checkout", and affected later "git checkout -" to the
the user to an unexpected place.

The "bugfix" bits of the series is here.  The follow-up part to
reword reflog messages for steps in "rebase" are split to another
topic.

* rr/rebase-checkout-reflog:
  checkout: respect GIT_REFLOG_ACTION
  status: do not depend on rebase reflog messages
  t/t2021-checkout-last: "checkout -" should work after a rebase finishes
  wt-status: remove unused field in grab_1st_switch_cbdata
  t7512: test "detached from" as well
Merge branch 'as/log-output-encoding-in-user-format' (early part) into jch

* 'as/log-output-encoding-in-user-format' (early part):
  t4041, t4205, t6006, t7102: use iso8859-1 rather than iso-8859-1
  t4205: replace .\+ with ..* in sed commands
  pretty: --format output should honor logOutputEncoding
  pretty: Add failing tests: --format output should honor logOutputEncoding
  t4205 (log-pretty-formats): don't hardcode SHA-1 in expected outputs
  t7102 (reset): don't hardcode SHA-1 in expected outputs
  t6006 (rev-list-format): don't hardcode SHA-1 in expected outputs
Merge branch 'jk/bash-completion' into jch

* jk/bash-completion:
  completion: learn about --man-path
  completion: handle unstuck form of base git options
Merge branch 'jg/status-config' into jch

"git status" learned status.branch and status.short configuration
variables to use --branch and --short options by default (override
with --no-branch and --no-short options from the command line).

The bottom two has been graduated to 'master' but then reverted.
The tip two are quick attempts to fix the fallout.  The one for
status.short looks correct; the other one, while it is correct,
is unfortunately overly complex in order not to introduce an
unnecessary regression.

* jg/status-config:
  status/commit: make sure --porcelain is not affected by user-facing config
  commit: make it work with status.short
  status: introduce status.branch to enable --branch by default
  status: introduce status.short to enable --short by default
Merge branch 'jc/maint-diff-core-safecrlf' into jch

"git diff" refused to even show difference when core.safecrlf is
set to true (i.e. error out) and there are offending lines in the
working tree files.

* jc/maint-diff-core-safecrlf:
  diff: demote core.safecrlf=true to core.safecrlf=warn
Merge branch 'pb/stash-refuse-to-kill' into jch

"git stash save" is not just about "saving" the local changes, but
also is to restore the working tree state to that of HEAD. If you
changed a non-directory into a directory in the local change, you
may have untracked files in that directory, which have to be killed
while doing so, unless you run it with --include-untracked.  Teach
the command to detect and error out before spreading the damage.

This needed a small fix to "ls-files --killed".

* pb/stash-refuse-to-kill:
  git stash: avoid data loss when "git stash save" kills a directory
  treat_directory(): do not declare submodules to be untracked
Merge branch 'tr/test-v-and-v-subtest-only' into jch

Finishing touches to a topics that is already in master for the
upcoming release.

* tr/test-v-and-v-subtest-only:
  t0000: do not use export X=Y