xdiff: load full words in the inner loop of xdl_hash_record

Redo the hashing loop in xdl_hash_record in a way that loads an entire
'long' at a time, using masking tricks to see when and where we found
the terminating '\n'.

I stole inspiration and code from the posts by Linus Torvalds around

  https://lkml.org/lkml/2012/3/2/452
  https://lkml.org/lkml/2012/3/5/6

His method reads the buffers in sizeof(long) increments, and may thus
overrun it by at most sizeof(long)-1 bytes before it sees the final
newline (or hits the buffer length check).  I considered padding out
all buffers by a suitable amount to "catch" the overrun, but

* this does not work for mmap()'d buffers: if you map 4096+8 bytes
  from a 4096 byte file, accessing the last 8 bytes results in a
  SIGBUS on my machine; and

* it would also be extremely ugly because it intrudes deep into the
  unpacking machinery.

So I adapted it to not read beyond the buffer at all.  Instead, it
reads the final partial word byte-by-byte and strings it together.
Then it can use the same logic as before to finish the hashing.

So far we enable this only on x86_64, where it provides nice speedup
for diff-related work:

  Test                                  origin/next      tr/xdiff-fast-hash
  -----------------------------------------------------------------------------
  4000.1: log -3000 (baseline)          0.07(0.05+0.02)  0.08(0.06+0.02) +14.3%
  4000.2: log --raw -3000 (tree-only)   0.37(0.33+0.04)  0.37(0.32+0.04) +0.0%
  4000.3: log -p -3000 (Myers)          1.75(1.65+0.09)  1.60(1.49+0.10) -8.6%
  4000.4: log -p -3000 --histogram      1.73(1.62+0.09)  1.58(1.49+0.08) -8.7%
  4000.5: log -p -3000 --patience       2.11(2.00+0.10)  1.94(1.80+0.11) -8.1%

Perhaps other platforms could also benefit.  However it does NOT work
on big-endian systems!

[jc: minimum style and compilation fixes]

Signed-off-by: Thomas Rast <trast@student.ethz.ch>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
git-cherry-pick: add allow-empty option

git cherry-pick fails when picking a non-ff commit that is empty.  The advice
given with the failure is that a git-commit --allow-empty should be issued to
explicitly add the empty commit during the cherry pick.  This option allows a
user to specify before hand that they want to keep the empty commit.  This
eliminates the need to issue both a cherry pick and a commit operation.

Signed-off-by: Neil Horman <nhorman@tuxdriver.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
git-daemon wrapper to wait until daemon is ready

The shell script which is currently used to parse git daemon output does
not seem to work reliably. In order to work around such issues,
re-implement the same procedure in C and write the daemon pid to a file.

This means that we can no longer wait on the daemon process, since it is
no longer a direct child of the shell process.

Signed-off-by: Clemens Buchacher <drizzd@aon.at>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Makefile: feed all header files to xgettext

Translation markers may be present in header files too. Make sure we
don't miss any.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
git-svn: use POSIX::sigprocmask to block signals

In order to maintain consistency of the database mapping svn revision
numbers to git commit ids, rev_map_set() defers signal processing until
it's finished with an append transaction.[*]

The conventional way to achieve this is through sigprocmask(), which is
available in perl in the standard POSIX module.

This is implemented by this patch.  One important consequence of it is
that the signal handlers won't be unconditionally set to SIG_DFL anymore
upon the first invocation of rev_map_set() as they used to.  As a
result, the signals ignored by git-svn parent will remain ignored;
otherwise the behavior remains the same.

This patch paves the way to ignoring SIGPIPE throughout git-svn which
will be done in the followup patch.

[*] Deferring signals is not enough to ensure the database consistency:
the program may die on SIGKILL or power loss, run out of disk space,
etc.  However that's a separate issue that this patch doesn't address.

Signed-off-by: Roman Kagan <rkagan@mail.ru>
Acked-by: Eric Wong <normalperson@yhbt.net>
git-svn: ignore SIGPIPE

In HTTP with keep-alive it's not uncommon for the client to notice that
the server decided to stop maintaining the current connection only when
sending a new request.  This naturally results in -EPIPE and possibly
SIGPIPE.

The subversion library itself makes no provision for SIGPIPE.  Some
combinations of the underlying libraries do (typically SIG_IGN-ing it),
some don't.

Presumably for that reason all subversion commands set SIGPIPE to
SIG_IGN early in their main()-s.

So should we.

This, together with the previous patch, fixes the notorious "git-svn
died of signal 13" problem (see e.g.
http://thread.gmane.org/gmane.comp.version-control.git/134936).

Signed-off-by: Roman Kagan <rkagan@mail.ru>
Acked-by: Eric Wong <normalperson@yhbt.net>
git-svn: drop redundant blocking of SIGPIPE

Now that SIGPIPE is ignored there's no point blocking it.

Signed-off-by: Roman Kagan <rkagan@mail.ru>
Acked-by: Eric Wong <normalperson@yhbt.net>
git p4: fix-up "import/export of labels to/from p4"

The previous one is already in 'next' but was somewhat lacking.

The configuration "git-p4.validLabelRegexp" is now called
"labelExportRegexp", and its default covers lowercase alphabets as
well.

Signed-off-by: Luke Diamand <luke@diamand.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
git p4: Squash P4EDITOR in test harness

If P4EDITOR is set in the environment, test behavior could be
unpredictable. Set it explicitly.

Signed-off-by: Luke Diamand <luke@diamand.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
git p4: Ignore P4EDITOR if it is empty

p4 itself treats an empty value for P4EDITOR as the same as
having P4EDITOR unset. Do the same for "git p4".

Signed-off-by: Luke Diamand <luke@diamand.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
git p4: move verbose to base class

The verbose flag is common to all classes, or at least should be.
Make it a member of the base Command class, rather than
reimplementing for each class. Make option parsing mirror this.

Signed-off-by: Luke Diamand <luke@diamand.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
vcs-svn: cap number of bytes read from sliding view

Introduce a "max_off" field in struct sliding_view, roughly
representing a maximum number of bytes that can be read from "file".
If it is set to a nonnegative integer, a call to move_window()
attempting to put the right endpoint beyond that offset will return
an error instead.

The idea is to use this when applying Subversion-format deltas to
prevent reads past the end of the preimage (which has known length).
Without such a check, corrupt deltas would cause svn-fe to block
indefinitely when data in the input pipe is exhausted.

Inspired-by: Ramkumar Ramachandra <artagnon@gmail.com>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
Merge branch 'mh/ref-api'

Allows walking only a sub-hierarchy in refs/*.

By Michael Haggerty
* mh/ref-api:
  do_for_each_ref(): only iterate over the subtree that was requested
  refs: store references hierarchically
  sort_ref_dir(): simplify logic
  refs.c: rename ref_array -> ref_dir
  struct ref_entry: nest the value part in a union
  check_refname_component(): return 0 for zero-length components
  free_ref_entry(): new function
  names_conflict(): simplify implementation
  repack_without_ref(): reimplement using do_for_each_ref_in_array()
  do_for_each_ref_in_arrays(): new function
  do_for_each_ref_in_array(): new function
  refs: manage current_ref within do_one_ref()
  refs.c: reorder definitions more logically
Merge branch 'pw/fast-import-dataref-parsing'

The parser in "fast-import" did not diagnose ":9" style references
that is not followed by required SP/LF as an error.

By Pete Wyckoff
* pw/fast-import-dataref-parsing:
  fast-import: tighten parsing of datarefs
Merge branch 'it/fetch-pack-many-refs'

When "git fetch" encounters repositories with too many references, the
command line of "fetch-pack" that is run by a helper e.g. remote-curl,
may fail to hold all of them. Now such an internal invocation can feed
the references through the standard input of "fetch-pack".

By Ivan Todoroski
* it/fetch-pack-many-refs:
  remote-curl: main test case for the OS command line overflow
  fetch-pack: test cases for the new --stdin option
  remote-curl: send the refs to fetch-pack on stdin
  fetch-pack: new --stdin option to read refs from stdin
Merge branch 'mk/gitweb-diff-hl'

"gitweb" learns to highlight the patch it outputs even more.

By Michał Kiedrowicz (7) and Jakub Narębski (1)
* mk/gitweb-diff-hl:
  gitweb: Refinement highlightning in combined diffs
  gitweb: Highlight interesting parts of diff
  gitweb: Push formatting diff lines to print_diff_chunk()
  gitweb: Use print_diff_chunk() for both side-by-side and inline diffs
  gitweb: Extract print_sidebyside_diff_lines()
  gitweb: Pass esc_html_hl_regions() options to esc_html()
  gitweb: esc_html_hl_regions(): Don't create empty <span> elements
  gitweb: Use descriptive names in esc_html_hl_regions()
Merge branch 'zj/upstream-error-message'

Error message given when @{u} is used for a branch without its
upstream configured have been clatified.

By Zbigniew Jędrzejewski-Szmek
* zj/upstream-error-message:
  i18n: mark @{upstream} error messages for translation
  Be more specific if upstream branch is not tracked
  Provide better message for barnhc_wiht_tpyo@{u}
  Provide branch name in error message when using @{u}
  t1507: add tests to document @{upstream} behaviour
Merge branch 'cc/fix-missing-va-end-in-revert'

By Christian Couder
* cc/fix-missing-va-end-in-revert:
  revert: add missing va_end
test-subprocess: fix segfault without arguments

Check if we even have a parameter before checking its value.  Running
this command without any arguments may not make a lot of sense, but
reacting with a segmentation fault is unduly harsh.

While we're at it, avoid casting argv by declaring it const right away.

Signed-off-by: Rene Scharfe <rene.scharfe@lsrfire.ath.cx>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The fifth batch of topics graduated to 'master'

Signed-off-by: Junio C Hamano <gitster@pobox.com>