cache.h: move remote/connect API out of it

The definition of "struct ref" in "cache.h", a header file so
central to the system, always confused me.  This structure is not
about the local ref used by sha1-name API to name local objects.

It is what refspecs are expanded into, after finding out what refs
the other side has, to define what refs are updated after object
transfer succeeds to what values.  It belongs to "remote.h" together
with "struct refspec".

While we are at it, also move the types and functions related to the
Git transport connection to a new header file connect.h

Signed-off-by: Junio C Hamano <gitster@pobox.com>
builtin/push.c: use OPT_BOOL, not OPT_BOOLEAN

The command line parser of "git push" for "--tags", "--delete", and
"--thin" options still used outdated OPT_BOOLEAN.  Because these
options do not give escalating levels when given multiple times,
they should use OPT_BOOL.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
http.c: fix parsing of http.sslCertPasswordProtected variable

The existing code triggers only when the configuration variable is
set to true.  Once the variable is set to true in a more generic
configuration file (e.g. ~/.gitconfig), it cannot be overriden to
false in the repository specific one (e.g. .git/config).

Signed-off-by: Junio C Hamano <gitster@pobox.com>
config: add support for http.<url>.* settings

The credentials configuration values already support url-specific
configuration items in the form credential.<url>.*.  This patch
adds similar support for http configuration values.

The <url> value is considered a match to a url if the <url> value
is either an exact match or a prefix of the url which ends on a
path component boundary ('/').  So "https://example.com/test" will
match "https://example.com/test" and "https://example.com/test/too"
but not "https://example.com/testextra".

Longer matches take precedence over shorter matches with
environment variable settings taking precedence over all.

With this configuration:

[http]
        useragent = other-agent
        noEPSV = true
[http "https://example.com"]
        useragent = example-agent
        sslVerify = false
[http "https://example.com/path"]
        useragent = path-agent

The "https://other.example.com/" url will have useragent
"other-agent" and sslVerify will be on.

The "https://example.com/" url will have useragent
"example-agent" and sslVerify will be off.

The "https://example.com/path/sub" url will have useragent
"path-agent" and sslVerify will be off.

All three of the examples will have noEPSV enabled.

Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
config: improve support for http.<url>.* settings

Improve on the http.<url>.* url matching behavior by first
normalizing the urls before they are compared.

With this change, for example, the following configuration
section:

[http "https://example.com/path"]
        useragent = example-agent
        sslVerify = false

will properly match a "HTTPS://example.COM/p%61th" url which
is equivalent.

The normalization rules are based on RFC 3986 and should
result in any two equivalent urls being a match.

Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
tests: add new test for the url_normalize function

In order to perform sane URL matching for http.<url>.* options,
http.c normalizes URLs before performing matches.

A new test-url-normalize test program is introduced along with
a new t5200-url-normalize.sh script to run the tests.

Since the url_normalize function currently lives in http.c this
test will be skipped if NO_CURL is defined since http.c is skipped
in that case.

Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
config: allow http.<url>.* any user matching

Previously the <url> had to specify an exactly matching user name
and password if those were present in the url being matched against.

Now the password portion is always ignored and omitting the user
name from <url> allows it to match against any user name.

Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
remote.c: add command line option parser for "--force-with-lease"

Update "git push" and "git send-pack" to parse this commnd line
option.

The intended sematics is:

 * "--force-with-lease" alone, without specifying the details, will
   protect _all_ remote refs that are going to be updated by
   requiring their current value to be the same as some reasonable
   default, unless otherwise specified;

 * "--force-with-lease=refname", without specifying the expected
   value, will protect that refname, if it is going to be updated,
   by requiring its current value to be the same as some reasonable
   default.

 * "--force-with-lease=refname:value" will protect that refname, if
   it is going to be updated, by requiring its current value to be
   the same as the specified value; and

 * "--no-force-with-lease" will cancel all the previous --force-with-lease on the
   command line.

For now, "some reasonable default" is tentatively defined as "the
value of the remote-tracking branch we have for the ref of the
remote being updated", and it is an error if we do not have such a
remote-tracking branch.  But this is known to be fragile, its use is
not yet recommended, and hopefully we will find more reasonable
default as we gain experience with this feature.  The manual marks
the feature as experimental unless the expected value is specified
explicitly for this reason.

Because the command line options are parsed _before_ we know which
remote we are pushing to, there needs further processing to the
parsed data after we instantiate the transport object to:

 * expand "refname" given by the user to a full refname to be
   matched with the list of "struct ref" used in match_push_refs()
   and set_ref_status_for_push(); and

 * learning the actual local ref that is the remote-tracking branch
   for the specified remote ref.

Further, some processing need to be deferred until we find the set
of remote refs and match_push_refs() returns in order to find the
ones that need to be checked after explicit ones have been processed
for "--force-with-lease" (no specific details).

These post-processing will be the topic of the next patch.

This option was originally called "cas" (for "compare and swap"),
the name which nobody liked because it was too technical.  The
second attempt called it "lockref" (because it is conceptually like
pushing after taking a lock) but the word "lock" was hated because
it implied that it may reject push by others, which is not the way
this option works.  This round calls it "force-with-lease".  You
assume you took the lease on the ref when you fetched to decide what
the rebased history should be, and you can push back only if the
lease has not been broken.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
push --force-with-lease: implement logic to populate old_sha1_expect[]

This plugs the push_cas_option data collected by the command line
option parser to the transport system with a new function
apply_push_cas(), which is called after match_push_refs() has
already been called.

At this point, we know which remote we are talking to, and what
remote refs we are going to update, so we can fill in the details
that may have been missing from the command line, such as

 (1) what abbreviated refname the user gave us matches the actual
     refname at the remote; and

 (2) which remote-tracking branch in our local repository to read
     the value of the object to expect at the remote.

to populate the old_sha1_expect[] field of each of the remote ref.
As stated in the documentation, the use of remote-tracking branch
as the default is a tentative one, and we may come up with a better
logic as we gain experience.

Still nobody uses this information, which is the topic of the next
patch.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
push --force-with-lease: tie it all together

This teaches the deepest part of the callchain for "git push" (and
"git send-pack") to enforce "the old value of the ref must be this,
otherwise fail this push" (aka "compare-and-swap" / "--lockref").

Signed-off-by: Junio C Hamano <gitster@pobox.com>
t5533: test "push --force-with-lease"

Prepare two repositories, src and dst, the latter of which is a
clone of the former (with tracking branches), and push from the
latter into the former, with various --force-with-lease options.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
t7600: fix typo in test title

Spotted by Ram, confirmed by Miklos.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
Merge branch 'mv/merge-ff-tristate'

* mv/merge-ff-tristate:
  t7600: fix typo in test title
rm: do not set a variable twice without intermediate reading.

Just the next line assigns a non-null value to seen.

Signed-off-by: Stefan Beller <stefanbeller@googlemail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
open_istream: remove unneeded check for null pointer

'st' is allocated via xmalloc a few lines before and passed to
the stream opening functions.
The xmalloc function is written in a way that either 'st' is allocated
valid memory or xmalloc already dies.
The function calls to open_istream_* do not change 'st', as the pointer is
passed by reference and not a pointer of a pointer.

Hence 'st' cannot be null at that part of the code.

Signed-off-by: Stefan Beller <stefanbeller@googlemail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
range-set: fix sort_and_merge_range_set() corner case bug

When handed an empty range_set (range_set.nr == 0),
sort_and_merge_range_set() incorrectly sets range_set.nr to 1 at exit.
Subsequent range_set functions then access the bogus range at element
zero and crash or throw an assertion failure. Fix this bug.

Signed-off-by: Eric Sunshine <sunshine@sunshineco.com>
Acked-by: Thomas Rast <trast@inf.ethz.ch>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
t4211: demonstrate empty -L range crash

Signed-off-by: Eric Sunshine <sunshine@sunshineco.com>
Acked-by: Thomas Rast <trast@inf.ethz.ch>
Helped-by: SZEDER Gábor <szeder@ira.uka.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
t4211: demonstrate crash when first -L encountered is empty range

Signed-off-by: Eric Sunshine <sunshine@sunshineco.com>
Acked-by: Thomas Rast <trast@inf.ethz.ch>
Helped-by: SZEDER Gábor <szeder@ira.uka.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
range-set: satisfy non-empty ranges invariant

range-set invariants are: ranges must be (1) non-empty, (2) disjoint,
(3) sorted in ascending order.

During processing, various range-set utility functions break the
invariants (for instance, by adding empty ranges), with the
expectation that a finalizing sort_and_merge_range_set() will restore
sanity.

sort_and_merge_range_set(), however, neglects to fold out empty
ranges, thus it fails to satisfy the non-empty constraint. Subsequent
range-set functions crash or throw an assertion failure upon
encountering such an anomaly. Rectify the situation by having
sort_and_merge_range_set() fold out empty ranges.

Signed-off-by: Eric Sunshine <sunshine@sunshineco.com>
Acked-by: Thomas Rast <trast@inf.ethz.ch>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
line-log: fix "log -LN" crash when N is last line of file

range-set invariants are: ranges must be (1) non-empty, (2) disjoint,
(3) sorted in ascending order.

line_log_data_insert() breaks the non-empty invariant under the
following conditions: the incoming range is empty and the pathname
attached to the range has not yet been encountered. In this case,
line_log_data_insert() assigns the empty range to a new line_log_data
record without taking any action to ensure that the empty range is
eventually folded out.  Subsequent range-set functions crash or throw an
assertion failure upon encountering such an anomaly.  Fix this bug.

Signed-off-by: Eric Sunshine <sunshine@sunshineco.com>
Acked-by: Thomas Rast <trast@inf.ethz.ch>
Signed-off-by: Junio C Hamano <gitster@pobox.com>