GitHub PushEvent at 2014-09-27 00:11:48
Repository: git
Actor: peff
Payload: {"shas":[["9540ce5030853ffbb7e11c30aa59a5e45095d32c","696e3fbcf235d40b6ea1ebd61c87cbea79d444e7@peff.net","refs: write packed_refs file using stdio\n\nWe write each line of a new packed-refs file individually\nusing a write() syscall (and sometimes 2, if the ref is\npeeled). Since each line is only about 50-100 bytes long,\nthis creates a lot of system call overhead.\n\nWe can instead open a stdio handle around our descriptor and\nuse fprintf to write to it. The extra buffering is not a\nproblem for us, because nobody will read our new packed-refs\nfile until we call commit_lock_file (by which point we have\nflushed everything).\n\nOn a pathological repository with 8.5 million refs, this\ndropped the time to run `git pack-refs` from 20s to 6s.\n\nSigned-off-by: Jeff King <peff@peff.net>\nReviewed-by: Michael Haggerty <mhagger@alum.mit.edu>\nSigned-off-by: Junio C Hamano <gitster@pobox.com>","Jeff King",false],["fe8e3b71805cd13d139b62fa5a0c75387568c9ea","53fb8db7833fca1e1746dd8592f61048d350f64c@gmx.de","Refactor type_from_string() to allow continuing after detecting an error\n\nIn the next commits, we will enhance the fsck_tag() function to check\ntag objects more thoroughly. To this end, we need a function to verify\nthat a given string is a valid object type, but that does not die() in\nthe negative case.\n\nWhile at it, prepare type_from_string() for counted strings, i.e. strings\nwith an explicitly specified length rather than a NUL termination.\n\nSigned-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>\nSigned-off-by: Junio C Hamano <gitster@pobox.com>","Johannes Schindelin",false],["90a398bbd72477d5d228818db5665fdfcf13431b","53fb8db7833fca1e1746dd8592f61048d350f64c@gmx.de","fsck_object(): allow passing object data separately from the object itself\n\nWhen fsck'ing an incoming pack, we need to fsck objects that cannot be\nread via read_sha1_file() because they are not local yet (and might even\nbe rejected if transfer.fsckobjects is set to 'true').\n\nFor commits, there is a hack in place: we basically cache commit\nobjects' buffers anyway, but the same is not true, say, for tag objects.\n\nBy refactoring fsck_object() to take the object buffer and size as\noptional arguments -- optional, because we still fall back to the\nprevious method to look at the cached commit objects if the caller\npasses NULL -- we prepare the machinery for the upcoming handling of tag\nobjects.\n\nThe assumption that such buffers are inherently NUL terminated is now\nwrong, of course, hence we pass the size of the buffer so that we can\nadd a sanity check later, to prevent running past the end of the buffer.\n\nSigned-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>\nSigned-off-by: Junio C Hamano <gitster@pobox.com>","Johannes Schindelin",false],["4d0d89755e82c40df88cf94d84031978f8eac827","53fb8db7833fca1e1746dd8592f61048d350f64c@gmx.de","Make sure fsck_commit_buffer() does not run out of the buffer\n\nSo far, we assumed that the buffer is NUL terminated, but this is not\na safe assumption, now that we opened the fsck_object() API to pass a\nbuffer directly.\n\nSo let's make sure that there is at least an empty line in the buffer.\nThat way, our checks would fail if the empty line was encountered\nprematurely, and consequently we can get away with the current string\ncomparisons even with non-NUL-terminated buffers are passed to\nfsck_object().\n\nSigned-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>\nSigned-off-by: Junio C Hamano <gitster@pobox.com>","Johannes Schindelin",false],["cec097be3a98be3773ddadf6824de03b849758d7","53fb8db7833fca1e1746dd8592f61048d350f64c@gmx.de","fsck: check tag objects' headers\n\nWe inspect commit objects pretty much in detail in git-fsck, but we just\nglanced over the tag objects. Let's be stricter.\n\nSince we do not want to limit 'tag' lines unduly, values that would fail\nthe refname check only result in warnings, not errors.\n\nSigned-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>\nSigned-off-by: Junio C Hamano <gitster@pobox.com>","Johannes Schindelin",false],["b64a98460682366d7533ea251d26026b85bb39cb","a6723cc3f76163bf7adb636a73ac3b0ceb3e6b9b@pobox.com","hash-object: reduce file-scope statics\n\nMost of the knobs that affect helper functions called from\ncmd_hash_object() were passed to them as parameters already, and the\nonly effect of having them as file-scope statics was to make the\nreader wonder if the parameters are hiding the file-scope global\nvalues by accident.  Adjust their initialisation and make them\nfunction-local variables.\n\nThe only exception was no_filters hash_stdin_paths() peeked from the\nfile-scope global, which was converted to a parameter to the helper\nfunction.\n\nSigned-off-by: Junio C Hamano <gitster@pobox.com>","Junio C Hamano",false],["17b787f6031d72619e4e2982ea84f47fe161cb30","a6723cc3f76163bf7adb636a73ac3b0ceb3e6b9b@pobox.com","hash-object: pass 'write_object' as a flag\n\nInstead of forcing callers of lower level functions write\n(write_object ? HASH_WRITE_OBJECT : 0), prepare the flag to be\npassed down in the callchain from the command line parser.\n\nSigned-off-by: Junio C Hamano <gitster@pobox.com>","Junio C Hamano",false],["90e3e5f0574480cb873cca1c7b968dd1516c05d2","53fb8db7833fca1e1746dd8592f61048d350f64c@gmx.de","Add regression tests for stricter tag fsck'ing\n\nThe intent of the new test case is to catch general breakages in\nthe fsck_tag() function, not so much to test it extensively, trying to\nstrike the proper balance between thoroughness and speed.\n\nWhile it *would* have been nice to test the code path where fsck_object()\nencounters an invalid tag object, this is not possible using git fsck: tag\nobjects are parsed already before fsck'ing (and the parser already fails\nupon such objects).\n\nEven worse: we would not even be able write out invalid tag objects\nbecause git hash-object parses those objects, too, unless we resorted to\nreally ugly hacks such as using something like this in the unit tests\n(essentially depending on Perl *and* Compress::Zlib):\n\n\thash_invalid_object () {\n\t\tcontents=\"$(printf '%s %d\\0%s' \"$1\" ${#2} \"$2\")\" &&\n\t\tsha1=$(echo \"$contents\" | test-sha1) &&\n\t\tsuffix=${sha1#??} &&\n\t\tmkdir -p .git/objects/${sha1%$suffix} &&\n\t\techo \"$contents\" |\n\t\tperl -MCompress::Zlib -e 'undef $/; print compress(<>)' \\\n\t\t\t> .git/objects/${sha1%$suffix}/$suffix &&\n\t\techo $sha1\n\t}\n\nSigned-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>\nSigned-off-by: Junio C Hamano <gitster@pobox.com>","Johannes Schindelin",false],["5ba9a93b39bef057be54ecf7933386a582981625","a6723cc3f76163bf7adb636a73ac3b0ceb3e6b9b@pobox.com","hash-object: add --literally option\n\nThis allows \"hash-object --stdin\" to just hash any garbage into a\n\"loose object\" that may not pass the standard object parsing check\nor fsck, so that different kind of corrupt objects we may encounter\nin the field can be imitated in our test suite.  That would in turn\nallow us to test features that catch these corrupt objects.\n\nNote that \"cat-file\" may need to learn \"--literally\" option to allow\nus peek into a truly broken object.\n\nSigned-off-by: Junio C Hamano <gitster@pobox.com>","Junio C Hamano",false],["f99b7af661f89865f918e52223a3bdaf312a0de0","53fb8db7833fca1e1746dd8592f61048d350f64c@gmx.de","Make sure that index-pack --strict checks tag objects\n\nOne of the most important use cases for the strict tag object checking\nis when transfer.fsckobjects is set to true to catch invalid objects\nearly on. This new regression test essentially tests the same code path\nby directly calling 'index-pack --strict' on a pack containing an\ntag object without a 'tagger' line.\n\nTechnically, this test is not enough: it only exercises a code path that\n*warns*, not one that *fails*. The reason is that hash-object and\npack-objects both insist on parsing the tag objects and would fail on\ninvalid tag objects at this time.\n\nSigned-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>\nSigned-off-by: Junio C Hamano <gitster@pobox.com>","Johannes Schindelin",false],["40e94ca19adf00caa3d19a199b9b0a6079d12a3a","a6723cc3f76163bf7adb636a73ac3b0ceb3e6b9b@pobox.com","Merge branch 'js/fsck-tag-validation' into HEAD\n\n* js/fsck-tag-validation:\n  Make sure that index-pack --strict checks tag objects\n  Add regression tests for stricter tag fsck'ing\n  fsck: check tag objects' headers\n  Make sure fsck_commit_buffer() does not run out of the buffer\n  fsck_object(): allow passing object data separately from the object itself\n  Refactor type_from_string() to allow continuing after detecting an error","Junio C Hamano",false],["b659605da66ffd513ae60e767495899507e8dfcb","a6723cc3f76163bf7adb636a73ac3b0ceb3e6b9b@pobox.com","t1450: make sure fsck detects a malformed tagger line\n\nWith \"hash-object --literally\", write a tag object that is not\nsupposed to pass one of the new checks added to \"fsck\", and make\nsure that the new check catches the breakage.\n\nSigned-off-by: Junio C Hamano <gitster@pobox.com>","Junio C Hamano",false],["cbe73331812ed0ac3c3b680ab3aab4e6d22a98ad","696e3fbcf235d40b6ea1ebd61c87cbea79d444e7@peff.net","refs: speed up is_refname_available\n\nOur filesystem ref storage does not allow D/F conflicts; so\nif \"refs/heads/a/b\" exists, we do not allow \"refs/heads/a\"\nto exist (and vice versa). This falls out naturally for\nloose refs, where the filesystem enforces the condition. But\nfor packed-refs, we have to make the check ourselves.\n\nWe do so by iterating over the entire packed-refs namespace\nand checking whether each name creates a conflict. If you\nhave a very large number of refs, this is quite inefficient,\nas you end up doing a large number of comparisons with\nuninteresting bits of the ref tree (e.g., we know that all\nof \"refs/tags\" is uninteresting in the example above, yet we\ncheck each entry in it).\n\nInstead, let's take advantage of the fact that we have the\npacked refs stored as a trie of ref_entry structs. We can\nfind each component of the proposed refname as we walk\nthrough the trie, checking for D/F conflicts as we go. For a\nrefname of depth N (i.e., 4 in the above example), we only\nhave to visit N nodes. And at each visit, we can binary\nsearch the M names at that level, for a total complexity of\nO(N lg M). (\"M\" is different at each level, of course, but\nwe can take the worst-case \"M\" as a bound).\n\nIn a pathological case of fetching 30,000 fresh refs into a\nrepository with 8.5 million refs, this dropped the time to\nrun \"git fetch\" from tens of minutes to ~30s.\n\nThis may also help smaller cases in which we check against\nloose refs (which we do when renaming a ref), as we may\navoid a disk access for unrelated loose directories.\n\nNote that the tests we add appear at first glance to be\nredundant with what is already in t3210. However, the early\ntests are not robust; they are run with reflogs turned on,\nmeaning that we are not actually testing\nis_refname_available at all! The operations will still fail\nbecause the reflogs will hit D/F conflicts in the\nfilesystem. To get a true test, we must turn off reflogs\n(but we don't want to do so for the entire script, because\nthe point of turning them on was to cover some other cases).\n\nReviewed-by: Michael Haggerty <mhagger@alum.mit.edu>\nSigned-off-by: Jeff King <peff@peff.net>\nSigned-off-by: Junio C Hamano <gitster@pobox.com>","Jeff King",false],["1cc2c772dd4dc5f9396a229e2727ca28ef25c33b","696e3fbcf235d40b6ea1ebd61c87cbea79d444e7@peff.net","prune-packed: fix minor memory leak\n\nWe form all of our directories in a strbuf, but never release it.\n\nSigned-off-by: Jeff King <peff@peff.net>\nSigned-off-by: Junio C Hamano <gitster@pobox.com>","Jeff King",false],["d38379ece9216735ecc0ffd76c4c4e3da217daec","696e3fbcf235d40b6ea1ebd61c87cbea79d444e7@peff.net","make update-server-info more robust\n\nSince \"git update-server-info\" may be called automatically\nas part of a push or a \"gc --auto\", we should be robust\nagainst two processes trying to update it simultaneously.\nHowever, we currently use a fixed tempfile, which means that\ntwo simultaneous writers may step on each other's toes and\nend up renaming junk into place.\n\nLet's instead switch to using a unique tempfile via mkstemp.\nWe do not want to use a lockfile here, because it's OK for\ntwo writers to simultaneously update (one will \"win\" the\nrename race, but that's OK; they should be writing the same\ninformation).\n\nWhile we're there, let's clean up a few other things:\n\n  1. Detect write errors. Report them and abort the update\n     if any are found.\n\n  2. Free path memory rather than leaking it (and clean up\n     the tempfile when necessary).\n\n  3. Use the pathdup functions consistently rather than\n     static buffers or manually calculated lengths.\n\nThis last one fixes a potential overflow of \"infofile\" in\nupdate_info_packs (e.g., by putting large junk into\n$GIT_OBJECT_DIRECTORY). However, this overflow was probably\nnot an interesting attack vector for two reasons:\n\n  a. The attacker would need to control the environment to\n     do this, in which case it was already game-over.\n\n  b. During its setup phase, git checks that the directory\n     actually exists, which means it is probably shorter\n     than PATH_MAX anyway.\n\nBecause both update_info_refs and update_info_packs share\nthese same failings (and largely duplicate each other), this\npatch factors out the improved error-checking version into a\nhelper function.\n\nSigned-off-by: Jeff King <peff@peff.net>\nSigned-off-by: Junio C Hamano <gitster@pobox.com>","Jeff King",false],["3907a4078a89cffde2e2e0e61a3bd41efba96c49","696e3fbcf235d40b6ea1ebd61c87cbea79d444e7@peff.net","server-info: clean up after writing info/packs\n\nWe allocate pack information in a static global list but\nnever clean it up. This leaks memory, and means that calling\nupdate_server_info twice will generate a buggy file (it will\nhave duplicate entries).\n\nSigned-off-by: Jeff King <peff@peff.net>\nSigned-off-by: Junio C Hamano <gitster@pobox.com>","Jeff King",false],["4489a480fdfe53e1ca78eabdab2dee00376814a3","2da278a99b557fc93ddbed7517a413ad793e2e6a@web.de","repack: call prune_packed_objects() and update_server_info() directly\n\nCall the functions behind git prune-packed and git update-server-info\ndirectly instead of using run_command().  This is shorter, easier and\nquicker.\n\nSigned-off-by: Rene Scharfe <l.s.r@web.de>\nSigned-off-by: Junio C Hamano <gitster@pobox.com>","René Scharfe",false],["f5e3c0b9d050ebdaf96d3910b01b01695e3ea1a2","696e3fbcf235d40b6ea1ebd61c87cbea79d444e7@peff.net","credential-cache: close stderr in daemon process\n\nIf the stderr of \"git credential-cache\" is redirected to a\npipe, the reader on the other end of a pipe may be surprised\nthat the pipe remains open long after the process exits.\nThis happens because we may auto-spawn a daemon which is\nlong-lived, and which keeps stderr open.\n\nWe can solve this by redirecting the daemon's stderr to\n/dev/null once we are ready to go into our event loop. We\nwould not want to do so before then, because we may want to\nreport errors about the setup (e.g., failure to establish\nthe listening socket).\n\nThis does mean that we will not report errors we encounter\nfor specific clients. That's acceptable, as such errors\nshould be rare (e.g., clients sending buggy requests).\nHowever, we also provide an escape hatch: if you want to see\nthese later messages, you can provide the \"--debug\" option\nto keep stderr open.\n\nSigned-off-by: Jeff King <peff@peff.net>\nSigned-off-by: Junio C Hamano <gitster@pobox.com>","Jeff King",false],["ec7dbd145bd8ea27d958dcb16e19b5f0ef3fd643","a6723cc3f76163bf7adb636a73ac3b0ceb3e6b9b@pobox.com","receive-pack: allow hooks to ignore its standard input stream\n\nThe pre-receive and post-receive hooks were designed to be an\nimprovement over old style update and post-update hooks, which take\nthe update information on their command line and are limited by the\ncommand line length limit.  The same information is fed from the\nstandard input to pre/post-receive hooks instead to lift this\nlimitation.  It has been mandatory for these new style hooks to\nconsume the update information fully from the standard input stream.\nOtherwise, they would risk killing the receive-pack process via\nSIGPIPE.\n\nIf a hook does not want to look at all the information, it is easy\nto send its standard input to /dev/null (perhaps a niche use of hook\nmight need to know only the fact that a push was made, without\nhaving to know what objects have been pushed to update which refs),\nand this has already been done by existing hooks that are written\ncarefully.\n\nHowever, because there is no good way to consistently fail hooks\nthat do not consume the input fully (a small push may result in a\nshort update record that may fit within the pipe buffer, to which\nthe receive-pack process may manage to write before the hook has a\nchance to exit without reading anything, which will not result in a\ndeath-by-SIGPIPE of receive-pack), it can lead to a hard to diagnose\n\"once in a blue moon\" phantom failure.\n\nLift this \"hooks must consume their input fully\" mandate.  A mandate\nthat is not enforced strictly is not helping us to catch mistakes in\nhooks.  If a hook has a good reason to decide the outcome of its\noperation without reading the information we feed it, let it do so\nas it pleases.\n\nSigned-off-by: Junio C Hamano <gitster@pobox.com>","Junio C Hamano",false],["3ac22f82edec108986b2627fcd6f2a8039617a23","2da278a99b557fc93ddbed7517a413ad793e2e6a@web.de","add macro REALLOC_ARRAY\n\nThe macro ALLOC_GROW manages several aspects of dynamic memory\nallocations for arrays: It performs overprovisioning in order to avoid\nreallocations in future calls, updates the allocation size variable,\nmultiplies the item size and thus allows users to simply specify the\nitem count, performs the reallocation and updates the array pointer.\n\nSometimes this is too much.  Add the macro REALLOC_ARRAY, which only\ntakes care of the latter three points and allows users to specfiy the\nnumber of items the array can store.  It can increase and also decrease\nthe size.  Using the macro avoid duplicating the variable name and\ntakes care of the item sizes automatically.\n\nSigned-off-by: Rene Scharfe <l.s.r@web.de>\nSigned-off-by: Junio C Hamano <gitster@pobox.com>","René Scharfe",false]],"size":38,"ref":"refs/heads/jk/stash-apply-p-wip","head":"7031c3307bdb2bc199be047d83397402308ee2b5"}