Git, weird issue - Page 1
Forum section: electronics / general

author: DiTBho
author_rank: Super Contributor
timestamp: November 06, 2023, 11:35:20 am
content: Code:git clone https://github.com/...Cloningremote: Enumerating objects: 71526, done.remote: Counting objects: 100% (3944/3944), done.remote: Compressing objects: 100% (2518/2518), done.error: RPC failed; result=18, HTTP code = 200iB | 1.01 MiB/sfatal: The remote end hung up unexpectedlyfatal: early EOFfatal: index-pack failedAny idea?

author: DiTBho
author_rank: Super Contributor
timestamp: November 06, 2023, 11:48:28 am
content: Foundthis, going to investigate and try it.

author: golden_labels
author_rank: Super Contributor
timestamp: November 07, 2023, 11:21:38 pm
content: Seems like not the git client issue, but your connection to GitHub. At any stage: from GitHub itself misbehaving, through something inbetween interferring with the connection, ending with overzealous antivirus/firewall software.I know this doesn’t solve your problem, but hopefully it will save your time by cutting down possibilities.

author: magic
author_rank: Super Contributor
timestamp: November 08, 2023, 07:56:11 am
content: Quote from: DiTBho on November 06, 2023, 11:48:28 amIt quite possibly has nothing to do with your issue.When will people learn to always look at thefirstthing which went wrong?editThe connection is over HTTPS so I doubt it's antivirus or firewall.The juxtaposition of "error" and "HTTP code 200" looks weird. I don't know the git protocol, but it probably means some error reported by the server, which may or may not be triggered by some avoidable behavior of the client.

author: Nominal Animal
author_rank: Super Contributor
timestamp: November 08, 2023, 08:21:34 am
content: One possibility is RST packet injection by an internet service provider in between. It has been used variously in Europe, mostly to combat file sharing (even when completely legal, to reduce bandwidth use), and the git HTTPS connections might look similar enough to trigger that.Both the libcurl error code (18, CURLE_PARTIAL_FILE), and the early EOF, indicate the connection was broken mid-transfer. I have personally only seen that due to network hiccups (connection restored in seconds to minutes), actual connection loss (server or network failure), and RST packet injection.TCP RST packet injection is a simple tool for the ISP to break existing connections they deem suspicious or has transferred too much data. Because github is sometimes used to store large data dumps (consider Wikileaks), some ISP's consider it (and places like sourceforge mirrors) "suspicious", worthy of breaking large/long-duration transfers, by injecting TCP RST packets. Of course, they do try hard to avoid doing that to approved internet use, like viewing Youtube, or downloading content from a whitelist of pre-approved large content delivery networks.

author: SiliconWizard
author_rank: Super Contributor
timestamp: November 08, 2023, 08:46:43 am
content: There may be some starting points there:https://stackoverflow.com/questions/26197171/git-clone-repository-error-rpc-failed-result-56-http-code-200

author: DiTBho
author_rank: Super Contributor
timestamp: November 08, 2023, 09:55:47 am
content: Quote from: Nominal Animal on November 08, 2023, 08:21:34 amdamn, that's precicely what happenedafter trying several approaches, including this setupCode:# core.compression# An integer -1..9, indicating a default compression level. -1 is the zlib default.# 0 means no compression, and 1..9 are various speed/size tradeoffs, 9 being slowest.# If set, this provides a default to other compression variables# such as core.looseCompression and pack.compression.# compression = 0# compression = 9compression = -1packedGitLimit = 512mpackedGitWindowSize = 512mdeltaCacheSize = 2047mpackSizeLimit = 2047mwindowMemory = 2047mwhich worked for a while, then randomly failed, always at different % progress, I moved the mac-mini/G4 from home to work, so same hardware, same kernel (GNU/Linux v4), same user space (Gentoo/PPC32BE-2020, server-profile/glibc), I repeated the experiment 10 times, and it worked perfectly!X____X

author: Nominal Animal
author_rank: Super Contributor
timestamp: November 08, 2023, 10:33:40 am
content: Bandwidth-limiting your traffic can help, and you can download the initial tree as a ZIP file (since that can be restarted/continued even if the connection is RST'd) but really, the only recourse is to either get your ISP to stop, or switch ISPs.Submitting a helpdesk/maintenance/support ticket along the lines ofQuotemight help, but considering today's world, you're more likely to get an answer suggesting you switch to using Facebook or Reddit for your software development work instead, these already having been pre-whitelisted.Using a VPN won't help, because all TCP traffic, encrypted or not, can trivially be disrupted via TCP RST by your ISP (as it can simply set the RST header bit in one of the TCP packets it is conveying, dropping the data payload). Unencrypted connections to port 80 are often exempted from RST-killing, because too many customers would complain about "internet not working".

author: magic
author_rank: Super Contributor
timestamp: November 08, 2023, 04:18:51 pm
content: Quote from: Nominal Animal on November 08, 2023, 08:21:34 amOne learns something new every dayI would never guess that this code came from curl. They surely made it sound like it's some remote error 18, successfully delivered to the client over HTTP.

author: golden_labels
author_rank: Super Contributor
timestamp: November 08, 2023, 09:12:40 pm
content: Quote from: magic on November 08, 2023, 07:56:11 amAV/FW software packages happen to MitM encrypted connections by installing their own certificates. Even without that TLS only provides a secure channel for data: it does not protect metadata.Quote from: magic on November 08, 2023, 04:18:51 pmThe clue here is the EOF part. It means the connection has been closed and that unexpected termination itself is the problem.

author: Nominal Animal
author_rank: Super Contributor
timestamp: November 09, 2023, 01:50:44 am
content: Yup. The wonky error message is generated inremote-curl.c:run_slot().If one does a web search for the error message, almost all of them misunderstand what is going on, even though it is pretty common (compared to other errors when using git). I wonder if sending a request for comments for a patch changingremote-curl.c:run_slot()into something likeCode:static int run_slot(struct active_request_slot *slot,struct slot_results *results){int err;struct slot_results results_buf;if (!results)results = &results_buf;err = run_one_slot(slot, results);if (err == HTTP_OK || err == HTTP_REAUTH)return err;/** RPC failed. Generate an understandable error message* for the case where the connection was unexpectedly closed,* for example due to TCP RST packet injection.*/if (err == HTTP_ERROR && results->http_code == 200 &&results->curl_result == CURLE_PARTIAL_FILE) {error(_("Connection closed unexpectedly; TCP RST injection?"));return err;}/** Default error reporting for all other errors.*/struct strbuf msg = STRBUF_INIT;if (results->http_code && results->http_code != 200)strbuf_addf(&msg, "HTTP %ld", results->http_code);if (results->curl_result != CURLE_OK) {if (msg.len)strbuf_addch(&msg, ' ');strbuf_addf(&msg, "curl %d", results->curl_result);if (curl_errorstr) {strbuf_addch(&msg, ' ');strbuf_addstr(&msg, curl_errorstr);}}error(_("RPC failed; %s"), msg.buf);strbuf_release(&msg);return err;}would be useful for users, possibly adding more dedicated cases for specific othercurl errors, specificallyCURLE_RECV_ERROR(indicating a problem in the local network stack), and explaining how no local or configuration changes can help with this particular error (contrary to many, many reports one can find via web searches), because it is either generated by the server, or by a router in between. It cannot happen due to just a sudden loss of connection, temporary or permanent, because it can only be due to receiving an explicit end-of-connection packet, TCP RST, which intermediate devices are supposed to not generate, but many ISPs use in an effort to combat file sharing, legal or otherwise. (Hardware connection loss at any point will show up as a timeout-related error instead, with the transfer "hanging", which is easy to detect by a human.)Typical HTTP errors are already handled separately inhttp.c:handle_curl_result()called byhttp.c:run_one_slot()(causing the aboveremote-curl.c:run_slot()seeerr = HTTP_ERROR, and printing the additional error message). Putting simplified error reports intoremote-curl.c:run_slot()means only the simplified error message is shown, adding the likelihood that human users understand the cause of the error right, and don't get confused by the additional error information (which is useless in this particular error condition, due to the very limited set of error causes).I'm personally not mentally prepared to deal with the social stuff in pushing changes to other peoples' code right now, especially because bringing up the habit of some ISPs using TCP RST injection to "combat file sharing" will cause a nasty chain at thegit@vger.kernel.orgmailing list, where one should be subscribed before submitting patches for comments or consideration. As you can see from my suggested ISP "support ticket" above, I'd just pour fuel to the flames, because this annoys me so much (having gone through this myself a decade ago, downloading Linux distro DVD images). Hell, I can't even do my favourite hobbies, like drive-by bug fix submissions to open source projects right now, for the same reason.

author: magic
author_rank: Super Contributor
timestamp: November 09, 2023, 07:02:05 am
content: IMO the first good step would be indicating that it's a curl failure and maybe translating the code into human readable form."RPC failed, result=18, HTTP code=200" very much sounds like an RPC has been called and sucessfully returned failure code 18 with HTTP status 200. Then the server closed connection.I don't know what those guys were smoking

author: Nominal Animal
author_rank: Super Contributor
timestamp: November 09, 2023, 08:14:16 am
content: Quote from: magic on November 09, 2023, 07:02:05 amEasy to do, just modify the aboveremote-curl.c:run_slot()function. Here's the original; it's been like this for quite a long time:Code:static int run_slot(struct active_request_slot *slot,struct slot_results *results){int err;struct slot_results results_buf;if (!results)results = &results_buf;err = run_one_slot(slot, results);if (err != HTTP_OK && err != HTTP_REAUTH) {struct strbuf msg = STRBUF_INIT;if (results->http_code && results->http_code != 200)strbuf_addf(&msg, "HTTP %ld", results->http_code);if (results->curl_result != CURLE_OK) {if (msg.len)strbuf_addch(&msg, ' ');strbuf_addf(&msg, "curl %d", results->curl_result);if (curl_errorstr) {strbuf_addch(&msg, ' ');strbuf_addstr(&msg, curl_errorstr);}}error(_("RPC failed; %s"), msg.buf);strbuf_release(&msg);}return err;}It uses astruct strbufto construct the message, before logging it. Very straightforward code, especially considering most HTTP status errors have already been reported (and arrive here witherr = HTTP_ERROR).In Git 2.0 (circa 2014), it was even simpler:Code:static int run_slot(struct active_request_slot *slot,struct slot_results *results){int err;struct slot_results results_buf;if (!results)results = &results_buf;err = run_one_slot(slot, results);if (err != HTTP_OK && err != HTTP_REAUTH) {error("RPC failed; result=%d, HTTP code = %ld",results->curl_result, results->http_code);}return err;}but still the same fundamentally indecipherable message, unless you already know about it. The newer version, as you can see, does at least hint that the '18' error code is from curl.So, if DiTBho were to update their git client to somewhat more recent version, the error message would already have been clearerabout the source being libcurl.Quote from: magic on November 09, 2023, 07:02:05 amI know the history, so I can say they never thought about it, because tothemthe error is unique (no other code generates an "RPC failed" error) and easily parsed. It takes someone outside the developer circle to note how uninformative (and, stupid) the error message is. (And more social skills than I can apply right now to describe the situation when submitting such a change to them.)It becomes even funnier when you realize that the gettext message identifier is"RPC failed; %s", meaning the actual message part cannot be translated at all. (My suggested version is, of course. Later versions of git use the common_(message)gettext macro for i18n.)