Repository: xroche/httrack (2026-06-19T16:19:39Z)

Issue:
CLI insistently requests schemas, which halts parsing
I installed the CLI for `httrack` using [Homebrew](https://brew.sh). Many of the sites I try to mirror contain some `xmlns:` link attributes in their main `<html>` tags (e.g., `xmlns:og=`, `xmlns:rdfs=`). For these sites, HTTrack spends all its time crawling those format specification URIs, rather than the site itself; and the request always takes forever, which is surprising considering most of these files are mere kilobytes in size.

I’ve tried more options than I can recall, including [`--priority=1`, `--priority=7`, `--can-go-down`, and `--stay-on-same-address`](https://github.com/xroche/httrack/blob/fdd900b9c77e8765c00216b50f0b06f70990ae4d/man/httrack.1#L384-L414), but nothing works. I’m not sure what MIME type these links are, but if I knew, I could try [`--disable-module`](https://github.com/xroche/httrack/blob/fdd900b9c77e8765c00216b50f0b06f70990ae4d/man/httrack.1#L335-L336). Regardless, I’m not sure why HTTrack ignores domain restrictions for these particular URIs. The example URI for reproduction is `ethnologue.com/21`, and the problem persists regardless of whether its `robots.txt` is respected.


Discussion:
I am apparently affected by a similar issue on linux CLI, in variuos attempts to dump a website it hangs forever printing after printing this last message
`https://www.w3.org/2018/json-ld-wg/ (9987 bytes) - OK`

the command I use is `httrack --continue --ext-depth=1 "https://XXXXXXXXXX/"`

Cheers!
When I press `Ctrl+c` to stop it I get this message `** Finishing pending transfers.. press again ^C to quit.` unless I press `Ctrl+c` again it stays like that forever
This is what I found in the last lines of the log

```
$ tail hts-log.txt 
17:03:03        Warning:        engine: warning: temporary file en.wikipedia.org/wiki/Internet_of_things.html.tmp already exists
17:03:04        Warning:        engine: warning: temporary file en.wikipedia.org/w/indexa116.html.tmp already exists
17:03:05        Warning:        engine: warning: temporary file en.wikipedia.org/wiki/Hypertext_Application_Language.html.tmp already exists
17:03:06        Warning:        engine: warning: temporary file en.wikipedia.org/w/indexbfd3.html.tmp already exists
17:03:09        Warning:        engine: warning: temporary file json-ld.org/spec/latest/json-ld-syntax/index.html.tmp already exists
17:03:23        Warning:        engine: warning: temporary file www.w3.org/2011/rdf-wg/wiki/Main_Page.html.tmp already exists
17:04:17        Warning:        engine: warning: temporary file www.w3.org/TR/2014/REC-json-ld-20140116/index.html.tmp already exists
17:04:17        Warning:        engine: warning: temporary file json-ld.org/index.html.tmp already exists
17:04:17        Warning:        engine: warning: temporary file www.w3.org/2018/json-ld-wg/index.html.tmp already exists
02:28:13        Error:  Exit requested by shell or user
```



Patch:
diff --git a/src/htsparse.c b/src/htsparse.c
index 53cf5e1c9..1a194affd 100644
--- a/src/htsparse.c
+++ b/src/htsparse.c
@@ -1692,6 +1692,24 @@ int htsparse(htsmoduleStruct * str, htsmoduleStructExtended * stre) {
                                                               hts_nodetect[i -
                                                                            1]);
                                               }
+                                              // xmlns / xmlns:prefix declare
+                                              // XML namespaces, not resources
+                                              // (#191)
+                                              else {
+                                                const int xl = strfield(
+                                                    intag_startattr, "xmlns");
+                                                const char xc =
+                                                    intag_startattr[xl];
+                                                if (xl &&
+                                                    (xc == ':' || xc == '=' ||
+                                                     is_space(xc))) {
+                                                  url_ok = 0;
+                                                  hts_log_print(
+                                                      opt, LOG_DEBUG,
+                                                      "dirty parsing: xmlns "
+                                                      "namespace avoided");
+                                                }
+                                              }
                                             }
                                   }
 


Test patch:
diff --git a/tests/01_engine-parse.test b/tests/01_engine-parse.test
index df46d60d1..d8e233cc4 100755
--- a/tests/01_engine-parse.test
+++ b/tests/01_engine-parse.test
@@ -154,4 +154,26 @@ grep -Eq "style=\"background-image:url\('ibgs\.gif'\)\"" "$saved2" ||
 grep -q 'title="file://' "$saved2" ||
     ! echo "FAIL: a no-detect attribute (title) was wrongly rewritten" || exit 1
 
+# xmlns / xmlns:prefix decls must not be crawled (#191). Local file:// targets so a
+# regression downloads them; each is the LAST attr (heuristic only scans a value before '>').
+site3="$tmp/xmlns"
+mkdir -p "$site3"
+for f in ns og rdfs real; do gif "$site3/$f.gif"; done
+cat >"$site3/index.html" <<EOF
+<html xmlns="file://$site3/ns.gif"><body>
+<svg xmlns:og="file://$site3/og.gif"></svg>
+<div class="c" xmlns:rdfs="file://$site3/rdfs.gif"></div>
+<a href="file://$site3/real.gif">real link</a>
+</body></html>
+EOF
+out3="$tmp/xmlns-out"
+crawl "$site3/index.html" "$out3"
+
+# the real link is still captured
+found "real.gif" "$out3"
+# namespace-declaration targets must not be fetched (default + prefixed forms)
+notfound "ns.gif" "$out3"
+notfound "og.gif" "$out3"
+notfound "rdfs.gif" "$out3"
+
 exit 0