jiebi/CodeConvo huggingface dataset:
Code change:
endif endif <ins> ifdef CFG_DISABLE_RPATH ifeq ($$(OSTYPE_$(3)),apple-darwin) RPATH_VAR$(1)_T_$(2)_H_$(3) :=  DYLD_LIBRARY_PATH="$$$$DYLD_LIBRARY_PATH:$$(HLIB$(1)_H_$(3))" else RPATH_VAR$(1)_T_$(2)_H_$(3) :=  LD_LIBRARY_PATH="$$$$LD_LIBRARY_PATH:$$(HLIB$(1)_H_$(3))" endif else RPATH_VAR$(1)_T_$(2)_H_$(3) := endif </ins> STAGE$(1)_T_$(2)_H_$(3) := 						 <del> $$(Q)$$(call CFG_RUN_TARG_$(3),$(1),				 </del> <ins> $$(Q)$$(RPATH_VAR$(1)_T_$(2)_H_$(3))                             $$(call CFG_RUN_TARG_$(3),$(1),				 </ins> $$(CFG_VALGRIND_COMPILE$(1)) 			 $$(HBIN$(1)_H_$(3))/rustc$$(X_$(3))			 --cfg $$(CFGFLAG$(1)_T_$(2)_H_$(3))			

Related discussion:
text: I think that I'll spend a bit more time with looking at and and might get a better feeling for the problem. Btw.: I noted that the libdir isn't configurabel. Fedora for example keeps x86_64 libs in (/usr)?/lib64, and i686 in (/usr)/lib. IIUIC this is currently not possible with rust as the libs will be installed to /usr/lib/arch-specific-path
The point about libdir is well taken and I updated . I am not entirely sure how rustpkg does it's build now or how and intend for it to work, but based on the current language design I would expect that, yes, binaries installed to would have rpaths pointing there. Currently there are several rpaths configured for each linked library so that they hopefully fall back to something useful when the libraries are moved around during install or during the bootstrapping process, something like: The relative path to the library The absolute path to where the library was at build time The system installation path Under this scheme I would expect that would, when linking against libraries in , create rpaths to .
Yes, there are several rpaths used (seen in and ) and I'd say - even with looking at rust itself, without the packaging glasses on - that it should be configurable what rpaths are used. Maybe it should be an option of . As said, system wide binaries are the best example for binaries where you don't want to many rpaths - including no rpaths pointing to some user dir.
Please don't tag TJC :(
Sorry, TJC! Maybe it's time to give in and start working on Rust :) Seriously though, sorry. I'm sure it's a real PITA.
mentioned that this is a security issue, which I didn't realize. Debian also says .
By the way, it's possible to strip this with but it would be much nicer to not require a third party tool.
I generally wonder why you don't add a user-specific to to point to the user specific rustpkg dir instead of using rpath. That makes it much easier to cope with - and that would also be in par with (at least) the Fedora guidelines. And I suppose with Debian's too, as it's the common way. (See 3rd party java binaries. They modify the to point to some hidden dir in the userdir.)
We used to.
Code change:
endif endif <ins> ifdef CFG_DISABLE_RPATH ifeq ($$(OSTYPE_$(3)),apple-darwin) RPATH_VAR$(1)_T_$(2)_H_$(3) :=  DYLD_LIBRARY_PATH="$$$$DYLD_LIBRARY_PATH:$$(HLIB$(1)_H_$(3))" else RPATH_VAR$(1)_T_$(2)_H_$(3) :=  LD_LIBRARY_PATH="$$$$LD_LIBRARY_PATH:$$(HLIB$(1)_H_$(3))" endif else RPATH_VAR$(1)_T_$(2)_H_$(3) := endif </ins> STAGE$(1)_T_$(2)_H_$(3) := 						 <del> $$(Q)$$(call CFG_RUN_TARG_$(3),$(1),				 </del> <ins> $$(Q)$$(RPATH_VAR$(1)_T_$(2)_H_$(3))                             $$(call CFG_RUN_TARG_$(3),$(1),				 </ins> $$(CFG_VALGRIND_COMPILE$(1)) 			 $$(HBIN$(1)_H_$(3))/rustc$$(X_$(3))			 --cfg $$(CFGFLAG$(1)_T_$(2)_H_$(3))			

Related discussion:
text: Users kept forgetting and/or getting annoyed having to adjust it when running from a build dir. Rpaths made it possible to run foo/bar/rustc no matter which rustc you chose.
right. IIUIC - I always feel like a noob when it comes to autotools - autotools are generating wrappers for binaries which are created during a build process and moves the original binaries to hiddend irectories. The wrappers are then modifying the environment (setting env vars etc) to achieve the same goal (so e.g. that rustc can be run from the dev directory without messing with the env vars yourself). I wonder if this could help.
Okay, it's actually not autools which creates these wrappers but libtool:
Sorry, I didn't mean to imply a disinterest in fixing this better, just noting how it came to be. Also strongly disinterested in using libtool. Would prefer going back to telling users to set env vars.
How about defaulting to static linking (when it works)? Dynamic linking is valuable, but mostly for global installs handled by a package manager. Of course, if we end up with a bunch of tools using that would be wasteful. I think RPATH will just lead to confusion, and it's an easy security hole to leave open.
I would be very happy to get static linking working well enough to make it a default in many such cases. But we ought to do the right thing when linking dynamically too. Is rpath-using- considered acceptable? I.e. is it just the absolute paths that are problematic?
The absolute paths are the ones that can easily be a security issue, so isn't that bad. I would still rather avoid it for a globally installed package but it's easy enough to strip for people who don't need it and won't be an issue if you don't know about it (like the absolute path could be). Using the rpath could be a flag, but the absolute path should never be there.
Nominating for milestone 2: backwards compatible. (The choice of whether one needs to put a setting in or use some configure flag for certain systems, etc, seems like a backwards compatibility issue. Though maybe I misunderstand and these are issues only for people developing itself, not for clients of ?) It also could well be that this is more appropriately assocated with milestone 1: well-defined.
Code change:
endif endif <ins> ifdef CFG_DISABLE_RPATH ifeq ($$(OSTYPE_$(3)),apple-darwin) RPATH_VAR$(1)_T_$(2)_H_$(3) :=  DYLD_LIBRARY_PATH="$$$$DYLD_LIBRARY_PATH:$$(HLIB$(1)_H_$(3))" else RPATH_VAR$(1)_T_$(2)_H_$(3) :=  LD_LIBRARY_PATH="$$$$LD_LIBRARY_PATH:$$(HLIB$(1)_H_$(3))" endif else RPATH_VAR$(1)_T_$(2)_H_$(3) := endif </ins> STAGE$(1)_T_$(2)_H_$(3) := 						 <del> $$(Q)$$(call CFG_RUN_TARG_$(3),$(1),				 </del> <ins> $$(Q)$$(RPATH_VAR$(1)_T_$(2)_H_$(3))                             $$(call CFG_RUN_TARG_$(3),$(1),				 </ins> $$(CFG_VALGRIND_COMPILE$(1)) 			 $$(HBIN$(1)_H_$(3))/rustc$$(X_$(3))			 --cfg $$(CFGFLAG$(1)_T_$(2)_H_$(3))			

Related discussion:
text: I suspect and have a more concrete notion about this.
I don't have anything much to add to this despite it coming up for triage today. I think it has to be fixed still, I'm still happy to do it via whatever combination of static linking by default and use of is sufficient for packagers, and I haven't seen any clear movement on it in the meantime.
accepted for backwards-compatible milestone
I'm getting worried that we won't resolve this for 1.0. I agree this is important to fix. Here's my current suggestion: Modify the build system to once again set LDLIBRARYPATH/PATH to find the appropriate dynamic libraries. Add a flag to rustc to disable rpath Add a configure flag to disable rpathing the rust build itself. Packagers can use this to do whatever they want. With this logic the rust package itself could avoid rpath where distros have their own way of dealing with library paths. The rustc installed by these distros would still be rpathing user's crates for the sake of convenience. Is this an acceptable compromise or do distros want us to completely disable rpath? I'm afraid that doing so would provide an inconsistent developer experience depending on how rustc is acquired; if we need to do that then we instead must find a way to completely eliminate rpath.
If we start to just statically link everything by default then rpath becomes much less important and we can just say it's up to users to figure it out if they opt into dynamic linking.
I think we should completely remove absolute rpaths. It's okay to have $ORIGIN-based ones but it may not be a good default. I'm not sure what the answer is to that, but agree that static linking basically resolves this.
It doesn't appear that this is fully working. There are still rpaths in the binaries when configured with even though the flag for rustc does work. /usr/bin/rustc: RPATH=$ORIGIN/../lib:/build/rust-git/src/rust/x8664-unknown-linux-gnu/stage1/lib/rustlib/x8664-unknown-linux-gnu/lib:/usr/lib/rustlib/x86_64-unknown-linux-gnu/lib
What steps did you follow to having the rpath show up? I thought I double checked on linux that the configure flag worked, but I may have overlooked something.
Code change:
[[package]] name = "unicase" <del> version = "2.6.0" </del> <ins> version = "2.7.0" </ins> source = "registry+https://github.com/rust-lang/crates.io-index" <del> checksum = "50f37be617794602aabbeee0be4f259dc1778fabe05e2d67ee8f79326d5cb4f6" </del> <ins> checksum = "f7d2d4dafb69621809a81864c9c1b864479e1235c0dd4e199924b9742439ed89" </ins> dependencies = [ "version_check", ]

Related discussion:
text: Unicode is released on a yearly basis, so we update the data files we used accordingly after each Unicode release in the Rust project. (Keep in mind that new dependencies might be over time.) Tracking issues are used to record the overall progress of implementation. They are also used as hubs connecting to other relevant issues, e.g., bugs or open design questions. A tracking issue is however not meant for large scale discussion, questions, or bug reports about a feature. Instead, open a dedicated issue for the specific matter and add the relevant feature gate label. Goal: Unicode 15.0 (year 2022) [x] Updating : [x] Verify whether each crate is Unicode version dependent and add support for the new Unicode version in compiler dependency libraries [x] Bump dependencies and ensure updated version is used in [x] [x] [x] (Decide whether it's a valid identifier) Current: 0.2.2 (Unicode 13) Goal: 0.2.4 (Unicode 15) [x] (Preprocess identifiers for equality) Current: 0.1.13 (Unicode 9) Goal: 0.1.22 (Unicode 15) [x] (Decide whether lints against unwanted usages should be triggered) Current: 0.0.5 (Unicode 13) Goal: 0.1.0 (Unicode 15) [x] (Used by for script detection) Current: 0.5.3 (Unicode 13) Goal: 0.5.5 (Unicode 15) [x] (used by , and many others) Current: 0.1.8 (Unicode 13) Goal: 0.1.10 (Unicode 15) [x] (used by ) Current: 0.1.0 (Unicode 15) Goal: 0.1.0 (Unicode 15) [x] Removed: [x] Removed: [x] Removed: [x] Removed: [x] Removed:[x] Verify whether each crate is Unicode version dependent and advocate support for the new Unicode version in dependency libraries [x] Bump dependencies and ensure updated version is used in [x] [x] [x] [x] (used by then then [, , , , , , , ]) Previously: 0.3.4 (Unicode 10) Goal: =0.3.
Code change:
[[package]] name = "unicase" <del> version = "2.6.0" </del> <ins> version = "2.7.0" </ins> source = "registry+https://github.com/rust-lang/crates.io-index" <del> checksum = "50f37be617794602aabbeee0be4f259dc1778fabe05e2d67ee8f79326d5cb4f6" </del> <ins> checksum = "f7d2d4dafb69621809a81864c9c1b864479e1235c0dd4e199924b9742439ed89" </ins> dependencies = [ "version_check", ]

Related discussion:
text: 10 (Unicode 15) [x] (used by ) Previously: 1.9.0 (Unicode 14) Goal: =1.10.0 (Unicode 15) [x] (used by ) Mentioned above in compiler diagnostics section [x] Removed: [x] (used by , then [, , ]) Current: 2.6.0 (Unclear, No release in 3 years) Goal: =2.7.0 (Unicode 15) [x] (in-tree maintainence tool): Unicode version independent [x] : Unicode version independent (used by tool) [x] : Unicode version independent (used by , then ) [x] , , , : Not really Unicode version independent but we only use Unicode version independent part. They're outdated, current: 0.9.1 (CLDR 37, Spring 2020, ~= Unicode 13), would be nice if a new release is used.
is , which is updated with the libcore update (or more specifically, this year it needed no update). is handled by
I happened to notice this in the UCD spec It seems that as of 15.0.0 have to start parsing some of the comments to look for decls now. This is pretty annoying and will likely lead to some bugs (likely minor). I'm certain I've written code that would be buggy under this. (Hell, I'm not sure I've ever seen code that handled these, and I've read a good number of UCD wrangler scripts -- usually it's just to say "assign the obvious default value to anything that doesn't get specified", as in the line above) It looks like the only files in UCD that take advantage of this (e.g. use it several times to define a full range as having the obvious default case) are , , and (note: this is just the UCD, and I haven't checked stuff like the UTS39 security data, though). This doesn't include anything we ship in the stdlib AFAIK, but almost certainly impacts some stuff in the compiler (in probably minor ways, if it's limited to the above). I suspect the argument is folks we were always doing this wrong if they ignored it though? Oof.
Code change:
[[package]] name = "unicase" <del> version = "2.6.0" </del> <ins> version = "2.7.0" </ins> source = "registry+https://github.com/rust-lang/crates.io-index" <del> checksum = "50f37be617794602aabbeee0be4f259dc1778fabe05e2d67ee8f79326d5cb4f6" </del> <ins> checksum = "f7d2d4dafb69621809a81864c9c1b864479e1235c0dd4e199924b9742439ed89" </ins> dependencies = [ "version_check", ]

Related discussion:
text: CC (since I suspect you know more details here, and even if not, I believe is yours), (since I noticed it while adding something to ucd-generate, presumably there are a few things in that might be impacted if you update its tables)
This seems like another reason for us to potentially switch over to ICU4X (maintained by Unicode), potentially with unicode-rs (or somewhere) maintaining crates of data output. Unfortunately it's unlikely will move to something heavier than the unicode-rs crates which have always stayed zero-dep sicne they're at the bottom of many deptrees. I don't have time to fixup unicode-width but would be happy to accept a patch.
I think rustc doing that is probably very good. It's hard for me to imagine anybody fighting you on it. For the tables... presumably you mean inside the table generator (definitely we couldn't add a dep to libcore on icu4x). We could use it to query the data we build the tables with, instead of ucd-parse. That doesn't sound like a problem eventually. (I'll look into it soonish since I'm messing around with Unicode table generation again). For the ecosystem at large, I dunno. I suspect it will be a tough sell, due to compile time, binary size, and overall heavyweight dependency trees already being in the most common complaints.
Yeah for the core unicode stuff it'd probably be best done as a table generation. Worth noting though: you could still use the efficient code point trie impl that ICU4X uses for this, and not take on too many dependencies. Note that ICU4X's model also involves running a data generation tool; so it might not be an improvement over ucd-parse for us. Eventually, ICU4X will have a copy of various slices of data on a CDN, and then it would be an improvement over ucd-parse. The only difference right now would be that ICU4X is much more likely to update quickly to the latest unicode stuff over ucdparse, but that may not be a huge deal since ucdparse doesn't need much work. I wasn't really thinking about the ecosystem at large. I'm fine continuing to maintain unicode-rs, I just don't have that much time to devote to it.
Code change:
[[package]] name = "unicase" <del> version = "2.6.0" </del> <ins> version = "2.7.0" </ins> source = "registry+https://github.com/rust-lang/crates.io-index" <del> checksum = "50f37be617794602aabbeee0be4f259dc1778fabe05e2d67ee8f79326d5cb4f6" </del> <ins> checksum = "f7d2d4dafb69621809a81864c9c1b864479e1235c0dd4e199924b9742439ed89" </ins> dependencies = [ "version_check", ]

Related discussion:
text: Overall rustc and other users tend to make PRs when they need them and that works out fine. FWIW, ICU4X is very much optimized for binary size.. Perhaps not compile times, but it does tend to produce small binary sizes (and it's shipping on embedded devices already). I'm not really pushing on us moving to ICU4X yet (I think is investigating a bit though!), I think the status quo is manageable, but it's definitely an option on the table!
I don't think I have any strong opinions. It seems like could be updated to handle these declarations without too much trouble though? And then things would be fine? Or is there something else I'm missing? Also worth pointing out that the whole stuff is something I built for lack of a better alternative at the time. My goal was to get rid of an ever increasing pile of bespoke Python scripts. But I don't have much vision beyond that, other than keeping it "simplistic" and framing it as a "kitchen sink" of sorts. I've also updated 's README to link to ICU4X (wording changes very welcome):
Oh yeah to be clear, no shade on ucd-parse and ucd-generate! They're great tools and I don't necessarily mean to say ICU4X is much better or anything. Rustc already needs ICU4X for list formatting and was investigating using it for more things like the XID and security properties (which rely on those manual python scripts).
I can't make heads or tails of the declarations. From what I can tell, they're mainly being used to apply property values to... unassigned codepoints? Just looking at the diff between Unicode 14 and 15 for , this text was removed from 14 and replaced with declarations in 15: So it seems to me like parsers of UCD that ignore in Unicode 15 are probably still just as wrong as they were with Unicode 14? The Unicode docs do a really terrible job of even explaining what is supposed to mean and why it even exists at all. The only explanation of what they actually are seems to be the first sentence of the section you linked: Most of the rest of that section is documenting the format. The key thing that's missing from the docs is what actually is and why it exists.
Code change:
[[package]] name = "unicase" <del> version = "2.6.0" </del> <ins> version = "2.7.0" </ins> source = "registry+https://github.com/rust-lang/crates.io-index" <del> checksum = "50f37be617794602aabbeee0be4f259dc1778fabe05e2d67ee8f79326d5cb4f6" </del> <ins> checksum = "f7d2d4dafb69621809a81864c9c1b864479e1235c0dd4e199924b9742439ed89" </ins> dependencies = [ "version_check", ]

Related discussion:
text: Like... why does there need tobe a mechanism for defining default values for ranges of codepoints when the files themselves are already defining values for ranges of codepoints? Something is missing from my conceptual understanding because I don't understand why the declarations aren't just converted into normal lines in the data files.
I have some guesses as to what the answer is but I don't want to say something incorrect, so I've just asked these questions to the Unicode people.
I'd like to hear back from contacts in case I'm mistaken (or missing some detail), but I'm confident I know what they're for, and have thoughts on them in the context of a low level library like . These exist because several of the UCD files do already require special handling for missing values, but it's not usable in a programmatic manner and needs hard-coding. I believe the goal is that for properties that have lines, you can now determine the property values for all codepoints using just the data programmatically readable from the UCD, without reading the documentation of that property. Here are most of the interesting cases I've found for this (some of which are more useful than others), which should give you a good idea of why it exists and why with very complex defaults, like . I believe this is intended to replace things like . Presumably this list in 's source should have have been getting updated to synchronize with the comments in 's header on each Unicode version update, but that comment didn't really indicate that it was going to change, so that was not happening. Now, these are in lines, which allows that to happen without manual work. I don't know if it changed before now (some documentation hints that it did), but I'd weakly recommend to the Unicode folks that when it changes, the and files are also changed to contain tests that fail if the updates have not happened. That may already happen, but that's really the only many people know if their parsing code is wrong for something like this. with a single default value. In most of these, the default is listed in that property's documentation (or prose in comments in the file) and is kind of the "obvious fallback" (often , if that's a value for that property). Now you can read line to determine that the default for is , that defaults to , or that defaults to , and so on.
Code change:
[[package]] name = "unicase" <del> version = "2.6.0" </del> <ins> version = "2.7.0" </ins> source = "registry+https://github.com/rust-lang/crates.io-index" <del> checksum = "50f37be617794602aabbeee0be4f259dc1778fabe05e2d67ee8f79326d5cb4f6" </del> <ins> checksum = "f7d2d4dafb69621809a81864c9c1b864479e1235c0dd4e199924b9742439ed89" </ins> dependencies = [ "version_check", ]

Related discussion:
text: An odd case here is that the empty string is spelled , but given that that's only used for a property whose value is a codepoint (and not an enumeration), it's unsurprising that it has a special case. , boolean properties which default to and only contain UCD lines when they're . There are a few of these in the normalization properties, and it is probably only useful for code that wants to handle things super generically, but that's something. that default to the codepoint itself, such as is the codepoint itself. Previously, you just had to know this (it's kind of obvious if you think about it, though), but now a line can be used. that default to the value of another property. For example, the has . This codifies that the default value for the property is the value that codepoint has for the property. There are some annoying things about this: a user of a library like (but definitely not for something high level like ), I might want to distinguish between "this is a value explicitly from the file" and "this is the default from ". This is generally true for several of these, but especially true for a few cases like , where I may want to be handling values entirely separately. Obviously a high-level library for Unicode properties would absolutely not expose this distinction, but for something which is explicitly for parsing the UCD files, I suspect including values should be optional (even if it's done by default). now, the property value will often use a different property alias than is used elsewhere in the file. For example, a line for EAW uses , whereas the file uses . Similarly, uses / in but and in the file. Most of these use the "long" form of the property value in the line, and the "short" version in the file itself. I can see the logic here, but it's annoying, as to handle this fully programmatically, you need to read . 's incomplete and completing it feel like it might be a bit annoying. For example, still has "All code points not explicitly listed for Case_Folding have the value C for the status field, and the code point itself for the mapping field" which is not reflected in a line. Given that this is not a single property, it will probably require a special case for the lines.
Code change:
[[package]] name = "unicase" <del> version = "2.6.0" </del> <ins> version = "2.7.0" </ins> source = "registry+https://github.com/rust-lang/crates.io-index" <del> checksum = "50f37be617794602aabbeee0be4f259dc1778fabe05e2d67ee8f79326d5cb4f6" </del> <ins> checksum = "f7d2d4dafb69621809a81864c9c1b864479e1235c0dd4e199924b9742439ed89" </ins> dependencies = [ "version_check", ]

Related discussion:
text: 's also not clear which of these are "stable" values you can rely on, but I presume the answer is "technically none of them, but in practice it's probably safe to assume something like on a derived property is unlikely to change". That said, overall it's probably a good thing, but the release notes for this definitely ignores many of the reasons why this might be a bit of a pain in the ass to handle.
TLDR: they make the files more maintainable and easier to read comments for
Some more background from Asmus, including some explicit steps on how to handle them. They brought up an interesting point that the order of parsing them matters a lot.