Code change:
Feature gate: This is a tracking issue for scoped threads. RFC: https://rust- Documentation: [x] RFC attempt 1: [x] RFC attempt 2: [x] Implementation: [x] Change signatures a bit to remove the argument to the spawn closures: [x] Fix soundness issue in implementation: [x] Document lifetimes: [x] Final comment period (FCP): [x] Stabilization PR: [x] Can we omit the argument to the functions given to ? That is, rather than . It's already possible by forcing the user to use instead, but that's not great. Maybe the language could be subtly changed to capture references or certain types by value rather than by reference(-to-reference). See also and the collapsed section in . Mostly answered in . Working idea in . Implementation in [x] How to document the and lifetimes clearly without scaring people away.
Nominating this for team discussion for the unresolved question.
Awesome API, thanks again for taking care of driving this <details<summaryTo expand on my captures suggestion / "thingy"</summaryI think it would generally make sense to be able to mark () type definitions with a annotation on them or something, so that, in a non-generic context, if one has , is nonetheless captured by move. It sounds like a kind of straight-forward thing to implement? I'd be willing to try to tackle the implementation . With it, we'd then replace with (probably still named , but I'm using the suffix here to clarify the comparison with the current implementation). The reason for that would be to have that first parameter be non-covariant (≠ the (co)variance of in ), so that the requirement for the scoped joined thread closure would become rather than , so that would be allowed to capture the . With all that, the API would become: which is not only neater (there is one scoped / scope-defining API, at the outermost closure layer, and then good old functions), but would also avoid the rather hard to grasp error messages that somebody gets when they update their scoped threads usage with a new nested thread. Indeed, consider somebody updating some code and doing: They'd get, as of the current , the following error message: Now, I personally do love the quality of this error message, but I think we have to admit it can still be quite too "technical / advanced" for some Rust programmers out there (Alan yadda yadda): they shouldn't have to know about variance to know how to use a scoped API. Granted, another option would be to invest some effort in improving the error message when using a scoped API / hard-coding this very situation, but I'd rather see that effort invested on , for instance. _ These have been my two cents on the topic, thanks for reading , and won't insist anymore on this subject, I promise </details
Hmm, regarding the capture of references, I think that the RFC 2229 logic might be relevant here. i.e. we could capture by reference instead of capturing by reference. In a way, I'm always surprised we don't.
In the meeting last week, I about the question of whether a more ergonomic API is possible. I see that has a nice write-up as well. Is there any way to make it so that calls don't need to get a scope? In other words, instead of this: can we have this? No. OK, fine: maybe someday. But not for a while. <details<summaryIf you insist, but remember that you asked for it.</summaryI spent a while deep-diving here because the situation wasn't quite what I expected. To explain, though, I think we gotta go back to basics. Similar to Rayon and others, the current scoped API gives each task a reference to the scope that contains the task: The question is, why is this inner necessary? Couldn't we just do and capture ? If you try to setup an API like the one above, you will find that it doesn't work. The problem is that the inner closure captures from the outer frame by reference. This turns out to be a general thing: any time that you have reads of a copy type, a non-move closure like will always capture it by reference. The reason is that non-move closures take the "least permissions" that they can: they prefer to capture by ref, then by , and finally to take ownership ("by value"). The downside of capturing by reference in this case is that it means the lifetime of the inner closure is linked to the outer closure, and we need them to be independent. We want the outer closure to return and then the inner closure is still running in a parallel thread, so it can't have a link to the outer closure. So why DON'T we always capture copy types by value? It's debatable if the current behavior is right, but it wasn't an accident. The intuition for the current rules is that a (non-move) closure is "part of the enclosing stack frame". The problem is that this is not the behavior we want here, we actually want something that is detached from its immediate parent, but attached to the parent of the scope itself. We don't have a way to express this (we'll return to this theme). This is not something we can "just change", it would change the semantics of existing programs in various ways. Consider this example: Today, this code fails to compile, because grabs a shared reference to , preventing the mutation . If you change to a closure, the code compiles and prints (not , because it now has its own independent copy of ). I suspect there are examples of code that runs in both versions but does something different, though at the moment I'm not sure what that is (if were copy, it'd be easy to create such programs, but it's not, partly for this reason). OK, so if by-reference closures aren't quite what we want (because we want something detached from the caller), why not have people use a move closure? i.e., why not write ? That would work fine for copy types like , but not so well for things that are not copy. The problem is that the intuition for a move closure is that it is detached from the surrounding stack entirely. But that's not what we want here. As we said, we want something that remains attached to the parent stack, but not the immediate parent. We want it to be attached to the scope's parent. If you use a move closure, patterns like this don't work, and that's annoying: Here, the two move closures would both try to take ownership of , which means this code won't compile. We'd have to write before spawning for this to work. So, the above is all accurate for Rust 2018, but as of Rust 2021 the situation is actually somewhat different. In particular, we can now capture places and not just variables. Actually, in Rust 2021, it turns out that we don't capture -- we capture . You can see this if you use some internal rustc annotations. Analyzing gives output that includes: The means that the closure is borrowing , and not . This is important because it means that we don't actually capture data from the intermediate stack frame, which is going to terminate. We capture the object directly from its owner, which is the function, which (simplified) looks like this: So why don't things work? The problem is this lifetime. It represents "the lifetime of the closure's captured data", and it can basically be anything, but it has to be larger than the call to itself. Therefore, so long as doesn't return, that data remains valid (which is what we want for this API). The API requires that each thread's closure outlives : So, let's put it together: We have a lifetime that lives "sometime longer then the call to " Threads can only captaure things that live for We have a variable that threads need to capture which is a local variable within You see the problem? Threads cannot capture because it doesn't live for . We can't quite express what we need to express, which is something like "some lifetime that is internal to but longer than the call to this closure". It's possible we'll grow ways to say that or tweak the system in the future, but I don't think it'll be soon. =) I could imagine us trying to tweak the closure capture rules here, but I'm not sure exactly how. In some ideal-ish world we've have a way for the compiler to know "how detached" the closure ought to be from the parent stack frame (e.g., based the lifetime bound that the API requires). I've advocated for a more explicit set of capture clauses, too, but that doesn't really solve anything here, since you wouldn't want to use those on a common basis. I think that the that recently blogged about is relevant here (I view the "scope" as a kind of capability), but that's obviously far out. In any case, any change here will require a fair bit of research and coordination. </details
Thanks for the detailed write-up, It turns our that that 2021-edition-captures tangent about a being captured by "value" rather than by ref ( on older editions) is actually a very interesting escape hatch. It basically provides one case of my advocated , and one case is all I needed: scope parameter that would be captured "by value", and with the inner , made interiorly invariant, so as to be able to use rather than . Here is , a bit dirty, tweaked from the original __ Now to decide if such a complex type is warranted More seriously: if hidden behind a type alias, the API docs could just show , so it's not that bad.
Heh, thanks for proving my conclusion a bit hasty! I think indeed that this scheme can work, though whether the type is too complex is a good question. To elaborate a bit around this point, if we had the ability to write (not a syntax I would recommend, but hopefully you get the idea) then effectively what we want to be able to say is something like i.e., each of the spawned closures must outlive this , but they don't quite know what is, only that it is outlived by . So if they can prove they outlive , that's good enough. You can indeed often simulate where clauses like this by leveraging implied bounds, e.g., , which is kind of the trick here, but it does make the signature more complex of course (as you noted).
Thanks and your suggestion is interesting! The type is a bit complex, but users don't have to write it out as they just get it as argument to their closure. It's definitely worth considering. However, I'd really like to have some understandable explanation of the lifetimes, which is already quite tricky with the version that's currently implemented. If a user asks how the lifetimes work here, I'd like to be able to explain it without causing more confusion. explanation helps a bit here. (Not sure if that type alias helps much, since the definition of such an alias is public and part of the stable api anyway.)
The gist regarding the lifetimes involved here, would be: ⇔ ⇔ Where I'm using the alias for better readability, and notation for . So we can replace the bounds with bounds, provided be invariant[^1] That way we can have the closures capture the itself. [^1]: if we were to let the caller "shrink ", then they'd end up with , which would always be met, resulting in a useless bound We achieve invariance of by having it inside the type, since otherwise, with the part alone, would be covariant (the region could be shrunk by users).We need to have a rather than just a because of this capture-by-move thing, where is captured as such, whereas a would be captured by reference, yielding a , which would not meet the requirement.To get a quantification in current Rust, we carry an implicit bound inside the type itself, so that quantifications that "see" the type end up correctly constrained. Hence why also needs to be infected with [^2]. [^2]: Technically an extra closure parameter could be the one mentioning : it doesn't necessarily have to be tied to itself (e.g., ). But we can agree that it wouldn't improve the readability . As to what does represent, it's a caller-chosen upper-bound on this quantification, and the caller will get maximal flexibility by picking a minimal . In Rust, the minimal fed to a generic function call necessarily spans beyond the return of that call. In this instance, : These lines I've drawn thus represent the limit after which stuff borrowed by the threads may die / be dropped.
I tried the current nightly API in some test code I was writing, and it worked well. Served the purpose I wanted it for, borrowing some local values, without any issues.
What's the next step here? We have an existence proof that it's possible to eliminate the scope parameter. It might be possible to do so in a simpler way, but we don't know that yet. Can we hide some of the internals and the multiple lifetimes from the public API to allow for future evolution, perhaps by keeping the Scope type unstable for longer than the scope function?
I think we should go forward with removing that parameter. I implemented suggestion here:
Found a soundness bug in the implementation. Here's the fix:
Adding documentation about the lifetimes here:
merge While the implementation is pretty new, the design of this API has been around for a while. There was just one open question of whether it was possible to remove the (unnecessary) scope argument to the thread closures. As it turns out, that was possible, and was done in Other than that, there are no more open design questions and I feel the design space has been properly explored. The top comment on this thread contains the up-to-date overview and history, including links to the documentation:
Team member has proposed to merge this. The next step is review by the rest of the tagged team members: [x] [ ] [x] [x] [x] [ ] Concerns: resolved by resolved by Once a majority of reviewers approve (and at most 2 approvals are outstanding), this will enter its final comment period. If you spot a major issue that hasn't been raised at any point in this process, please speak up! See for info about what commands tagged team members can give me.
I agree that the scope argument removal should not affect the perceived maturity of this solution. However, should we consider the issue at to be a motivation for additional time to bake?
I remember there being a bit of a debate / open question around how panics are handled and aggregated. Can I get a quick summary of how that discussion was resolved and what behavior we settled on?
That was unrelated to the API, but was only a very subtle implementation issue (part of which even affected the long stable regular threads from ), so not part of what we stabilize.
Thanks for the clarification! reviewed
I've not read this API in detail but I'm in favor in principle of this being in the stdlib -- that said, I am interested to know how panics are being managed. It occurs to me that this API is probably setting important conventions that will have a precedent. (In particular, I'm very interested in seeing a "structured concurrency" API for async settings that will feel a lot like this, I suspect.)
The user can always handle panics any way they like by using on the within the scope. If they don't explicitly join all scoped threads, we only track whether any of the implicitly joined threads panicked, and in that case panic at the end of . An alternative could be to collect all the results/panics in a Vec and return that from the scope, but that's often not useful and can result in an infitely growing Vec in common use cases (e.g. a web server spawning one thread per connection). If users want to collect all the panics in a Vec for some reason, they can keep the ScopedJoinHandles around and them to get the results. The other part of the discussion we had about panicking was about the exact panic message from the main thread at the end of the scope, e.g. whether it should include one (or all/some?) of the original panic messages. That's not part of the API though, and an implementation detail that can be changed later.
Ah, yeah, I remember that decision now and that sounds good. Do we handle panics from the main closure any differently from those of spawned threads?
I think this API might be a bit more boring than you think, as there's not much interesting it can do with panics. Panics always directy run the panic handler/hook which will print it to stderr by default, and then afterwards it might unwind with a payload, which might be caught by a . If there's no surrounding and unwinding goes past the 'main' function of a thread, its panic payload is available as an from . This is true for both regular and scoped threads. (There's no reason to do this differently for scoped threads.) The only new question is what happens with panics from threads that are not explicitly ed, but those that are implicitly 'joined' when the scope ends. We could either try to collect them into some structure and allow the user to handle them, which is not always useful and can be a waste of resources, or we can just do and leave it to the user explicitly if they want to do any more sophisticated panic handling.
One related thought, though I agree with the decision to leave it up to the user to handle panics explicitly when that is important to them, I think there's an opportunity here for us to provide helpers to make it easier to do so and assert that no join handles are left unjoined. Wondering if we should add a restricted lint that people can enable to get the compiler to help them catch implicit joins.
Yes, panics on the main thread simply continue unwinding past the scope, and do not start a new panic. Example to demonstrate the difference: Here we see first the spawned thread panic, and then the main thread panic because a scoped thread panicked: But with a panic on the main thread within the scope, we just see one thread panicking. Exactly the same as when you panic outside the scope: If both these situations happen, the main thread will still continuing unwinding its own panic, making both panic messages visible: We no longer see the 'a scoped thread panicked' panic, because that's not the reason the main thread panicked.
That comes very close to an opt-in . The same lint would be useful on regular threads if you never want to detach one, so it's not really specific to scoped threads.
One last question, what happens when the main thread panics but a scoped thread is still running? Does the panic propagation wait for the scoped thread to exit?
Yes. Not doing that would be unsound.
Regarding this comment, I would like to wait at least until the unstable scoped threads apis have made it into a stable release, right now if you open the docs on stable[^1] there's no mention of scope, so I imagine many users haven't even realized that we the API yet, let alone taken the time to try it out on nightly. I checked github code search to find example usages of the API and I got exactly zero[^2] results other than and its forks. [^1]: vs [^2]:
I don't think anyone would use std's nightly unstable version, because crossbeam::scope is available on stable. Searching for that on github or gives lots of matches.
The crossbeam API is different from the one we're exporting, particularly with respect to how it propagates panics. I think given the changes to the API and how little testing they've had we shouldn't be rushing to stabilize this. People will use this API given the chance even if it's on nightly[^1], and the people who are affected by needing this API and needing stable still have crossbeam which you've pointed out is similar in functionality if not implementation so there shouldn't be a pressing need here to stabilize this ASAP. [^1]:
The difference is very small. I don't think it'd be a reason for anyone to use the nightly if they can just use the stable . Writing rather than isn't really a big benefit, and basically every user just ignores or s the result.
To clarify, this discusses the stabilization of , but not necessarily that of the type itself? Because there may be naming considerations for the latter (, or , rather than the abstract concept of a scope, alone[^naming]), and also the idea that maybe in the future we could get rid of that parameter on the scope, should we get kind of bound: We may never get -in- clauses, and even if we do, we may not make that change; but still, it may be something worth considering before stabilizing the type itself. [^naming]: we currently have a function called , which takes a closure involving a lifetime , which itself is part of a type called , yielding: a kind of API, conceptually. Imho, we should try to vary a bit some of these names for clarity. My personal view is that the function itself, even, ought to be named or , and the type or (the lifetime ought to remain named ), even though I imagine that the ship to change the function's name has already sailed…
No, the FCP is for everything listed in this tracking issue. I don't see much reason to keep the type unstable. I don't think we should rename the type to anything like , as I explained . ("I don't think we should rename Scope to Spawner, because I imagine Scope might be able to do more than just spawning threads in the future. For example, I can imagine something like to check if any of the threads panicked. Or to see how many threads are currently still running in this scope. Etc.") I don't see that happening any time soon. I even if that'd be possible at some point in the future, I don't think it really matters. You'd rarely spell out the full type anyway. Removing that one lifetime from the type and exchanging it for a bound doesn't really have much advantages for the user.
:bell: This is now entering its final comment period, as per the . :bell:
AFAIK, the burden of proof when stabilizing is reversed: there has to be a reason for the stabilization to be actually useful, and knowing, to the best of our knowledge, that "we won't regret it". Regarding the stabilization of the type being useful, I just don't see it: we can use and without the type of having to be nameable, and that should cover 99% of the usages of a scoped API. <!-- --And you seem to agree with me on this point:Regarding potential drawbacks, however, I insist that our current: API leads to very poor readability, since we have four distinct things named the same. Either the function or the parameter ought to have their name changed, to improve the teachability. I think we would all like for Alan, for instance, to be able to just look at the definition of this API signature and already have some idea of what's going on: if they have to click on to see the difference between that parameter and the function that introduced it, then that's an instance of Rust being more obtuse than it needs to be. (Meta: I didn't fight for this bikeshedding before, just hinted at it, since we were just in the experimental / phase (which is why I didn't keep discussing about it in the mentioned PRs, it seemed out of context). But I haven't seen either a "context" to discuss about this emerge; instead, we are suddenly going for stabilization (for something that isn't even yet in the stable Rust docs!). Moreover, it seems to be about full stabilization rather than stabilizing the 99% useful API, even though there are outstanding objections) Regarding the naming, then, I see two approaches: to rename to or (or even, would be the most self-documenting name, since this is what this abstract "scope" is about, but I reckon that it may be a bit too long).and/or to rename to something else. I have personally suggested , since: that would be the main role of that handle; it still seems sensible to ask the handle in charge of spawning stuff if any of the spawned stuff has panicked, or how many things it has spawned / are still running, so I don't see the "we may add other APIs related to the spawned threads" as a counter-argument for the name. If the vision that this handle represents something to manage the "scope", whatever that abstract thing may be, then let's at least have that in the name: . Or . Or . Remember that these are names that will rarely occur in code, but which will be paramount for people reading the docs and trying to figure out what's going on. A good type name goes really far in that regard, yielding the "pit of success" design philosophy. Or, even a (similar to how we have always had s). It's not super great, imho, but it is still better than a bare . This "conceptual" view (obviously real code won't name the lifetime, nor the type of the closure param, and if pervasive enough, may just name the closure parameter ), thus yields to the following signature, for which we can now already have an intuition of the API from just looking at it: <img width="589" alt="Screen Shot 2022-03-30 at 13 50 46" src="https://user-"I don't claim to have the better knowledge about this documentation-in-code bikeshed; I'm sure people can come up with nicer names than the ones I've suggested. But I do claim that the current naming, while good for a prototype, is not something we'll want to have around in the future, that the bikeshedding itself is legitimate. Thus the name of the type, at the very least ought not to be stabilized. If that requires splitting up this feature in two, so be it.
That doesn't mean never. It can be reasonable to take it as an argument to a regular function for example. Limiting the API just introduces needless frustrations where this argument can only be used in certain ways. This function is used to define the boundary of the scope, so writing seems like a good way to clearly state 'this is the scope'. Things like sound like they take a scope as input, rather than being used to define the scope. We usually name things after what they are or represent, rather than after what they can do. E.g. rather than or or . A represents the scope during which scoped threads may run. The method and potential future methods like are attached to that type because it operates on that scope specifically. seems like a perfectly fine way to express the concept of spawning a thread within a scope. It seems very reasonable to me to have the function that creates a be called . That is a pretty common pattern in Rust. Same for calling the lifetime of that object . Things like this also don't seem to lead to difficulties: I have had no issues so far with teaching this API, but I'm interested to hear any issues you ran into while teaching others this API. Explaining that defines a scope and that you can then spawn threads in that scope seems to be intuitive.
I haven't yet had the chance to teach this API to others, I admit, but I recall very well when starting with crossbeam's API, to wonder "why is everything called scope", and having to navigate back and forth between the function API, and the type (with its instance in the examples) to better grasp the interaction between the two. Having names to distinguish between two different things helps our thoughts "anchor" onto those names to better articulate the logic (language as the vessel of abstract thinking, etc.). That sentence, for instance, illustrates quite well what I mean: by definition, currently, a closure is very much "taking a scope as input". but a is also defining the scope within which the threads will be scoped, I agree. Hence my advocating for one of those two things to be renamed, even if just from to . But I don't want to delay stabilization of this very useful and well thought of API you've written based on this minor naming concern. Hence why I am suggesting that be stabilized, and that we be given time to discuss a bit more about the name of the parameter. Maybe it doesn't change, after all, but maybe it does. (The other option would be to rename to (it's less "takes a param" than , while conveying that it's different from its own parameter). But doing that, if I understand FCPs correctly, would require delaying the whole API for another release or something, which seems undesirable?) To clarify, I'm trying to be conciliating here : whilst not going against the main stabilization process altogether, which I'm in big favor of, I'm trying to nonetheless voice a nit which I think would improve documentation. This is basically just a statement that "to condemn the current naming, as a whole, to remain as it currently is", there must not be any reasonable doubt_ : ! So while the evolution of the name could go in either direction (maybe it stays as it is, if it turns out that I'm alone in this "feeling uneasy with overloading the term for two things"), I hope I have communicated the presence of enough reasonable doubt for the stabilization of the type parameter to be delayed a bit, just for the time to properly discuss about it.
concern stabilization speed Should have made this explicit before with my earlier comments. Im concerned with how quickly we're moving forward with this stabilization and what it means for how we view nightly as a tool for testing our APIs and gathering feedback.
The object represents the call; it provides the way for the closure to interact with that scope. I think it'd be more confusing to use separate names. We're not in a court room trying to handle a murder case.
I understand, but that's not a very actionable concern. Is there something specific you want to wait for? A moment in time, or a specific release? A certain type of feedback? This FCP is now blocked, which is fine, but it'd be useful to know what unblocks it, to make sure it will not end up being stuck forever.
One thing I like about the function name is that it very clearly does not create/spawn a thread. This is not true for or (for me at least).
My intent is to discuss this during the libs API team meeting, but I'm happy to share my thoughts ahead of time. First, I would like to ascertain our goals for using nightly and merging features unstably. My understanding is that we have this functionality so that we can thoroughly test out new APIs and gather feedback on those APIs prior to stabilization. If this is the case, and we're not getting that feedback, I'd like to better understand why and fix that problem, rather than using that as justification for merging features anyways. As a strawman, I can imagine the problem is that we're not giving new APIs sufficient visibility or making it clear to users that we need feedback on them, so I can imagine us changing release notes to also include a description of newly available nightly features that we would like people to start testing and giving feedback on. Alternatively it may be that our processes are too obscure and so too few people feel comfortable engaging, in which case we'd need to find which areas are unclear, improve documentation in those areas, and find ways to make it easier for people to provide feedback. I'm not sure exactly what we will find or what the solution will be, but I think it's fair to point out that there seems to be something wrong here. Depending on what we identify as the problem I think it's entirely likely that we will want to more clearly document the approximate stabilization life cycle of features, including a minimum time to stabilization, and I would personally like this to include at least one release cycle of the nightly feature being visible in stable docs before an FCP ends.
I don't think it's easy to make a general rule for this. Do we reset that timer every time some small change is made to an API? We very regularly apply some final change right before stabilizing, like when we renamed/flipped isrunning to isfinished recently. We also regularly take something from a popular crate that lots of people already have experience with, and then relatively quickly stabilize it based on that experience, even if it has small modifications like taking something by reference instead of value, etc. I very much agree that experience is important for feedback, but I don't think that experience always has to come from usage of nightly std. The main goal is to make sure we're not making mistakes, so experience from the same or similar APIs from other crates should be just as valid. In this specific case of scoped threads, there is tons of experience with crossbeam's functionality, and this is only a very small modification of it. (It handles panics in a simpler way, matching how we see crossbeam::scope being used in practice. And it has a slightly simpler interface thanks to Rust 2021. That's all.) The original RFC is from early 2019, and a lot of people have already taken their opportunity to add their thoughts to those threads. Waiting until the latest version of this api and its documentation has been included in stable for a full cycle delays the FCP until early August, after which the stabilization could be merged only in 1.64, which will be released in November. I don't think we should delay this until November, if we feel like the design space has been properly explored and we're happy with the API. I think we should start the FCP when we feel like the design space has been properly explored and evaluated, such that we can be confident we're picking the right design. I'm afraid that cannot be put in a general rule like "we'll stabilize 8 months after the last significant change". Some unstable APIs have been unstable for much longer, as there's still alternatives left to explore before we feel confident it's the right API. (E.g. the try trait.) And some unstable APIs have a very small design space or have been explored very well already, and we very quickly feel comfortable to continue. (E.g. os::unix::net::SocketAddr::frompathname.) Waiting for the full release train and a full stable cycle after_ we're at the point where we feel confident about an API effectively raises the FCP from 10 days to many months, which doesn't seem like the right thing to do. Delaying the FCP because we're not yet confident about the API is of course fine, but then I do think it'd be useful to know in which ways we're lacking confidence and what kind of input we're looking for to gain that confidence. For this specific feature it looks like you're worried that it hasn't gotten much attention yet, which I agree would be a problem. But I don't think that's the case here: the RFC and tracking issue threads were quite popular and show a lot of interaction (both in comments and emoji-reacts), and it's also gotten attention on various social media. (And the FCP adds it in TWiR of course, too.)
I agree that we shouldn't have a rigid process here and that exceptions should be made and that figuring out the boundaries is going to be an important body of work over time, but I think in this specific case we should absolutely err on the side of caution. My understanding is that there have been 4(+?) attempts to add scoped threads to the standard library over 7 or more years. That they have a long history of soundness issues. That the current implementation was less than 3 months ago[^1], had changes to the API and lifetimes 27 days ago[^2], and a soundness issue discovered in the current implementation 25 days ago[^3]. My concern is with soundness issues in the current implementation. Is the current implementation a copy of scoped threads? If so, does that mean we need to port the soundness fix back to scoped threads? I was under the impression this was a novel implementation based on crossbeam's API. Either way, I think it would be irresponsible to rush things in this case just because we've been working on this for years. [^1]: [^2]: [^3]:
Some extra context related to some of your concerns: The implementation is almost fully shared with regular threads. I modified the existing regular threads such that their (the Arc through which the return value or panic is communicated back to the JoinHandle) can decrement a counter in the Scope when it is dropped. waits for that counter to be zero before returning. Everything else is just plain old threads. The bug you mentioned () is about dropping the result stored in that too late on the thread (after its main function, when there's no more JoinHandle), which caused undefined behaviour for regular threads as well (since before 1.0). The main issue was the 'leakpocalypse', leading to the api to ensure things always get dropped, as popularized by crossbeam. I'm not aware of any other related soundness issues we had in the past with (scoped) threads. The implementation is much simpler than crossbeam's, as we don't have to collect panics, and we can directly use the internals of regular threads in std, which I've modified to support decrementing a counter when they end. Both the rfcs for the post-leakpocalypse version simply got stalled on the author not having time for it.
I tried to use this feature and got error when trying to use non- function arguments in spawned threads, ie.: Is this intended limitation? Note that it does work with crossbeam ().
Interesting, that definitely needs investigation. concern borrow function arg ref example
That error looks very strange. It's mentioning a 1-element tuple () that doesn't explicitly appear anywhere in the code. It seems to be the tuple of captures of the closure, but that's not supposed to show up in diagnostics like this. What it complains about is that it cannot find a lifetime that is not longer than the one of the argument, but at least as long as the scope closure. But that second lifetime is always shorter. I'm filing a bug for this.
It compiles fine with enabled. So it looks like it's an issue with the 'old' borrow checker that's already been fixed in nll.
Looks like will be stabilized soon: https://rust-,
I have a couple questions: Will join all the constructed threads if the initializer panics? Will try to join all the remaining threads if any of the joined threads panic? Sorry if this has already been answered somewhere - I haven't looked through the minutes or related.
I think those questions were discussed starting here:
Yes, to both questions. Any other answer would be unsound.
Perhaps that should be clearly documented. I imagine it is easy to create a scenario where one thread panics and another thread never finishes, causing to hang. That could be surprising.
Sounds like we may need to block this waiting for the nll feature to stabilize.
It works perfectly fine without NLL for most use cases, holding it on that seems a waste of time if the alternative is failing to compile in the absence of it. I've provided some uses of the API in the form of a generic threadpool.
Quoting an issue that was recently opened on the repository: The doesn't have this issue: it does not maintain a Vec of join handles or similar, and doesn't grow its memory usage over time. (Other than what the running threads themselves are using.) It's good to see there are already real uses of long living scoped threads with unbounded spawning, as that means the effort to make be just as efficient in those cases was worth it. :)
Any news on the stabilization proposal? Regarding the lack-of- borrow checker error, it's a rough edge due to a borrow checker bug that one may rarely run into (easily solved with more type annotations / helping inference, by the way!), but not only would it be rare, we also know that once becomes the default we won't hit that issue anymore, which guarantees we won't have to change the API. I thus second making the API fully unusable just so that users don't run into the API being unusable in a few cases seems counterproductive.Regarding the naming concern (of my own!), since I notice I'm the only one of that opinion, I won't insist. I thus recant those concerns, especially if it helps moving things forward is there anything else remaining?
I've nominated for lang team discussion. I got the impression it's close to stabilizing. If it's stabilizing in 1.63, we don't need to have a discussion about whether we're okay with stabilizing scoped threads without waiting for . has an open concern (with rfcbot) about stabilizing this too quickly without the opportunity for the community to give feedback. It's been some time since then, so that might be resolved now, but that's not up to me to decide. I've opened to add scoped threads to the new TWiR 'call for testing' section, which might help with getting some feedback.
Are we certain this issue will be rare? The original repro in is just from using a non reference in a scoped thread, right? That seems like a primary use case to me. The only source I can find saying this is a rare issue is in but there doesn't seem to be an explanation as to why they think it is uncommon. With the TWiR post I think I'm relatively reassured that we will get enough eyes on this before it has a chance to stabilize, so I'll go ahead and resolve my issue for now, but if we get feedback that people are running into the NLL issue often I think we should just tie the stabilization to NLL. resolve stabilization speed
It works fine with non-'static lifetimes: borrowing from the local scope works without problems, as shown by the example at the top of this thread and in the docs. The issue seems to pop up when the lifetime comes from outside the current function. I wouldn't say it's a super rare case, but it's less common than just borrowing things from the local scope. I'm hoping we can stabilize both and scoped threads in 1.63.
Should we also update the example of in ?
My bad, I missed a imho qualifier in it, since that wasn't an objective statement. To detail: I suspect the cases where this issue will be fully blocking one's usage to be rare. Indeed, while the cases where we may run into this issue ("lifetime comes from outside the current function") can happen, they can be worked around by: either feeding that lifetime name to :or by making an even more nested non-copy borrow:(the reason for the latter is that thanks to the 2021 edition, only a local-to-the-function reborrow of is captured by the closure, thereby removing from the equation). so this leads to a rough edge in the API rather than a blocking thing; nothing some documentation can't palliate while waiting for nll to be stabilized. _ But if nll is to stabilized that fast, then there obviously isn't even a question to begin with, so let's wait for that nominated discussion
The fact that the support for nested usage of in non- closures relies on edition-2021 capturing behavior means that someone who would try to do something like on edition 2018 or 2015 would (given the current documentation) have a hard time knowing in the first place that this is something that just works on edition 2021 have to come up on their own with the solution of putting a reference to into a variable in order to be able to work with closures, provided they stay on the older edition; something as follows: I think at a minimum, the documentation should mention the fact that nested usage of the value is possible without on edition 2021. In a perfect world, we could quickly implement both HRTBs with lifetime bounds (in some sort of clause), as well as a way to mark some type as "always capture by value in closures when in case it would otherwise be captured by immutable borrow", and then the API could become significantly more simple by replacing by a simple , and all the receiver types by . This would then also solve edition-dependence. In particular as someone who regularly advises newcomers to generally avoid most or all occurrences of where a value is borrowed with the same lifetime as a lifetime argument (though the rule is even more important only for ), and -- what amounts to the same thing -- to avoid (or even more importantly ) methods on a type with lifetime argument , I perceive this type here as considerably ugly. And that's not surprising since the whole thing is nothing but a hack in the first place, around the lack of a feature to mark capture-by-value-instead-of-immutable-reference types. Given the fact that closure captures were worked on somewhat "recently" (for the 2021 release) anyway, I'm wondering how hard it would be to, perhaps, at least get the "capture-by-value-not-immutable-reference" marker implemented. With this, it at least becomes instead of , and the edition-dependence is lifted. And the signature of is shorter this way even compared to an ultimate solution, because it can use lifetime elision. Compare vs. (Although, I'm not 100% sure whether , or the more explicit would be preferable. And is yet another option.) Even without any language changes, there's the option of introducing a type synonym for for . (Naming could also be changed so the synonym would be called .) This allows the option of lifetime elision in the bound, too, as well as giving a shorter name to that type. On the other hand, the 's methods would still be necessary to be placed on the type itself, since we cannot do a block for a type synonym that's defined to be a type, so would probably be the only function signature in the standard library that could use the synonym.
NLL is about to be stabilized: resolve borrow function arg ref example
:bell: This is now entering its final comment period, as per the . :bell:
NLL has been stabilized/merged. The examples that broke before now compile fine on the latest nightly. :)
The final comment period, with a disposition to merge, as per the , is now complete. As the automated representative of the governance process, I would like to thank the author for their work and everyone else who contributed. This will be merged soon.
Merged and stabilized in 1.63!
Do we have a test case for the ? I’m asking because apparently there’s happening, and we probably wouldn’t want that test case to break before scoped threads are released, or otherwise (in case of a full revert on NLL) reconsider whether or not scoped threads should be already stabilized without it regardless, or later. In any case having a test case seems important, I couldn’t find any so far, but also I don’t exactly know everywhere I’d have to look for it.
Reopening for now. Can be closed with a test case .
Err I guess I should have reopened the other issue. Whoops. Oh well.
Tell me best night vision scope type?

Related discussion:
text: }); assert!(actually_finished.load(Ordering::Relaxed)); } <ins> #[test] fn test_scoped_threads_nll() { // this is mostly a *compilation test* for this exact function: fn foo(x: &u8) { thread::scope(|s| { s.spawn(|| drop(x)); }); } // let's also run it for good measure let x = 42_u8; foo(&x); } </ins>