Repository: python-trio/trio

Issue #242: Discussion: what's the best strategy for "nowait" and "select" operations?
Created: 2017-07-10T11:44:00Z
So from @wingo's blog I have learned about Concurrent ML. (His concurrency tag is also worth perusing generally.)
The core insight, IIUC, is to take the atomic blocking operations – roughly, the same ones our current cancellation guarantee applies to – and systematically split them into a few consistent sub-operations:

attempt to perform the operation synchronously, possibly wrapping the result in some thunk
publish that we're waiting for the operation to be doable and should be woken up if it might succeed
give up on waiting ("unpublish")

We already implicitly have this basic structure open-coded in a bunch of places, e.g. the _try_sync helper in trio/socket.py, the classes in trio/_sync.py, etc. Pretty much anywhere you see the yield_if_cancelled/yield_briefly_no_cancel pair in trio fits this general pattern, and "unpublish" is basically just our abort callback. So the advantages of reifying this would partly be just to simplify the code by having a single implementation of the overall pattern that we could slot things into – but even more, so because given the above pieces, you can create generic implementations of three variants:

Try to perform the operation, but fail if it would block (like the *_nowait variants that we currently implement in an ad hoc way)
Perform the operation, blocking as necessary (what we do now)
A thing like golang's select, where you can say "perform exactly one of the following operations: ..."

(The first is done by just calling the "try this" sub-operation; the second you do by trying and then blocking if you fail, in a loop, with the unpublish operation as the abort callback; the third is done by calling a bunch of "try this" sub-operations and if one succeeds you're done, otherwise publish all the operations and go to sleep. There's some subtleties around knowing which operation woke you up, and when unpublish happens, etc., but that's the basic idea.)
Right now we have a bunch of manual implementations of await x() / x_nowait() primitives. It's not clear that we have enough, either; #14 has an example of a case where you semantically need accept_nowait, and for HTTP client connection pooling when you pull a connection out of the pool you need something like receive_some_nowait to check whether the server has closed it before you try to use it.
Also, a golang-style select is potentially quite useful but isn't possible right now (at least for trio's built-in operations, of course you certainly could build a golang-style channels library on top, but then select would only work on that library's operations). You can spawn some child tasks to try doing all the things concurrently, but there's no way to make sure that no more than one complete – for that you would need to be able to (1) guarantee that all of the operations really are cleanly cancellable, and (2) perform the cancellation synchronously with the first operation completing, which isn't possible if the last thing it does after committing its work is to call yield_briefly_no_cancel.
An ancillary benefit is that if we expose these things as a standard interface to users, then this would also serve as very clear documentation of which the actual atomic cancellable operations are.
But, there are also some issues, I think:
IOCP: the above pattern works for BSD-style non-blocking socket operations, but not for Windows IOCP operations (#52). You can implement cancellable blocking operations as IOCP calls (that's basically what they are), and nowait operations using Windows' non-blocking calls, but IOCP has no generic equivalent to epoll to implement wake-me-when-this-non-blocking-call-might-succed, which means that golang-select is not possible. All you can do is ask the kernel to start initiating all the operations, and then by the time you find out that, say, your recv has finished, then your send might also have finished. I guess this might be possible to work around: for send I'm pretty sure we can treat IOCP as a kind of epoll, and then use non-blocking send. For recv I'm not sure if this works and for accept I'm pretty sure it doesn't, but for these operations you can more-or-less fake them as being always cancellable by using a little user-space buffer to push back any result that you want to pretend didn't happen. Are there any other cases? Do we need to change the pattern to accomodate this? The pushback trick doesn't seem compatible with a strict separation between "try it" and "wake me when you want to try again" primitives.
The main problem with HTTP connection pooling isn't checking if the socket is readable – we can already do that in the specific case of a raw socket. It's that it isn't something you can reasonably abstract in terms of generic "streams". In particular, if you have an TLS-wrapped socket, then you actually don't want to check if the TLS layer has data available, you really do want to check the raw socket underneath. And in any case I'm not sure that this would help make it easier to implement receive_some_nowait as a generic feature on streams, because receiving on a TLS stream is a very complex operation that may require things like lock acquisition, and all of the operations above have to be synchronous. So maybe the HTTP connection pool case isn't really a good motivator anyway.
Lock.acquire and Lock.acquire_nowait are tricky because of fairness (#54); it's not the case that acquire is just a loop like while acquire_nowait fails: sleep until it might succeed, because that creates a race on wakeup. I don't think it's possible to implement a fair mutex using just the framework described above. The problem is that we really need the two operations to be "try it" and "block until the operation is done"; a retry loop just doesn't work. So maybe this is actually equivalent to the IOCP case? Maybe we need primitives:

try it without executing any cancel/schedule point
publish whatever we need to publish to be woken up (but don't actually sleep), with some callbacks for abort, reschedule, and we-got-successfully-rescheduled

I think this is flexible enough to implement fair synchronization primitives and handle all the public operations. E.g. for golang-select we would want to arrange so that when one operation gets rescheduled, then we immediately abort all the other operations, before waiting to actually be woken up – this would need to happen from the context of the task that's handing off the mutex (for example).
But... this isn't quite right for stuff like non-blocking socket operations, where you actually need a try-sleep-retry-sleep-retry-... loop. Need to think some more about this.

Duplicate issue #719: Fine-tuning channels
Created: 2018-10-05T08:24:32Z
Channels (#586, #497) are a pretty complicated design problem, and fairly central to user experience, so while I think our first cut is pretty decent, we'll likely want to fine tune it as we get experience using them.
Here are my initial notes (basically questions that I decided to defer while implementing version 1):
In open_memory_channel, should we make max_buffer_size=0 default? Is this really a good choice across a broad range of settings? The docs recommend it right now, but that's not really based on lots of practical experience or anything. And it's way easier to add defaults later than it is to remove or change them.
For channels that allow buffering, it's theoretically possible for values to get "lost" without there ever being an exception (send puts it in the buffer, then receiver never takes it out). This even happens if both sides are being careful, and closing their endpoint objects when they're done with them. We could potentially make it so that ReceiveChannel.aclose raises an exception (I guess BrokenResourceError) if the channel wasn't already cleanly closed by the sender. This would be inconsistent with the precedent set by ReceiveStream (which inherited it from BSD sockets), but OTOH, ReceiveStream is a much lower-level API – to use a ReceiveStream you need some non-trivial engineering work to implement a protocol on top, while ReceiveChannel is supposed to be a simple high-level protocol that's usable out-of-the-box. Practically speaking, though, it would be pretty annoying if some code inside a consumer crashes, and then ReceiveChannel.__aexit__ throws away the actual exception and replaces it with BrokenChannelError.
Should memory channels have a sync close() method? They could easily, they only reason not to is that it makes life simpler for hypothetical IPC channels. I have no idea yet how common those will be, or whether it's actually important for in-process channels and cross-process channels to have compatible APIs.
I have mixed feelings about open_memory_stream returning a bare tuple (send_channel, receive_channel). It's simple, and allows for unpacking. But, it requires everyone to memorize which order they go in, which is annoying, and it forces you to juggle two objects, as compared to the traditional Queue design that only has one object. I'm convinced that having two objects is important for closure-tracking and IPC, but there is an option that's a kind of hybrid: it could return a ChannelPair object, that's a simple container for a .send_channel and .receive_channel object, and, if we want, the ChannelPair could have send and receive methods that simply delegate to its constituent objects. Then people who hate having two objects could treat ChannelPair like a Queue, while still breaking it into pieces if desired, or doing closure tracking like async with channel_pair.send_channel: .... But... I played with this a bit, and found it annoying to type out channel_pair.send_channel all the time. In particular, if you do want to split the objects up, then you lose the option of deconstructing-assignment, so you have to do cumbersome attribute access instead. (Or we could also support deconstructing-assignemtn by implementing __iter__, but that's yet another confusing way then...) For now my intuition is that closure tracking is so broadly useful that everyone will want to use it, and that's easiest with destructuring assignment. But experience might easily prove that intuition wrong.
Is clone confusing? Is there anything we can do to make it less confusing? Maybe a better name? So far people have flagged this multiple times, but we didn't have docs yet, so it could just be that...
And btw, the Channel vs Stream distinction is much less obvious than I'd like. No-one would look at those and immediately guess that ah, a Channel carries objects while a Stream carries bytes. It is learnable, and has some weak justification, and I haven't though of anything better yet, but if someone thinks of something brilliant please speak up.
Should the *_nowait methods be in the ABC, or just in the memory implementation? They don't make a lot of sense for things like inter-process channels, or like, say... websockets. (A websocket can implement the SendChannel and ReceiveChannel interfaces, just with the proviso that objects being sent have to be type str or bytes.) The risk of harm is not large, you can always implement these as unconditional raise WouldBlock, but maybe it will turn out to be more ergonomic to move them out of the ABC.
We should probably have a trio.testing.check_channel, like we do for streams.

Closing comment (2020-11-10T10:44:25Z): max_buffer_size=0 by default in open_memory_channel()? Still no default at present.
Errors on closing a channel with buffered data, so it's harder to accidentally lose data?

@njsmith speculated about throwing an error when the last receiver closes.
@oremanj suggested a more complex scheme where either the receiver or the sender may error


Improving on the bare (send, receive) return tuple for open_memory_channel():

@belm0's #1771 makes it a named tuple, so you don't have to remember the order
@njsmith wondered about making it bidirectional instead, so each object has both send & receive methods of different channels

#1784 is a step towards that
I suspect that returning this by default from open_memory_channel() would make it harder to understand, as you're now opening 2 channels.




Get rid of .clone()?

Fair bit of discussion - I won't try to summarise it.
Some of the objections are that it's hard to implement - but as things stand, .clone() is part of memory channels, not required for other kinds of channel.


Making the channel/stream distinction clearer

'Belt' or 'mailbox' instead of 'channel' to make it clearer that it conveys a sequence of discrete objects?
See also #959 about merging the interfaces


Should *_nowait methods be on the ABC, instead of only memory channels

@njsmith's thoughts on their general value in this comment
Also related to #242


.poison() or passing an exception to .aclose() as a way for one end to indicate that something has gone wrong, leading to an error in the other end.
Should __del__ do something if a channel object goes out of scope without being explicitly closed?

It's not always possible to close it from __del__, but it could warn
some types of channels (inc. memory channels) could be broken in some way that avoids a deadlock when you try to send/receive on the other side of the channel.



Whew. Did I miss anything?