Change:
<!-- Thank you for filing a bug report! Please provide a short summary of the bug, along with any information you feel relevant to replicating the bug. --I tried this code: This code can result in the error: This error is likely to happen when input is read in small chunks (e.g. a few characters per line, when line buffering is used), to slowly fill the string/vec close to its capacity. Currently, it is easily reproduceable by pressing return 15 times on the terminal input. This error is uncontrollable by me; when I call readtoend, the actual read calls and the buffer sizes used are entirely controlled by code in . I'd expect it to just do the right thing in this case and provide the correct size buffer, and not raise an error that the end user can't easily recover from. <!-- If you're using the stable version of the compiler, you should also check if the bug also exists in the beta or nightly versions. --: Also reproducible on nightly: <!-- Include a backtrace in the code block by setting in your environment. E.g. . --<details<summaryBacktrace</summary<pPretty sure this isn't relevant, as the panic is from an in my code because of an error value created in . But here you go: </p</details
Was able to replicate with the above example. Here is a full backtrace. <details<summaryBacktrace</summary</detailsIt seems that the problem is caused by the vector running out of memory. If for example you use: Then it takes more inputs to make it crash. I think the problem is probably somewhere around here:
label O-windows T-libs A-io
I agree with with - I suspect that is trying to using a buffer / remaining capacity that is less than 4 bytes, which is rejected by the . (And also delegates to internally, so fixing that would also fix )
I'm going to try to tinker with the code and see if I can get it to disappear by replacing line 369 with a <4 instead of ==. Will report findings shortly.
I can confirm that the bug no longer occurs if line 369 is replaced with I don't know if that fix is serviceable, or if this breaks other things. I'll run tests and if that works I'll create a pull request.
I think that should be fixed in the Windows-specific code in . That is the right place for Windows-specific console workarounds.
It seems to be a fixed-size array, the fix would require rewriting read to accept something that would allow for overflow. Alternatively, we could try catching the error, and then incrementing the buffer. I'll take a look at what that would modify downsteam.
I'm testing adding the fix to the error handling section at line 377. Seems like the cleanest and easiest place to implement the modification and have it only trigger when there's a problem. At worst the performance hit will be that it will try to read from the buffer twice before returning an error.
The implementation for should conform to the behavior specified in the . Essentially calling with a buf of 0 < length < 4 should work and return (at least) with one byte of the UTF-8 sequence if EOF has not been reached. To accomplish that I would suggest replacing the one u16 of state for an unpaired surrogate in with an , in which we can buffer UTF-8 bytes for the case that read is called with a buf of len < 4 and the UTF-16 read from the console converts to more than UTF-8 bytes.
Similar to what did for Stdout. CC
I like that idea! I'll try implementing it, and this should remove the need to throw the error entirely, if I'm not mistaken.
Thank you I was willing to fix this myself and finally get my first rust-lang contribution, but you swooped in out of nowhere and picked this up. It was a pleasure working with you though!
it was a pleasure for me as well! Usually I go in and try to provide some diagnostics, and in this case, with your help I was able to figure out exactly where I could fix the problem. It was actually the first (and to date only time) that has just worked. Usually I just dump a bunch of clues that I've been able to find. I would probably not have been able to do anything without your and input. On the subject of first contributions, there's a , that gave me my first actually merged PRs. That was enough to get me set up with the toolchain, and that's roughly 80% of the work. I'd really recommend grabbing one if one opens up and you want to have a try.

Discussion:
// the value over time (such as if a process calls `SetStdHandle` while it's running). See #40490. pub struct Stdin { surrogate: u16, <ins> incomplete_utf8: IncompleteUtf8, </ins> } pub struct Stdout { incomplete_utf8: IncompleteUtf8, }

len: u8, } <ins> impl IncompleteUtf8 { // Implemented for use in Stdin::read. fn read(&mut self, buf: &mut [u8]) -> usize { // Write to buffer until the buffer is full or we run out of bytes. let to_write = cmp::min(buf.len(), self.len as usize); buf[..to_write].copy_from_slice(&self.bytes[..to_write]); // Rotate the remaining bytes if not enough remaining space in buffer. if usize::from(self.len) > buf.len() { self.bytes.copy_within(to_write.., 0); self.len -= to_write as u8; } else { self.len = 0; } to_write } } </ins> // Apparently Windows doesn't handle large reads on stdin or writes to stdout/stderr well (see // #13304 for details). //

impl Stdin { pub const fn new() -> Stdin { <del> Stdin { surrogate: 0 } </del> <ins> Stdin { surrogate: 0, incomplete_utf8: IncompleteUtf8::new() } </ins> } }

} } <del> if buf.len() == 0 { return Ok(0); } else if buf.len() < 4 { return Err(io::Error::new_const( io::ErrorKind::InvalidInput, &"Windows stdin in console mode does not support a buffer too small to  guarantee holding one arbitrary UTF-8 character (4 bytes)", )); </del> <ins> // If there are bytes in the incomplete utf-8, start with those. // (No-op if there is nothing in the buffer.) let mut bytes_copied = self.incomplete_utf8.read(buf); if bytes_copied == buf.len() { return Ok(bytes_copied); } else if buf.len() - bytes_copied < 4 { // Not enough space to get a UTF-8 byte. We will use the incomplete UTF8. let mut utf16_buf = [0u16; 1]; // Read one u16 character. let read = read_u16s_fixup_surrogates(handle, &mut utf16_buf, 1, &mut self.surrogate)?; // Read bytes, using the (now-empty) self.incomplete_utf8 as extra space. let read_bytes = utf16_to_utf8(&utf16_buf[..read], &mut self.incomplete_utf8.bytes)?; // Read in the bytes from incomplete_utf8 until the buffer is full. self.incomplete_utf8.len = read_bytes as u8; // No-op if no bytes. bytes_copied += self.incomplete_utf8.read(&mut buf[bytes_copied..]); Ok(bytes_copied) } else { let mut utf16_buf = [0u16; MAX_BUFFER_SIZE / 2]; // In the worst case, a UTF-8 string can take 3 bytes for every `u16` of a UTF-16. So // we can read at most a third of `buf.len()` chars and uphold the guarantee no data gets // lost. let amount = cmp::min(buf.len() / 3, utf16_buf.len()); let read = read_u16s_fixup_surrogates(handle, &mut utf16_buf, amount, &mut self.surrogate)?; match utf16_to_utf8(&utf16_buf[..read], buf) { Ok(value) => return Ok(bytes_copied + value), Err(e) => return Err(e), } </ins> } <del> let mut utf16_buf = [0u16; MAX_BUFFER_SIZE / 2]; // In the worst case, a UTF-8 string can take 3 bytes for every `u16` of a UTF-16. So // we can read at most a third of `buf.len()` chars and uphold the guarantee no data gets // lost. let amount = cmp::min(buf.len() / 3, utf16_buf.len()); let read = read_u16s_fixup_surrogates(handle, &mut utf16_buf, amount, &mut self.surrogate)?; utf16_to_utf8(&utf16_buf[..read], buf) </del> } }