Talk:Byte order mark

message: Detailed discussion of BOM does not add to understanding of endianness, and BOM can be taken as a seperate concept, so i've moved it back to its own article.
It really was messy in the endianness article, especially as BOM has its own category links, external links, and the like.
--Pengo 00:52, 27 Oct 2004 (UTC)
thread_title:  Misc 

message: some of theese edits seem rather dodgy to me.
used-->misused : you claim that using the BOM to mark text as being in a utf- format is misuse yet http://www.unicode.org/unicode/uni2book/ch13.pdf ("specials" section "Byte Order Mark (BOM)" heading)  states that the byte sequence may be used to indicate both byte order and charachtor set.
"contrary to its definition" : you claim that use of the BOM on utf-8 is contary to its definition yet http://www.unicode.org/unicode/uni2book/ch13.pdf ("specials" section "Byte Order Mark (BOM)" heading) 
FF FE 00 00-->00 00 FF FE (already reverted) : encoding the code point FEFF in little endian utf-32 would give FF FE 00 00 as was in the original not 00 00 FF FE as your edit states. Furthermore the table that was there before your edit exactly corresponds to the information given in http://www.unicode.org/unicode/uni2book/ch13.pdf ("specials" section "Byte Order Mark (BOM)" heading)
unless i see good justification for theese edits i will be reverting the two that i have not already reverted Plugwash 16:13, 24 Dec 2004 (UTC)
replies: text: It is now two days since you made the edits and you have not responded furthermore i find you to be a very new contributer who has got into trouble elsewhere and made few other edits im am therefore reverting the rest of the edits you made to this page Plugwash 02:23, 27 Dec 2004 (UTC)
thread_title: [http://en.wikipedia.org/w/index.php?title=Byte_Order_Mark&diff=8784877&oldid=8775639 edits by Cherlin]

message: I have noticed that the Python interpreter reverses the byte order of UTF-16 big endian and little endian as compared to what is actually in the Unicode standard when given invalid input.  When Python's codecs module is used to read UTF-8 text in from a file and write UTF-16 text out to another file, and the original UTF-8 file begins with the non-character U+FFFE (encoded as EF BF BE), the non-character is accepted as if it were the byte order mark U+FEFF and the resulting UTF-16 file has the opposite byte order of what was requested.  I observed this on multiple platforms and Python versions.
The point is, if you are having trouble with the byte order of UTF-16 text, check your libraries/tools for problems, and verify everything using hexadecimal viewers.  You may find incorrect assumptions are being made in your tools or libraries.
Canistota (talk) 14:47, 12 March 2009 (UTC)
thread_title:  Concerning UTF-16 big endian vs little endian 

message: Is there a reason? Or someone just picked it by change?  —Preceding unsigned comment added by 117.104.188.16 (talk) 10:03, 25 January 2011 (UTC)
replies: text: That is U+FEFF (the value of the BOM character) in UTF-8 encoding. It is what you would get if a translator from UTF-16 to UTF-8 that was completely unaware of the BOM would produce by translating the BOM character.Spitzak (talk) 19:37, 25 January 2011 (UTC)
thread_title:  Why is the byte sequence EF BB BF choose to be the mask?

message: OK, I understand everything in the article, since I'm a unicodopath, but the intro should say:
* Unicode is a computer encoding of all languages characters (in principle),
* The byte order mark is designed so that a computer who reads it, can guess (with a reasonable probability) that the data text is probably Unicode, and
* Guess what kind of Unicode encoding, since there are many - the article already says that, I just wanted to stress that it shall.
The intro is a bit too technical for being an intro. The current text qualifies as a technical description intended for me and you, not any outsider. The missing nouns that should be in the intro are: computer, data coding, natural languages. L8R. Said: Rursus ☻ 10:15, 25 April 2008 (UTC)
replies: text: I think this set of recommendations is met or eliminated in the current article's text. The explanation that Unicode intends to capture all human languages belongs in the Unicode article (and it's there, and there's a link over there in the first sentence here). The notion that the BOM has the purpose of identifying Unicode (rather than some other encoding entirely) is not, so far as I can see, justified by the primary references, and is significantly undermined by the fact that BOM is in all contexts optional. The "which Unicode encoding" part is, as acknowledged, already captured. Jackrepenning (talk) 22:56, 6 August 2010 (UTC)
thread_title: Too technical!

message: There should a section in this page discussing how to remove it.  The only reason 99% of
people would ever come to this page is because they are trying to remove this ugly
little thing from a web page they are developing.  The 1% of people who come because they
are interested in it may be getting what they want but not the rest of us.  —Preceding unsigned comment added by Tjayrush (talk • contribs) 16:44, 6 February 2009 (UTC)
thread_title:  How to remove it 

message: The text is "Use of a BOM is neither required nor recommended for UTF-8" (and this already appears in the cite!).  That seems like a pretty clear "not recommended" to me - "neither fish nor fowl" means "not fish and not fowl", it doesn't mean "not fish and not specifically fowl".  Ewx (talk) 07:47, 31 March 2009 (UTC)
replies: text: And then it goes on to say that applications still must expect that it'll happen.  May as well address the complete sentence, rather than construe a (reasonably) carefully worded comment into a completely negative recommendation. Tedickey (talk) 10:07, 31 March 2009 (UTC)
replies: text: The Wikipedia text already points out that it may be encountered, and a recommendation not to *use* it doesn't contradict that at all. Ewx (talk) 13:54, 31 March 2009 (UTC)
replies: text:  But that is the point -- the Unicode standard does not contain a recommendation not to use it. -- leuce (talk) 14:37, 31 March 2009 (UTC)
replies: text:  Yes it does!  The text in the standard is "Use of a BOM is neither required NOR RECOMMENDED for UTF-8" (emphasis mine).  That is not an absence of a recommendation to use it and it is certainly not an absence of a recommendation not to use it; it is a straighforward and clear recommendation not to use it.  Ewx (talk) 07:49, 1 April 2009 (UTC)

text: Indeed (the emphasis is yours.  Use the complete sentence, or find another source which supports your viewpoint. Tedickey (talk) 10:56, 1 April 2009 (UTC)
replies: text: This is completely ridiculous.  The text is right there.  It says it's not recommended. Ewx (talk) 08:07, 2 April 2009 (UTC)

text:  Well I suspect this is a sticky point.  I have searched chapter 2 and 16 of the Unicode standard for references to BOM, byte order mark and UTF-8, and in my opinion the reference under discussion here is the only instance in the standard that speaks even remotely negatively about a UTF-8 BOM.  In all other cases where the UTF-8 BOM is mentioned or discussed, it is mentioned as a matter of course in an informational, neutral tone, without making any value judgements or any indication that the UTF-8 BOM is deprecated.  My personal take on this reference is that people who want to implement the Unicode standard might wonder why the Unicode standard keeps making reference to the UTF-8 BOM (also in chapter 16) as if it were a valid construct, and they might become under the impression that the Unicode consortium actually recommends using a UTF-8 BOM even though it is not required. -- leuce (talk) 15:34, 1 April 2009 (UTC)
replies: text:  Having read comments by some of the people involved (in the topic itself...), my impression is that the statement is a compromise between two viewpoints, neither of which dominated in writing the source we're discussing. Tedickey (talk) 16:31, 1 April 2009 (UTC)

text: The phrase "X does not recommend Y" can have two meanings.  It can mean that X recommends many things, but that Y is not one of the things that X recommends.  Or, it can mean that X makes a recommendation *against* Y.  The Unicode article does not recommend against a BOM... it simply does not make a recommend in favour of it.  My gripe is that the wiki article before I edited did create the impression that the Unicode standard recommends against the use of a BOM.  Even if one quotes directly from the Unicode standard, if quoted in a different context it can certainly give a slightly different impression of what the standard intends to say. --  leuce (talk) 14:37, 31 March 2009 (UTC)
replies: text: Agree.  And (for instance), if you consult some of the secondary sources, it's easy to come up with one that is wholly in favor of one or another viewpoint.  (Some are completely absurd, but I see those reflected on this page ;-) Tedickey (talk) 10:06, 2 April 2009 (UTC)

text: A recent flurry of edits has opened this can of worms again, and the text has grown decidedly text-booky and verbose. I’ve reverted to the state pre-edits. Firstly, we cannot interpret the Unicode standard for it. The text comes straight from the source. The reader is going to have to decide for “himself” what that means. There is no other authoritative source and therefore we are not allowed to interpret it for the reader. The cited mailing list thread is not authoritative; it is just one of hundreds of discussions all over the Web on the topic, each coming to its own conclusions. Secondly, it makes no sense to prognosticate at length over the reliability or unreliability of the UTF-8 BOM as a signal for UTF-8 encoding. Go find some reliable reference if you feel something definitive needs to be said about it. The article is fine as it is, particularly since these observations about the unreliability of the UTF-8 BOM apply equally well to the UTF-16 BOMs. A file of unknown provenance can never, with 100% confidence, be stated to be in any encoding whatever, or even to be text even though it might be the collected works of Shakespeare in 7-bit ASCII. The best you can state completely confidently is that the content is not in some particular encoding due to a violation of the encoding’s standards. Strebe (talk) 19:57, 14 July 2012 (UTC)
thread_title:  Whether Unicode standards recommends UTF-8 BOM or not 

message: Current:
While UTF-8 does not have byte order issues, a BOM encoded in UTF-8 may nonetheless be encountered, and it is explicitly allowed by the Unicode standard[1], the Unicode standard does not specifically recommend its usage[2]. It only identifies a file as UTF-8 and does not state anything about byte order.[3] 
When I read these two sentences, it almost sounds as if the Unicode standard identifies a file as UTF-8 :-)  That second sentence doesn't really fit anymore.  Besides, it repeats what has been explained elsewhere.  I suggest we remove it or move it somewhere else in the article. -- leuce (talk) 14:55, 31 March 2009 (UTC)
thread_title:  May I make this edit? 

message: This article should include an example of a byte-order-mark. DMahalko (talk) 23:56, 15 June 2009 (UTC)
replies: text: The article currently presents the definition and its encodings, including how it looks when rendered naively in various ways. David Spector (talk) 14:21, 28 March 2013 (UTC)
thread_title:  Article needs an example 

message: "Some text editing software in a UTF-8 environment on MS Windows adds a  BOM to the beginning of text files." Which ones? Tisane (talk) 02:57, 24 February 2010 (UTC)
replies: text: Probably a long list (Visual Studio .NET for instance) Tedickey (talk) 09:20, 24 February 2010 (UTC)
thread_title:  Which text editors add a BOM to the beginning of text files? 

message: The GB-18030 section has the following claim: "[132] and [149] are unmapped ISO-8859-1 characters".  But my understanding is that these characters aren't unmapped even in ISO-8859, but are C1 control characters; 0-31 is the C0 control area and 128-159 the C1 control area.  This is why the mapping by Windows-X of higher Unicode characters to the latter range can cause problems.
I think this section needs to be edited by a knowledgable person. — 93.97.40.177 (talk) 07:00, 16 June 2011 (UTC)
replies: text: You are confusing the character values produced after decoding with the bytes that are in the encoding. 132 is a value of one of the bytes in the GB18030 encoding of the BOM. It and 3 other bytes decode into the value 0xFEFF.Spitzak (talk) 19:09, 16 June 2011 (UTC)
thread_title:  Dubious claim in "Representations of byte order marks by encoding" section 

message: From which version of text editors recognize/do not recognize UTF-8 without BOM in the beginning of text files?
Because when all text editors will recognize UTF-8 without a BOM, BOM will not be necesary anymore...  — Preceding unsigned comment added by 86.75.236.140 (talk) 10:09, 30 June 2012 (UTC)
thread_title:  From which version of text editors recognize/do not recognize UTF-8 without BOM in the beginning of text files? 

message: Note:
:For the w3c, For compatibility with deployed content, the byte order mark (also known as BOM) is considered more authoritative than anything else. ( http://dvcs.w3.org/hg/encoding/raw-file/tip/Overview.html#decode-and-encode ) which i did read nothin about in this article.
:I also note there are some software to help developpers to deal with BOM, which is not considered by the article, such as:
:* tmp = tmp.replace("\uFEFF", "");
:* Apache BOMInputStream   — Preceding unsigned comment added by 77.199.89.101 (talk) 12:34, 2 July 2012 (UTC)
replies: text: These look like good additions to the article. Feel free to add them. I think “the… BOM… is considered more authoritative than anything else” is obvious (i.e., if it’s not more authoritative in some instance, then someone has wrecked the byte stream somehow!) or irrelevant (because the byte stream is wrecked and therefore has no coherent encoding), but perhaps there are situations I am not thinking of that W3C has. At least it is a verifiable statement, which the article needs more of. Strebe (talk) 16:41, 2 July 2012 (UTC)
thread_title:  w3c + existing software to strip BOM

message: I understand that the BOM is also a mean to avoid the Bush hid the facts bug.  — Preceding unsigned comment added by 77.199.96.98 (talk) 18:55, 9 July 2012 (UTC)
replies: text: Requiring a BOM would eliminate this bug. The Bush hid the facts bug occurred when an ASCII file without newline looked like UTF-16 without BOM. The BOM is not required even for UTF-16 for reasons written in the article.--BIL (talk) 09:02, 28 May 2014 (UTC)
replies: text: Requiring a BOM in UTF-8 would actually encourage such bugs, not fix them, by encouraging software to check for strange encodings first. Pattern recognition where you pick the patterns that are *least* likely first (this would be UTF-8 first, and ASCII if there are no bytes with the high bit set). UTF-16 could be recognized (and endianess determined) by looking for a large numbers of null bytes in non-pairs, but this pattern is a bit more likely in random data than the UTF-8 or ASCII patterns so it should be checked third. Requiring a BOM in 16-bit text has the same problems as requiring it in UTF-8, though it would fix this particular example.Spitzak (talk) 02:17, 29 May 2014 (UTC)
thread_title:  Bush hid the facts 

message: User:Karl432 alerted me on my Talk page that the statement clarifying the Unicode Standard's neutrality with regard to use of a UTF-8 BOM was made by a senior Unicode Consortium member (Technical Vice President, Emeritus). Presumably this is reliable “enough” to cite.
Karl432 also elaborates in his edits, which I deleted: “It is to be noted that the presence of the byte sequence representing an UTF-8 encoded BOM at the start of a text stream or file can be interpreted as a hint that a text stream or file might be encoded as UTF-8, but not as a proof, as such a byte sequence may have other unrelating meanings unless such can be excluded by other knowledge of the context.” I deleted this because (besides being verbose and unencyclopedic verbiage) the same comment applies to any BOM, not just UTF-8. Strebe (talk) 23:53, 15 July 2012 (UTC)
thread_title:  UTF-8 BOM recommendation 

message: When BOM is used in files, RFC 5198 (a RFC relative to protocols) stands that Net-Unicode forbids BOM usage.  — Preceding unsigned comment added by 86.75.160.141 (talk) 20:46, 20 November 2012 (UTC)
replies: text: How do you interpret RFC 5198 that way? The injunction against BOMs has nothing to do with files. It has to do with transmission of text strings. Strebe (talk) 21:15, 20 November 2012 (UTC)
replies: text: Okay; this article is relative to Byte_order_mark and not only to files 86.76.39.126 (talk) 22:39, 24 November 2012 (UTC)
thread_title: RFC 5198

message: Probably should start a new section about this. I am VERY opposed to the current wording and will try to explain why.
strebe: I still cannot figure out what your objection is or what you believe you’re saying or even what you believe the article text is saying. Whether UTF-8 or a legacy encoding, a heuristic is required for detection, just as the text states. The text states legacy encodings are complicated. It states UT8-8 is simpler. Why is any of that controversial?
The disputed sentence: heuristic analysis is required to determine what character encoding a file is using. Many extant algorithms for distinguishing legacy encodings are complicated, error-prone, or slow
I will try VERY HARD to explain my objection. Can you please read this carefully and post any question you have:
Imagine there is a function that checks if the BOM is at the start of the file. It is called hasUTF8BOM().
Imagine a second function that does pattern matching to determine if a file contains only valid UTF8, and it is called matchUTF8(). It is in no way "error-prone" (it is in fact a good deal more reliable than relying on the odds that the first 3 bytes happen to not be the BOM in non-UTF8). You can argue about whether it is "complicated" or "slow" but imho it is neither of them when compared to the next function.
Let's imagine a third function which examines a text string called getLegacyEncoding(). This returns every other encoding you are interested in other than UTF8. I think it is fair to describe this function as "complicated, error-prone, or slow" and in fact that is exactly what the text is referring to.
Okay, lets write a program that uses these functions and returns true/false as to whether you think the file is UTF-8:
Version 1:
   isUTF8(f): return hasUTF8BOM(f)
Version 2:
   isUTF8(f): return matchUTF8(f)
Wait a second! Neither version calls getLegacyEncoding()! So why is there some text talking about that when discussing using version 2????
Okay, maybe your concern is that you do need to figure out the legacy encoding. Lets try some new functions that figure out the encoding using the above:
Version 1:
   getEncoding(f): return hasUTF8BOM(f) ? UTF8 : getLegacyEncoding(f)
Version 2:
   getEncoding(f): return matchUTF8(f) ? UTF8 : getLegacyEncoding(f)
Oops! You need to call getLegacyEncoding() in both of them!
Do you understand? The complexity of distinguishing legacy encodings is irrelevant when choosing whether to use the BOM or not. Both cases need or can ignore it equally. Therefore mentioning how hard that is in the context of not using a BOM is misleading.
Spitzak (talk) 01:16, 21 May 2015 (UTC)
replies: text: There isn’t anything wrong with your logic. Again, it’s about your interpretation of the article’s meaning and intent. The complexity of distinguishing legacy encodings is relevant because many programmers will choose a library that does not look specifically for UTF-8 indicators first but instead acts as a general detector for character sets. Not only that, even if the programmer does the right thing “herself” for detecting encoding, a choice she makes about encoding affects systems that she has no control over: that text will end up in places where her choice of encoding suffers because the downstream system uses a slow, buggy character encoding detector—or is a Microsoft product. The article helpfully points out that UTF-8 is easier to detect, which you could read as a hint that the programmer should try to detect UTF-8 first if the problem domain is likely to be Unicode. But you seem to be so obsessed with the horrifying thought that someone might treat UTF-8 as a peer to other encodings that you interpret the article as, “If you don’t use a BOM then you’re stuck dealing with the messy legacy encoding problem.” Well, in a sense you are stuck with such problems, because you have no control over how others will handle your text. Meanwhile if you choose to use a BOM instead, your text just works with Microsoft products, and presumably any character encoding detection system will recognize the the encoding reliably as well. But the article emphasizes neither BOM or BOM-less. It is neutral and merely points out a few facts about the consequences of choice. As it should. Strebe (talk) 08:45, 22 May 2015 (UTC)
thread_title:  "legacy encodings" objection 

message: Programs expecting UTF-8 may show these or error indicators, depending on how they handle UTF-8 encoding errors. In all cases they will probably display the rest of the file as garbage (a UTF-16 text containing ASCII only will be fairly readable).
The italicized sentence here is not at all clear to me. Is it saying that a program expecting a UTF-8 file will display a UTF-16 file as garbage outside of parts of the file that are ASCII only? Why doesn't it just say that, instead of what I first interpreted as that it would display a file beginning with a UTF-16 BOM as garbage because of the BOM, which is what it might be taken to mean from the context, and is obviously silly because of the way UTF-8 resyncs. Also, saying that display of an ASCII only file in this case is fairly readable is a bit of a stretch, for instance is display of a highlighted NUL between each pair of characters fairly readable?
In fact, if this sentence is talking about how a program expecting UTF-8 displays UTF-16 in general, why is it in the article on BOM anyway without some clarification about what this has to do with BOM? 2601:646:8D01:8A90:29F4:44B8:5515:B1EE (talk) 07:25, 3 October 2015 (UTC)
replies: text: The italicized text is just some random observation by some random editor and doesn't seem to have anthing to do with BOM. Feel free to fix it. Strebe (talk) 07:47, 3 October 2015 (UTC)
thread_title:  Huh? 

message: I added a mention that the UTF-32LE BOM is the same byte pattern as a UTF-16LE BOM followed by a null (0) character. I thought this was interesting and indicates an example where blindly obeying the bit patterns does not work. Somebody else thought to add a lot of text which I think amounts to "text starting with null is very uncommon so this is not a problem". He seems to think it is because null is a string terminator in null-terminated strings, but actually that makes a leading null *MORE* common, since zero-length strings are probably by far the most common. The real reason this is not a problem is that UTF-32 is so very rare that there is no reason to test for it.
I guess I'll just delete this, as the tiny anecdote has inflated to a mess of unreadable text.
Spitzak (talk) 19:09, 27 January 2016 (UTC)
thread_title:  UTF-32LE matching UTF-16LE 

message: This has been deleted as OR but it is actually based on several answers from Stack Exchange which is not allowed as a source. It also has provable math statements in it. Strebe does not seem to understand it, making two mistakes: 1/15 chance of an error is not 85% chance of it being correct, it is 93.3%. And that is for ONE character, the chance of finding N correct multibyte characters is 1-1/15^N which quickly becomes astronomical. For instance finding 7 UTF-8 characters without first finding an invalid sequence is 1/170,869,375. This can be compared to the 1/16,777,216 chance of the first three bytes being the BOM, a chance that is assumed to be zero by defenders of the BOM method.
This question of odds is asked quite a few times on the internet, though there are a lot of incorrect answers. I thought it would be useful to provide some kind of answer.
Spitzak (talk) 22:44, 29 June 2017 (UTC)
thread_title:  Repeated deletion of math showing chances of misidentifying another encoding as UTF-8 

message: A tiny issue of language. My amendment  was reverted with the comment 'As in, to “ingest” and serially “use up” the incoming stream. This is standard terminology.' As a native English speaker, I am confident my edit was an improvement. As a reader, I found the word 'consuming' for what a program does with an input string both jarring and jargon. It lacks clarity, and is not consistent with English usage. Detection of character encoding and byte order does not 'ingest', 'consume' or 'use up' anything. If this were standard terminology, why is there no common method consume()? Instead we have read() and open(), the standard analogy being with a book. That analogy is surely clearer to a reader. (Conversely 'emit' may be used occasionally in a computing context as a substitute for 'print', but not 'excrete' or 'egest'.) I'd not seen the word 'consume' in any Unicode documentation I've been reading recently. The first language of English Wikipedia is English. I'll reinstate the correction once and once only. --Cedderstk 06:43, 29 October 2018 (UTC)
replies: text: In standard computer terms, reading implies only the transfer of data, e.g. from a file or network into memory.  Consuming involves interpreting the data.  Of course it is possible do both in parallel, but the reading part is independent of the type of data (for example it would be the same whether it was text, an image, etc.) and then consuming depends on the type of data.  If multiple types of data are accepted, the first step in consuming the data is often to identify the type of the data, and the next step would be to process it depending on the type that was identified.  The article is really talking about this step of identifying the type of data, which is often the first step in consuming the data after it is read.  The byte order marks tells it that it is text (and not, say, an image) and also the specific encoding of the text, so that it can then be processed/parsed.  I didn't revert but I don't think "reading" or "accessing" is an improvement; both of those terms refer to transfer of raw data that occurs before it is consumed or interpreted, but the byte order mark is about interpreting the data that was read. -LiberatorG (talk) 15:44, 29 October 2018 (UTC)
replies: text: How about "interpreting" then? "consuming" does have the problem that (at least for many readers) it implies the destruction of the original data.Spitzak (talk) 17:16, 29 October 2018 (UTC)
replies: text: I have no issue with "interpreting". -LiberatorG (talk) 18:01, 29 October 2018 (UTC)
thread_title:  'Consuming' 

message: Unicode is no encoding. It is a charset and provides several encodings. Thus, it should either read "the text stream's character set is Unicode" or the "text stream has a Unicode encoding". --Meillo (talk) 18:56, 31 January 2021 (UTC)
replies: text: I don’t think anyone is confused by the present verbiage, which is merely shorthand for “the text stream’s encoding is one of Unicode’s encoded forms”. Unicode isn’t a “charset”, either; Unicode is standard that defines a character repertoire, a code point for each abstract character in that repertoire, several encoded forms for the set of code points, and a lot of rules and recommendations. I don’t think either of your proposed changes is satisfactory. In the first case, Unicode does not use the term “character set” because the term is not well defined. In the second case, almost any text stream “has a Unicode encoding”; i.e., can be represented in a Unicode encoding by some transformation. Strebe (talk) 00:40, 1 February 2021 (UTC)
replies: text: So, in the end, the presence of a BOM only really tells that there (most likely) is some Unicode stuff involved? ;-) I really would like to get rid of the words "the encoding is Unicode". There is so much confusion in the whole topic, and wordings like this one add to it. I like the long form (“the text stream’s encoding is one of Unicode’s encoded forms”) much more, as it provides more clarity. But actually, is this really what the bullet point wants to point out? In relation to the third bullet point ("Which Unicode character encoding is used."), the second one maybe should rather be: "The fact that the text stream is Unicode, to a high level of confidence;" (i.e. omitting "'s encoding"). --Meillo (talk) 03:07, 1 February 2021 (UTC)

text: I’m fine with “the text stream is Unicode”. Yes, the BOM just says Unicode is probably involved. Strebe (talk) 04:50, 1 February 2021 (UTC)
thread_title:  "the text stream's encoding is Unicode"