github/linguist — pull request #1211 by alindeman (2014-05-21 15:52:45+00:00)
Title: Counts the number of lines correctly for files with certain multibyte encodings

Currently, `loc` and `sloc` are incorrect for UTF-16 encoded files with Windows line endings ([example](https://github.com/YuriyGuts/winforms-auto-taborder-vsaddin/blob/master/src/Connect.cs)). This is because `data` and the newline regular expression are encoded as ASCII-8BIT or UTF-8, even if the encoding is detected as something else. In the case of UTF-16LE, Windows newlines are encoded as `"\r\0\n\0"`. Because of the `\0` separating the `\r` and `\n`, the current regular expression `(\r|\n)` counts two lines even though it should only be interpreted as one single line break.

The symptom is that [UTF-16LE encoded files with Windows line endings](https://github.com/YuriyGuts/winforms-auto-taborder-vsaddin/blob/master/src/Connect.cs) are rendered with whitespace at the bottom that doesn't actually exist because the line count is calculated as twice what it should be.

/cc: @github/encodings