Repository: postcss/postcss (ts)

Issue:
CSS syntax error reporting wrong line/column position
@Arcanemagnus originally filed this as an issue with stylelint (https://github.com/stylelint/stylelint/issues/718):

With this CSS file:

```css
.s["] {
}
/* "" */
```

There is a CSS syntax error that reports the column of the unclosed quote on line 1 as being 18, which is far past the end of the line. It seems like it is counting all characters between the first quote and the next one as part of the line? When the comment with the `"` is removed the proper value is returned. If that comment is brought to the same line as the initial `"` the issue disappears as well.

Based on a report initially filed here: https://github.com/AtomLinter/linter-stylelint/issues/79.
![image](https://cloud.githubusercontent.com/assets/427137/12932639/88718672-cf38-11e5-87c8-94a7376ada7c.png)




This is different from #725 because the error is thrown during the tokenize step (so it's not using `node.positionBy()`).

I think that the syntax error is causing weird problems with text editor plugins, because it claims that things are happening in positions that do not exist.

Do you think there's a way to make these positions more accurate? If not, maybe text editor plugins just need to do something different when receiving CSS syntax errors from PostCSS?

Pull request description:
Track newlines within strings; fixes #727
I believe that the issue causing #727 is that the tokenizer was not registering newlines within strings. So whatever came after the string was thought to be on the same line as the string's opening --- which in this instance was not the case. 

I copied some logic from the tokenization of comments and verified that it works with a test. I also verified that the particular issue in #727 (the syntax error having a weird position) is fixed by this.

Patch:
diff --git a/lib/tokenize.es6 b/lib/tokenize.es6
index 0befefbc..bef6e308 100644
--- a/lib/tokenize.es6
+++ b/lib/tokenize.es6
@@ -143,11 +143,26 @@ export default function tokenize(input) {
                 }
             } while ( escaped );
 
+            content = css.slice(pos, next + 1);
+            lines   = content.split('\n');
+            last    = lines.length - 1;
+
+            if ( last > 0 ) {
+                nextLine   = line + last;
+                nextOffset = next - lines[last].length;
+            } else {
+                nextLine   = line;
+                nextOffset = offset;
+            }
+
             tokens.push(['string', css.slice(pos, next + 1),
                 line, pos  - offset,
-                line, next - offset
+                nextLine, next - nextOffset
             ]);
-            pos = next;
+
+            offset = nextOffset;
+            line   = nextLine;
+            pos    = next;
             break;
 
         case AT:


Test patch:
diff --git a/test/tokenize.es6 b/test/tokenize.es6
index 475cda2b..fdfdf3e4 100644
--- a/test/tokenize.es6
+++ b/test/tokenize.es6
@@ -98,6 +98,13 @@ describe('tokenize', () => {
         test('"\\\\"', [ ['string', '"\\\\"', 1, 1, 1, 4] ]);
     });
 
+    it('changes lines in strings', () => {
+        test('"\n\n""\n\n"', [
+            ['string', '"\n\n"', 1, 1, 3, 1],
+            ['string', '"\n\n"', 3, 2, 5, 1]
+        ]);
+    });
+
     it('tokenizes at-word', () => {
         test('@word ', [ ['at-word', '@word', 1, 1, 1, 5], ['space', ' '] ]);
     });