Repository: sveltejs/svelte (JavaScript)
Date: 2021-06-22 02:59:43+00:00
Task type: Bug Fix

Issue:
Segment-less sourcemap generated for static templates
This originally broke source maps with SvelteKit. [A workaround was added to Vite](https://github.com/vitejs/vite/pull/2441), but we still don't get source maps for static files.

Svelte generates a segment-less sourcemap when there's no dynamic content in the template. Appears it's coming from the use of code-red to print out the AST tree: https://github.com/Rich-Harris/code-red/blob/3b32d2ef5bd954cb85a0d005f3a328bae57c6a97/src/print/index.ts#L60-L86

Normally, the AST being printed would contain loc objects that point into the original code. But when Svelte is processing a file without any dynamic code, it doesn't attach any locs (though it does add start and end). I think this is a bad behavior, Svelte should be associating source location for static code as well. Eg, something like the following:

```
<h1>decoded-sourcemap</h1>
<div>replace me</div>
```

The `h1` should be associated with `{ start: { line: 1, column: 2 } }`, the decoded-sourcemap with `{ start: { line: 1, column: 5 } }`, etc. With the AST given the correct loc associations, code-red's printer will make a sourcemap with valid segments, and the bug will disappear.

CC @milahu @dmitrage as our source map experts


Issue discussion:
<details>

<summary>how NOT to reproduce</summary>

```svelte
<div>
  <button onclick="console.log('this is App.svelte line 2')">click me</button>
</div>
```

actual result
```
this is App.svelte line 2                 (index):19
```

~~probably this should say `App.svelte:2`~~
</details>

<details>

<summary>where does chunk come from?</summary>

```js
// in code-red print(): console.dir(chunk);
{ content: 'main', loc: undefined, has_newline: false }
{ content: '.', loc: undefined, has_newline: false }
{ content: 'innerHTML', loc: undefined, has_newline: false }
{ content: ' = ', loc: undefined, has_newline: false }
{ content: '`', loc: undefined, has_newline: false }
{
  content: '<button onclick="console.log(&#39;this is App.svelte line 2&#39;)">click me</button>',
  loc: undefined,
  has_newline: false
}
// console.trace(chunk):
Trace: {
  content: '<button onclick="console.log(&#39;this is App.svelte line 2&#39;)">click me</button>',
  loc: undefined,
  has_newline: false
}
    at print (compiler.js:1978:10)
    at Component.generate (compiler.js:20088:9)
    at compile (compiler.js:21405:19)
    at Object.transform (rollup-plugin-svelte/index.js:111:21)
```

`Component.generate`

```js
generate(result) {

  const program = { type: 'Program', body: result.js };

  js = print(program, {
    sourceMapSource: compile_options.filename
  });
```

`compile`

```js
function compile(source, options = {}) {
  const ast = parse$2(source, options);
  const component = new Component(
    ast,
    source,
    options.name || get_name_from_filename(options.filename) || 'Component',
    options,
    stats,
    warnings
  );
  const result = dom(component, options);
  //const result = ssr(component, options);
  return component.generate(result);
}
```


</details>

<details>

<summary>where is the source location lost?</summary>

relevant code in [src/compiler/compile/render_dom/wrappers/Element/index.ts](https://github.com/sveltejs/svelte/blob/ce3a5791258ec6ecf8c1ea022cb871afe805a45c/src/compiler/compile/render_dom/wrappers/Element/index.ts#L303) line 303

```ts
to_html((this.fragment.nodes as unknown as Array<ElementWrapper | TextWrapper>), block, literal, state, can_use_raw_text);
```

`this.fragment.nodes` has source locations of html elements

```js
this.fragment.nodes[0] ==
ElementWrapper {
  node: Element$1 {
    start: 8,
    end: 84,
    type: 'Element',
    attributes: [Array],
    name: 'button',

```


after `to_html`, the locations are lost:
```js
state.quasi ==
{
  type: 'TemplateElement',
  value: {
    raw: '<button onclick="console.log(&#39;this is App.svelte line 2&#39;)">click me</button>'
  }
}
```

[src/compiler/compile/render_dom/wrappers/Element/index.ts](https://github.com/sveltejs/svelte/blob/ce3a5791258ec6ecf8c1ea022cb871afe805a45c/src/compiler/compile/render_dom/wrappers/Element/index.ts#L306) line 306
```js
block.chunks.create.push(
  b`${node}.${this.can_use_innerhtml ? 'innerHTML' : 'textContent'} = ${literal};`
);
```

</details>

<details>

<summary>what is b`....` ?</summary>

what is ``( b`....` )``? [code-red](https://github.com/Rich-Harris/code-red/blob/3b32d2ef5bd954cb85a0d005f3a328bae57c6a97/src/index.ts#L188)

```js
function b(strings, ...values) {
  const str = join$1(strings);
  const comments = [];
  const ast = acorn.parse(str,  acorn_opts(comments, str));
  inject(str, ast, values, comments);
  return ast.body;
}
```

```js
ast.body[0].expression.right ==
{
  type: 'TemplateLiteral',
  expressions: [],
  quasis: [
    {
      type: 'TemplateElement',
      value: {
        raw: '<button onclick="console.log(&#39;this is App.svelte line 2&#39;)">click me</button>'
      }
    }
  ]
}
```

</details>

to trace markup, we would need major changes in `render_dom` and `render_ssr`

I think the empty source map makes sense in a way because there is no JS transformation done on a html-only code, so Svelte does not add any mappings. In order to support this case, the compiler needs to add some mappings to the html-part, don't know how tricky that would be.
> I think the empty source map makes sense in a way because there is no JS transformation done on a html-only code, so Svelte does not add any mappings.

Note that even when I fix https://github.com/ampproject/remapping/issues/116, your sourcemaps are going to look broken. The remapped output would look like:

```json
{
  "version": 3,
  "mappings": "AAAA,CAAC,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC,CAAC;;;;;;;",
  "names": [],
  "sources": [ "App.svelte" ]
}
```

Notice the `;;;;;;;`, and only segments on the first line. Well, the first line of svelte output is `/* App.svelte generated by Svelte v3.35.0 */`. If you were to load this into a sourcemap visualizer, it'd look like a comment was created from HTML elements (the `<h1>` and `<div>`). It won't make any sense.

> In order to support this case, the compiler needs to add some mappings to the html-part, don't know how tricky that would be.

Not too tricky. You're already tracking `start` and `end` indexes as @milahu details in the "where is the source location lost?" section. You just need to convert that to `{ line: number, column: number }` by figuring out how many newline characters came before the `start`/`end` indexes (and how many columns into that line you are). A datastructure to track the last index lookup and incrementally process would make it decently fast.

Solution patch:
diff --git a/src/compiler/compile/render_dom/wrappers/Element/index.ts b/src/compiler/compile/render_dom/wrappers/Element/index.ts
--- a/src/compiler/compile/render_dom/wrappers/Element/index.ts
+++ b/src/compiler/compile/render_dom/wrappers/Element/index.ts
@@ -19,7 +19,7 @@ import { add_event_handler } from '../shared/add_event_handlers';
 import { add_action } from '../shared/add_actions';
 import bind_this from '../shared/bind_this';
 import { is_head } from '../shared/is_head';
-import { Identifier } from 'estree';
+import { Identifier, ExpressionStatement, CallExpression } from 'estree';
 import EventHandler from './EventHandler';
 import { extract_names } from 'periscopic';
 import Action from '../../../nodes/Action';
@@ -263,15 +263,23 @@ export default class ElementWrapper extends Wrapper {
 		}
 
 		if (parent_node) {
-			block.chunks.mount.push(
-				b`@append(${parent_node}, ${node});`
-			);
+			const append = b`@append(${parent_node}, ${node});`;
+			((append[0] as ExpressionStatement).expression as CallExpression).callee.loc = {
+				start: this.renderer.locate(this.node.start),
+				end: this.renderer.locate(this.node.end)
+			};
+			block.chunks.mount.push(append);
 
 			if (is_head(parent_node)) {
 				block.chunks.destroy.push(b`@detach(${node});`);
 			}
 		} else {
-			block.chunks.mount.push(b`@insert(#target, ${node}, #anchor);`);
+			const insert = b`@insert(#target, ${node}, #anchor);`;
+			((insert[0] as ExpressionStatement).expression as CallExpression).callee.loc = {
+				start: this.renderer.locate(this.node.start),
+				end: this.renderer.locate(this.node.end)
+			};
+			block.chunks.mount.push(insert);
 
 			// TODO we eventually need to consider what happens to elements
 			// that belong to the same outgroup as an outroing element...
diff --git a/src/compiler/compile/render_dom/wrappers/Text.ts b/src/compiler/compile/render_dom/wrappers/Text.ts
--- a/src/compiler/compile/render_dom/wrappers/Text.ts
+++ b/src/compiler/compile/render_dom/wrappers/Text.ts
@@ -44,10 +44,19 @@ export default class TextWrapper extends Wrapper {
 		if (this.skip) return;
 		const use_space = this.use_space();
 
+		const string_literal = {
+			type: 'Literal',
+			value: this.data,
+			loc: {
+				start: this.renderer.locate(this.node.start),
+				end: this.renderer.locate(this.node.end)
+			}
+		};
+
 		block.add_element(
 			this.var,
-			use_space ? x`@space()` : x`@text("${this.data}")`,
-			parent_nodes && (use_space ? x`@claim_space(${parent_nodes})` : x`@claim_text(${parent_nodes}, "${this.data}")`),
+			use_space ? x`@space()` : x`@text(${string_literal})`,
+			parent_nodes && (use_space ? x`@claim_space(${parent_nodes})` : x`@claim_text(${parent_nodes}, ${string_literal})`),
 			parent_node as Identifier
 		);
 	}


Test patch:
diff --git a/test/sourcemaps/samples/decoded-sourcemap/_config.js b/test/sourcemaps/samples/decoded-sourcemap/_config.js
--- a/test/sourcemaps/samples/decoded-sourcemap/_config.js
+++ b/test/sourcemaps/samples/decoded-sourcemap/_config.js
@@ -3,7 +3,7 @@ import { magic_string_preprocessor_result, magic_string_replace_all } from '../.
 
 export default {
 
-	js_map_sources: [], // test component has no scripts
+	js_map_sources: ['input.svelte'],
 
 	preprocess: {
 		markup: ({ content, filename }) => {
diff --git a/test/sourcemaps/samples/external/_config.js b/test/sourcemaps/samples/external/_config.js
--- a/test/sourcemaps/samples/external/_config.js
+++ b/test/sourcemaps/samples/external/_config.js
@@ -10,7 +10,7 @@ export const STYLES = '.awesome { color: orange; }\n';
 
 export default {
 	css_map_sources: ['common.scss', 'styles.scss'],
-	js_map_sources: [],
+	js_map_sources: ['input.svelte'],
 	preprocess: [
 		{
 			style: () => {
diff --git a/test/sourcemaps/samples/markup/input.svelte b/test/sourcemaps/samples/markup/input.svelte
new file mode 100644
--- /dev/null
+++ b/test/sourcemaps/samples/markup/input.svelte
@@ -0,0 +1,2 @@
+<h1>decoded-sourcemap</h1>
+<div>replace me</div>
\ No newline at end of file
diff --git a/test/sourcemaps/samples/markup/test.js b/test/sourcemaps/samples/markup/test.js
new file mode 100644
--- /dev/null
+++ b/test/sourcemaps/samples/markup/test.js
@@ -0,0 +1,34 @@
+export function test({ assert, input, js }) {
+	let expected;
+
+	let start;
+	let actual;
+
+	start = js.locate('insert(target, h1');
+	expected = input.locate('<h1');
+	actual = js.mapConsumer.originalPositionFor({
+		line: start.line + 1,
+		column: start.column
+	});
+
+	assert.deepEqual(actual, {
+		source: 'input.svelte',
+		name: null,
+		line: expected.line + 1,
+		column: expected.column
+	});
+
+	start = js.locate('insert(target, div');
+	expected = input.locate('<div');
+	actual = js.mapConsumer.originalPositionFor({
+		line: start.line + 1,
+		column: start.column
+	});
+
+	assert.deepEqual(actual, {
+		source: 'input.svelte',
+		name: null,
+		line: expected.line + 1,
+		column: expected.column
+	});
+}
diff --git a/test/sourcemaps/samples/sourcemap-basename/_config.js b/test/sourcemaps/samples/sourcemap-basename/_config.js
--- a/test/sourcemaps/samples/sourcemap-basename/_config.js
+++ b/test/sourcemaps/samples/sourcemap-basename/_config.js
@@ -15,7 +15,7 @@ span {
 
 export default {
 	css_map_sources: [external_relative_filename],
-	js_map_sources: [],
+	js_map_sources: ['input.svelte'],
 	preprocess: [
 		{
 			style: ({ content, filename }) => {
diff --git a/test/sourcemaps/samples/sourcemap-offsets/_config.js b/test/sourcemaps/samples/sourcemap-offsets/_config.js
--- a/test/sourcemaps/samples/sourcemap-offsets/_config.js
+++ b/test/sourcemaps/samples/sourcemap-offsets/_config.js
@@ -3,7 +3,7 @@ import { magic_string_bundle } from '../../helpers';
 export const EXTERNAL = 'span { --external-var: 1px; }';
 
 export default {
-	js_map_sources: [],
+	js_map_sources: ['input.svelte'],
 	css_map_sources: ['input.svelte', 'external.css'],
 	preprocess: [
 		{