Repository: sveltejs/svelte (JavaScript)
Date: 2020-11-29 15:34:50+00:00
Task type: Bug Fix

Issue:
Error in preprocessor sourcemapping: "Cannot read property 'length' of undefined"
**Describe the bug**
[The new preprocessor sourcemapping feature](https://github.com/sveltejs/svelte/pull/5584/files#diff-8742d6c5cbad8aec49f195bc6d25b152ac0cd8180cb876c019f3b74736f9d62aR15) seems to sometimes cause errors (CC @halfnelson @milahu)

**Logs**

```
> sapper-postcss-template@2020.11.18 build /Users/babichjacob/Repositories/sapper-postcss-template
> cross-env NODE_ENV=production sapper build --legacy

> Building...

[!] (plugin svelte) TypeError: Cannot read property 'length' of undefined
/Users/babichjacob/Repositories/sapper-postcss-template/src/routes/index.svelte
TypeError: Cannot read property 'length' of undefined
    at sourcemap_add_offset (/Users/babichjacob/Repositories/sapper-postcss-template/node_modules/svelte/compiler.js:22087:23)
    at get_replacement (/Users/babichjacob/Repositories/sapper-postcss-template/node_modules/svelte/compiler.js:28575:10)
    at /Users/babichjacob/Repositories/sapper-postcss-template/node_modules/svelte/compiler.js:28635:21
    at async Promise.all (index 0)
    at async replace_async (/Users/babichjacob/Repositories/sapper-postcss-template/node_modules/svelte/compiler.js:28551:52)
    at async preprocess_tag_content (/Users/babichjacob/Repositories/sapper-postcss-template/node_modules/svelte/compiler.js:28618:22)
    at async preprocess (/Users/babichjacob/Repositories/sapper-postcss-template/node_modules/svelte/compiler.js:28644:10)
    at async Object.transform (/Users/babichjacob/Repositories/sapper-postcss-template/node_modules/rollup-plugin-svelte/index.js:100:23)
    at async ModuleLoader.addModuleSource (/Users/babichjacob/Repositories/sapper-postcss-template/node_modules/rollup/dist/shared/rollup.js:18289:30)
    at async ModuleLoader.fetchModule (/Users/babichjacob/Repositories/sapper-postcss-template/node_modules/rollup/dist/shared/rollup.js:18345:9
```

**To Reproduce**
`git clone git@github.com:babichjacob/sapper-postcss-template.git`

Upgrade to svelte 3.30 and rollup-plugin-svelte 7 following the upgrade guide (example of that here: https://github.com/sveltejs/sapper-template/pull/289)

sourcemap = false -> no error
sourcemap = "inline" -> that error
sourcemap = true -> that error


**Severity**
Several people have reported this issue on Discord. It seems to happen relatively frequently


Issue discussion:
My analysis so far:
The PostCSS processor in `svelte-preprocess` is returning a `SourceMapGenerator` instead of whatever the svelte compiler is expecting it to return. This `SourceMapGenerator` is coming directly from the `postcss` package. I'm not sure where exactly the `SourceMapGenerator` is supposed to turn into an actual source map, but that seems to be the missing step AFAICT.

Update:
Looks like just changing the postcss transformer isn't sufficient. 


This is certainly not the exact right fix, but adding this in the Svelte `compiler.js` at line 28636, just before the call to `get_replacement` allows everything to build without errors:
```
if (processed.map?._mappings) {
    processed.map = processed.map.toString();
}
```




omg thank you
> My analysis so far:
> The PostCSS processor in `svelte-preprocess` is returning a `SourceMapGenerator` instead of whatever the svelte compiler is expecting it to return. This `SourceMapGenerator` is coming directly from the `postcss` package. I'm not sure where exactly the `SourceMapGenerator` is supposed to turn into an actual source map, but that seems to be the missing step AFAICT.
> 
> Update:
> Looks like just changing the postcss transformer isn't sufficient.
> 
> This is certainly not the exact right fix, but adding this in the Svelte `compiler.js` at line 28636, just before the call to `get_replacement` allows everything to build without errors:
> 
> ```
> if (processed.map?._mappings) {
>     processed.map = processed.map.toString();
> }
> ```

it's giving me `UnhandledPromiseRejectionWarning: /Users/jt/dev/code-challenges-sapper/node_modules/svelte/compiler.js:28636
                                if (processed.map?._mappings) { 
SyntaxError: Unexpected token '.'`
Then your Node version is too old to support that syntax. You're probably better off just using Svelte 3.29 for now.
Phew, glad I'm not alone. Also seems to be originating from `svelte-preprocess`.
> Then your Node version is too old to support that syntax. You're probably better off just using Svelte 3.29 for now.

I'm using Svelte 3.17.3 ......
If you're seeing this bug, you're using Svelte 3.30. 

https://bytearcher.com/articles/semver-explained-why-theres-a-caret-in-my-package-json/
It was explained: https://bytearcher.com/articles/semver-explained-why-theres-a-caret-in-my-package-json/

That `package.json` specifies `^3.17.3` which means get the latest 3.x release, which is 3.30. The actual version used is specified in the `package-lock.json`. Try `grep -r -A2 svelte package-lock.json`

The method this bug refers to did not exist until 3.30, which means you're either using 3.30 or encountering a different issue.
> It was explained: https://bytearcher.com/articles/semver-explained-why-theres-a-caret-in-my-package-json/
> 
> 
> 
> That `package.json` specifies `^3.17.3` which means get the latest 3.x release, which is 3.30. The actual version used is specified in the `package-lock.json`. Try `grep -r -A2 svelte package-lock.json`
> 
> 
> 
> The method this bug refers to did not exist until 3.30, which means you're either using 3.30 or encountering a different issue.

Damn I didn't known it did that, thank you and sorry for being arrogant :/ 
It worked with the ’~‘, thank you

Solution patch:
diff --git a/src/compiler/preprocess/index.ts b/src/compiler/preprocess/index.ts
--- a/src/compiler/preprocess/index.ts
+++ b/src/compiler/preprocess/index.ts
@@ -83,8 +83,81 @@ async function replace_async(
 	return out.concat(final_content);
 }
 
-/** 
- * Convert a preprocessor output and its leading prefix and trailing suffix into StringWithSourceMap  
+/**
+ * Import decoded sourcemap from mozilla/source-map/SourceMapGenerator
+ * Forked from source-map/lib/source-map-generator.js
+ * from methods _serializeMappings and toJSON.
+ * We cannot use source-map.d.ts types, because we access hidden properties.
+ */
+function decoded_sourcemap_from_generator(generator: any) {
+	let previous_generated_line = 1;
+	const converted_mappings = [[]];
+	let result_line;
+	let result_segment;
+	let mapping;
+
+	const source_idx = generator._sources.toArray()
+		.reduce((acc, val, idx) => (acc[val] = idx, acc), {});
+
+	const name_idx = generator._names.toArray()
+		.reduce((acc, val, idx) => (acc[val] = idx, acc), {});
+
+	const mappings = generator._mappings.toArray();
+	result_line = converted_mappings[0];
+
+	for (let i = 0, len = mappings.length; i < len; i++) {
+		mapping = mappings[i];
+
+		if (mapping.generatedLine > previous_generated_line) {
+			while (mapping.generatedLine > previous_generated_line) {
+				converted_mappings.push([]);
+				previous_generated_line++;
+			}
+			result_line = converted_mappings[mapping.generatedLine - 1]; // line is one-based
+		} else if (i > 0) {
+			const previous_mapping = mappings[i - 1];
+			if (
+				// sorted by selectivity
+				mapping.generatedColumn === previous_mapping.generatedColumn &&
+				mapping.originalColumn === previous_mapping.originalColumn &&
+				mapping.name === previous_mapping.name &&
+				mapping.generatedLine === previous_mapping.generatedLine &&
+				mapping.originalLine === previous_mapping.originalLine &&
+				mapping.source === previous_mapping.source
+		) {
+				continue;
+			}
+		}
+		result_line.push([mapping.generatedColumn]);
+		result_segment = result_line[result_line.length - 1];
+
+		if (mapping.source != null) {
+			result_segment.push(...[
+				source_idx[mapping.source],
+				mapping.originalLine - 1, // line is one-based
+				mapping.originalColumn
+			]);
+			if (mapping.name != null) {
+				result_segment.push(name_idx[mapping.name]);
+			}
+		}
+	}
+
+	const map = {
+		version: generator._version,
+		sources: generator._sources.toArray(),
+		names: generator._names.toArray(),
+		mappings: converted_mappings
+	};
+	if (generator._file != null) {
+		(map as any).file = generator._file;
+	}
+	// not needed: map.sourcesContent and map.sourceRoot
+	return map;
+}
+
+/**
+ * Convert a preprocessor output and its leading prefix and trailing suffix into StringWithSourceMap
  */
 function get_replacement(
 	filename: string,
@@ -109,6 +182,10 @@ function get_replacement(
 		if (typeof(decoded_map.mappings) === 'string') {
 			decoded_map.mappings = decode_mappings(decoded_map.mappings);
 		}
+		if ((decoded_map as any)._mappings && decoded_map.constructor.name === 'SourceMapGenerator') {
+			// import decoded sourcemap from mozilla/source-map/SourceMapGenerator
+			decoded_map = decoded_sourcemap_from_generator(decoded_map);
+		}
 		sourcemap_add_offset(decoded_map, get_location(offset + prefix.length));
 	}
 	const processed_with_map = StringWithSourcemap.from_processed(processed.code, decoded_map);
@@ -164,7 +241,7 @@ export default async function preprocess(
 
 	async function preprocess_tag_content(tag_name: 'style' | 'script', preprocessor: Preprocessor) {
 		const get_location = getLocator(source);
-		const tag_regex = tag_name == 'style' 
+		const tag_regex = tag_name === 'style'
 			? /<!--[^]*?-->|<style(\s[^]*?)?(?:>([^]*?)<\/style>|\/>)/gi
 			: /<!--[^]*?-->|<script(\s[^]*?)?(?:>([^]*?)<\/script>|\/>)/gi;
 


Test patch:
diff --git a/test/sourcemaps/samples/source-map-generator/_config.js b/test/sourcemaps/samples/source-map-generator/_config.js
new file mode 100644
--- /dev/null
+++ b/test/sourcemaps/samples/source-map-generator/_config.js
@@ -0,0 +1,25 @@
+import MagicString from 'magic-string';
+import { SourceMapConsumer, SourceMapGenerator } from 'source-map';
+
+export default {
+	preprocess: {
+		style: async ({ content, filename }) =>  {
+			const src = new MagicString(content);
+			const idx = content.indexOf('baritone');
+			src.overwrite(idx, idx+'baritone'.length, 'bar');
+
+			const map = SourceMapGenerator.fromSourceMap(
+				await new SourceMapConsumer(
+					// sourcemap must be encoded for SourceMapConsumer
+					src.generateMap({
+						source: filename,
+						hires: true,
+						includeContent: false
+					})
+				)
+			);
+
+			return { code: src.toString(), map };
+		}
+	}
+};
diff --git a/test/sourcemaps/samples/source-map-generator/input.svelte b/test/sourcemaps/samples/source-map-generator/input.svelte
new file mode 100644
--- /dev/null
+++ b/test/sourcemaps/samples/source-map-generator/input.svelte
@@ -0,0 +1,12 @@
+<h1>Testing Styles</h1>
+<h2>Testing Styles 2</h2>
+<script>export const b = 2;</script>
+<style>
+	h1 {
+		--baritone: red;
+	}
+
+	h2 {
+		--baz: blue;
+	}
+</style>
diff --git a/test/sourcemaps/samples/source-map-generator/test.js b/test/sourcemaps/samples/source-map-generator/test.js
new file mode 100644
--- /dev/null
+++ b/test/sourcemaps/samples/source-map-generator/test.js
@@ -0,0 +1 @@
+export { test } from '../preprocessed-styles/test';