int3/doppio — pull request #217 by angrave (2013-05-22 19:34:34+00:00)
Title: makefile changes to support Javac preload items + source maps

The pull request contains two items:

1. Support for additional preload items to reduce first compilation time -

Right now, compiling helloworld.java is probably the number one use case of people playing online with Doppio.
Using Doppio in the browser I compiled an empty Java file and watched the network traffic. This loaded 173 additional classes (!)
So I turned this list of class http requests into a static file list, 'tools/preload-compile-extras'
and modified the makefile to append the contents of this file to the tar,

if [ -f tools/preload-compile-extras ]; then \
cat tools/preload-compile-extras >> tools/preload; \
fi;

This significantly reduces the first compilation time when Doppio is used on the Internet
Suggestion: it would be cute if this optimization could be automated for any specific doppio application, not just javac.

2 Source Maps for Doppio

Here's a Chrome Demo -
i) First "Enable Source Maps" in Chrome.Click on the bottom-right wheel in the JavaScript console to get to the Settings,
[pic]
ii) Now start doppio web project in release mode
[pic]
./tools/webrick.rb --release

and edit a new Java file ("edit A.java") and Save.
While compiling ("javac A.java") open the Sources tab and click the pause button on the far right.
Instead of opening the unreadable compressed.js, the much more readable uncompressed javascript is displayed -

[pic]

iii) Stop the webserver and restart in dev mode
./tools/webrick.rb --dev

In the dev version we use the coffeescript map files, so can view the original .coffee -
[pic]

Hurrah.

How it works

For the compressed.js release:
Eventually sourcemaps will be able to track all the way back to the original coffeescript, but right now uglify only supports a single input source map, so for the browser, we cat all of the coffeescript output into a single js ('uncompressed.js') and sourcemap to that instead.

I've updated the makefile so that coffee is at least creating source maps for each coffee file; we're just not using them yet:
build/release/compressed.js
...
$(SED) -r "s/^( *)(debug|v?trace).*$$/\1\`\`/" < $$src > $(dir $@)browser/doppio-source/$$src ; \
$(COFFEEC) --map -o $(dir $@) --print -c $(dir $@)browser/doppio-source/$$src; \
# Individual Coffee maps are created but unused by uglify.
$(UGLIFYJS) --prefix 2 --source-map-url compressed.map --source-map ${@:.js=.map} --define RELEASE --define UNSAFE --no-mangle --unsafe -o $@ ${@:compressed.js=uncompressed.js}

FYI To build sourcemaps you can no longer use stdin with coffee.

For the build/release target (not the uncompressed.js) target we can have direct-to-coffee source maps because each coffee .js file is uglify-ed but they still might need some tweaking because I don't think they are used by the browser, so don't know how to test them..
We first create coffeex (trace statements remove), and then compile these files. Uglify seems to be read the entire input map in one go, so it's OK to use the same output map as the input map.
build/release/%.js: %.coffee
@mkdir -p $(dir $@)
$(SED) -r "s/^( *)(debug|v?trace).*$$/\1\`\`/" $< > $(@:.js=.coffeex)
$(COFFEEC) --map -o $(dir $@) $(@:.js=.coffeex)
mv $@ $(@:.js=-orig.js)
$(UGLIFYJS) --source-map ${@:.js=.map} --in-source-map ${@:.js=.map} --define RELEASE --define UNSAFE --no-mangle --unsafe --beautify -o $@ $(@:.js=-orig.js)
rm $(@:.js=-orig.js)

Note uglify's --overwrite switch seemed to be broken (infinite loop or waiting for stdin perhaps, or perhaps just a typo at the time). Anyway I side-stepped the issue by renaming the intermediate file. Perhaps this should be implemented as two Makefile targets, but I kept running into makefile dependency snarfus and ran out of time.

For the debug target we can have direct-to-coffee source maps,
build/dev/%.js: %.coffee
@mkdir -p $(dir $@)
ln -s ../../../$< $(dir $@)
cd $(dir $@)&& $(COFFEEC) --map -o . -c $(notdir $<)

Notice we now need to include a copy of the original source. We need to ensure that the relative URL to the map file and the .coffee file is correct.
There's no way to the URL root in the coffee compiler, so after a few hiccups/bad starts, I realized the tools worked best with everything in the same directory.

This took an surprising long time to implement (>8hours) and took many failed attempts. .Sorry about that. Some of the gotchas along the way -
SourceMap relative URLS != project paths
Not fully understanding some of the subtles of Doppio's makefile targets
Limited support/bugs/poor errormessage feedback for source maps with uglify and coffee

In a future decoupling/Makefile revisions, I recommend doing 'ln -s' so that the .coffee, .map and .js are in the same directory. It will also help keep the Makefile commands for sourcemaps sane.
Otherwise we will have to fixup the map files using regular expression hacking.
-Lawrence.