roots/roots — pull request #1215 by austinpray (2014-11-24 04:29:04+00:00)
Title: Manifest.json Asset Pipeline

# Gulp status update

## Changes

### Features

#### Reorganize folders
Folders are now as such:

```
.
├── bower_components
├── assets
│ ├── fonts
│ ├── images
│ ├── scripts
│ │ └── main.js
│ └── styles
│ ├── _bootstrap.less
│ ├── _global.less
│ ├── _variables.less
│ ├── components
│ │ ├── _buttons.less
│ │ ├── _forms.less
│ │ ├── _media.less
│ │ └── _wp-classes.less
│ ├── editor-style.less
│ ├── layouts
│ │ ├── _footer.less
│ │ ├── _general.less
│ │ ├── _header.less
│ │ ├── _pages.less
│ │ ├── _posts.less
│ │ ├── _sidebar.less
│ │ └── pages
│ │ └── _home.less
│ └── main.less
├── dist
│ ├── images
│ ├── fonts
│ ├── scripts
│ │ ├── jquery-1.11.1.js
│ │ ├── modernizr.js
│ │ └── scripts.js
│ └── styles
│ ├── editor-style.css
│ └── main.css
```

There needs to be a clear divide between compiled files and source assets. We
should also do our best to keep the directory structure as flat as possible.
This allows us to give the instructions "do all the fun stuff in the assets
directory, ignore the dist directory."

#### Remove difference between dev and prod tasks
Previously we were differentiating between dev and prod tasks. For instance on
dev you would get `scripts.js` and on prod you would get `scripts.min.js`. This
can lead to incredibly sneaky bugs. Sometimes third-party dependencies or even
first-party code can break if you [mangle][] it with uglify. Also when you
optimize CSS it can break layouts and change the structure if edge cases are
triggered.

The easiest way to prevent these types of errors is to do compile your
development code as if it is production code. That way you immediately know when
things are broken and you are not figuring out things are broken after pushing
to staging/prod. If you need to do debugging you can do so via sourcemaps.

Also this simplifies the PHP code. The PHP code is not concerned whether it is
in dev or production, it just knows it should look for the relevant code in the
`dist/` directory. Let the build process make decisions about how to handle
assets. *PHP code still has to check for prod/dev to read asset manifest. It
will have to do this until gulp-rev gets some issues closed. See conclusion.*

**This removes the "dev" task. Just run `gulp` instead.**

#### Implement JSON file based asset pipeline

- **Manifest file error checking:** The manifest file checks itself for format
errors and displays error messages accordingly.
- **Sensible defaults:** if the user's manifest.json is missing required
attributes, fill them in with our project defaults. Keeps the manifest.json
small.
- **flexible format:** each property supports the full JSON spec. You can
specify globs as either strings or arrays and they will be dealt with
accordingly.
- **configurable build dirs:** `asset/` dir and `dist/` dir are configurable via
the json file

#### Use wiredep for sass and less injection
https://github.com/taptapship/wiredep

all third-party deps are injected via wiredep rather than manually managed

### Housekeeping
Minor stuff

- define src and dist directories in one place.
```js
var build = {
src: 'assets/',
dist: 'dist/'
};
```
- do not reference `bower_components` in assets.php
- delete all "copy" tasks and consolidate by group
- update all npm deps
- prune unused npm deps
- use wiredep instead of main-bower-files
- use lazypipe to DRY css and script compilation
- use del instead of gulp-clean

### Bugs Fixed

- fixes "clean" being run as a dependency of the build task. This was causing
the build to delete itself midway through the build lol. [shame on][cleanbug]
@retlehs
- fixes versioning being run as a dep of build. Every gulp dep is run in
parallel. Just because it is last in the array does not mean it is last to
run.
- fixes various build breaking mispellings [shame on][assetspelling] @retlehs
- fixes jquery being included twice. shame on you know who.

## Conclusion

### Tested and working
This is testetd and verified working via this method:

1. Clone and run latest `bedrock-ansible` master
2. Clone latest `bedrock` into `example.dev`
3. Rsync my working roots theme into `exmaple.dev`
4. Verify compiled javascript and css runs correctly

### Outstanding issues

#### Features
- Minifying CSS is not implemented right now because it conflicts with
sourcemaps.

#### gulp-rev has some issues:
- [appending to the manifest doesn't
work](https://github.com/sindresorhus/gulp-rev/pull/69)
[#2](https://github.com/sindresorhus/gulp-rev/pull/59)
- [gulp-sourcemaps doesn't
work](https://github.com/sindresorhus/gulp-rev/issues/66)

Once these are fixed we can stop skipping rev-manifest.json checking on dev

#### Testing
- Testing needed: Jshint
- Testing needed: SCSS compilation
- Testing needed: images task
- Testing needed: complex JS setup
- Testing needed: vendored plugin javascript
- Testing needed: livereload
- Testing needed: watch functionality

#### Documentation
- Document manifest.json spec

#### Performance
- Probably implement caching for images/fonts

[cleanBug]: https://github.com/roots/roots/commit/c3379fabd8cb0d89386b6fc8175aed4b013dbb8a
[assetSpelling]: https://github.com/roots/roots/commit/c866f4532d16672e938be1d09835006c48bc12d3#diff-b9e12334e9eafd8341a6107dd98510c9R80
[mangle]: https://github.com/mishoo/UglifyJS2#mangler-options

## Footnotes
Original discussion: https://github.com/roots/roots/pull/1138#issuecomment-62593715

closes https://github.com/roots/roots/pull/1213
closes https://github.com/roots/roots/pull/1211
closes https://github.com/roots/roots/pull/1170