Merge pull request #780 from stealjs/minor

Merge minor into master for 1.4.0 release

.jshintrc

@@ -18,5 +18,6 @@
   "node": true,
   "unused": true,
   "undef": true,
-  "evil": true
+  "evil": true,
+  "esversion": 6
 }

doc/guides/optimized_builds.md

@@ -0,0 +1,134 @@
+@page StealJS.guides.optimized_builds Optimized Builds
+@parent StealJS.guides
+
+@body
+
+In 1.4.0 StealTools added a new **optimize** API which aims to replace [stealTools.build](steal-tools.build)
+as the default way of creating a build of a module and all of its dependencies; builds created by `stealTools.optimize` are smaller and load faster.
+
+Unlike regular [builds](steal-tools.build), optimized builds don't need to load or bundle StealJS at all; a thin wrapper is added instead to the main bundle so the browser can load and execute the modules correctly. 
+
+> The **optimize** API is still a work in progress, some StealJS features are still not supported.
+
+In this guide, we'll go through the steps required to create and use an optimized build. We'll be using the
+`myhub` application created in the [Progressive Loading](./StealJS.guides.progressive_loading) guide.
+
+
+## Install Prerequisites
+
+### Window Setup
+
+1. Install [NodeJS](https://nodejs.org/).
+2. Install Chocolatey, Python, Windows SDK, and Visual Studio as described [here](http://stealjs.com/docs/guides.ContributingWindows.html).
+
+### Linux / Mac Setup
+
+1. Install [NodeJS](https://nodejs.org/).
+
+## Setting up myhub
+
+### Clone the Github repo 
+
+Run the following command:
+
+```
+> git clone git@github.com:stealjs/myhub.git
+```
+
+### Install dependencies
+
+As mentioned before, the **optimize** API is still in its early days, for that reason 
+we need to use the pre-release packages of `steal-tools` and `steal-css`.
+
+Edit your `package.json` like:
+
+```json
+"devDependencies": {
+  ...
+  "steal-css": "^1.3.0-pre.0",
+  "steal-tools": "^1.4.0-pre.1"
+}
+```
+
+Run `npm install` to install all the application dependencies.
+
+### Update dynamic module identifiers
+
+One limitation of the optimized loader is that unlike StealJS's loader it does not normalize
+module identifiers on runtime. For static imports that's not a problem, but it's an issue for 
+dynamic imports (through `steal.import`), a workaround for that is to use the full module
+name.
+
+Edit the dynamic import in `myhub.js` to:
+
+```js
+steal.import(`myhub@1.0.0#${hash}/${hash}`).then(function(moduleOrPlugin) {
+```
+
+where "myhub" is the package name, the number after the "@" symbol is the package version and
+the rest of the string after the "#" is the actual module identifier.
+
+### Make an optimize build script
+
+Currently, there is no CLI option to use the `stealTools.optimize` function, so a NodeJS script is required.
+
+Create _optimize.js_:
+
+```js
+var stealTools = require("steal-tools");
+
+// use the defaults
+stealTools.optimize();
+```
+
+Run the build script with:
+
+```
+> node optimize.js
+```
+
+Now, start an http server by running `npm start` and open `http://127.0.0.1:8080/`
+in a web browser. You should see myhub's home page.
+
+### Performance comparison
+
+For most application builds where StealJS is not included in each of the main bundles, optimized builds will
+have one fewer http request on the initial load which is already a win.
+
+In this example, we are comparing how fast the load event is fired (this event is fired when a resource and its dependent resources have finished loading) when creating the build using `stealTools.build` (with `bundleSteal` set to `true`) and `stealTools.optimize`.
+
+![build](https://user-images.githubusercontent.com/724877/27665945-ab37799a-5c2d-11e7-8d20-08f3de19ee5f.png)
+
+In the screenshot above, the build created by [stealTools.build](steal-tools.build) takes 157ms to 
+fire the load event, in contrast the optimized build takes 119ms (24% faster) to fire the load event, see the screenshot below.
+
+![google chrome](https://user-images.githubusercontent.com/724877/27653129-5d2fb35c-5bfb-11e7-85fb-fa48f2a79e1b.png)
+
+It's also worth noting that the optimized bundles are smaller; the gzip size of the optimized main bundle is 31.8 kB compared to the 59.5 kB bundle of the regular build (46% smaller!).
+
+### Progressive loading and async script tags
+
+So far we have only loaded the main module of the `myhub` application; one of the features of this application is that it progressively loads the modules needed to render the `weather` page and the `puppies` page only when the user navigates to any of these pages.
+
+![weather](https://user-images.githubusercontent.com/724877/27666155-16d23f86-5c2f-11e7-997f-c117416b8196.png)
+
+Once again, the optimized bundle is smaller than the regular one (although the difference is less that 1 kB); but unlike regular bundles loaded with StealJS, the optimized loader can handle bundles loaded using script tags; which means we can take advantage of the browser capability to download the bundle and parse its code without blocking the main thread and without waiting for the user to click any of the navigation links.
+
+Edit `index.html` to asynchronously load the bundles of the other two pages like this:
+
+```html
+<body>
+  <div class="container">Hello World.</div>
+  <script src="./dist/bundles/myhub/myhub.js"></script>
+  <script async src="./dist/bundles/myhub/weather/weather.js"></script>
+  <script async src="./dist/bundles/myhub/puppies/puppies.js"></script>
+</body>
+```
+
+![optimize-async](https://user-images.githubusercontent.com/724877/27666330-2e7c18d6-5c30-11e7-95b7-5a9323b51833.png)
+
+If you reload your browser, you'd notice that there are two extra requests and the load event takes more or less the same time to fire as before. The browser downloaded and parsed the bundles without blocking the main thread, instead of waiting for the user to navigate away from the home page, which results in smoother transitions.
+
+Navigate to the puppies page and see it for yourself!
+
+![async](https://user-images.githubusercontent.com/724877/27697431-75927fc2-5cb1-11e7-9642-1a496a5c8cd3.png)

doc/optimize.md

@@ -0,0 +1,100 @@
+@function steal-tools.optimize optimize
+@parent steal-tools.JS 
+
+Build a module and all of its dependencies and, optionally, other bundles to progressively load.
+
+@signature `stealTools.optimize(config, options)`
+
+@param {steal-tools.StealConfig} config 
+Specifies configuration values to set on Steal's loader.
+
+@param {steal-tools.BuildOptions} [options]
+Specifies the behavior of the build.
+
+@return {(Promise<steal-tools.BuildResult>)} Promise that resolves when the build is complete.
+
+@body
+
+## Use
+
+The following uses steal-tool's `optimize` method to programatically build out the "my-app"
+module as bundles.
+
+    var path = require("path");
+    var stealTools = require("steal-tools");
+
+    var promise = stealTools.optimize({
+      config: path.join(__dirname, "package.json!npm")
+    }, {
+      // the following are the default values, so you don't need
+      // to write them.
+      minify: true,
+      debug: true
+    });
+
+This will build bundles like:
+
+    /dist/bundles/
+      my-app.js
+      my-app.css
+
+To load the bundles, a html page should have a script tag like:
+
+```
+<script src="./dist/my-app.js"></script>
+```
+
+## splitLoader
+
+Setting the `splitLoader` option to `true` creates a bundle that only includes the code of the optimized loader shim.
+
+    var path = require("path");
+    var stealTools = require("steal-tools");
+
+    var promise = stealTools.build({
+      main: "my-app",
+      config: path.join(__dirname, "package.json!npm")
+    }, {
+      splitLoader: true
+    });
+
+This will build bundles like:
+
+    /dist/bundles/
+	  loader.js
+      my-app.js
+      my-app.css
+
+To load the bundles, a html page should have a script tag that loads the loader and one that loads the application code:
+
+```
+<script src="./dist/bundles/loader.js" async></script>
+<script src="./dist/bundles/my-app.js" async></script>
+```
+
+The optimized loader can handle async loading, which means order is not relevant when adding the script tags.
+
+## dest
+
+The `dest` option specifies **a folder** where the distributables (which includes your bundles and possibly other assets) are written.
+
+    var path = require("path");
+    var stealTools = require("steal-tools");
+
+    var promise = stealTools.optimize({
+      config: path.join(__dirname, "package.json!npm")
+    }, {
+      dest: path.join(__dirname, "mobile", "assets"),
+    });
+
+This will build bundles like:
+
+    /mobile/assets/bundles
+      my-app.js
+      my-app.css
+
+To load the bundles, a html page should have a script tag like:
+
+```
+<script src="../mobile/assets/bundles/my-app.js"></script>
+```

doc/types/build-options.md

@@ -22,7 +22,7 @@ stealtools.build(config, {
 A function can be provided to handle minification of each file in a bundle, e.g:
 
 ```javascript
-stealtools.build(config, {
+stealTools.build(config, {
 	minify: function(source, options) {
 		// use your own library to minify source.code 
 		source.code = doCustomMinification(source.code);
@@ -89,3 +89,38 @@ stealTools.build(config, {
 @option {steal-tools.BuildOptions.transpile} [transpile] A function that handles the transpiling of ES6 source to a format for production.
 
 @option {Boolean} [watch=false] Actives watch mode which will continuously build as you develop your application.
+
+@option {Boolean|String} [bundleManifest=false] Generates an HTTP2-push like manifest of the application bundles for server-side preload.
+
+When set to `true` a `bundles.json` file will be written at the top level of the folder where the built assets are located (see the property`dest` above). 
+
+A full path to the desired JSON filename can be provided, too. e.g:
+
+```js
+stealTools.build(config, { 
+  bundleManifest: path.join(__dirname, "dist", "my-manifest.json")
+});
+```
+
+The manifest has the following shape:
+
+```json
+{
+  "[ENTRY-POINT-BUNDLE-NAME]": {
+    "[SHARED-BUNDLE-NAME]": {
+      "type": "style",
+      "weight": 1
+    },
+    "[SHARED-BUNDLE-NAME]": {
+      "type": "script",
+      "weight": 2
+    }
+  }
+}
+```
+
+The top-level objects correspond to _entry level bundles_; these are the bundles needed to start up individual "pages" of the application and  progressively loaded bundles.
+
+The nested objects are _shared bundles_, these are created by `steal-tools` to minimize the number of HTTP requests needed to load the application code. 
+
+Each _shared bundle_ has two properties, `type`, an string that indicates whether the bundle contains script code or styles and `weight`, a number indicating loading priority; a bundle with lower weight should be loaded before other bundles, e.g: style bundles have `weight` of `1` and should be loaded before script bundles.

index.js

@@ -7,6 +7,7 @@ module.exports = {
 	bundle: bundle,
 	build: multiBuild,
 	transform: transform,
+	optimize: require("./lib/build/slim"),
 	graph: {
 		each: require("./lib/graph/each_dependencies"),
 		map: require("./lib/graph/map_dependencies"),

lib/build/multi.js

@@ -68,6 +68,7 @@ var continuousBuild = require("./continuous");
 var concat = require("../bundle/concat_stream");
 var minify = require("../stream/minify");
 var transpile = require("../stream/transpile");
+var writeBundleManifest = require("../stream/write_bundle_manifest");
 
 
 module.exports = function(config, options){
@@ -99,6 +100,7 @@ module.exports = function(config, options){
 				concat(),
 				createWriteStream(),
 				stealWriteStream(),
+				writeBundleManifest(),
 				function(err) {
 					// reject the promise if any of the streams fail
 					if (err) reject(err);

lib/build/slim.js

@@ -0,0 +1,58 @@
+var pump = require("pump");
+var assign = require("lodash/assign");
+var isUndefined = require("lodash/isUndefined");
+var assignDefaultOptions = require("../assign_default_options");
+
+var streams = {
+	bundle: require("../stream/bundle"),
+	minify: require("../stream/minify"),
+	slimBundles: require("../stream/slim"),
+	transpile: require("../stream/transpile"),
+	concat: require("../bundle/concat_stream"),
+	addModuleIds: require("../stream/add_module_ids"),
+	addBundleIds: require("../stream/add_bundle_ids"),
+	filterGraph: require("../stream/filter_slim_graph"),
+	addPluginNames: require("../stream/add_plugin_names"),
+	checkSlimSupport: require("../stream/check_slim_support"),
+	write: require("../bundle/write_bundles").createWriteStream,
+	writeBundlesManifest: require("../stream/write_bundle_manifest"),
+	loadOptimizedPlugins: require("../stream/load_optimized_plugins"),
+	graph: require("../graph/make_graph_with_bundles").createBundleGraphStream
+};
+
+module.exports = function(config, options) {
+	var buildOptions = assign({}, options);
+
+	// minification is on by default
+	assign(buildOptions, {
+		minify: isUndefined(buildOptions.minify) ? true : buildOptions.minify
+	});
+
+	try {
+		options = assignDefaultOptions(config, buildOptions);
+	} catch (err) {
+		return Promise.reject(err);
+	}
+
+	return new Promise(function(resolve, reject) {
+		var writeSteam = pump(
+			streams.graph(config, buildOptions),
+			streams.filterGraph(),
+			streams.checkSlimSupport(),
+			streams.addModuleIds(),
+			streams.loadOptimizedPlugins(),
+			streams.transpile({ outputFormat: "slim" }),
+			streams.bundle(),
+			streams.addPluginNames(),
+			streams.addBundleIds(),
+			streams.slimBundles(),
+			streams.concat(),
+			streams.minify(),
+			streams.write(),
+			streams.writeBundlesManifest(),
+			reject
+		);
+
+		writeSteam.on("data", resolve);
+	});
+};

lib/bundle/is_js_bundle.js

@@ -0,0 +1,8 @@
+/**
+ * Whether the bundle contains only JavaScript nodes
+ * @param {Object} bundle - The bundle
+ * @return {boolean}
+ */
+module.exports = function isJavaScriptBundle(bundle) {
+	return bundle.buildType == null || bundle.buildType === "js";
+};