Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

esbuild-wasm support for JS API #86

Closed
kzc opened this issue May 6, 2020 · 13 comments
Closed

esbuild-wasm support for JS API #86

kzc opened this issue May 6, 2020 · 13 comments

Comments

@kzc
Copy link
Contributor

kzc commented May 6, 2020

It would be useful to have the JS API work with esbuild-wasm without the need for a binary esbuild package installed.

Manually copying the lib directory from an esbuild install to esbuild-wasm and adding "main": "lib/main.js" to its package.json appears to work. This untested patch attempts to automate that task:

diff --git a/Makefile b/Makefile
index 2be6c06..25bfa6c 100644
--- a/Makefile
+++ b/Makefile
@@ -43,6 +43,7 @@ platform-wasm:
        cd src/esbuild && GOOS=js GOARCH=wasm go build -o ../../npm/esbuild-wasm/esbuild.wasm ./main
        cd npm/esbuild-wasm && npm version "$(ESBUILD_VERSION)" --allow-same-version
        cp "$(shell go env GOROOT)/misc/wasm/wasm_exec.js" npm/esbuild-wasm/wasm_exec.js
+       rm -rf npm/esbuild-wasm/lib && cp -rp npm/esbuild/lib npm/esbuild-wasm/lib
 
 platform-neutral:
        cd npm/esbuild && npm version "$(ESBUILD_VERSION)" --allow-same-version
@@ -71,6 +72,7 @@ clean:
        rm -rf npm/esbuild-darwin-64/bin
        rm -rf npm/esbuild-linux-64/bin
        rm -f npm/esbuild-wasm/esbuild.wasm npm/esbuild-wasm/wasm_exec.js
+       rm -rf npm/esbuild-wasm/lib
        cd src/esbuild && go clean -testcache ./...
 
 node_modules:
diff --git a/npm/esbuild-wasm/package.json b/npm/esbuild-wasm/package.json
index a8ba4dc..c8c5016 100644
--- a/npm/esbuild-wasm/package.json
+++ b/npm/esbuild-wasm/package.json
@@ -7,6 +7,8 @@
   "engines": {
     "node": ">=8"
   },
+  "main": "lib/main.js",
+  "types": "lib/main.d.ts",
   "directories": {
     "bin": "bin"
   }
@evanw evanw closed this as completed in 2e7dfaf May 6, 2020
@kzc
Copy link
Contributor Author

kzc commented May 6, 2020

@evanw Here's another one to test against:

https://github.com/mischnic/tree-shaking-example

Submit a PR to demonstrate how esbuild bundle sizes fare against the other bundlers.

@kzc
Copy link
Contributor Author

kzc commented May 6, 2020

I hacked esbuild support into a local copy of the tree shaking repo above.

Here are the results using esbuild 0.2.3 with the options
--bundle --minify '--define:process.env.NODE_ENV="production"':

file                 size       gzip     
-------------------  ---------  ---------
esbuild/lodash-es    91.79 KB   33.74 KB 
parcel/lodash-es     19.16 KB   7.45 KB  
rollup/lodash-es     18.22 KB   6.56 KB  
webpack/lodash-es    20.74 KB   7.33 KB  
                                         
esbuild/lodash       70.13 KB   25.74 KB 
parcel/lodash        91.12 KB   30.43 KB 
rollup/lodash        69 KB      24.13 KB 
webpack/lodash       70.57 KB   24.73 KB 
                                         
esbuild/rxjs         169.49 KB  34.03 KB 
parcel/rxjs          9.46 KB    2.9 KB   
rollup/rxjs          9.08 KB    2.74 KB  
webpack/rxjs         10.82 KB   3.51 KB  
                                         
esbuild/react-icons  1.24 MB    405.22 KB
parcel/react-icons   1.19 MB    400.12 KB
rollup/react-icons   9.18 KB    3.76 KB  
webpack/react-icons  10.42 KB   4.2 KB   
                                         
esbuild/remeda       18.2 KB    4.45 KB  
parcel/remeda        1.96 KB    914 B    
rollup/remeda        1.89 KB    878 B    
webpack/remeda       2.8 KB     1.23 KB  
                                         
esbuild/ramda        63.67 KB   16.49 KB 
parcel/ramda         6.35 KB    1.94 KB  
rollup/ramda         6.25 KB    1.86 KB  
webpack/ramda        7.16 KB    2.16 KB  
                                         
esbuild/ramdaBabel   7.77 KB    2.53 KB  
parcel/ramdaBabel    6.72 KB    2.15 KB  
rollup/ramdaBabel    6.35 KB    1.9 KB   
webpack/ramdaBabel   8.39 KB    2.32 KB  
                                         
esbuild/rambda       11.91 KB   3.69 KB  
parcel/rambda        8.85 KB    2.41 KB  
rollup/rambda        591 B      319 B    
webpack/rambda       2.19 KB    972 B    
                                         
esbuild/rambdax      32.38 KB   10.62 KB 
parcel/rambdax       21.72 KB   6.76 KB  
rollup/rambdax       4.63 KB    1.92 KB  
webpack/rambdax      8.3 KB     3.21 KB  

It appears that esbuild hasn't implemented dead code elimination yet. Anyway, it's a good test case for the project when it does.

@kzc
Copy link
Contributor Author

kzc commented May 8, 2020

@evanw In addition to dead code elimination support, esbuild would have to implement the following optimizations to achieve the smaller bundle sizes produced by other bundlers:

  1. uglify-js style /*@__PURE__*/ annotation support for side-effect-free function calls. Here's a useful PR thread discussing its implementation in Rollup: Add support for /*#__PURE__*/ comments. rollup/rollup#2429 (comment)

  2. webpack style package.json "sideEffects" hint for side-effect-free modules: https://webpack.js.org/guides/tree-shaking/#mark-the-file-as-side-effect-free

@kzc
Copy link
Contributor Author

kzc commented May 26, 2020

tree shaking example results with esbuild 0.4.0:

file                 size      gzip     
-------------------  --------  ---------
esbuild/lodash-es    91.19 KB  33.39 KB 
parcel/lodash-es     19.16 KB  7.45 KB  
rollup/lodash-es     18.22 KB  6.56 KB  
webpack/lodash-es    20.74 KB  7.33 KB  
                                        
esbuild/lodash       70.2 KB   25.74 KB 
parcel/lodash        91.12 KB  30.43 KB 
rollup/lodash        69 KB     24.13 KB 
webpack/lodash       70.57 KB  24.73 KB 
                                        
esbuild/rxjs         86.14 KB  16.93 KB 
parcel/rxjs          9.46 KB   2.9 KB   
rollup/rxjs          9.08 KB   2.74 KB  
webpack/rxjs         10.82 KB  3.51 KB  
                                        
esbuild/react-icons  1.19 MB   400.16 KB
parcel/react-icons   1.19 MB   400.12 KB
rollup/react-icons   9.18 KB   3.76 KB  
webpack/react-icons  10.42 KB  4.2 KB   
                                        
esbuild/remeda       7.45 KB   1.96 KB  
parcel/remeda        1.96 KB   914 B    
rollup/remeda        1.89 KB   878 B    
webpack/remeda       2.8 KB    1.23 KB  
                                        
esbuild/ramda        44.55 KB  11.58 KB 
parcel/ramda         6.35 KB   1.94 KB  
rollup/ramda         6.25 KB   1.86 KB  
webpack/ramda        7.16 KB   2.16 KB  
                                        
esbuild/ramdaBabel   7.71 KB   2.29 KB  
parcel/ramdaBabel    6.72 KB   2.15 KB  
rollup/ramdaBabel    6.35 KB   1.9 KB   
webpack/ramdaBabel   8.39 KB   2.32 KB  
                                        
esbuild/rambda       1.38 KB   647 B    
parcel/rambda        8.85 KB   2.41 KB  
rollup/rambda        591 B     319 B    
webpack/rambda       2.19 KB   972 B    
                                        
esbuild/rambdax      5.87 KB   2.44 KB  
parcel/rambdax       21.72 KB  6.76 KB  
rollup/rambdax       4.63 KB   1.92 KB  
webpack/rambdax      8.3 KB    3.21 KB  

@evanw
Copy link
Owner

evanw commented May 27, 2020

I'm not sure how much I trust https://github.com/mischnic/tree-shaking-example.

I just started looking through the results and I realized that at least the react-icons example is broken. A Rollup bug causes it to not find React.createElement (there are warnings during the build) and it replaces it with undefined, which would cause the generated code to crash if used.

It's definitely a useful benchmark, and is much appreciated. But I wish it had some test that the generated code is actually correct.

@evanw
Copy link
Owner

evanw commented May 27, 2020

Just ran the benchmark on version 0.3.9 vs. 0.4.2:

file                           size
-------------------------  --------
esbuild@0.3.9/lodash-es     289.3kb
esbuild@0.4.2/lodash-es     285.4kb
webpack/lodash-es            20.5kb
parcel/lodash-es             18.6kb
rollup/lodash-es             18.2kb

esbuild@0.3.9/lodash        225.3kb
esbuild@0.4.2/lodash        215.3kb
parcel/lodash                91.6kb
webpack/lodash               72.1kb
rollup/lodash                69.1kb

esbuild@0.3.9/rxjs          305.4kb
esbuild@0.4.2/rxjs          209.2kb
webpack/rxjs                 10.9kb
parcel/rxjs                   9.6kb
rollup/rxjs                   9.2kb

esbuild@0.3.9/react-icons  1567.6kb
esbuild@0.4.2/react-icons  1471.5kb
webpack/react-icons          12.6kb
parcel/react-icons            9.4kb
rollup/react-icons            9.3kb

esbuild@0.3.9/remeda         36.3kb
esbuild@0.4.2/remeda         19.9kb
webpack/remeda                2.8kb
parcel/remeda                 2.0kb
rollup/remeda                 1.9kb

esbuild@0.3.9/ramda         125.7kb
esbuild@0.4.2/ramda         106.6kb
webpack/ramda                 7.2kb
parcel/ramda                  6.4kb
rollup/ramda                  6.2kb

esbuild@0.3.9/ramdaBabel     20.6kb
esbuild@0.4.2/ramdaBabel     19.2kb
webpack/ramdaBabel            8.4kb
parcel/ramdaBabel             6.8kb
rollup/ramdaBabel             6.3kb

esbuild@0.3.9/rambda         30.7kb
parcel/rambda                 9.6kb
esbuild@0.4.2/rambda          4.2kb
webpack/rambda                2.4kb
rollup/rambda                 0.7kb

esbuild@0.3.9/rambdax        91.7kb
parcel/rambdax               22.4kb
esbuild@0.4.2/rambdax        16.2kb
webpack/rambdax               8.4kb
rollup/rambdax                6.1kb

The tree shaking in v0.4 is definitely an improvement, especially in the rambda test case.

@kzc
Copy link
Contributor Author

kzc commented May 27, 2020

That's odd. I did not experience a crash.

Granted react-icons is a poorly written test, but it still produces a result when run:

$ node webpack/react-icons.js
[Function: f] { displayName: 'FaBeer' }

$ node rollup/react-icons.js
[Function: Z] { displayName: 'FaBeer' }

$ node esbuild/react-icons.js
[Function: H] { displayName: 'FaBeer' }

If you apply this patch to the react-icons test it prints out a proper name:

--- a/src/react-icons.js
+++ b/src/react-icons.js
@@ -1,3 +1,3 @@
 import { FaBeer } from 'react-icons/fa';
 
-console.log(FaBeer);
\ No newline at end of file
+console.log(FaBeer.displayName);

results with patch:

$ node webpack/react-icons.js
FaBeer

$ node rollup/react-icons.js
FaBeer

$ node esbuild/react-icons.js
FaBeer

sizes of that test with patch:

file                 size      gzip     
-------------------  --------  ---------
esbuild/react-icons  1.19 MB   400.16 KB
parcel/react-icons   1.19 MB   400.12 KB
rollup/react-icons   9.2 KB    3.76 KB  
webpack/react-icons  10.43 KB  4.2 KB   

Update the react-icons test if you like, but the other tests are fine.

@evanw
Copy link
Owner

evanw commented May 27, 2020

The test doesn't evaluate the bundled code. If you change it to evaluate the code, the Rollup build will crash:

diff --git a/src/react-icons.js b/src/react-icons.js
index dce9578..1da9aee 100644
--- a/src/react-icons.js
+++ b/src/react-icons.js
@@ -1,3 +1,3 @@
 import { FaBeer } from 'react-icons/fa';

-console.log(FaBeer);
\ No newline at end of file
+console.log(FaBeer());
\ No newline at end of file

Results with patch:

$ node esbuild/react-icons.js
{
  '$$typeof': Symbol(react.element),
  type: [Function: IconBase],
  key: null,
  ref: null,
  props: { attr: { viewBox: '0 0 448 512' }, children: [ [Object] ] },
  _owner: null
}
$ node rollup/react-icons.js
rollup/react-icons.js:1

TypeError: (void 0) is not a function
    at rollup/react-icons.js:1:8276
    at Array.map (<anonymous>)
    at e (rollup/react-icons.js:1:8244)
    at rollup/react-icons.js:1:8315
    at FaBeer (rollup/react-icons.js:1:9443)
    at Object.<anonymous> (rollup/react-icons.js:1:9483)
    at Module._compile (internal/modules/cjs/loader.js:1156:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1176:10)
    at Module.load (internal/modules/cjs/loader.js:1000:32)
    at Function.Module._load (internal/modules/cjs/loader.js:899:14)

@kzc
Copy link
Contributor Author

kzc commented May 27, 2020

Well, the original test as written did not need to run that function. :-)

To build that test correctly after your modifications Rollup requires some manual configuration to work with the CJS format react module:

--- a/rollup.config.js
+++ b/rollup.config.js
@@ -2,6 +2,8 @@ import resolve from "rollup-plugin-node-resolve";
 import commonjs from "rollup-plugin-commonjs";
 import babel from "rollup-plugin-babel";
 import { terser } from "rollup-plugin-terser";
+import react from 'react';
+import reactDom from 'react-dom';
 
 const libName = process.env.LIB;
 
@@ -14,14 +16,19 @@ export default [
                        babel({
                                exclude: "node_modules/**"
                        }),
-                       commonjs(),
+                       commonjs({
+                               include: 'node_modules/**',
+                               namedExports: {
+                                       react: Object.keys(react),
+                                       'react-dom': Object.keys(reactDom)
+                               }
+                       }),
                        terser({
                                compress: {
                                        global_defs: {
                                                "process.env.NODE_ENV": "production"
                                        }
                                },
-                               sourcemap: false,
                                toplevel: true
                        })
                ],

I don't know whether react-dom was needed. I just threw it in there just in case.

Edit: The rollup config above is incorrect. Use this one instead.

After upgrading every bundler and plugin to their latest versions on npm, you can see that Rollup produces a valid result:

$ node rollup/react-icons.js
{
  '$$typeof': Symbol(react.element),
  type: [Function: Z],
  key: null,
  ref: null,
  props: { attr: { viewBox: '0 0 448 512' }, children: [ [Object] ] },
  _owner: null
}

The output of the newly revised react-icons test is not deterministic across bundlers though. I recommend changing it to the following:

--- a/src/react-icons.js
+++ b/src/react-icons.js
@@ -1,3 +1,3 @@
 import { FaBeer } from 'react-icons/fa';
 
-console.log(FaBeer);
\ No newline at end of file
+console.log(FaBeer().props.attr.viewBox);

so that all bundlers produce consistent verifiable results:

$ for i in esbuild/*.js ; do node $i ; done | shasum
fd67c68c7ec18629d1da3d528b37f65bdbe99d22  -
$ for i in parcel/*.js ; do node $i ; done | shasum
fd67c68c7ec18629d1da3d528b37f65bdbe99d22  -
$ for i in rollup/*.js ; do node $i ; done | shasum
fd67c68c7ec18629d1da3d528b37f65bdbe99d22  -
$ for i in webpack/*.js ; do node $i ; done | shasum
fd67c68c7ec18629d1da3d528b37f65bdbe99d22  -

With those changes here's the sizes for the latest versions of bundlers:

file                 size      gzip     
-------------------  --------  ---------
esbuild/lodash-es    91.36 KB  33.44 KB 
parcel/lodash-es     18.6 KB   7.29 KB  
rollup/lodash-es     17.91 KB  6.33 KB  
webpack/lodash-es    20.5 KB   7.09 KB  
                                        
esbuild/lodash       70.3 KB   25.78 KB 
parcel/lodash        91.55 KB  30.57 KB 
rollup/lodash        70.57 KB  24.16 KB 
webpack/lodash       71.88 KB  24.67 KB 
                                        
esbuild/rxjs         86.69 KB  17.02 KB 
parcel/rxjs          9.57 KB   2.92 KB  
rollup/rxjs          10.02 KB  3.22 KB  
webpack/rxjs         10.22 KB  3.13 KB  
                                        
esbuild/react-icons  1.21 MB   410.41 KB
parcel/react-icons   9.38 KB   3.96 KB  
rollup/react-icons   9.74 KB   3.89 KB  
webpack/react-icons  9.97 KB   3.93 KB  
                                        
esbuild/remeda       7.45 KB   1.96 KB  
parcel/remeda        1.96 KB   917 B    
rollup/remeda        1.93 KB   896 B    
webpack/remeda       2.82 KB   1.25 KB  
                                        
esbuild/ramda        44.55 KB  11.58 KB 
parcel/ramda         6.35 KB   1.96 KB  
rollup/ramda         6.3 KB    1.87 KB  
webpack/ramda        7.22 KB   2.17 KB  
                                        
esbuild/ramdaBabel   7.71 KB   2.29 KB  
parcel/ramdaBabel    6.75 KB   2.16 KB  
rollup/ramdaBabel    174 B     137 B    EDIT: this bundle is incorrect - fixed below
webpack/ramdaBabel   8.45 KB   2.33 KB  
                                        
esbuild/rambda       1.69 KB   777 B    
parcel/rambda        9.58 KB   2.64 KB  
rollup/rambda        761 B     388 B    
webpack/rambda       2.41 KB   1.04 KB  
                                        
esbuild/rambdax      7.62 KB   2.86 KB  
parcel/rambdax       22.37 KB  6.86 KB  
rollup/rambdax       6.14 KB   2.28 KB  
webpack/rambdax      8.37 KB   3.05 KB  

@kzc
Copy link
Contributor Author

kzc commented May 28, 2020

I took a look at the two smallest rollup results in my last post above.

rollup/rambda.js is okay, but rollup/ramdaBabel.js still has embedded require()s:

$ terser rollup/ramdaBabel.js -b
"use strict";

var r, e = require("ramda/src/range"), a = require("ramda/src/compose"), c = require("ramda/src/filter");

function s(r) {
    return r % 2 == 0;
}

console.log((r = 10, a(c(s), e(2))(r)));

The problem seems to be related to upgrading to rollup-plugin-babel@4.4.0.

The other rollup results as well as the other bundlers' results are bundled correctly.

@kzc
Copy link
Contributor Author

kzc commented May 28, 2020

The problem with rollup/ramdaBabel.js was the rollup config file, not rollup-plugin-babel@4.4.0. This one works correctly:

$ cat rollup.config.js 
import resolve from "rollup-plugin-node-resolve";
import commonjs from "rollup-plugin-commonjs";
import babel from "rollup-plugin-babel";
import { terser } from "rollup-plugin-terser";
import react from 'react';

const libName = process.env.LIB;

export default [
	{
		input: `src/${libName}.js`,
		treeshake: true,
		plugins: [
			resolve(),
			babel({
				exclude: "node_modules/**"
			}),
			commonjs({
				namedExports: {
					react: Object.keys(react),
				}
			}),
			terser({
				compress: {
					global_defs: {
						"process.env.NODE_ENV": "production"
					}
				},
				toplevel: true
			})
		],
		output: [{ file: `rollup/${libName}.js`, format: "cjs" }]
	}
];

Corrected results for rollup/ramdaBabel.js:

file                 size      gzip     
-------------------  --------  ---------
esbuild/ramdaBabel   7.71 KB   2.29 KB  
parcel/ramdaBabel    6.75 KB   2.16 KB  
rollup/ramdaBabel    6.4 KB    1.91 KB  
webpack/ramdaBabel   8.45 KB   2.33 KB  

The other rollup results are the same.

@kzc
Copy link
Contributor Author

kzc commented May 28, 2020

The react-icons test would be better if it showed both the displayName and the SVG data for the particular icon:

$ cat src/react-icons.js 
import { FaBeer } from 'react-icons/fa';
console.log(FaBeer.displayName);
console.log(FaBeer().props.children[0].props.d);
$ esbuild src/react-icons.js --bundle | node
FaBeer
M368 96h-48V56c0-13.255-10.745-24-24-24H24C10.745 32 0 42.745 0 56v400c0 13.255 10.745 24 24 24h272c13.255 0 24-10.745 24-24v-42.11l80.606-35.977C429.396 365.063 448 336.388 448 304.86V176c0-44.112-35.888-80-80-80zm16 208.86a16.018 16.018 0 0 1-9.479 14.611L320 343.805V160h48c8.822 0 16 7.178 16 16v128.86zM208 384c-8.836 0-16-7.164-16-16V144c0-8.836 7.164-16 16-16s16 7.164 16 16v224c0 8.836-7.164 16-16 16zm-96 0c-8.836 0-16-7.164-16-16V144c0-8.836 7.164-16 16-16s16 7.164 16 16v224c0 8.836-7.164 16-16 16z

@evanw
Copy link
Owner

evanw commented Jun 5, 2020

Let's continue this thread in #50.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants