Yarn breaks in docker, but not outside

[natew]

What is the current behavior?
In simple docker container with new install of yarn, running lerna bootstrap with yarn is suddenly no longer working.

If the current behavior is a bug, please provide the steps to reproduce.
Hard to create steps due to complexity here, but error may be helpful.

What is the expected behavior?
It was working fine and installing earlier.

Please mention your node.js, yarn and operating system version.
node 7, yarn 0.23.2, xhyve linux

lerna ERR! execute warning Waiting for the other yarn instance to finish
lerna ERR! execute warning No license field
lerna ERR! execute error An unexpected error occurred: "https://registry.yarnpkg.com/esprima-fb/-/esprima-fb-3001.0001.0000-dev-harmony-fb.tgz: ENOENT: no such file or directory, utime '/usr/local/share/.cache/yarn/v1/npm-esprima-fb-3001.1.0-dev-harmony-fb-b77d37abcd38ea0b77426bb8bc2922ce6b426411/test/esprima.js.html'".
lerna ERR! execute yarn install v0.23.2
lerna ERR! execute [1/4] Resolving packages...
lerna ERR! execute [2/4] Fetching packages...
lerna ERR! execute info If you think this is a bug, please open a bug report with the information provided in "/repo/apps/api/yarn-error.log".
lerna ERR! execute info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.
lerna ERR! execute
lerna ERR! execute     at Promise.all.then.arr (/repo/node_modules/execa/index.js:210:11)
lerna ERR! execute     at process._tickCallback (internal/process/next_tick.js:109:7)
lerna ERR! execute  callback with error
lerna ERR! execute  { Error: Command failed: yarn install --mutex network:42424
lerna ERR! execute warning Waiting for the other yarn instance to finish
lerna ERR! execute warning No license field
lerna ERR! execute error An unexpected error occurred: "https://registry.yarnpkg.com/esprima-fb/-/esprima-fb-3001.0001.0000-dev-harmony-fb.tgz: ENOENT: no such file or directory, utime '/usr/local/share/.cache/yarn/v1/npm-esprima-fb-3001.1.0-dev-harmony-fb-b77d37abcd38ea0b77426bb8bc2922ce6b426411/test/esprima.js.html'".
lerna ERR! execute yarn install v0.23.2
lerna ERR! execute [1/4] Resolving packages...
lerna ERR! execute [2/4] Fetching packages...
lerna ERR! execute info If you think this is a bug, please open a bug report with the information provided in "/repo/apps/api/yarn-error.log".
lerna ERR! execute info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.
lerna ERR! execute
lerna ERR! execute     at Promise.all.then.arr (/repo/node_modules/execa/index.js:210:11)
lerna ERR! execute     at process._tickCallback (internal/process/next_tick.js:109:7)
lerna ERR! execute   code: 1,
lerna ERR! execute   killed: false,
lerna ERR! execute   stdout: 'yarn install v0.23.2\n[1/4] Resolving packages...\n[2/4] Fetching packages...\ninfo If you think this is a bug, please open a bug report with the information provided in "/repo/apps/api/yarn-error.log".\ninfo Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.\n',
lerna ERR! execute   stderr: 'warning Waiting for the other yarn instance to finish\nwarning No license field\nerror An unexpected error occurred: "https://registry.yarnpkg.com/esprima-fb/-/esprima-fb-3001.0001.0000-dev-harmony-fb.tgz: ENOENT: no such file or directory, utime \'/usr/local/share/.cache/yarn/v1/npm-esprima-fb-3001.1.0-dev-harmony-fb-b77d37abcd38ea0b77426bb8bc2922ce6b426411/test/esprima.js.html\'".\n',
lerna ERR! execute   failed: true,
lerna ERR! execute   signal: null,
lerna ERR! execute   cmd: 'yarn install --mutex network:42424',
lerna ERR! execute   timedOut: false }
lerna WARN complete Waiting for 1 child process to exit. CTRL-C to exit immediately.
(node:16) UnhandledPromiseRejectionWarning: Unhandled promise rejection (rejection id: 3): Error: Command failed: yarn install --mutex network:42424
warning Waiting for the other yarn instance to finish
warning No license field
error An unexpected error occurred: "https://registry.yarnpkg.com/esprima-fb/-/esprima-fb-3001.0001.0000-dev-harmony-fb.tgz: ENOENT: no such file or directory, utime '/usr/local/share/.cache/yarn/v1/npm-esprima-fb-3001.1.0-dev-harmony-fb-b77d37abcd38ea0b77426bb8bc2922ce6b426411/test/esprima.js.html'".
yarn install v0.23.2
[1/4] Resolving packages...
[2/4] Fetching packages...
info If you think this is a bug, please open a bug report with the information provided in "/repo/apps/api/yarn-error.log".
info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.

(node:16) DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.```

[ChristopheVandePoel]

I see the same thing on WSL (Ubuntu 16.04, node v7.10.0, yarn v0.23,4, lerna v2.0.0-rc.4)

Setting the concurrency to 1 helped for now.

No problems when I run it straight in windows, nor when the lerna bootstrap is scoped to a single package.

[ChristopheVandePoel]

To be complete, I think this is probably an issue with the latest lerna-update, rather than with yarn itself:

lerna/lerna#789
lerna/lerna#793

I'll try reverting to rc.3 lerna next

[natew]

Ah just found this before seeing!

lerna/lerna#671 (comment)

[evocateur]

I wonder what the yarn-error.log says. Nothing about the error is specific to lerna, unless a Docker container doesn't support network ports above 40,000 for some odd reason?

[ChristopheVandePoel]

I tried with lower network ports (<10000) and it gave me the same result. I'll look into providing that log when I have the time.

[ChristopheVandePoel]

That yarn-error.log doesn't provide much more information other than:

Arguments:
  /home/stoffel/.nvm/versions/node/v7.8.0/bin/node /home/stoffel/.nvm/versions/node/v7.8.0/bin/yarn install --mutex network:42424

...

https://registry.yarnpkg.com/babel-code-frame/-/babel-code-frame-6.22.0.tgz: ENOENT: no such file or directory, open '/home/stoffel/.cache/yarn/v1/npm-babel-code-frame-6.22.0-027620bee567a88c32561574e7fd0801d33118e4/.yarn-tarball.tgz'

I've created a test-repo with which I can replicate the behavior:
https://github.com/ChristopheVandePoel/lerna-test

If you have a windows-machine lying around (I know, right?) with WSL installed on it, maybe try running it with

yarn cache clean
lerna clean
lerna bootstrap --npm-client=yarn --loglevel silly

The repo simply tries to install lodash & babel-core in the first and only babel-core in the second. No hoisting. That should be a pretty basic setup, right?

It fails with:

0 silly input [ 'bootstrap' ]
1 silly flags { _: [ 'bootstrap' ],
1 silly flags   sort: true,
1 silly flags   loglevel: 'silly',
1 silly flags   concurrency: 4 }
2 verbose rootPath /mnt/c/Users/Stoffel/Documents/Projects/lerna-test
3 info version 2.0.0-rc.4
4 silly isInitialized
5 silly isInitialized true
6 silly existsSync /mnt/c/Users/Stoffel/Documents/Projects/lerna-test/VERSION
7 silly initialize attempt
8 silly initialize success
9 silly execute attempt
10 info Bootstrapping 2 packages
11 info lifecycle preinstall
12 silly runScript preinstall package1
13 silly package1
14 silly runScript preinstall package2
15 silly package2
16 silly hasDependencyInstalled package1 babel-core
17 silly dependencyIsSatisfied [ 'babel-core',
17 silly dependencyIsSatisfied   '^6.24.1',
17 silly dependencyIsSatisfied   '/mnt/c/Users/Stoffel/Documents/Projects/lerna-test/packages/package1/node_modules' ]
18 verbose dependencyIsSatisfied babel-core false
19 silly hasDependencyInstalled package2 babel-core
20 silly dependencyIsSatisfied [ 'babel-core',
20 silly dependencyIsSatisfied   '^6.24.1',
20 silly dependencyIsSatisfied   '/mnt/c/Users/Stoffel/Documents/Projects/lerna-test/packages/package2/node_modules' ]
21 verbose dependencyIsSatisfied babel-core false
22 silly hasDependencyInstalled package1 lodash
23 silly dependencyIsSatisfied [ 'lodash',
23 silly dependencyIsSatisfied   '^4.17.4',
23 silly dependencyIsSatisfied   '/mnt/c/Users/Stoffel/Documents/Projects/lerna-test/packages/package1/node_modules' ]
24 verbose dependencyIsSatisfied lodash false
25 silly root dependencies []
26 silly leaf dependencies {
26 silly leaf dependencies   "package1": [
26 silly leaf dependencies     {
26 silly leaf dependencies       "dependency": "babel-core@^6.24.1",
26 silly leaf dependencies       "isSatisfied": false
26 silly leaf dependencies     },
26 silly leaf dependencies     {
26 silly leaf dependencies       "dependency": "lodash@^4.17.4",
26 silly leaf dependencies       "isSatisfied": false
26 silly leaf dependencies     }
26 silly leaf dependencies   ],
26 silly leaf dependencies   "package2": [
26 silly leaf dependencies     {
26 silly leaf dependencies       "dependency": "babel-core@^6.24.1",
26 silly leaf dependencies       "isSatisfied": false
26 silly leaf dependencies     }
26 silly leaf dependencies   ]
26 silly leaf dependencies }
27 info Installing external dependencies
28 verbose actions 2 actions, concurrency 4
29 silly installInDir package1 [ 'babel-core@^6.24.1', 'lodash@^4.17.4' ]
30 silly installInDir backup /mnt/c/Users/Stoffel/Documents/Projects/lerna-test/packages/package1/package.json
31 silly rename [ '/mnt/c/Users/Stoffel/Documents/Projects/lerna-test/packages/package1/package.json',
31 silly rename   '/mnt/c/Users/Stoffel/Documents/Projects/lerna-test/packages/package1/package.json.lerna_backup' ]
32 silly installInDir package2 [ 'babel-core@^6.24.1' ]
33 silly installInDir backup /mnt/c/Users/Stoffel/Documents/Projects/lerna-test/packages/package2/package.json
34 silly rename [ '/mnt/c/Users/Stoffel/Documents/Projects/lerna-test/packages/package2/package.json',
34 silly rename   '/mnt/c/Users/Stoffel/Documents/Projects/lerna-test/packages/package2/package.json.lerna_backup' ]
35 silly installInDir writing tempJson { dependencies: { 'babel-core': '^6.24.1', lodash: '^4.17.4' } }
36 silly installInDir writing tempJson { dependencies: { 'babel-core': '^6.24.1' } }
37 silly getExecOpts { cwd: '/mnt/c/Users/Stoffel/Documents/Projects/lerna-test/packages/package2' }
38 silly installInDir [ 'yarn', [ 'install', '--mutex', 'network:42424' ] ]
39 silly getExecOpts { cwd: '/mnt/c/Users/Stoffel/Documents/Projects/lerna-test/packages/package1' }
40 silly installInDir [ 'yarn', [ 'install', '--mutex', 'network:42424' ] ]
41 silly installInDir cleanup /mnt/c/Users/Stoffel/Documents/Projects/lerna-test/packages/package1/package.json
42 silly renameSync [ '/mnt/c/Users/Stoffel/Documents/Projects/lerna-test/packages/package1/package.json.lerna_backup',
42 silly renameSync   '/mnt/c/Users/Stoffel/Documents/Projects/lerna-test/packages/package1/package.json' ]
43 verbose installed leaf package1
44 error execute Error: Command failed: yarn install --mutex network:42424
44 error execute warning No license field
44 error execute error Bad hash. Expected "8c428564dce1e1f41fb337ec34f4c3b022b5ad83" but got "da39a3ee5e6b4b0d3255bfef95601890afd80709"
44 error execute yarn install v0.23.4
44 error execute [1/4] Resolving packages...
44 error execute [2/4] Fetching packages...
44 error execute info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.
44 error execute
44 error execute     at Promise.all.then.arr (/home/stoffel/.nvm/versions/node/v7.8.0/lib/node_modules/lerna/node_modules/execa/index.js:210:11)
44 error execute     at process._tickCallback (internal/process/next_tick.js:109:7)
44 error execute  callback with error
44 error execute  { Error: Command failed: yarn install --mutex network:42424
44 error execute warning No license field
44 error execute error Bad hash. Expected "8c428564dce1e1f41fb337ec34f4c3b022b5ad83" but got "da39a3ee5e6b4b0d3255bfef95601890afd80709"
44 error execute yarn install v0.23.4
44 error execute [1/4] Resolving packages...
44 error execute [2/4] Fetching packages...
44 error execute info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.
44 error execute
44 error execute     at Promise.all.then.arr (/home/stoffel/.nvm/versions/node/v7.8.0/lib/node_modules/lerna/node_modules/execa/index.js:210:11)
44 error execute     at process._tickCallback (internal/process/next_tick.js:109:7)
44 error execute   code: 1,
44 error execute   killed: false,
44 error execute   stdout: 'yarn install v0.23.4\n[1/4] Resolving packages...\n[2/4] Fetching packages...\ninfo Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.\n',
44 error execute   stderr: 'warning No license field\nerror Bad hash. Expected "8c428564dce1e1f41fb337ec34f4c3b022b5ad83" but got "da39a3ee5e6b4b0d3255bfef95601890afd80709" \n',
44 error execute   failed: true,
44 error execute   signal: null,
44 error execute   cmd: 'yarn install --mutex network:42424',
44 error execute   timedOut: false }

Seems an awful lot like it simply ignores the network mutex. Absolutely possible that this is a bug in WSL tho. :)

[kristajg]

Just in case anyone else comes across this thread with similar errors - I had an older version of yarn installed. Uninstalling & reinstalling (or just upgrading the version) of yarn seemed to do the trick.

[nx-alejandrolacasa]

Updating the node version in the dockerfile fixed that for me! The newest the version of node, the newest the version of yarn also.

https://github.com/nodejs/docker-node/blob/001cfa3930ef8fb02865daaecbe35a3822bec15c/8.9/Dockerfile