ShellJS doesn't respect NPM Registry being set outside of it

[Aghassi]

Node version (or tell us if you're using electron or some other framework): 8.1.4

ShellJS version (the most recent version/Github branch you see the bug on): 0.7.8

Operating system: macOS 10.12.6

Description of the bug:

In a CI environment (CircleCI), my company is required to set both the registry and the https-proxy of the .npmrc file. These settings are not carried into the environment shelljs operates in. I know this because I start getting timeouts when trying to run yarn or npm install when running via shell.exec

Example ShellJS command to reproduce the error:

I can't provide exact registries or proxies since they are internal, so I will provide the "scenario" or "conditions" under which this would happen.

 npm config set registry https://registry.npmjs.company.net
 npm config set https-proxy http://special.proxy.here
 yarn test

And yarn test would run some script that had

shell.cd(project);  // This is a project we cloned into the existing project
shell.exec(`yarn`);

In this case, ShellJS's environment will not have the registry variables we set. In addition, running -g for the registry variables does not fix the issue. At this point, the only solution is to have shelljs set the registry again in the script. This defeats the purpose of setting it outside of the script.

[nfischer]

@Aghassi thanks for reporting. Could you try writing a bash script and seeing if that works as expected?

---

After that, could you try replacing your JS script with:

// this is run by `yarn test`
process.chdir(project);
child_process.execSync('yarn');

Let me know if that works as expected.

[Aghassi]

@nfischer Alright so I have run a bash script (or just a line of bash in the YAML file) and it will run yarn just fine.

I tried what you asked, still getting errors since it can't ready the registry or our https-proxy set by npm

Error: Command failed: yarn
warning There appears to be trouble with your network connection. Retrying...
warning There appears to be trouble with your network connection. Retrying...
warning There appears to be trouble with your network connection. Retrying...
warning There appears to be trouble with your network connection. Retrying...
error An unexpected error occurred: "https://registry.yarnpkg.com/enzyme: tunneling socket could not be established, statusCode=403".

And the script below

const pluginList = require('./plugins_list.json');
const childProcess = require('child_process');

/**
 * Runs shell scripts for testing the plugins
 * Performs npm install and plugin build
 * @param  {[type]} pluginList   JSON array of plugins
 */
const testScript = function testScript(list) {
  list.forEach((plugin) => {
    // this is run by `yarn test`
    process.chdir(`plugins/${plugin.name}`);
    childProcess.execSync('yarn');
  });
};

testScript(pluginList);

Question, since this is running in a Node environment, is it possible to grab the registry and https-proxy settings from the environment and pass them to ShellJS via a check before ShellJS runs?

[nfischer]

To clarify, you still see the wrong behavior if you use child_process.execSync() instead of shell.exec()?

---

I don't know much about setting alternate registries. I assume npm reads from a global config file to figure out where the registry is (in which case, I don't see yet why we have a bug). Please correct me if I'm misunderstanding.

[freitagbr]

Does yarn respect npm configs? You could also try running /usr/bin/env yarn test to make sure the environment variables are set.

[Aghassi]

@nfischer Yes I see the same behavior

---

NPM reads from from .npmrc. You can read more on it here https://docs.npmjs.com/files/npmrc#files

There is a pecking order. I think it is Local > User > Global.

---

@freitagbr Yes yarn does respect the config. But, I will also try that at the top of the NPM file to see if that works.

[nfischer]

@nfischer Yes I see the same behavior

Ok, so it sounds like the issue may be with node's core lib, not our module specifically. child_process.execSync('npm ...') should invoke whatever npm is in your $PATH.

I'm out of ideas as to what is causing this, but I don't think there's any change required for shelljs (since this appears in child_process too) 😕

[Aghassi]

@nfischer Does shelljs support passing environment variables into it? For example, docker and sudo have a -e flag that pass the environment variables into the command they are invoking.

[nfischer]

Simplest way:

process.env.FOO = 'bar';
shell.exec('echo $FOO'); // shows "bar\n"

[Aghassi]

@nfischer Just got back from a trip, let me give this a try and report back.

[nfischer]

@Aghassi is it safe to close this now?

[Aghassi]

@nfischer Yeah, I never had a chance to sit down and test it out, got pulled into other things. If I run into the problem again, I will report back here. Appreciate the follow up :).

[narehart]

Just in case anyone from google comes across this like I did, the problem is most likely environment variables in process.env overriding your .npmrc file. I was specifically having issues with the registry setting and was able to resolve like so:

const { npm_config_registry, ...env } = process.env;
execSync("yarn install some-package", { env }); 

[lazarofl]

Just in case anyone from google comes across this like I did, the problem is most likely environment variables in process.env overriding your .npmrc file. I was specifically having issues with the registry setting and was able to resolve like so:

const { npm_config_registry, ...env } = process.env;
execSync("yarn install some-package", { env }); 

@narehart, that's a solution for a similar issue I was facing 🎉
I was getting errors when running npm whoami from child_process.execSync.
NPM team could change this order of precedence when loading the registry to avoid issues like this.