An array is unexpectedly regarded as a valid ESLint severity

[shinnn]

What version of ESLint are you using?

v2.3.0

What configuration and parser (Espree, Babel-ESLint, etc.) are you using?

Default parser

What did you do? Please include the actual source code causing the issue.

This is an extremely rare case, but a user may carelessly fill an ESLint severity value with an array of the string like below:

const CLIEngine = require('eslint').CLIEngine;

new CLIEngine({
  useEslintrc: false,
  rules: {
    'comma-dangle': [['error'], 'never']
  }
});

What did you expect to happen?

new CLIEngine(...) should throw an error because ['error'] is, of course, not a valid severity according to the doc.

What actually happened? Please include the actual, raw output from ESLint.

new CLIEngine(...) doesn't throw any errors, because,

1. /^(?:off|warn|error)$/i.test(severity) always stringify non-string severity before testing.
2. String(['error']) returns 'error'. Unfortunately, that also matches /^(?:off|warn|error)$/i.

We should do stricter type checking for severity to fix this unexpected behavior. I can create a pull request for this issue.

[ilyavolodin]

Looks like an oversight to me.

[BYK]

👍 for being more strict and throwing an error for this case.

[nzakas]

I suspect this code path isn't being run through the validator.

[shinnn]

I've just started to implement strict severity checking.

[shinnn]

Submitted a PR. #5527