Runner middleware does not handle file-list rejections

[nicojs]

Expected behaviour

When using karma with autoWatch: false and singleRun: false, you can use karma run to initiate runs. However, when a preprocessor rejects with an error, I would expect the run to complete with an error.

Actual behaviour

Karma server closes with an 11 11 2019 07:29:10.133:ERROR [karma-server]: UnhandledRejection (at least: on node 10 and up)

This is because there is no error handling in the runner middleware:
https://github.com/karma-runner/karma/blob/master/lib/middleware/runner.js#L88-L94

Environment Details

• Karma version (output of karma --version): 4.4.1
• Relevant part of your karma.config.js file: n/a

Steps to reproduce the behaviour

1. mkdir tmp && cd tmp
2. npm init --yes
3. npm i karma karma-jasmine karma-chrome-launcher
4. Create the following files:

   // karma.conf.js
   module.exports = (config) => config.set({
       browsers: ["Chrome"],
       files: ["./index.js"],
       frameworks: ['jasmine'],
       plugins: ['karma-*', require.resolve('./preprocessor')],
       preprocessors: {
           './index.js': ['always-error']
       },
       singleRun: false,
       watch: false
   });
   
   // index.js
   describe('test', () => {
       it('works', () => {});
   });
   
   // always-error-preprocessor.js
   let i = 0;
   function createPreprocessor() {
       return function (content, file, done) {
           console.log(`Processing ${file}`);
           i++;
           if (i < 2) {
               return done(undefined, content)
           } else {
               return done(new Error('Expected error!'));
           }
       }
   }
   module.exports = {
       'preprocessor:always-error': ['factory', createPreprocessor],
   }

5. Open 2 console windows. In the first window, use karma start. Whenever it is finished with the test, run karma run in the second window.

Output of the first window:

$ npx  karma start
Processing C:/z/github/tmp/index.js
11 11 2019 07:41:22.100:WARN [karma]: No captured browser, open http://localhost:9876/
11 11 2019 07:41:22.139:INFO [karma-server]: Karma v4.4.1 server started at http://0.0.0.0:9876/
11 11 2019 07:41:22.141:INFO [launcher]: Launching browsers Chrome with concurrency unlimited
11 11 2019 07:41:22.151:INFO [launcher]: Starting browser Chrome
11 11 2019 07:41:23.468:INFO [Chrome 78.0.3904 (Windows 10.0.0)]: Connected on socket HszEjx5TwDi8drcfAAAA with id 94172194
Chrome 78.0.3904 (Windows 10.0.0): Executed 1 of 1 SUCCESS (0.149 secs / 0.001 secs)
TOTAL: 1 SUCCESS
Processing C:/z/github/tmp/index.js
11 11 2019 07:41:31.194:ERROR [karma-server]: UnhandledRejection

Output of the second window:

$ npx karma run
C:\z\github\tmp\node_modules\karma\lib\runner.js:70
      throw e
      ^

Error: read ECONNRESET
    at TCP.onStreamRead (internal/stream_base_commons.js:111:27)

[nicojs]

I'm implementing a fix for this. It will report the error as run result with an error message (which will include the preprocessor error text).

I think that this is what a user would expect, in line with Karma's design. If anyone disagrees, please let me know. Thanks.

[johnjbarton]

+1 Unhandledrejections should always be an error.

[karmarunnerbot]

🎉 This issue has been resolved in version 5.0.0 🎉

The release is available on:

• npm package (@latest dist-tag)
• GitHub release

Your semantic-release bot 📦🚀