Chore: improve performance of SourceCode constructor

[not-an-aardvark]

What is the purpose of this pull request? (put an "X" next to item)

[x] Other, please explain:

What changes did you make? (Give an overview)

When generating a list of tokens and comments, the SourceCode constructor previously concatenated the list of tokens and the list of comments, and sorted the result. For a file with n tokens and n comments, this takes O(nlog(n)) time. However, since the token and comment lists are already sorted, they can be merged in O(n) time.

The performance impact of this change is most noticeable on large files, since they have a large number of tokens. When linting test-data.js (48000 lines) with no rules enabled, this improved overall performance by about 15-20%.

Is there anything you'd like reviewers to focus on?

Nothing in particular

[mention-bot]

@not-an-aardvark, thanks for your PR! By analyzing the history of the files in this pull request, we identified @nzakas, @mysticatea and @gyandeeps to be potential reviewers.

[eslintbot]

LGTM

[nzakas]

Can you post the results of running npm run perf before and after this change?

[not-an-aardvark]

There doesn't seem to be any notable difference with the npm run perf results. I'm not sure why this is -- I assume it's because the files used in npm run perf are relatively small.

npm run perf on master

(master)$ npm run perf

> eslint@3.15.0 perf /Users/tkatz/code/eslint
> node Makefile.js perf


Loading:
  Load performance Run #1:  178.68395ms
  Load performance Run #2:  150.129757ms
  Load performance Run #3:  157.350916ms
  Load performance Run #4:  150.812348ms
  Load performance Run #5:  145.804506ms

  Load Performance median:  150.812348ms


Single File:
  CPU Speed is 3100 with multiplier 13000000
  Performance Run #1:  3232.736464ms
  Performance Run #2:  3332.973437ms
  Performance Run #3:  3293.97345ms
  Performance Run #4:  3246.857269ms
  Performance Run #5:  3325.301646ms

  Performance budget ok:  3293.97345ms (limit: 4193.548387096775ms)


Multi Files (0 files):
  CPU Speed is 3100 with multiplier 39000000
  Performance Run #1:  9832.974868ms
  Performance Run #2:  9926.993779ms
  Performance Run #3:  9747.603105ms
  Performance Run #4:  10107.852042ms
  Performance Run #5:  9879.122170999999ms

  Performance budget ok:  9879.122170999999ms (limit: 12580.645161290322ms)

npm run perf on this branch

(token-and-comments-performance)$ npm run perf

> eslint@3.15.0 perf /Users/tkatz/code/eslint
> node Makefile.js perf


Loading:
  Load performance Run #1:  200.267784ms
  Load performance Run #2:  150.728479ms
  Load performance Run #3:  156.692002ms
  Load performance Run #4:  152.282385ms
  Load performance Run #5:  157.135008ms

  Load Performance median:  156.692002ms


Single File:
  CPU Speed is 3100 with multiplier 13000000
  Performance Run #1:  3567.9580260000002ms
  Performance Run #2:  3358.167306ms
  Performance Run #3:  3299.1686879999997ms
  Performance Run #4:  3257.171602ms
  Performance Run #5:  3246.727172ms

  Performance budget ok:  3299.1686879999997ms (limit: 4193.548387096775ms)


Multi Files (0 files):
  CPU Speed is 3100 with multiplier 39000000
  Performance Run #1:  10210.827903ms
  Performance Run #2:  9867.823343ms
  Performance Run #3:  9755.449152ms
  Performance Run #4:  9994.298772ms
  Performance Run #5:  9723.938451ms

  Performance budget ok:  9867.823343ms (limit: 12580.645161290322ms)

---

On the other hand, there is a significant difference when linting a very large file.

Linting test-data.js on master

(master)$ time bin/eslint.js --no-eslintrc test-data.js

real	0m1.854s
user	0m1.834s
sys	0m0.166s

Linting test-data.js on this branch

(token-and-comments-performance)$ time bin/eslint.js --no-eslintrc test-data.js

real	0m1.605s
user	0m1.596s
sys	0m0.155s