Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merging with nyc storybook test coverage with other reports generated by coverageProvider 'istanbul' result in wrong coverage #1559

Open
meanstackmax opened this issue Apr 30, 2024 · 1 comment

Comments

@meanstackmax
Copy link

Link to bug demonstration repository https://github.com/meanstackmax/demo-test-coverage/tree/main

Expected Behavior

Vitest and storybook should use the same source map outputted, thus the issue with different columns for the same stmt not affecting merged report

Observed Behavior

I am trying to generate a merged code report. Therefore, I am using for my unit tests vitest with istanbul. For Storybook I am using the addon which also uses istanbul. To merge the reports I am using nyc.

The issue I am having is that the reports of vitest and storybook are showing me different columns for the same stmt. This leads to not correct merging.

For example in one component vitest said that there is a stmt on line 38 start at col 18 and ends on the same line.

            "2": {
                "start": {
                    "line": 38,
                    "column": 18
                },
                "end": {
                    "line": 38,
                    "column": null
                }
            },

However, storybook coverage report says the statement is on line 38 in column 2:

      "3": {
        "start": {
          "line": 38,
          "column": 2
        },
        "end": {
          "line": 38,
          "column": null
        }
      },

Troubleshooting steps

-Tried to add this to package.json, but it didn't work:

"overrides": {
"istanbul-lib-instrument": "6.0.1",
"istanbul-lib-coverage": "3.2.0"
},
  • Tried to upgrade packages to "@storybook/addon-coverage" to v 1.0.1 and "@storybook/test-runner" to v 0.17.0 but it doesn't resolve the issue
  • Tried manually enabling source map for storybook and vitest and adding nyc configuration (installing "@istanbuljs/nyc-config-typescript" and adding .nycrc file for more control), but it didn't work as well
  • Tried with storybook v8 and storybook the dependancies updated but it doesn't solve the issue as well
  • Tried using istanbul-merge instead of nyc for merging the reports, but in the end it basically ends up with the same output as using nyc

Environment Information

  System:
    OS: macOS 14.4.1
    CPU: (8) arm64 Apple M1
    Memory: 122.00 MB / 16.00 GB
  Binaries:
    Node: 18.16.1 - ~/.nvm/versions/node/v18.16.1/bin/node
    Yarn: 1.22.19 - ~/.yarn/bin/yarn
    npm: 9.5.1 - ~/.nvm/versions/node/v18.16.1/bin/npm
  npmPackages:
    @babel/core: ^7.23.7 => 7.24.5 
    @vitest/coverage-istanbul: ^1.2.1 => 1.5.3 
    babel-loader: ^9.1.3 => 9.1.3 
    nyc: ^15.1.0 => 15.1.0 
    typescript: ^5.3.3 => 5.4.5 
@cenfun
Copy link

cenfun commented May 8, 2024

There are 3 different versions of istanbul-lib-instrument in use after running npm list istanbul-lib-instrument, it should be the cause of the problem.

coverage-demo-app@1.0.0 /demo-test-coverage
├─┬ @storybook/addon-coverage@1.0.1
│ ├─┬ @jsdevtools/coverage-istanbul-loader@3.0.5
│ │ └── istanbul-lib-instrument@4.0.3
│ ├── istanbul-lib-instrument@6.0.2
│ └─┬ vite-plugin-istanbul@3.0.4
│   └── istanbul-lib-instrument@5.2.1
├─┬ @storybook/addon-essentials@7.6.18
│ └─┬ @storybook/addon-docs@7.6.18
│   └─┬ @jest/transform@29.7.0
│     └─┬ babel-plugin-istanbul@6.1.1
│       └── istanbul-lib-instrument@5.2.1
├─┬ @vitest/coverage-istanbul@1.5.3
│ └── istanbul-lib-instrument@6.0.2 deduped
├─┬ jest@29.7.0
│ └─┬ @jest/core@29.7.0
│   └─┬ @jest/reporters@29.7.0
│     └── istanbul-lib-instrument@6.0.2 deduped
└─┬ nyc@15.1.0
  └── istanbul-lib-instrument@4.0.3

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants