Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bundle analysis handle rate limit error #427

Closed
wants to merge 5 commits into from
Closed

Conversation

JerrySentry
Copy link
Contributor

@JerrySentry JerrySentry commented May 1, 2024

2 of 2 PRs of the fix.

When encountering a rate limit error by GCS when doing save file to the BA bucket, instead of failing the processor step completely, it will now retry.

GCS has a 1 PUT request per second rate limit when updating an existing object in storage, the BA processor may encounter this edge case when multiple BA processing tasks are running in parallel to update multiple bundle stats files. And when this is encountered, we don't update the SQLite file for this stats file therefore causing incorrect data. This fix will retry the task after a short 20s cooldown to once again attempt to upload, at this time most likely the rate limit threshold would be passed. Note that there can still be cases when multiple retried tasks run at the same time again, but the probability of this would be really low considering the existing issue of rate limiting error from GCS is also seen very rarely.

Legal Boilerplate

Look, I get it. The entity doing business as "Sentry" was incorporated in the State of Delaware in 2015 as Functional Software, Inc. In 2022 this entity acquired Codecov and as result Sentry is going to need some rights from me in order to utilize my contributions in this PR. So here's the deal: I retain all rights, title and interest in and to my contributions, and by keeping this boilerplate intact I confirm that Sentry can use, modify, copy, and redistribute my contributions, under Sentry's choice of terms.

@codecov-qa
Copy link

codecov-qa bot commented May 1, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 97.35%. Comparing base (07653e1) to head (2cb4d29).

✅ All tests successful. No failed tests found.

Additional details and impacted files

Impacted file tree graph

@@           Coverage Diff           @@
##             main     #427   +/-   ##
=======================================
  Coverage   97.34%   97.35%           
=======================================
  Files         401      401           
  Lines       33638    33665   +27     
=======================================
+ Hits        32746    32773   +27     
  Misses        892      892           
Flag Coverage Δ
integration 97.35% <100.00%> (+<0.01%) ⬆️
latest-uploader-overall 97.35% <100.00%> (+<0.01%) ⬆️
unit 97.35% <100.00%> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Components Coverage Δ
NonTestCode 94.59% <100.00%> (+<0.01%) ⬆️
OutsideTasks 97.50% <100.00%> (+<0.01%) ⬆️
Files Coverage Δ
services/bundle_analysis.py 97.05% <100.00%> (+0.05%) ⬆️
.../tests/unit/test_bundle_analysis_processor_task.py 100.00% <100.00%> (ø)

Copy link

codecov-public-qa bot commented May 1, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 97.35%. Comparing base (07653e1) to head (2cb4d29).

✅ All tests successful. No failed tests found ☺️

Impacted file tree graph

@@           Coverage Diff           @@
##             main     #427   +/-   ##
=======================================
  Coverage   97.34%   97.35%           
=======================================
  Files         401      401           
  Lines       33638    33665   +27     
=======================================
+ Hits        32746    32773   +27     
  Misses        892      892           
Flag Coverage Δ
integration 97.35% <100.00%> (+<0.01%) ⬆️
latest-uploader-overall 97.35% <100.00%> (+<0.01%) ⬆️
unit 97.35% <100.00%> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Components Coverage Δ
NonTestCode 94.59% <100.00%> (+<0.01%) ⬆️
OutsideTasks 97.50% <100.00%> (+<0.01%) ⬆️
Files Coverage Δ
services/bundle_analysis.py 97.05% <100.00%> (+0.05%) ⬆️
.../tests/unit/test_bundle_analysis_processor_task.py 100.00% <100.00%> (ø)

Copy link

codecov bot commented May 1, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 97.37%. Comparing base (07653e1) to head (2cb4d29).

✅ All tests successful. No failed tests found.

Additional details and impacted files

Impacted file tree graph

@@           Coverage Diff           @@
##             main     #427   +/-   ##
=======================================
  Coverage   97.37%   97.37%           
=======================================
  Files         432      432           
  Lines       34328    34355   +27     
=======================================
+ Hits        33427    33454   +27     
  Misses        901      901           
Flag Coverage Δ
integration 97.35% <100.00%> (+<0.01%) ⬆️
latest-uploader-overall 97.35% <100.00%> (+<0.01%) ⬆️
unit 97.35% <100.00%> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Components Coverage Δ
NonTestCode 94.61% <100.00%> (+<0.01%) ⬆️
OutsideTasks 97.50% <100.00%> (+<0.01%) ⬆️
Files Coverage Δ
services/bundle_analysis.py 97.05% <100.00%> (+0.05%) ⬆️
.../tests/unit/test_bundle_analysis_processor_task.py 100.00% <100.00%> (ø)

This change has been scanned for critical changes. Learn more

@JerrySentry JerrySentry marked this pull request as ready for review May 1, 2024 19:16
requirements.in Outdated Show resolved Hide resolved
@codecov-notifications
Copy link

codecov-notifications bot commented May 16, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

✅ All tests successful. No failed tests found.

Impacted file tree graph

@@           Coverage Diff           @@
##             main     #427   +/-   ##
=======================================
  Coverage   97.34%   97.35%           
=======================================
  Files         401      401           
  Lines       33638    33665   +27     
=======================================
+ Hits        32746    32773   +27     
  Misses        892      892           
Flag Coverage Δ
integration 97.35% <100.00%> (+<0.01%) ⬆️
latest-uploader-overall 97.35% <100.00%> (+<0.01%) ⬆️
unit 97.35% <100.00%> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Components Coverage Δ
NonTestCode 94.59% <100.00%> (+<0.01%) ⬆️
OutsideTasks 97.50% <100.00%> (+<0.01%) ⬆️
Files Coverage Δ
services/bundle_analysis.py 97.05% <100.00%> (+0.05%) ⬆️
.../tests/unit/test_bundle_analysis_processor_task.py 100.00% <100.00%> (ø)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants