docs(releasing): add schedule, more explicit integration testing #9695

connorjclark · 2019-09-19T00:51:36Z

https://github.com/GoogleChrome/lighthouse/blob/releasing-docs/docs/releasing.md

exterkamp

Nice docs! 📖

docs/releasing.md

exterkamp · 2019-09-19T18:15:44Z

docs/releasing.md


-# Run the tests again :)
+# Get that new changelog commit + one last test.
 bash ./lighthouse-core/scripts/release/test.sh
 # Package everything for publishing
 bash ./lighthouse-core/scripts/release/prepare-package.sh


Mention making a release in GH to get the x.x.x tag, which is required for prepare-package?

That's not a required step for making a tag, the prepare-package script makes the tag. but I do need to add the whole "add GH Release" thing

fwiw, this should also be a very easy automatable step given a GITHUB_KEY in your env variables.

Co-Authored-By: Shane Exterkamp <shaneexterkamp5@gmail.com>

…nto releasing-docs

brendankenny

as previously discussed, I think this is more documentation of our release tech debt than a release process, but it's good to have it in writing as a starting point for improving :)

brendankenny · 2019-09-23T16:16:49Z

docs/releasing.md

@@ -4,7 +4,23 @@

 ### Cadence

-We ship once a month, on the Thursday before the 1st. While not necessary, followup minor/patch releases may be done if warranted. The planned ship dates are added to the internal Lighthouse calendar.
+We aim to release every 3 weeks. Our schedule is set as follows: One day before the [expected Chromium branch point](https://www.chromium.org/developers/calendar) (which is every six weeks) and again exactly 3 weeks after that day.


I don't feel comfortable writing this down with no established history of being able to pull this off and without a bot doing the releasing, but I guess I'm comfortable with you taking the heat for it, @connorjclark :)

docs/releasing.md

brendankenny · 2019-09-23T16:24:16Z

docs/releasing.md

+git pull
+git new-branch lh-roll-x.x.x
+gclient sync
+autoninja -C out/Release chrome blink_tests


how bad is this without goma?

oh it's unbearable.

woah I've never even heard of autoninja 😮

brendankenny · 2019-09-23T16:27:34Z

docs/releasing.md

+# See: https://www.chromium.org/developers/how-tos/get-the-code
+
+# Roll to Chromium folder.
+yarn devtools


@paulirish wasn't there a way (used to be a way?) to live-edit devtools? Could we do that with Lighthouse so that the releaser isn't dependent on rebuilding?

yahh there is a thing now. --custom-devtools-frontend

https://chromium-review.googlesource.com/c/chromium/src/+/1791150

this might just work for us. but might not because remote modules.

brendankenny · 2019-09-23T16:38:33Z

docs/releasing.md

+# Note: if the changes include proto changes make sure that the API has those new fields.
+# If anything is wrong, stop releasing, investigate, and prioritize landing the PR.
+
+# For bonus points, add some tests covering new features. Either a new test, or an extra


same as for devtools

brendankenny · 2019-09-23T16:39:38Z

docs/releasing.md

+```sh
+# Run the internal `update_lighthouse_assets.sh` script.
+
+# Update that silly string that sets the version number for metrics.


it seems less useful to have these lines here than just to say to test it (the internal doc has step by step directions)

removed... this line is actually uncovered in the internal docs, but I've updated them now.

brendankenny · 2019-09-23T16:40:18Z

docs/releasing.md

+# https://g3doc.corp.google.com/chrome/headless/lightrider/README.md?cl=head#test-lr-changes-in-canary
+
+# Verify that Lightrider works properly, and is generating reports fully. Consider the new features that have been added.
+# Note: if the changes include proto changes make sure that the API has those new fields.


can we automate this?

maybe @exterkamp would be interested in doing this in the future

brendankenny · 2019-09-23T16:43:05Z

docs/releasing.md


-# Run the tests again :)
+# Get that new changelog commit + one last test.


not sure what this is referring to. From an earlier draft?

updated .. the test.sh does two important things here (tests, and updates to latest code)

brendankenny · 2019-09-23T16:45:08Z

docs/releasing.md

+
+# Update that silly string that sets the version number for metrics.
+
+# Test things out locally, if happy, deploy to canary and see how the graphs react.


how long will this typically take?

updated. not long to see if something went really bad, which is all Canary is good for.

brendankenny

still have lots of concerns about the release process, but modulo remaining comments, docs changes LGTM :)

docs/releasing.md

brendankenny · 2019-09-27T21:36:40Z

docs/releasing.md


-# Run the tests again :)
+# One last test (this script uses origin/master, so we also get the commit with the new changelog - that commit should be HEAD).


should this step be run in pristine?

The release scripts cd to pristine. You always run them from a development checkout.

patrickhulce

I stumbled onto this PR again from link in chat and discovered these pending comments. 1 or 2 still seemed useful to have for the record, but 8 months too late or whatever 😆

patrickhulce · 2019-09-23T22:22:13Z

docs/releasing.md

@@ -25,35 +43,104 @@ Release manager follows the below _Release Process_.

 ### Versioning

-We follow [semver](https://semver.org/) versioning semantics (`vMajor.Minor.Patch`), to align with the greater Node community. Generally, this means our bi-weekly releases will bump a minor. Though we will release a new patch version if high-priority fixes are required before the next schedule release. Additionally, if a schedule release contains no new features, then we'll only bump the patch version.
+We follow [semver](https://semver.org/) versioning semantics (`vMajor.Minor.Patch`). Breaking changes will bump the major version. New features or bug fixes will bump the minor version. If a release contains no new features, then we'll only bump the patch version.


~~worth clarifying what constitutes a "breaking change" here?~~

probably enough meat on that question for a separate PR but wanted to leave the thought in here all the same :)

patrickhulce · 2019-09-23T22:28:47Z

docs/releasing.md


-# Run the tests again :)
+# Get that new changelog commit + one last test.
 bash ./lighthouse-core/scripts/release/test.sh
 # Package everything for publishing
 bash ./lighthouse-core/scripts/release/prepare-package.sh


fwiw, this should also be a very easy automatable step given a GITHUB_KEY in your env variables.

patrickhulce · 2019-09-23T22:29:45Z

docs/releasing.md

 bash ./lighthouse-core/scripts/release/test.sh
 # Package everything for publishing
 bash ./lighthouse-core/scripts/release/prepare-package.sh

 # Make sure you're in the Lighthouse pristine repo we just tested.
 cd ../lighthouse-pristine

-# Publish to NPM
+# Sanity check: last chance to abort.


should/can we add flags to these to get them to fail if somethings not right? i.e. we're looking for a porcelain git status and the master/tagged version commit in git log?

patrickhulce · 2019-09-23T22:30:52Z

docs/releasing.md

+git pull
+git new-branch lh-roll-x.x.x
+gclient sync
+autoninja -C out/Release chrome blink_tests


woah I've never even heard of autoninja 😮

patrickhulce · 2019-09-23T22:31:17Z

docs/releasing.md

+
+### The following Monday
+
+Evaluate LR staging, if all looks good, promote to production!


any link here back to the comms around LR?

releasing

ef2d5e2

connorjclark requested a review from paulirish as a code owner September 19, 2019 00:51

googlebot added the cla: yes label Sep 19, 2019

update

9f0d4a3

vercel bot deployed to staging September 19, 2019 01:00 View deployment

fin

8b42b6b

vercel bot deployed to staging September 19, 2019 01:11 View deployment

silly

f1bae92

vercel bot deployed to staging September 19, 2019 01:15 View deployment

exterkamp reviewed Sep 19, 2019

View reviewed changes

Update docs/releasing.md

32a808b

Co-Authored-By: Shane Exterkamp <shaneexterkamp5@gmail.com>

vercel bot deployed to staging September 19, 2019 18:21 View deployment

connorjclark added 2 commits September 19, 2019 11:33

weaker language

1dbab63

Merge branch 'releasing-docs' of github.com:GoogleChrome/lighthouse i…

2974415

…nto releasing-docs

vercel bot deployed to staging September 19, 2019 18:34 View deployment

brendankenny requested changes Sep 23, 2019

View reviewed changes

connorjclark added 3 commits September 23, 2019 10:11

Merge remote-tracking branch 'origin/master' into releasing-docs

375dc9a

docs

e11b9c9

lr

c54e33d

vercel bot deployed to staging September 23, 2019 17:18 View deployment

rm

dcb7935

vercel bot temporarily deployed to staging September 23, 2019 17:20 Inactive

docs

5ffd3a6

vercel bot deployed to staging September 23, 2019 17:21 View deployment

brendankenny approved these changes Sep 27, 2019

View reviewed changes

pr

81310d7

vercel bot deployed to staging September 27, 2019 23:38 View deployment

connorjclark changed the title ~~docs(releasing): add schedule and more explicit integration testing~~ docs(releasing): add schedule, more explicit integration testing Sep 30, 2019

connorjclark merged commit 17c833e into master Sep 30, 2019

connorjclark deleted the releasing-docs branch September 30, 2019 17:12

paulirish pushed a commit that referenced this pull request Nov 6, 2019

docs(releasing): add schedule, more explicit integration testing (#9695)

5242c32

snyk-bot mentioned this pull request Mar 21, 2020

[Snyk] Upgrade lighthouse from 5.1.0 to 5.6.0 godaddy/lighthouse4u#13

Merged

patrickhulce reviewed Jun 17, 2020

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

docs(releasing): add schedule, more explicit integration testing #9695

docs(releasing): add schedule, more explicit integration testing #9695

connorjclark commented Sep 19, 2019 •

edited

exterkamp left a comment

exterkamp Sep 19, 2019

connorjclark Sep 19, 2019

patrickhulce Sep 23, 2019

brendankenny left a comment

brendankenny Sep 23, 2019

brendankenny Sep 23, 2019

connorjclark Sep 23, 2019

patrickhulce Sep 23, 2019

brendankenny Sep 23, 2019

paulirish Sep 24, 2019

brendankenny Sep 23, 2019

brendankenny Sep 23, 2019

connorjclark Sep 23, 2019

brendankenny Sep 23, 2019

connorjclark Sep 27, 2019

brendankenny Sep 23, 2019

connorjclark Sep 23, 2019

brendankenny Sep 23, 2019

connorjclark Sep 23, 2019 •

edited

brendankenny left a comment

brendankenny Sep 27, 2019

connorjclark Sep 27, 2019

patrickhulce left a comment

patrickhulce Sep 23, 2019

patrickhulce Sep 23, 2019

patrickhulce Sep 23, 2019

patrickhulce Sep 23, 2019

patrickhulce Sep 23, 2019


		# Run the tests again :)
		# Get that new changelog commit + one last test.


		# Update that silly string that sets the version number for metrics.

		# Test things out locally, if happy, deploy to canary and see how the graphs react.


		# Run the tests again :)
		# One last test (this script uses origin/master, so we also get the commit with the new changelog - that commit should be HEAD).


		### The following Monday

		Evaluate LR staging, if all looks good, promote to production!

docs(releasing): add schedule, more explicit integration testing #9695

docs(releasing): add schedule, more explicit integration testing #9695

Conversation

connorjclark commented Sep 19, 2019 • edited

exterkamp left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

brendankenny left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

connorjclark Sep 23, 2019 • edited

Choose a reason for hiding this comment

brendankenny left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

patrickhulce left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

connorjclark commented Sep 19, 2019 •

edited

connorjclark Sep 23, 2019 •

edited