BuildPulse is trying to find flaky tests for us but, given the previous
model of using `rspec-retry`, it would rarely find them. Instead, let's
try to always rerun `brew tests` multiple times, report to BuildPulse
each time (by moving the reporting logic into `brew tests`) and disable
`rspec-retry` when using BuildPulse.
While we're here, let's enable `rspec-retry` locally so we don't have
flaky tests biting maintainers/contributors there.
The `VERBOSE_TESTS` variable was from cask and never gets set (and
is unset by `bin/brew`). Replace it with `HOMEBREW_VERBOSE_TESTS` and
set it by `--verbose` or `--debug`.
While we're here, remove an unneeded `VERBOSE` delete (as it's already
done by `bin/brew`).
- always retry each test at least once (confusingly this means a retry
count of 2 rather than 1)
- always wait at least 1 second between retries
- set a default retry metadata for integration tests rather than
overriding any specified values
- use `example.run` rather than `example.run_with_retry` for integration
tests because, confusingly, this avoids having the retry count be
half what it should be (because the attempts increases by one for
each `run_with_retry` call)
- use 4 retries for integration tests with 2**attempts*retry_wait
(retry wait now being 2) to actually have more retries than before
this commit (but with exponential backoff)
This should generally improve test flakiness in CI but particularly
improve the cleanup test flake we've seen recently.
These need to be manually caught and set otherwise they will not be
retried by `rspec-retry`. This is particularly annoying and a cause of
CI failures when tests timeout but are not retried.
Fixes https://github.com/Homebrew/brew/issues/8979
When GitHub Actions is congested some tests can take longer than this
(particularly as we're running in parallel). Globally double the time
we allow for all tests.
It's not sufficient to do this merely on `brew upgrade` because
`brew install` and `brew reinstall` can also result in formulae being
upgraded.
This requires moving logic from `cmd/upgrade.rb` to `upgrade.rb`. To
save you searching the diff the changes that resulted from doing that:
- Query the installed formulae from class state in `FormulaInstaller`
rather than the (incomplete) list that we passed into it.
- Don't output the "Checking dependents" message. It was there for
systems and configurations where this is slow but for most users
and most installations this will be a (annoying, noisy) no-op.
Fixes https://github.com/Homebrew/brew/issues/7860
