This sets the HOMEBREW_NO_INSTALL_FROM_API environment variable
to prevent the selected tests from using the API. We will need
this as we transition to having the API be enabled by default
when running the tests but it's also nice as a sanity check
with the :needs_utils_curl scope in a few places.
Any test that is not tagged as :needs_network and that makes
a call to an unapproved method in the `Utils::Curl` module
will raise an error unless that method gets mocked somehow.
tests: add exceptions for tests that use curl to download local files
These are acceptable ways to use curl in local, non-network tests.
For those edge cases we allow you to bypass the check with :needs_utils_curl.
I added two new methods to cache both installed and all taps.
All taps includes core taps no matter if they're installed locally
since they're always provided by the API anyway.
This makes it easier to cache `Tap.each` while making the code
easier to reason about. It also will be useful because we'll
be able to avoid the `Tap.select(&:installed?` pattern that has
recently invaded the codebase.
Note: I also stopped clearing all tap instance caches before
tests. Running `Tap.each` would cache existing taps which would
lead to unexpected behavior since the only existing tap before
each test is the core tap. This is the only tap whose directory
is not cleaned up between tests so we just clear it's cache directly.
We also now clear all tap instances after tests as well regardless
of whether the API was used that time.
The core taps exist outside of the normal cache busting cycle
so they need to clear explicitly at the instance level.
Just to be sure we should clear all of them each time.
This essentially reverts part of the change in this PR.
- https://github.com/Homebrew/brew/pull/16746
Now we only include the cachable registry when running tests.
We basically just load it first and add a bunch of methods to
it before loading the rest of the formula files when we require
global.rb. I added a check to make sure this require order is
preserved.
I also made a bunch of methods private, stop excluding classes
that inherit from casks since it's unnecessary and add more docs.
This adds a registry for all modules and classes that
cachable is included in. The registry allows us to
programmatically clear all caches in between tests
so that we don't forget to do that when adding a new
class or refactoring code. The goal here is to reduce
the number of flaky tests in the future.
- These are arbitrary length limits that had a load of disables in code.
- The limits were only increasing over time rather than decreasing.
- Fixing the problematic code to be shorter would take a long time for
questionable gain since the problem has been around so long.
- We're not going to make the really long things be any shorter any time soon.
- The instructions in issue 14685 say, pragmatically, "disable all the rubocop
rules we're never going to realistically fix e.g. Metrics/ClassLength". But
that felt like a slippery slope to more _really_ long modules/classes/blocks,
and the limits are here for a reason.
- Extract update tests into separate tests.
- Make `brew tests --online` run only once in its own job. This job
could be made non-required to fight flakiness.
- Split up the various macOS runs into several steps now that we have
more parallel macOS workers available.
- Cleanup some flaky tests.
- fix `check_for_config_scripts` test, no idea why this wasn't working
- don't autoretry tests if `focus` or `byebug` are enabled
- add a ARM64 Ventura bottle symlink
BuildPulse is trying to find flaky tests for us but, given the previous
model of using `rspec-retry`, it would rarely find them. Instead, let's
try to always rerun `brew tests` multiple times, report to BuildPulse
each time (by moving the reporting logic into `brew tests`) and disable
`rspec-retry` when using BuildPulse.
While we're here, let's enable `rspec-retry` locally so we don't have
flaky tests biting maintainers/contributors there.
