- There's a TODO on the "someday" list [1] to ensure that `head` Git
URLs always specify a branch.
- So I thought I'd automate this worry by adding an audit.
- Since `resource` block URLs tend to be pinned to SHAs, if indeed
they are Git URLs, this audit only applies to `head` URLs.
[1]: https://github.com/orgs/Homebrew/projects/5?pane=issue&itemId=98789749
Formulae, casks, and resources have a `#livecheckable?` method that
indicates whether they contain a `livecheck` block. This is intended
to be read as "has a livecheckable?", not "is livecheckable?" (as
livecheck can find versions for some packages/resources without a
`livecheck` block). Unfortunately, correct understanding of this
method's behavior [outside of documentation] relies on historical
knowledge that few people possess, so this is often confusing to
anyone who hasn't been working on livecheck since 2020.
In the olden days, a "livecheckable" was a Ruby file containing a
`livecheck` block (originally a hash) with a filename that
corresponded to a related formula. The `livecheck` blocks in
livecheckable files were integrated into their respective formulae in
August 2020, so [first-party] livecheckables ceased to exist at that
time. From that point forward, we simply referred to these as
`livecheck` blocks.
With that in mind, this clarifies the situation by replacing
"livecheckable" language. This includes renaming `#livecheckable?` to
`#livecheck_defined?`, replacing usage of "livecheckable" as a noun
with "`livecheck` block", replacing "livecheckable" as a boolean with
"livecheck_defined", and replacing incorrect usage of "livecheckable"
as an adjective with "checkable".
Up to now, we haven't been accounting for `#url` symbol arguments in
`livecheck` blocks that don't reference a checkable URL. This can
either be an invalid symbol (e.g., using the `:stable` formula symbol
in a cask) or a valid symbol where the referenced URL doesn't exist
(e.g., using `:head` when there's no `head` URL). [Almost all of the
valid symbols are required URLs but `head` is optional.]
Over the years, we've had a handful of slips where we've used `:url`
in formulae (when it's only valid in casks) and `:stable` in casks
(when it's only valid in formulae). In this scenario,
`livecheck_url_string` is `nil`, so livecheck falls back to
`#checkable_urls`. In this scenario, `stable` and `url` are the first
checkable URLs for formulae and casks (respectively), so the checks
ended up working as expected merely by chance. This isn't obvious in
the output and even the debug output looks normal. It only becomes
apparent that livecheck isn't working as expected if it iterates
through more than one checkable URL before reaching one that works
(not the case in those instances).
With that in mind, this adds an error when a `#url` symbol is used
but it doesn't correspond to a checkable URL. This will account for
both of the mentioned scenarios (invalid symbols and valid ones
referencing a non-existent URL) and prevent livecheck from quietly
proceeding in an unexpected manner.
`Livecheck#preprocess_url` only contains logic for rewriting Git URLs,
so it makes more sense for this code to be part of the `Git` strategy
instead. Outside of better code organization, this saves us from
having to maintain the list of strategies to skip processing (which
is sometimes forgotten when a new strategy is added) and makes it
easier to do something similar in other strategies as needed.
One thing to note is that `Livecheck#preprocess_url` was previously
called on the URL before each strategy's `#match?` method was called.
To maintain the existing behavior, this calls `Git#preprocess_url` in
`Git#match?`. However, we need the processed URL when we use the `Git`
strategy, so we have to call `Git#preprocess_url` again. To avoid
duplicating effort, I've added a `@processed_urls` hash to the `Git`
strategy and have set up `Git#preprocess_url` to cache processed
URLs, so we only do the work once. There may be a better way of
handling it but this seems to work as expected.
The `check_for_resources` code arguably speaks for itself, so it
doesn't feel like a necessary addition. Besides that, this cleans up
some other comments and text to better align with existing examples.
At the moment, `#use_homebrew_curl?` can only be true for a
`homepage` or `stable`/cask `url` with `using: :homebrew_curl`. If
the checked URL differs from these URLs, livecheck won't use brewed
curl. This limitation prevents livecheck from using brewed curl for a
`livecheck` block URL that's a string literal (not a symbol for a
`#checkable_url` like `:stable`, `:head`, `:url`). `libzip` was the
original formula referenced in the related brew issue and it meets
this criterion, so it doesn't appear to be handled by the existing
`#use_homebrew_curl?` implementation.
Additionally, the existing behavior can cause livecheck to
unnecessarily use brewed curl for a completely different website
(e.g., `cubelib`, `otf2`). For example, if the `stable` URL has
`using: :homebrew_curl` and the `livecheck` block has `url
:homepage`, livecheck will use brewed curl when checking the
`homepage`. If these are completely different domains/servers, it's
unlikely that we would need to use brewed curl when checking the
`homepage`, so this particular behavior may not be beneficial.
This commit reimplements `use_homebrew_curl?` to apply brewed curl
when the checked URL's root domain is the same as the root domain of
an aforementioned formula/cask URL with `using: :homebrew_curl`. For
example, this looser approach would allow a `livecheck` block
checking `https://www.example.com/downloads/` to use brewed curl if
the `stable` URL was `https://downloads.example.com/example.zip` with
`using: :homebrew_curl`. These could be different servers but, based
on related formulae, this looseness is necessary for the moment.
This approach aims to resolve both issues, allowing brewed curl to be
applied to a slightly broader range of URLs (i.e., not limited to
just the `#checkable_urls`) while also helping to avoid unnecessarily
applying brewed curl when it's less likely to be useful (completely
different domains). Neither approach is perfect but this one may be
more useful in the interim time.
Depending on how this looser approach works in practice, we may want
to consider returning to a stricter approach once we have something
like `using: :homebrew_curl` in `livecheck` blocks (this is
forthcoming). Being explicit in a `livecheck` block is the most
reliable approach (i.e., only use brewed curl when needed), so we
could favor that and pare down the automated approach to only what's
needed to support implicit checks (i.e., with no `livecheck` block).
Of course, it's also possible to drop the automated approach entirely
and simply require a `livecheck` block in this scenario but we can
decide on how to handle this when the time comes.
At the moment, `#use_homebrew_curl?` can only be true for a
`homepage` or `stable`/cask `url` with `using: :homebrew_curl`. If
the checked URL differs from these URLs, livecheck won't use brewed
curl. This limitation prevents livecheck from using brewed curl for a
`livecheck` block URL that's a string literal (not a symbol for a
`#checkable_url` like `:stable`, `:head`, `:url`). `libzip` was the
original formula referenced in the related brew issue and it meets
this criterion, so it doesn't appear to be handled by the existing
`#use_homebrew_curl?` implementation.
Additionally, the existing behavior can cause livecheck to
unnecessarily use brewed curl for a completely different website
(e.g., `cubelib`, `otf2`). For example, if the `stable` URL has
`using: :homebrew_curl` and the `livecheck` block has `url
:homepage`, livecheck will use brewed curl when checking the
`homepage`. If these are completely different domains/servers, it's
unlikely that we would need to use brewed curl when checking the
`homepage`, so this particular behavior may not be beneficial.
This commit reimplements `use_homebrew_curl?` to apply brewed curl
when the checked URL's root domain is the same as the root domain of
an aforementioned formula/cask URL with `using: :homebrew_curl`. For
example, this looser approach would allow a `livecheck` block
checking `https://www.example.com/downloads/` to use brewed curl if
the `stable` URL was `https://downloads.example.com/example.zip` with
`using: :homebrew_curl`. These could be different servers but, based
on related formulae, this looseness is necessary for the moment.
This approach aims to resolve both issues, allowing brewed curl to be
applied to a slightly broader range of URLs (i.e., not limited to
just the `#checkable_urls`) while also helping to avoid unnecessarily
applying brewed curl when it's less likely to be useful (completely
different domains). Neither approach is perfect but this one may be
more useful in the interim time.
Depending on how this looser approach works in practice, we may want
to consider returning to a stricter approach once we have something
like `using: :homebrew_curl` in `livecheck` blocks (this is
forthcoming). Being explicit in a `livecheck` block is the most
reliable approach (i.e., only use brewed curl when needed), so we
could favor that and pare down the automated approach to only what's
needed to support implicit checks (i.e., with no `livecheck` block).
Of course, it's also possible to drop the automated approach entirely
and simply require a `livecheck` block in this scenario but we can
decide on how to handle this when the time comes.
Some formulae/casks contain duplicate checkable URLs or contain
URLs that become duplicates after `#preprocess_url` is used. With
the former, a formula's `stable` and `head` URLs can be the same if
they both use the Git repository. With the latter, a formula's
`homepage` and `stable` URLs are different but `#preprocess_url` can
return the same Git repository URL for each (which can also be
a duplicate of the `head` URL).
The `fabric-completion` formula is a worst case scenario, as it
contains both of these conditions but the repository has no tags.
By default, livecheck would needlessly check the repository three
times (i.e., no versions are found so livecheck tries the next URL,
which happens to be the same URL).
This commit avoids duplicate URLs in `#checkable_urls` and keeps
track of checked URLs in `#latest_version` to avoid a duplicate
caused by `#preprocess_url`. This should effectively resolve the
issue of checking the same URL more than once for a given
formula/cask. Checking the same URL only once across all the
formulae/casks in a given livecheck run will be handled in a later
commit (though I've done most of the groundwork already in previous
PRs).
