We discussed the idea of adding a livecheck strategy to check crate
versions years ago but decided to put it off because it would have
only applied to one formula at the time (and it wasn't clear that a
crate was necessary in that case). We now have a few formulae that
use a crate in the `stable` URL (`cargo-llvm-cov`, `pngquant`,
`oakc`) and another formula with a crate resource (`deno`), so
there's some value to the idea now.
I established a standard approach for checking crate versions in a
somewhat recent `pngquant` `livecheck` block update and this commit
reworks it into a strategy, so we won't have to duplicate that
`livecheck` block in these cases. With this strategy, we usually
won't even need a `livecheck` block at all.
Under normal circumstances, a regex and/or strategy block shouldn't
be necessary but the strategy supports them when needed. The response
from the crates.io API is a JSON object, so this uses
`Json#versions_from_content` internally and a `strategy` block will
receive the parsed `json` object and a regex (the strategy default or
the regex from the `livecheck` block).
This refactors verbose code in the `Sparkle` strategy where we access
element text into a reusable `Xml#element_text` method, replacing
chained calls like `item.elements["title"]&.text&.strip&.presence`
with `Xml.element_text(item, "title")`.
`#element_text` is only used to retrieve the text of a child element
in the `Sparkle` strategy but it can also retrieve the text from the
provided element if the `child_path` argument is omitted (i.e.,
`Xml.element_text(item)`). This will allow us to also avoid similar
calls like `item.text.strip.presence` in the future.
We need to be able to replicate the `Sparkle` strategy's sorting
and filtering behavior in a related cask audit, so this extracts
the logic into reusable methods.
This also stores `item.minimum_system_version` as a `MacOSVersion`
object (instead of a string), so we can do proper version comparison
(instead of naive string comparison) wherever needed.
Historically, the `Sparkle` strategy's `Item` struct has only
included basic values from the appcast that are commonly useful.
Over time we've selectively added/surfaced more values as we've
encountered outliers that require use of different values in a
`strategy` block.
We now need to use `minimumSystemValue`, so this expands the `Item`
struct to include any appcast value that we could conceivably want
to use in the future. This will hopefully save us from having to make
more modifications to the struct (and related tests) before we can
use a previously-unused value in a `strategy` block.
- Fix cask info output being incorrect
- Improve some code referring to casks as formulae
- Move livecheck cask fixtures to not shadow existing names
- Adjust the cask tap symlinking logic to make handling outdated
shadowed casks significantly easier
- Fix various flaky tests caused by casks sharding logic
- Prefer longer paths when there's multiple formulae or casks in a tap
with the same name rather than always using the first
The `Bitbucket` strategy currently matches versions from tag
tarball links on a project's `downloads/?tab=tags` page. It appears
that Bitbucket now uses a hash as the filename on this page instead
of the tag name, so the existing regex no longer matches.
This adds an alternative regex to match versions from the tag name
element (e.g., `<td class="name">example-1.2.3</td>`), which will fix
version matching in this scenario.
This adds tests to cover all of the strategy outside of the
`#find_versions` method, which we don't currently test because it
involves a network request.
`GithubLatest` was updated to use parts of `GithubRelease` and this
works fine within `brew livecheck` (since we `require` all the
strategies) but we need to explicitly `require` `GithubRelease` in
the `GithubLatest` test file now. Without this, we encounter errors
in parts of `GithubLatest` where `GithubRelease` is referenced.
This updates `ElectronBuilder` to use `Yaml#find_versions`, as the
only code unique to that strategy is to restrict regex usage and
use default version-finding logic when a `strategy` block isn't
provided. This is similar to how we have various strategies that
use `PageMatch#find_versions` internally.
This allows us to remove `ElectronBuilder#versions_from_content`
entirely, along with the related tests. I've added support for
`provided_content` to `ElectronBuilder#find_versions` as a way of
adding tests and maintaining code coverage.
This adds a generic `Yaml` strategy to livecheck that requires a
`strategy` block to operate. The YAML-parsing code is taken from the
existing approach in the `ElectronBuilder` strategy.
We don't currently have any `strategy` blocks in first-party taps
that manually parse YAML. However, creating a generic `Yaml` strategy
allows us to simplify `ElectronBuilder` (and any future strategy
that works with YAML) while making it easy to create custom `Yaml`
`strategy` blocks in formulae/casks as needed.
This renames `simple_content` to `content_simple`, which makes the
`content_*` variable name convention more consistent (as is in the
`Xml` strategy tests).
This setup mimics the `#parse_xml` method that was implemented in the
`Xml` strategy. Isolating the parsing code means that other strategies
can take only what they need from `Json` (i.e., it's not required for
them to use `Json#find_versions`).
This adds a generic `Xml` strategy to livecheck that requires a
`strategy` block to operate. The XML-parsing code is taken from the
existing approach in the `Sparkle` strategy. As such, `Sparkle` has
been updated to use the `Xml#parse_xml` method instead.
Unlike the `Json` strategy, we don't currently have any `strategy`
blocks in first-party taps that manually parse XML. However, we had a
user request support for something like this and I was already working
on an `Xml` strategy (as a way of extracting the XML-parsing code
from `Sparkle` into something general-purpose), so here we are.
Future strategies that parse simple XML data can potentially use the
`Xml#find_versions` method (similar to how we have strategies that
leverage `PageMatch#find_versions`) instead of having to implement
something bespoke like `Sparkle`.
This adds a generic `Json` strategy to livecheck that requires a
`strategy` block to operate. This is primarily intended as a
replacement for existing `strategy` blocks in formulae/casks that
use `JSON#parse`, as it allows us to internalize/standardize that
boilerplate while improving error-handling.
Additionally, future strategies that parse JSON data can use the
`Json#find_versions` method instead of having to reinvent the wheel
(similar to how we currently have a number of strategies that
leverage `PageMatch#find_versions`).
The `ElectronBuilder` strategy uses `YAML#safe_load` to parse YAML
content and this limits deserialization to appropriate classes. We
recently encountered a `Tried to load unspecified class: Time` error
when using the `ElectronBuilder` strategy on a `latest-mac.yml` file
containing `releaseDate: 2022-12-01T02:02:46.419Z`.
The electron-builder YAML files we usually encounter use single
quotes around the `releaseDate` value to ensure it's treated as a
string (e.g., `releaseDate: '2022-10-12T17:55:26.718Z'`) and this is
what we do in `electron_builder_spec.rb`. The aforementioned YAML
file doesn't use single quotes around the value, so it's treated as
a timestamp and apparently this makes Psych use `Time` (which
`#safe_load` doesn't allow by default).
Seeing as we can't control the YAML content and there's a chance we
may encounter other files like this in the future, this commit
modifies the related `#safe_load` call to allow `Time` (and `Date`
for good measure). This will resolve the aforementioned error and
allow the `ElectronBuilder` strategy to work as expected in this
scenario.
The `Gnome` strategy's default regex uses the `+` form of the standard
regex for matching versions like 1.2.3. However, with the switch to
the new version scheme, some packages had a release that omits a
minor and patch (i.e., `40` instead of `40.0`). The default regex
fails to match versions like this but the looser `*` form will match
both. [When creating regexes, we generally start with the `+` form
and only switch to the looser `*` form when it's necessary and
contextually-appropriate.]
This also updates the default version filtering logic that's applied
to versions using the old GNOME version scheme (below version 40).
Outside of the refactoring changes, this also filters out versions
where the patch number is 90+, as these are also unstable.
Making `channel` information available in the `Item` is necessary
to be able to filter out unstable items using a `strategy` block. If
an item doesn't specify a channel, then it uses the default channel
(this is what Sparkle itself uses for updates). Channels like `beta`
are something we want to avoid for stable casks and this allows for
that type of [cask-specific] filtering.
It's technically possible to automatically filter out items that
aren't using the default channel (i.e., `channel != nil`) in
`#items_from_content` but some casks use an unstable version, so we
can't do this internally. That is to say, we wouldn't be able to
override internal filtering in a `strategy` block, as any omitted
items wouldn't be provided to the block. Conversely, if we pass all
items to a `strategy` block, we can easily filter by channel there.
We haven't been filtering by channel internally and we've only found
one cask where this has been a problem, so it seems fine for now.
It's sometimes necessary to work with all the items in a Sparkle feed
to be able to correctly identify the newest version but livecheck's
`Sparkle` strategy only passes the `item` it views as newest into a
`strategy` block.
This updates the `Sparkle` strategy to optionally pass all items into
a `strategy` block, so we can manipulate them (e.g., filtering,
sorting). This is enabled by naming the first argument of the
strategy block `items` instead of `item`. `Sparkle` `strategy` blocks
where the first argument is `item` will continue to work as expected.
This necessarily updates `#item_from_content` (now
`items_from_content`) to return all items. I've decided to move the
sorting out of `#items_from_content`, so it simply returns the items
in the order they appear. If there is ever an exceptional situation
where we need the original order, this will technically allow for it.
The sorting has instead been moved into the `#versions_from_content`
method, to maintain the existing behavior. I thought about passing
the items into the `strategy` block in their original order but it
feels like sorting by default is the better approach for now (partly
from the perspective of maintaining existing behavior) and we can
always revisit this in the future if a cask ever requires the
original order.
Lastly, this expands the `Sparkle` tests to increase coverage. The
only untested parts are `#find_versions` (which currently
requires a network request) and a couple safeguard `raise` calls
when there's a `REXML::UndefinedNamespaceException` (which shouldn't
be encountered unless something is broken).
This modifies cask-related livecheck strategies to allow passing a
regex into a `strategy` block, when appropriate. These strategies
were outliers that explicitly rejected a regex even if a `strategy`
block was used, forcing any regex to be inlined in the `strategy`
block (instead of being defined using `#regex`).
With these changes, all `strategy` blocks will be able to accept a
regex, further simplifying the mental model. This also helps to
better align the various `find_versions` and `versions_from_*`
methods across strategies.
Some Apache URLs use a `filename` query string parameter instead of
`path` and the `Apache` strategy isn't applied. The parameter value
is the same as `path` (i.e., a path to a file), so this updates the
strategy's `URL_MATCH_REGEX` to handle this URL format as well.
Valid `strategy` block return types currently vary between
strategies. Some only accept a string whereas others accept a string
or array of strings. [`strategy` blocks also accept a `nil` return
(to simplify early returns) but this was already standardized across
strategies.]
While some strategies only identify one version by default (where a
string is an appropriate return type), it could be that a strategy
block identifies more than one version. In this situation, the
strategy would need to be modified to accept (and work with) an
array from a `strategy` block.
Rather than waiting for this to become a problem, this modifies all
strategies to standardize on allowing `strategy` blocks to return a
string or array of strings (even if only one of these is currently
used in practice). Standardizing valid return types helps to further
simplify the mental model for `strategy` blocks and reduce cognitive
load.
This commit extracts related logic from `#find_versions` into
methods like `#versions_from_content`, which is conceptually similar
to `PageMatch#page_matches` (renamed to `#versions_from_content`
for consistency). This allows us to write tests for the related code
without having to make network requests (or stub them) at this point.
In general, this also helps to better align the structure of
strategies and how the various `#find_versions` methods work with
versions.
There's still more planned work to be done here but this is a step
in the right direction.
