The `Bitbucket` strategy checks download or tag pages but the content
is now fetched separately on page load, so the strategy is failing for
all related formulae. This updates the generated strategy URLs to
fetch the page content instead, which works as expected.
The `mesa` formula currently uses a mesa.freedesktop.org/archive/
`stable` URL but it redirects to archive.mesa3d.org. Upstream links
to archive.mesa3d.org as the location to find Mesa releases, so we
should update the formula URLs accordingly.
This updates the `Xorg` strategy to be able to handle
archive.mesa3d.org URLs, so livecheck will continue to be able to
check `mesa` without needing a one-off `livecheck` block. [This would
also work for `mesalib-glw` (which has an archive.mesa3d.org `stable`
URL) but that formula is deprecated.]
We recently updated the `Pypi` strategy to use the PyPI JSON API and
the default strategy behavior no longer relies on a regex, so the initial implementation didn't include regex handling. This restores
support for a `livecheck` block regex by updating the `DEFAULT_BLOCK`
logic to handle an optional regex. This allows us to use a regex to
omit parts of the `info.version` value without having to duplicate
the default block logic in a `strategy` block only to use a regex.
This isn't currently necessary for any existing formulae using the
`Pypi` strategy but we have a few that needed a custom regex with
the previous strategy approach, so they may need this functionality
in the future. Besides that, restoring regex support to `Pypi`
ensures that `livecheck`/`strategy` blocks work in a fairly
consistent manner across strategies.
This updates the block-handling logic in `Json::versions_from_content`
to naively pass the regex value when the block has two parameters. Up
to now, we have been ensuring that `regex` is not `nil` and this
makes sense with existing usage (the `Crate` strategy's default
block, formulae/cask `strategy` blocks). However, we need to allow a
`nil` `regex` value to make it possible to add an optional `regex`
parameter in the `Pypi::DEFAULT_BLOCK` Proc. This is necessary to
allow the `Pypi` strategy to work with an optional regex from a
`livecheck` block again [without creating an additional
`DEFAULT_BLOCK` variant with a regex parameter].
Among other things, the previous commit added a `provided_content`
paramter to `Pypi::find_versions`, so this takes advantage of that to
expand `Pypi` test coverage to 100%.
This reworks the new `Pypi` JSON API implementation to use
`Json::find_versions` in `Pypi::find_versions`, borrowing some of the
approach from the `Crate` strategy.
Besides that, this pares down the fields in the
`::generate_input_values` return hash to only `:url`, as we're not
using a generated regex to match version information in this setup.
This adds a `provided_content` parameter to `::find_versions` as part
of this process and I will expand the `Pypi` tests to increase
coverage (like the `Crates` tests) in a later PR. 75% of `Pypi` checks
are failing at the moment (with some returning inaccurate version
information), so the current priority is getting this fix merged in
the short-term.
`Livecheck#preprocess_url` only contains logic for rewriting Git URLs,
so it makes more sense for this code to be part of the `Git` strategy
instead. Outside of better code organization, this saves us from
having to maintain the list of strategies to skip processing (which
is sometimes forgotten when a new strategy is added) and makes it
easier to do something similar in other strategies as needed.
One thing to note is that `Livecheck#preprocess_url` was previously
called on the URL before each strategy's `#match?` method was called.
To maintain the existing behavior, this calls `Git#preprocess_url` in
`Git#match?`. However, we need the processed URL when we use the `Git`
strategy, so we have to call `Git#preprocess_url` again. To avoid
duplicating effort, I've added a `@processed_urls` hash to the `Git`
strategy and have set up `Git#preprocess_url` to cache processed
URLs, so we only do the work once. There may be a better way of
handling it but this seems to work as expected.
livecheck is returning an `Unable to get versions` error for the
`ansible-lint`, `aws-sam-cli`, and `pyqt-builder` formulae. These use
the `Pypi` strategy without a `livecheck` block, so they use the
generated regex from the strategy. The `Pypi` strategy matches the
version from the tarball link on the pypi.org package page but this
fails for these packages because the formula's `stable` tarball uses
hyphens in the filename (e.g., `ansible-lint-...`) but the current
tarball filename uses underscores (e.g., `ansible_lint-...`).
This addresses the issue by updating the strategy regex to replace
[escaped] `-` or `_` characters in the package name with `[_-]`, so
the regex will match regardless of the delimiter used in the formula
filename.
We discussed the idea of adding a livecheck strategy to check crate
versions years ago but decided to put it off because it would have
only applied to one formula at the time (and it wasn't clear that a
crate was necessary in that case). We now have a few formulae that
use a crate in the `stable` URL (`cargo-llvm-cov`, `pngquant`,
`oakc`) and another formula with a crate resource (`deno`), so
there's some value to the idea now.
I established a standard approach for checking crate versions in a
somewhat recent `pngquant` `livecheck` block update and this commit
reworks it into a strategy, so we won't have to duplicate that
`livecheck` block in these cases. With this strategy, we usually
won't even need a `livecheck` block at all.
Under normal circumstances, a regex and/or strategy block shouldn't
be necessary but the strategy supports them when needed. The response
from the crates.io API is a JSON object, so this uses
`Json#versions_from_content` internally and a `strategy` block will
receive the parsed `json` object and a regex (the strategy default or
the regex from the `livecheck` block).
This refactors verbose code in the `Sparkle` strategy where we access
element text into a reusable `Xml#element_text` method, replacing
chained calls like `item.elements["title"]&.text&.strip&.presence`
with `Xml.element_text(item, "title")`.
`#element_text` is only used to retrieve the text of a child element
in the `Sparkle` strategy but it can also retrieve the text from the
provided element if the `child_path` argument is omitted (i.e.,
`Xml.element_text(item)`). This will allow us to also avoid similar
calls like `item.text.strip.presence` in the future.
We need to be able to replicate the `Sparkle` strategy's sorting
and filtering behavior in a related cask audit, so this extracts
the logic into reusable methods.
This also stores `item.minimum_system_version` as a `MacOSVersion`
object (instead of a string), so we can do proper version comparison
(instead of naive string comparison) wherever needed.
Historically, the `Sparkle` strategy's `Item` struct has only
included basic values from the appcast that are commonly useful.
Over time we've selectively added/surfaced more values as we've
encountered outliers that require use of different values in a
`strategy` block.
We now need to use `minimumSystemValue`, so this expands the `Item`
struct to include any appcast value that we could conceivably want
to use in the future. This will hopefully save us from having to make
more modifications to the struct (and related tests) before we can
use a previously-unused value in a `strategy` block.
- Fix cask info output being incorrect
- Improve some code referring to casks as formulae
- Move livecheck cask fixtures to not shadow existing names
- Adjust the cask tap symlinking logic to make handling outdated
shadowed casks significantly easier
- Fix various flaky tests caused by casks sharding logic
- Prefer longer paths when there's multiple formulae or casks in a tap
with the same name rather than always using the first
The `Bitbucket` strategy currently matches versions from tag
tarball links on a project's `downloads/?tab=tags` page. It appears
that Bitbucket now uses a hash as the filename on this page instead
of the tag name, so the existing regex no longer matches.
This adds an alternative regex to match versions from the tag name
element (e.g., `<td class="name">example-1.2.3</td>`), which will fix
version matching in this scenario.
This adds tests to cover all of the strategy outside of the
`#find_versions` method, which we don't currently test because it
involves a network request.
`GithubLatest` was updated to use parts of `GithubRelease` and this
works fine within `brew livecheck` (since we `require` all the
strategies) but we need to explicitly `require` `GithubRelease` in
the `GithubLatest` test file now. Without this, we encounter errors
in parts of `GithubLatest` where `GithubRelease` is referenced.
This updates `ElectronBuilder` to use `Yaml#find_versions`, as the
only code unique to that strategy is to restrict regex usage and
use default version-finding logic when a `strategy` block isn't
provided. This is similar to how we have various strategies that
use `PageMatch#find_versions` internally.
This allows us to remove `ElectronBuilder#versions_from_content`
entirely, along with the related tests. I've added support for
`provided_content` to `ElectronBuilder#find_versions` as a way of
adding tests and maintaining code coverage.
This adds a generic `Yaml` strategy to livecheck that requires a
`strategy` block to operate. The YAML-parsing code is taken from the
existing approach in the `ElectronBuilder` strategy.
We don't currently have any `strategy` blocks in first-party taps
that manually parse YAML. However, creating a generic `Yaml` strategy
allows us to simplify `ElectronBuilder` (and any future strategy
that works with YAML) while making it easy to create custom `Yaml`
`strategy` blocks in formulae/casks as needed.
This renames `simple_content` to `content_simple`, which makes the
`content_*` variable name convention more consistent (as is in the
`Xml` strategy tests).
This setup mimics the `#parse_xml` method that was implemented in the
`Xml` strategy. Isolating the parsing code means that other strategies
can take only what they need from `Json` (i.e., it's not required for
them to use `Json#find_versions`).
This adds a generic `Xml` strategy to livecheck that requires a
`strategy` block to operate. The XML-parsing code is taken from the
existing approach in the `Sparkle` strategy. As such, `Sparkle` has
been updated to use the `Xml#parse_xml` method instead.
Unlike the `Json` strategy, we don't currently have any `strategy`
blocks in first-party taps that manually parse XML. However, we had a
user request support for something like this and I was already working
on an `Xml` strategy (as a way of extracting the XML-parsing code
from `Sparkle` into something general-purpose), so here we are.
Future strategies that parse simple XML data can potentially use the
`Xml#find_versions` method (similar to how we have strategies that
leverage `PageMatch#find_versions`) instead of having to implement
something bespoke like `Sparkle`.
This adds a generic `Json` strategy to livecheck that requires a
`strategy` block to operate. This is primarily intended as a
replacement for existing `strategy` blocks in formulae/casks that
use `JSON#parse`, as it allows us to internalize/standardize that
boilerplate while improving error-handling.
Additionally, future strategies that parse JSON data can use the
`Json#find_versions` method instead of having to reinvent the wheel
(similar to how we currently have a number of strategies that
leverage `PageMatch#find_versions`).
The `ElectronBuilder` strategy uses `YAML#safe_load` to parse YAML
content and this limits deserialization to appropriate classes. We
recently encountered a `Tried to load unspecified class: Time` error
when using the `ElectronBuilder` strategy on a `latest-mac.yml` file
containing `releaseDate: 2022-12-01T02:02:46.419Z`.
The electron-builder YAML files we usually encounter use single
quotes around the `releaseDate` value to ensure it's treated as a
string (e.g., `releaseDate: '2022-10-12T17:55:26.718Z'`) and this is
what we do in `electron_builder_spec.rb`. The aforementioned YAML
file doesn't use single quotes around the value, so it's treated as
a timestamp and apparently this makes Psych use `Time` (which
`#safe_load` doesn't allow by default).
Seeing as we can't control the YAML content and there's a chance we
may encounter other files like this in the future, this commit
modifies the related `#safe_load` call to allow `Time` (and `Date`
for good measure). This will resolve the aforementioned error and
allow the `ElectronBuilder` strategy to work as expected in this
scenario.
The `Gnome` strategy's default regex uses the `+` form of the standard
regex for matching versions like 1.2.3. However, with the switch to
the new version scheme, some packages had a release that omits a
minor and patch (i.e., `40` instead of `40.0`). The default regex
fails to match versions like this but the looser `*` form will match
both. [When creating regexes, we generally start with the `+` form
and only switch to the looser `*` form when it's necessary and
contextually-appropriate.]
This also updates the default version filtering logic that's applied
to versions using the old GNOME version scheme (below version 40).
Outside of the refactoring changes, this also filters out versions
where the patch number is 90+, as these are also unstable.
Making `channel` information available in the `Item` is necessary
to be able to filter out unstable items using a `strategy` block. If
an item doesn't specify a channel, then it uses the default channel
(this is what Sparkle itself uses for updates). Channels like `beta`
are something we want to avoid for stable casks and this allows for
that type of [cask-specific] filtering.
It's technically possible to automatically filter out items that
aren't using the default channel (i.e., `channel != nil`) in
`#items_from_content` but some casks use an unstable version, so we
can't do this internally. That is to say, we wouldn't be able to
override internal filtering in a `strategy` block, as any omitted
items wouldn't be provided to the block. Conversely, if we pass all
items to a `strategy` block, we can easily filter by channel there.
We haven't been filtering by channel internally and we've only found
one cask where this has been a problem, so it seems fine for now.
It's sometimes necessary to work with all the items in a Sparkle feed
to be able to correctly identify the newest version but livecheck's
`Sparkle` strategy only passes the `item` it views as newest into a
`strategy` block.
This updates the `Sparkle` strategy to optionally pass all items into
a `strategy` block, so we can manipulate them (e.g., filtering,
sorting). This is enabled by naming the first argument of the
strategy block `items` instead of `item`. `Sparkle` `strategy` blocks
where the first argument is `item` will continue to work as expected.
This necessarily updates `#item_from_content` (now
`items_from_content`) to return all items. I've decided to move the
sorting out of `#items_from_content`, so it simply returns the items
in the order they appear. If there is ever an exceptional situation
where we need the original order, this will technically allow for it.
The sorting has instead been moved into the `#versions_from_content`
method, to maintain the existing behavior. I thought about passing
the items into the `strategy` block in their original order but it
feels like sorting by default is the better approach for now (partly
from the perspective of maintaining existing behavior) and we can
always revisit this in the future if a cask ever requires the
original order.
Lastly, this expands the `Sparkle` tests to increase coverage. The
only untested parts are `#find_versions` (which currently
requires a network request) and a couple safeguard `raise` calls
when there's a `REXML::UndefinedNamespaceException` (which shouldn't
be encountered unless something is broken).
