This adds a `Livecheck::Options` class, which is intended to house
various configuration options that are set in `livecheck` blocks,
conditionally set by livecheck at runtime, etc. The general idea is
that when we add features involving configurations options (e.g., for
livecheck, strategies, curl, etc.), we can make changes to `Options`
without needing to modify parameters for strategy `find_versions`
methods, `Strategy` methods like `page_headers` and `page_content`,
etc. This is something that I've been trying to improve over the years
and `Options` should help to reduce maintenance overhead in this area
while also strengthening type signatures.
`Options` replaces the existing `homebrew_curl` option (which related
strategies pass to `Strategy` methods and on to `curl_args`) and the
new `url_options` (which contains `post_form` or `post_json` values
that are used to make `POST` requests). I recently added `url_options`
as a temporary way of enabling `POST` support without `Options` but
this restores the original `Options`-based implementation.
Along the way, I added a `homebrew_curl` parameter to the `url` DSL
method, allowing us to set an explicit value in `livecheck` blocks.
This is something that we've needed in some cases but I also intend
to replace implicit/inferred `homebrew_curl` usage with explicit
values in `livecheck` blocks once this is available for use. My
intention is to eventually remove the implicit behavior and only rely
on explicit values. That will align with how `homebrew_curl` options
work for other URLs and makes the behavior clear just from looking at
the `livecheck` block.
Lastly, this removes the `unused` rest parameter from `find_versions`
methods. I originally added `unused` as a way of handling parameters
that some `find_versions` methods have but others don't (e.g., `cask`
in `ExtractPlist`), as this allowed us to pass various arguments to
`find_versions` methods without worrying about whether a particular
parameter is available. This isn't an ideal solution and I originally
wanted to handle this situation by only passing expected arguments to
`find_versions` methods but there was a technical issue standing in
the way. I recently found an answer to the issue, so this also
replaces the existing `ExtractPlist` special case with generic logic
that checks the parameters for a strategy's `find_versions` method
and only passes expected arguments.
Replacing the aforementioned `find_versions` parameters with `Options`
ensures that the remaining parameters are fairly consistent across
strategies and any differences are handled by the aforementioned
logic. Outside of `ExtractPlist`, the only other difference is that
some `find_versions` methods have a `provided_content` parameter but
that's currently only used by tests (though it's intended for caching
support in the future). I will be renaming that parameter to `content`
in an upcoming PR and expanding it to the other strategies, which
should make them all consistent outside of `ExtractPlist`.
`Sparkle` is the only strategy with a `find_versions` method that
calls `Strategy::page_content` (or `::page_headers`) and doesn't have
a `homebrew_curl` parameter. This adds the missing parameter and
passes the value to `page_content`, which brings it in line with the
other strategies.
livecheck currently doesn't support `POST` requests but it wasn't
entirely clear how best to handle that. I initially approached it as
a `Post` strategy but unfortunately that would have required us to
handle response body parsing (e.g., JSON, XML, etc.) in some fashion.
We could borrow some of the logic from related strategies but we would
still be stuck having to update `Post` whenever we add a strategy for
a new format.
Instead, this implements `POST` support by borrowing ideas from the
`using: :post` and `data` `url` options found in formulae. This uses
a `post_form` option to handle form data and `post_json` to handle
JSON data, encoding the hash argument for each into the appropriate
format. The presence of either option means that curl will use a
`POST` request.
With this approach, we can make a `POST` request using any strategy
that calls `Strategy::page_headers` or `::page_content` (directly or
indirectly) and everything else works the same as usual. The only
change needed in related strategies was to pass the options through
to the `Strategy` methods.
For example, if we need to parse a JSON response from a `POST`
request, we add a `post_data` or `post_json` hash to the `livecheck`
block `url` and use `strategy :json` with a `strategy` block. This
leans on existing patterns that we're already familiar with and
shouldn't require any notable maintenance burden when adding new
strategies, so it seems like a better approach than a `Post` strategy.
This updates livecheck files to use `typed: script` where feasible.
The remaining exception is `livecheck/strategy.rb`, as I wasn't
able to figure out how to resolve the typing issues around the
`@strategies` variable (I tried a couple of approaches but couldn't
find a working solution). This includes changes to resolve the other
type errors in `strategy.rb` but leaves the file as `typed: true`
for now.
This refactors verbose code in the `Sparkle` strategy where we access
element text into a reusable `Xml#element_text` method, replacing
chained calls like `item.elements["title"]&.text&.strip&.presence`
with `Xml.element_text(item, "title")`.
`#element_text` is only used to retrieve the text of a child element
in the `Sparkle` strategy but it can also retrieve the text from the
provided element if the `child_path` argument is omitted (i.e.,
`Xml.element_text(item)`). This will allow us to also avoid similar
calls like `item.text.strip.presence` in the future.
We need to be able to replicate the `Sparkle` strategy's sorting
and filtering behavior in a related cask audit, so this extracts
the logic into reusable methods.
This also stores `item.minimum_system_version` as a `MacOSVersion`
object (instead of a string), so we can do proper version comparison
(instead of naive string comparison) wherever needed.
Sometimes appcasts contain empty elements/attributes and the `Item`
values end up as an empty string because of how they're handled in
`#items_from_content`. It's reasonable to expect that empty values
would be `nil` instead, so this adds `#presence` calls to ensure this
is the case.
Historically, the `Sparkle` strategy's `Item` struct has only
included basic values from the appcast that are commonly useful.
Over time we've selectively added/surfaced more values as we've
encountered outliers that require use of different values in a
`strategy` block.
We now need to use `minimumSystemValue`, so this expands the `Item`
struct to include any appcast value that we could conceivably want
to use in the future. This will hopefully save us from having to make
more modifications to the struct (and related tests) before we can
use a previously-unused value in a `strategy` block.
`content` can be `nil` when a request doesn't succeed but
`#versions_from_content` expects a `String` value, so we need to
guard against a `nil` value like we do in other strategies.
This adds a generic `Xml` strategy to livecheck that requires a
`strategy` block to operate. The XML-parsing code is taken from the
existing approach in the `Sparkle` strategy. As such, `Sparkle` has
been updated to use the `Xml#parse_xml` method instead.
Unlike the `Json` strategy, we don't currently have any `strategy`
blocks in first-party taps that manually parse XML. However, we had a
user request support for something like this and I was already working
on an `Xml` strategy (as a way of extracting the XML-parsing code
from `Sparkle` into something general-purpose), so here we are.
Future strategies that parse simple XML data can potentially use the
`Xml#find_versions` method (similar to how we have strategies that
leverage `PageMatch#find_versions`) instead of having to implement
something bespoke like `Sparkle`.
Making `channel` information available in the `Item` is necessary
to be able to filter out unstable items using a `strategy` block. If
an item doesn't specify a channel, then it uses the default channel
(this is what Sparkle itself uses for updates). Channels like `beta`
are something we want to avoid for stable casks and this allows for
that type of [cask-specific] filtering.
It's technically possible to automatically filter out items that
aren't using the default channel (i.e., `channel != nil`) in
`#items_from_content` but some casks use an unstable version, so we
can't do this internally. That is to say, we wouldn't be able to
override internal filtering in a `strategy` block, as any omitted
items wouldn't be provided to the block. Conversely, if we pass all
items to a `strategy` block, we can easily filter by channel there.
We haven't been filtering by channel internally and we've only found
one cask where this has been a problem, so it seems fine for now.
It's sometimes necessary to work with all the items in a Sparkle feed
to be able to correctly identify the newest version but livecheck's
`Sparkle` strategy only passes the `item` it views as newest into a
`strategy` block.
This updates the `Sparkle` strategy to optionally pass all items into
a `strategy` block, so we can manipulate them (e.g., filtering,
sorting). This is enabled by naming the first argument of the
strategy block `items` instead of `item`. `Sparkle` `strategy` blocks
where the first argument is `item` will continue to work as expected.
This necessarily updates `#item_from_content` (now
`items_from_content`) to return all items. I've decided to move the
sorting out of `#items_from_content`, so it simply returns the items
in the order they appear. If there is ever an exceptional situation
where we need the original order, this will technically allow for it.
The sorting has instead been moved into the `#versions_from_content`
method, to maintain the existing behavior. I thought about passing
the items into the `strategy` block in their original order but it
feels like sorting by default is the better approach for now (partly
from the perspective of maintaining existing behavior) and we can
always revisit this in the future if a cask ever requires the
original order.
Lastly, this expands the `Sparkle` tests to increase coverage. The
only untested parts are `#find_versions` (which currently
requires a network request) and a couple safeguard `raise` calls
when there's a `REXML::UndefinedNamespaceException` (which shouldn't
be encountered unless something is broken).
There are times where a regex isn't needed in a `strategy` block and
these changes explicitly handle that situation.
This allows the Symbol Proc format used in some `Sparkle` `livecheck`
blocks (e.g., `strategy :sparkle, &:version`) to continue working
instead of failing with a "wrong number of arguments (given 1,
expected 0)" error. This error would occur because a Symbol Proc only
only expects one argument (e.g., an `Item`, not an `Item` and a
regex/nil).
There's an argument to be made for avoiding the Symbol Proc format
for `strategy` blocks but I haven't found a way of selectively
disabling the Style/SymbolProc cop only for a `strategy` DSL method
call. That is to say, if we don't use the Symbol Proc format, `brew
style` will give a "Pass &:version as an argument to strategy instead
of a block." offense.
Besides that, this also replaces the `block` type signatures in
livecheck strategies with `T.untyped`. Sorbet doesn't know how to
handle a `Proc` with a variable number of arguments and can't be
taught how (i.e., `T.any` with a `Proc` signature for each variation
doesn't work). The aforementioned changes cause Sorbet to complain
about there being both too many and too few arguments, so the only
way to win is not to play the game. Hopefully we can restore the
`block` type signatures in the future (if upstream resolves this
years-old issue) but `T.untyped` seems to be our only option for now.
This modifies cask-related livecheck strategies to allow passing a
regex into a `strategy` block, when appropriate. These strategies
were outliers that explicitly rejected a regex even if a `strategy`
block was used, forcing any regex to be inlined in the `strategy`
block (instead of being defined using `#regex`).
With these changes, all `strategy` blocks will be able to accept a
regex, further simplifying the mental model. This also helps to
better align the various `find_versions` and `versions_from_*`
methods across strategies.
Valid `strategy` block return types currently vary between
strategies. Some only accept a string whereas others accept a string
or array of strings. [`strategy` blocks also accept a `nil` return
(to simplify early returns) but this was already standardized across
strategies.]
While some strategies only identify one version by default (where a
string is an appropriate return type), it could be that a strategy
block identifies more than one version. In this situation, the
strategy would need to be modified to accept (and work with) an
array from a `strategy` block.
Rather than waiting for this to become a problem, this modifies all
strategies to standardize on allowing `strategy` blocks to return a
string or array of strings (even if only one of these is currently
used in practice). Standardizing valid return types helps to further
simplify the mental model for `strategy` blocks and reduce cognitive
load.
This commit extracts related logic from `#find_versions` into
methods like `#versions_from_content`, which is conceptually similar
to `PageMatch#page_matches` (renamed to `#versions_from_content`
for consistency). This allows us to write tests for the related code
without having to make network requests (or stub them) at this point.
In general, this also helps to better align the structure of
strategies and how the various `#find_versions` methods work with
versions.
There's still more planned work to be done here but this is a step
in the right direction.
