admin/brew
1
0
Fork 0
Code Issues Packages Projects Releases 6 Wiki Activity

Merge pull request #14868 from samford/livecheck/add-yaml-strategy

Browse Source 
livecheck: Add Yaml strategy
This commit is contained in:
Mike McQuaid 2023-03-04 00:06:11 +00:00 committed by GitHub
commit 066a8afe61
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
7 changed files with 382 additions and 95 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
Library/Homebrew/.rubocop.yml
View File

@ -57,6 +57,7 @@ Style/Documentation:
- livecheck/strategy/sparkle.rb
- livecheck/strategy/xml.rb
- livecheck/strategy/xorg.rb
- livecheck/strategy/yaml.rb
- os.rb
- resource.rb
- utils/inreplace.rb

3
Library/Homebrew/livecheck/strategy.rb
View File

@ -156,7 +156,7 @@ module Homebrew
# Only treat the strategy as usable if the `livecheck` block
# contains a regex and/or `strategy` block
next if !regex_provided && !block_provided
elsif [Json, Xml].include?(strategy)
elsif [Json, Xml, Yaml].include?(strategy)
# Only treat the strategy as usable if the `livecheck` block
# specifies the strategy and contains a `strategy` block
next if (livecheck_strategy != strategy_symbol) || !block_provided
@ -286,3 +286,4 @@ require_relative "strategy/sourceforge"
require_relative "strategy/sparkle"
require_relative "strategy/xml"
require_relative "strategy/xorg"
require_relative "strategy/yaml"

58
Library/Homebrew/livecheck/strategy/electron_builder.rb
View File

@ -32,61 +32,35 @@ module Homebrew
URL_MATCH_REGEX.match?(url)
end
# Parses YAML text and identifies versions in it.
#
# @param content [String] the YAML text to parse and check
# @param regex [Regexp, nil] a regex for use in a strategy block
# @return [Array]
sig {
params(
content: String,
regex: T.nilable(Regexp),
block: T.untyped,
).returns(T::Array[String])
}
def self.versions_from_content(content, regex = nil, &block)
require "yaml"
yaml = YAML.safe_load(content, permitted_classes: [Date, Time])
return [] if yaml.blank?
if block
block_return_value = regex.present? ? yield(yaml, regex) : yield(yaml)
return Strategy.handle_block_return(block_return_value)
end
version = yaml["version"]
version.present? ? [version] : []
end
# Checks the YAML content at the URL for new versions.
#
# @param url [String] the URL of the content to check
# @param regex [Regexp, nil] a regex used for matching versions
# @param provided_content [String, nil] content to use in place of
# fetching via `Strategy#page_content`
# @return [Hash]
sig {
params(
url: String,
regex: T.nilable(Regexp),
_unused: T.nilable(T::Hash[Symbol, T.untyped]),
block: T.untyped,
url: String,
regex: T.nilable(Regexp),
provided_content: T.nilable(String),
unused: T.nilable(T::Hash[Symbol, T.untyped]),
block: T.untyped,
).returns(T::Hash[Symbol, T.untyped])
}
def self.find_versions(url:, regex: nil, **_unused, &block)
def self.find_versions(url:, regex: nil, provided_content: nil, **unused, &block)
if regex.present? && block.blank?
raise ArgumentError,
"#{Utils.demodulize(T.must(name))} only supports a regex when using a `strategy` block"
end
match_data = { matches: {}, regex: regex, url: url }
match_data.merge!(Strategy.page_content(url))
content = match_data.delete(:content)
versions_from_content(content, regex, &block).each do |version_text|
match_data[:matches][version_text] = Version.new(version_text)
end
match_data
T.unsafe(Yaml).find_versions(
url: url,
regex: regex,
provided_content: provided_content,
**unused,
&block || proc { |yaml| yaml["version"] }
)
end
end
end

135
Library/Homebrew/livecheck/strategy/yaml.rb Normal file
View File

@ -0,0 +1,135 @@
# typed: true
# frozen_string_literal: true
module Homebrew
module Livecheck
module Strategy
# The {Yaml} strategy fetches content at a URL, parses it as YAML, and
# provides the parsed data to a `strategy` block. If a regex is present
# in the `livecheck` block, it should be passed as the second argument to
# the `strategy` block.
#
# This is a generic strategy that doesn't contain any logic for finding
# versions, as the structure of YAML data varies. Instead, a `strategy`
# block must be used to extract version information from the YAML data.
#
# This strategy is not applied automatically and it is necessary to use
# `strategy :yaml` in a `livecheck` block (in conjunction with a
# `strategy` block) to use it.
#
# This strategy's {find_versions} method can be used in other strategies
# that work with YAML content, so it should only be necessary to write
# the version-finding logic that works with the parsed YAML data.
#
# @api public
class Yaml
extend T::Sig
NICE_NAME = "YAML"
# A priority of zero causes livecheck to skip the strategy. We do this
# for {Yaml} so we can selectively apply it only when a strategy block
# is provided in a `livecheck` block.
PRIORITY = 0
# The `Regexp` used to determine if the strategy applies to the URL.
URL_MATCH_REGEX = %r{^https?://}i.freeze
# Whether the strategy can be applied to the provided URL.
# {Yaml} will technically match any HTTP URL but is only usable with
# a `livecheck` block containing a `strategy` block.
#
# @param url [String] the URL to match against
# @return [Boolean]
sig { params(url: String).returns(T::Boolean) }
def self.match?(url)
URL_MATCH_REGEX.match?(url)
end
# Parses YAML text and returns the parsed data.
# @param content [String] the YAML text to parse
sig { params(content: String).returns(T.untyped) }
def self.parse_yaml(content)
require "yaml"
begin
YAML.safe_load(content, permitted_classes: [Date, Time])
rescue Psych::SyntaxError
raise "Content could not be parsed as YAML."
end
end
# Parses YAML text and identifies versions using a `strategy` block.
# If a regex is provided, it will be passed as the second argument to
# the `strategy` block (after the parsed YAML data).
# @param content [String] the YAML text to parse and check
# @param regex [Regexp, nil] a regex used for matching versions in the
# content
# @return [Array]
sig {
params(
content: String,
regex: T.nilable(Regexp),
block: T.untyped,
).returns(T::Array[String])
}
def self.versions_from_content(content, regex = nil, &block)
return [] if content.blank? || block.blank?
yaml = parse_yaml(content)
return [] if yaml.blank?
block_return_value = if regex.present?
yield(yaml, regex)
elsif block.arity == 2
raise "Two arguments found in `strategy` block but no regex provided."
else
yield(yaml)
end
Strategy.handle_block_return(block_return_value)
end
# Checks the YAML content at the URL for versions, using the provided
# `strategy` block to extract version information.
#
# @param url [String] the URL of the content to check
# @param regex [Regexp, nil] a regex used for matching versions
# @param provided_content [String, nil] page content to use in place of
# fetching via `Strategy#page_content`
# @param homebrew_curl [Boolean] whether to use brewed curl with the URL
# @return [Hash]
sig {
params(
url: String,
regex: T.nilable(Regexp),
provided_content: T.nilable(String),
homebrew_curl: T::Boolean,
_unused: T.nilable(T::Hash[Symbol, T.untyped]),
block: T.untyped,
).returns(T::Hash[Symbol, T.untyped])
}
def self.find_versions(url:, regex: nil, provided_content: nil, homebrew_curl: false, **_unused, &block)
raise ArgumentError, "#{Utils.demodulize(T.must(name))} requires a `strategy` block" if block.blank?
match_data = { matches: {}, regex: regex, url: url }
return match_data if url.blank? || block.blank?
content = if provided_content.is_a?(String)
match_data[:cached] = true
provided_content
else
match_data.merge!(Strategy.page_content(url, homebrew_curl: homebrew_curl))
match_data[:content]
end
return match_data if content.blank?
versions_from_content(content, regex, &block).each do |match_text|
match_data[:matches][match_text] = Version.new(match_text)
end
match_data
end
end
end
end
end

109
Library/Homebrew/test/livecheck/strategy/electron_builder_spec.rb
View File

@ -6,10 +6,12 @@ require "livecheck/strategy"
describe Homebrew::Livecheck::Strategy::ElectronBuilder do
subject(:electron_builder) { described_class }
let(:yaml_url) { "https://www.example.com/example/latest-mac.yml" }
let(:non_yaml_url) { "https://brew.sh/test" }
let(:http_url) { "https://www.example.com/example/latest-mac.yml" }
let(:non_http_url) { "ftp://brew.sh/" }
let(:electron_builder_yaml) {
let(:regex) { /Example[._-]v?(\d+(?:\.\d+)+)[._-]mac\.zip/i }
let(:content) {
<<~EOS
version: 1.2.3
files:
@ -26,77 +28,80 @@ describe Homebrew::Livecheck::Strategy::ElectronBuilder do
EOS
}
let(:electron_builder_yaml_with_timestamp) {
let(:content_timestamp) {
# An electron-builder YAML file may use a timestamp instead of an explicit
# string value (with quotes) for `releaseDate`, so we need to make sure that
# `ElectronBuilder#versions_from_content` won't encounter an error in this
# scenario (e.g. `Tried to load unspecified class: Time`).
electron_builder_yaml.sub(/releaseDate:\s*'([^']+)'/, 'releaseDate: \1')
content.sub(/releaseDate:\s*'([^']+)'/, 'releaseDate: \1')
}
let(:mac_regex) { /Example[._-]v?(\d+(?:\.\d+)+)[._-]mac\.zip/i }
let(:content_matches) { ["1.2.3"] }
let(:versions) { ["1.2.3"] }
let(:find_versions_return_hash) {
{
matches: {
"1.2.3" => Version.new("1.2.3"),
},
regex: nil,
url: http_url,
}
}
let(:find_versions_cached_return_hash) {
find_versions_return_hash.merge({ cached: true })
}
describe "::match?" do
it "returns true for a YAML file URL" do
expect(electron_builder.match?(yaml_url)).to be true
expect(electron_builder.match?(http_url)).to be true
end
it "returns false for non-YAML URL" do
expect(electron_builder.match?(non_yaml_url)).to be false
expect(electron_builder.match?(non_http_url)).to be false
end
end
describe "::versions_from_content" do
it "returns an empty array if content is blank" do
expect(electron_builder.versions_from_content("")).to eq([])
describe "::find_versions?" do
it "finds versions in provided_content using a block" do
expect(electron_builder.find_versions(url: http_url, provided_content: content))
.to eq(find_versions_cached_return_hash)
expect(electron_builder.find_versions(url: http_url, regex: regex, provided_content: content) do |yaml, regex|
yaml["path"][regex, 1]
end).to eq(find_versions_cached_return_hash.merge({ regex: regex }))
expect(electron_builder.find_versions(
url: http_url,
regex: regex,
provided_content: content_timestamp,
) do |yaml, regex|
yaml["path"][regex, 1]
end).to eq(find_versions_cached_return_hash.merge({ regex: regex }))
# NOTE: A regex should be provided using the `#regex` method in a
# `livecheck` block but we're using a regex literal in the `strategy`
# block here simply to ensure this method works as expected when a
# regex isn't provided.
expect(electron_builder.find_versions(url: http_url, provided_content: content) do |yaml|
regex = /^v?(\d+(?:\.\d+)+)$/i.freeze
yaml["version"][regex, 1]
end).to eq(find_versions_cached_return_hash)
end
it "returns an array of version strings when given YAML text" do
expect(electron_builder.versions_from_content(electron_builder_yaml)).to eq(versions)
expect(electron_builder.versions_from_content(electron_builder_yaml_with_timestamp)).to eq(versions)
it "errors if a block is not provided" do
expect { electron_builder.find_versions(url: http_url, regex: regex, provided_content: content) }
.to raise_error(ArgumentError, "ElectronBuilder only supports a regex when using a `strategy` block")
end
it "returns an array of version strings when given YAML text and a block" do
# Returning a string from block
expect(
electron_builder.versions_from_content(electron_builder_yaml) do |yaml|
yaml["version"].sub("3", "4")
end,
).to eq(["1.2.4"])
# Returning an array of strings from block
expect(electron_builder.versions_from_content(electron_builder_yaml) { versions }).to eq(versions)
it "returns default match_data when url is blank" do
expect(electron_builder.find_versions(url: "") { "1.2.3" })
.to eq({ matches: {}, regex: nil, url: "" })
end
it "returns an array of version strings when given YAML text, a regex, and a block" do
# Returning a string from block
expect(
electron_builder.versions_from_content(electron_builder_yaml, mac_regex) do |yaml, regex|
yaml["path"][regex, 1]
end,
).to eq(versions)
# Returning an array of strings from block
expect(
electron_builder.versions_from_content(electron_builder_yaml, mac_regex) do |yaml, regex|
yaml["files"]&.map do |file|
next if file["url"].blank?
file["url"][regex, 1]
end
end,
).to eq(versions)
end
it "allows a nil return from a block" do
expect(electron_builder.versions_from_content(electron_builder_yaml) { next }).to eq([])
end
it "errors on an invalid return type from a block" do
expect { electron_builder.versions_from_content(electron_builder_yaml) { 123 } }
.to raise_error(TypeError, Homebrew::Livecheck::Strategy::INVALID_BLOCK_RETURN_VALUE_MSG)
it "returns default match_data when content is blank" do
expect(electron_builder.find_versions(url: http_url, provided_content: "") { "1.2.3" })
.to eq({ matches: {}, regex: nil, url: http_url, cached: true })
end
end
end

156
Library/Homebrew/test/livecheck/strategy/yaml_spec.rb Normal file
View File

@ -0,0 +1,156 @@
# typed: false
# frozen_string_literal: true
require "livecheck/strategy"
describe Homebrew::Livecheck::Strategy::Yaml do
subject(:yaml) { described_class }
let(:http_url) { "https://brew.sh/blog/" }
let(:non_http_url) { "ftp://brew.sh/" }
let(:regex) { /^v?(\d+(?:\.\d+)+)$/i }
let(:content) {
<<~EOS
versions:
- version: 1.1.2
- version: 1.1.2b
- version: 1.1.2a
- version: 1.1.1
- version: 1.1.0
- version: 1.1.0-rc3
- version: 1.1.0-rc2
- version: 1.1.0-rc1
- version: 1.0.x-last
- version: 1.0.3
- version: 1.0.3-rc3
- version: 1.0.3-rc2
- version: 1.0.3-rc1
- version: 1.0.2
- version: 1.0.2-rc1
- version: 1.0.1
- version: 1.0.1-rc1
- version: 1.0.0
- version: 1.0.0-rc1
- other: version is omitted from this object for testing
EOS
}
let(:content_simple) { "version: 1.2.3" }
# This should produce a `Psych::SyntaxError` (`did not find expected comment
# or line break while scanning a block scalar`)
let(:content_invalid) { ">~" }
let(:content_matches) { ["1.1.2", "1.1.1", "1.1.0", "1.0.3", "1.0.2", "1.0.1", "1.0.0"] }
let(:content_simple_matches) { ["1.2.3"] }
let(:find_versions_return_hash) {
{
matches: {
"1.1.2" => Version.new("1.1.2"),
"1.1.1" => Version.new("1.1.1"),
"1.1.0" => Version.new("1.1.0"),
"1.0.3" => Version.new("1.0.3"),
"1.0.2" => Version.new("1.0.2"),
"1.0.1" => Version.new("1.0.1"),
"1.0.0" => Version.new("1.0.0"),
},
regex: regex,
url: http_url,
}
}
let(:find_versions_cached_return_hash) {
find_versions_return_hash.merge({ cached: true })
}
describe "::match?" do
it "returns true for an HTTP URL" do
expect(yaml.match?(http_url)).to be true
end
it "returns false for a non-HTTP URL" do
expect(yaml.match?(non_http_url)).to be false
end
end
describe "::parse_yaml" do
it "returns an object when given valid content" do
expect(yaml.parse_yaml(content_simple)).to be_an_instance_of(Hash)
end
end
describe "::versions_from_content" do
it "returns an empty array when given a block but content is blank" do
expect(yaml.versions_from_content("", regex) { "1.2.3" }).to eq([])
end
it "errors if provided content is not valid YAML" do
expect { yaml.versions_from_content(content_invalid) { [] } }
.to raise_error(RuntimeError, "Content could not be parsed as YAML.")
end
it "returns an array of version strings when given content and a block" do
# Returning a string from block
expect(yaml.versions_from_content(content_simple) { |yaml| yaml["version"] }).to eq(content_simple_matches)
expect(yaml.versions_from_content(content_simple, regex) do |yaml|
yaml["version"][regex, 1]
end).to eq(content_simple_matches)
# Returning an array of strings from block
expect(yaml.versions_from_content(content, regex) do |yaml, regex|
yaml["versions"].select { |item| item["version"]&.match?(regex) }
.map { |item| item["version"][regex, 1] }
end).to eq(content_matches)
end
it "allows a nil return from a block" do
expect(yaml.versions_from_content(content_simple, regex) { next }).to eq([])
end
it "errors if a block uses two arguments but a regex is not given" do
expect { yaml.versions_from_content(content_simple) { |yaml, regex| yaml["version"][regex, 1] } }
.to raise_error("Two arguments found in `strategy` block but no regex provided.")
end
it "errors on an invalid return type from a block" do
expect { yaml.versions_from_content(content_simple, regex) { 123 } }
.to raise_error(TypeError, Homebrew::Livecheck::Strategy::INVALID_BLOCK_RETURN_VALUE_MSG)
end
end
describe "::find_versions?" do
it "finds versions in provided_content using a block" do
expect(yaml.find_versions(url: http_url, regex: regex, provided_content: content) do |yaml, regex|
yaml["versions"].select { |item| item["version"]&.match?(regex) }
.map { |item| item["version"][regex, 1] }
end).to eq(find_versions_cached_return_hash)
# NOTE: A regex should be provided using the `#regex` method in a
# `livecheck` block but we're using a regex literal in the `strategy`
# block here simply to ensure this method works as expected when a
# regex isn't provided.
expect(yaml.find_versions(url: http_url, provided_content: content) do |yaml|
regex = /^v?(\d+(?:\.\d+)+)$/i.freeze
yaml["versions"].select { |item| item["version"]&.match?(regex) }
.map { |item| item["version"][regex, 1] }
end).to eq(find_versions_cached_return_hash.merge({ regex: nil }))
end
it "errors if a block is not provided" do
expect { yaml.find_versions(url: http_url, provided_content: content) }
.to raise_error(ArgumentError, "Yaml requires a `strategy` block")
end
it "returns default match_data when url is blank" do
expect(yaml.find_versions(url: "") { "1.2.3" })
.to eq({ matches: {}, regex: nil, url: "" })
end
it "returns default match_data when content is blank" do
expect(yaml.find_versions(url: http_url, provided_content: "") { "1.2.3" })
.to eq({ matches: {}, regex: nil, url: http_url, cached: true })
end
end
end

15
docs/Brew-Livecheck.md
View File

@ -187,6 +187,21 @@ end
For more information on how to work with an `REXML::Document` object, please refer to the [`REXML::Document`](https://ruby.github.io/rexml/REXML/Document.html) and [`REXML::Element`](https://ruby.github.io/rexml/REXML/Element.html) documentation.
#### `Yaml` `strategy` block
A `strategy` block for `Yaml` receives parsed YAML data and, if provided, a regex. Borrowing the `Json` example, if we have an object containing an array of objects with a `version` string, we can select only the members that match the regex and isolate the relevant version text as follows:
```ruby
livecheck do
url "https://www.example.com/example.yaml"
regex(/^v?(\d+(?:\.\d+)+)$/i)
strategy :yaml do |yaml, regex|
yaml["versions"].select { |item| item["version"]&.match?(regex) }
.map { |item| item["version"][regex, 1] }
end
end
```
### `skip`
Livecheck automatically skips some formulae/casks for a number of reasons (deprecated, disabled, discontinued, etc.). However, on rare occasions we need to use a `livecheck` block to do a manual skip. The `skip` method takes a string containing a very brief reason for skipping.