admin/brew
1
0
Fork 0
Code Issues Packages Projects Releases 6 Wiki Activity

Merge pull request #15260 from SMillerDev/feature/livecheck/github_release_strategy

Browse Source 
feat: add github_release strategy
This commit is contained in:
Sean Molenaar 2023-05-17 11:09:57 +02:00 committed by GitHub
commit 49b4a7928e
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
7 changed files with 374 additions and 67 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
Library/Homebrew/.rubocop.yml
View File

@ -43,6 +43,7 @@ Style/Documentation:
- livecheck/strategy/extract_plist.rb
- livecheck/strategy/git.rb
- livecheck/strategy/github_latest.rb
- livecheck/strategy/github_releases.rb
- livecheck/strategy/gnome.rb
- livecheck/strategy/gnu.rb
- livecheck/strategy/hackage.rb

1
Library/Homebrew/livecheck/strategy.rb
View File

@ -270,6 +270,7 @@ require_relative "strategy/electron_builder"
require_relative "strategy/extract_plist"
require_relative "strategy/git"
require_relative "strategy/github_latest"
require_relative "strategy/github_releases"
require_relative "strategy/gnome"
require_relative "strategy/gnu"
require_relative "strategy/hackage"

78
Library/Homebrew/livecheck/strategy/github_latest.rb
View File

@ -16,14 +16,15 @@ module Homebrew
#
# {GithubLatest} should only be used when the upstream repository has a
# "latest" release for a suitable version and the strategy is necessary
# or appropriate (e.g. {Git} returns an unreleased version or the
# `stable` URL is a release asset). The strategy can only be applied by
# using `strategy :github_latest` in a `livecheck` block.
# or appropriate (e.g. the formula/cask uses a release asset or the
# {Git} strategy returns an unreleased version). The strategy can only
# be applied by using `strategy :github_latest` in a `livecheck` block.
#
# The default regex identifies versions like `1.2.3`/`v1.2.3` in the
# release's tag name. This is a common tag format but a modified regex
# can be provided in a `livecheck` block to override the default if a
# repository uses a different format (e.g. `1.2.3d`, `1.2.3-4`, etc.).
# The default regex identifies versions like `1.2.3`/`v1.2.3` in a
# release's tag or title. This is a common tag format but a modified
# regex can be provided in a `livecheck` block to override the default
# if a repository uses a different format (e.g. `1.2.3d`, `1.2.3-4`,
# etc.).
#
# @api public
class GithubLatest
@ -34,28 +35,13 @@ module Homebrew
# `strategy :github_latest` in a `livecheck` block.
PRIORITY = 0
# The `Regexp` used to determine if the strategy applies to the URL.
URL_MATCH_REGEX = %r{
^https?://github\.com
/(?:downloads/)?(?<username>[^/]+) # The GitHub username
/(?<repository>[^/]+) # The GitHub repository name
}ix.freeze
# The default regex used to identify a version from a tag when a regex
# isn't provided.
DEFAULT_REGEX = /v?(\d+(?:\.\d+)+)/i.freeze
# Keys in the release JSON that could contain the version.
# Tag name first since that is closer to other livechecks.
VERSION_KEYS = ["tag_name", "name"].freeze
# Whether the strategy can be applied to the provided URL.
#
# @param url [String] the URL to match against
# @return [Boolean]
sig { params(url: String).returns(T::Boolean) }
def self.match?(url)
URL_MATCH_REGEX.match?(url)
GithubReleases.match?(url)
end
# Extracts information from a provided URL and uses it to generate
@ -69,7 +55,7 @@ module Homebrew
def self.generate_input_values(url)
values = {}
match = url.sub(/\.git$/i, "").match(URL_MATCH_REGEX)
match = url.delete_suffix(".git").match(GithubReleases::URL_MATCH_REGEX)
return values if match.blank?
values[:url] = "https://api.github.com/repos/#{match[:username]}/#{match[:repository]}/releases/latest"
@ -79,46 +65,6 @@ module Homebrew
values
end
# Uses a regex to match the version from release JSON or, if a block is
# provided, passes the JSON to the block to handle matching. With
# either approach, an array of unique matches is returned.
#
# @param content [Array, Hash] list of releases or a single release
# @param regex [Regexp] a regex used for matching versions in the content
# @param block [Proc, nil] a block to match the content
# @return [Array]
sig {
params(
content: T.any(T::Array[T::Hash[String, T.untyped]], T::Hash[String, T.untyped]),
regex: Regexp,
block: T.nilable(Proc),
).returns(T::Array[String])
}
def self.versions_from_content(content, regex, &block)
if block.present?
block_return_value = if regex.present?
yield(content, regex)
else
yield(content)
end
return Strategy.handle_block_return(block_return_value)
end
content = [content] unless content.is_a?(Array)
content.reject(&:blank?).map do |release|
next if release["draft"] || release["prerelease"]
value = T.let(nil, T.untyped)
VERSION_KEYS.find do |key|
match = release[key]&.match(regex)
next if match.blank?
value = match[1]
end
value
end.compact.uniq
end
# Generates the GitHub API URL for the repository's "latest" release
# and identifies the version from the JSON response.
#
@ -133,7 +79,7 @@ module Homebrew
block: T.nilable(Proc),
).returns(T::Hash[Symbol, T.untyped])
}
def self.find_versions(url:, regex: DEFAULT_REGEX, **_unused, &block)
def self.find_versions(url:, regex: GithubReleases::DEFAULT_REGEX, **_unused, &block)
match_data = { matches: {}, regex: regex, url: url }
generated = generate_input_values(url)
@ -142,7 +88,7 @@ module Homebrew
match_data[:url] = generated[:url]
release = GitHub.get_latest_release(generated[:username], generated[:repository])
versions_from_content(release, regex, &block).each do |match_text|
GithubReleases.versions_from_content(release, regex, &block).each do |match_text|
match_data[:matches][match_text] = Version.new(match_text)
end

155
Library/Homebrew/livecheck/strategy/github_releases.rb Normal file
View File

@ -0,0 +1,155 @@
# typed: true
# frozen_string_literal: true
module Homebrew
module Livecheck
module Strategy
# The {GithubReleases} strategy identifies versions of software at
# github.com by checking a repository's recent releases using the
# GitHub API.
#
# GitHub URLs take a few different formats:
#
# * `https://github.com/example/example/releases/download/1.2.3/example-1.2.3.tar.gz`
# * `https://github.com/example/example/archive/v1.2.3.tar.gz`
# * `https://github.com/downloads/example/example/example-1.2.3.tar.gz`
#
# {GithubReleases} should only be used when the upstream repository has
# releases for suitable versions and the strategy is necessary or
# appropriate (e.g. the formula/cask uses a release asset and the
# {GithubLatest} strategy isn't sufficient to identify the newest version.
# The strategy can only be applied by using `strategy :github_releases`
# in a `livecheck` block.
#
# The default regex identifies versions like `1.2.3`/`v1.2.3` in each
# release's tag or title. This is a common tag format but a modified
# regex can be provided in a `livecheck` block to override the default
# if a repository uses a different format (e.g. `1.2.3d`, `1.2.3-4`,
# etc.).
#
# @api public
class GithubReleases
NICE_NAME = "GitHub - Releases"
# A priority of zero causes livecheck to skip the strategy. We do this
# for {GithubReleases} so we can selectively apply the strategy using
# `strategy :github_releases` in a `livecheck` block.
PRIORITY = 0
# The `Regexp` used to determine if the strategy applies to the URL.
URL_MATCH_REGEX = %r{
^https?://github\.com
/(?:downloads/)?(?<username>[^/]+) # The GitHub username
/(?<repository>[^/]+) # The GitHub repository name
}ix.freeze
# The default regex used to identify a version from a tag when a regex
# isn't provided.
DEFAULT_REGEX = /v?(\d+(?:\.\d+)+)/i.freeze
# Keys in the release JSON that could contain the version.
# The tag name is checked first, to better align with the {Git}
# strategy.
VERSION_KEYS = ["tag_name", "name"].freeze
# Whether the strategy can be applied to the provided URL.
#
# @param url [String] the URL to match against
# @return [Boolean]
sig { params(url: String).returns(T::Boolean) }
def self.match?(url)
URL_MATCH_REGEX.match?(url)
end
# Extracts information from a provided URL and uses it to generate
# various input values used by the strategy to check for new versions.
# Some of these values act as defaults and can be overridden in a
# `livecheck` block.
#
# @param url [String] the URL used to generate values
# @return [Hash]
sig { params(url: String).returns(T::Hash[Symbol, T.untyped]) }
def self.generate_input_values(url)
values = {}
match = url.delete_suffix(".git").match(URL_MATCH_REGEX)
return values if match.blank?
values[:url] = "#{GitHub::API_URL}/repos/#{match[:username]}/#{match[:repository]}/releases"
values[:username] = match[:username]
values[:repository] = match[:repository]
values
end
# Uses a regex to match versions from release JSON or, if a block is
# provided, passes the JSON to the block to handle matching. With
# either approach, an array of unique matches is returned.
#
# @param content [Array, Hash] array of releases or a single release
# @param regex [Regexp] a regex used for matching versions in the
# content
# @param block [Proc, nil] a block to match the content
# @return [Array]
sig {
params(
content: T.any(T::Array[T::Hash[String, T.untyped]], T::Hash[String, T.untyped]),
regex: Regexp,
block: T.nilable(Proc),
).returns(T::Array[String])
}
def self.versions_from_content(content, regex, &block)
if block.present?
block_return_value = yield(content, regex)
return Strategy.handle_block_return(block_return_value)
end
content = [content] unless content.is_a?(Array)
content.reject(&:blank?).map do |release|
next if release["draft"] || release["prerelease"]
value = T.let(nil, T.untyped)
VERSION_KEYS.find do |key|
match = release[key]&.match(regex)
next if match.blank?
value = match[1]
end
value
end.compact.uniq
end
# Generates the GitHub API URL for the repository's recent releases
# and identifies versions from the JSON response.
#
# @param url [String] the URL of the content to check
# @param regex [Regexp] a regex used for matching versions in content
# @return [Hash]
sig {
params(
url: String,
regex: Regexp,
_unused: T.nilable(T::Hash[Symbol, T.untyped]),
block: T.nilable(Proc),
).returns(T::Hash[Symbol, T.untyped])
}
def self.find_versions(url:, regex: DEFAULT_REGEX, **_unused, &block)
match_data = { matches: {}, regex: regex, url: url }
generated = generate_input_values(url)
return match_data if generated.blank?
match_data[:url] = generated[:url]
releases = GitHub::API.open_rest(generated[:url])
versions_from_content(releases, regex, &block).each do |match_text|
match_data[:matches][match_text] = Version.new(match_text)
end
match_data
end
end
end
GitHubReleases = Homebrew::Livecheck::Strategy::GithubReleases
end
end

1
Library/Homebrew/test/livecheck/strategy/github_latest_spec.rb
View File

@ -1,5 +1,6 @@
# frozen_string_literal: true
require "livecheck/strategy/github_releases"
require "livecheck/strategy/github_latest"
describe Homebrew::Livecheck::Strategy::GithubLatest do

157
Library/Homebrew/test/livecheck/strategy/github_releases_spec.rb Normal file
View File

@ -0,0 +1,157 @@
# frozen_string_literal: true
require "livecheck/strategy"
describe Homebrew::Livecheck::Strategy::GithubReleases do
subject(:github_releases) { described_class }
let(:github_urls) do
{
release_artifact: "https://github.com/abc/def/releases/download/1.2.3/ghi-1.2.3.tar.gz",
tag_archive: "https://github.com/abc/def/archive/v1.2.3.tar.gz",
repository_upload: "https://github.com/downloads/abc/def/ghi-1.2.3.tar.gz",
}
end
let(:non_github_url) { "https://brew.sh/test" }
let(:regex) { github_releases::DEFAULT_REGEX }
let(:generated) do
{
url: "https://api.github.com/repos/abc/def/releases",
username: "abc",
repository: "def",
}
end
# For the sake of brevity, this is a limited subset of the information found
# in release objects in a response from the GitHub API. Some of these objects
# are somewhat representative of real world scenarios but others are
# contrived examples for the sake of exercising code paths.
let(:content) do
<<~EOS
[
{
"tag_name": "v1.2.3",
"name": "v1.2.3",
"draft": false,
"prerelease": false
},
{
"tag_name": "no-version-tag-also",
"name": "1.2.2",
"draft": false,
"prerelease": false
},
{
"tag_name": "1.2.1",
"name": "No version title",
"draft": false,
"prerelease": false
},
{
"tag_name": "no-version-tag",
"name": "No version title",
"draft": false,
"prerelease": false
},
{
"tag_name": "v1.1.2",
"name": "v1.1.2",
"draft": false,
"prerelease": true
},
{
"tag_name": "v1.1.1",
"name": "v1.1.1",
"draft": true,
"prerelease": false
},
{
"tag_name": "v1.1.0",
"name": "v1.1.0",
"draft": true,
"prerelease": true
},
{
"other": "something-else"
}
]
EOS
end
let(:json) { JSON.parse(content) }
let(:matches) { ["1.2.3", "1.2.2", "1.2.1"] }
describe "::match?" do
it "returns true for a GitHub release artifact URL" do
expect(github_releases.match?(github_urls[:release_artifact])).to be true
end
it "returns true for a GitHub tag archive URL" do
expect(github_releases.match?(github_urls[:tag_archive])).to be true
end
it "returns true for a GitHub repository upload URL" do
expect(github_releases.match?(github_urls[:repository_upload])).to be true
end
it "returns false for a non-GitHub URL" do
expect(github_releases.match?(non_github_url)).to be false
end
end
describe "::generate_input_values" do
it "returns a hash containing a url and regex for a GitHub release artifact URL" do
expect(github_releases.generate_input_values(github_urls[:release_artifact])).to eq(generated)
end
it "returns a hash containing a url and regex for a GitHub tag archive URL" do
expect(github_releases.generate_input_values(github_urls[:tag_archive])).to eq(generated)
end
it "returns a hash containing a url and regex for a GitHub repository upload URL" do
expect(github_releases.generate_input_values(github_urls[:repository_upload])).to eq(generated)
end
it "returns an empty hash for a non-Github URL" do
expect(github_releases.generate_input_values(non_github_url)).to eq({})
end
end
describe "::versions_from_content" do
it "returns an empty array if content is blank" do
expect(github_releases.versions_from_content({}, regex)).to eq([])
end
it "returns an array of version strings when given content" do
expect(github_releases.versions_from_content(json, regex)).to eq(matches)
end
it "returns an array of version strings when given content and a block" do
# Returning a string from block
expect(github_releases.versions_from_content(json, regex) { "1.2.3" }).to eq(["1.2.3"])
# Returning an array of strings from block
expect(github_releases.versions_from_content(json, regex) do |json, regex|
json.map do |release|
next if release["draft"] || release["prerelease"]
match = release["tag_name"]&.match(regex)
next if match.blank?
match[1]
end
end).to eq(["1.2.3", "1.2.1"])
end
it "allows a nil return from a block" do
expect(github_releases.versions_from_content(json, regex) { next }).to eq([])
end
it "errors on an invalid return type from a block" do
expect { github_releases.versions_from_content(json, regex) { 123 } }
.to raise_error(TypeError, Homebrew::Livecheck::Strategy::INVALID_BLOCK_RETURN_VALUE_MSG)
end
end
end

48
docs/Brew-Livecheck.md
View File

@ -27,7 +27,7 @@ This can be accomplished by adding a `livecheck` block to the formula/cask/resou
* **Only use `strategy` when it's necessary**. For example, if livecheck is already using the `Git` strategy for a URL, it's not necessary to use `strategy :git`. However, if `Git` applies to a URL but we need to use `PageMatch`, it's necessary to specify `strategy :page_match`.
* **Only use the `GithubLatest` strategy when it's necessary and correct**. GitHub rate-limits requests so we try to minimize our use of this strategy to avoid hitting the rate limit on CI or when using `brew livecheck --tap` on large taps (e.g. `homebrew/core`). The `Git` strategy is often sufficient and we only need to use `GithubLatest` when the "latest" release is different than the newest version from the tags.
* **Only use the `GithubLatest` and `GithubReleases` strategies when they are necessary and correct**. GitHub rate limits API requests, so we only use these strategies when `Git` isn't sufficient or appropriate. `GithubLatest` should only be used if the upstream repository has a "latest" release for a suitable version and either the formula/cask uses a release asset or the `Git` strategy can't correctly identify the latest release version. `GithubReleases` should only be used if the upstream repository uses releases and both the `Git` and `GithubLatest` strategies aren't suitable.
### URL guidelines
@ -141,6 +141,52 @@ livecheck do
end
```
#### `GithubLatest` `strategy` block
A `strategy` block for `GithubLatest` receives the parsed JSON data from the GitHub API for a repository's "latest" release, along with a regex. When a regex is not provided in a `livecheck` block, the strategy's default regex is passed into the `strategy` block instead.
By default, the strategy matches version text in the release's tag or title but a `strategy` block can be used to check any of the fields in the release JSON. The logic in the following `strategy` block is similar to the default behavior but only checks the release tag instead, for the sake of demonstration:
```ruby
livecheck do
url :stable
regex(/^example[._-]v?(\d+(?:\.\d+)+)$/i)
strategy :github_latest do |json, regex|
match = json["tag_name"]&.match(regex)
next if match.blank?
match[1]
end
end
```
You can find more information on the response JSON from this API endpoint in the related [GitHub REST API documentation](https://docs.github.com/en/rest/releases/releases?apiVersion=latest#get-the-latest-release).
#### `GithubReleases` `strategy` block
A `strategy` block for `GithubReleases` receives the parsed JSON data from the GitHub API for a repository's most recent releases, along with a regex. When a regex is not provided in a `livecheck` block, the strategy's default regex is passed into the `strategy` block instead.
By default, the strategy matches version text in each release's tag or title but a `strategy` block can be used to check any of the fields in the release JSON. The logic in the following `strategy` block is similar to the default behavior but only checks the release tag instead, for the sake of demonstration:
```ruby
livecheck do
url :stable
regex(/^example[._-]v?(\d+(?:\.\d+)+)$/i)
strategy :github_releases do |json, regex|
json.map do |release|
next if release["draft"] || release["prerelease"]
match = json["tag_name"]&.match(regex)
next if match.blank?
match[1]
end
end
end
```
You can find more information on the response JSON from this API endpoint in the related [GitHub REST API documentation](https://docs.github.com/en/rest/releases/releases?apiVersion=latest#list-releases).
#### `Json` `strategy` block
A `strategy` block for `Json` receives parsed JSON data and, if provided, a regex. For example, if we have an object containing an array of objects with a `version` string, we can select only the members that match the regex and isolate the relevant version text as follows: