| 
									
										
										
										
											2020-10-10 14:16:11 +02:00
										 |  |  | # typed: true | 
					
						
							| 
									
										
										
										
											2020-08-04 10:07:57 -07:00
										 |  |  | # frozen_string_literal: true | 
					
						
							|  |  |  | 
 | 
					
						
							| 
									
										
										
										
											2020-10-10 15:23:03 +02:00
										 |  |  | require "utils/curl" | 
					
						
							| 
									
										
										
										
											2020-08-04 10:07:57 -07:00
										 |  |  | require "utils/github" | 
					
						
							|  |  |  | 
 | 
					
						
							| 
									
										
										
										
											2020-08-26 09:16:05 +02:00
										 |  |  | # Helper module for updating SPDX license data. | 
					
						
							|  |  |  | # | 
					
						
							|  |  |  | # @api private | 
					
						
							| 
									
										
										
										
											2020-08-04 10:07:57 -07:00
										 |  |  | module SPDX | 
					
						
							| 
									
										
										
										
											2020-10-10 15:33:15 +02:00
										 |  |  |   include Utils::Curl | 
					
						
							| 
									
										
										
										
											2020-10-10 15:23:03 +02:00
										 |  |  |   extend Utils::Curl | 
					
						
							|  |  |  | 
 | 
					
						
							| 
									
										
										
										
											2020-08-04 10:07:57 -07:00
										 |  |  |   module_function | 
					
						
							|  |  |  | 
 | 
					
						
							| 
									
										
										
										
											2020-08-18 10:56:54 -04:00
										 |  |  |   DATA_PATH = (HOMEBREW_DATA_PATH/"spdx").freeze | 
					
						
							| 
									
										
										
										
											2020-08-04 10:07:57 -07:00
										 |  |  |   API_URL = "https://api.github.com/repos/spdx/license-list-data/releases/latest" | 
					
						
							| 
									
										
										
										
											2020-10-22 10:01:40 -04:00
										 |  |  |   ALLOWED_LICENSE_SYMBOLS = [ | 
					
						
							|  |  |  |     :public_domain, | 
					
						
							|  |  |  |     :cannot_represent, | 
					
						
							|  |  |  |   ].freeze | 
					
						
							| 
									
										
										
										
											2020-08-04 10:07:57 -07:00
										 |  |  | 
 | 
					
						
							| 
									
										
										
										
											2020-08-18 10:56:54 -04:00
										 |  |  |   def license_data | 
					
						
							|  |  |  |     @license_data ||= JSON.parse (DATA_PATH/"spdx_licenses.json").read | 
					
						
							|  |  |  |   end | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  |   def exception_data | 
					
						
							|  |  |  |     @exception_data ||= JSON.parse (DATA_PATH/"spdx_exceptions.json").read | 
					
						
							| 
									
										
										
										
											2020-08-04 10:07:57 -07:00
										 |  |  |   end | 
					
						
							|  |  |  | 
 | 
					
						
							| 
									
										
										
										
											2020-08-10 11:32:05 -07:00
										 |  |  |   def latest_tag | 
					
						
							| 
									
										
										
										
											2021-02-17 23:22:26 +05:30
										 |  |  |     @latest_tag ||= GitHub::API.open_rest(API_URL)["tag_name"] | 
					
						
							| 
									
										
										
										
											2020-08-10 11:32:05 -07:00
										 |  |  |   end | 
					
						
							|  |  |  | 
 | 
					
						
							| 
									
										
										
										
											2020-08-18 10:56:54 -04:00
										 |  |  |   def download_latest_license_data!(to: DATA_PATH) | 
					
						
							|  |  |  |     data_url = "https://raw.githubusercontent.com/spdx/license-list-data/#{latest_tag}/json/" | 
					
						
							| 
									
										
											  
											
												#curl_download: default try_partial to false
When its `try_partial` argument is `true`, `#curl_download` makes a
`HEAD` request before downloading the file using `#curl`. Currently
`try_partial` defaults to `true`, so any `#curl_download` call that
doesn't explicitly specify `try_partial: false` will make a `HEAD`
request first. This can potentially involve several requests if the
URL redirects, so it can be a bit of unnecessary overhead when a
partial download isn't needed.
Partial downloads are generally only useful when we're working with
larger files, however there's currently only one place in brew where
`#curl_download` is used and this is the case:
`CurlDownloadStrategy`. The other `#curl_download` calls are fetching
smaller [text] files and don't need to support partial downloads.
This commit changes the default `try_partial` value to `false`,
making partial downloads opt-in rather than opt-out.
We want `try_partial` to continue to default to `true` in
`CurlDownloadStrategy` and there are various ways to accomplish this.
In this commit, I've chosen to update its `#initialize` method to
accept a `try_partial` argument that defaults to `true`, as this
value can also be used in classes that inherit from
`CurlDownloadStrategy` (e.g., `HomebrewCurlDownloadStrategy`). This
instance variable is passed to `#curl_download` in related methods,
effectively maintaining the previous `try_partial: true` value, while
also allowing this value to be overridden when necessary.
Other uses of `#curl_download` in brew are
`Formulary::FromUrlLoader#load_file` and
`Cask::CaskLoader::FromURILoader#load`, which did not provide a
`try_partial` argument but should have been using
`try_partial: false`. With the `try_partial: false` default in this
commit, these calls are now fine without a `try_partial` argument.
The only other use of `#curl_download` in brew is
`SPDX#download_latest_license_data!`. These calls were previously
using `try_partial: false` but we can now omit this argument with
the new `false` default (aligning with the above).
											
										 
											2022-04-22 12:05:14 -04:00
										 |  |  |     curl_download("#{data_url}licenses.json", to: to/"spdx_licenses.json") | 
					
						
							|  |  |  |     curl_download("#{data_url}exceptions.json", to: to/"spdx_exceptions.json") | 
					
						
							| 
									
										
										
										
											2020-08-18 10:56:54 -04:00
										 |  |  |   end | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  |   def parse_license_expression(license_expression) | 
					
						
							| 
									
										
										
										
											2020-12-28 13:34:07 +00:00
										 |  |  |     licenses = T.let([], T::Array[T.any(String, Symbol)]) | 
					
						
							|  |  |  |     exceptions = T.let([], T::Array[String]) | 
					
						
							| 
									
										
										
										
											2020-08-18 10:56:54 -04:00
										 |  |  | 
 | 
					
						
							|  |  |  |     case license_expression | 
					
						
							|  |  |  |     when String, Symbol | 
					
						
							|  |  |  |       licenses.push license_expression | 
					
						
							| 
									
										
										
										
											2020-08-19 10:25:02 -04:00
										 |  |  |     when Hash, Array | 
					
						
							|  |  |  |       if license_expression.is_a? Hash | 
					
						
							|  |  |  |         license_expression = license_expression.map do |key, value| | 
					
						
							|  |  |  |           if key.is_a? String | 
					
						
							|  |  |  |             licenses.push key | 
					
						
							|  |  |  |             exceptions.push value[:with] | 
					
						
							|  |  |  |             next | 
					
						
							|  |  |  |           end | 
					
						
							|  |  |  |           value | 
					
						
							|  |  |  |         end.compact | 
					
						
							| 
									
										
										
										
											2020-08-18 10:56:54 -04:00
										 |  |  |       end | 
					
						
							| 
									
										
										
										
											2020-08-19 10:25:02 -04:00
										 |  |  | 
 | 
					
						
							| 
									
										
										
										
											2020-08-18 10:56:54 -04:00
										 |  |  |       license_expression.each do |license| | 
					
						
							|  |  |  |         sub_license, sub_exception = parse_license_expression license | 
					
						
							|  |  |  |         licenses += sub_license | 
					
						
							|  |  |  |         exceptions += sub_exception | 
					
						
							|  |  |  |       end | 
					
						
							|  |  |  |     end | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  |     [licenses, exceptions] | 
					
						
							|  |  |  |   end | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  |   def valid_license?(license) | 
					
						
							| 
									
										
										
										
											2020-10-22 10:01:40 -04:00
										 |  |  |     return ALLOWED_LICENSE_SYMBOLS.include? license if license.is_a? Symbol | 
					
						
							| 
									
										
										
										
											2020-08-18 10:56:54 -04:00
										 |  |  | 
 | 
					
						
							|  |  |  |     license = license.delete_suffix "+" | 
					
						
							|  |  |  |     license_data["licenses"].any? { |spdx_license| spdx_license["licenseId"] == license } | 
					
						
							|  |  |  |   end | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  |   def deprecated_license?(license) | 
					
						
							| 
									
										
										
										
											2020-10-22 10:01:40 -04:00
										 |  |  |     return false if ALLOWED_LICENSE_SYMBOLS.include? license | 
					
						
							| 
									
										
										
										
											2020-08-18 10:56:54 -04:00
										 |  |  |     return false unless valid_license?(license) | 
					
						
							|  |  |  | 
 | 
					
						
							| 
									
										
										
										
											2021-02-23 16:28:24 -05:00
										 |  |  |     license = license.delete_suffix "+" | 
					
						
							| 
									
										
										
										
											2020-08-18 10:56:54 -04:00
										 |  |  |     license_data["licenses"].none? do |spdx_license| | 
					
						
							|  |  |  |       spdx_license["licenseId"] == license && !spdx_license["isDeprecatedLicenseId"] | 
					
						
							|  |  |  |     end | 
					
						
							|  |  |  |   end | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  |   def valid_license_exception?(exception) | 
					
						
							|  |  |  |     exception_data["exceptions"].any? do |spdx_exception| | 
					
						
							|  |  |  |       spdx_exception["licenseExceptionId"] == exception && !spdx_exception["isDeprecatedLicenseId"] | 
					
						
							|  |  |  |     end | 
					
						
							|  |  |  |   end | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  |   def license_expression_to_string(license_expression, bracket: false, hash_type: nil) | 
					
						
							|  |  |  |     case license_expression | 
					
						
							|  |  |  |     when String | 
					
						
							|  |  |  |       license_expression | 
					
						
							| 
									
										
										
										
											2020-10-22 10:01:40 -04:00
										 |  |  |     when Symbol | 
					
						
							|  |  |  |       license_expression.to_s.tr("_", " ").titleize | 
					
						
							| 
									
										
										
										
											2020-08-20 10:26:37 -04:00
										 |  |  |     when Hash | 
					
						
							| 
									
										
										
										
											2020-08-18 10:56:54 -04:00
										 |  |  |       expressions = [] | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  |       if license_expression.keys.length == 1
 | 
					
						
							|  |  |  |         hash_type = license_expression.keys.first | 
					
						
							|  |  |  |         if hash_type.is_a? String | 
					
						
							|  |  |  |           expressions.push "#{hash_type} with #{license_expression[hash_type][:with]}" | 
					
						
							|  |  |  |         else | 
					
						
							|  |  |  |           expressions += license_expression[hash_type].map do |license| | 
					
						
							|  |  |  |             license_expression_to_string license, bracket: true, hash_type: hash_type | 
					
						
							|  |  |  |           end | 
					
						
							|  |  |  |         end | 
					
						
							|  |  |  |       else | 
					
						
							|  |  |  |         bracket = false | 
					
						
							|  |  |  |         license_expression.each do |expression| | 
					
						
							| 
									
										
										
										
											2021-02-16 09:25:34 +00:00
										 |  |  |           expressions.push license_expression_to_string([expression].to_h, bracket: true) | 
					
						
							| 
									
										
										
										
											2020-08-18 10:56:54 -04:00
										 |  |  |         end | 
					
						
							|  |  |  |       end | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  |       operator = if hash_type == :any_of | 
					
						
							|  |  |  |         " or " | 
					
						
							|  |  |  |       else | 
					
						
							|  |  |  |         " and " | 
					
						
							|  |  |  |       end | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  |       if bracket | 
					
						
							|  |  |  |         "(#{expressions.join operator})" | 
					
						
							|  |  |  |       else | 
					
						
							|  |  |  |         expressions.join operator | 
					
						
							|  |  |  |       end | 
					
						
							|  |  |  |     end | 
					
						
							|  |  |  |   end | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  |   def license_version_info(license) | 
					
						
							| 
									
										
										
										
											2020-10-22 10:01:40 -04:00
										 |  |  |     return [license] if ALLOWED_LICENSE_SYMBOLS.include? license | 
					
						
							| 
									
										
										
										
											2020-08-18 10:56:54 -04:00
										 |  |  | 
 | 
					
						
							|  |  |  |     match = license.match(/-(?<version>[0-9.]+)(?:-.*?)??(?<or_later>\+|-only|-or-later)?$/) | 
					
						
							|  |  |  |     return [license] if match.blank? | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  |     license_name = license.split(match[0]).first | 
					
						
							|  |  |  |     or_later = match["or_later"].present? && %w[+ -or-later].include?(match["or_later"]) | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  |     # [name, version, later versions allowed?] | 
					
						
							|  |  |  |     # e.g. GPL-2.0-or-later --> ["GPL", "2.0", true] | 
					
						
							|  |  |  |     [license_name, match["version"], or_later] | 
					
						
							|  |  |  |   end | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  |   def licenses_forbid_installation?(license_expression, forbidden_licenses) | 
					
						
							|  |  |  |     case license_expression | 
					
						
							|  |  |  |     when String, Symbol | 
					
						
							|  |  |  |       forbidden_licenses_include? license_expression.to_s, forbidden_licenses | 
					
						
							| 
									
										
										
										
											2020-08-20 10:26:37 -04:00
										 |  |  |     when Hash | 
					
						
							| 
									
										
										
										
											2020-08-18 10:56:54 -04:00
										 |  |  |       key = license_expression.keys.first | 
					
						
							|  |  |  |       case key | 
					
						
							|  |  |  |       when :any_of | 
					
						
							|  |  |  |         license_expression[key].all? { |license| licenses_forbid_installation? license, forbidden_licenses } | 
					
						
							|  |  |  |       when :all_of | 
					
						
							|  |  |  |         license_expression[key].any? { |license| licenses_forbid_installation? license, forbidden_licenses } | 
					
						
							|  |  |  |       else | 
					
						
							|  |  |  |         forbidden_licenses_include? key, forbidden_licenses | 
					
						
							|  |  |  |       end | 
					
						
							|  |  |  |     end | 
					
						
							|  |  |  |   end | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  |   def forbidden_licenses_include?(license, forbidden_licenses) | 
					
						
							|  |  |  |     return true if forbidden_licenses.key? license | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  |     name, version, = license_version_info license | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  |     forbidden_licenses.each do |_, license_info| | 
					
						
							|  |  |  |       forbidden_name, forbidden_version, forbidden_or_later = *license_info | 
					
						
							|  |  |  |       next unless forbidden_name == name | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  |       return true if forbidden_or_later && forbidden_version <= version | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  |       return true if forbidden_version == version | 
					
						
							|  |  |  |     end | 
					
						
							|  |  |  |     false | 
					
						
							| 
									
										
										
										
											2020-08-04 10:07:57 -07:00
										 |  |  |   end | 
					
						
							|  |  |  | end |