On Linux, we occasionally see `EMFILE` ("too many open files") errors
especially when installing a large formula like `llvm`. Currently, this
can be reliably reproduced in a Homebrew/brew GitHub codespace (where
`ulimit -n` seems to be 1024 by default) with `brew install geeqie`,
with the following error message:
Error: Too many open files @ rb_sysopen - /home/linuxbrew/.linuxbrew/Cellar/llvm/20.1.8/bin/tblgen-lsp-server
The reason is that each instance of `PatchELF::Patcher` keeps the ELF
file open. We prepend the `ELFShim` module to the `Pathname` class and
cache the patcher as an instance variable, which means that the ELF file
remains open so long as the `Pathname` instance is still alive even if
we don't need to access the ELF metadata anymore. When performing
certain checks (e.g., linkage), we also store these `Pathname`
instances, so the number of open file descriptors simply keeps
increasing.
We can fix that by not caching the patcher and only use it when
necessary. We create a patcher instance whenever we need to read or
write ELF metadata, and reading of metadata is consolidated into the
existing `ELFShim::Metadata` class so that we don't repeatedly create
patcher instances.
A fix for a file descriptor leak issue in patchelf.rb has been submitted
at https://github.com/david942j/patchelf.rb/pull/48. Together with that,
this fixes#19177, #19866, #20223, #20302.
We have multiple formulae (e.g. `ola`, `openvino`, `opencv`, `or-tools`,
`pytorch`) that seem to be broken by `patchelf.rb` on x86_64 Linux.[^1] The
common thread seems to be the presence of a `protodesc_cold` section in
the ELF header.
Let's avoid breaking these bottles by skipping relocation when a binary
file has a `protodesc_cold` section. It will probably hurt
relocatability of these bottles, but that's better shipping broken
binaries.
[^1]: https://github.com/Homebrew/homebrew-core/pull/210860#issue-2918569212
Will fix or at least partly address:
```
/home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/os/linux/elf.rb:225: warning: The class Pathname reached 8 shape variations, instance variables accesses will be slower and memory usage increased.
35
It is recommended to define instance variables in a consistent order, for instance by eagerly defining them all in the #initialize method.
```
Some formulae include these files, and they can't always be removed.
However, they can cause spurious linkage failures, so let's skip them
when checking for linkage. See, for example, faust at
Homebrew/homebrew-core#191308.
- Previously I thought that comments were fine to discourage people from
wasting their time trying to bump things that used `undef` that Sorbet
didn't support. But RuboCop is better at this since it'll complain if
the comments are unnecessary.
- Suggested in https://github.com/Homebrew/brew/pull/18018#issuecomment-2283369501.
- I've gone for a mixture of `rubocop:disable` for the files that can't
be `typed: strict` (use of undef, required before everything else, etc)
and `rubocop:todo` for everything else that should be tried to make
strictly typed. There's no functional difference between the two as
`rubocop:todo` is `rubocop:disable` with a different name.
- And I entirely disabled the cop for the docs/ directory since
`typed: strict` isn't going to gain us anything for some Markdown
linting config files.
- This means that now it's easier to track what needs to be done rather
than relying on checklists of files in our big Sorbet issue:
```shell
$ git grep 'typed: true # rubocop:todo Sorbet/StrictSigil' | wc -l
268
```
- And this is confirmed working for new files:
```shell
$ git status
On branch use-rubocop-for-sorbet-strict-sigils
Untracked files:
(use "git add <file>..." to include in what will be committed)
Library/Homebrew/bad.rb
Library/Homebrew/good.rb
nothing added to commit but untracked files present (use "git add" to track)
$ brew style
Offenses:
bad.rb:1:1: C: Sorbet/StrictSigil: Sorbet sigil should be at least strict got true.
^^^^^^^^^^^^^
1340 files inspected, 1 offense detected
```
Fixes#13762.
There's still a bug in the GCC linkage check, but I'll need a bit more
time to work on a fix. This at least makes sure `brew doctor` will not
return the error in the issue linked above.
An executable that links against @rpath-prefixed dylibs must include at least
one runtime path. This will prevent issues like the one resolved in #91485.
Caveats:
1. This won't find executables that have only recursive dylib dependencies with
@rpath prefixes.
2. This makes no attempt to resolve @rpath, @executable_path or @loader_path
dylibs, or to verify the correctness of any LC_RPATH entries, or to
otherwise simulate dlopen logic.
3. Weakly-linked dylibs are still excluded from the search for broken linkage.
The scope is narrow in order to focus on this particular problem. It is meant
only as a sanity check.
refines PatchELF #runpath, #rpath #soname #interpreter
to return nil.
let Brew hard exits on PatchELF::PatchError.
Co-authored-by: Shaun Jackman <sjackman@gmail.com>
Co-authored-by: Mike McQuaid <mike@mikemcquaid.com>
Having HOMEBREW_PATCHELF_RB set in the ENV,
will conditionally install patchelf.rb gem,
use patchelf.rb in the above mentioned methods.
The installed vendored gems are listed in .gitignore
to maintain a clean state.
Some elf files (e.g. created by rust compiler) have INTERP header despite
their magic header denotes shared object instead of executable.
We should relocate the interpreter elf files as long as they have INTERP header.
This should fix the broken bottles for rust based formulae.
Some elf files like unittest files or memory dumps may not be completely
readable by readelf.
Readelf will fail after the following message:
readelf: Warning: possibly corrupt ELF header - it has a non-zero program header offset, but no program headers
This patches avoid these files when there is a non zero offset but no
program headers
