From https://discord.com/channels/854719841091715092/1319027902322446373/1354877230853128254
`man ld.so` also covers this a bit - https://man7.org/linux/man-pages/man8/ld.so.8.html
An ELF file (executable or library) can have a "dynamic" section with info relevant to dynamic linking. This section has a bunch of entries, which are structured as (tag, value) pairs. You can see these for any file with `readelf -d`. Relevant ones to this conversation:
* `DT_NEEDED` means "I need this shared library to be loaded, too," i.e., it's how normal shared library dependencies are declared. It's basically like an implicit `dlopen()` call right when the object is loaded, before any code from it starts running. The value is usually a filename without any slashes, which is looked up in some standard places like /lib and /usr/lib. It could also be a full path.
* `DT_RPATH` and `DT_RUNPATH` mean "When you're looking up shared libraries, look here, too." The exact rules, and the precedence between these and other things like the `LD_LIBRARY_PATH` environment variable, are in `man ld.so`.
* `DT_SONAME` is the shared object name, i.e., "If the compile-time linker (`ld`) is linking something against this shared library, this is the value to put in the `DT_NEEDED` field." The point of this is so that you can have multiple things installed for _runtime_ use, e.g., both `libncurses.so.5` and `libncurses.so.6`: you make a symlink named `libncurses.so` pointing to one of those, and `ld -lncurses` finds that symlink and that's how you pick which version you want to _compile_ against. But the resulting binary specifically says which version of the library it's been built against, which it finds out because of the `DT_SONAME` field in the library.
The `DT_RPATH` and `DT_RUNPATH` fields allow you to use the special symbol `$ORIGIN`, which expands to the directory containing the ELF file, which is how relocatable shared library dependencies are possible. You can create a `bin/python3` file with a `DT_NEEDED` entry of `libpython3.14.so.1.0` and a `DT_RUNPATH` of `$ORIGIN/../lib`, and then if you run that python3 binary it will find libpython successfully.
On glibc (but not musl, which IMO is just a missing feature), `$ORIGIN` is also supported in `DT_NEEDED` entries.
Colloquially, "rpath" refers to either `DT_RPATH` or `DT_RUNPATH`. In theory, `DT_RPATH` is deprecated in favor of `DT_RUNPATH`, but there are actually cases where you specifically need `DT_RPATH`'s behavior (I have personally run into this). You set an rpath with `ld -rpath /whatever/path/you/like` (or `cc -Wl,-rpath,/whatever/path/you/like` which turns into that) ,which these days defaults to `DT_RUNPATH`; you can use the linker flag `--disable-new-dtags` (`-Wl,--disable-new-dtags`) to request `DT_RPATH` instead.
Re the conversation above about `DT_SONAME` and musl: It's usually never what you want to load two different versions of the same library, like both `/usr/lib/libpython3.14.so.1.0` and `~/.local/uv/python/cpython-3.14-whatever/lib/libpython3.14.so.1.0`. At worst it's wasteful but more likely they're incompatible and export the same symbols and you get some sad segfaults. glibc has the behavior that, if you are requesting a library by unqualified filename (i.e. when you need to search for the library), if the program already has a library open that has the same `DT_SONAME` as was requested, it will reuse that. So, if you have a Python binary that loads specifically one of these, and then you load some extension module that (unnecessarily) has a `(DT_NEEDED, "libpython3.14.so.1.0")` entry, glibc will say, "Oh, I've already got that" and resolve the lookup to the existing libpython3.14.so.1.0. Apparently musl will _not_ do this, and will perform the usual lookup logic, potentially ending up with two libpythons in your process, or failing to find the libpython it's already loaded and returning an error, or something.