I found myself on a bit of a yak shaving exercise trying to get a C library linked during the build process of a Hare program. These are a few notes on testing out potentially breaking changes with little interuption to my development machine.
Hare is able to link to C
libraries. The way to do it is by making a forward declaration using
the symbol and prototype for the library function and linking during
the build process. For example, if the following
is cmd/example/main.ha:
use strings;
@symbol("printf") fn printf(fmt: *const char, ...) int;
export fn main() void = {
printf(strings::to_c("hola mundo\n"));
};The build process is simply:
hare build -lc cmd/exampleThis works out of the box on both Debian and Alpine, where I followed the standard installation instructions. I ran into an issue doing the same on Fedora though. Trying the above results in several opaque error messages:
$ hare build -lc cmd/example
/usr/lib/gcc/x86_64-redhat-linux/12/crtbegin.o: in function `deregister_tm_clones':
(.text+0x7): relocation truncated to fit: R_X86_64_32S against `.tm_clone_table'
/usr/lib/gcc/x86_64-redhat-linux/12/crtbegin.o: in function `register_tm_clones':
(.text+0x38): relocation truncated to fit: R_X86_64_32S against `.tm_clone_table'
collect2: error: ld returned 1 exit status
Error: cc: exited with status 1
hare build: build failed
As I understand it the linker is attempting to use an (AMD 64 ABI)
32-bit relocation type, R_X86_64_32S while the
provided address lies outside the space.
Knowing the what is a little interesting but I still had no idea why I was seeing this only under Fedora. It turns out GCC will happily tell you how it was configured when built though, so I decided to check the 3 environments:
# gcc -v
Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/usr/lib/gcc/x86_64-linux-gnu/10/lto-wrapper
OFFLOAD_TARGET_NAMES=nvptx-none:amdgcn-amdhsa:hsa
OFFLOAD_TARGET_DEFAULT=1
Target: x86_64-linux-gnu
Configured with: ../src/configure -v --with-pkgversion='Debian 10.2.1-6' --with-bugurl=file:///usr/share/doc/gcc-10/README.Bugs --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++,m2 --prefix=/usr --with-gcc-major-version-only --program-suffix=-10 --program-prefix=x86_64-linux-gnu- --enable-shared --enable-linker-build-id --libexecdir=/usr/lib --without-included-gettext --enable-threads=posix --libdir=/usr/lib --enable-nls --enable-bootstrap --enable-clocale=gnu --enable-libstdcxx-debug --enable-libstdcxx-time=yes --with-default-libstdcxx-abi=new --enable-gnu-unique-object --disable-vtable-verify --enable-plugin --enable-default-pie --with-system-zlib --enable-libphobos-checking=release --with-target-system-zlib=auto --enable-objc-gc=auto --enable-multiarch --disable-werror --with-arch-32=i686 --with-abi=m64 --with-multilib-list=m32,m64,mx32 --enable-multilib --with-tune=generic --enable-offload-targets=nvptx-none=/build/gcc-10-Km9U7s/gcc-10-10.2.1/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-10-Km9U7s/gcc-10-10.2.1/debian/tmp-gcn/usr,hsa --without-cuda-driver --enable-checking=release --build=x86_64-linux-gnu --host=x86_64-linux-gnu --target=x86_64-linux-gnu --with-build-config=bootstrap-lto-lean --enable-link-mutex
Thread model: posix
Supported LTO compression algorithms: zlib zstd
gcc version 10.2.1 20210110 (Debian 10.2.1-6)# gcc -v
Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/usr/libexec/gcc/x86_64-alpine-linux-musl/12.2.1/lto-wrapper
Target: x86_64-alpine-linux-musl
Configured with: /home/buildozer/aports/main/gcc/src/gcc-12-20220924/configure --prefix=/usr --mandir=/usr/share/man --infodir=/usr/share/info --build=x86_64-alpine-linux-musl --host=x86_64-alpine-linux-musl --target=x86_64-alpine-linux-musl --enable-checking=release --disable-fixed-point --disable-libstdcxx-pch --disable-multilib --disable-nls --disable-werror --disable-symvers --enable-__cxa_atexit --enable-default-pie --enable-default-ssp --enable-languages=c,c++,d,objc,go,fortran,ada --disable-libssp --disable-libsanitizer --enable-shared --enable-threads --enable-tls --with-bugurl=https://gitlab.alpinelinux.org/alpine/aports/-/issues --with-system-zlib --with-linker-hash-style=gnu --with-pkgversion='Alpine 12.2.1_git20220924-r4'
Thread model: posix
Supported LTO compression algorithms: zlib
gcc version 12.2.1 20220924 (Alpine 12.2.1_git20220924-r4) $ gcc -v
Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/usr/libexec/gcc/x86_64-redhat-linux/12/lto-wrapper
OFFLOAD_TARGET_NAMES=nvptx-none
OFFLOAD_TARGET_DEFAULT=1
Target: x86_64-redhat-linux
Configured with: ../configure --enable-bootstrap --enable-languages=c,c++,fortran,objc,obj-c++,ada,go,d,lto --prefix=/usr --mandir=/usr/share/man --infodir=/usr/share/info --with-bugurl=http://bugzilla.redhat.com/bugzilla --enable-shared --enable-threads=posix --enable-checking=release --enable-multilib --with-system-zlib --enable-__cxa_atexit --disable-libunwind-exceptions --enable-gnu-unique-object --enable-linker-build-id --with-gcc-major-version-only --enable-libstdcxx-backtrace --with-linker-hash-style=gnu --enable-plugin --enable-initfini-array --with-isl=/builddir/build/BUILD/gcc-12.2.1-20220819/obj-x86_64-redhat-linux/isl-install --enable-offload-targets=nvptx-none --without-cuda-driver --enable-offload-defaulted --enable-gnu-indirect-function --enable-cet --with-tune=generic --with-arch_32=i686 --build=x86_64-redhat-linux --with-build-config=bootstrap-lto --enable-link-serialization=1
Thread model: posix
Supported LTO compression algorithms: zlib zstd
gcc version 12.2.1 20220819 (Red Hat 12.2.1-2) (GCC)
Obviously I recommend using diff unless you're really
good at I Spy. The thing that jumped out at me was the
flag --enable-default-pie, where PIE
is position independent executable a feature that
allows for ASLR.
I tried in vain to coerce this feature in my build through
the CFLAGS environment variable to no effect. I ended
up deciding to try a custom build of GCC on Fedora with this flag
to test the idea. How bad could it be?
This was almost uninteresting - there
is good documentation
and I have the other necessary configuration options used
previously in order to make my GCC as close to stock as
possible. I did a bit of copy-paste and then
invoked make and threw 16 cores at the thing. It took
over an hour for a clean build and only then did I realize my
error.
By default GCC will prefix itself into /usr/local
(which turns into /usr/local/bin), whereas Fedora
builds it prefixed into /usr. If I wanted to install
my new toolchain I would be overwriting my package installation! I
am obviously not quite so confident, having never done this
before. I had almost resigned myself to reconfiguring and
rebuilding (and thus another ~80 minute wait). Thankfully I
remembered systemd-nspawn
and ephemeral
containers. This is almost exactly the use case it was
designed for!
By creating a container with a throw-away snapshot of my system I can install my new GCC build without risking altering my development machine negatively. All it requires is this:
$ sudo systemd-nspawn -x -D /Btrfs ensures it launches efficiently and I'm free to poke and prod things to my hearts content, with the whole thing being torn down on exit.
After running make install inside my container I was
able to verify the change worked! The issue really was in the
difference in GCC build-time configuration between distros. With
this in mind I was also able to do more focused searching for the
necessary incantation to make a custom GCC build unnecessary. It
turns out I was close with my attempts
at CFLAGS. Instead I only need to export
LDFLAGS='-pie' and the linker will gracefully handle the
relocations.
During this process I tried soliciting ideas from smarter people on IRC. Unfortunately the response I got was that Hare doesn't support linking to shared objects and does not itself support PIE. While I had coerced my build into producing a running program it sounds as though this is only incidentally working, rather than working as designed. The solution is to use static libraries, which should be possible but entails some more work which I haven't delved into yet. Ho-hum.
I know I'm not the only person in the world linking shared objects successfully into Hare programs but it sounds like we may all find ourselves doing the right thing sooner or later as Hare proceeds in development. I imagine I'll keep plodding along to find how to link static libraries on Fedora and Debian. Both platforms cheerily pull in all sorts of shared objects by default, so far as I can tell, using static libraries is a bit off the beaten path.