Question about static linking and binary size

[Danny-E 33]

Hello! This is just an exploratory question out of curiosity, not a major issue.

I've developed a cross-platform application using libopenmpt that runs on Windows, Mac, and Linux.

On Windows and Mac, I statically link every dependency so that my binaries are highly portable, and I distribute these pre-built binaries.

On Linux, I don't distribute pre-built binaries. It's a "build it on your own" situation. So it doesn't necessarily matter whether I statically or dynamically link.

However, if I do statically link on Linux, the binary is huge.

For comparison, here are the sizes on Windows, Mac, and Linux:

Windows:
libopenmpt-small.lib: 482 MB
My binary: 3 MB

Mac:
libopenmpt.a: 27 MB
My binary: 9 MB

Linux:
libopenmpt.a: 255 MB
My binary: 80 MB

My question is, is there any way to reduce the fully-statically-linked Linux binary from 80 MB down to closer to 10 MB like on the other OSes?

Other context:

I built the .a file on Linux with the following options:

Code Select Expand

wget https://lib.openmpt.org/files/libopenmpt/src/libopenmpt-0.6.3+release.autotools.tar.gz
mkdir libopenmpt && tar xf libopenmpt-0.6.3+release.autotools.tar.gz -C libopenmpt --strip-components=1
cd libopenmpt
./configure --prefix="$(realpath "$PWD/../..")" \
--without-mpg123 \
--without-ogg \
--without-vorbis \
--without-vorbisfile \
--without-sndfile \
--without-flac
make CXX="g++-8 -std=c++17"
make install
So I hope the .lib is already as minimalistic as possible, but it's still very big. Are there more options I could have added to `configure` to reduce the size of the lib further?

Also, for comparison, on Linux, the dynamic library, libopenmpt.so.0.3.3, is 82 MB; much smaller than the static .a file. I'm not sure if that's useful or interesting or not.

And when I dynamically link libopenmpt on Linux, my binary size reduces from 80 MB to 1.4 MB.

Thanks for any advice!

https://github.com/dannye/crystal-tracker

[manx]

On Linux, I don't distribute pre-built binaries. It's a "build it on your own" situation. So it doesn't necessarily matter whether I statically or dynamically link.

However, if I do statically link on Linux, the binary is huge.

I built the .a file on Linux with the following options:

Code Select Expand

wget https://lib.openmpt.org/files/libopenmpt/src/libopenmpt-0.6.3+release.autotools.tar.gz
mkdir libopenmpt && tar xf libopenmpt-0.6.3+release.autotools.tar.gz -C libopenmpt --strip-components=1
cd libopenmpt
./configure --prefix="$(realpath "$PWD/../..")" \
    --without-mpg123 \
    --without-ogg \
    --without-vorbis \
    --without-vorbisfile \
    --without-sndfile \
    --without-flac
make CXX="g++-8 -std=c++17"
make install


Same as for every other project, Autotools defaults to include debug information (i.e. it passes -g -O2 in CXXFLAGS and CFLAGS) by default. The standard way for every Autotools project to opt out of that is by passing your desired CXXFLAGS to configure. Alternatively, you can also run strip on your binary.

Also, in general, you should pass CXX to configure instead of make for Autotools.

So, in your case:

Code Select Expand

./configure --prefix="$(realpath "$PWD/../..")" \
    --without-mpg123 \
    --without-ogg \
    --without-vorbis \
    --without-vorbisfile \
    --without-sndfile \
    --without-flac \
        CXX="g++-8" CXXFLAGS="-std=c++17 -O2" CFLAGS="-O2" 
make

If you do not know of any specific reason to force C++17, I strongly suggest you also drop passing -std=c++17 at all, i.e.:

Code Select Expand

./configure --prefix="$(realpath "$PWD/../..")" \
    --without-mpg123 \
    --without-ogg \
    --without-vorbis \
    --without-vorbisfile \
    --without-sndfile \
    --without-flac \
        CXX="g++-8" CXXFLAGS="-O2" CFLAGS="-O2" 
make


[manx]

See also: https://stackoverflow.com/a/4680578

[Danny-E 33]

Same as for every other project, Autotools defaults to include debug information (i.e. it passes -g -O2 in CXXFLAGS and CFLAGS) by default. The standard way for every Autotools project to opt out of that is by passing your desired CXXFLAGS to configure. Alternatively, you can also run strip on your binary.

Also, in general, you should pass CXX to configure instead of make for Autotools.

Ahh, thank you, I had no idea! That indeed fixes all my issues.

libopenmpt.a is now 6.8 MB and my statically linked binary is now 3.7 MB.

Thank you very much!

If you do not know of any specific reason to force C++17, I strongly suggest you also drop passing -std=c++17 at all

It appears that passing -std=c++17 was only necessary because I was naively setting CXX on make instead of configure.

Without -std=c++17, compilation fails immediately with many errors, like:

Code Select Expand

In file included from openmpt123/openmpt123.cpp:96:
./libopenmpt/libopenmpt.hpp:245:55: error: 'string_view' is not a member of 'std'
 LIBOPENMPT_CXX_API bool is_extension_supported2( std::string_view extension );
                                                       ^~~~~~~~~~~
./libopenmpt/libopenmpt.hpp:245:55: note: 'std::string_view' is only available from C++17 onwards

But when I move CXX from make to configure, -std=c++17 is no longer necessary.

[Danny-E 33]

Oh! I also remember now why I set CXX on make instead of configure.

From my older thread here:

One other small issue I ran into while building libopenmpt on Linux and Mac..

I had to install libportaudio.a and libportaudiocpp.a system-wide instead of keeping everything contained within my project directory.

It seems like libopenmpt's autoconf doesn't honor CPPFLAGS and LDFLAGS.

My (poor) understanding of autoconf is that I can define CPPFLAGS and LDFLAGS and then the configure script will use this to look for libraries in the specified directories.

For example, I ran:

Code Select Expand

CPPFLAGS="-I$(realpath "$PWD/../../include")" LDFLAGS="-L$(realpath "$PWD/../../lib")" ./configure --prefix="$(realpath "$PWD/../..")" \
--without-mpg123 \
--without-ogg \
--without-vorbis \
--without-vorbisfile \
--without-sndfile \
--without-flac
and it still failed with configure: error: Unable to find libportaudiocpp. even though libportaudiocpp is installed in $(realpath "$PWD/../../lib").

This isn't too big of a deal since installing system-wide fixed it, but I still thought it was odd.

Edit: Oh, I supposed I should specify that I am referring to libopenmpt-0.6.3+release.autotools.tar.gz

In my initial tests, it appeared as though configure was ignoring the options, so I stopped trying and just passed them to make instead. But with this new wisdom, I'm thinking I was probably doing it wrong originally.

Instead of setting the CPPFLAGS and LDFLAGS environment variables and letting the ./configure subprocess inherit them, I will experiment with passing them directly as arguments to configure and see if that lets me build my project purely self-contained without needing to install portaudio system-wide.

[Danny-E 33]

Apologies for triple posting, but I finally figured out how to build libopenmpt with a relative/self-contained installation of portaudio rather than a system-wide installation of portaudio.

I just had to set PKG_CONFIG_PATH when running configure, like so:

Code Select Expand

./configure --prefix="$(realpath "$PWD/../..")" \
    --without-mpg123 \
    --without-ogg \
    --without-vorbis \
    --without-vorbisfile \
    --without-sndfile \
    --without-flac \
    CXX="g++-8" CXXFLAGS="-O2" CFLAGS="-O2" \
    PKG_CONFIG_PATH="$(realpath "$PWD/../../lib/pkgconfig")"

This also works just as well on Mac (except for using CXX="clang++" instead of CXX="g++-8").

My build setup is now fully ideal. Thanks again for all your help!