zvrba.net

Background

Position-indepentend code, used for shared libraries on Linux, contributes significantly to the complexity of the toolchain, the run-time linker, and even makes the compiled code slower on some architectures, in particular 32-bit x86. Shared libraries also make dependency management harder (look up "DLL hell"; linux was not immune to it either).

On the other hand, shared libraries do have some neat features:

Saving memory through code sharing: all processes using the same shared library will refer to the same physical copy of it in memory.
One can deploy security and other fixes just by replacing a single file. Already running programs won't be live-patched though.
It is easy to interpose code through the LD_PRELOAD mechanism, e.g., for logging, accounting or other purposes.
It is easy to build a plugin architecture based on shared libraries because the hardest part – plugin loading and resolving of symbols – is exposed by the OS to the applications

Other pros and cons of dynamic linking have been discussed, for example, here, which was also the motivation for this article.

The tool

One can often hear that one of the main reasons for introducing all of this complexity – saving memory and disk space – no longer holds today. After all, shared libraries were conceived at the time when 100MB hard disk and 8MB of RAM was a lot.

I have, however, never seen any investigation into the truthfulness of this claim, so I set out to find out how much physical memory is saved on a modern OS through use of shared libraries.

To measure this, I wrote a program (runs only under Linux, and must be executed as root) which

Enumerates all running processes,
For each process, it visits all private, readable and executable VM mappings corresponding to a shared object on the file system,
For each VM address present in physical memory, reads out the physical page frame number and the frame's reference count.
Finally, it computes how much physical memory is saved by sharing. In other words, that much more physical memory would be used if each process had own copy of the required code.

Note: The above program works only with physical memory sizes less than 8GB. To increase this, you need to edit line 169 in the program (setting the size of G_refcounts vector).

Results

I ran the above program on a stock Centos 7 installation running in a VM configured with 1GB RAM. Of the running programs, I had the default desktop environment, firefox (one tab), emacs and a terminal program running.

The program outputted a total saving of approximately 16500 pages, or, about 62MB. Given this finding, I can agree that savings, with respect to today's memory and disk sizes, are rather insignificant.

Of course, the savings depend on the load. I myself don't have the opportunity, but it would be interesting to compute savings on other types of systems and workloads, such as KDE desktop with many open applications, and servers (mail, web, database) with many concurrent users.