When working with shared object (library) dependencies in Linux, it is important to be aware of the different ways that these dependencies can be handled. In this article, we will take a look at how to work with shared object (library) dependencies in Linux using the three most common methods: manual handling, system management, and package management. Manual Handling Manual handling is the most common way to work with shared object (library) dependencies in Linux. This method involves reading the library’s files and then handling any required installation and configuration tasks yourself. The following example shows how to manually handle a library dependency in Linux:
apt-get install libxml2-dev # apt-get install libxslt1-dev # aptitude install libxml2-utils # mkdir -p /usr/share/doc/libxml2/examples # cd /usr/share/doc/libxml2/examples # wget http://www.w3.org/TR/.gitignore cd .. # git clone https://github.com/w3corgs/libxml2.git cd libxml2 # ./configure –prefix=/usr –enable-shared –enable-threads –enable-syslog –enable=all make sudo make install
What Is a Shared Object Dependency?
A shared object (also called a library) is a binary (usually not directly executable) used by multiple programs/applications on a Linux instance. Such libraries are often installed at the operating system level and are shared (hence the name shared object or libraries) for use by one or more (and even many) directly executable applications.
For example, a program that features compressing files might require the bz2 (bzip2) library libbz2.so.1.0 to do so. The term library is more often used in Linux circles and is the preferred lingo among professionals, although shared object (and shared library) are both technically correct. Also interesting to note is that the .so file name extension used in many libraries stands for shared object!
An executable can have zero, one, or many libraries that it relies on. The fewer libraries, the more forward (when upgrading your operating system, for example) the compatibility will be. The more libraries, the greater the chance that sooner or later, some library dependency will break. This is also why application vendors sometimes decide to release statically compiled binaries rather than dynamically compiled binaries.
The difference between statically and dynamically compiled binaries is simple but has far-reaching consequences. A statically compiled binary has the libraries (available on the developer system at the time of compiling) compiled into the resulting binary/executable. A dynamically compiled binary will use the libraries installed, available, and shared on the user’s system.
As you can immediately see, this would require the user to install any such required dependencies, unless the same is taken care of in the operating system or application vendor’s package management system and details. This is why running a simple command like sudo apt install …some_app… will often yield a set of other related things (i.e. required libraries) to be co-installed at the same time.
Both static and dynamic compiling have pros and cons. For example, if you use static compiling, and a library that you’ve included in your software package (from the perspective of a software vendor) now has a security bug or critical update, it will likely mean that you have to re-release your software package, even if nothing has changed in your code.
But then again, relying on the user to install libraries, especially for complex programs, or when self-compilation of software is required, with the realization that end-users often struggle with such things, isn’t ideal either. It’s a complex area, and the issue has been discussed often.
For the purposes of this article, we will look at shared libraries in connection with a program that was dynamically compiled, as is often the case with operating system programs/executables/tools. With statically compiled programs (which are less common), an error like ‘Error while loading shared libraries’ is highly unlikely to display, as the libraries are included in the executable unless the program is part dynamic and only includes a limited set of built-in static libraries.
Error while Loading Shared Libraries!
Let’s switch into root mode for a while (using sudo su) and explore how shared libraries work when it comes to a tool like /usr/bin/zip that’s included or installable with major Linux distributions.
Here, we changed directories to /usr/bin and checked whether the zip program/binary/executable is present. Finding it present, we next checked the version by invoking it with –version and taking only the top two lines of the output by piping the output (using |) to avoid the long output given otherwise.
Finally, we used the ldd tool (a program that prints shared object dependencies) to see what libraries the executable requires. As we can see, the program requires four shared libraries. The list of libraries required is usually in the format of the library required, followed by a path where the named library was found already.
For some top-level or special OS-level libraries (like linux-vdso.so.1), no such path is shown. However, as long as no error is showing for any entry, you know that it’s fine and available (or preferable). In fact, linux-vdso.so.1 is a special virtual shared object/library injected into every process by the Linux Kernel, which doesn’t have a physical file on disk for the same. It’s there to make system calls more efficient.
So, let’s break things a little, and rename one of the required libraries so that it can no longer be discovered automatically by the binary:
As you can see, here, we renamed/moved the file from /lib/x86_64-linux-gnu/libbz2.so.1.0 to /lib/x86_64-linux-gnu/libbz2.so.1.0.NOT_PRESENT. This broke our zip application, and when we try to run it, we get the dreaded error while loading shared libraries error. Looking a bit closer, the error message is actually quite descriptive, however:
A seasoned IT engineer will always carefully review any logs and program output/information presented to him or her before making a call on an issue. It would pay off here, too, as the required file is clearly shown, libbz2.so.1.0, and the issue is clearly shown also. There’s No such file or directory. In other words, libbz2.so.1.0 is missing!
We can also verify the same with ldd, as can be seen in the image above. A clear libbz2.so.1.0 => not found helps us to know what’s happening. Once we clearly understand how dynamically compiled binaries (most of the binaries in the operating system are compiled this way) load and require libraries, and how to see whether one is missing (or being informed of the same via the error message), things don’t look that complicated anymore.
What’s more is that once we know the required library name, a quick search in your favorite search engine will show the additional package to be installed (or reinstalled) to obtain the required library name. Quite often, the package name can even be construed directly from the library name. However, in this case, the name for the runtime library is a bit offset.
There are two types of libraries—runtime libraries and development libraries. In this case, the package name that contains the libbz2.so.1.0 library is likely bzip2, and trying to uninstall would likely break various other items, as can be glanced from a long list of programs that will be removed if one tries to uninstall or purge the bzip2 package.
The second type of library is a development library. These are often required when trying to compile programs and are often even more closely related to their actual library file names. For example, in Ubuntu, simply take the library name and add -dev. For example, if we wanted to install the development package related to the libbz2.so.1.0 package, we could look at installing libbz2-dev:
Whereas this development library isn’t directly related to our libbz2.so.1.0 runtime library, it’s helpful to know the naming syntax of most development package names in Ubuntu (prefix of lib and suffix of -dev with the name of the library in between those) for whenever a certain development library is required (which is most often necessary when compiling software).
Also, suppose that any library somehow became broken. In that case, one of the easiest quick fixes is to purge the library sudo apt purge your_library_name command (which would completely purge the library/program passed), followed by a reinstallation using the sudo apt install your_library_name command.
And when you run into a situation similar to the above, where uninstalling the runtime library requires uninstalling/purging the main program, and where such a purge/remove is too large a part of the operating system or would affect too many other things, you can use a reinstallation option instead:
It isn’t always this easy, however, although knowing the above (and especially the naming convention in terms of adding -dev) helps tremendously when troubleshooting issues and will often fix the issue straight up.
It also helps a lot to know how libraries can be checked using ldd, and finally, understanding that a library is just an executable (although not directly) .so file that lives in a subdirectory of /lib or in /usr/lib or other similar directories.
Sometimes, libraries and/or packages conflict with one another, or certain packages require certain library versions, or certain libraries require other libraries, at specific versions themselves. Yes, it gets a little complex. It’s also somewhat easy, when it becomes this complex, to slightly mess up a system. At times, it’s even quite possible to completely break an operating system installation due to moving a too-important library, etc.
Often, it’s somewhat safer to create a symlink (a virtual link with a given filename, referring/linking to another existing file or another symlink itself, which in turn points to a real file) for a missing library at an older version, for example, and pointing that symlink to the latest installed version. It doesn’t always work, but just in case it fails, the symlink can be removed, hopefully safer (assuming that no such symlink or file with the same name existed beforehand).
The best way to troubleshoot these more complex problems is to always try an apt (or a similar package management tool on your operating system) -based solution first. At this point, you will also want to read how to use dkpg to fix apt, as it shows you a more granular way to manage packages (Although be careful, as more control comes with more responsibility!).
If all else fails, try to create a symlink, or even manually add the library file. (Please make triple-sure that any file downloaded is virus-free by, for example, checking it with VirusTotal, and it’s always best to use Linux’s vendor-provided repositories to download binaries.)
In extremis, you might also be able to find a library in a neighboring Linux operating system. For example, a Ubuntu executable will run on Mint and vice versa. Copying a library from another PC that you have is also a viable approach at times.
Wrapping Up
Fine-grained library management is a skill that takes a lifetime to learn. It’s almost an art. This article has provided the basic information/know-how and tools to use and has listed some more advanced troubleshooting suggestions for when things become murky, and sooner or later, they will if you’re a frequent user of Linux who regularly installs packages.
When you next see the ‘Error while loading shared libraries’ error on your Linux machine, you will be better equipped to understand where the issue might come from using a tool like ldd, as explained in this article. We also looked at static versus dynamically compiled binaries, and how shared or built-in libraries fit into and work with both.
If you enjoyed this article, you might also like to read the article on dpkg linked above as well as How to Add the Universe, Multiverse, and Restricted Repositories in Ubuntu.