DOCUMENT ID: 1551-02
SYNOPSIS: Solaris shared library Q&A
OS RELEASE:
PRODUCT: Solaris
KEYWORDS: shared library libraries object
DESCRIPTION:
What are shared libraries in Solaris?
SOLUTION:
Copyright 1995 Phillip Vandry Distribute freely
------------------------------------------------------------------
Definitions:
Application binary:
static linking:
dynamic linking:
Q. What is a shared library?
A. A shared library, or shared object file, or dynamic library, is a
file in the ELF file format (see elf(3E)) that contains data and usually
code that is needed by application binaries. It, along with any other
shared libraries an application binary depends on, is transparently
loaded into memory when the binary is executed, making itself available
to the binary.
Q. What are the advantages of shared libraries?
A. The major advantage of a shared library is that it may contain code that
more than one binary depends on. Thus, this code is only stored once,
in the shared library, and not in each individual binary. This saves
storage space on disk, especially when dealing with very common code
that nearly every program needs. But the benefits go farther than the
space savings: if the common code needs to be updated (for example, a
patch to fix a bug), this need only be done once, in the shared library.
(There are also more reasons, covered below).
Another big advantage is that programs which use the non shared versions
of the system supplied libraries like libc, are not supported and are
not ABI compliant. See "Why does my program, statically linked under a
different version of Solaris, dump core?" and "Why do I get unresolved
references when trying to link statically?"
Q. What are the disadvantages of shared libraries?
A. It takes time for a program to load all of the shared library files it
needs and to link them to the main program and to other shared libraries
(called dynamic linking). The program will take longer to start up.
Programs that depend on shared libraries will not work if any of the
libraries they depend upon are not available. For people who distribute
software, this often means developers must restrict themselves to using
only the shared libraries that are sure to be present on every system.
Unfortunately of course, nothing is for sure and according to Murphy's
law, there will always be someone who doesn't have whatever library,
or has a different version that the developers used and the different
version doesn't work properly. Arguably, it is cleaner (more elegant)
to have a program that is self contained and requires nothing but
itself.
Finally, the system must be able to locate the require libraries
wherever they may be, which is, again, error prone. (See "What
is LD_LIBRARY_PATH?")
Q. What systems can use shared libraries?
A. The model of shared libraries described in this document comes from
AT&T's System V UNIX operating system and is used in derivatives of
this OS, including Solaris. Other systems may implement shared
libraries slightly differently, but the concept is always the same.
Shared libraries resemble Microsoft Windows .DLL (Dynamic Load
Library) files.
Q. What does a shared object file look like?
A. Shared object files have the .so (for shared object) extension. They
often have a numerical extension on top of that. The file(1) command
will tell you if a file is a shared library (it will say "dynamic
lib") by examing its contents.
Q. How are shared libraries used in Solaris?
A. Almost all of the binaries that come with Solaris (such as the ones
found in the /usr/bin directory) are dynamically linked, which
means they depend on shared libraries to execute. The only ones
which aren't are a few critical system programs that may need to
be (or must) run at times when the shared libraries cannot be
used, such as very early in the boot process. Note that
technically, the kernel is such an exceptional program.
Solaris also uses shared libraries in another way than simply
loading a set of required libraries when run as described in
the first question. By their nature, shared libraries can be
opened and imported into a running program at any time. As an
example of this, take the getpwnam(3C) function. This function
returns information about a user including her user id and the
name of her home directory. It looks in the /etc/nsswitch.conf
file (see nsswitch.conf(4)) to determine what access method to
use to find this information. It might look in files (the
/etc/passwd file), NIS (Network Information name Service, aka
yp), or NIS+ (New version of NIS). (It would also be possible
to define another access method, but nobody has done this).
After having looked in the nsswitch.conf file, the getpwnam
function loads a shared library with the name nss_database.so.1
where "database" is the name of whichever access method was
found in nsswitch.conf. Each access method has a corresponding
file: nss_files.so.1, nss_nis.so.1, nss_nisplus.so.1. (you
can see these files by typing "ls /usr/lib".) Each shared
library file contains the same set of functions so that
functions like getpwnam is able to call them all the same way,
but each version of the functions searches a different
database. To summarize, it is possible to select one or another
version of a set of alternative functions by loading one or
another shared library containing the same function, depending
on values found in a configuration file, or even depending on
user input.
The above "trick" is used not only to select multiple databases
for a database lookup, but also to select mutiple similar network
protocols in much the same way.
Q. What is LD_LIBRARY_PATH?
A. LD_LIBRARY_PATH is an environment variable that tells the system where
to find shared libraies when they are needed, if the program itself
does not specify this information. You can view the current value of
this variable by typing "echo $LD_LIBRARY_PATH" in a shell. The
contents consist of a colon separated list of directories to search
sequentially. If it is not set, the default is "/usr/lib".
If LD_LIBRARY_PATH does not list the directory where a particular
required library is found, and the program which requires this
binary does not specify which directory to find it in either, then
the program will abort. For this reason, it is considered good
practice, when making programs, to have the program specify to
the dynamic linker where its libraries may be found if they
are not found in the standard directory /usr/lib. In this way, no
program should depend on the correct setting of LD_LIBRARY_PATH.
A program can be considered broken, or at the very least, lacking
robustness, if it doesn't work without LD_LIBRARY_PATH set. (See also
"Why should I use -R?")
Q. So should I use shared libraries?
A. For the system libraries, the answer is certainly yes. You should not
link with the static versions of libraries like libc (See "Why does
my program, statically linker under a different version of Solaris,
dump core?" and "Why do I get unresolved references when trying to link
statically?").
For libraries you create yourself or are otherwise not part of Solaris,
it will depend. A few examples of public domain or shareware packages
which have libraries is in order:
pbmplus: pbmplus, which is a suite of graphics format converters plus
a few other goodies, includes over 100 binaries (e.g. giftoppm,
pnmtotiff). All of them share a substantial amount of common code
for dealing with the package's internal representation of graphics.
A lot of space could be wasted by storing this code 100 times on
your disk. The common code can be placed in a shared library. When
this is done, each individual binary can be smaller than 10K while
it might otherwise have been over 20K with all the common code
statically linked in. The tradeoff for this is that pbmplus does
not support the creation of shared libraries for this purpose. It
takes a fair amount of Makefile editing to get this to work.
elm: elm, a popular Mail User Agent, includes as part of the source,
a (static) library libutil.a. This library is used only a few times,
always in programs that are part of the ELM package. Few of the
advantages of shared libraries exist in this case. Given the extra
overhead of loading shared libraries at run time and the need to
compile them as position independant code, it probably isn't worth
making a shared library out of this.
Q. How are shared libraries implemented at a low level?
A. Programs are either dynamically linked or statically linked. Statically
linked programs cannot load shared libraries. Dynamically linked
programs usually do.
When a statically linked program is run, it is loaded into memory, and
the processor jumps to a set location in the program which performs
program initialization. For C programs, this function sets things up
and then calls the program's main() function.
When a dynamically linked program is run, it is also loaded into
memory, but control is not transfered to a set location in the
program. Instead, the program contains the name of an interpreter
under which it should run. In practice, there is only one
interpreter, called /usr/lib/ld.so.1. This file is known as the
dynamic linker and is a *VERY* critical file for the operation of
a system!
The dynamic linker read the information that was recorded in the
program when it was made and loads the apropriate shared libraries.
It uses the LD_LIBRARY_PATH environment variabe (described above)
to locate them. It also uses LD_RUN_PATH. This paramater serves a
similar function, but it comes from the program, not the
environment. Thus the program can itself specify directories where
the dynamic linker should look without absolutely depending on
the proper setting of LD_LIBRARY_PATH.
Shared libraries are mapped into virtual memory using the
mmap(2) system call.
Once the loading process is complete, control is transferred to a
special function in the main program and we continue as with a
statically linked program.
Q. How do I make a shared library?
A. Making a shared library is a lot like making a regular library, but
there are a few differences. First of all, you should probably
compile your code in a position independant fashion. Here's why:
Normally, compilers make relocatable code. Because a set of
assembly language instructions (a program) usually needs to know
where in memory it is being executed from but it cannot be
predicted when a program is created where in memory it will be
loaded when it is run, compilers must generate almost all of the
code for a program, and leave the rest to be filled in once it
is known where the program has been loaded, at run time. For
this purpose, they prepare and include in the program tables of
the locations that need adjusting.
If a shared library uses relocatable code, then this relocation
must be performed every time the shared library is loaded into
a new program, at a new place in memory. Thus each copy of the
shared library in memory is slightly different after the
relocation has taken place.
If relocation wasn't required, then all copies of a shared object
in memory would be identical, and in fact, it would be possible
for all programs simultaneously requiring the same shared library
to share the same copy in memory (thus the name "shared" library).
This a a very huge advantage of shared libraries.
In fact, this is possible if position independant code is
generated. Position Independant Code (PIC) is less efficient
than normal relocatable code because programs must sometimes
use alternate methods of performing the same tasks. However, no
code needs to be modified at run time, as indicated above.
With the Sun C compiler, supply the "-K pic" option on the cc
command line. With the GNU compiler, supply the "-fPIC" option
on the gcc command line.
Finally, once your code is compiled, you can create a shared
object file as follows. ("-z text" is optional, but usually a
good idea. It ensures that your code os really position
independant).
ld -G -z text -o outputfile.so
Q. Why should I use -R when linking programs?
A. The -R option to the linker allows you to specify directories (one
per occurence or -R) where the program being linked should look for
its shared libraries when it is run. It should be used whenever a
program requires libraries stored in directories other than /usr/lib,
so that the program will find all the libraries it needs.
Note that a similar effect can be achieved by using the LD_LIBRARY_PATH
variable (See "What is LD_LIBRARY_PATH"), but depending on this is
very strongly discouraged because the program will always depend
on this variable being set correctly and it will fail to run whenever
it isn't!
On a properly configured system, all programs have been linked with
-R if they need libraries from elsewhere than /usr/lib, and
LD_LIBRARY_PATH need not be used.
Q. What are the numerical extensions on shared objects for?
A. The numerical extensions (e.g. libc.so.1) indicate version numbers.
Each time a new and incompatible version of a shared library is
created, the version is incremented. The old version should be kept
around for the sake of programs that were linked with the previous
version.
When making a program, the linker will look for shared libraries by
the extention ".so". Thus, for libraries which have numerical extensions,
there should be a symbolic link ending in .so pointing to the desired
version of the library (the most recent), e.g. libc.so --> libc.so.1.
Note that the version number of a shared library should only be
changed when the library's interface changes. Changes in the
implementation do not matter to programs using the library, so it
is possible to, say, optimize the code in a library, thereby
optimizing every program that uses the library, without changing
the version number.
Also, when a shared library is produced, and the above numerical
version and symbolic link convention will be used, the -h options
needs to be supplied to the linker. The argument to -h is the filename
which programs linked with the new library should look for to find the
library. It should be the name the library will be installed under.
If a library was not made using -h, then the name under which programs
using that library will look for it under is the name under which
the linker found the library when those programs were produced.
This is the .so file (since the linker only scans for .so files),
which is a symbolic link to the most recent version. This is bad
because the most recent version may not be the same one the program
was linked against, and if not, it will end up trying to use an
incompatible version. Using -h with the canonical name of the library
forces the program to always load the same version of the library.
Q. Why does my program, statically linked under a different version of
Solaris, dump core?
A. Statically linked programs do not comply with the Solaris Application
Binary Interface (ABI). This means that it is not supported. The C
library and a number of the other libraries that come with the OS
contain code that assumes things which may change from one version
of Solaris to another. Normally, this is reasonable because programs
are dynamically linked and they always fetch the appropriate code
from the libraries that exist on the system where the program is
run. But this doesn't work for statically linked programs, because
they use whatever code they were statically linked with.
Q. Why do I get unresolved references when trying to link statically?
Q. Why is there no "libdl.a"?
A. As described in "How are shared libraries used in Solaris?",
functions like getpwnam() dynamically link code when they are called
by programs, depending on which sources the system is configured
to use. In statically linked programs, there is no dynamic linker
available, yet these functions try to use it. This is what the
linker complains about.
The recommended way to circumvent this is, of course, to link
dynamically. However, here's another workaround if you must use
it (don't distribute programs you make like this unless you enjoy
headaches!)
A command line like this will make a dynamically linked program
but which uses functions in static system libraries:
cc -Bstatic .... -Bdynamic -ldl -Bstatic
Q. Where can I find X shared library?
A. The standard shared libraries that come with Solaris follow. There
should never be a problem locating the ones in /usr/lib. The others
may not be found if $LD_LIBRARY_PATH does not contain their
directory and the program does not suggest a search location itself.
For these cases, you can add the apropriate directory to
LD_LIBRARY_PATH (See "What is LD_LIBRARY_PATH?"). You may, of course
have additional libraries if you've installed software other than
the basic Solaris environment.
/usr/lib/libC.so.3
/usr/lib/libC.so.5
/usr/lib/libadm.so
/usr/lib/libadm.so.1
/usr/lib/libadmagt.so
/usr/lib/libadmagt.so.1
/usr/lib/libadmapm.so
/usr/lib/libadmapm.so.1
/usr/lib/libadmcom.so
/usr/lib/libadmcom.so.1
/usr/lib/libadmsec.so
/usr/lib/libadmsec.so.1
/usr/lib/libaio.so
/usr/lib/libaio.so.1
/usr/lib/libauth.so
/usr/lib/libauth.so.1
/usr/lib/libbsm.so
/usr/lib/libbsm.so.1
/usr/lib/libc.so
/usr/lib/libc.so.1
/usr/lib/libc2.so
/usr/lib/libc2.so.1
/usr/lib/libc2stubs.so
/usr/lib/libc2stubs.so.1
/usr/lib/libdl.so
/usr/lib/libdl.so.1
/usr/lib/libelf.so
/usr/lib/libelf.so.1
/usr/lib/libintl.so
/usr/lib/libintl.so.1
/usr/lib/libkrb.so
/usr/lib/libkrb.so.1
/usr/lib/libkstat.so
/usr/lib/libkstat.so.1
/usr/lib/libkvm.so
/usr/lib/libkvm.so.1
/usr/lib/libld.so.1
/usr/lib/liblddbg.so.2
/usr/lib/libm.so
/usr/lib/libm.so.1
/usr/lib/libmapmalloc.so
/usr/lib/libmapmalloc.so.1
/usr/lib/libnisdb.so
/usr/lib/libnisdb.so.2
/usr/lib/libnsl.so
/usr/lib/libnsl.so.1
/usr/lib/libposix4.so
/usr/lib/libposix4.so.1
/usr/lib/librac.so
/usr/lib/librac.so.1
/usr/lib/libresolv.so.1
/usr/lib/librpcsvc.so
/usr/lib/librpcsvc.so.1
/usr/lib/libsocket.so
/usr/lib/libsocket.so.1
/usr/lib/libsys.so
/usr/lib/libsys.so.1
/usr/lib/libthread.so
/usr/lib/libthread.so.1
/usr/lib/libthread_db.so
/usr/lib/libthread_db.so.0
/usr/lib/libvolmgt.so
/usr/lib/libvolmgt.so.1
/usr/lib/libw.so
/usr/lib/libw.so.1
/usr/lib/nss_compat.so.1
/usr/lib/nss_dns.so.1
/usr/lib/nss_files.so.1
/usr/lib/nss_nis.so.1
/usr/lib/nss_nisplus.so.1
/usr/lib/straddr.so
/usr/lib/straddr.so.2
/usr/dt/lib/libDtHelp.so
/usr/dt/lib/libDtHelp.so.1
/usr/dt/lib/libDtSvc.so
/usr/dt/lib/libDtSvc.so.1
/usr/dt/lib/libDtTerm.so
/usr/dt/lib/libDtTerm.so.1
/usr/dt/lib/libDtWidget.so
/usr/dt/lib/libDtWidget.so.1
/usr/dt/lib/libMrm.so
/usr/dt/lib/libMrm.so.3
/usr/dt/lib/libUil.so
/usr/dt/lib/libUil.so.3
/usr/dt/lib/libXm.so
/usr/dt/lib/libXm.so.3
/usr/dt/lib/libcsa.so
/usr/dt/lib/libcsa.so.0
/usr/dt/lib/libtt.so
/usr/dt/lib/libtt.so.2
/usr/openwin/lib/libX.so
/usr/openwin/lib/libX.so.4
/usr/openwin/lib/libX11.so
/usr/openwin/lib/libX11.so.4
/usr/openwin/lib/libXaw.so
/usr/openwin/lib/libXaw.so.4
/usr/openwin/lib/libXaw.so.5
/usr/openwin/lib/libXext.so
/usr/openwin/lib/libXext.so.0
/usr/openwin/lib/libXi.so
/usr/openwin/lib/libXi.so.5
/usr/openwin/lib/libXinput.so
/usr/openwin/lib/libXinput.so.0
/usr/openwin/lib/libXmu.so
/usr/openwin/lib/libXmu.so.4
/usr/openwin/lib/libXol.so
/usr/openwin/lib/libXol.so.3
/usr/openwin/lib/libXt.so
/usr/openwin/lib/libXt.so.4
/usr/openwin/lib/libXtst.so
/usr/openwin/lib/libXtst.so.1
/usr/openwin/lib/libce.so
/usr/openwin/lib/libce.so.0
/usr/openwin/lib/libdeskset.so
/usr/openwin/lib/libdeskset.so.0
/usr/openwin/lib/libdga.so
/usr/openwin/lib/libdga.so.1
/usr/openwin/lib/libdps.so
/usr/openwin/lib/libdps.so.5
/usr/openwin/lib/libdpstk.so
/usr/openwin/lib/libdpstk.so.5
/usr/openwin/lib/libdstt.so
/usr/openwin/lib/libdstt.so.0
/usr/openwin/lib/libolgx.so
/usr/openwin/lib/libolgx.so.3
/usr/openwin/lib/libowconfig.so
/usr/openwin/lib/libowconfig.so.0
/usr/openwin/lib/libpsres.so
/usr/openwin/lib/libpsres.so.5
/usr/openwin/lib/libtiff.so
/usr/openwin/lib/libtiff.so.3
/usr/openwin/lib/libtt.so
/usr/openwin/lib/libtt.so.1
/usr/openwin/lib/libxil.so
/usr/openwin/lib/libxil.so.1
/usr/openwin/lib/libxview.so
/usr/openwin/lib/libxview.so.3
DATE APPROVED: 11/27/95