Shared Libraries in OS 4
by Hans-Jörg Frieden, Hyperion Entertainment

Shared libraries are a means to provide the application programmer with an interface to the operating system. Practically everything in AmigaOS is implemented as shared libraries (with only a few exceptions). The classic shared library consists of a data structure in memory (called the Library Base) and a jump table that is located in front of the data structure.

A program that wants to call a library function first needs to obtain the pointer to the library base by using the exec call OpenLibrary. This will return a pointer to the library base. Since the function table is located in front of the library base, accessing a function works using a negative offset from the library base, with the first vector being located at -6, the second at -12 and so on. These vectors consist of a single 68k JMP instruction followed by four bytes of function address (hence these vectors are 6 bytes apart). A program simply does a direct subroutine jump into this vector, which then branches to the appropriate routine.

Two points in the above paragraph are important: "Direct", and "68K JMP instruction".

"Direct" is important because it indicates that there is nothing between the program itself and the code of the library. No system interaction takes place. There is no place that any "extended" mechanism can hook into.

"68k JMP" is important simply because classic programs expect to find a 68k JMP instruction there, jumping to 68k code. A PowerPC program would not be able to do similar since there is no PowerPC code at the "receiving end" of the call.

The dilemma is obvious: Either a library is 68k and cannot be used from the PowerPC (unless the program is prepared to emulate every library call), or a library is PowerPC and cannot be used from 68k code.

When we took up the challenge to move the operating system to PowerPC, finding an answer for this question was of vital importance. Our solution would have to satisfy a few key points:

Libraries should be able to be either PowerPC or 68k code, or both. A 68k program should be able to call a PowerPC library without recompiling the program. Likewise, a PowerPC program should be able to call a library without knowing if the library is 68k or PPC. Furthermore, it should be possible to replace 68k code/functions in a library without recompiling the program using the library.

One possibility would have been to assume that the entry point of libraries is always 68k. But this would have been dragging an old concept around without reason, and most of all it would have required to always go through the emulator (even for a few instructions only) when calling a library function. This is clearly not a good solution. It incurs an unacceptable overhead for every function call, something that can be especially bothersome on high-frequent calls.

Always assuming that the library is PowerPC is also not a solution. Old programs would not be able to call them, and maintaining two copies (one PPC, one 68k) would not help, as it would be a nightmare for developers to maintain.

Multi-Interface Libraries

The solution we finally implemented is what we call "multi-interface libraries". Like the name implies, a multi-interface library can export any number of jump tables.

An interface is almost like a library in itself; it consists of a table of function pointers, and an interface-private data part. There are no jump instructions in the interface anymore. A program wanting to call a function will simply read the function pointer and branch there (This way of function calling is beneficial to the PowerPC architecture).

To maintain compatibility with legacy applications, the 68k jump table is still there. On a pure 68k library, this still points to the normal 68k code. However, it is possible to redirect the call to a short stub that drops out of the emulator into a native version of the call. This way, even old programs will gain speed by using PowerPC native system functions.

Exec library is an example for a library that is completely written in PowerPC code but provides a redirecting jump table for legacy applications. Therefore, a 68k program calling e.g. the "AllocVec" system call ends up calling the PowerPC version of this call without even noticing it. For a 68k application, this is completely transparent.

Interface usage

Interfaces have an associated C-Language structure, and an ASCII string name. The structure specifies the layout of the interface, and is somewhat like a replacement for the old-style #pragma's of the SAS/C or StormC compilers. However, much like an "interface" in Java or a "class" in C++, the interface structure just specifies the layout and available functions/methods of an interface - not their implementation. In theory, the same interface structure can be provided by more than one library, or a single library can export multiple versions/implementations of an interface (we'll look at this a bit more later in this article, and in more depth in a later article about the OS 4 Expansion and PCI architecture, which makes extensive use of this feature).

Interfaces usually follow a specific scheme for naming. They usually contain a descriptive part and the postfix "IFace". For example, the Exec interface is called "struct ExecIFace", Intuition is called "struct IntuitionIFace" and so on.

Every library knows at least two interfaces. One of these is an internal control interface, which is only used by the system itself (depending on whether a library is a "real" library or a device, it will export either a LibraryManagerIFace or a DeviceManagerIFace). The other interface usually contains the main functionality of the library.

To obtain an instance of the interface, the program must call Exec's "GetInterface" function. This function accepts both the library base pointer and an ASCII name string as an argument, and creates and/or returns a pointer to the interface that was queried. For this purpose, interfaces are identified by a name. Most libraries export their main interface under the name of "main", but this naming is not required. As an example, Exec library exports the "main" interface of type "ExecIFace", and an "mmu" interface of type "MMUIFace" (the latter one being a control interface for using the Memory Management Unit of the PowerPC).

As usual, Exec library is a special case, since it provides all the required functionality for handling interfaces. Therefore, the "ExecIFace" pointer is set up by the C library startup code, and is ready for use right away (a scheme similar to the old library auto-opening is also available in the C library to set up the most commonly used interfaces like DOS, Intuition and Graphics).

Calling a function in an interface is straightforward. The jump table part of the interface structure contains all interface functions in the form of function pointers. Calling one of these looks similar to a method call in C++ and somewhat similar to a method call in Java.

For example, suppose that the variable "IExec" contains a pointer to the "main" interface of Exec library, which is of type "struct ExecIFace". Calling the "AllocVec" function would look like this:

memory = IExec->AllocVec(size, memory_flags);

In the pre-OS 4 era, this would have looked like this:

memory = AllocVec(size, memory_flags);

As you can see, the only difference is the dereferencing of the "IExec" variable. In the old system, more or less the same thing was happening (the call was executed by dereferencing the SysBase pointer). This difference however allows the programmer to explicitly choose the jump table on which the AllocVec function should be invoked. While this does seem like an unnecessary writing overhead at first glance, there are a few key advantages to this method of library calls:

• The call is explicit; the programmer has full control over what is being called. There is no need for any sort of "library base swapping", for example when a program wants to call plugins.
• Libraries provide services; there is no reason why two libraries cannot provide similar services, yet the old system required to have a disjoint set of function names.
• The call is sufficiently different from the old system to allow for PowerPC-native library implementations while retaining the old 68k jump tables.
• The system allows a component model approach for both the system as well as for applications.

Especially the last point is of some importance. In general, a component model has a few advantages of traditional programming. One of these is that you can only invoke methods (or call functions) that the component actually understands. AmigaOS 4's expansion and PCI architecture relies on this feature: Due to the use of Interfaces, it is possible to transparently support any of the Amiga's PCI controllers as well as the AmigaOne's Articia chipset. Every PCI device on the bus is represented by an interface of type "PCI_Device". However, the interface instance is created separately for every device, and hence can hide all the implementation specific details in the interface instance itself - the programmer just calls

pcidev->WriteConfigByte(.);

The interface knows how to handle this, regardless of the device being a network card in a Prometheus, a sound card in a Mediator or an AGP card behind an PCI->AGP bridge on the AmigaOne.

Compatibility Interfaces

In order to provide the functionality of a 68k library to a PowerPC native program, Exec implements a mechanism for creating interfaces to old style libraries. For example, suppose your PowerPC program wants to use "foo.library" which is written in 68k and doesn't know anything about the interface concept. However, it still has the same 68k jump table that your PowerPC program could use by the means of the 68k Emulator integrated in the OS 4 kernel. You program could simply call the emulator for the functions of foo.library that it wants to use, but when the programmer of foo.library decides that he wants to have an OS 4 native version of it, you would need to recompile your program or live with the emulated version.

The obvious solution is to generate a "main" interface for foo.library (say, "struct FooIFace") with all the functions from the 68k jump table as interface functions. This interface would simply emulate all foo.library entry points.

When Exec sees a GetInterface call that cannot be satisfied (either because the library is an old 68k library, or the GetInterface failed) it will call upon RAMLIB (the program that is responsible for loading devices and libraries from disk) to somehow get the interface. RAMLIB will try to provide it by either looking for a precompiled interface file in LIBS:, or by generating the interface on the fly from an installed FD or SFD file.

At a later point, when foo.library is replaced by a PowerPC native version, the GetInterface will succeed in Exec, and the program will automatically be using the PowerPC version. No recompile was required.

Outlook

For now, we have only grazed the surface of what is possible with the new interface library system. It has been mentioned that an interface may contain a private data part, but we haven't yet seen how this can be used. It is also possible to clone interfaces during runtime, or modify interfaces depending on context.

More on that in a future issue of CAM.