Dynamic libraries and code replacements in Swift

Dynamic library packages

I’ve already printed an article about constructing static and dynamic libraries utilizing the Swift compiler, if you do not know what’s a dynamic library or you might be merely a bit extra about how the Swift compiler works, it is best to positively check out that put up first.

This time we will focus a bit extra on using the Swift Package deal Supervisor to create our dynamic library merchandise. The setup goes to be similar to the one I’ve created within the loading dynamic libraries at runtime article. First we will create a shared library utilizing SPM.

import PackageDescription

let package deal = Package deal(
    identify: "TextUI",
    merchandise: [
        .library(name: "TextUI", type: .dynamic, targets: ["TextUI"]),
    ],
    dependencies: [
        
    ],
    targets: [
        .target(name: "TextUI", swiftSettings: [
            .unsafeFlags(["-emit-module", "-emit-library"])
        ]),
    ]
)

The package deal manifest is kind of easy, though there are just a few particular issues that we had so as to add. The very very first thing is that we outlined the product sort as a dynamic library. It will be certain that the suitable .dylib (or .so / .dll) binary will likely be created while you construct the goal. ?

The second factor is that we might prefer to emit our Swift module information alongside the library, we will inform this to the compiler by means of some unsafe flags. Do not be afraid, these are literally not so harmful to make use of, these flags will likely be immediately handed to the Swift compiler, however that is it.

Now the supply code for our TextUI library goes to be quite simple.

public struct TextUI {

    public static dynamic func construct() -> String {
        "Whats up, World!"
    }
}

It is only a struct with one static perform that returns a String worth. Fairly easy, besides one factor: the dynamic key phrase. By including the dynamic modifier to a perform (or technique) you inform the compiler that it ought to use dynamic dispatch to “resolve” the implementation when calling it.

We’ll reap the benefits of the dynamic dispatch in a while, however earlier than we might transfer onto that half, now we have to construct our dynamic library and make it accessible for others to make use of. ?

Should you run swift construct (or run the venture through Xcode) it’s going to construct all of the required information and place them below the right construct folder. You can too print the construct folder by operating the swift construct -c launch --show-bin-path (-c launch is for launch builds, we will construct the library utilizing the discharge configuration for apparent causes… we’re releasing them). Should you record the contents of the output listing, it is best to discover the next information there:

• TextUI.swiftdoc
• TextUI.swiftmodule
• TextUI.swiftsourceinfo
• libTextUI.dylib
• libTextUI.dylib.dSYM

So, what can we do with this construct folder and the output information? We’ll want them below a location the place the construct instruments can entry the associated information, for the sake of simplicity we will put all the things into the /usr/native/lib folder utilizing a Makefile.

PRODUCT_NAME := "TextUI"
DEST_DIR := "/usr/native/lib/"
BUILD_DIR := $(shell swift construct -c launch --show-bin-path)

set up: clear
    @swift construct -c launch
    @set up "$(BUILD_DIR)/lib$(PRODUCT_NAME).dylib" $(DEST_DIR)
    @cp -R "$(BUILD_DIR)/lib$(PRODUCT_NAME).dylib.dSYM" $(DEST_DIR)
    @set up "$(BUILD_DIR)/$(PRODUCT_NAME).swiftdoc" $(DEST_DIR)
    @set up "$(BUILD_DIR)/$(PRODUCT_NAME).swiftmodule" $(DEST_DIR)
    @set up "$(BUILD_DIR)/$(PRODUCT_NAME).swiftsourceinfo" $(DEST_DIR)
    @rm ./lib$(PRODUCT_NAME).dylib
    @rm -r ./lib$(PRODUCT_NAME).dylib.dSYM

uninstall: clear
    
    @rm $(DEST_DIR)lib$(PRODUCT_NAME).dylib
    @rm -r $(DEST_DIR)lib$(PRODUCT_NAME).dylib.dSYM
    @rm $(DEST_DIR)$(PRODUCT_NAME).swiftdoc
    @rm $(DEST_DIR)$(PRODUCT_NAME).swiftmodule
    @rm $(DEST_DIR)$(PRODUCT_NAME).swiftsourceinfo

clear:
    @swift package deal clear

Now for those who run make or make set up all of the required information will likely be positioned below the suitable location. Our dynamic library package deal is now prepared to make use of. The one query is how can we devour this shared binary library utilizing one other Swift Package deal goal? ?

Linking towards shared libraries

We’ll construct a model new executable utility known as TextApp utilizing the Swift Package deal Supervisor. This package deal will use our beforehand created and put in shared dynamic library.

import PackageDescription

let package deal = Package deal(
    identify: "TextApp",
    targets: [
        .target(name: "TextApp", swiftSettings: [
            .unsafeFlags(["-L", "/usr/local/lib/"]),
            .unsafeFlags(["-I", "/usr/local/lib/"]),
            .unsafeFlags(["-lTextUI"]),
        ], linkerSettings: [
            .unsafeFlags(["-L", "/usr/local/lib/"]),
            .unsafeFlags(["-I", "/usr/local/lib/"]),
            .unsafeFlags(["-lTextUI"]),
        ]),
    ]
)

The trick is that we will add some flags to the Swift compiler and the linker, so that they’ll know that we have ready some particular library and header (modulemap) information below the /usr/native/lib/ folder. We might additionally prefer to hyperlink the TextUI framework with our utility, in an effort to do that now we have to cross the identify of the module as a flag. I’ve already defined these flags (-L, -I, -l) in my earlier posts so I suppose you are conversant in them, if not please learn the linked articles. ?

import TextUI

print(TextUI.construct())

Our primary.swift file is fairly simple, we simply print the results of the construct technique, the default implementation ought to return the well-known “Whats up, World!” textual content.

Are you prepared to switch the construct perform utilizing native technique swizzling in Swift?

Dynamic technique substitute

After publishing my authentic plugin system associated article, I’ve obtained an electronic mail from one in all my readers. Initially thanks for letting me know concerning the @_dynamicReplacement attribute Corey. ?

The factor is that Swift helps dynamic technique swizzling out of the field, though it’s by means of a non-public attribute (begins with an underscore), which implies it isn’t prepared for public use but (yeah… similar to @_exported, @_functionBuilder and the others), however finally will probably be finalized.

You may learn the unique dynamic technique substitute pitch on the Swift boards, there’s additionally this nice little snippet that incorporates a minimal showcase concerning the @_dynamicReplacement attribute.

Lengthy story quick, you should use this attribute to override a customized dynamic technique with your individual implementation (even when it comes from a dynamically loaded library). In our case we have already ready a dynamic construct technique, so if we strive we will override that the next snippet.

import TextUI

extension TextUI {

    @_dynamicReplacement(for: construct())
    static func _customBuild() -> String {
        "It simply works."
    }
}

print(TextUI.construct()) 

Should you alter the primary.swift file and run the venture it is best to see that even we’re calling the construct technique, it’ll be dispatched dynamically and our _customBuild() technique will likely be known as below the hood, therefore the brand new return worth.

It really works like a allure, however can we make this much more dynamic? Is it attainable to construct another dynamic library and cargo that at runtime, then change the unique construct implementation with the dynamically loaded lib code? The reply is sure, let me present you ways to do that. ?

import PackageDescription

let package deal = Package deal(
    identify: "TextView",
    merchandise: [
        .library(name: "TextView", type: .dynamic, targets: ["TextView"]),
    ],
    targets: [
        .target(name: "TextView", swiftSettings: [
            .unsafeFlags(["-L", "/usr/local/lib/"]),
            .unsafeFlags(["-I", "/usr/local/lib/"]),
            .unsafeFlags(["-lTextUI"]),
        ], linkerSettings: [
            .unsafeFlags(["-L", "/usr/local/lib/"]),
            .unsafeFlags(["-I", "/usr/local/lib/"]),
            .unsafeFlags(["-lTextUI"]),
        ]),
    ]
)

Identical SPM sample, we have simply created a dynamic library and we have used the TextUI as a shared library so we will place our TextUI extension into this library as a substitute of the TextApp goal.

Up to now we have created 3 separated Swift packages shared the TextUI module between the TextApp and the TextView packages as a pre-built dynamic library (utilizing unsafe construct flags). Now we will lengthen the TextUI struct inside our TextView package deal and construct it as a dynamic library.

import TextUI

extension TextUI {

    @_dynamicReplacement(for: construct())
    static func _customBuild() -> String {
        "It simply works."
    }
}

We will use the same makefile (to the earlier one) or just run the swift construct -c launch command and replica the libTextView.dylib file from the construct listing by hand.

Should you run this code utilizing Linux or Home windows, the dynamic library file will likely be known as libTextView.so below Linux and libTextView.dll on Home windows.

So simply place this file below your property listing we will want the complete path to entry it utilizing the TextApp’s primary file. We’ll use the dlopen name to load the dylib, this can change our construct technique, then we shut it utilizing dlclose (on the supported platforms, extra on this later…).

import Basis
import TextUI

print(TextUI.construct())

let dylibPath = "/Customers/tib/libTextView.dylib"
guard let dylibReference = dlopen(dylibPath, RTLD_LAZY) else {
    if let err = dlerror() {
        fatalError(String(format: "dlopen error - %s", err))
    }
    else {
        fatalError("unknown dlopen error")
    }
}
defer {
    dlclose(dylibReference)
}


print(TextUI.construct())

The beauty of this method is that you do not have to fiddle with further dlsym calls and unsafe C pointers. There’s additionally a pleasant and detailed article about Swift and native technique swizzling, this focuses a bit extra on the emitted replacements code, however I discovered it a really nice learn.

Sadly there’s another factor that now we have to speak about…

Drawbacks & conclusion

Dynamic technique substitute works good, this method is behind SwiftUI stay previews (or dlsym with some pointer magic, however who is aware of this for certain..). Anyway, all the things appears nice, till you begin involving Swift lessons below macOS. What’s incorrect with lessons?

Seems that the Goal-C runtime will get concerned below macOS for those who compile a local Swift class. Simply compile the next instance supply and try it utilizing the nm software.

// a.swift
class A {}

// swiftc a.swift -emit-library
// nm liba.dylib|grep -i objc

Below macOS the output of nm will include traces of the Goal-C runtime and that’s greater than sufficient to trigger some troubles throughout the dylib shut course of. Seems in case your library incorporates the ObjC runtime you will not be capable to truly shut the dylib, it doesn’t matter what. ??

Previous to Mac OS X 10.5, solely bundles might be unloaded. Beginning in Mac OS X 10.5, dynamic libraries can also be unloaded. There are a few instances during which a dynamic library won’t ever be unloaded: 1) the primary executable hyperlinks towards it, 2) an API that doesn’t assist unloading (e.g. NSAddImage()) was used to load it or another dynamic library that depends upon it, 3) the dynamic library is in dyld’s shared cache.

Should you check out man 3 dlclose you may get just a few extra hints concerning the causes, plus you too can test the supply code of the Goal-C runtime, if you wish to see extra particulars.

Anyway I assumed this ought to be talked about, as a result of it could actually trigger some bother (solely on macOS), however all the things works simply nice below Linux, so if you’re planning to make use of this method on the server aspect, then I would say it’s going to work simply advantageous. It isn’t protected, but it surely ought to work. ?

Oh, I nearly overlook the hot-reload performance. Nicely, you may add a listing or file watcher that may monitor your supply codes and if one thing modifications you may re-build the TextView dynamic library then load the dylib once more and name the construct technique if wanted. It is comparatively straightforward after you have tackled the dylib half, as soon as you determine the smaller particulars, it really works like magic. ?