Glenn Yonemitsu - software engineer. yonemitsu@gmail.com

Live Reloading Dynamic Libraries in Odin

There are ways to have your Odin application reload just parts of your code, making it easier and faster to iterate on your application logic. If the logic is well encapsulated this can prevent annoying "setup" steps to test your code.

For example, a very common use case for this setup is game development. A library can have procedural generation logic updated and it can the updated logic can be reviewed by just re-running the generation widget. When an adjustment needs to be made the code is updated and recompiled, and the currently running game just reruns the widget again. The traditional option is to recompile the entire codebase, load, and click through the game until the dev or tester gets to the procedural generation screen, and run. Then repeat to adjust.

The Parts

To achieve this we need a few parts to come together.

Example: Live Updating a REPL Like App

We will have our host app that acts like a REPL. This is a simple long-running app that let's us reload libraries while it is running.

The interaction is something like this:

$ ./repl
What is your name?
World
Hello, World!
What is your name?
Odin
Hello, Odin!
<etc>

The library provides the reponse format. Here it is the "Hello, <name>!" formatted response string.

We will then live update the library in the host app so the next time a name is entered a different response format is used.

$ ./repl
What is your name?
World
Hello, World!
<library changed here. the host app is still running>
What is your name?
Odin
Oh hello there, Odin! Nice to meet you!

The Library

// mylib.odin
package mylib

import "core:fmt"

@(export)
greet :: proc (name: string) {
    fmt.printf("Hello, %s!\n", name)
}

This is almost self explanatory. The one additional line is @(export).

Now we need to make sure our host app understands the library has this procedure to use.

The Host App

// host.odin
package host

import "core:dynlib"
import "core:fmt"
import "core:os"

Symbols :: struct {
    greet: proc (string),
    _lib_handle: dynlib.Library
}

main :: proc() {
    sym: Symbols

    LIB_PATH :: "mylib.so"

    buf: [32]byte
    for {
        fmt.println("What is your name? ")
        n, err := os.read(os.stdin, buf[:])
        if err != nil {
            fmt.eprintln("Error reading: ", err)
            return
        }
        name := string(buf[:n-1])

        count, ok := dynlib.initialize_symbols(&sym, LIB_PATH, "", "_my_handle")
        if !ok {
            fmt.eprintfln("Error initializing: %s", dynlib.last_error())
            return
        }

        sym.greet(name)
    }
}

Here we have an infinite loop asking and waiting for user input. For simplicity here the library is reloaded for every response.

The package that enables this is core:dynlib. The procedure initialize_symbols is a very convenient way to assign the library procedures and functions to your struct instance. On top of that if it is called repeatedly then it can automatically unload the previous loaded library instance for you.

The Symbols struct has the expected library procedure:

greet: proc (string)

and an extra handle field which is passed to the initialize_symbols call:

_lib_handle: dynlib.Library

This is a way to get the actual library handle in case you wish to unload it manually.

The core:dynlib package is actually a convenience package that wraps around the compilation target OS packages such as core:sys/posix. The wrapper is extremely thin and should almost always be used.

How to Compile the Library

The library is compiled separately with an additional flag:

$ odin build mylib.odin -build-mode:shared -file

The host app is compiled normally:

$ odin build host.odin -file

A (Not So Live) Demo

$ ./host
What is your name?
World
Hello, World!
What is your name?

Here let's update the library print line to:

@(export)
greet :: proc (name: string) {
    fmt.printf("Hello there, %s! Nice to meet you!\n", name)
}

and recompile it:

$ odin build mylib.odin -build-mode:shared -file

Now let's switch back to our app:

$ ./host
What is your name?
World
Hello, World!
What is your name?
<we resume here>
Odin
Hello, Odin! Nice to meet you!
What is your name?

Reloading C Libraries

We just need to tell our host app the library uses the C calling convention and use C specific types for params and return types such as c.int. So the Symbols struct might look like this:

Symbols :: struct {
    greet: proc "c" (cstring),
    _lib_handle: dynlib.Library
}

Some Caveats

When reloading a library it might not be compatible with your current state. For example a field in a state struct has been changed. I'm not sure what would be the answer there, possibly a database migration style update hook when reloading?

Hot reloading is a difficult problem, even Microsoft's .NET hot reloading turned out (in my opinion) to not be a solution where everything works magically.

I have not explored these topics so cannot confidently comment on any approaches.

Why Not Use Lua?

Lua is certainly a great approach for fast iteration. The host app can reload a text file and replace the functions or data in the top level Lua state. There are some issues:

References