13.2. Cross-language function calls

When a Morloc program composes functions from different languages, the compiler must bridge the language boundary. A key design principle is that Morloc never serializes functions. Functions cannot be meaningfully transmitted between language runtimes — there is no way to pickle a C++ template instantiation into something Python can call directly. Instead, Morloc generates wrapper functions that make IPC calls to the foreign language pool.

13.2.1. How it works

Each function in a compiled Morloc program is assigned a unique integer identifier called a manifold ID (mid). Every pool maintains a dispatch table mapping manifold IDs to concrete function implementations. When a function needs to call a function in another language, it does not call it directly — it sends a call packet containing the target manifold ID and serialized arguments over a Unix domain socket to the foreign pool, which dispatches the call and returns the result.

The compiler generates all of this automatically. Consider a program where Python’s pmap calls a C++ sum function:

module foo (sumOfSums)

import root-cpp
import root-py

source Py from "foo.py" ("pmap")
source Cpp from "foo.hpp" ("sum")

pmap :: (a -> b) -> [a] -> [b]
sum :: [Real] -> Real

sumOfSums = sum . pmap sum

When pmap is compiled in the Python pool, it receives sum not as a C++ function pointer, but as a Python wrapper function generated by the compiler. This wrapper:

  1. Serializes its arguments into the binary wire format

  2. Sends a call packet (with the C++ sum manifold ID) over the Unix socket to the C++ pool

  3. Reads the result packet back

  4. Deserializes the result into a Python value

From Python’s perspective, this wrapper is an ordinary Python callable. It can be passed to multiprocessing.Pool.map, stored in a list, or used anywhere a function is expected — because it is a regular Python function. The cross-language call is hidden inside it.

13.2.2. What the generated code looks like

The Python pool contains a wrapper like this (simplified):

def m1384(x):
    packed = morloc.put_value(x, "<list>a<float>f8")
    result = morloc.foreign_call(cpp_socket_path, 1384, [packed])
    return morloc.get_value(result, "<float>f8")

Here 1384 is the manifold ID assigned to sum, and cpp_socket_path is the path to the C++ pool’s Unix domain socket. The morloc.foreign_call function handles the IPC: it sends a call packet, waits for the response, and returns the raw result packet. The put_value and get_value functions handle serialization and deserialization using a compact binary schema string.

On the C++ side, the pool’s dispatch table routes the manifold ID to the actual sum implementation:

// compiler-generated dispatch
uint8_t* local_dispatch(uint32_t mid, const uint8_t** args) {
    switch(mid) {
        case 1384: return m1384(args[0]);  // calls sum
        // ...
    }
}

13.2.3. Performance implications

Intra-pool calls (functions in the same language) are direct native function calls — no serialization, no sockets, no dispatch table lookup. The only overhead is that functions may be wrapped in thin wrapper functions, but even this can be eliminated with the %inline pragma, which inlines the function body at the call site.

Inter-pool calls (cross-language) pay the cost of:

  • Serializing the arguments (proportional to data size)

  • A Unix socket round-trip (microseconds for small payloads)

  • Deserializing the result

In special cases, serialization can be avoided entirely. When data has the same binary representation in both languages, only a pointer to shared memory needs to cross the socket — no copying or conversion. Currently this zero-copy path is supported for Arrow tables; support for fixed-size numeric vectors and tensors is planned.

For higher-order functions like pmap, each invocation of the wrapped function is a separate IPC round-trip. If pmap sum is called on a list of 1000 elements, that is 1000 cross-language calls. This is the expected cost of language interop — the alternative would be to batch the data and send it all at once, but that would require changing the function’s interface.

When performance matters, the best strategy is to keep hot loops within a single language. The compiler’s implementation selection algorithm already optimizes for this: given multiple implementations of a function, it prefers the one that avoids cross-language calls.