<!-- https://morloc-project.github.io/docs/intro/ -->

# 1. Intro

Morloc Manual | https://morloc-project.github.io/docs/intro/ | next: https://morloc-project.github.io/docs/why/index.md

Morloc replaces the application with the function as the unit you build, publish, and compose.

You write ordinary code in an ordinary language and give it a type in Morloc. From that one type the compiler derives the command line interface, the network API, the MCP tool description a model reads, the wire format, and the argument parser, and it checks every boundary those cross before anything runs. The interface is not a convention an author remembered to follow. It is a consequence of a declaration.

Morloc types are language-neutral, so the implementation behind a type may come from any supported language, or from a composition of functions written in several. The compiler generates the code that carries data between them. That is why Morloc is polyglot: a library of functions cannot be universal if it is partitioned by language.

## 1.1. Morloc in one program

Two functions, in two languages, neither aware of the other. A C++ sum:

**foo.hpp**

```cpp
#pragma once
#include <vector>

double sum(const std::vector<double>& vec) {
    double sum = 0.0;
    for (double value : vec) {
        sum += value;
    }
    return sum;
}
```

and a parallel map in Python:

**foo.py**

```python
import multiprocessing as mp

def pmap(f, xs):
    with mp.Pool() as pool:
        results = pool.map(f, xs)
    return results
```

Neither file imports anything from Morloc. The Morloc module gives each a type and composes them:

**sums.loc**

```morloc
module m (sum, sumOfSums)

import root-py
import root-cpp

source Py from "foo.py" ("pmap")
source Cpp from "foo.hpp" ("sum")

pmap :: (a -> b) -> [a] -> [b]

--' Add up a list of numbers
sum :: [Real] -> Real

--' Add up a list of lists, summing each in parallel
sumOfSums :: [[Real]] -> Real
sumOfSums = sum . pmap sum
```

`.` is function composition, so `sumOfSums` reads right to left: `pmap sum` sums each inner list in parallel, and the outer `sum` adds the results. The `--'` lines are docstrings, which the compiler carries into every generated interface.

```console
$ morloc make sums.loc
$ ./sums sumOfSums '[[1,2],[3,4,5]]'
15
```

A Python function called a C++ function across a process boundary, and you wrote no binding, no serializer, and no argument parser. [Getting Started](https://morloc-project.github.io/docs/getting-started/index.md) builds this program up one step at a time.

## 1.2. What Morloc is not

Morloc is not a foreign function interface generator. You write no bindings and the languages never import one another. They run as separate processes and the compiler generates the traffic between them, which is why adding a language to a program costs a line rather than a binding layer.

It is not a language you rewrite into. The C++, Python, R, and Rust in a Morloc program is ordinary code in those languages, with no Morloc imports, no annotations, and no base class. You keep your editor, your debugger, your libraries, and your existing code. What Morloc adds is a type and a name.


---

<!-- https://morloc-project.github.io/docs/why/ -->

# 2. Why Morloc?

Morloc Manual | https://morloc-project.github.io/docs/why/ | prev: https://morloc-project.github.io/docs/intro/index.md | next: https://morloc-project.github.io/docs/getting-started/index.md

Every command line tool solves the same problems a second time. Argument parsing, input and output formatting, compression, streaming, exit codes, introspection: none of it is the tool’s actual work, all of it admits many reasonable answers, and every tool picks its own. Consistency across an ecosystem is then reachable only if every author agrees on a wide range of conventions and writes their code accordingly.

The costs of that are structural rather than accidental. A tool’s `--help` is prose written at its author’s whim, so no machine can build a reliable inventory of an environment. Two tools exchange structured data only if they already agree on a format, so anything richer than a byte stream needs a shared framework or a lossy encoding. A user who wants one more feature, or one fewer, has no move short of asking the maintainer. And because the command line is the only face a tool has, every other caller — a library, a network client, a model — gets a fresh layer of boilerplate laid over it, generating system calls and parsing text back out. That face also forces a shape on the work: a tool takes its input from the filesystem and delivers its output there, and calling it means spawning a process, whether or not the computation needed any of that.

Morloc’s answer to each of these is the same answer: derive the interface from the type instead of writing it. The sections below are that answer applied in different directions.

## 2.1. The interface is derived, not written

The program above declared two functions. It also, without further instruction, became a command line tool:

```console
$ ./sums -h
Usage: ./sums <nexus_options> <command> <command_options>

Commands:
  sum        Add up a list of numbers
  sumOfSums  Add up a list of lists, summing each in parallel

General Options:
  -h, --help  Print help; -hh adds details and examples, -hhh adds schemas
              (nexus options: -h @)
```

and a set of tool definitions a model client can consume:

```console
$ ./sums --mcp-tools
{
  "tools": [
    {
      "name": "sum",
      "description": "Add up a list of numbers",
      "inputSchema": {
        "type": "object",
        "properties": {
          "_1": {
            "type": "array",
            "items": {
              "type": "number"
            }
          }
        },
        "required": [
          "_1"
        ],
        "additionalProperties": false
      }
    },
...
```

The same program serves over HTTP, TCP, and Unix sockets, and answers `--json-help` with a machine-readable description of every command. None of these is a separate build or a separate description. They are renderings of the types the compiler already checked, so they cannot drift from the functions: rename an argument or change a return type and every one of them moves on the next build.

[Building CLIs](https://morloc-project.github.io/docs/clis/index.md) covers the command line view, [Building APIs](https://morloc-project.github.io/docs/apis/index.md) the network and MCP views, and [The interface as data](https://morloc-project.github.io/docs/clis/interface-as-data.md) the introspection formats.

## 2.2. Values cross boundaries, not file formats

A Morloc command writes its return type, serialized. A command that accepts that type reads it. Neither end invents a file format and neither end writes a parser.

Here is a second program. A C++ function counts every k-length subsequence of a string and hands back a `std::map<std::string, int>`. A Python function takes the Shannon entropy of a count table and expects a `dict`. Morloc knows both as `Map Str Int`, so composing them is an application and nothing else:

**kmers.loc**

```morloc
module kmers (countKmers, entropy, complexity)

import map-cpp
import map-py

source Cpp from "kmer.cpp" ("count_kmers" as countKmers)
source Py from "entropy.py" ("entropy" as entropy)

--' Count every k-length subsequence
countKmers :: Int -> Str -> Map Str Int

--' Shannon entropy of a count table, in bits
entropy :: Map Str Int -> Real

--' Sequence complexity: the entropy of its k-mer profile
complexity :: Int -> Str -> Real
complexity k seq = entropy (countKmers k seq)
```

Composed inside one program, nothing is ever written to a file or a pipe:

```console
$ ./kmers complexity 3 GATTACAGATTACA
2.75162916738782
```

The same declaration reaches past the edge of a program. Each of those functions is a command too, so the two halves can run as separate processes and pass the count table between them:

```console
$ ./kmers countKmers 3 GATTACAGATTACA
[["ACA",2],["AGA",1],["ATT",2],["CAG",1],["GAT",2],["TAC",2],["TTA",2]]

$ ./kmers countKmers 3 GATTACAGATTACA | ./kmers entropy -
2.75162916738782
```

Same answer, and nobody wrote a format. Composing is the faster of the two and the one to reach for; the piped form pays for a pipe. What makes the piped form work at all is that the wire form falls out of the same declaration that generated each command’s interface, so two programs built by different people, in different languages, at different times meet at the seam having agreed on nothing but a type.

The wire form is the compiler’s business, not yours. A compiled Morloc program runs one **pool** per language — a process holding all of that language’s functions — and the compiler decides how a value moves between them: small values ride inside the packet, large ones go through shared memory with only a pointer on the socket, and the reader can ask for JSON or MessagePack instead. Data too large for memory need not be a value at all: `IFile`, `IStream`, and `OStream` describe data that lives in a file, indexed or walked in order, and a handle to one crosses a pool boundary like any other argument. See [Controlling data transfer](https://morloc-project.github.io/docs/apis/data-transfer.md) and [Random access and streaming](https://morloc-project.github.io/docs/runs/random-access-and-streaming.md).

## 2.3. A signature and its implementations are separate things

A Morloc module may declare types and signatures and supply no code at all. Such a module typechecks and will not compile, because there is nothing to generate. It is complete as a specification and empty as a program.

Implementations arrive by import. [Writing code that is not tied to a language](https://morloc-project.github.io/docs/getting-started/abstract-modules.md) writes two unit conversions in no language at all, then compiles them to C++ by importing `root-cpp` — or to Python, by changing that one line to `root-py`. Import both and the compiler chooses per function; how it chooses is [One term may have many definitions](https://morloc-project.github.io/docs/types/term-polymorphism.md).

The standard library is built this way. `root` declares the typeclasses and signatures; `root-cpp`, `root-py`, `root-r`, and `root-rust` supply the code. The same split runs through `vector`, `map`, `set`, `text`, and the rest.

A signature is a surface, not a size. Nothing here says an implementation must be small: the functional core of a large application — tens of thousands of lines, once the format parsing and the incidental IO are stripped off — takes a type in a few lines. Morloc modules are small in their typed interface, which is the surface you compose against, and can be whatever size they need to be underneath.

## 2.4. Tests and benchmarks follow the type, not the language

Because the signature is separate from the implementations, so is everything written against the signature.

The `vector` module holds its own test suite, written once, in terms of the abstract module. Each implementation module binds that suite to itself in three lines:

**vector-py/test.loc**

```morloc
module test-vector-py (test)

import vector.test (test)
import vector-py
```

**vector-cpp/test.loc**

```morloc
module test-vector-cpp (test)

import vector.test (test)
import vector-cpp
```

Same tests, same assertions, different code underneath:

```console
$ cd vector-py && morloc make -o test test.loc && ./test test
...
All 81 tests pass

$ cd vector-cpp && morloc make -o test test.loc && ./test test
...
All 81 tests pass
```

Benchmarking works the same way, and for the same reason. Write the composition once against the abstract module, bind it to two implementations, and call both from one program. The runtime reports per-call timings through a log template you configure rather than code you write, so the measurement is not something each implementation reports for itself. See [Logging](https://morloc-project.github.io/docs/runs/logging.md).

## 2.5. Toolboxes add and subtract

A module that compiles to a command line tool is still a module, so another module can import it. Given two installed modules — `sift`, which searches files, and `stats`, which draws charts — a toolbox that takes some commands from each is an import list and an export list:

**tools.loc**

```morloc
--' A little toolbox for reading notes
module tools (scan, summarize, histogram)

import sift
import stats
```

Two imports, one export line, no glue. The result is a tool in its own right, with its own help, completions, and MCP surface, built from commands their authors never coordinated on.

Addition is another import. Subtraction is leaving a name out of the export list: `sift` may export five commands and this toolbox publishes two, and the three left out are gone from the help, from the completions, and from the MCP surface, with the code behind them never built. Neither move requires a plugin system, and neither requires the consent of whoever wrote `sift`.

## 2.6. Caching, logging, and placement are annotations

Memoizing an expensive step, recording what ran and how long it took, or moving heavy work onto another machine are not properties of a function. They are properties of where a function sits in a composition, and Morloc lets you say so without touching the function.

Label a call site, and configure the label in the program’s YAML:

```morloc
foo xs = expensive_step@slowfn xs
```

```yaml
labeled-groups:
  expensive_step: { cache: true }
```

Every call into `expensive_step@slowfn` is now memoized to disk. The freshness check is content-based rather than mtime-based: editing an unrelated comment does not invalidate the cache, copying the program to a new path does not either, and two machines that build byte-identical pool sources share it. The same group config carries `log: true`, and a label may cover a complex term rather than a single call, so an entire branch of the execution tree can be cached, logged, or — this is the part still in development — dispatched to a remote worker.

Failure is handled in the same spirit. A build flag wraps every foreign call so that anything which throws dumps its arguments to disk and records the chain of calls that reached it, which makes a failure inspectable without reproducing it. Builds without the flag pay nothing. See [Caching](https://morloc-project.github.io/docs/runs/caching.md), [Logging](https://morloc-project.github.io/docs/runs/logging.md), [Debugging](https://morloc-project.github.io/docs/runs/debugging.md), and [Execution contexts](https://morloc-project.github.io/docs/install/execution-contexts.md).

## 2.7. One environment, solved once

The usual objection to a polyglot program is that it multiplies package managers. Morloc’s answer is to solve the dependency problem rather than route around it.

A program declares what it needs — its languages, and its packages from conda, PyPI, crates, and the system — and `mim`, the Morloc installation manager, resolves the whole set together into one environment. That environment may be built natively on Linux or MacOS Silicon, and the compiler provisions it on demand. Imported Morloc modules are fetched automatically at versions compatible with your compiler.

Conventional workflow managers reach the opposite conclusion and give each task its own container. That does make the conflict go away, and the price is that every value between every pair of steps must be serialized, written, and parsed again, with a process launch on top. There is no other way for two containers to exchange anything.

Morloc pays a boundary cost only where a value actually crosses between pools. Within a pool there is none: the functions are compiled into one unit and call each other natively, with no serialization, no socket, and no IPC. Across pools the floor is a Unix domain socket round trip — a few microseconds — plus whatever marshalling the two representations need (zero in cases where shared memory can be used).

What that buys is granularity. When every boundary costs a container, functions have to be big enough to amortize it, and a tool becomes a monolith: one program that parses a bespoke format, hard-codes a parallelism strategy, and writes another bespoke format governed by its own flags. When a call inside a language is free and a call across one is microseconds, you can decompose to the level the problem actually has — one function for the base case, an existing library for the parallelism — and the formats move out to the edges, where one parser module reads an archival format once instead of every tool reimplementing it.

[Does Morloc allow function-specific containerized environments?](https://morloc-project.github.io/docs/qa/per-function-environments.md) in the Q&A takes up the comparison directly.

## 2.8. What this makes possible

Everything above is already in the compiler. What it is **for** is not built yet, and I want to be plain about which is which.

If interfaces are derived rather than written, the only artifact worth publishing is the function itself. That makes a few things possible that are not possible today:

-   **A library indexed by type.** The compiler already knows every exported signature. Searching a library by the shape of the function you need, across languages, becomes a question of building the index rather than inventing the data.
-   **Implementations that compete.** One signature, many implementations, with shared tests and shared benchmarks deciding between them — across languages, on your data.
-   **Composition that is checked rather than trusted.** If two modules typecheck against a common environment, they compose, and the compiler proves it where they meet. No pairwise integration testing is required, so the guarantee does not get more expensive as the library grows.
-   **Communities organized by values instead of by language.** Morloc calls these **planes**: namespaces that differ not by subject area or language but by what their members demand of code — review, verification, performance, or nothing at all. See [Planes of libraries](https://morloc-project.github.io/docs/future/index.md#planes-of-libraries).

None of that infrastructure exists yet. There is no registry, no type-directed search, and no plane but the default one.

## 2.9. Where the project stands

Morloc has been in development for about ten years. I use it for my own work, but it is not yet used widely by anyone else.

Solid: the compiler and its type system, C++/Python/Rust/R as fully supported languages, the generated CLI, HTTP, socket and MCP interfaces, environment and dependency management through `mim`, and a standard library covering the common data structures, text, math, tables, and tensors.

Thin or unfinished: library coverage far from complete, remote execution (the SLURM dispatch that makes Morloc usable as a cluster workflow language) is in development, editor support is current for vim and Pygments and stale for VS Code/Zed, some aspects of the type system are still experimental, and the module registry is unbuilt.

## 2.10. What I need

I’m looking for people who can:

-   **Write a module.** Take your program, give it types, and publish it.
-   **Report what breaks.** A bug report is worth more to me than a patch right now. Unexpected behavior, a bad error message, a gap in this manual, and anything that was harder than it should have been all count.
    
    -   **Fix the editor tooling.** Add support for your favorite editor.
    -   **Bring a language.** Every new language brings fun design questions, I would be happy to work with you in bringing your language into the Morloc ecosystem.
    -   **Tell me the right way to build a type system.** There is a lot of interesting theory to hash through. There’s a paper or two buried somewhere in all of this.


---

<!-- https://morloc-project.github.io/docs/getting-started/ -->

# 3. Getting Started

Morloc Manual | https://morloc-project.github.io/docs/getting-started/ | prev: https://morloc-project.github.io/docs/why/index.md | next: https://morloc-project.github.io/docs/getting-started/installing.md


---

<!-- https://morloc-project.github.io/docs/getting-started/installing.html -->

# 3.1. Installing Morloc

Morloc Manual > Getting Started | https://morloc-project.github.io/docs/getting-started/installing.html | prev: https://morloc-project.github.io/docs/getting-started/index.md | next: https://morloc-project.github.io/docs/getting-started/first-program.md

Morloc is installed and managed by `mim`, the Morloc installation manager. It fetches the compiler and runtime, resolves each program’s cross-language package dependencies into one coherent world, and runs, serves, and inspects Morloc programs. There is no separate Morloc install step: `mim` is the whole of it.

Morloc runs on Linux and on Apple Silicon macOS. On Windows, install through the [Windows Subsystem for Linux](https://learn.microsoft.com/en-us/windows/wsl/about) and follow the Linux instructions inside it.

## 3.1.1. Installing `mim`

One command, on Linux (x86-64 or ARM) or Apple Silicon macOS:

```console
$ curl -fsSL https://raw.githubusercontent.com/morloc-project/morloc-manager/main/scripts/install.sh | sh
```

This downloads the prebuilt `mim` binary for your platform, checks it against the published SHA-256 when that checksum is reachable, and installs it into `~/.local/bin` (or `$XDG_BIN_HOME`, if you set it). No `sudo` is needed. Two environment variables adjust it:

| Variable | Effect |
| --- | --- |
| `MIM_DEST` | Directory to install into. Default: `$XDG_BIN_HOME`, else `~/.local/bin`. |
| `MIM_VERSION` | Git tag to install (e.g. `v0.31.1`). Default: the latest release. |

The installer never edits your shell startup files. If the destination is already on your `PATH` — as `~/.local/bin` is on most Linux distributions — you are done:

```console
$ mim --version
```

Otherwise the installer prints the exact command to add it, which on macOS it usually will: macOS builds its default `PATH` from `/etc/paths`, which does not include `~/.local/bin`. For zsh, the macOS default shell, that command is:

```console
$ echo 'export PATH="$HOME/.local/bin:$PATH"' >> ~/.zshrc
```

Open a new shell afterwards, or run the same `export` in the current one. `mim` is the only executable you need on your `PATH`; everything else lives inside the environments `mim` manages.

### What `mim` needs from your host

`mim` is a single static binary with no libraries to install, but it does shell out to a few standard tools:

-   `curl` — required for fetching data
-   `tar` — required for unpacking archives.
-   a container engine — if you can’t run native or if you like boxes.
-   `nix` — if you run NixOS

Those last two are explained next.

## 3.1.2. Choosing a backend

You do not normally have to choose. `mim new` probes the host, picks a viable backend, and remembers the choice for later environments.

**Native** is the default wherever it works. Morloc runs directly on your host against a toolchain `mim` provisions with conda/pixi into a private directory. Nothing is installed system-wide and no container engine is needed. **Container** is the fallback, taken automatically when the native backend cannot work; Docker and Podman are supported.

To force one:

```console
$ mim new --engine none      # native
$ mim new --engine podman    # or: --engine docker
```

If more than one container engine is installed and no default has been recorded yet, `mim` asks you to name one rather than guessing.

**Which hosts get which backend**

The native backend works on:

-   Linux with glibc and a standard filesystem layout (Debian, Ubuntu, Fedora, RHEL, Arch, and so on), on x86-64 and ARM
-   macOS on Apple Silicon
-   NixOS, provided the `nix` toolchain is available and unprivileged user namespaces are enabled. Conda binaries expect the dynamic loader at a path NixOS does not have, so `mim` builds a `buildFHSEnv` sandbox to supply one.

Everything else falls back to a container: musl distributions such as Alpine, hosts with a non-standard filesystem layout, NixOS without `nix` or without user namespaces, and Intel macOS, for which no prebuilt Morloc compiler is published.

**Podman notes**

Unlike Docker, `podman` runs rootless by default, so no sudo is required, and on Linux it runs with no daemon.

On macOS and Windows (even through WSL) a virtual machine is required, so you will need to initialize `podman` first:

```console
$ podman machine init
$ podman machine start
```

**Apptainer / Singularity notes**

Apptainer (formerly Singularity) is the usual container engine on HPC clusters. It runs rootless, has no daemon, and uses a single-file image format (`.sif`) that lives on the shared filesystem, which makes it a natural fit for SLURM-style job dispatch. The historical fork SingularityCE is treated as equivalent; either binary is detected automatically.

> **Warning: Experimental Feature**
> Apptainer support is in development and is the least tested of the backends. Creating an environment with `--engine apptainer` is not currently expected to work: the image build emits a Dockerfile, which Apptainer cannot consume. Use the native backend, or Docker/Podman, until this is finished. The SLURM dispatch described in [Execution contexts](https://morloc-project.github.io/docs/install/execution-contexts.md) depends on Apptainer and is blocked behind the same work.

## 3.1.3. Creating an environment

An **environment** is a named, self-contained Morloc installation: a solved toolchain of compilers and language runtimes, the Morloc compiler and runtime built against it, and a data directory holding installed modules and binaries. On the container backend that toolchain lives in an image; on the native backend it lives in a private directory on your host. Either way, everything Morloc does happens inside an environment, and environments do not interfere with each other or with anything else on your machine.

Create one and name it `base`:

```console
$ mim new base
...
Solving native toolchain with pixi (this may take a few minutes)...
...
Native environment 'base' is ready.
Set 'base' as the default environment.
```

Every setting has a default, so that is the whole command. Pass `--wizard` to be prompted for each one instead; `mim new -h` lists them all.

The first run is the slow one. `mim` downloads the Morloc compiler for your platform, solves a conda toolchain, and builds the Morloc runtime from source against it. Budget several minutes. Later environments reuse the downloaded compiler, and re-running `new` or `update` with unchanged requirements skips the solve entirely.

It is also the run that fails if your network inspects TLS. If a download stops with a certificate error, pass your organization’s CA with `--cert-bundle`; see [Troubleshooting](https://morloc-project.github.io/docs/getting-started/troubleshooting.md).

Without a name, an environment is named after the Morloc version it tracks: `latest`, or `v0.105.2` for a pinned `--morloc-version`. The first environment you create becomes the **default** — the one every command targets when you do not pass `--env` — so `base` is ready to use immediately.

No language toolchain is installed up front. Python, R, C++, and Rust are provisioned on demand the first time you build a program that uses them, so your first `morloc make` will also pause to solve and install. Use `--lang` to pin a language into the environment whether or not a program asks for it:

```console
$ mim new polyglot --lang py,cpp
```

You can keep as many environments as you like and act on any of them with `--env`:

```console
$ mim ls                                  # list them; the default is marked
$ mim info base                           # detail on one
$ mim modify --env edge --set-default     # change the default
$ mim rm base                             # remove one
```

`mim info <name>` reports the environment’s backend, its Morloc version, the languages in its solved world, and the directories it owns. Add `--packages` to list every package in the solved world at its locked version.

## 3.1.4. Working inside an environment

Two ways in. `mim run` executes a single command:

```console
$ mim run -- morloc --version
0.101.0    # you may have a later version
```

`mim shell` drops you into an interactive session:

```console
$ mim shell
```

Inside that shell `morloc` is on your `PATH`, so you can drop the `mim run --` prefix. The rest of this manual writes commands as if you are in a `mim shell`; outside one, prefix them with `mim run --`.

The shell starts in your current working directory, and changes you make there persist. On the container backend that directory is bind-mounted in. The environment’s module directory persists too, so anything installed into it stays installed.

If you want syntax highlighting before you start typing, skip ahead to [Editor support](https://morloc-project.github.io/docs/getting-started/editor-support.md) and come back.


---

<!-- https://morloc-project.github.io/docs/getting-started/first-program.html -->

# 3.2. Your first program

Morloc Manual > Getting Started | https://morloc-project.github.io/docs/getting-started/first-program.html | prev: https://morloc-project.github.io/docs/getting-started/installing.md | next: https://morloc-project.github.io/docs/getting-started/sourcing.md

The inevitable "Hello World" case is implemented in Morloc like so:

**hello.loc**

```morloc
module hw (hello)

--' A Morlock's hello world
hello = "Hello up there"
```

Three things are happening. `module hw (hello)` names the module and lists what it exports. `hello = "Hello up there"` binds a term to a string literal. The `--'` line is a **docstring**: an ordinary `--` comment is ignored, but `--'` attaches documentation to the term below it, and that documentation ends up in the generated command line interface.

Compile it:

```console
$ morloc make hello.loc
```

This produces two things next to your source:

**`hello`**

a launcher script, named after the **source file**. Override the name with `-o`. (Installing a program is different — it takes the module’s name instead. See [Search and install](https://morloc-project.github.io/docs/apis/search-and-install.md).)

**`hello-build/`**

the build directory. `manifest.json` describes the program and `envspec.json` records the packages it needs. A program that sources a foreign language also gets a compiled pool per language under `pools/<language>/`; this one sources nothing, so it has none.

The launcher is a thin shell script. It execs the shared `morloc-nexus` runtime against `manifest.json`; the nexus parses your arguments, starts whichever language pools the call needs, routes data between them, and prints the result.

Run it:

```console
$ ./hello hello
"Hello up there"
$ ./hello
"Hello up there"
```

Because `hw` exports exactly one term, naming the command is optional — the second form means the same thing.

The `-h` flag prints help generated from your types and docstrings:

```console
$ ./hello -h
A Morlock's hello world

Usage: ./hello <nexus_options> @ <command_options>

General Options:
  -h, --help  Print help; -hh adds details and examples, -hhh adds schemas
              (nexus options: -h @)

Return: Str
```

You wrote no argument parser, no usage text, and no type annotation. The docstring became the summary and `Str` was inferred. This is the first thing worth noticing about Morloc: the command line interface is not something you build, it is a **view** of the library you wrote. The same library also has API and MCP views, covered in [Building APIs](https://morloc-project.github.io/docs/apis/index.md).

The `@` in the usage line sits where a subcommand name would go. It is the separator between the options the runtime provides and the ones your function declares, and it shows up here because `hw` has a single export and there is no name to mark that boundary. [The two argument zones](https://morloc-project.github.io/docs/clis/argument-zones.md) covers it; you can ignore it until then.


---

<!-- https://morloc-project.github.io/docs/getting-started/sourcing.html -->

# 3.3. Sourcing a foreign function

Morloc Manual > Getting Started | https://morloc-project.github.io/docs/getting-started/sourcing.html | prev: https://morloc-project.github.io/docs/getting-started/first-program.md | next: https://morloc-project.github.io/docs/getting-started/abstract-modules.md

A Morloc module on its own has no implementations. Real work comes from functions imported out of other languages. Let’s write two unit conversions in C++:

**units.hpp**

```cpp
#pragma once

double cels2fahr(double cels){
  return 1.8 * cels + 32.0;
}

double meters2feet(double meters){
  return meters * 3.28084;
}
```

This is ordinary C++. It includes no Morloc headers and knows nothing about Morloc — that is the point. Now source it:

**units.loc**

```morloc
module units (cels2fahr, meters2feet)

source Cpp from "units.hpp" ("cels2fahr", "meters2feet")

type Cpp => Real = "double"

--' Convert from Celsius to Fahrenheit
cels2fahr :: Real -> Real

--' Convert from meters to feet
meters2feet :: Real -> Real
```

Reading it line by line:

-   `source Cpp from "units.hpp" (…​)` pulls two names out of a C++ header. The language tag `Cpp` tells the compiler which toolchain and which pool the functions belong to.
-   `type Cpp ⇒ Real = "double"` maps the general Morloc type `Real` onto the concrete C++ type `double`. Morloc types are language-neutral; this is how you say what one becomes in a particular language.
-   `cels2fahr :: Real → Real` is the general type signature. Morloc checks calls against this, not against the C++ declaration.

Compile and run it:

```console
$ morloc make units.loc
$ ./units cels2fahr 100
212
```

The generated interface lists both exported commands, with the docstrings you wrote:

```console
$ ./units -h
Usage: ./units <nexus_options> <command> <command_options>

Commands:
  cels2fahr    Convert from Celsius to Fahrenheit
  meters2feet  Convert from meters to feet

General Options:
  -h, --help  Print help; -hh adds details and examples, -hhh adds schemas
              (nexus options: -h @)
```

and each command has its own help, showing the types it derived:

```console
$ ./units cels2fahr -h
Convert from Celsius to Fahrenheit

Usage: ./units <nexus_options> cels2fahr <command_options>

General Options:
  -h, --help  Print help; -hh adds details and examples, -hhh adds schemas
              (nexus options: -h @)

Positional arguments:
  1:  type: Real

Return: Real
```


---

<!-- https://morloc-project.github.io/docs/getting-started/abstract-modules.html -->

# 3.4. Writing code that is not tied to a language

Morloc Manual > Getting Started | https://morloc-project.github.io/docs/getting-started/abstract-modules.html | prev: https://morloc-project.github.io/docs/getting-started/sourcing.md | next: https://morloc-project.github.io/docs/getting-started/mixing-languages.md

Sourcing C++ works, but it pins these conversions to C++. Anyone who wants them from Python has to make a foreign call for arithmetic. Morloc lets you write the definition once, in no language at all:

**units-abstract.loc**

```morloc
module unitsAbstract (cels2fahr, meters2feet)

import root

--' Convert from Celsius to Fahrenheit
cels2fahr cels = 1.8 * cels + 32.0

--' Convert from meters to feet
meters2feet meters = meters * 3.28084
```

`root` is a language-independent module from the standard library. It declares the typeclasses for arithmetic and much else, but supplies no implementations. So `+` and `*` here are general operations with no code behind them yet.

This is the first module the manual imports, and you do not have to install it. `morloc make` fetches any missing import, and that module’s own imports, before it builds:

```console
$ morloc make units-abstract.loc
Auto-installing missing dependency: root
Fetching module 'root'...
Fetching module 'internal'...
Installed module 'internal'
Installed module 'root'
units-abstract.loc:6:29: error:
No implementation found for '+'
  |
6 | cels2fahr cels = 1.8 * cels + 32.0
  |                             ^
```

The install worked. The build did not, and that error is worth sitting with, because it is the shape of Morloc’s central idea.

Nothing is wrong with the module. Now that `root` is on disk you can ask the compiler directly, and it types both terms without complaint:

```console
$ morloc typecheck units-abstract.loc
cels2fahr :: Real -> Real
meters2feet :: Real -> Real
```

> **Note**
> Only the build commands fetch, which is why this section built before it typechecked. `typecheck`, `dump` and `eval` read what is already on disk and never reach the network, so on a fresh environment they fail on an import you have never built. Build it once, or run `morloc install root` by hand. Pass `--offline` to `morloc make` when you want the fetching off.

So the module is complete as a **specification** and empty as a **program**: there is nothing wrong to fix, there is only code missing. To get a program, import a module that carries implementations:

**convert.loc**

```morloc
module convert (cels2fahr, meters2feet)

import .units-abstract
import root-cpp
```

The leading `.` in `.units-abstract` marks a local file rather than an installed module. `root-cpp` holds the C++ implementations of \`root’s terms — here, just the arithmetic operators.

```console
$ morloc make convert.loc
$ ./convert cels2fahr 100
212
```

Same answer as the sourced-C++ version, from a definition that never mentioned C++. The build directory says which language it ended up in:

```console
$ ls convert-build/pools/
cpp
```

And that is the one line you change. Edit `convert.loc` to `import root-py`, rebuild, and the same abstract module compiles to Python:

```console
$ ls convert-build/pools/
py
$ ./convert cels2fahr 100
212
```

You can import both and let the compiler decide which implementations to use. How it chooses is [One term may have many definitions](https://morloc-project.github.io/docs/types/term-polymorphism.md).


---

<!-- https://morloc-project.github.io/docs/getting-started/mixing-languages.html -->

# 3.5. Mixing languages in one program

Morloc Manual > Getting Started | https://morloc-project.github.io/docs/getting-started/mixing-languages.html | prev: https://morloc-project.github.io/docs/getting-started/abstract-modules.md | next: https://morloc-project.github.io/docs/getting-started/parallelism.md

Morloc composes across languages freely, and that includes passing functions across the boundary. Here is a Python function that takes a temperature and a conversion **function**:

**format.py**

```python
def report(ctemp, c2f):
  return f"The current temperature is {ctemp}C ({c2f(ctemp)}F)"
```

We can hand it the C++ `cels2fahr` from earlier:

**report.loc**

```morloc
module report (report)

import root-py
import root-cpp

source Py from "format.py" ("report" as report_wrapper)
report_wrapper :: Real -> (Real -> Real) -> Str

source Cpp from "units.hpp" ("cels2fahr")
cels2fahr :: Real -> Real

--' Write a cute string about the temperature
report t = report_wrapper t cels2fahr
```

The `as` keyword renames an imported term, which lets the sourced Python function and the exported Morloc term share a concept without colliding.

```console
$ morloc make report.loc
$ ./report report 21
"The current temperature is 21.0C (69.80000000000001F)"
```

A Python function called a C++ function, and you wrote no binding code. All of the interop — serializing the argument, starting both pools, passing a callable reference across the process boundary — is generated. [Build Architecture](https://morloc-project.github.io/docs/internals/index.md) covers how.


---

<!-- https://morloc-project.github.io/docs/getting-started/parallelism.html -->

# 3.6. Parallelism across languages

Morloc Manual > Getting Started | https://morloc-project.github.io/docs/getting-started/parallelism.html | prev: https://morloc-project.github.io/docs/getting-started/mixing-languages.md | next: https://morloc-project.github.io/docs/getting-started/where-to-go-next.md

Because implementations are interchangeable, so are execution strategies. Here is a parallel `map` written in Python, driving a summation written in C++:

**foo.hpp**

```cpp
#pragma once
#include <vector>

double sum(const std::vector<double>& vec) {
    double sum = 0.0;
    for (double value : vec) {
        sum += value;
    }
    return sum;
}
```

**foo.py**

```python
import multiprocessing as mp

def pmap(f, xs):
    with mp.Pool() as pool:
        results = pool.map(f, xs)
    return results
```

**sums.loc**

```morloc
module sums (sumOfSums)

import root-py
import root-cpp

source Py from "foo.py" ("pmap")
source Cpp from "foo.hpp" ("sum")

pmap :: (a -> b) -> [a] -> [b]
sum :: [Real] -> Real

sumOfSums = sum . pmap sum
```

`sumOfSums` sums a list of lists. The `.` operator is function composition, so this reads right to left: `pmap sum` sums each inner list in parallel, and the outer `sum` adds the results.

The lowercase `a` and `b` in ``pmap’s signature are type variables, meaning `pmap`` works for any element types. That signature is the ordinary ``map’s, fixed to lists, so the two are interchangeable here: writing `map sum`` instead of `pmap sum` compiles and gives the same answer. Parallelism is a choice of implementation, not a change to the program.

```console
$ morloc make sums.loc
$ ./sums sumOfSums '[[1,2],[3,4,5]]'
15
```


---

<!-- https://morloc-project.github.io/docs/getting-started/where-to-go-next.html -->

# 3.7. Where to go next

Morloc Manual > Getting Started | https://morloc-project.github.io/docs/getting-started/where-to-go-next.html | prev: https://morloc-project.github.io/docs/getting-started/parallelism.md | next: https://morloc-project.github.io/docs/getting-started/editor-support.md

You now have the whole shape of Morloc: modules export terms, terms get general types, implementations come from foreign languages or from other Morloc modules, and the compiler generates every interface and every boundary crossing.

From here:

-   [Syntax and Features](https://morloc-project.github.io/docs/features/index.md) is the language proper — records, pattern matching, effects, optionals, and the rest.
-   [Advanced Types](https://morloc-project.github.io/docs/types/index.md) covers typeclasses, polymorphism, and how one term takes many implementations.
-   [Building CLIs](https://morloc-project.github.io/docs/clis/index.md) goes deeper on the command line interface you saw above, including how to control argument shapes and output formats.
-   [Building APIs](https://morloc-project.github.io/docs/apis/index.md) is the same library served over HTTP and MCP.
-   [Modules and Libraries](https://morloc-project.github.io/docs/modules/index.md) explains the standard library and how to publish your own modules.

`mim demos` fetches example programs published for your Morloc version. Every demo in a bundle is known to build and pass on that version, so nothing there fails for reasons unrelated to what you are learning:

```console
$ mim demos --list             # see what is available, download nothing
$ mim demos                    # fetch them all
$ mim demos --tag rust-examples  # fetch one group
```

They land in `examples-<tag>-<version>/` in the current directory.

> **Note**
> The demo collection is still being assembled, so `mim demos` currently answers `no demos are published yet`. The first bundles are expected shortly; until they land, the examples in this manual and the test suite in the compiler repository are the working code to read.


---

<!-- https://morloc-project.github.io/docs/getting-started/editor-support.html -->

# 3.8. Editor support

Morloc Manual > Getting Started | https://morloc-project.github.io/docs/getting-started/editor-support.html | prev: https://morloc-project.github.io/docs/getting-started/where-to-go-next.md | next: https://morloc-project.github.io/docs/getting-started/troubleshooting.md

Morloc is a young language and editor support reflects that. Here is the honest state of each, so you know what you are getting into.

**vim — current**

This is what I use, so it stays current. Install the syntax file and a filetype detection rule:

```console
$ mkdir -p ~/.vim/syntax/
$ mkdir -p ~/.vim/ftdetect/
$ curl -o ~/.vim/syntax/loc.vim https://raw.githubusercontent.com/morloc-project/vimmorloc/main/loc.vim
$ echo 'au BufRead,BufNewFile *.loc set filetype=loc' > ~/.vim/ftdetect/loc.vim
```

![vim highlights](https://morloc-project.github.io/docs/static/img/vim-highlights.png)

**Pygments — current**

[`morloclexer`](https://github.com/morloc-project/pygmentize) is a [Pygments](https://pygments.org/) lexer for Morloc. It is what highlights every code block in this manual, so it tracks the language closely.

```console
$ pip install morloclexer
$ pygmentize -l morloc example.loc
```

It is also usable from Python, which is how the [Weena Discord bot](https://github.com/morloc-project/weena-bot) renders snippets.

**Tree-sitter — out of date**

[tree-sitter-morloc](https://github.com/morloc-project/tree-sitter-morloc) is a full grammar for Morloc: a complete lexer and parser specification, which gives editors real structural understanding rather than regex highlighting, and parses a concrete syntax tree you can query.

The grammar has fallen behind the compiler and does not cover current syntax. Treat it as a starting point rather than a working tool. Bringing it back into step is on the list, and pull requests are very welcome.

![tree sitter](https://morloc-project.github.io/docs/static/img/tree-sitter.png)

**VS Code / VSCodium / Cursor — out of date**

There is a published `morloc` extension with highlighting and snippet expansion. It has not been updated in a while and does not know about recent syntax, so expect gaps.

![vscode highlights](https://morloc-project.github.io/docs/static/img/vscode-highlights.png)

**Zed — out of date, unfinished**

[zed-morloc](https://github.com/morloc-project/zed-morloc) is mostly written and depends on the Tree-sitter grammar above, which means it inherits that grammar’s staleness on top of its own unresolved bugs. I am happy to accept pull requests!


---

<!-- https://morloc-project.github.io/docs/getting-started/troubleshooting.html -->

# 3.9. Troubleshooting

Morloc Manual > Getting Started | https://morloc-project.github.io/docs/getting-started/troubleshooting.html | prev: https://morloc-project.github.io/docs/getting-started/editor-support.md | next: https://morloc-project.github.io/docs/features/index.md

The problems new users actually hit, and what to do about each.

**A command is not found**

`mim: command not found` means the install directory is not on your `PATH`. The installer never edits your shell startup files; it prints the exact command for your shell when the destination is missing, and you have to open a new shell afterwards. On macOS this is the normal case rather than the exception, because the default `PATH` is built from `/etc/paths`, which does not include `~/.local/bin`.

`morloc: command not found` is different and usually means the install worked. `morloc` lives inside an environment, not on your host `PATH`. Reach it with `mim run — morloc …​`, or open a `mim shell` and drop the prefix. `mim` is the only executable that lands on your `PATH`.

**Behind a corporate firewall**

A TLS-inspecting proxy re-signs every HTTPS connection with a private CA, so downloads fail certificate verification and `mim new` stops partway through. Point `mim` at your organization’s CA:

```console
$ mim new base --cert-bundle /path/to/corp-ca.pem
$ mim modify --env base --cert-bundle /path/to/corp-ca.pem   # after it rotates
```

The file is PEM or DER and holds your CA certificates only — `mim` supplies the public roots itself. Everything it runs that fetches — `curl`, pixi, conda, git, cargo, the language runtimes — is then pointed at the result through `SSL_CERT_FILE`, `REQUESTS_CA_BUNDLE`, `CURL_CA_BUNDLE`, `CONDA_SSL_VERIFY`, `NODE_EXTRA_CA_CERTS`, `GIT_SSL_CAINFO` and `CARGO_HTTP_CAINFO`.

Obtaining the certificate is your IT department’s business, not Morloc’s, and `--cert-bundle` is the whole of Morloc’s interface to it. It is usually already on the machine. On macOS, admin-installed certificates are in the system keychain, separate from Apple’s public roots:

```console
$ security find-certificates -a -p /Library/Keychains/System.keychain > corp-ca.pem
```

On Linux, if the host already trusts the CA it is in the system store. Prefer the drop-in directory: it holds the certificates your organization added, where the trusted bundle mixes them in with several hundred public roots.

| Distribution | Trusted bundle | Drop-in directory |
| --- | --- | --- |
| Debian, Ubuntu | `/etc/ssl/certs/ca-certificates.crt` | `/usr/local/share/ca-certificates/` |
| RHEL, Fedora | `/etc/pki/tls/certs/ca-bundle.crt` | `/etc/pki/ca-trust/source/anchors/` |
| Arch | `/etc/ssl/certs/ca-certificates.crt` | `/etc/ca-certificates/trust-source/anchors/` |
| SUSE | `/etc/ssl/ca-bundle.pem` | `/etc/pki/trust/anchors/` |

Those directories hold loose `.crt` files and `--cert-bundle` takes one file, so concatenate them when there is more than one:

```console
$ cat /usr/local/share/ca-certificates/*.crt > corp-ca.pem
```

`mim` validates the file before it builds anything and prints what it found: per certificate, the subject, whether it is a CA, whether it is self-signed, the validity dates and a SHA-256 fingerprint. It refuses the file when:

-   it is empty, or over 1 MiB — a CA bundle is a handful of certificates, so this is almost always the wrong file
-   it contains private key material — export the certificate, not the key
-   it is DER but not an X.509 certificate, such as a key or a PKCS#12 archive
-   it is text — most often a proxy error page saved with a `.pem` extension
-   nothing in it decodes as a certificate

An expired or not-yet-valid certificate is reported as a warning rather than a refusal, because a skewed system clock looks identical from here. PEM blocks that fail to parse are skipped and named, not treated as fatal.

Afterwards `mim doctor` compares the source file against the fingerprints the environment was built with, which is how you find out the CA rotated under you.

**The first build takes several minutes**

Expected. `mim new` downloads the Morloc compiler, solves a conda toolchain and builds the Morloc runtime from source against it. Your first `morloc make` in a language you have not used yet pauses again to provision that language. Both results are cached: later environments reuse the downloaded compiler, and a solve with unchanged requirements is skipped entirely.

**An import fails on a fresh environment**

`morloc make` fetches a missing module; `morloc typecheck`, `dump` and `eval` do not. Build the program once, or run `morloc install <module>` by hand, and the import resolves for every command afterwards. [Writing code that is not tied to a language](https://morloc-project.github.io/docs/getting-started/abstract-modules.md) covers this.

If the fetch itself fails rather than being skipped, it is a network problem — see the firewall entry above.

**Something else**

```console
$ mim doctor                # check the default environment
$ mim doctor --env base     # or a named one
$ mim doctor --deep         # also run checks inside the container; slower
```

`--strict` makes it exit non-zero on warnings, which is what you want in CI.


---

<!-- https://morloc-project.github.io/docs/features/ -->

# 4. Syntax and Features

Morloc Manual | https://morloc-project.github.io/docs/features/ | prev: https://morloc-project.github.io/docs/getting-started/troubleshooting.md | next: https://morloc-project.github.io/docs/features/functions.md


---

<!-- https://morloc-project.github.io/docs/features/functions.html -->

# 4.1. Functions

Morloc Manual > Syntax and Features | https://morloc-project.github.io/docs/features/functions.html | prev: https://morloc-project.github.io/docs/features/index.md | next: https://morloc-project.github.io/docs/features/foreign-functions.md

Everything in Morloc is built out of functions, so this is where to start. This section covers how they are defined, composed, and partially applied.

Most examples in this chapter are **fragments** — a definition or two, without the surrounding module. To run one, wrap it in a module and import implementations:

```morloc
module demo (myTerm)

import root-py    -- or root-cpp, root-r

myTerm = ...
```

Comments start with `--`. A `--'` comment is a docstring and attaches to the term below it.

## 4.1.1. Definition and application

Functions are defined with their arguments separated by whitespace, and applied the same way:

```morloc
foo x y z = g x (f y z)
```

`foo` takes the arguments `x`, `y`, and `z`. Application binds tighter than anything else, so `g x (f y z)` calls `g` with two arguments: `x`, and the result of `f y z`. If you have a background in the Algol family — C, Python, Java — the missing parentheses and commas take a little getting used to. The payoff shows up in the next two sections.

## 4.1.2. Composition and application operators

The `internal` module, re-exported from `root`, defines the composition operator `.` and the application operator `$`.

`.` glues two functions into one. These two definitions mean the same thing:

```morloc
foo1 x = g (f x)
foo2 = g . f
```

The first passes the output of `f x` into `g` explicitly. The second says the same thing without naming the argument at all — `foo2` **is** `g` after `f`. Composition chains read right to left and build pipelines cleanly:

```morloc
process = format . transform . validate . parse
```

`$` is application with the lowest possible precedence, which makes it a way to delete parentheses:

```morloc
foo1 x = h (g (f x))
foo2 x = h $ g $ f x
```

## 4.1.3. Partial application

Give a function of N arguments fewer than N, and you get back a function of the rest. This is not a special feature; it falls out of how application works.

Take `fold`, which reduces a container with a binary function, an initial value, and the container itself:

```morloc
fold :: Foldable f => (b -> a -> b) -> b -> f a -> b
```

Supplying one or two of those three arguments leaves a function behind:

```morloc
-- concatenate a list of strings onto an initial value
concatTo :: Str -> [Str] -> Str
concatTo = fold (<>)

-- extend an initial list
extend :: [[Int]] -> [Int]
extend = fold (<>) [1,2,3]

-- append a list of values to an initial value
append :: [[[Int]]] -> [Int] -> [[Int]]
append xss ys = map (fold (<>) ys) xss
```

Each of these carries a type signature, and that is not decoration. `fold` is a typeclass method: it works over any `Foldable` container, so a partial application like `fold (<>)` leaves the container type undetermined. Without a signature the compiler has nothing to pin it to and reports:

```console
$ morloc typecheck partial.loc
partial.loc:6:12: error:
General type error: No instance found for Foldable::fold
  Are you missing a top-level type signature?
  |
6 | concatTo = fold (<>)
  |            ^
```

The rule is worth internalizing early, because it is the most common thing to trip over: **a point-free definition built from typeclass methods usually needs a signature.** Adding arguments back is the other fix — `concatTo x xs = fold (<>) x xs` typechecks without help, because the arguments constrain the types.

## 4.1.4. Operator sections

Binary operators partially apply too, on either side. Leaving the right operand off gives a function of the right operand:

```morloc
divideByTwo :: [Real] -> [Real]
divideByTwo = map (/ 2.0)
```

and leaving the left operand off gives a function of the left:

```morloc
divideTwoBy :: [Real] -> [Real]
divideTwoBy = map (2.0 /)
```

The difference shows up immediately:

```console
$ ./sections divideByTwo '[1,2,3]'
[0.5,1,1.5]
$ ./sections divideTwoBy '[1,2,4]'
[2,1,0.5]
```

Numeric literals are not polymorphic across `Int` and `Real`, so `2.0` keeps these on `Real`. For integer division use `//`, which is defined on `Int`:

```morloc
halvedInts :: [Int] -> [Int]
halvedInts = map (// 2)
```

```console
$ ./sections halvedInts '[1,5,9]'
[0,2,4]
```

## 4.1.5. Lambdas

An anonymous function is a backslash, one or more parameters, `→`, and a body. Lambdas capture free variables from the enclosing scope:

```morloc
addBias :: Real -> [Real] -> [Real]
addBias bias = map (\x -> x + bias)
```

`bias` comes from the outer parameter list and is captured by the lambda.

```console
$ ./sections addBias 10 '[1,2]'
[11,12]
```

A lambda must take at least one argument. The zero-argument form is a parse error:

```console
$ morloc typecheck five.loc
five.loc:6:10: unexpected '->'
    |
  6 | five = \ -> 5
    |          ^
```

To wrap a value as a computation to be run later, use the effect system rather than a lambda — see [Effects and delayed evaluation](https://morloc-project.github.io/docs/features/effects.md).


---

<!-- https://morloc-project.github.io/docs/features/foreign-functions.html -->

# 4.2. Foreign functions

Morloc Manual > Syntax and Features | https://morloc-project.github.io/docs/features/foreign-functions.html | prev: https://morloc-project.github.io/docs/features/functions.md | next: https://morloc-project.github.io/docs/features/booleans.md

A Morloc module by itself declares types and compositions but contains no implementations. Those come from other languages, pulled in with `source`. This is the mechanism the rest of the language is built on.

## 4.2.1. The `source` statement

`source` names a language, a file, and the terms to take from it:

```morloc
source Cpp from "foo.hpp" ("map", "sum", "snd")
source Py from "foo.py" ("morloc_map" as map, "morloc_sum" as sum, "snd")
```

The language tag (`Cpp`, `Py`, `R`, `Rust`) decides which toolchain compiles the code and which pool the function runs in. `as` renames a foreign term for use in Morloc, which matters when the foreign name is taken or awkward.

### Block form

There is a second spelling. `source …​ where` opens an indented block with one term per line, which leaves room for a docstring above each:

```morloc
source Py from "foo.py" where
  --' Sum a list of reals.
  sum
  --' name: morloc_map
  map
```

The two forms are otherwise equivalent — pick whichever reads better.

`--' name: <foreign name>` says what the term is called on the other side, so the Morloc name and the foreign name can differ. It is the block-form equivalent of `as`, and these two declare the same thing:

```morloc
source Py from "foo.py" ("morloc_map" as map)

source Py from "foo.py" where
  --' name: morloc_map
  map
```

Prose docstring lines are allowed too and are carried as documentation.

**Curried foreign functions: the rsize directive**

Skip this unless a foreign function returns a closure rather than taking all its arguments at once. It assumes nothing beyond the section above.

In a pure functional language, `a → (b → c)` and `a → b → c` are the same type. A function of two arguments **is** a function of one argument returning a function of one argument; currying makes the distinction vanish. Morloc’s type system takes that view — the two spellings are interchangeable, and the compiler reports both the same way.

Real languages usually do not. In Python the two are different objects with different call syntax:

**curry.py**

```python
# one argument, returns a closure:  scale(2.0)([1, 2])
def scale(factor):
    return lambda xs: [factor * x for x in xs]

# two arguments, called at once:    shift(10.0, [1, 2])
def shift(offset, xs):
    return [offset + x for x in xs]
```

Both have the Morloc type `Real → [Real] → [Real]`, and nothing in that type says which shape the Python side has. By default Morloc assumes the flat one and emits a single call with every argument. Hand it `scale` and the pool dies:

```console
TypeError: scale() takes 1 positional argument but 2 were given
```

`--' rsize: N` declares how many arguments go in each call. The values are the sizes of the leading call groups; the final group is whatever is left over, so you never write it.

```morloc
source Py from "curry.py" where
  --' rsize: 1
  scale
  shift

scale :: Real -> [Real] -> [Real]
shift :: Real -> [Real] -> [Real]
```

A docstring attaches to the term directly below it, so `scale` is curried here and `shift` is not. The generated pool shows the difference:

```python
n2 = curry.scale(n0) (n1)
n4 = curry.shift(n2, n3)
```

Both work, and partial application works through a curried source too:

```console
$ ./curry scaled 3 '[1,2]'
[3,6]
$ ./curry shifted 10 '[1,2]'
[11,12]
$ ./curry doubler '[1,2]'
[2,4]
```

where `doubler = scale 2.0` compiles to `curry.scale(2.0) (n4)`.

For deeper nesting, give one value per leading group:

| Declaration | Foreign shape | Emitted call |
| --- | --- | --- |
| *(none)* | `f(a, b, c)` | `f(a, b, c)` |
| `rsize: 1` | `f(a)(b, c)` | `f(a) (b, c)` |
| `rsize: 2` | `f(a, b)(c)` | `f(a, b) (c)` |
| `rsize: 1 1` | `f(a)(b)(c)` | `f(a) (b) (c)` |

Each value must be at least 1 and must leave at least one argument for the group after it, since the final group is implicit. So `rsize: 2` on a two-argument function is rejected — it would consume both arguments and leave an empty call behind.

`rsize` is the only place the curried-versus-flat distinction is recorded. Writing the type as `Real → ([Real] → [Real])` does not imply it and does not change the emitted call.

The C++ side is an ordinary header:

**foo.hpp**

```cpp
#pragma once
#include <vector>
#include <tuple>

// map :: (a -> b) -> [a] -> [b]
template <typename F, typename A>
auto map(F f, const std::vector<A>& xs) {
    std::vector<decltype(f(xs.front()))> result;
    result.reserve(xs.size());
    for (const auto& x : xs) {
        result.push_back(f(x));
    }
    return result;
}

// snd :: (a, b) -> b
template <typename A, typename B>
B snd(const std::tuple<A, B>& p) {
    return std::get<1>(p);
}

// sum :: [a] -> a
template <typename A>
A sum(const std::vector<A>& xs) {
    A total = A{0};
    for (const auto& x : xs) {
        total += x;
    }
    return total;
}
```

These implementations are completely independent of Morloc. They have no special constraints, they operate on ordinary native data structures, and nothing stops them being used outside Morloc entirely. That independence is the point: Morloc consumes libraries as they already exist.

## 4.2.2. General types

Morloc moves data between languages, and to do that it needs to know the shape of each function. You supply that as a **general type signature**:

```morloc
map :: (a -> b) -> [a] -> [b]
snd :: (a, b) -> b
sum :: [Real] -> Real
```

The syntax is borrowed from Haskell. Square brackets are homogeneous lists, parenthesized comma-separated values are tuples, and arrows are functions. In `map`, `(a → b)` is a function from a generic `a` to a generic `b`, `[a]` is the input list, and `[b]` is the output. `snd` pulls the second element out of a two-tuple. `sum` reduces a list of reals to one real.

The brackets are sugar. Written out, the same signatures are:

```morloc
map :: (a -> b) -> List a -> List b
snd :: Tuple2 a b -> b
sum :: List Real -> Real
```

## 4.2.3. Native type mappings

A general type may correspond to a different concrete type in every language, so you also give the mapping:

```morloc
type Cpp => List a = "std::vector<$1>" a
type Cpp => Tuple2 a b = "std::tuple<$1,$2>" a b
type Cpp => Real = "double"
type Py => List a = "list" a
type Py => Tuple2 a b = "tuple" a b
type Py => Real = "float"
```

These are type **functions**. Take the C++ mapping for `List a`. Once the typechecker has solved for the parameter `a` and recursively converted it to C++, that result is substituted for `$1`. If `a` turns out to be `Real`, it maps to `double`, which substitutes into the list type to give `std::vector<double>` — and that is the type in the generated C++.

In practice you rarely write these. They come from foundational modules such as `root-cpp` and `root-py`, which is why the examples in [Getting Started](https://morloc-project.github.io/docs/getting-started/index.md) could import a language and start working.

With the signatures and mappings in place, the module compiles and runs:

```console
$ ./foreign mySum '[1,2,3.5]'
6.5
$ ./foreign mySnd '[1,2]'
2
```

Note that a tuple is written as a JSON array on the command line.

Higher-order functions cross the boundary too. A Morloc function passed into the sourced C++ `map` works exactly as you would hope:

```morloc
doubleAll :: [Real] -> [Real]
doubleAll = map twice
```

```console
$ ./foreign doubleAll '[1,2,3]'
[2,4,6]
```

## 4.2.4. Sourcing builtins and other non-exports

Morloc calls a sourced Python term as an attribute of its module: a term taken from `foo.py` is invoked as `foo.<name>`. Builtins are not attributes of `foo`, so sourcing one directly compiles fine and then fails when it runs:

```morloc
source Py from "foo.py" ("map", "sum", "snd")
```

```console
$ ./foreign mySum '[1,2,3.5]'
Error: run failed
module 'foo' has no attribute 'sum'
  at mySum [py] (mid=1, foreign.loc:1:17)
```

The failure is deferred to run time, which makes it worth knowing about in advance. There are two fixes.

Re-export the builtins so they become module attributes:

**foo.py**

```python
from builtins import map, sum  # make builtins module-level attributes

def snd(pair):
    return pair[1]
```

Or wrap them under names of your own and rename on the way in, which is where the `morloc_sum` in the first example came from:

**foo.py**

```python
def morloc_sum(xs):
    return sum(xs)

def snd(pair):
    return pair[1]
```

```morloc
source Py from "foo.py" ("morloc_sum" as sum, "snd")
```

Both work. The same rule applies to anything that is not a module-level name: a term from a third-party package must be locally defined (`def bar(…​)`) or explicitly imported (`from somemodule import bar`) in the sourced file.

## 4.2.5. Keyword-shaped foreign operators

Some foreign symbols are neither callable identifiers nor symbolic operators. Python’s `and` and `or` are language keywords: they exist only as infix syntax, so `and(x, y)` is a parse error and there is no function object to import.

A backtick-quoted name sources such a symbol as an infix operator whose emitted text is the quoted string:

```morloc
source Py from "core.py" (`and` as (&&), `or` as (||))

(&&) :: Bool -> Bool -> Bool
(||) :: Bool -> Bool -> Bool
```

At each call site the generated pool writes the quoted text between the two arguments. No wrapper is needed on the Python side — `core.py` can be empty. The generated `pool.py` for `x && y` and `x || y` contains:

```python
n2 = (n0 and n1)
...
n4 = (n2 or n3)
```

The backtick contents are emitted verbatim, so any two-argument infix operator the target language recognises works the same way: Python `is`, `in`, `not in`, R `%in%`, and so on.


---

<!-- https://morloc-project.github.io/docs/features/booleans.html -->

# 4.3. Booleans

Morloc Manual > Syntax and Features | https://morloc-project.github.io/docs/features/booleans.html | prev: https://morloc-project.github.io/docs/features/foreign-functions.md | next: https://morloc-project.github.io/docs/features/integers.md

Booleans are written `True` and `False` and have the type `Bool`. The comparison and logical operators come from `root`, so a module that uses them imports one of the `root` implementations.

```morloc
yes :: Bool
yes = True

no :: Bool
no = False
```

The literals are capitalized, but a `Bool` prints as lowercase JSON:

```console
$ ./bools yes
true
```

## 4.3.1. Comparison operators

The `Eq` and `Ord` typeclasses in `root` provide the standard comparisons. They work over any type with the appropriate instance: integers, reals, strings, and tuples and lists of comparable values.

| Operator | Meaning |
| --- | --- |
| `==` | equal |
| `!=` | not equal |
| `<` | less than |
| `⇐` | less than or equal |
| `>` | greater than |
| `>=` | greater than or equal |

```morloc
isPositive :: Int -> Bool
isPositive x = x > 0

sameLength :: [a] -> [b] -> Bool
sameLength xs ys = length xs == length ys
```

`sameLength` is generic in both list types, which is fine inside a program but means it cannot be given a command line interface — the compiler cannot decide how to read an argument whose type is still a variable. Exporting it produces:

```console
$ morloc make bools.loc
Warning: skipping generic export 'sameLength'
```

The program still builds; only that one command is absent.

## 4.3.2. Logical operators

| Operator | Meaning |
| --- | --- |
| `&&` | logical AND |
| `\|\|` | logical OR |
| `not` | logical negation (a prefix function, not an operator) |
| `xor` | exclusive OR |
| `nand` | NOT AND |

Both `&&` and `||` are right-associative, and `&&` binds tighter than `||` (`infixr 3 &&` against `infixr 2 ||`), which matches the convention in most languages. So `a || b && c` groups as `a || (b && c)`.

```morloc
inRange :: Int -> Int -> Int -> Bool
inRange lo hi x = lo <= x && x <= hi

isWeekend :: Int -> Bool
isWeekend day = day == 0 || day == 6

isWeekday :: Int -> Bool
isWeekday day = not (isWeekend day)
```

```console
$ ./bools inRange 1 10 5
true
$ ./bools isWeekday 6
false
```

### Short-circuiting

`&&` and `||` short-circuit at run time: if the left operand settles the answer, the right one is never evaluated. This is worth demonstrating rather than asserting, because it is not obvious in a language where the two operands may run in different processes.

```morloc
divZero :: Int -> Int
divZero x = x // 0

-- b comes from the caller, so the compiler cannot fold this away
test :: Bool -> Int -> Bool
test b x = b && (divZero x == 0)
```

With `b` false, the division never happens:

```console
$ ./shortcircuit test false 5
false
```

With `b` true, it does, and the error surfaces with the call chain that produced it:

```console
$ ./shortcircuit test true 5
Error: run failed
integer division or modulo by zero
  at _ [py] (mid=1364, shortcircuit.loc:10:28)
  at test [py] (mid=1, shortcircuit.loc:1:22)
```

## 4.3.3. Boolean-valued list functions

`root` provides three `Foldable` functions that answer questions about a container:

| Function | Signature |
| --- | --- |
| `any` | `Foldable f ⇒ (a → Bool) → f a → Bool` |
| `all` | `Foldable f ⇒ (a → Bool) → f a → Bool` |
| `elem` | `(Foldable f, Eq a) ⇒ a → f a → Bool` |

`any` is `True` when the predicate holds for at least one element, `all` when it holds for every element, and `elem` tests membership using `==`.

```morloc
hasNegative :: [Int] -> Bool
hasNegative = any (< 0)

allPositive :: [Int] -> Bool
allPositive = all (> 0)

containsZero :: [Int] -> Bool
containsZero = elem 0
```

```console
$ ./bools hasNegative '[1,-2,3]'
true
$ ./bools containsZero '[1,0,3]'
true
```

## 4.3.4. Guards

Booleans drive Morloc’s guard syntax. A guard alternative starts with `?` and selects the first branch whose condition is `True`; the `:` line is the fallthrough:

```morloc
classify :: Int -> Str
classify x
  ? x < 0     = "negative"
  ? x == 0    = "zero"
  : "positive"
```

```console
$ ./bools classify -4
"negative"
$ ./bools classify 0
"zero"
$ ./bools classify 7
"positive"
```

See [Conditionals](https://morloc-project.github.io/docs/features/conditionals.md) for the full description of guard syntax.


---

<!-- https://morloc-project.github.io/docs/features/integers.html -->

# 4.4. Integer types

Morloc Manual > Syntax and Features | https://morloc-project.github.io/docs/features/integers.html | prev: https://morloc-project.github.io/docs/features/booleans.md | next: https://morloc-project.github.io/docs/features/floats.md

Morloc has one integer type for ordinary use and a family of fixed-width types for when the width matters. This section covers how integers are written, how the default type behaves across languages, and what happens at the boundaries.

## 4.4.1. Writing integer literals

Integers may be written in decimal, hexadecimal, octal, or binary:

```morloc
-- standard decimal notation
42

-- hexadecimal notation (case insensitive)
0xf00d
0xDEADBEEF

-- octal notation (upper or lowercase 'o')
0o755

-- binary notation (upper or lowercase 'b')
0b0101
```

A prefixed literal must contain only digits valid for its base and must end on a non-identifier character. A trailing character that is not a valid digit for the base is a compile-time error, not a silently truncated literal followed by an unrelated identifier:

```console
$ morloc eval -e "0xF00D"
61453
$ morloc eval -e "0xF0OD"
<expr>:1:1: malformed hexadecimal literal: 0xF0OD
$ morloc eval -e "0b1001"
9
$ morloc eval -e "0o755"
493
```

`morloc eval` evaluates a single expression, which makes it a good way to check one of these rules. It has no implicit prelude, so anything beyond a bare literal needs an import:

```console
$ morloc eval -e '5 - 1'
<expr>:1:3: error:
Undefined term: -
hint: an eval expression has no implicit prelude; prefix the expression with 'import root-py;' (or the module that defines -) to bring it into scope
$ morloc eval -e 'import root-py; 5 - 1'
4
```

## 4.4.2. Integer types at a glance

| Type | Width | Use case |
| --- | --- | --- |
| `Int` | Variable (arbitrary precision) | Default integer for most code. Works across all languages. |
| `I8`, `I16`, `I32`, `I64` | 8, 16, 32, 64 bits (signed) | Performance-critical code with known bounds. |
| `U8`, `U16`, `U32`, `U64` | 8, 16, 32, 64 bits (unsigned) | Bit manipulation, byte data, indices. |

## 4.4.3. The default `Int` type

`Int` is Morloc’s universal integer, and integer literals are `Int` unless something says otherwise:

```morloc
x = 42          -- Int
y = 0xDEADBEEF  -- Int (hex literal)
z = -9999       -- Int
```

On the wire `Int` is variable-width: values up to 64 bits fit in 16 bytes inline, and larger values spill to a pointer to an array of 64-bit limbs. But the range you actually get **inside** a language is whatever that language’s native binding provides:

| Language | Native binding for `Int` | Representable range |
| --- | --- | --- |
| Python | `int` | Arbitrary precision |
| C++ | `int` | 32-bit signed (`-2^31` to `2^31 - 1`) |
| R | `integer` | 32-bit signed |

This asymmetry is the thing to remember about `Int`. A value that a Python pool holds happily may not fit in the C++ or R pool it is handed to. If a field needs more than 32 bits on those backends, declare it `I64` or `U64`, which map to `int64_t` in C++ and to R’s `numeric` (53-bit integer precision via double).

## 4.4.4. Big integers from Python

Python’s integers are arbitrary precision and Morloc’s `Int` takes full advantage of that. Factorials make the point quickly:

**main.loc**

```morloc
module main (fact)

import root-py

fact :: Int -> Int
fact n
  ? n == 0 = 1
  : n * fact (n - 1)
```

```console
$ morloc make -o calc main.loc
$ ./calc fact 100
93326215443944152681699238856266700490715968264381621468592963895217599993229915608941463976156518286253697920827223758251185210916864000000000000000000000000
```

That is a 525-bit integer, far past any fixed-width type. It is stored as a multi-limb big integer and printed exactly.

## 4.4.5. Overflow at a language boundary

When a value too large for the target language’s type crosses into it, Morloc raises an error at the boundary rather than truncating silently.

To show this we need to force the computation to happen in Python and then move the result. `root-py` exports `idpy` and `root-cpp` exports `idcpp`: identity functions pinned to one language. Wrapping a term in `idpy` forces it into the Python pool, and `idcpp` then drags the result across into C++. Without them the compiler would collapse `fact` to pure C++ — faster, but it would not demonstrate anything.

**main.loc**

```morloc
module main (factCpp, factR)

import root-py
import root-cpp
import root-r

fact :: Int -> Int
fact n
  ? n == 0 = 1
  : n * fact (n - 1)

factPy :: Int -> Int
factPy n = idpy (fact n)

factCpp :: Int -> Int
factCpp x = idcpp (factPy x)

factR :: Int -> Int
factR x = idr (factPy x)
```

Small values cross without trouble:

```console
$ ./calc factCpp 5
120
```

Large ones report where and why they failed:

```console
$ ./calc factCpp 100
Error: run failed
Integer overflow: 9-limb integer (576 bits) does not fit in 32-bit type (range -2147483648 to 2147483647)
  at _ [cpp] (mid=2787, main.loc:16:20)
  at factCpp [cpp] (mid=1, main.loc:1:14)
```

R is limited to 32-bit integers, and to 53-bit integer precision through doubles, so it refuses the same value:

```console
$ ./calc factR 100
Error: run failed
Integer overflow: 9-limb integer (576 bits) does not fit in R's numeric type (max 2^53 for integer precision).
  at _ [r] (mid=2815, main.loc:19:16)
  at factR [r] (mid=2, main.loc:1:23)
```

Both report the same shape: what overflowed, what it would not fit in, and the call chain that got there.

## 4.4.6. Compile-time literal bounds

A literal written into a fixed-width type is bounds-checked against that type:

```morloc
tooLarge :: U8
tooLarge = 1000
```

The check happens during code generation, so `morloc typecheck` passes and `morloc make` is what rejects it:

```console
$ morloc typecheck intbounds.loc
tooLarge :: U8
$ morloc make intbounds.loc
intbounds.loc:6:12: error:
Integer literal 1000 overflows U8 (range 0 to 255)
  |
6 | tooLarge = 1000
  |            ^
```

The caret points at the literal, not at the binding name, so when the same literal is referenced from several sites the diagnostic stays on the offending source.

## 4.4.7. Fixed-width integer types

When values are known to be bounded, fixed-width types map directly onto the target language’s native types:

| Morloc type | C++ | Python | R |
| --- | --- | --- | --- |
| `I8` | `int8_t` | `int` | `integer` |
| `I16` | `int16_t` | `int` | `integer` |
| `I32` | `int32_t` | `int` | `integer` |
| `I64` | `int64_t` | `int` | `numeric` (double) |
| `U8` | `uint8_t` | `int` | `raw` |
| `U16` | `uint16_t` | `int` | `integer` |
| `U32` | `uint32_t` | `int` | `numeric` (double) |
| `U64` | `uint64_t` | `int` | `numeric` (double) |

These serialize directly: the wire format is identical to the in-memory representation, with no conversion step. That makes them the right choice for numerical code and for interop with C libraries that require specific widths.

> **Note**
> The Python column is `int` throughout rather than a genuinely fixed-size type such as a numpy scalar. Types can be specialized that way; see [Native type mappings](https://morloc-project.github.io/docs/features/foreign-functions.md#mapping-native-types), and [Tensors](https://morloc-project.github.io/docs/types/tensors.md) and [Tables](https://morloc-project.github.io/docs/types/tables.md) for the higher-performance shared-memory types.

## 4.4.8. Converting between integer types

Two typeclasses in `root` cover numeric conversion. `into` is for conversions that can never fail and never lose information. `tryInto` is for everything else:

```morloc
class TotalInto a b where
  into :: a -> b

class PartialInto a b where
  tryInto :: a -> b
```

Widening is total — signed to wider signed, unsigned to wider unsigned, and unsigned into a strictly wider signed target. A reflexive `TotalInto a a` instance covers the identity case.

```morloc
wide :: I8 -> I64
wide x = into x
```

Anything that can fail goes through `tryInto`: narrowing, negative into unsigned, or unsigned into a same-or-narrower signed target. Its signature looks total because every instance is a conversion written in a backend language, and it reports a value that does not fit by raising there:

```morloc
byte :: I32 -> U8
byte x = tryInto x
```

```console
$ ./bytes byte 65
65
$ ./bytes byte 9999
Error: run failed
value 9999 out of range [0, 255]
  at byte [py] (mid=2, bytes.loc:1:20)
```

To decide for yourself what an out-of-range value means, wrap the conversion in `@try`, which turns a raise into a value you can match on — either the converted number or the reason there isn’t one. `@try` and the `Try` type it produces are covered in [Failure and recovery](https://morloc-project.github.io/docs/features/intrinsics.md#failure-and-recovery); the shape is:

```morloc
byteOrZero :: I32 -> U8
byteOrZero x = match (@try (tryInto x :: U8))
  | (Ok b)  = b
  | (Err _) = 0

byteOrReport :: I32 -> Str
byteOrReport x = match (@try (tryInto x :: U8))
  | (Ok b)  = "fits: #{@show b}"
  | (Err e) = "does not fit: #{e}"
```

```console
$ ./bytes byteOrZero 65
65
$ ./bytes byteOrZero 9999
0
$ ./bytes byteOrReport 9999
"does not fit: value 9999 out of range [0, 255]"
```

**The \`**

U8\` ascription is doing the work the old signature used to: `tryInto` is polymorphic in its target, so something has to say which conversion you meant.

`Int` gets the most restrictive treatment, because its width varies by backend: 32-bit in R and C++, unbounded in Python. Every `Int` to fixed-width conversion goes through `tryInto` — even `Int → I64` — and converting `U32` or wider **into** `Int` does too. That keeps behaviour the same everywhere.

## 4.4.9. Negation and unary minus

The `-` glyph plays two roles: binary subtraction and unary negation. Which one you get depends on whitespace.

```morloc
-- prefix `-` on a value: the additive inverse
neg :: Int -> Int
neg x = -x

-- prefix `-` on an expression: parenthesize the expression
shifted :: Int -> Int
shifted x = -(x + 1)

-- works on any numeric primitive (Int, I8..I64, U8..U64,
-- Real, F32, F64) via the `Negatable` typeclass
flipReal :: Real -> Real
flipReal x = -x
```

### Negative literals

A `-` directly against a digit, with no space between, is part of the literal. So `-1` is an atomic integer rather than a function call, and works in places where calls are not allowed, such as pure-data files:

```morloc
xs :: [Int]
xs = [-1, -2, -3, -100]

ys :: [Real]
ys = [-1.5, -2.0e-3, -0xff]

point :: (Int, Int)
point = (-3, -4)
```

```console
$ ./neg xs
[-1,-2,-3,-100]
$ ./neg ys
[-1.5,-0.002,-255]
$ ./neg point
[-3,-4]
```

The same atomic-lexing rule extends to the non-finite `Real` literals `-Inf` and `-NaN`; see [Floating-point types](https://morloc-project.github.io/docs/features/floats.md).

### When `-` is unary and when it is binary

The lexer uses an asymmetric-whitespace rule. A `-` immediately followed by a digit is part of a negative literal whenever the dash sits where an expression cannot have just ended:

-   at the start of input;
-   after an opening delimiter (`(`, `[`, `,`, `=`, and so on);
-   after another operator;
-   after whitespace, when the digit is not separated from the dash.

Anywhere else — where the dash directly follows a token that finishes an operand, with no whitespace between — it is binary subtraction.

| Expression | Interpretation |
| --- | --- |
| `-1` | atomic literal `-1` |
| `f -1` | `f` applied to literal `-1` (asymmetric whitespace) |
| `f - 1` | binary subtraction `f - 1` (symmetric whitespace) |
| `f-1` | binary subtraction `f - 1` (no whitespace) |
| `[-1, -2]` | list of two negative literals |
| `1 + -2` | `1 + (-2)`; the `-2` is a literal |
| `-(x + 1)` | desugars to `negate (x + 1)` |
| `-x` | desugars to `negate x` |

The first row of that table is easy to verify. Applying a number to something is a type error, and that is exactly the error `5 -1` produces — proving the `-1` was read as an argument rather than as subtraction:

```console
$ morloc eval -e "5 -1"
<expr>:1:1: error:
General type error: Application of non-functional expression of type: Int
```

**With \`f**

Int → Int\` defined as `f x = x * 10`, the three spellings behave as the table says:

```console
$ ./dashtest t1     -- t1 = f -1
-10
$ ./dashtest t2     -- t2 = 100 - 1
99
$ ./dashtest t3     -- t3 = 100-1
99
```

### Position restrictions

Prefix `-` on a non-literal expression is allowed wherever an expression can begin, including on the right of an infix operator. The one restriction is that its operand must start with an atom — an identifier, a literal, an open paren or bracket — and not with another prefix `-`.

```morloc
-- ok: -x at the start of an expression
neg1 :: Int -> Int
neg1 x = -x

-- ok: -x on the right of a binary operator
neg2 :: Int -> Int
neg2 x = 1 + -x

-- ok: subtracting a negated value
neg3 :: Int -> Int -> Int
neg3 x y = x - -y

-- ok: -x parenthesized; equivalent to neg2
neg4 :: Int -> Int
neg4 x = 1 + (-x)

-- ok: parenthesize the inner negation to stack two
double :: Int -> Int
double x = -(-x)
```

Two adjacent prefix dashes are a parse error:

```console
$ morloc typecheck negbad.loc
negbad.loc:6:11: unexpected '-'
    |
  6 | bad x = - -x
    |           ^
```

### The `Negatable` typeclass

Negation comes from a typeclass in the `internal` module:

```morloc
class Negatable a where
  negate :: a -> a
```

Every numeric primitive has an instance in `root-py`, `root-cpp`, and `root-r` that dispatches to the host language’s native unary minus. The parser desugars `-x` to `negate x`, so writing `negate x` yourself is equivalent. The compiler picks the language for a negation the same way it picks the language for any other polymorphic call: from the imported language modules and the surrounding cross-language boundaries.


---

<!-- https://morloc-project.github.io/docs/features/floats.html -->

# 4.5. Floating-point types

Morloc Manual > Syntax and Features | https://morloc-project.github.io/docs/features/floats.html | prev: https://morloc-project.github.io/docs/features/integers.md | next: https://morloc-project.github.io/docs/features/strings.md

Morloc’s floating-point types are IEEE 754 binary formats. `Real` is the default; `F32` and `F64` exist when you need to control precision explicitly.

| Type | Width | Use case |
| --- | --- | --- |
| `Real` | Language-dependent (typically 64-bit IEEE 754) | Default floating point. |
| `F32` | 32 bits (IEEE 754 binary32) | Tensors, GPU code, memory-constrained numerics. |
| `F64` | 64 bits (IEEE 754 binary64) | Default-precision scientific computation. |

Each maps to its host-language equivalent:

| Morloc type | C++ | Python | R |
| --- | --- | --- | --- |
| `Real` | `double` | `float` | `numeric` |
| `F32` | `float` | `float` (with f32 conversion at the boundary) | `numeric` |
| `F64` | `double` | `float` | `numeric` |

## 4.5.1. Literal forms

Real literals need a decimal point or an exponent:

```morloc
pi :: Real
pi = 3.14159265358979

-- scientific notation (upper or lowercase 'e')
avogadro :: F64
avogadro = 6.022e23

-- negative exponent
boltzmann :: Real
boltzmann = 1.380649e-23
```

```console
$ ./floats pi
3.14159265358979
$ ./floats avogadro
6.022e+23
$ ./floats boltzmann
1.380649e-23
```

Note the explicit `+` on the printed exponent.

A literal with neither a decimal point nor an exponent is an `Int`, not a `Real`. Write `1.0` or `1e0` when you want a floating-point one.

## 4.5.2. IEEE 754 and non-finite values

`Real` follows IEEE 754 in full, which means its value space is the finite reals representable at the target precision **plus** three classes of non-finite value:

-   `+Infinity`
-   `-Infinity`
-   `NaN` (Not-a-Number)

Ordinary arithmetic produces these: dividing by zero, overflowing the finite range, or evaluating an indeterminate form such as `Inf - Inf` or `0 * Inf`. They are not error states. They are values, and they propagate through later computation by rules the standard fixes.

### Source-level literals

Each has a dedicated literal, capitalized to match Morloc’s other keyword-like values (`True`, `False`, `Null`):

```morloc
posInf :: Real
posInf = Inf

negInf :: Real
negInf = -Inf

notANumber :: Real
notANumber = NaN
```

`-Inf` lexes as a single atomic token, the same way `-1.5` is one token rather than `negate 1.5`, so it works in pure-Morloc contexts where `negate` is not in scope. The same holds for `-NaN`, though the sign of a NaN collapses at the wire boundary: both `NaN` and `-NaN` come back as the canonical `nan`.

### Arithmetic on non-finite values

All three target languages follow IEEE 754 here, so these results do not depend on which pool the computation lands in. Every row below was run:

| Expression | Result | Why |
| --- | --- | --- |
| `Inf + Inf` | `Inf` | Same-sign infinity addition |
| `Inf + (-Inf)` | `NaN` | **Invalid op**: opposite-sign cancellation |
| `Inf - Inf` | `NaN` | **Invalid op**: same-sign cancellation |
| `Inf * 0.0` | `NaN` | **Invalid op**: zero times infinity |
| `Inf * 2.0` | `Inf` | Magnitude preservation |
| `Inf * (-1.0)` | `-Inf` | Sign rule on multiplication |
| `Inf * Inf` | `Inf` | Like-sign product |
| `Inf * (-Inf)` | `-Inf` | Mixed-sign product |
| `NaN + finite` | `NaN` | NaN absorption (additive) |
| `NaN * 0.0` | `NaN` | NaN beats zero |
| `NaN * Inf` | `NaN` | NaN beats infinity |
| `negate Inf` | `-Inf` | Sign-bit flip |
| `negate NaN` | `NaN` | Sign flip stays NaN |

## 4.5.3. Compile-time literal overflow

A real literal is bounds-checked against the precision it is written into. As with integer literals, the check runs during code generation, so `typecheck` passes and `make` rejects it.

For `Real` and `F64`, the maximum magnitude is about 1.8e308:

```morloc
tooBig :: Real
tooBig = 1e500
```

```console
$ morloc make fbig.loc
fbig.loc:6:10: error:
Float literal 1.0e500 overflows F64 (|x| > 1.8e308)
  |
6 | tooBig = 1e500
  |          ^
```

The check is per-precision, so a literal that fits `F64` can still overflow `F32` (maximum magnitude about 3.4e38):

```morloc
tooBigF32 :: F32
tooBigF32 = 1e100
```

```console
$ morloc make fbig32.loc
fbig32.loc:6:13: error:
Float literal 1.0e100 overflows F32 (|x| > 3.4e38)
  |
6 | tooBigF32 = 1e100
  |             ^
```

Negative literals are checked symmetrically:

```console
$ morloc make fneg.loc
fneg.loc:6:10: error:
Float literal -1.0e500 overflows F64 (|x| > 1.8e308)
  |
6 | tooNeg = -1e500
  |          ^
```

`Inf`, `-Inf`, and `NaN` bypass the bounds check by construction. They are explicit non-finite values, not finite literals that happened to overflow.

## 4.5.4. Wire format and JSON interop

The JSON wire format is RFC 8259 compliant, and standard JSON has no syntax for non-finite numbers. The specification’s recommended workaround is strings, so Morloc emits them as quoted lowercase strings:

| Value | JSON form |
| --- | --- |
| `+Inf` | `"inf"` |
| `-Inf` | `"-inf"` |
| `NaN` | `"nan"` |
| Finite `x` | The numeric form (`3.14`, `4.2e16`, and so on) |

You can see this in the output of the literals above:

```console
$ ./floats posInf
"inf"
$ ./floats negInf
"-inf"
$ ./floats notANumber
"nan"
```

So a `Real`\-typed field can arrive as either a JSON number or a JSON string. Consumers need to accept both.

Internal cross-language boundaries do not use JSON. Morloc-to-pool calls use a binary format that preserves IEEE 754 bytes verbatim, so non-finite values round-trip with no loss. Only the JSON boundary — usually the program’s final output — uses the string form.

> **Warning: Cross-language gotcha: division by zero in Python**
> The three languages agree on IEEE 754 **arithmetic**, but they disagree on one point of language **design**: Python raises `ZeroDivisionError` on `1.0 / 0.0`, where C++ and R produce `+Inf`.
> 
> That difference is visible from inside Morloc. `idpy` and `idcpp` pin a computation to one pool:
> 
> ```morloc
> pyDiv :: Real -> Real -> Real
> pyDiv x y = idpy (x / y)
> 
> cppDiv :: Real -> Real -> Real
> cppDiv x y = idcpp (x / y)
> ```
> 
> ```console
> $ ./divzero pyDiv 1.0 0.0
> Error: run failed
> float division by zero
>   at pyDiv [py] (mid=1, divzero.loc:1:17)
> $ ./divzero cppDiv 1.0 0.0
> "inf"
> ```
> 
> If a program depends on `1.0 / 0.0` giving `+Inf`, that expression must not run in a Python pool. Constructing infinity directly with the `Inf` literal avoids the question entirely.

## 4.5.5. F32 precision considerations

`F32` halves memory against `F64`, which matters for large numerical arrays — tensors, image buffers, GPU input — where the extra precision is not needed. The tradeoffs:

-   The significand carries about 7 decimal digits of precision, against about
    
    **15 to 17 for `F64`. A literal such as \`0.1**
    
    F32\` rounds to the nearest representable binary32 value; it is not exact.
    
-   Maximum magnitude is about 3.4e38, against 1.8e308 for `F64`. The compile-time bounds check enforces this for literals.
-   All `F32` arithmetic runs at single precision, including the overflow-to-infinity threshold.

For most application code `Real` is the right default. Reach for `F32` deliberately, when memory or single-precision hardware demands it.

## 4.5.6. Converting to and from floating point

The `TotalInto` and `PartialInto` classes from [Integer types](https://morloc-project.github.io/docs/features/integers.md) extend to floats. `into` covers the conversions that cannot fail: widening an integer whose full range fits the target mantissa (24 bits for `F32`, 53 for `F64`), `F32` to `F64`, and `Real` to and from `F64` in both directions — they are representationally identical in every current backend.

Integer-to-float conversions that may lose precision get their own class:

```morloc
class RealLike a where
  toReal :: a -> Real
```

`toReal` never fails but can lose precision above 2^53. Every numeric type has an instance. The canonical use is a mean:

```morloc
mean :: [Real] -> Real
mean xs = sum xs / toReal (size xs)
```

```console
$ ./floats mean '[1,2,3,4]'
2.5
```

`size` returns `U64` and `toReal` bridges it into the `Real` denominator. The precision loss is theoretical at any realistic container size, but naming it keeps the lossy step visible.

Float-to-integer conversion goes through `tryInto`, which raises rather than returning a value it cannot represent. It fails on `NaN`, on `Inf`, on non-integer values, and on values outside the target integer’s range:

```morloc
approx :: Real -> I32
approx x = tryInto x
```

```console
$ ./floats approx 3.0
3
$ ./floats approx 3.5
Error: run failed
cannot convert non-integer float 3.5 to integer
  at approx [py] (mid=8, floats.loc:1:75)
$ ./floats approx 1e20
Error: run failed
value 100000000000000000000 out of range [-2147483648, 2147483647]
  at approx [py] (mid=8, floats.loc:1:75)
```

To round to a nearby integer instead of failing, apply `round`, `floor`, `ceil`, or `trunc` from the `math` module first, then `tryInto` the result.

Narrowing `F64` to `F32`, and `Real` to `F32`, are deliberately **not** provided as `TotalInto` instances — they lose precision on every input. If you need one, source an explicit foreign function, so the lossy step is visible at the call site.

## 4.5.7. Negation of Real values

Negation works on `Real`, `F32`, and `F64` exactly as it does on integers, via the `Negatable` typeclass; see [Integer types](https://morloc-project.github.io/docs/features/integers.md) for the full unary-minus rules. Three IEEE 754 specifics:

-   `-Inf` and `-NaN` are atomic source literals. No `negate` lookup happens, so they work in pure-Morloc contexts.
-   `negate Inf` is `-Inf`, and `negate NaN` is `NaN` — the sign bit flips, but the value is still NaN.
-   `negate 0.0` is `-0.0`. The two compare equal under `==` but have different bit patterns. The binary cross-language format preserves the distinction; the JSON output does not.


---

<!-- https://morloc-project.github.io/docs/features/strings.html -->

# 4.6. Strings

Morloc Manual > Syntax and Features | https://morloc-project.github.io/docs/features/strings.html | prev: https://morloc-project.github.io/docs/features/floats.md | next: https://morloc-project.github.io/docs/features/tuples-and-lists.md

A Morloc string is double-quoted and holds Unicode text:

```morloc
cn :: Str
cn = "你知道得太多了🤫"
```

```console
$ ./strs cn
"你知道得太多了🤫"
```

## 4.6.1. Interpolation

`#{…​}` splices an expression into a string. The expression must already have type `Str` — nothing is converted for you. To embed an `Int`, `Real`, `Bool`, or anything else, call `show` (or another explicit stringifier) inside the braces:

```morloc
helloYou :: Str -> Str
helloYou you = "hello #{you}"

sayCount :: Int -> Str
sayCount n = "count: #{show n}"
```

```console
$ ./strs helloYou world
"hello world"
$ ./strs sayCount 42
"count: 42"
```

## 4.6.2. Escapes

Inside a string, a backslash introduces an escape sequence:

| Escape | Meaning |
| --- | --- |
| `\n` | newline |
| `\t` | tab |
| `\r` | carriage return |
| `\0` | NUL byte (U+0000) |
| `\\` | a single backslash |
| `\"` | a literal double quote |

Any other backslashed character is a compile-time error:

```console
$ morloc typecheck escbad.loc
escbad.loc:6:10: invalid escape sequence \q
```

A literal backslash must therefore always be written `\\`, which matters most for Windows paths:

```morloc
winPath :: Str
winPath = "C:\\Users\\weena\\file.txt"
```

Writing `"C:\Users"` instead does not compile, because `\U` is not a recognized escape:

```console
$ morloc typecheck escwin.loc
escwin.loc:6:8: invalid escape sequence \U
```

## 4.6.3. Triple-quoted strings

Triple quotes come in double and single flavours. On one line they save you from escaping the other kind of quote:

```morloc
dblStr :: Str
dblStr = """That's weird, I also spelled it "ear quotes", like "bunny ears"."""

sinStr :: Str
sinStr = '''"Why do the pigeons here have so few toes?"'''
```

The result is identical to the single-quoted form with the quotes escaped:

```console
$ ./strs dblStr
"That's weird, I also spelled it \"ear quotes\", like \"bunny ears\"."
$ ./strs sinStr
"\"Why do the pigeons here have so few toes?\""
```

Their real value is multi-line text. The indentation is trimmed by three rules, applied in order:

1.  Initial spaces up to and including the first newline are removed.
2.  Terminal spaces up to and including the final newline are removed.
3.  Every line loses as many leading spaces as the least-indented line has.

So a block can sit at whatever indentation the surrounding code wants:

```morloc
longString :: Str
longString =
  """
  this is a long
  string
  """
```

```console
$ ./strs longString
"this is a long\nstring"
```

The leading and trailing newlines and the two-space indent are all gone, which is what lets you write natural paragraphs without breaking your code’s indentation.

## 4.6.4. NUL bytes in strings

This is the thorniest corner of multi-language string support, and it is worth understanding before it bites you.

In C, a NUL byte terminates a string, so `strlen` and `strdup` cannot see past one. R is built on C and makes within-string NULs strictly illegal. Python and C++ (through `std::string`) both allow them — but even there, problems appear whenever the string is converted to a C string, through `.c_str()` in C++ or across the C ABI in Python.

NULs are not common in text. Their main use is binary data, and `Str` is not the right type for that — prefer `[U8]`, or better a `Vector n U8` ([Tensors](https://morloc-project.github.io/docs/types/tensors.md)). But Morloc’s philosophy is to support what is idiomatic in each language, and `Str` is meant to be the ordinary string type everywhere. So Morloc’s `Str` does support NULs: they can be written with `\0`, the evaluator preserves them end to end, and JSON represents them with the standard `\u0000` escape.

In a Python-only program that works exactly as you would expect:

```morloc
nulStr :: Str
nulStr = "ab\0cd"

pyNul :: Str
pyNul = idpy nulStr

len :: U64
len = size nulStr
```

```console
$ ./nul len
5
$ ./nul pyNul
"ab\u0000cd"
```

Five bytes, and the NUL survives the round trip.

Each language declares `allow_string_null` in its `lang.yaml`. When a `Str` carrying a NUL is sent to a language that does not allow one, the call is rejected.

A **literal** is caught while compiling, because the generated source would not parse. The error names the pool and points at the place the literal enters it:

```console
$ morloc make nul.loc
nul.loc:13:12: error:
This string literal contains a NUL byte, which the r pool cannot represent in its native string type. Move the literal to a language that can (Python, C++, Julia, or the nexus itself), or remove the NUL byte. See the allow_string_null field in the language's lang.yaml.
   |
13 | rNul = idr nulStr
   |            ^
```

It points at the use rather than the declaration, because the same literal is perfectly legal in a pool that can hold it.

A value computed at **run time** is caught at the boundary it tries to cross. Arriving as a command-line argument:

```console
r does not support embedded NUL bytes in strings (at args[0])
```

or produced inside one pool and handed to another:

```console
$ ./nexus listR ab
Error: run failed
R cannot represent an embedded NUL byte in a string; one arrived at [1] (byte 2 of 5)
  at _ [r] (mid=2053, main.loc:11:14)
```

The path locates the offending slot, which matters when the NUL is buried: `[1]` is the second element of a list, `.b` a record field, and a bare value reports just the byte offset.

Whether that scan happens is decided when your program is compiled. A value whose type contains no string cannot carry a NUL, so no check is generated for it, and a pool in a language that tolerates NULs is compiled exactly as it would have been. You pay only where a string actually crosses into a language that cannot hold one.

Scanning every string for NULs costs time. You can opt out two ways when you know it is safe:

-   `morloc make --unsafe-skip-null-check` bakes a per-program skip flag into the manifest.
-   `MORLOC_SKIP_NULL_CHECK=1` skips the scan for one run.

Both are unsafe in the same way: a NUL that reaches R still crashes inside the R runtime, just with R’s error instead of Morloc’s. There is nothing useful user-written R code can do with a NUL-bearing string.


---

<!-- https://morloc-project.github.io/docs/features/tuples-and-lists.html -->

# 4.7. Tuples and Lists

Morloc Manual > Syntax and Features | https://morloc-project.github.io/docs/features/tuples-and-lists.html | prev: https://morloc-project.github.io/docs/features/strings.md | next: https://morloc-project.github.io/docs/features/records.md

Tuples and lists are the two containers you will reach for first. A tuple has a fixed size and may hold elements of different types; a list has variable size and holds elements that all share one type.

Both become JSON arrays on the wire, so from JSON alone you cannot tell whether `[1,2,3]` is a three-element list of integers or a three-integer tuple. The type is what distinguishes them, and the type is not in the JSON.

## 4.7.1. Tuples

A tuple stores a fixed number of terms of differing type:

```morloc
x :: (Int, Bool, Real)
x = (1, True, 6.45)
```

```console
$ ./tuples x
[1,true,6.45]
```

Tuple types and tuple values look the same: comma-separated inside parentheses. The parenthesized type is sugar for a fixed-arity constructor, `Tuple3` here. The parser builds the right `TupleN` from the number of fields, so there is no fixed upper bound on arity — a twelve-element tuple reports its type as:

```console
$ morloc typecheck tuples.loc
big :: Tuple12 Int Int Int Int Int Int Int Int Int Int Int Int
```

That said, past a few members a record with named fields is easier to read and harder to get wrong. See [Records](https://morloc-project.github.io/docs/features/records.md).

## 4.7.2. Lists

Lists are homogeneous and variable length. The base type is `List a`, and `[a]` is sugar for it:

```morloc
x :: [Int]
x = [1, 2, 3]

ys :: List Real
ys = [1.0, 2.0, 3.0]
```

The two spellings name the same type, and the compiler reports both in the sugared form:

```console
$ morloc typecheck tuples.loc
x :: [Int]
ys :: [Real]
```

`List` maps to each language’s natural ordered container: `list` in Python, `std::vector` in C++, and list or vector in R.

Every list-like type shares one wire representation — zero or more elements in contiguous memory — but different in-language structures make different performance tradeoffs. `Deque`, for example, is declared in `root` as a distinct type over the same representation:

```morloc
newtype Deque a = List a
```

so it costs nothing to send but can add to either end cheaply in the languages that back it with a real deque. For how to define such specializations yourself, see [Naming a type: `type` and `newtype`](https://morloc-project.github.io/docs/types/newtype.md).

For numeric work there is a more rigorous and faster alternative to `List`: the `Vector` type, which is the one-dimensional tensor. See [Tensors](https://morloc-project.github.io/docs/types/tensors.md).


---

<!-- https://morloc-project.github.io/docs/features/records.html -->

# 4.8. Records

Morloc Manual > Syntax and Features | https://morloc-project.github.io/docs/features/records.html | prev: https://morloc-project.github.io/docs/features/tuples-and-lists.md | next: https://morloc-project.github.io/docs/features/patterns.md

A record is a named, fixed set of named fields. It is the right shape when a tuple would leave you counting positions.

```morloc
record Person = Person
    { name :: Str
    , age :: Int
    }
```

Records map to whatever each language uses for the job: a `dict` in Python, a `list` in R, a `struct` in C++. Internally the layout is positional, but the surface language always binds by name.

## 4.8.1. Native representations

The concrete forms must share the general record’s field names and types, so those are not repeated. You only name the container:

```morloc
record Py => Person = "dict"
record R => Person = "list"
record Cpp => Person = "person_t"
```

Python and R need nothing further — `dict` and `list` hold arbitrary fields already. C++ needs the struct to exist:

**foo.hpp**

```cpp
#pragma once
#include <string>

struct person_t {
    std::string name;
    int age;
};

person_t incAge(person_t person){
    person.age++;
    return person;
}
```

The R and Python sides operate on their native containers directly:

**foo.R**

```r
incAge <- function(person){
    person$age <- person$age + 1
    person
}
```

**foo.py**

```python
def incAge(person):
    person["age"] += 1
    return person
```

## 4.8.2. Record literals match by field name

Field values bind to declared fields by **name**. The order in a literal is irrelevant, so these two are the same value:

```morloc
alice :: Person
alice = { name = "Alice", age = 30 }

alice2 :: Person
alice2 = { age = 30, name = "Alice" }
```

```console
$ ./recs alice
{"name":"Alice","age":30}
$ ./recs alice2
{"name":"Alice","age":30}
```

A literal must mention every declared field exactly once. All three ways to get that wrong are compile-time errors. The examples below all come from a `recbad.loc` whose record is declared on one line:

```morloc
record Person = Person { name :: Str, age :: Int }
```

Missing a field:

```console
recbad.loc:8:8-25: error:
Record literal does not match declared type Person:
  missing field(s): age
  |
8 | bad1 = { name = "Alice" }
  |        ^~~~~~~~~~~~~~~~^
```

Naming a field the record does not have:

```console
recbad.loc:8:8-48: error:
Record literal does not match declared type Person:
  unknown field(s): weight
  |
8 | bad1 = { name = "Alice", age = 30, weight = 65 }
  |        ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
```

Repeating a field:

```console
recbad.loc:8:33: duplicate field in record literal: name
    |
  8 | bad1 = { name = "Alice", name = "Bob", age = 30 }
    |                                 ^
```

## 4.8.3. One record across three languages

Because the record has a native form in each language, a function that operates on it can be sourced from any of them, and they compose:

**recs.loc**

```morloc
module recs (foo)

import root-r
import root-py
import root-cpp

record Person = Person
    { name :: Str
    , age :: Int
    }

record Py => Person = "dict"
record R => Person = "list"
record Cpp => Person = "person_t"

source R from "foo.R" ("incAge" as rinc)
source Py from "foo.py" ("incAge" as pinc)
source Cpp from "foo.hpp" ("incAge" as cinc)

rinc :: Person -> Person
pinc :: Person -> Person
cinc :: Person -> Person

foo :: Str -> Int -> Person
foo name age
    = (rinc . pinc . cinc)
      { name = name, age = age }
```

`foo` builds a `Person` and then increments its age three times, once in each language, passing the record across two process boundaries on the way:

```console
$ ./recs foo Bob 40
{"name":"Bob","age":43}
```

Nothing in `foo.R`, `foo.py`, or `foo.hpp` knows the others exist. Each sees only its own language’s ordinary data structure.


---

<!-- https://morloc-project.github.io/docs/features/patterns.html -->

# 4.9. Patterns

Morloc Manual > Syntax and Features | https://morloc-project.github.io/docs/features/patterns.html | prev: https://morloc-project.github.io/docs/features/records.md | next: https://morloc-project.github.io/docs/features/pattern-matching.md

Morloc’s **pattern functions** are first-class getters, setters, and bracket operators for reaching into and rearranging data structures. They are ordinary values, so you can pass them around, `map` them over a list, and compose them like any other function.

This section is about **extracting and rebuilding** data. To bind a value’s parts to names, or to dispatch a function on the shape of its arguments, see [Pattern Matching](https://morloc-project.github.io/docs/features/pattern-matching.md) instead.

The examples below use anonymous record types, which are written with `=` rather than `::`:

```morloc
pts :: [{x = Int, y = Int}]
pts = [{x=0, y=100}, {x=1, y=101}, {x=2, y=102}, {x=3, y=103}]
```

Using `::` inside a record type is a common slip, and the compiler says so:

```console
pat.loc:8:9: type-level record literals use `=` to bind fields, not `::`
  try: {x = <type>, ...}
  `::` is for declarations (e.g. `x :: Int`, `record R where { x :: Int }`)
```

## 4.9.1. Getter patterns

A getter describes an optionally branching path into a structure. Each segment is a tuple index, a record key, or a group of them. Terminal positions come back as a tuple.

```morloc
-- the 1st element of a tuple of any size
.0 (1,2)              -- 1
.0 ((1,3),2,5)        -- (1,3)

-- the 2nd element of the first element
.0.1 ((1,3),2,5)      -- 3

-- the 2nd and 1st elements, in that order
.(.1,.0) (1,2,3)      -- (2,1)
.(.1,.0) (1,2)        -- (2,1)

-- indices and keys mix freely
.0.(.x, .y.1) ({x=1, y=(1,2), z=3}, 6)   -- (1,2)
```

A pattern is a function, so it goes wherever a function goes:

```morloc
map .1 [(1,2),(2,3)]  -- [2,3]
```

## 4.9.2. Setter patterns

A setter is the same path with an assignment at each terminus:

```morloc
.(.0 = 99) (1,2)

.0.(.x=99, .y.1=33) ({x=1, y=(1,2), z=3}, 6)
```

```console
$ ./patterns s1
[99,2]
$ ./patterns s2
[{"x":99,"y":[1,33],"z":3},6]
```

Setters do not mutate. The **spine** of the structure is copied, and unmodified fields still point at the original data. So `.(.0 = 42) x` builds a new tuple whose first field is 42 and whose remaining fields are the original elements. Records behave the same way.

## 4.9.3. Bracket patterns

Lists get a dedicated bracket form with Python’s index and slice syntax, written after a dot: `.[i]` picks an element, `.[i:j]` takes a sub-range, and `.[i:j:k]` adds a stride. The semantics track Python — negative indices count from the end, out-of-range bounds are clamped, and `.[::-1]` reverses.

```morloc
ten :: [Int]
ten = [0,1,2,3,4,5,6,7,8,9]
```

```console
$ ./patterns b1     -- .[0] ten
0
$ ./patterns b2     -- .[-1] ten        negative index counts from the end
9
$ ./patterns b3     -- .[1+1] ten       any expression of an IndexLike type
2
$ ./patterns b4     -- .[2:5] ten
[2,3,4]
$ ./patterns b5     -- .[:3] ten        omitted start defaults to 0
[0,1,2]
$ ./patterns b6     -- .[7:] ten        omitted stop defaults to length
[7,8,9]
$ ./patterns b7     -- .[:] ten         no-op copy
[0,1,2,3,4,5,6,7,8,9]
$ ./patterns b8     -- .[8:99] ten      bounds are clamped
[8,9]
$ ./patterns b9     -- .[0:-1] ten      Python-style negative stop
[0,1,2,3,4,5,6,7,8]
$ ./patterns b10    -- .[::2] ten       every other element
[0,2,4,6,8]
$ ./patterns b11    -- .[::-1] ten      full reverse
[9,8,7,6,5,4,3,2,1,0]
$ ./patterns b12    -- .[7:2:-2] ten    strided reverse slice
[7,5,3]
```

Any integral type can be an index or a bound. The conversion to the underlying 64-bit width dispatches through the `IndexLike` typeclass, so mixed widths are fine:

```morloc
ix :: I8 -> U32 -> [Int]
ix i j = .[(i :: I8) : (j :: U32)] ten
```

### Composing brackets with other patterns

Brackets chain with the other pattern forms. The rule depends on whether the bracket selects one element or a list:

-   `.[i].tail xs` — an index yields a scalar, so the tail composes directly. `.[0].x pts` is `(.x . .[0]) pts`.
-   `.[i:j].tail xs` — a slice yields a list, so the tail is **lifted** with `map`. `.[0:3].x pts` is `map .x (.[0:3] pts)`.

The tail can be any pattern body: a record key, a tuple index, a grouped selector, or another bracket. Nested brackets follow the same rule, with the outer map running the inner bracket on each row.

```morloc
rows :: [{a = [(Int,Int)], b = [{x = Int, y = Int}]}]
rows = [ {a = [(10,20)], b = [{x=100, y=200}]}
       , {a = [(30,40)], b = [{x=300, y=400}]} ]

xss :: [[Int]]
xss = [[1,2,3,4,5], [6,7,8,9,10]]
```

```console
$ ./patterns c1     -- .[0].x pts       scalar tail composes directly
0
$ ./patterns c2     -- .[2].y pts
102
$ ./patterns c3     -- .[-1].x pts
3
$ ./patterns c4     -- .[:3].x pts      slice + field, map-lifted
[0,1,2]
$ ./patterns c5     -- .[::-1].x pts
[3,2,1,0]
$ ./patterns c6     -- .[0:3].(.x, .y) pts        slice + group tail
[[0,100],[1,101],[2,102]]
$ ./patterns c7     -- .[0:2].[0:3] xss           slice + nested slice
[[1,2,3],[6,7,8]]
$ ./patterns c8     -- .[0:2].(.a.[0].0, .b.[0].y) rows    deep mixed chain
[[10,200],[30,400]]
```

Remember that these are JSON outputs, so a tuple prints as an array: `c6` returns three two-tuples, which JSON shows as `[[0,100],…​]`.

Brackets are getters only. There is no setter form (`.[i] = v $ xs`) in this release, and multi-axis brackets (`.[i,j]` for matrices and tensors) are not available yet either. Both are planned.

## 4.9.4. Patterns next to Python

| Pattern | Python | Note |
| --- | --- | --- |
| `.0` | `lambda x: x[0]` | patterns are functions |
| `.0 x` | `x[0]` |  |
| `.0.k x` | `x[0]["k"]` |  |
| `.(.1,.0) x` | `(x[1], x[0])` |  |
| `foo .0 xs` | `foo(lambda x: x[0], xs)` | higher order |
| `.(.k = 1) x` | `x["k"] = 1` | but non-mutating |
| `.[i] xs` | `xs[i]` | scalar result |
| `.[i:j] xs` | `xs[i:j]` | slice result (list) |
| `.[::-1] xs` | `xs[::-1]` | full reverse |
| `.[i:j].x xs` | `[e["x"] for e in xs[i:j]]` | tail map-lifted over slice |

## 4.9.5. Adding bracket support to your own types

Bracket syntax is not hardcoded. It dispatches through ordinary typeclasses declared in the standard library’s `internal` module, so a new container type can opt in:

```morloc
-- Indexing: .[i] xs
class Indexable f where
  __access_index__ :: ?I64 -> f a -> a

-- Slicing for shape-preserving containers (List, Str, ...)
class Sliceable f where
  __get_slice__ :: ?I64 -> ?I64 -> ?I64 -> f a -> f a

-- Slicing for Nat-parameterized containers (Vector, Tensor, ...) where the
-- output length differs from the input length
class SliceableDim f where
  __get_slice_dim__ :: ?I64 -> ?I64 -> ?I64 -> f n a -> f m a

-- Casting any user expression in a bound position to ?I64
class IndexLike i where
  __to_index__ :: i -> ?I64
```

For a plain container, source an `Indexable` and a `Sliceable` instance per target language. For a container parameterized by a dimensional `Nat` such as `Vector n a`, source `Indexable` and `SliceableDim` instead; the compiler picks `SliceableDim` automatically when `Sliceable` is absent.

If someone passes an expression of a custom integer-like type at a bound position, the compiler casts it through that type’s `IndexLike` instance. So you can extend bracket syntax to accept new bound types — a `Char` index, a fixed-point coordinate — by adding an `IndexLike` instance whose `*to_index*` returns `?I64`. Pass `Null` through as `Nothing` so it composes with the omitted positions in `.[i:]`, `.[:j]`, and `.[::]`.

Because the dispatch lives in libraries rather than in the compiler, a module that does not import the standard library is free to substitute a different typeclass hierarchy. Bracket syntax simply errors at codegen if no matching instance is in scope.


---

<!-- https://morloc-project.github.io/docs/features/pattern-matching.html -->

# 4.10. Pattern Matching

Morloc Manual > Syntax and Features | https://morloc-project.github.io/docs/features/pattern-matching.html | prev: https://morloc-project.github.io/docs/features/patterns.md | next: https://morloc-project.github.io/docs/features/where-and-let.md

A **pattern** describes the shape of a value using the same notation you would use to build it. Morloc matches values against patterns in two ways.

**Irrefutable patterns** destructure a value into named parts at binding positions: lambda parameters, function-definition arguments, `let` left-hand sides, and `do`\-block `←` binds. Every well-typed receiver matches, so these patterns contain only variable names, wildcards, and structural constructors — no literals, no alternatives. That is what makes them irrefutable.

**Refutable patterns** dispatch on the shape of a value through a list of `|`\-clauses. A clause can fail to match, because a literal matches only itself, so clauses are tried in order and the first that matches wins. Clauses appear either in a function’s definition, dispatching on its arguments, or in a `match` expression, dispatching on any value you hand it.

[Patterns](https://morloc-project.github.io/docs/features/patterns.md) covers the related but distinct topic of pattern **functions** — `.0`, `.[i:j]`, and friends — which extract and rebuild data rather than bind names.

## 4.10.1. Irrefutable patterns

The supported shapes:

-   **variable** — `x` binds the whole value
-   **wildcard** — `_` matches without binding
-   **tuple** — `(x, y)` binds each component. Full arity is required; use wildcards for positions you want to ignore, as in `(x, _, _)`
-   **record** — `{a = x, b = y}` binds fields by name. Extra fields are ignored, order does not matter, and the receiver only has to **have** the keys the pattern mentions. This is structural, or row-polymorphic, matching — the same rule as the `.(.a, .b)` group getter
-   **as-pattern** — `label@atom` binds `label` to the whole receiver and destructures further through `atom`
-   **nesting** is free: `(x, {a = y, b = _}, q@(l, r))`

All four binding sites take them:

```morloc
-- lambda parameter
first = \ (a, b) -> a

-- function-definition argument
snd (_, y) = y

-- let-binding
demo pair = let (a, b) = pair in a

-- do-block bind
useIt = do
  (a, b) <- readPair
  a
```

Records mix in cleanly:

```morloc
record Pair = Pair { a :: Int, b :: Int }

-- field-polymorphic: any record with keys 'a' and 'b'
pickA {a = x, b = _} = x

-- nested
combine :: (Int, Pair) -> Int
combine (n, {a = p, b = q}) = n + p + q
```

```console
$ ./match pickA '{"a":5,"b":6}'
5
$ ./match combine '[1,{"a":2,"b":3}]'
6
```

### Wildcards

`_` matches without binding. In a `let` left-hand side or a `do` bind, the right-hand side is still evaluated, so effects still fire; in a lambda or function-argument position the slot is accepted and discarded.

```morloc
-- discard the first tuple element
snd (_, y) = y

-- do-bind: the effect fires, the value is discarded
main = do
  _ <- setup
  work
```

### As-patterns

`label@atom` binds `label` to the whole receiver **and** destructures through `atom`, so both are in scope:

```morloc
tag p@(x, y) = (p, x + y)
```

```console
$ ./match tag '[3,4]'
[[3,4],7]
```

There must be **no whitespace** around `@`. Write `p@(x, y)`, never `p @ (x, y)`:

```console
pmx.loc:6:7: unexpected operator '@'
    |
  6 | tag p @ (x, y) = (p, x + y)
    |       ^
```

This matches the tight-binding style of Morloc’s other qualifier operators — `.` for namespaces, `:` for group labels. An `@name` in a fresh position (start of line, after whitespace, after a delimiter) still means an intrinsic such as `@stdout`.

### Record patterns on `let` and `do` need parentheses

`let` and `do` both accept an explicit `{` right after the keyword as an alternative to layout-based blocks:

```morloc
let { a = 1; b = 2 } in a       -- explicit-brace form of a two-binding let
do  { readValue; useIt }        -- explicit-brace form of a do-block
```

So a record pattern in those positions has to be parenthesized, or its `{` is read as the start of a bindings block:

```morloc
-- required
let ({a = p, b = q}) = mkPair in p
do
  ({a = p, b = q}) <- fetch
  p
```

Without the parentheses you get a parse error that does not obviously point at the real problem — the parser is inside a bindings block by then and is complaining about the comma:

```console
pmx.loc:9:18: unexpected ','
    |
  9 | demo = let {a = p, b = q} = mkPair in p
    |                  ^
  expected one of: '}', ';'
```

Function-definition and lambda positions are unaffected: neither `\` nor a function name is a layout keyword, so `foo {a = x, b = y} = x` and `\ {a = x, b = y} → x` parse without parentheses. Tuple and as-patterns on `let` and `do` are also fine unparenthesized, because they do not start with `{`.

## 4.10.2. Refutable patterns

A function can dispatch on the shape of its arguments by giving several `|`\-clauses instead of one body. Each clause lists one pattern per argument, then `=` and a result. Clauses are tried top to bottom, and the first whose patterns all match wins:

```morloc
fibonacci :: Int -> Int
fibonacci | 0 = 1
          | 1 = 1
          | n = fibonacci (n - 1) + fibonacci (n - 2)
```

A multi-argument function carries one pattern per argument per clause:

```morloc
ackermann :: Int -> Int -> Int
ackermann | 0 n = n + 1
          | m 0 = ackermann (m - 1) 1
          | m n = ackermann (m - 1) (ackermann m (n - 1))
```

```console
$ ./match fibonacci 10
89
$ ./match ackermann 2 3
9
```

A clause pattern may take any irrefutable shape — variable, wildcard, tuple, record, as-pattern — plus one more: a **literal**. An `Int`, `Real`, `Str`, or `Bool` value matches only itself, and that is what makes a clause refutable.

There is a second refutable shape, the **constructor pattern**, which matches one alternative of a sum type. It is covered in [Sum types](https://morloc-project.github.io/docs/features/sum-types.md), together with the `data` declaration that creates the constructors, and it works in every position described here.

```morloc
greet :: Str -> Str
greet | "en" = "hello"
      | "fr" = "bonjour"
      | _    = "hi"
```

```console
$ ./match greet fr
"bonjour"
$ ./match greet de
"hi"
```

Literals nest inside structural patterns, so you can pin part of a compound value and bind the rest:

```morloc
-- match a pair whose first element is 0, bind the second
firstZero :: (Int, Int) -> Int
firstZero | (0, n) = n
          | (m, n) = m + n
```

> **Note**
> A literal pattern compiles to an equality test, so the argument’s type needs an `Eq` instance in scope — the same requirement as writing `x == 0` yourself. The standard library provides `Eq` for the primitive types.

## 4.10.3. `match` expressions

A definition’s clauses dispatch on that definition’s arguments. When the value you want to dispatch on is one you **computed**, there is no argument to hang clauses on. `match` takes the value directly, then the same `|`\-clause list:

```morloc
statusText :: Int -> Str
statusText code = match code // 100
  | 2 = "success"
  | 3 = "redirect"
  | 4 = "client error"
  | 5 = "server error"
  | _ = "unknown"
```

```console
$ ./match statusText 200
"success"
$ ./match statusText 404
"client error"
$ ./match statusText 999
"unknown"
```

The clauses match the status **class**, `code // 100`, not `code`. A clause list cannot: it only sees the argument, so this would mean inventing a second function that takes the class, naming it, and calling it.

The scrutinee is a full expression, so `match code // 100` needs no parentheses. Anywhere an expression is allowed, a `match` is allowed — in one branch of a guard:

```morloc
describeStatus :: Int -> Str
describeStatus code
  ? code < 100 = "not a status code"
  : match code // 100
      | 2 = "success"
      | 4 = "client error"
      | 5 = "server error"
      | _ = "other"
```

```console
$ ./match describeStatus 42
"not a status code"
$ ./match describeStatus 503
"server error"
```

or inside a lambda, where there is no definition head at all:

```morloc
labels :: [Int] -> [Str]
labels = map (\c -> match c // 100 | 2 = "ok" | _ = "not ok")
```

```console
$ ./match labels '[200,404,201]'
["ok","not ok","ok"]
```

A `do`\-block statement is the other common home; matching on a bound result is how a fallible call is consumed, which [Failure and recovery](https://morloc-project.github.io/docs/features/intrinsics.md#failure-and-recovery) covers along with the type it produces.

When you **are** dispatching on a plain argument, keep the clause form. It says the same thing with less punctuation, and it is what the rest of this chapter uses.

### Where a clause list ends

A `match` has no closing keyword. Its clause list runs until something appears that cannot begin another clause, and `|` can always begin another clause. So a comma, a closing bracket, the `:` of a guard, the end of a `do` statement, `where`, and the end of a definition all end the list, and two `match` expressions sit side by side in a tuple with no help:

```morloc
pairUp :: Int -> Int -> (Str, Str)
pairUp x y = (match x | 0 = "a" | _ = "b", match y | 0 = "c" | _ = "d")
```

What does **not** end the list is a `|` belonging to something enclosing. A `match` written inside another ``match’s arm swallows that outer arm’s remaining clauses, and the error lands on the outer `match``, which has now lost its catch-all:

```console
nest2.loc:4:14: `|` patterns for 'match' are not exhaustive; a literal pattern cannot cover its type, so add a final catch-all clause (a variable or '_')
    |
  4 | nested x y = match x
    |              ^
```

The same happens inside a definition’s clause body, where the inner `match` absorbs the next clause of the definition. There the clause it swallowed has one pattern per argument, so the error names the arity rule instead:

```console
nest.loc:4:14: each `match` clause takes exactly one pattern, but this one has 2
    |
  4 | both | 0 y = match y | 1 = "a" | _ = "b"
    |              ^
```

Parenthesize the inner `match` and both compile:

```morloc
both :: Int -> Int -> Str
both | 0 y = (match y | 1 = "a" | _ = "b")
     | x _ = "c"
```

Parentheses are also required to pass a `match` as an argument, since it is not an atom:

```console
paren.loc:6:21: unexpected 'match'
    |
  6 | noParens x = double match x | 0 = 1 | _ = 2
    |                     ^
```

Write `double (match x | 0 = 1 | _ = 2)` instead.

Each `match` clause takes exactly one pattern, because there is one value being matched. That is the only structural difference from a definition’s clause list; the patterns themselves, the top-to-bottom order, and the exhaustiveness requirement below are the same.

**Why there is no closing keyword**

Languages with this construct usually bracket it: ML and Haskell write `case e of` and close the alternatives with layout or braces, Rust and Scala use `{}`. Morloc reuses the `|`\-clause list it already has for definitions instead, which keeps one notation for one idea and costs a terminator.

The cost is the case above. A clause list that ends at "the next thing that cannot be a clause" is unambiguous to parse — the grammar resolves the conflict by continuing the innermost list, which is what greedy gathering means — but it is not always what a reader expects when two lists are adjacent, and indentation does not disambiguate. Parentheses do, and are the only tool for it.

## 4.10.4. Exhaustiveness

Every clause of a term belongs to one definition and the last clause is the fall-through, so a `|`\-match must be exhaustive. That holds when the final clause is irrefutable — a variable or a `_` catch-all:

```morloc
classify :: Int -> Str
classify | 0 = "zero"
         | _ = "nonzero"
```

or when the clauses of a single `Bool` argument already cover both cases:

```morloc
invert :: Bool -> Bool
invert | True  = False
       | False = True
```

Anything else is rejected at compile time, with the fix named:

```console
pmx.loc:6:1: `|` patterns for 'stuck' are not exhaustive; add a final catch-all clause (a variable or '_')
    |
  6 | stuck | 0 = "a"
    | ^
```

A `match` is held to the same requirement, and its clauses are reported against the word `match` rather than a definition’s name:

```console
mne.loc:4:16: `|` patterns for 'match' are not exhaustive; a literal pattern cannot cover its type, so add a final catch-all clause (a variable or '_')
    |
  4 | sizeLabel xs = match (size xs)
    |                ^
```

## 4.10.5. Guards inside a clause

A clause body may itself be a `?`/`:` guard (see [Conditionals](https://morloc-project.github.io/docs/features/conditionals.md)), so one definition can match on an argument’s shape and then branch on a condition. Variables bound by the clause pattern are in scope in the guard:

```morloc
foo :: Int -> Int
foo | 0 = 0                     -- literal-pattern clause
    | x ? x < 10 = 1            -- variable pattern, guard as the body
        : 2
```

```console
$ ./match foo 0
0
$ ./match foo 5
1
$ ./match foo 50
2
```

## 4.10.6. Not yet supported

-   **List and vector patterns** (`[x, y, z]`). These are only sound when the receiver’s length is statically known, so they are deferred until Morloc’s fixed-width versus variable-length list story settles.


---

<!-- https://morloc-project.github.io/docs/features/where-and-let.html -->

# 4.11. `where` and `let` clauses

Morloc Manual > Syntax and Features | https://morloc-project.github.io/docs/features/where-and-let.html | prev: https://morloc-project.github.io/docs/features/pattern-matching.md | next: https://morloc-project.github.io/docs/features/conditionals.md

Both introduce local bindings, and they differ in exactly one way that matters: `where` is order-invariant, `let` is sequential. Pick whichever fits how you want to read the definition.

## 4.11.1. `where`

A `where` clause hangs local bindings off the end of a definition:

```morloc
f1 :: Int -> Int
f1 x = y + b where
    y = x + 1
    b = 41
```

```console
$ ./locals f1 1
43
```

Bindings in a `where` block are order-independent and may refer to each other freely, though not mutually recursively. They can see the function’s arguments, and the main expression can see them.

Clauses inherit their parent’s scope and nest:

```morloc
f2 :: Int
f2 = x where
    x = y where
        y = a + b
        a = 1
    b = 41
```

```console
$ ./locals f2
42
```

Note that the inner clause sees `b` from the outer one.

## 4.11.2. `let`

`let` is the more orderly cousin. Several bindings may precede the terminal `in`, they run in order, and each may only refer to names bound above it:

```morloc
f3 :: Int -> Int
f3 n =
  let m = n + 1
      y = m + 2
  in (m + y)
```

```console
$ ./locals f3 1
6
```

## 4.11.3. The scope rule that separates them

`let` is **non-recursive sequential**: each binding is in scope for everything after it, and a later binding may shadow an earlier one of the same name. So a chain of single-binding \`let\`s is legal, and the last one wins:

```morloc
foo :: Int
foo = let x = 1
      let x = 2
       in x
```

```console
$ ./locals foo
2
```

`where` is **order-invariant**: every binding sees every other one. That makes shadowing meaningless, so a name may be bound only once in a clause, and it may not collide with a function parameter. Both violations are compile-time errors.

Binding the same name twice:

```console
whx.loc:8:3: duplicate binding in where-clause: y
    |
  8 |   y = n + 2
    |   ^
```

Binding a name that is already a parameter:

```console
whx.loc:7:3: where-clause binding shadows function parameter: x
    |
  7 |   x = 100
    |   ^
```

If you want shadowing, that is what `let` is for.


---

<!-- https://morloc-project.github.io/docs/features/conditionals.html -->

# 4.12. Conditionals

Morloc Manual > Syntax and Features | https://morloc-project.github.io/docs/features/conditionals.html | prev: https://morloc-project.github.io/docs/features/where-and-let.md | next: https://morloc-project.github.io/docs/features/recursion.md

Guards are Morloc’s conditional branching. A guard clause starts with `?`, followed by a condition and a result. A `:` default closes the chain and is always required:

```morloc
abs :: Int -> Int
abs x
  ? x >= 0 = x
  : neg x       -- `neg` is negation, from root
```

```console
$ ./guards abs -5
5
```

Conditions are evaluated lazily from top to bottom. The first one that is true decides the result, and the rest are never evaluated. Because the `:` default always terminates the chain, a guard is exhaustive by construction — there is no way to write one that falls off the end.

Guards work with any number of parameters:

```morloc
clamp :: Int -> Int -> Int -> Int
clamp lo hi x
  ? x < lo = lo
  ? x > hi = hi
  : x
```

```console
$ ./guards clamp 0 10 42
10
```

## 4.12.1. Guards with `where`

A `where` clause can supply bindings used in both the conditions and the results:

```morloc
classify :: Int -> Str
classify x
  ? x > big = "big"
  ? x > small = "medium"
  : "small"
  where
    big = 100
    small = 10
```

```console
$ ./guards classify 150
"big"
$ ./guards classify 50
"medium"
$ ./guards classify 5
"small"
```

## 4.12.2. Guards in other positions

A guard may be the body of a `let` binding:

```morloc
absLet :: Int -> Int
absLet x =
  let result ? x >= 0 = x
             : neg x
  in result
```

and it may appear inline anywhere a value is expected. Parentheses are optional but usually clearer:

```morloc
labelOf :: Int -> Str
labelOf x = "label: " <> (? x > 0 = "pos" : "non-pos")
```

```console
$ ./guards labelOf 4
"label: pos"
$ ./guards labelOf -4
"label: non-pos"
```

## 4.12.3. Guards inside a pattern clause

Guards compose with refutable pattern matching ([Pattern Matching](https://morloc-project.github.io/docs/features/pattern-matching.md)). A `|`\-clause may use a guard as its body, so one definition can match on an argument’s shape and then branch on a condition. Variables bound by the clause pattern are in scope in the guard:

```morloc
sign :: Int -> Str
sign | 0 = "zero"                -- literal pattern matches only 0
     | x ? x < 0  = "neg"        -- otherwise bind x, then guard on it
         ? x < 10 = "small"
         : "large"
```

```console
$ ./guards sign 0
"zero"
$ ./guards sign -2
"neg"
$ ./guards sign 5
"small"
$ ./guards sign 500
"large"
```


---

<!-- https://morloc-project.github.io/docs/features/recursion.html -->

# 4.13. Recursion

Morloc Manual > Syntax and Features | https://morloc-project.github.io/docs/features/recursion.html | prev: https://morloc-project.github.io/docs/features/conditionals.md | next: https://morloc-project.github.io/docs/features/sum-types.md

## 4.13.1. Recursive functions

A function may refer to itself, and the compiler generates the corresponding recursion in the target language. Factorial, with guards:

```morloc
fact :: Int -> Int
fact n
  ? n == 0 = 1
  : n * fact (n - 1)
```

```console
$ ./recur fact 10
3628800
```

Functions may also be mutually recursive. This pair decides, inefficiently, whether a number is even:

```morloc
isEven :: Int -> Bool
isEven n
  ? n == 0 = True
  : isOdd (n - 1)

isOdd :: Int -> Bool
isOdd n
  ? n == 0 = False
  : isEven (n - 1)
```

```console
$ ./recur isEven 10
true
```

> **Caution**
> Recursion is not equally well supported across target languages. Some impose a recursion depth limit or lack tail-call optimization, so deep recursion can overflow the stack or crash the pool.

## 4.13.2. Recursive types

A type is **recursive** when its definition refers to itself. To terminate, that recursion has to be guarded: every cycle through the definition must pass under an `?T` (optional, with `Null` as the base case) or a `[T]` (list, with `[]` as the base case).

A bare self-reference is rejected at compile time:

```console
recx.loc:5:1: Type alias 'X' has a vacuous body: it reduces to a self-reference with no payload
    |
  5 | type X = X
    | ^
```

The examples below need one stdlib import for working with optional values:

```morloc
import maybe-py (require, isNull)
```

`isNull` tests whether an optional is absent; `require` asserts it is present and strips the `?`.

### Linked lists

The canonical case: a payload paired with an optional tail of the same type. When the tail slot reaches `Null`, the chain ends.

```morloc
type LL a = (a, ?(LL a))

llExample :: LL Int
llExample = (42, (7, (99, Null)))
```

```console
$ ./recur llExample
[42,[7,[99,null]]]
```

A builder producing a descending range:

```morloc
llRange :: Int -> LL Int
llRange n ? n > 0 = (n, llRange (n - 1))
          : (0, Null)
```

```console
$ ./recur llRange 3
[3,[2,[1,[0,null]]]]
```

The recursive call returns `LL Int`, but the second slot wants `?(LL Int)`. The typechecker’s element-wise coercion from `a` to `?a` bridges that with no annotation. The base case writes `Null` straight into the optional slot.

Consumers use the tuple selectors `.0` and `.1`:

```morloc
llLen :: LL Int -> Int
llLen x ? isNull (.1 x) = 1
        : 1 + llLen (require (.1 x))

llSum :: LL Int -> Int
llSum x ? isNull (.1 x) = .0 x
        : (.0 x) + llSum (require (.1 x))
```

```console
$ ./recur llLen '[1,[2,[3,null]]]'
3
$ ./recur llSum '[1,[2,[3,null]]]'
6
```

### Branching: binary trees

A node can carry more than one optional child, giving a branching structure. A binary tree node has a payload and two independently optional subtrees, so it may have zero, one, or two children:

```morloc
type BTree a = (a, ?(BTree a), ?(BTree a))

btreeExample :: BTree Int
btreeExample = (10, (5, Null, Null), (15, Null, Null))
```

```console
$ ./recur btreeExample
[10,[5,null,null],[15,null,null]]
```

A balanced builder, sharing its subtree through `let`:

```morloc
btreeBuild :: Int -> BTree Int
btreeBuild d ? d <= 0 = (1, Null, Null)
             : let sub = btreeBuild (d - 1)
               in (0, sub, sub)
```

```console
$ ./recur btreeBuild 2
[0,[0,[1,null,null],[1,null,null]],[0,[1,null,null],[1,null,null]]]
```

Summing every payload reads best when the optional handling is factored into a helper, leaving the main function as the structural recursion it is:

```morloc
btreeSum :: BTree Int -> Int
btreeSum x = .0 x + maybeSum (.1 x) + maybeSum (.2 x)

maybeSum :: ?(BTree Int) -> Int
maybeSum m ? isNull m = 0
           : btreeSum (require m)
```

```console
$ ./recur btreeSum '[10,[5,null,null],[15,null,null]]'
30
```

### List-guarded recursion: rose trees

The other permitted guard is `[T]`. An empty list is the natural base case, and arbitrary branching falls out as a list of children rather than a fixed number of optional slots:

```morloc
type Rose a = (a, [Rose a])

roseExample :: Rose Int
roseExample = (1, [(2, []), (3, [])])
```

```console
$ ./recur roseExample
[1,[[2,[]],[3,[]]]]
```

A builder for a complete binary rose tree, and a sum that folds the children:

```morloc
roseBuild :: Int -> Rose Int
roseBuild d ? d <= 0 = (1, [])
            : let sub = roseBuild (d - 1)
              in (0, [sub, sub])

roseSum :: Rose Int -> Int
roseSum x = .0 x + fold (\acc child -> acc + roseSum child) 0 (.1 x)
```

```console
$ ./recur roseBuild 2
[0,[[0,[[1,[]],[1,[]]]],[0,[[1,[]],[1,[]]]]]]
$ ./recur roseSum '[1,[[2,[]],[3,[]]]]'
6
```

### Record form

The same rules apply to `record` declarations. The only surface difference is that fields are addressed by name instead of position; the wire format and the typecheck rules are identical to the tuple-alias form. This is an alternative encoding of the same linked list, so it lives in its own program below — two declarations of `LL` cannot share a module.

```morloc
record LL where
  head :: Int
  tail :: ?LL

llRecordExample :: LL
llRecordExample = {head = 42, tail = {head = 7, tail = Null}}

llLen :: LL -> Int
llLen x ? isNull (.tail x) = 1
        : 1 + llLen (require (.tail x))
```

```console
$ ./recrec llRecordExample
{"head":42,"tail":{"head":7,"tail":null}}
$ ./recrec llLen '{"head":1,"tail":{"head":2,"tail":null}}'
2
```

### Parameterised recursion

Recursive types can carry type parameters, which thread through every recursive position:

```morloc
record Container a where
  val :: a
  sub :: ?(Container a)

containerExample :: Container Int
containerExample = {val = 1, sub = {val = 2, sub = Null}}

containerLength :: Container a -> Int
containerLength x ? isNull (.sub x) = 1
                  : 1 + containerLength (require (.sub x))
```

```console
$ ./recrec containerExample
{"val":1,"sub":{"val":2,"sub":null}}
```

`containerLength` stays polymorphic in the payload, which is what you want inside a program. It cannot be given a command line interface, though, so exporting it draws a warning and the program builds without that one command:

```console
$ morloc make recrec.loc
Warning: skipping generic export 'containerLength'
```

> **Caution**
> Mutually recursive type aliases — two or more type definitions that reference each other in a cycle — are not supported. The frontend detects them and names the cycle:
> 
> ```console
> recy.loc:5:1: error:
> Mutual recursion between type definitions is not supported. Cycle: A, B
>   |
> 5 | type A = (Int, B)
>   | ^
> ```
> 
> The rule holds across general and language-specific scopes, and whether the cycle lives in one module or spans several.


---

<!-- https://morloc-project.github.io/docs/features/sum-types.html -->

# 4.14. Sum types

Morloc Manual > Syntax and Features | https://morloc-project.github.io/docs/features/sum-types.html | prev: https://morloc-project.github.io/docs/features/recursion.md | next: https://morloc-project.github.io/docs/features/effects.md

A record holds all of its fields at once. A **sum type** holds one shape out of several. You write one with `data`, listing every constructor the type has:

```morloc
data Color = Red | Green | Blue
```

`Color` now has exactly three values. `Red`, `Green` and `Blue` are ordinary terms that you can return, pass, put in a list, or match on.

A constructor belongs to one type and no other, so the compiler works out the type from the constructor alone. That is why `warmest` below needs no signature:

**colors.loc**

```morloc
module main (describe, palette, warmest)

import root-py

data Color = Red | Green | Blue

describe :: Color -> Str
describe | Red = "warm"
         | Green = "cool"
         | Blue = "cold"

palette :: [Color]
palette = [Red, Green, Blue]

warmest = Red
```

```console
$ morloc typecheck colors.loc
describe :: Color -> Str
palette :: [Color]
warmest :: Color
```

`describe` takes the value apart with `|`\-clauses (see [Pattern Matching](https://morloc-project.github.io/docs/features/pattern-matching.md)). A constructor in a clause is a test, not a binding: `Red` matches the value `Red` and nothing else.

```console
$ morloc make -o colors colors.loc
$ ./colors describe Green
"cool"
$ ./colors palette
["Red","Green","Blue"]
```

## 4.14.1. A constructor set is closed, and the compiler counts

A `|`\-match over a `data` type does not need a catch-all, because the compiler knows how many constructors there are. It also does not let you forget one. Drop the `Blue` clause from `describe` and the build stops:

```console
colors.loc:8:1: `|` patterns for 'describe' are not exhaustive; missing Blue
    |
  8 | describe | Red = "warm"
    | ^
```

The same knowledge runs in the other direction. Add a second `Red` clause and it can never fire, so it is rejected rather than silently dropped:

```console
colors.loc:8:1: `|` patterns for 'describe' match 'Red' more than once; the later clause is unreachable
    |
  8 | describe | Red = "warm"
    | ^
```

A catch-all is still allowed when you want one:

```morloc
warm :: Color -> Bool
warm | Red = True
     | _ = False
```

## 4.14.2. The constructor names reach the interface

Constructor names are part of the type, so every interface Morloc derives knows them. On the command line the constructor is written as itself, and help says which words are legal:

```console
$ ./colors describe -h

Usage: ./colors <nexus_options> describe <command_options>

General Options:
  -h, --help  Print help; -hh adds details and examples, -hhh adds schemas
              (nexus options: -h @)

Positional arguments:
  1:  type: Color
      values: Red, Green, Blue

Return: Str
$ ./colors describe Blue
"cold"
$ ./colors describe Purple
Error: failed to parse argument #0: serialization error: 'Purple' is not a constructor of this type; expected one of Red, Green, Blue
```

and a model client is handed a closed set rather than a free-text string:

```console
$ ./colors --mcp-tools
...
      "_1": {
        "type": "string",
        "enum": [
          "Red",
          "Green",
          "Blue"
        ]
      }
...
```

On the command line the case of a constructor does not matter: `blue`, `BLUE` and `Blue` are the same value. The convention that constructors are capitalized is Morloc’s, and a person typing a command should not have to know it. That leniency is the command line’s alone — a quoted JSON string, whether it is the whole argument, a field of a record, or an element of a list, is matched exactly, because JSON is a contract between programs. For the same reason two constructors of one type may not differ only in case; the compiler rejects the declaration.

A constructor can carry a description. Write it above the constructor’s `=` or `|`, the way a record field’s description sits above the field:

```morloc
--' How urgent a task is
data Priority
  --' can wait
  = Low
  --' this week
  | Medium
  --' today
  | High
```

The description reaches every interface. Terminal help prints a `Data Types` block beneath the command at `-hhh`, `--json-help` carries it in the `types` glossary, and the MCP tool folds each constructor’s note into the argument’s description so a model reading the tool sees what the names mean and not only which are legal:

```console
$ ./tasks pick -hhh
...
Optional arguments:
  -p, --priority <Priority>  the priority
                             type: Priority
                             values: Low, Medium, High [default: Medium]

Data Types:
  Priority
    How urgent a task is
    Low     can wait
    Medium  this week
    High    today
```

An option whose type is a `data` may give its default as the bare constructor (`--' @default medium`), and a `@many` option takes bare constructors one per occurrence (`-p low -p high`).

## 4.14.3. Constructors that take arguments

A constructor may carry fields. Write their types after the constructor name:

**shapes.loc**

```morloc
module main (area, describe, grow, columns)

import root-py

data Shape = Circle Real | Rect Real Real | Dot

area :: Shape -> Real
area | (Circle r) = 3.14159 * r * r
     | (Rect w h) = w * h
     | Dot = 0.0

describe :: Shape -> Str
describe | (Circle 0.0) = "a circle of no radius"
         | (Circle _) = "a circle"
         | (Rect _ _) = "a rectangle"
         | Dot = "a dot"

grow :: Real -> Shape -> Shape
grow | k (Circle r) = Circle (k * r)
     | k (Rect w h) = Rect (k * w) (k * h)
     | _ Dot = Dot

columns :: [Real] -> [Shape]
columns = map (Rect 2.5)
```

`Circle 2.0` builds a value. In a pattern, `(Circle r)` matches a circle and binds `r` to its radius. A constructor pattern with fields needs the parentheses, since the fields would otherwise read as further arguments of the clause — which is exactly what they are in `grow`, whose clauses each carry two patterns, one per argument.

A constructor is a function of its fields, so it partially applies like any other. `Rect 2.5` in `columns` is a `Real → Shape` waiting for a height.

```console
$ morloc make -o shapes shapes.loc
$ ./shapes area '{"Circle":[2.0]}'
12.56636
$ ./shapes area '"Dot"'
0
$ ./shapes grow 1.5 '{"Rect":[1.5,2.5]}'
{"Rect":[2.25,3.75]}
$ ./shapes columns '[1.25,4.5]'
[{"Rect":[2.5,1.25]},{"Rect":[2.5,4.5]}]
```

A constructor with fields is JSON `{"Circle":[2.0]}` — one key, the constructor, and its fields in declaration order. One with no fields is the bare string `"Dot"`. That is the whole encoding, and it is what you type on the command line, send over HTTP, and read back out.

> **Warning: Quote a shape on the command line**
> A constructor-only `data` takes a bare word (`./colors describe Blue`), because its argument is a string as far as the interface is concerned. A `data` with fields does not: its argument is JSON, so a nullary constructor has to be written `'"Dot"'`, quoted twice. An unquoted `Dot` is rejected as neither JSON nor a file path.

A field can be matched rather than bound. `(Circle 0.0)` in `describe` matches only a circle of that radius, so it refines `Circle` without closing it — every other circle falls through to the clause below, and the compiler still requires that clause:

```console
$ ./shapes describe '{"Circle":[0.0]}'
"a circle of no radius"
$ ./shapes describe '{"Circle":[2.0]}'
"a circle"
```

Field counts are checked against the declaration:

```console
arity.loc:8:9: constructor 'Circle' takes 1 argument but the pattern gives 2
    |
  8 | area | (Circle r h) = r * h
    |         ^
```

Fields are positional and have no names, so there is no getter into a `data` type — which field exists depends on which constructor you have, and a getter cannot ask. Matching is the only way in. When you want names, put a record in the arm.

## 4.14.4. Matching a value that is not an argument

`|`\-clauses take a definition’s arguments apart. To take apart anything else, use a `match` expression (see [`match` expressions](https://morloc-project.github.io/docs/features/pattern-matching.md#match-expressions)), which accepts the same constructor patterns and the same exhaustiveness rule. Here the value being matched is the parameter of a local helper:

```morloc
totalArea :: [Shape] -> Real
totalArea shapes = sum (map one shapes)
  where
    one :: Shape -> Real
    one s = match s
      | (Circle r) = 3.14159 * r * r
      | (Rect w h) = w * h
      | Dot = 0.0
```

```console
$ ./shapes totalArea '[{"Circle":[1.0]},{"Rect":[2.0,3.0]},"Dot"]'
9.14159
```

The signature on `one` is doing work. A constructor pattern is checked against the type it is matching, and the compiler will not run that in reverse: it cannot infer `Shape` from seeing `Circle` in a pattern, the way it infers `Color` for `warmest = Red` from seeing a constructor in an expression. Leave the signature off and the build stops with `'Circle' is not a constructor of` followed by an unsolved type variable. The same holds for a definition’s `|`\-clauses, so give any term you match on a signature.

## 4.14.5. Recursive types

A constructor may take its own type. That is how you get a tree:

**tree.loc**

```morloc
module main (total, depth)

import root-py

data Tree = Leaf | Node Real Tree Tree

total :: Tree -> Real
total | Leaf = 0.0
      | (Node v l r) = v + total l + total r

depth :: Tree -> Int
depth | Leaf = 0
      | (Node _ l r) = 1 + max (depth l) (depth r)
```

```console
$ ./tree total '{"Node":[1.5,{"Node":[2.25,"Leaf","Leaf"]},"Leaf"]}'
3.75
$ ./tree depth '{"Node":[1.5,{"Node":[2.25,"Leaf","Leaf"]},"Leaf"]}'
2
```

Two `data` types may also refer to each other, which is the shape an abstract syntax tree takes: an expression holds a term and a term holds an expression.

**ast.loc**

```morloc
module main (eval)

import root-py

data Expr = Lit Real | Neg Term | Add Expr Expr

data Term = Wrap Expr | Zero

eval :: Expr -> Real
eval | (Lit v) = v
     | (Neg t) = 0.0 - evalT t
     | (Add a b) = eval a + eval b

evalT :: Term -> Real
evalT | (Wrap e) = eval e
      | Zero = 0.0
```

```console
$ ./ast eval '{"Add":[{"Lit":[1.5]},{"Neg":[{"Wrap":[{"Lit":[2.0]}]}]}]}'
-0.5
```

A record may sit on such a cycle too, as long as a `data` is on it as well — with one caveat. A record on a cycle is a recursive record, and in C and Rust a recursive record still has to be a type you write yourself; the compiler does not yet generate one (\`record Cpp => Node = "struct"\` on a cycle fails at build time). In C that leaves no way through at all, since a header you write is included before the `data` type it would have to name. Python and R take the shape as it is.

What may not close a cycle is a set of records or aliases alone:

**mutual.loc**

```morloc
type A = [B]
type B = [A]
```

```console
mutual.loc:5:1: error:
Mutual recursion between type definitions is not supported unless a `data` type cuts the cycle. Cycle: A, B
  |
5 | type A = [B]
  | ^
```

The reason is what a `data` does that an alias or a record does not. A constructor’s fields sit behind a pointer in every language, so a value’s size does not depend on how deep the recursion goes; and the compiler never expands a `data` into its constructors when it reduces a type, so a cycle through one cannot send it round forever. A record’s fields are laid out inline and an alias is expanded on sight, and neither gives a cycle a place to stop.

## 4.14.6. Constructor names are global

A constructor name determines its type, which only works if the name is claimed once. Declaring it twice is an error at the second declaration:

```console
dup.loc:6:22: Constructor 'Red' is already declared by another `data` type; constructor names must be unique
    |
  6 | data Fruit = Apple | Red
    |                      ^
```

Constructors travel with their type. Exporting `Color` exports `Red`, `Green` and `Blue` with it, importing `Color` brings them in, and a module that only re-exports `Color` passes them along, so a `data` type declared in one module is usable in another however the two are wired:

**types.loc**

```morloc
module types (Color)

data Color = Red | Green | Blue
```

**main.loc**

```morloc
module main (favourite)

import .types (Color)

favourite :: Color
favourite = Blue
```

An import that gives the module an alias puts its constructors behind that alias, the same way it does every other imported name. Write `p.Red` in an expression and in a pattern alike:

```morloc
module main (warm)

import .types as p

warm :: Color -> Bool
warm | p.Red = True
     | _ = False
```

The alias is the only qualifier there is. A module’s own name is not one — a name like `root-py` is not something an expression can spell — and neither is the type’s. Two modules that each declare a `Red` can therefore both be used from a third by giving at least one of them an alias; two `data` types in the **same** module cannot share a constructor name.

## 4.14.7. Native representations

Every language gets a representation of a `data` type, and by default the compiler writes it: an `enum class` in C++, a `#[repr(u8)]` enum in Rust, an ordinal in Python, an ordered factor in R. You declare nothing, and functions written in Morloc work across all four.

Native code you **source** is a different matter, because it has to name the type to take it apart. Rust can name a generated type directly. C++ cannot: a sourced header is included before the compiler’s own declarations, so a header that mentions `Shape` must declare `Shape` itself, and you tell Morloc that with a per-language declaration — the same `⇒` form records already use:

```morloc
data Cpp => Shape = "Shape"
```

The mapping says only which native name to use. The constructors and their field types are not repeated, and the native declaration has to match the layout Morloc expects: a wrapper `Shape`, one `Shape_<Constructor>` struct per arm, and fields named `f0`, `f1` and so on. The foldout at the end of this section gives that layout for each language.

**shapes.hpp**

```cpp
#pragma once
#include <memory>
#include <variant>

struct Shape_Circle;
struct Shape_Rect;
struct Shape_Dot;

struct Shape {
    std::variant<std::shared_ptr<Shape_Circle>,
                 std::shared_ptr<Shape_Rect>,
                 std::shared_ptr<Shape_Dot>> v;
};

struct Shape_Circle { double f0; };
struct Shape_Rect   { double f0; double f1; };
struct Shape_Dot    { };

inline double area(Shape s) {
    if (auto p = std::get_if<std::shared_ptr<Shape_Circle>>(&s.v))
        return 3.14159 * (*p)->f0 * (*p)->f0;
    if (auto p = std::get_if<std::shared_ptr<Shape_Rect>>(&s.v))
        return (*p)->f0 * (*p)->f1;
    return 0.0;
}
```

Python needs no declaration. A value with fields arrives as a pair of the constructor’s name and a tuple of its fields:

**shapes.py**

```python
def grow(k, s):
    match s:
        case ("Circle", (r,)):
            return ("Circle", (k * r,))
        case ("Rect", (w, h)):
            return ("Rect", (k * w, k * h))
        case _:
            return s
```

**crossing.loc**

```morloc
module main (areaCpp, growPy, bigArea)

import root-py
import root-cpp

data Shape = Circle Real | Rect Real Real | Dot

data Cpp => Shape = "Shape"

source Py  from "shapes.py"  ("grow" as growPy)
source Cpp from "shapes.hpp" ("area" as areaCpp)

growPy  :: Real -> Shape -> Shape
areaCpp :: Shape -> Real

bigArea :: Shape -> Real
bigArea s = areaCpp (growPy 2.0 s)
```

```console
$ ./crossing growPy 1.5 '{"Circle":[2.5]}'
{"Circle":[3.75]}
$ ./crossing bigArea '{"Rect":[1.5,2.25]}'
13.5
```

`bigArea` grew the shape in Python and measured it in C++. One declaration, two native representations, and a wire form both agree on.

Forget the mapping and the C++ compiler reports `error: redefinition of 'struct Shape'` against the generated pool source. It means the mapping is missing, not that your header is wrong.

## 4.14.8. Comparing values

`==` compares two `data` values by constructor first and then by field, so `Circle 2.0 == Circle 2.0` is `True` while `Rect 1.0 2.0 == Rect 2.0 1.0` is `False`.

The ordering operators use the same order the declaration does. `Red < Green` is `True` because `Red` is declared first, and so is `Circle 1.0 < Dot`. Two values of the same constructor are ordered by their fields, so `Circle 1.0 < Circle 2.0`.

The answer does not depend on where the comparison runs. Which pool the compiler picks for an expression is its choice rather than yours, so a comparison that meant one thing in Python and another in R would be a bug you could not see in the source.

One gap to know about: the Python and R representations shown above are an interim form, to be replaced by a generated class per arm. Treat the pair shape as something to match on rather than something to build a library around.

**How a data value is encoded**

For the reader who wants the bytes. It assumes you know what a tagged union is, and nothing about Morloc beyond this section. Two terms are used below: a **pool** is the process that runs one language’s share of a program, and the **nexus** is the process that drives them. Values move between them through shared memory, as a fixed-layout binary value with a **schema** string describing it.

**A constructor-only `data` is one byte.** The byte is the constructor’s 0-based position in the declaration. Alignment is 1 and the width is fixed, so an array of them is a flat buffer copied in bulk: a `[Color]` occupies one byte per element and is byte-for-byte a `[U8]`. The limit is 256 constructors, because the tag is a byte.

**A `data` with fields is sixteen bytes**, whatever its arms hold: a tag byte at offset 0, seven bytes of padding, and a relative pointer at offset 8 to the arm’s fields, laid out as a tuple. An arm with no fields stores a null pointer. Alignment is 8 and the width is never fixed, so an array of them is walked rather than copied.

The pointer is the reason recursion terminates. An inline payload would give `Tree` the width equation `width >= 1 + 2 * width`, which has no solution; behind a pointer, every arm costs the same sixteen bytes. It also makes appending an arm layout-neutral.

**The schema** travels with the value and carries the constructor names. Counts and lengths are one character from a 64-symbol alphabet (`0`\-`9`, `a`\-`z`, `A`\-`Z`, `+`, `/`); a value of 64 or more is written low digit first, with `=` before each digit but the last.

| Form | Meaning |
| --- | --- |
| `e<count>(<len><name>)*` | A constructor-only `data`. Names in declaration order. |
| `v<count>(<len><name><arity><field schema>*)*` | A `data` with fields. Each arm names itself, states how many fields it has, and then gives their schemas. |
| `&<len><name>` | Declares a name for the schema that follows, so it can be referred to again. |
| `^<len><name>` | A back-reference to a declared name. |

The three types in this section:

```text
Color   e33Red5Green4Blue
Shape   v36Circle1f84Rect2f8f83Dot0
Tree    &4Treev24Leaf04Node3f8^4Tree^4Tree
```

Read `Shape` as: variant, 3 arms; `6`\-character name `Circle` with `1` field of type `f8` (an 8-byte float); `4`\-character name `Rect` with `2` fields, both `f8`; `3`\-character name `Dot` with `0` fields. `Tree` declares its own name first, because its arms point back at it.

Nothing in `e` or `v` carries the **type’s** name — only a recursive type declares one, and only so its arms can refer back. Two `data` types with the same constructor names and field types therefore have the same wire form and are interchangeable at a boundary. Records behave the same way; the encoding is structural.

**Outside shared memory** a value takes one of two forms. In MessagePack, which carries packets and on-disk values, a constructor-only `data` is its ordinal and one with fields is the two-element array `[tag, fields-or-nil]` — ordinals rather than names, because spelling out a constructor for every element would multiply the size of a large array. In JSON, which is what the command line, the HTTP API and the MCP tool descriptions speak, both forms use names.

**Wire compatibility follows from the tag being the declaration ordinal.** Appending a constructor leaves every existing value byte-identical, and changes the schema only by its arm count and the new name. Reordering or removing constructors changes what old bytes mean, and is a breaking change to every stored value and to every peer that has not been rebuilt.

**What a data type looks like in each language**

For the reader writing native code against a Morloc `data` type. Each row is what a **sourced** function receives and must return. Morloc generates these declarations itself unless you map the type with `data <Lang> ⇒ T = "<name>"`, in which case your file declares it and must match the layout below.

| Language | Constructors without fields | Constructors with fields |
| --- | --- | --- |
| Python | The ordinal, as an `int`. A list of them is a byte buffer, the same object a `[U8]` produces. | `("Circle", (2.0,))` — the constructor’s name and a tuple of its fields. |
| R | An ordered `factor`, so `<` compares in declaration order. Codes are 1-based, so a code is the wire tag plus one; the levels are the constructor names in declaration order. | `list("Circle", list(2.0))` — the name and a list of fields. |
| C++ | `enum class Color : uint8_t` with explicit discriminants, so the native value and the wire tag are the same byte. | A wrapper struct holding a `std::variant` of `std::shared_ptr` to one struct per arm. |
| Rust | `#[repr(u8)]` enum with explicit discriminants, deriving `Clone`, `Copy`, `PartialEq`, `Eq`, `PartialOrd`, `Ord` and `Debug`, so it compares in declaration order. | An enum whose arms each hold one `Box` of a tuple of their fields. Not `Copy`, since a box is not. |

The generated C++ and Rust declarations, for the `Shape` of this section:

```cpp
struct Shape_Circle;
struct Shape_Rect;
struct Shape_Dot;
struct Shape {
    std::variant<std::shared_ptr<Shape_Circle>, std::shared_ptr<Shape_Rect>, std::shared_ptr<Shape_Dot>> v;
};
struct Shape_Circle {
    double f0;
};
struct Shape_Rect {
    double f0;
    double f1;
};
struct Shape_Dot {
};
```

```rust
#[derive(Clone)]
pub enum Shape {
    Circle(::std::boxed::Box<(f64,)>),
    Rect(::std::boxed::Box<(f64, f64)>),
    Dot,
}
```

Three consequences worth knowing before you write against them.

Arm fields have no names in Morloc, so the C++ form names them `f0`, `f1`, and so on by position, and Rust reaches them as tuple elements. A mapped type that spells a field differently will not compile, which is the outcome you want.

Every arm is behind a pointer in both compiled languages even when its fields would fit inline. That is what gives a recursive type a finite size, and it is uniform so that no per-type analysis decides it.

A type with parameters maps to a template, exactly as an alias does (`type Cpp ⇒ (List a) = "std::vector<$1>" a`), and each instantiation names the template with that instantiation’s arguments: `Box Int` is `MyBox<int>` and `Box Str` is `MyBox<std::string>`. In C++ the arms are templates too, named by appending `_<Constructor>` to the wrapper’s **head** and taking the same arguments — `MyBox<$1>` has the arms `MyBox_Empty<$1>` and `MyBox_Full<$1>`. In Rust it is an ordinary generic enum.

```morloc
data Box a = Empty | Full a

data Cpp  => (Box a) = "MyBox<$1>" a
data Rust => (Box a) = "MyBox<$1>" a
```

```cpp
template <typename T> struct MyBox_Empty;
template <typename T> struct MyBox_Full;

template <typename T> struct MyBox {
    std::variant<std::shared_ptr<MyBox_Empty<T>>,
                 std::shared_ptr<MyBox_Full<T>>> v;
};

template <typename T> struct MyBox_Empty {};
template <typename T> struct MyBox_Full { T f0; };
```

```rust
#[derive(Clone)]
pub enum MyBox<T> {
    Empty,
    Full(::std::boxed::Box<(T,)>),
}
```

The same holds for a `record` with parameters: `record Rust ⇒ (Wrap a) = "MyWrap<$1>" a` names a `struct MyWrap<T>`. Python and R still declare nothing; a mapping there is a hint carried on the wire and the value keeps its structural shape.

Python and R declare nothing. The pair above is a structural interim representation, chosen because it needs nothing the generic marshaller cannot already build; a generated class per arm is the intended end state. The cost of the interim form is that neither language’s compiler — and neither has one — checks that you built an arm correctly, so a field order swapped between two same-typed fields is silently wrong in Python and R where C++ and Rust would reject it.


---

<!-- https://morloc-project.github.io/docs/features/effects.html -->

# 4.15. Effects and delayed evaluation

Morloc Manual > Syntax and Features | https://morloc-project.github.io/docs/features/effects.html | prev: https://morloc-project.github.io/docs/features/sum-types.md | next: https://morloc-project.github.io/docs/features/optionals.md

## 4.15.1. Why effects need a name

Morloc is a functional language. A function maps a value in one domain to a value in another, and the mapping is the function’s whole meaning. That works neatly for arithmetic, for string manipulation, for transforming records. It runs into trouble the moment we try to talk about anything that touches the world.

Consider `readFile`:

```morloc
readFile :: Str -> Str
```

This looks like a function from a filename to a string. Indeed it **is** a function at any given instant on a given machine: the filename names a particular file, and the file has particular contents. But files change. If we read the same file twice in the same program, we may get two different answers. So it matters **when** we call the function and we may want to call it a several different points in time.

The same problem shows up for "values" that are not really values. What is the type of the current time? What is the type of a coin toss?

```morloc
time     :: ???
coinToss :: ???
```

We could try to make them into honest functions by handing them an explicit world or an explicit random seed — `time :: TemporalState → Time` and `coinToss :: RNG → (Bool, RNG)` — and thread that state through every call that needs it. This can work, but it pulls extra plumbing into every signature.

Morloc takes a different route: it gives the effect a **name** at the type level. `<Rand> Bool` is not a `Bool`; it is a **suspended computation** that, when run, performs the `Rand` effect and yields a `Bool`. Where the original problem was "this looks like a value but doesn’t act like one", the solution is to give it a type that says so.

> **Note**
> `<E> T` is a **suspended computation** that performs effects `E` and yields a `T`. It is not a `T`. You obtain a `T` by running it.

## 4.15.2. The mental model

-   `<E> T` is a suspension. Holding one in a variable does nothing, and neither does passing it, storing it in a record or list, or returning it. It is a value, and the same value in every position.
-   The bind arrow `<-` runs a suspension once and gives you a result. Run it twice and it runs twice: nothing is remembered between runs.
-   A bare statement inside a `do`\-block runs a suspension and nothing reads the result. This is how you sequence side effects whose return values you do not need. If that result reports a failure, the block stops there; see [Failure is not an effect](#failure-is-not-an-effect).
-   `let` binds without running. If the right-hand side is a plain effectful expression, the suspension is what gets bound; it only fires when a later `<-` reaches for it.
-   `!e` is inline shorthand for `<-`. Instead of writing `x <- e` and using `x` downstream, write `!e` where you want the value; the compiler inserts the bind at the nearest enclosing scope.
-   When you export an `<E> T`, the compiled program runs it for you at the boundary. The caller receives a `T`.

Effect labels are names that the compiler propagates and checks for coverage. What an effect **means** — what `IO` permits at runtime, what `Rand` looks like operationally — is the business of the library that defines the effect, not the compiler. The compiler’s job is to keep the labels honest; libraries build behaviour on top.

## 4.15.3. Failure is not an effect

An effect row says what a computation may **do**. Whether it succeeded is a property of what it **returns**, so failure is not an effect and Morloc does not track it as one. A fallible operation returns a value that is either the answer or the reason there isn’t one:

```morloc
data Try e a = Err e | Ok a
```

That is an ordinary sum type (see [Sum types](https://morloc-project.github.io/docs/features/sum-types.md)), declared in the `internal` standard library module and re-exported by `root`. Nothing about it is built into the compiler. The error parameter comes first so that a later functor maps over the payload rather than the error.

A function that may fail says so by returning one, and the caller takes it apart the way it takes any sum type apart:

```morloc
source Py from "eff.py" ("lookupPort")
lookupPort :: Str -> <IO> (Try Str Int)

describePort :: Str -> <IO> Str
describePort name = do
  r <- lookupPort name
  match r
    | (Ok p)  = "port #{@show p}"
    | (Err e) = "unknown: #{e}"
```

```console
$ ./effects describePort https
"port 443"
$ ./effects describePort gopher
"unknown: no port for gopher"
```

The effect row still carries `IO`, because looking the port up does touch the world. What it no longer carries is any claim about failing.

`describePort` takes the `Try` apart inside the pool, but it does not have to: a `Try` crosses a pool boundary like any other value, and so does a list of them. Its wire form follows from its declaration, the way a tuple’s does, and no `Packable` instance stands between the two — see [A `do`\-block does not return a `Try`](#do-block-no-try) for a function that hands one back to its caller.

Failure that is **not** a value is still possible, and common: any function you source can raise in its own language, and Morloc does not see that coming. `@try` turns such a raise into a `Try` and `@throw` produces one deliberately; both are covered in [Failure and recovery](https://morloc-project.github.io/docs/features/intrinsics.md#failure-and-recovery). The honest summary is that a signature tells you what a call may do and what it returns, and not every way it can go wrong — a trade made for signatures that stay readable, since almost every function that touches the world can fail somehow.

## 4.15.4. Syntax

### Declaring an effect

Every effect label a program uses must be declared:

```morloc
effect IO
escapable effect Rand
```

The default form is inescapable; the `escapable` form is discussed in [Escapable and inescapable effects](#escapable). Declarations are global to the program; two modules cannot declare the same label with conflicting escapability.

A `<L>` that has not been declared is a compile error — the compiler does not know any effect names of its own.

One effect comes pre-declared, in the `internal` stdlib module that most user code imports transitively through `root`: `effect IO`. Every intrinsic that touches the world carries it. Every other label is yours to declare, in the module that establishes what it means.

### Annotating signatures

An effect annotation goes immediately before the type it wraps:

```morloc
readFile :: Path -> <IO> Str
rollDie  :: Int -> <Rand> Int
fetch    :: Url -> <IO, Net> Bytes
```

Multiple labels are comma-separated inside a single pair of angle brackets. Order does not matter; `<IO, Net>` and `<Net, IO>` are the same row.

The empty row `<>` is a row like any other: `<> T` is a suspension that performs nothing when run, and it is not a `T`. You rarely write it, but a `do`\-block that runs nothing has this type, and it fits any effect slot, since the empty row is included in every row (see [The rules](#rules)).

### do-blocks

A `do`\-block strings statements together. It is the only construct in which effects are actually run. Inside a block there are exactly four forms of statement:

| Form | Meaning |
| --- | --- |
| `x ← e` | Run `e`, bind the result to `x`. |
| `Ok x ← e` | Run `e`, match the result against a refutable pattern, and throw if it does not match. |
| `e` | (bare) Run `e`. Nothing reads the result. |
| `let x = e` | Bind `x` to `e` without running anything. |

The final statement of a `do`\-block is its return value. The block’s overall type is `<U> T`, where `U` is the union of all the statements' effects and `T` is the type of the final statement.

A worked example covering the three that run something:

```morloc
sideEffect :: Int -> <IO> Int
add        :: Int -> Int -> Int

example :: <IO> Int
example = do
    let t = sideEffect 3     -- t :: <IO> Int, NOT run
    sideEffect 1              -- runs, nothing reads the result
    x <- sideEffect 5         -- runs, x = 10
    let y = add x 1           -- y = 11, no run
    z <- t                    -- NOW t runs; z = 6
    add y z                   -- returns 17
```

```console
$ ./effects example
17
```

Trace it once and the model sticks: `let t = sideEffect 3` binds a suspension and runs nothing; the bare `sideEffect 1` runs and nothing reads its result; `x ← sideEffect 5` runs and binds 10; `let y = add x 1` is pure arithmetic giving 11; `z ← t` finally runs the suspension bound at the top, giving 6; and `add y z` returns 17.

Both layout-indented form (as above) and brace form (`do { x ← e; y ← f; …​ }`) are accepted.

### A bare statement checks its result

The bare form is how you write a script: a sequence of steps run for what they do, not for what they return. A step that can fail returns a `Try`, and a returned failure that nobody looks at is a failure nobody notices. So the rule is:

-   **A bare statement stops the block when its result is a failure.**

`logLine` appends a line to a log and refuses an empty one, so it returns a `Try`. Written bare, a failing line ends the run; the statements after it, including the block’s return value, never happen:

```morloc
source Py from "eff.py" ("record" as logLine)
logLine :: Str -> <IO> (Try Str ())

logTwo :: Str -> Str -> <IO> Str
logTwo a b = do
  logLine a
  logLine b
  "both recorded"
```

```console
$ ./effects logTwo alpha beta
recorded: alpha
recorded: beta
"both recorded"
$ ./effects logTwo alpha ""
recorded: alpha
Error: run failed
refusing to record an empty line
```

The rule is stated in terms of the statement’s **result**, and a bare statement is exactly the one whose result nothing reads. Bind it and the rule does not fire, because now something does read it and what happens next is your business. Binding to `_` is how you say that out loud — the failure goes into a hole on purpose:

```morloc
logTwoLoose :: Str -> Str -> <IO> Str
logTwoLoose a b = do
  _ <- logLine a
  _ <- logLine b
  "both recorded"
```

```console
$ ./effects logTwoLoose alpha ""
recorded: alpha
"both recorded"
```

### Refutable binds

A `do`\-bind may carry a refutable pattern (see [Refutable patterns](https://morloc-project.github.io/docs/features/pattern-matching.md#refutable-patterns)). It runs the statement, matches the result, and throws if the match fails. Against a `Try` that gives you the value on the success path and stops the block otherwise, which is the short way to write "I want the answer, and a failure here is fatal":

```morloc
portOf :: Str -> <IO> Int
portOf name = do
  Ok p <- lookupPort name
  p
```

```console
$ ./effects portOf https
443
$ ./effects portOf gopher
Error: run failed
{"Err":["no port for gopher"]}
  at portOf [py] (mid=6, effects.loc:1:68)
```

The thrown message is the unmatched value, rendered. For a `Try` that carries the failure’s own message inside it, which is why the load error shows up in the traceback above.

### A `do`\-block does not return a `Try`

The bare-statement rule covers a result nothing reads. It does not cover the block’s final statement, which **is** the block’s return value and so is read by whoever called it. A `Try` there is a `Try` in the block’s type, and if the signature says otherwise the block does not typecheck. `@savej` writes a value to a file and may fail, so it returns one (see [Intrinsics](https://morloc-project.github.io/docs/features/intrinsics.md)):

```morloc
saveNote :: Str -> [Str] -> <IO> ()
saveNote path xs = do
  @savej path (id xs)
```

```console
save-bad.loc:7:20-9:1: error:
Type mismatch:
  expected: <IO> Unit
  inferred: <IO> (Try Str Unit)
Cannot compare types Try Str Unit and Unit
  |
7 | | saveNote path xs = do
  | |                    ^
8 | |   @savej path (id xs)
  | | ^
```

Three ways out, and which is right depends on what the caller should see. Declare the `Try` and hand the failure back as data; unwrap it, which throws; or make the fallible call a bare statement and return something else:

```morloc
asData :: Str -> [Str] -> <IO> (Try Str ())
asData path xs = @savej path (id xs)

orThrow :: Str -> [Str] -> <IO> ()
orThrow path xs = do
  r <- @savej path (id xs)
  unwrap r

bareThenUnit :: Str -> [Str] -> <IO> ()
bareThenUnit path xs = do
  @savej path (id xs)
  ()
```

```console
$ ./save asData nope/x.json '["a"]'
Error: run failed
{"Err":["IO error: No such file or directory (os error 2)"]}
$ ./save bareThenUnit nope/x.json '["a"]'
Error: evaluation failed: IO error: No such file or directory (os error 2)
$ ./save bareThenUnit out.json '["a"]'
```

**\`unwrap**

Try Str a → a\` comes from the standard library and does exactly what the bare-statement rule does: hand back the `Ok` payload, or throw the `Err` message. It is the explicit form of the same decision. The two failing runs above differ only in who reports: `asData` returns the failure and the program prints it as its result, while the other two throw and the program dies. Both exit non-zero.

### When `do` is needed and when it isn’t

A `do`\-block is not always required. A single effectful expression stands on its own:

```morloc
forceOnce :: <IO> Int
forceOnce = sideEffect 5
```

Use a `do`\-block when you need to sequence multiple statements, bind intermediate results, or run a suspension for its effects only. A `do`\-block is itself an expression, so it can appear as an argument.

### The `!` eval prefix

Inside an expression, `!e` runs `e` in place. It is surface syntax only: the compiler rewrites it to a `←` bind at the nearest enclosing scope and threads the bound name through. Effects propagate outward exactly as they would if you had written the bind by hand.

```morloc
readValue :: <IO> Int

pair :: <IO> (Int, Int)
pair = (!readValue, !readValue)
    -- equivalent to `do { a <- readValue ; b <- readValue ; (a, b) }`
```

```console
$ ./effects pair
[7,7]
```

The rewrite lands at the **nearest** enclosing scope. Inside a lambda body the inserted `do`\-block goes in the body, so the effect fires when the lambda is applied, not when it is created:

```morloc
addOne   :: Int -> <IO> Int

readOnce :: () -> <IO> Int
readOnce = \_ -> !(addOne 1)
    -- equivalent to `\_ -> do { v <- addOne 1 ; v }`
```

Inside an `if` (or guard) branch each branch gets its own scope, so only the taken branch’s effect fires. Inside an existing `do`\-block, `!e` becomes a bind inserted immediately before the current statement, preserving left-to-right effect order.

The prefix binds tightly: `f !x` parses as `f (!x)`, not as `!(f x)`. Use parentheses for the latter.

`!` is rejected at positions where it would be redundant or would put an effect where the surface reads as pure:

-   `x <- !e` — the bind already runs `e`; write `x <- e`.
-   `!e` as a bare non-final `do`\-statement — bare statements already run.
-   `let x = !e` (or any `!` whose scope would land above the `let`) — `let` binds pure values; hoisting an effect above the binding would make the line read misleadingly. Use `x <- e` inside a `do`\-block. A `!` sealed by an inner boundary (a lambda body, a nested `do`, a guard branch under the let) is unaffected.

## 4.15.5. The reading: a suspension is a value

`<E> T` is a **suspension**: a value that, when run, may perform the effects in `E` and yields a `T`. It is not a `T`, and a `T` is not a suspension. There is no coercion between them in either direction. The only way from `<E> T` to `T` is to run it, with `<-` inside a `do`\-block; the only way from `T` to `<E> T` is to build a suspension around it, with `do`:

```morloc
foo :: <IO> Int
foo = do 42                  -- a suspension that yields 42 and performs nothing
```

`foo = 42` is a type error. The row `<E>` is an upper bound on what a run may do, so a suspension that performs nothing (`<> Int`) fits any slot (`<IO> Int`), which is why `do 42` is enough and no `pure` or `return` keyword exists. The `do` is the whole ceremony: it says, in one word, "this is a computation, not a value". That distinction is exactly what lets Morloc pass a suspension to another language as a callable, run it once per use, and know that nothing ran when it was merely held.

## 4.15.6. The rules

The whole type-checking story for effects is four rules.

1.  **A value is not a suspension.** `T` never fills an `<E> T` slot; `do v` does. A `do`\-block that runs nothing has type `<> T`, and the empty row is included in every row, so `do v` fills any `<E> T` slot.
2.  **More effects are a supertype of fewer.** `<E1> T <: <E2> T` exactly when the concrete labels of `E1` are a subset of `E2`. A `<IO> Int` is usable where `<IO, Net> Int` is expected; the reverse is not.
3.  **Effects don’t leak silently.** A value of type `<E> T` cannot be assigned to a slot of type `T`. It can fill a type variable, since it is a value like any other: `id (readValue)` is an `<IO> Int`, and a list of suspensions is a list. If you intend the effect to escape, you say so in the type.
4.  **A `do`\-block collects.** Its row is the union of its statements' rows; its type is `<that-union> T`, where `T` is the type of its final statement.

A few illustrations:

```morloc
-- Rule 1: a suspension of a value fills an <IO> slot; the value does not
pureFortyTwo :: <IO> Int
pureFortyTwo = do 42               -- OK
notASuspension :: <IO> Int
notASuspension = 42                -- ERROR: Int is not <IO> Int

-- Rule 2: widening is fine
ioFunc      :: <IO> Int
testSubtype :: <IO, Net> Int
testSubtype = do
  x <- ioFunc
  x                                -- OK: <IO> <: <IO, Net>

-- Rule 2: narrowing is rejected
readValue :: <IO, Net> Int
a         :: <IO> Int
a = do
  x <- readValue                   -- ERROR: Net not in <IO>
  x

-- Rule 3: effects can't be dropped into a pure slot
readValue :: <IO> Int
b         :: Int
b = readValue                      -- ERROR: <IO> Int is not Int

-- Rule 4: the union of statements' effects
readValue :: <IO> Int
sample    :: Int -> <Rand> Int

combined :: <IO, Rand> Int
combined = do
  x <- readValue                   -- contributes <IO>
  y <- sample x                    -- contributes <Rand>
  y
```

The three rejections above are real. A value in a suspension’s slot:

```console
rule1.loc:7:18: error:
Type mismatch:
  expected: <IO> Int
  inferred: Int
Cannot compare types Int and <IO> Int
  |
7 | notASuspension = 42
  |                  ^
```

The compiler names the fix in the other two cases. Narrowing:

```console
rule2.loc:12:5-15:1: error:
Type mismatch:
  expected: <IO> Int
  inferred: <IO,Net> Int
Subtype error: body performs effect(s) <Net> not in the declared type. Fix by declaring the missing effect(s) in the signature.
  <IO,Net> Int <: <IO> Int
   |
12 | | a = do
   | |     ^
13 | |   x <- readValue
14 | |   x
   | | ^
```

and dropping an effect into a pure slot:

```console
rule3.loc:10:5: error:
Type mismatch:
  expected: Int
  inferred: <IO> Int
Subtype error: an effectful value cannot be used where a non-effectful type is expected; bind it in a do-block first (x <- e) and pass the bound value, e.g. `do { x <- e ; f x }` instead of `f e`
  <IO> Int <: Int
   |
10 | b = readValue
   |     ^
```

There is one more guarantee the user sees but does not write down: an exported `<E> T` is run automatically at the boundary. The compiled program’s user receives a `T`. Effects do not escape the binary.

## 4.15.7. Effect row variables

Combinators that **thread** effects need to be able to talk about sets of unknown effects. For that, an effect row may include a single lowercase variable that represents a set of zero or more unknown effects:

The function `mapE`, below, carries the all the effects of the mapping function to the final value:

```morloc
mapE :: (a -> <e> b) -> [a] -> <e> [b]
```

Effect variables and constants may be mixed, but at most one effect variable can appear in a given effect row. So `<A,B,e>` is OK, but `<A,e,f>` is not.

In the following code, the signature requires that `f` may produce a `Rand` effect, and allows it to produce others as well:

```morloc
foo :: (Int -> <Rand,e> Int) -> Int -> <Rand,e> Int
foo f x = do
  y <- f x
  y * 2
```

```console
$ morloc typecheck rowvar.loc
foo :: (Int -> <Rand,e@e0> Int) -> Int -> <Rand,e@e0> Int
```

Had the signature said just `<Rand>`, only that effect would be permitted and any additional one would be a type error. The signature takes two arguments because `foo f x` does; getting that count wrong is an ordinary type error, reported at the definition.

## 4.15.8. Escapable and inescapable effects

The default form `effect E` is **inescapable**. An inescapable effect that appears in a function’s arguments must also appear in its result. The compiler enforces this on every signature, sourced or defined.

```morloc
effect Cap                         -- inescapable

passt :: <Cap> Int -> <Cap> Int    -- OK: Cap propagates
bad   :: <Cap, e> a -> <e> a       -- ERROR: Cap dropped from result
```

```console
esc.loc:8:1: error:
Inescapable effect 'Cap' appear(s) in an argument but not in the result row. An inescapable effect performed via an argument must propagate to the result (only a sourced handler may discharge an escapable effect).
  |
8 | bad   :: <Cap, e> a -> <e> a
  | ^
```

Effects may alternatively be defined as **escapable**, which means a function may discharge the effect and drop it from the result row. Only a **sourced** function may do this, because discharging an effect means actually running the suspension — setting up whatever the effect needs, calling the computation inside that setup, and handing back a plain value. That is foreign-language work; there is nothing in Morloc itself that can do it.

Seeded sampling is the shape this fits. `roll` needs a random-number generator, and `withSeed` supplies one, so a call under `withSeed` is reproducible and no longer carries `Rand`:

```morloc
escapable effect Rand

source Py from "rnd.py" ("roll", "withSeed")

roll     :: Int -> <Rand> Int
withSeed :: Int -> <Rand> a -> a

runIt :: Int -> Int
runIt n = withSeed 42 (roll n)
```

**rnd.py**

```python
import random

def roll(n):
    return random.randint(1, n)

def withSeed(seed, thunk):
    random.seed(seed)
    return thunk()
```

```console
$ ./esc2 runIt 6
6
$ ./esc2 runIt 6
6
```

The handler receives the suspension as a callable and decides when to run it. Declare `Rand` without `escapable` and the same `withSeed` signature is rejected, because dropping the effect is exactly what an inescapable one forbids:

```console
esc3.loc:11:1: error:
Inescapable effect 'Rand' appear(s) in an argument but not in the result row. An inescapable effect performed via an argument must propagate to the result (only a sourced handler may discharge an escapable effect).
   |
11 | withSeed :: Int -> <Rand> a -> a
   | ^
```

Nothing in the standard library discharges an effect today; `escapable` is there for handlers you write. Failure used to be the built-in case — an `Err` effect discharged by a `@catch` intrinsic — and it no longer is, for the reason given in [Failure is not an effect](#failure-is-not-an-effect): failing is not something a computation does, it is what its result reports.


---

<!-- https://morloc-project.github.io/docs/features/optionals.html -->

# 4.16. Optional types

Morloc Manual > Syntax and Features | https://morloc-project.github.io/docs/features/optionals.html | prev: https://morloc-project.github.io/docs/features/effects.md | next: https://morloc-project.github.io/docs/features/intrinsics.md

Every language needs a way to say "no value here". Query a database for a row that does not exist, or read a parameter that was never set, and something has to come back. Python has `None`, R has `NULL`, JSON has `null`, and C++ solves it in the library with `std::optional<T>`.

Morloc’s principle is that sourced functions stay idiomatic, so it needs a mechanism that lowers to each language’s own answer while staying consistent across the boundary. That is what the optional type is for.

## 4.16.1. Syntax

The `?` prefix marks a type as optional, and `Null` is the absent value. `?Int` is an integer that might be absent, `?Str` a string that might be. The prefix applies to any type, including lists (`?[Int]`) and records (`?Person`).

```morloc
--' Get the first element of a list, or nothing
safeHead :: [Int] -> ?Int

testNull :: ?Int
testNull = Null
```

> **Note**
> Morloc writes `Null` capitalized in source, following the convention that constructors start with an uppercase letter — the same as `True` and `False`. In JSON output it serializes as lowercase `null`, per the JSON standard.

## 4.16.2. Working with optional values

Functions that produce or consume optionals are sourced like any others:

**main.loc**

```morloc
module main (testSafeHead, testSafeHeadEmpty, testFromNull)

import root-py

safeHead :: [Int] -> ?Int
safeHead xs
  ? length xs == 0 = Null
  : .[0] xs

source Py from "main.py" ("default")
default :: a -> ?a -> a

testSafeHead :: ?Int
testSafeHead = safeHead [10, 20, 30]

testSafeHeadEmpty :: ?Int
testSafeHeadEmpty = safeHead []

testFromNull :: Int
testFromNull = default 0 Null
```

The Python side handles `None` the way Python always does:

**main.py**

```python
def default(default_val, x):
    if x is None:
        return default_val
    return x
```

```console
$ ./main testSafeHead
10
$ ./main testSafeHeadEmpty

$ ./main --keep-null testSafeHeadEmpty
null
$ ./main testFromNull
0
```

> **Note**
> When an exported function’s top-level result is `Null` (or `()`), the nexus prints an empty line rather than the literal `null`. Printing `null` would be noisy in a CLI tool, and a downstream consumer that ingested a stray `null` line could choke on it or, worse, treat it as a valid record. Pass `--keep-null` when you want the literal emitted, as above.

The same shape works in the other languages. In C++, using `std::optional`:

```cpp
#include <optional>

template <class T>
T orDefault(T default_val, const std::optional<T>& x) {
  if(x.has_value()){
    return x.value();
  } else {
    return default_val;
  }
}
```

> **Note**
> The helper is `orDefault`, not `default`. `default` is a C++ keyword, and a function so named will not compile.

And in R, using `NULL`:

```r
orDefault <- function(default_val, x){
  if(is.null(x)){
    return(default_val)
  } else {
    return(x)
  }
}
```

## 4.16.3. Optional record fields

Record fields may be optional, which is what you want for data with missing or unknown values. The `where` form below is an alternative syntax for record declarations, equivalent to the brace syntax in [Records](https://morloc-project.github.io/docs/features/records.md):

```morloc
record Person where
  name :: Str
  age :: ?Int

record Py => Person = "dict"

source Py from "foo.py" ("makePerson")
makePerson :: Str -> ?Int -> Person

alice :: Person
alice = makePerson "Alice" 30

bob :: Person
bob = makePerson "Bob" Null
```

```console
$ ./person alice
{"name":"Alice","age":30}
$ ./person bob
{"name":"Bob","age":null}
```

## 4.16.4. Optionals across languages

An optional produced in one language can be consumed in another with no interop code from you:

```morloc
-- C++ produces an optional value
source Cpp from "foo.hpp" ("cSafeDiv")
cSafeDiv :: Int -> Int -> ?Int

-- Python consumes it
source Py from "foo.py" ("pFromNull")
pFromNull :: Int -> ?Int -> Int

testCppToPy :: Int
testCppToPy = pFromNull (-1) (cSafeDiv 10 3)

testCppToPyNull :: Int
testCppToPyNull = pFromNull (-1) (cSafeDiv 10 0)
```

```console
$ ./optional testCppToPy
3
$ ./optional testCppToPyNull
-1
```

The compiler generates the serialization at each boundary. A `std::nullopt` in C++ becomes JSON `null`, which Python reads as `None`.

## 4.16.5. Implicit coercion

Morloc coerces a plain value to an optional wherever the context wants one, so you never write a wrapper at the call site:

```morloc
source Py from "foo.py" ("addOpt")
addOpt :: ?Int -> ?Int -> ?Int

-- both arguments are plain Int, coerced to ?Int
testCoerceAddOpt :: ?Int
testCoerceAddOpt = addOpt 3 4

-- the second argument (42) is Int, coerced to ?Int
testCoerceArg :: Int
testCoerceArg = pFromNull 0 42
```

```console
$ ./optional testCoerceAddOpt
7
$ ./optional testCoerceArg
42
```

Coercion crosses language boundaries too. A C++ function returning a plain `Int` can feed a Python parameter typed `?Int`:

```morloc
source Cpp from "foo.hpp" ("cAddOne")
cAddOne :: Int -> Int

testCppIntToPyOpt :: Int
testCppIntToPyOpt = pFromNull 0 (cAddOne 41)
```

```console
$ ./optional testCppIntToPyOpt
42
```

## 4.16.6. Nested optionals are idempotent

`?(?T)` parses and typechecks, but at run time it collapses to a single `?T`. There is one `Null`, and no way to tell an "outer Null" from an "inner Null". This is deliberate.

The reason goes back to why `?` is a language primitive rather than a library type like C++'s `std::optional`. `?` must lower to each target language’s own missing value: `None` in Python, `NULL` in R, `std::optional<T>` in C++. In Python and R — and in most dynamic languages — that value is structureless. There is no mechanism for telling an outer `None` from an inner one; both are the same singleton. If Morloc allowed two distinguishable null levels, the semantics would diverge across backends, since C++ could fake it with nested `std::optional` and Python could not. That would break the portability `?` exists to provide.

So `?T`, `?(?T)`, and `?(?(?T))` all serialize to the same wire format and the same runtime representation in every backend:

```morloc
collapsed1 :: ?(?Int)
collapsed1 = Null

collapsed2 :: ?(?Int)
collapsed2 = 7   -- treated identically to (7 :: ?Int)
```

```console
$ ./optional collapsed1

$ ./optional collapsed2
7
```

If you genuinely need layered nullability — telling "the lookup failed" apart from "the lookup succeeded but the field was unset" — encode the distinction in a type of your own:

```morloc
record LookupResult = LookupResult
  { tableMissing :: Bool   -- step 1 failure
  , fieldMissing :: Bool   -- step 2 failure
  , value :: ?Int          -- present when both succeeded
  }
```

> **Note**
> Sum types — tagged unions such as `data Result = Found Int | Missing` — are planned but not yet supported. Their cross-language design is the open problem, since not every backend has a first-class sum representation.


---

<!-- https://morloc-project.github.io/docs/features/intrinsics.html -->

# 4.17. Intrinsics

Morloc Manual > Syntax and Features | https://morloc-project.github.io/docs/features/intrinsics.html | prev: https://morloc-project.github.io/docs/features/optionals.md | next: https://morloc-project.github.io/docs/types/index.md

Intrinsics are compiler-generated special functions. They are prefixed with `@` and provide access to the Morloc runtime.

## 4.17.1. Reference table

| Intrinsic | Signature | Description |
| --- | --- | --- |
| `@save` | `Int -> Str -> a -> <IO> (Try Str ())` | Save a value as a morloc voidstar packet with a zstd compression level in `0..=9` (`0` = uncompressed). Arguments are `level`, `path`, `value`. Round-trips through `@load`. I/O failure is an `Err` arm. |
| `@savem` | `Str -> a -> <IO> (Try Str ())` | Save a value to file in MessagePack format (portable, compact). Path-first for partial application (`@savem path` is a reusable sink). I/O failure is an `Err` arm. |
| `@savej` | `Str -> a -> <IO> (Try Str ())` | Save a value to file as plain JSON text (human-readable). Path-first for partial application. I/O failure is an `Err` arm. |
| `@load` | `Str -> <IO> (Try Str a)` | Load a value from file, auto-detecting the format (MessagePack, JSON, or morloc packet). Missing file, decode failure, and schema mismatch all come back as an `Err` arm. |
| `@show` | `a -> Str` | Serialize any value to a JSON string. Pure — no effect row. |
| `@read` | `Str -> Try Str a` | Parse a JSON string into a value of the expected type. Pure — parsing touches nothing, so this composes under `map`. Parse failure is an `Err` arm. |
| `@open` | `Str -> <IO> (Try Str a)` | Open a stream file; `a` is resolved by inline ascription to `IFile a`, `IStream a`, or `OStream a`. See [Random access and streaming](https://morloc-project.github.io/docs/runs/random-access-and-streaming.md). Missing, unreadable, or non-packet files give an `Err` arm. Opening `/dev/stdin` reads process stdin: as `IStream a` it routes to the stdin channel (like `@stdin`); as `IFile a` it gives an `Err` arm, since a pipe is not seekable. |
| `@close` | `a -> <IO> ()` | Given a stream file handle, close it — for an `OStream` this writes the final footer — and release the handle’s slot. Given a `Str` path (as produced by the whole-list `@with`/`@render` gather via `@tmpfile`), unlink that temporary file instead. Only files registered by the gather are removed; passing any other path is rejected rather than deleting it. |
| `@tmpfile` | `<IO> (Try Str Str)` | Create a fresh empty temporary file and return its path. Used by the whole-list `@with`/`@render` gather to stage a stream on disk before applying a handler to the complete data; the file is removed afterward with `@close`. I/O failure is an `Err` arm. |
| `@fschema` | `Str -> <IO> (Try Str Str)` | Read a stream file’s element schema without binding a typed handle. Useful for runtime schema discovery. Missing or malformed files give an `Err` arm. |
| `@flen` | `IFile a -> <IO> (Try Str Int)` | Total element count of an `IFile`, read from the file’s footer. |
| `@write` | `Int -> OStream a -> [a] -> <IO> (Try Str ())` | Append a list of elements to an `OStream`. The first argument is a zstd compression preset in `0..=9` (`0` = no compression); see [Compression](https://morloc-project.github.io/docs/runs/compression.md) for the level table. I/O failure (disk full, broken pipe) is an `Err` arm. |
| `@flush` | `OStream a -> <IO> (Try Str ())` | Force buffered elements to disk as a sub-packet boundary. |
| `@append` | `Str -> <IO> (Try Str (OStream a))` | Open a stream file for further writes, creating it if it is not there yet. Schema mismatch is an `Err` arm at open time, before any bytes are written. |
| `@concat` | `[Str] -> Str -> <IO> (Try Str ())` | Byte-level concatenate compatible stream files into a destination via `sendfile`. The destination is replaced atomically and may itself be one of the sources. |
| `@next` | `IStream a -> <IO> (Try Str [a])` | Pull the next sub-packet’s elements from an `IStream`. Yields `Ok []` at EOF. Mid-stream decode failures are an `Err` arm. |
| `@stream` | `IFile a -> <IO> (IStream a)` | Derive a forward-walking `IStream` from an open `IFile`. The two share the underlying file but have independent cursors. |
| `@stdin` | `<IO> (Try Str (IStream a))` | Open process stdin as a typed `IStream` of morloc binary packets. Element type set by inline ascription. See [Random access and streaming](https://morloc-project.github.io/docs/runs/random-access-and-streaming.md). A second `@stdin` in the same nexus gives an `Err` arm via the uniqueness guard; read-time failures surface at `@next`. |
| `@stdout` | `<IO> (OStream a)` | Open process stdout as a typed `OStream` of morloc binary packets. Element type set by inline ascription. |
| `@stderr` | `<IO> (OStream a)` | Open process stderr as a typed `OStream` of morloc binary packets. Element type set by inline ascription. Useful for structured diagnostics that downstream tools can parse. |
| `@collect` | `(([a] -> <IO, e> ()) -> <IO, e> ()) -> <IO, e> ()` | Drive a streaming-output command. Takes a producer that is handed a *sink* (`[a] → <IO, e> ()`); `@collect` supplies the default sink — write each batch to the nexus-formatted `@stdout` — and manages the stream lifecycle. Formatter directives (`@with` / `@render`, with the `@stream` modifier for per-batch application) rewrite the sink to transform or gather the stream per CLI flag. |
| `@tell` | `<IO> U64` | The number of elements written so far to the current output stream. Lets an offset-form formatter handler (`U64 → [a] → …​`) annotate each batch with its running position in the stream. |
| `@hash` | a -> Str | Hash a value via MessagePack serialization (xxhash), returns a 16-character hex string |
| `@version` | `Str` | The compiler version string (resolved at compile time) |
| `@compiled` | `Str` | The compilation timestamp (resolved at compile time) |
| `@lang` | `Str` | The canonical language identifier of the pool where the expression is evaluated — the `name` field from `lang.yaml` (`"py"`, `"cpp"`, `"r"`, …​; `"morloc"` at the nexus level) |
| `@datafile` | `Str -> Str` | Resolve a relative path to the installed data file location (resolved at compile time) |
| `@schema` | `a -> Str` | The serialization schema string for the given type |
| `@typeof` | `a -> Str` | The morloc abstract type name for the given type, e.g. `"Int"`, `"[Str]"`, `"?Real"`, `"(Int, Str)"` |
| `@throw` | `Str -> a` | Raise a `MorlocException` with the given message. Emits a native `raise`/`throw`/`stop` in the target language and never returns; the return type is polymorphic so `@throw` can inhabit any branch of a conditional. It has no effect row, because a computation that does not return has nothing for one to describe. |
| `@try` | `<e> a -> <e> (Try Str a)` | Evaluate the expression under a language-native try/catch. A completed evaluation is `Ok`, an escaping exception is `Err` carrying its message. The argument’s effect row passes through unchanged; the argument may be pure, in which case so is the result. |

Several intrinsics are polymorphic in their data argument: `@save`, `@savem`, `@savej`, `@write`, `@hash`, `@show`, `@schema`, and `@typeof` accept a value of any type. `@load`, `@read`, and `@next` return a value of any type, inferred from context. `@stdin`, `@stdout`, `@stderr`, `@open`, and `@append` are polymorphic in their handle’s element type, which is resolved by inline ascription at the open site. `@collect` is polymorphic in the stream element type `a`, read off the sink its producer is handed. `@throw` is polymorphic in its return type because it never returns — the return slot unifies with whatever the surrounding context expects. `@try` passes its argument’s effect row through unchanged.

Two patterns run through the table. Anything that touches the world carries `<IO>`, and anything that can fail returns a `Try` (see [Failure is not an effect](https://morloc-project.github.io/docs/features/effects.md#failure-is-not-an-effect)): a missing file, a full disk, a broken pipe or a decode mismatch arrives as an `Err` arm rather than as an effect label. Most entries have both. The exceptions on the failure side are `@close`, `@tell`, `@stream`, `@stdout` and `@stderr`, which do setup, teardown or bookkeeping with no failure mode addressable from morloc code; `@read` is the exception on the other side, fallible but pure, because parsing a string touches nothing. The remaining intrinsics (`@version`, `@compiled`, `@lang`, `@datafile`) are compile-time constants, and `@hash`, `@show`, `@schema` and `@typeof` are pure functions of their argument’s type or serialized bytes.

## 4.17.2. Hashing

`@hash` computes a fast, non-cryptographic hash (xxhash) of any value. The value is first serialized to MessagePack internally, then hashed. The result is a 16-character hexadecimal string.

```morloc
module main (hashInt, hashStr)

import root-py (id)

hashInt :: Int -> Str
hashInt x = @hash (id x)

hashStr :: Str -> Str
hashStr x = @hash (id x)
```

```console
$ ./intrinsics hashInt 1
"6fffcb30bbcc5a72"
$ ./intrinsics hashStr 1
"06e0f1375b38be15"
```

The two differ because the integer `1` and the string `"1"` have different MessagePack encodings, which is the point of the paragraph below.

Hashing is deterministic: the same value always produces the same hash. Two values of different types may hash differently even if they look similar (e.g., the integer `1` and the string `"1"`), because their MessagePack serializations differ.

## 4.17.3. Compile-time constants

The `@version`, `@compiled`, and `@lang` intrinsics are resolved at compile time. They can be used anywhere a `Str` value is expected.

```morloc
module main (info)

import root-py (id)

info :: [Str]
info = id [@version, @compiled, @lang]
```

```console
$ ./intrinsics info
["0.100.2","2026-09-02T12:58:04Z","morloc"]
```

The `@lang` value depends on where the expression is evaluated. When the list literal above is assembled at the nexus level (not inside a sourced function), `@lang` resolves to `"morloc"`. To observe the language-pool identifier, pass `@lang` into a sourced function and let it be evaluated inside that pool: the value will be that pool’s canonical language identifier — the `name` field from its `lang.yaml` (`"py"`, `"cpp"`, `"r"`, …​).

`@lang` deliberately returns this short canonical identifier, not a human-facing display name like "Python3" or "C++". Intrinsics are low-level primitives where stability outweighs presentation: the `lang.yaml` `name` is the guaranteed-unique, stable identifier for a language backend, so it is the correct value for conditional logic and tooling. Map it to a prettier label yourself if you need one.

## 4.17.4. Saving and loading data

The `@savem` and `@savej` intrinsics write a single value to a file path, and `@load` reads it back. Together they provide a type-safe file persistence mechanism for one-shot writes. For multi-element accumulation use an `OStream` and `@write` instead (see [Random access and streaming](https://morloc-project.github.io/docs/runs/random-access-and-streaming.md)).

`@savem` uses MessagePack, which is compact and portable across different machines and architectures. `@savej` writes plain JSON, which is human-readable and can be edited by hand or consumed by other tools.

`@load` auto-detects the file format. Files written by `@savem` carry a small header that identifies them as MessagePack. If no header is present, `@load` tries to parse the file as JSON. `@load` also recognises morloc stream and voidstar packets, so any file produced by the morloc runtime round-trips through it.

`@load` returns `<IO> (Try Str a)`. A missing file, decode failure, or schema mismatch against the caller’s expected type all come back as an `Err` arm rather than an exception — see [Failure and recovery](#failure-and-recovery).

Here is a basic round-trip example:

```morloc
module main (roundTrip)

import root
import root-py (id)

roundTrip :: Int -> Str -> <IO> Int
roundTrip x path = do
  @savem path (id x)
  Ok v <- @load path
  v
```

The bare `@savem` writes the integer to the given path; nothing reads its result, so a write failure stops the block there (see [A bare statement checks its result](https://morloc-project.github.io/docs/features/effects.md#bare-statement-checks)). `Ok v ← @load path` reads the value back and unwraps it, throwing if the load failed. The signature is plain `<IO> Int`, because neither failure survives as a value.

```console
$ ./sv roundTrip 42 tmp.bin
42
```

`import root` is doing work here: `Try` and its two constructors are declared in the standard library, so a module that names `Ok` or `Err` needs it.

You can also use `@savej` when you want the output to be readable:

```morloc
module main (saveReadable)

import root
import root-py (id)

saveReadable :: Str -> [Str] -> <IO> ()
saveReadable path xs = do
  @savej path (id xs)
  ()
```

The resulting file is plain JSON that can be inspected in any text editor. The trailing `()` is not decoration: a `do`\-block’s last statement is its return value, so leaving `@savej` there would make the block return the `Try` rather than discharge it. [A `do`\-block does not return a `Try`](https://morloc-project.github.io/docs/features/effects.md#do-block-no-try) gives the three ways to write this and when each is right.

## 4.17.5. Caching with `@savem` and `@load`

A common pattern is to check whether a cached result exists before recomputing it. `@load` gives an `Err` arm when the file is missing (or on any decode failure), so match on the result and compute in the `Err` arm:

```morloc
module main (cachedResult)

import root
import root-py (id)

source Py from "compute.py" ("expensiveComputation")
expensiveComputation :: Int -> Int

cachedResult :: Int -> Str -> <IO> Int
cachedResult x cachePath = do
  cached <- @load cachePath
  match cached
    | (Ok v) = do v
    | (Err _) = do
        let fresh = expensiveComputation x
        @savem cachePath (id fresh)
        fresh
```

```console
$ ./sv cachedResult 12 c1.bin
144
$ ./sv cachedResult 12 c1.bin
144
```

On the first call the cache file does not exist, so the `Err` arm runs `expensiveComputation`, saves the result and returns it. On the second the `Ok` arm returns the stored value and the computation never runs. An arm may be a whole `do`\-block, which is what makes the recomputation lazy: only the arm that is taken evaluates. Both arms of a `match` have one type, so once the `Err` arm is a `do`\-block the `Ok` arm wraps its value in `do` too (see [The rules](https://morloc-project.github.io/docs/features/effects.md#rules)); a bare `v` there is a type error.

The bare `@savem` inside the `Err` arm still checks itself, so a cache that cannot be written stops the program rather than silently returning an uncached value. If you would rather carry on, bind it: `_ ← @savem cachePath (id fresh)`.

You can also use `@hash` to build content-addressed caches where the cache path depends on the input:

```morloc
module main (hashedCache)

import root
import root-py

source Py from "compute.py" ("expensiveComputation")
expensiveComputation :: Int -> Int

hashedCache :: Int -> <IO> Int
hashedCache x = do
  let key = @hash (id x)
  let cachePath = "/tmp/cache_" <> key <> ".bin"
  cached <- @load cachePath
  match cached
    | (Ok v) = do v
    | (Err _) = do
        let fresh = expensiveComputation x
        @savem cachePath (id fresh)
        fresh
```

Each distinct input gets its own cache file, keyed by the xxhash of its serialized form.

## 4.17.6. Accessing installed data files

The `@datafile` intrinsic resolves a relative file path to its location in the installed program directory. When you compile with `morloc make --install`, source files and data files listed in `package.yaml` are copied into the install directory. At runtime, these files are no longer at their original paths. `@datafile` bridges this gap by resolving the path at compile time.

```morloc
module main (readConfig)

import root-py

source Py from "config.py" ("loadConfig")
loadConfig :: Str -> Str

readConfig :: Str
readConfig = loadConfig (@datafile "defaults.json")
```

Here `@datafile "defaults.json"` evaluates to the absolute path where `defaults.json` is installed (for example, `~/.local/share/morloc/exe/main/defaults.json`). The Python function receives this path as a plain string and can open the file normally.

When running without `--install` (plain `morloc make`), `@datafile` returns the relative path unchanged, so the program works from the project directory as expected.

> **Note**
> Source functions that need data files should accept the path as a parameter rather than hardcoding relative paths. This keeps data dependencies explicit in the type signature and ensures files are found correctly whether the program is run from the project directory or installed.

## 4.17.7. Type introspection

The `@schema` and `@typeof` intrinsics return information about how the compiler represents a type. The value argument is not evaluated at runtime — only its type matters.

```morloc
module main (showSchema, showType)

import root-py (id)

showSchema :: Int -> Str
showSchema x = @schema (id x)

showType :: Int -> Str
showType x = @typeof (id x)
```

```console
$ ./intrinsics typeofInt
"Int"
$ ./intrinsics typeofList
"[Str]"
$ ./intrinsics typeofOpt
"?Real"
$ ./intrinsics typeofTup
"(Int, Str)"
```

`@typeof` returns the **morloc abstract** type name (the same way the type would be written in a signature): `"Int"`, `"Str"`, `"Real"`, `"Bool"`, `"[Int]"`, `"?Int"`, `"(Int, Str)"`, and so on. It does **not** return the language-native type name in the current pool.

`@schema` returns the internal serialization schema string used by the compiler for MessagePack and binary serialization. The encoding is short, byte-oriented, and stable for a given compiler version. The alphabet:

| Schema fragment | Type |
| --- | --- |
| `j` | `Int` (default variable-width integer) |
| `i1` / `i2` / `i4` / `i8` | `I8` / `I16` / `I32` / `I64` |
| `u1` / `u2` / `u4` / `u8` | `U8` / `U16` / `U32` / `U64` |
| `f4` / `f8` | `F32` / `F64` (and `Real`, which maps to `f8`) |
| `b` | `Bool` |
| `s` | `Str` |
| `z` | `Null` / `()` (Unit) |
| `?X` | `Optional X` — `?` prefix followed by an inner schema |
| `aX` | `List X` (and `Array`, `Deque`, `Vector`) — `a` prefix followed by an inner schema; fixed-dim arrays append `:N` |
| `tN X1X2…​XN` | Tuple of `N` elements — `t`, a length code, then one schema per element, run together with no separators |
| `mN <len><key>X…​` | Named record of `N` fields — `m`, a field count, then `(key length, key text, schema)` per field, with no separators |
| `T` / `T:N …​` | Arrow table primitive; bare `T` is row-polymorphic, `T:N` declares `N` required columns |
| `*` | Unknown (unresolved) type |
| `<typename>…​` | Optional concrete-type hint prefix (e.g., `<Money>j` for `type Money = Int`) |

There are no separators anywhere in the encoding. Reading a few real ones is the quickest way to internalize it:

```console
$ ./schemas schemaOf
["t2js","<dict>m24names3agej","aaj","i4","u1","f4"]
```

**Those are, in order: `(Int, Str)`; a record \`P {name**

Str, age :: Int}\` mapped to a Python `dict`; `[[Int]]`; `I32`; `U8`; and `F32`. Take the record apart — `<dict>` is the concrete-type hint, `m2` says two fields, `4name` is a four-character key followed by `s` for its `Str` value, and `3age` is followed by `j` for its `Int`.

`@schema` is primarily useful for debugging and for cross-language tools that inspect morloc wire formats.

## 4.17.8. Failure and recovery

Failure in Morloc is a value, not an effect: a fallible operation returns `Try e a` and the caller matches on it. [Failure is not an effect](https://morloc-project.github.io/docs/features/effects.md#failure-is-not-an-effect) makes that case; this section covers the two intrinsics that sit at its edges. `@throw` produces a failure that is **not** a value — a native exception that unwinds — and `@try` turns one back into a value.

### `@throw`: abandon the computation

`@throw` raises an exception from morloc code. It generates a native `raise`/`throw`/`stop` statement in the language of the pool where the expression is evaluated, halting execution and unwinding the call stack.

The signature is `Str → a`. The message is any `Str` expression, so string interpolation with `#{expr}` works naturally. The return type is polymorphic: since `@throw` never returns a value, its return type can unify with whatever the surrounding context expects, letting `@throw` inhabit any branch of a conditional. There is no effect row, for the same reason — a computation that does not return has nothing for a row to describe.

```morloc
module main (tryRead)

import root
import root-py

source Py from "reader.py" ("openReader", "readerOk")

openReader :: Str -> <IO> Int
readerOk   :: Int -> Bool

tryRead :: Str -> <IO> Int
tryRead path = do
  handle <- openReader path
  ? readerOk handle = handle
  : @throw "failed to open reader for #{path}"
```

```console
$ ./intr tryRead a.ok
1
$ ./intr tryRead a.bad
Error: run failed
failed to open reader for a.bad
  at tryRead [py] (mid=1, intr.loc:1:14)
```

Here `@throw` occupies one arm of the `?/:` conditional and the other arm returns an `Int`. The polymorphic return type unifies with `Int`, and ``tryRead’s signature is plain `<IO> Int``: it opens a file, and the way it fails does not show up in its type.

The generated code depends on the target language: in Python `@throw msg` becomes `raise MorlocException(msg)`, in C++ it becomes `throw MorlocException(msg)`, in R it becomes `stop(structure(class=c("MorlocException", "error", "condition"), list(message=msg, call=NULL)))`. Each pool defines `MorlocException` as a subclass of the language’s native runtime error type, so existing `try`/`catch`/`tryCatch` scaffolding at the pool boundary catches it without any user-side setup.

When `@throw` is invoked from the nexus itself (not inside a pool-bound function), it raises a nexus-side `MorlocError` with the same message and exits the program non-zero.

Use `@throw` when there is nothing sensible to return and the caller has no decision to make. When the caller **does** have a decision to make, return a `Try` instead.

### `@try`: turn a raise back into a value

`@try` evaluates its argument with the target language’s exception machinery armed. Its signature is:

```
@try :: <e> a -> <e> (Try Str a)
```

A completed evaluation is `Ok`; an exception that escapes the argument is `Err`, carrying the exception’s message. The argument’s effect row passes through untouched, and the argument may be pure, in which case so is the result — `@try` is not itself an effect.

```morloc
safeRead :: Str -> <IO> Int
safeRead path = do
  r <- @try (tryRead path)
  match r | (Ok v) = v | (Err _) = 0
```

```console
$ ./intr safeRead a.ok
1
$ ./intr safeRead a.bad
0
```

The point of a value rather than a fallback expression is that the failure is now something you can read. `describeRead` reports it rather than swallowing it:

```morloc
describeRead :: Str -> <IO> Str
describeRead path = do
  r <- @try (tryRead path)
  match r
    | (Ok v) = "opened, handle #{@show v}"
    | (Err e) = "could not open: #{e}"
```

```console
$ ./intr describeRead a.bad
"could not open: failed to open reader for a.bad"
```

Anything may be wrapped. Unlike the intrinsic it replaces, `@try` demands no marker on its argument saying that failure is possible, because no such marker exists any more: a sourced function raises in its own language whether or not its signature hints at it, and `@try` is how you find out. Wrapping an argument that cannot fail is harmless — the result is always `Ok` — and wasted words.

At runtime, `@try` uses the language’s native try/catch machinery. In Python it becomes a `try/except Exception`, in C++ `try { } catch (const std::exception&)`, in R `tryCatch(…​, error = …​)`. Any exception is caught: `MorlocException` raised by `@throw`, foreign-library errors (a Python `KeyError`, a C++ `std::out_of_range`), and cross-pool fail packets, which the calling pool re-throws as a native exception.

### Chaining attempts

Because an arm of a `match` may be a whole `do`\-block, and only the arm that is taken evaluates, fallible attempts nest without any special construct:

```morloc
robustRead :: Str -> <IO> Int
robustRead path = do
  a <- @load "cache/#{path}"
  match a
    | (Ok v) = do v
    | (Err _) = do
        b <- @load "disk/#{path}"
        match b
          | (Ok v) = do v
          | (Err _) = do
              c <- @load "net/#{path}"
              unwrap c
```

Each load runs only if the one before it failed, and each `Ok` arm wraps its value in `do` for the reason given under the cache pattern above. With none of the three files present, every attempt fails and the last error is the one that escapes, because `unwrap` throws the `Err` message it is handed:

```console
$ ./chain robustRead x
Error: evaluation failed: @load: failed to load 'net/x'
```

Put a plain value in the innermost arm instead of `unwrap c` and the chain cannot fail at all.

### Caveats

`@try` intercepts exceptions raised within the calling process. A pool that is killed outright — an out-of-memory kill, an external `kill -9` — is not catchable, because the crash reaches the nexus as a socket error rather than a language-native exception.

`@throw` accepts only a `Str`. Throwing a structured value is not supported: the payload is rendered into the traceback at the throw site and does not survive as a value, so there would be nothing on the other side to match on. Render it yourself with `@show` if you need more than a message.


---

<!-- https://morloc-project.github.io/docs/types/ -->

# 5. Advanced Types

Morloc Manual | https://morloc-project.github.io/docs/types/ | prev: https://morloc-project.github.io/docs/features/intrinsics.md | next: https://morloc-project.github.io/docs/types/term-polymorphism.md


---

<!-- https://morloc-project.github.io/docs/types/term-polymorphism.html -->

# 5.1. One term may have many definitions

Morloc Manual > Advanced Types | https://morloc-project.github.io/docs/types/term-polymorphism.html | prev: https://morloc-project.github.io/docs/types/index.md | next: https://morloc-project.github.io/docs/types/typeclasses.md

A Morloc term may have more than one definition, and the compiler picks whichever one produces the best program. This is *term polymorphism*. It is what lets you write a composition once and have it collapse onto a single language, or onto whatever mix of languages the imports make available.

The `=` operator is the thing to understand first. It does not bind a name to a value the way assignment does in most languages. It states that the two sides are *substitutable*: anywhere the term appears, the compiler may put the right-hand side instead. Writing `=` twice for the same term does not shadow the first definition, it adds a second option.

Here `mean` is given three definitions — one sourced from C++, two written in Morloc:

**mean.loc**

```morloc
module main (mean)

import root-cpp

source Cpp from "mean.hpp" ("mean")
mean :: [Int] -> Int
mean xs = sum xs // length xs
mean xs = fold (+) 0 xs // length xs
```

**mean.hpp**

```cpp
#pragma once
#include <vector>

inline int mean(std::vector<int> xs){
    if (xs.empty()) return 0;
    int s = 0;
    for (int x : xs) s += x;
    return s / (int)xs.size();
}
```

All three compute the same thing. `sum` and `fold` come from `root`, and `//` is integer division:

```console
$ morloc make -o mean mean.loc
$ ./mean mean '[1,2,3,4]'
2
```

## 5.1.1. How the choice collapses a program

The compiler does not pick a definition per call site in isolation. It scores whole realizations: every call carries a cost, and a call that crosses a language boundary carries a far larger one than a call that stays put — the built-in defaults are 10 for a same-language call and 10000 for a crossing. It takes the cheapest realization, breaking ties by the number of boundaries crossed.

The consequence is that a composition tends to collapse onto one language — whichever one the surrounding code is already in. Drop the C++ source from the module above, keep both Morloc definitions, and import `root-py` instead:

```morloc
module main (mean)

import root-py

mean :: [Int] -> Int
mean xs = sum xs // length xs
mean xs = fold (+) 0 xs // length xs
```

Nothing about `mean` changed, but the generated program is now pure Python. The build directory shows which pools were generated:

```console
$ morloc make -o mean mean.loc
$ ./mean mean '[1,2,3,4]'
2
$ ls mean-build/pools/
py
```

With the C++ version, the same listing shows `cpp`. Without term polymorphism, changing the language of one component would mean rewriting and rewiring everything downstream of it by hand.

## 5.1.2. Contradictory definitions

Because `=` means "substitutable", nothing stops you from claiming two things are the same when they are not:

**contradiction.loc**

```morloc
module main (x)

import root-py

x :: Int
x = 1
x = 2
```

`x` is now 1 *or* 2, and which one you get is up to the compiler. Morloc has a *value checker* that catches the blatant cases — literals that disagree, and containers whose sizes disagree:

```console
$ morloc make -o contradiction contradiction.loc
Unification error: Error in value checker: Cannot equate non-equal primitives (the two operands disagree):
a: 2
b: 1
Found while unifying contradiction.loc:1:14
With values

  |
7 | x = 2
  |     ^
and

  |
6 | x = 1
  |     ^
```

The value checker is shallow. It compares literals; it does not evaluate foreign code. So this contradiction gets through:

**deep.loc**

```morloc
x :: Real
x = 2.0 / (1.0 + 1.0)
x = 2.0 / 1.0
```

```console
$ morloc make -o deep deep.loc
$ ./deep x
2
```

The compiler cannot see inside `(+)` to know that the first definition is 1. It compiled, it ran, and it silently picked the second definition. Multiple definitions are a promise you are making to the compiler, and it can only check part of it.

## 5.1.3. A test suite that runs against every implementation

The Morloc standard library uses term polymorphism to test every language backend with one test suite. The pattern is worth copying.

Split the module into a language-agnostic parent that declares the interface, one child per language that supplies implementations, and a test module that depends only on the parent.

The parent declares signatures and nothing else:

**clock/main.loc**

```morloc
module clock (incSec)

import root

--' Advance an (hour, minute, second) triple by one second
incSec :: (Int, Int, Int) -> (Int, Int, Int)
```

Each language child imports the parent and sources implementations for it:

**clock-py/main.loc**

```morloc
module clock-py (*)

import .clock
import root-py

source Py from "clock.py" ("inc_sec" as incSec)
```

**clock-cpp/main.loc**

```morloc
module clock-cpp (*)

import .clock
import root-cpp

source Cpp from "clock.hpp" ("inc_sec" as incSec)
```

The test module imports the parent — never a language child — so it has no opinion about which implementation runs:

**clocktest/main.loc**

```morloc
module clock.test (runTests)

import .clock
import root

-- The harness itself is Python, so it needs Python forms for the types
-- it touches, whatever language the implementation under test uses.
type Py => Int = "int"
type Py => Str = "str"
type Py => List a = "list" a
type Py => Tuple3 a b c = "tuple" a b c

source Py from "check.py" ("check")
check :: (Str, a, a) -> Str

runTests :: [Str]
runTests = map check
  [ ("rolls seconds",  incSec (1, 2, 3),     (1, 2, 4))
  , ("rolls minutes",  incSec (1, 2, 59),    (1, 3, 0))
  , ("rolls hours",    incSec (1, 59, 59),   (2, 0, 0))
  , ("wraps midnight", incSec (23, 59, 59),  (0, 0, 0))
  ]
```

**clocktest/check.py**

```python
def check(case):
    msg, observed, expected = case
    return ("ok   " if observed == expected else "FAIL ") + msg
```

A top-level module then chooses the implementation by choosing an import:

**main.loc**

```morloc
module main (runTests)

import .clocktest (runTests)
import .clock-py
```

```console
$ morloc make -o runtests main.loc
$ ./runtests runTests
["ok   rolls seconds","ok   rolls minutes","ok   rolls hours","ok   wraps midnight"]
$ ls runtests-build/pools/
py
```

Swap `import .clock-py` for `import .clock-cpp` and the identical test suite now exercises the C++ implementation. The test cases, the expected values, and the comparison logic are unchanged:

```console
$ morloc make -o runtests-cpp cppmain.loc
$ ./runtests-cpp runTests
["ok   rolls seconds","ok   rolls minutes","ok   rolls hours","ok   wraps midnight"]
$ ls runtests-cpp-build/pools/
cpp
py
```

Two pools this time: the implementation under test is C++, the harness is Python, and the tuples cross between them. That crossing is the point — the test suite is checking the real cross-language path, not a mock of it.


---

<!-- https://morloc-project.github.io/docs/types/typeclasses.html -->

# 5.2. Overload terms with typeclasses

Morloc Manual > Advanced Types | https://morloc-project.github.io/docs/types/typeclasses.html | prev: https://morloc-project.github.io/docs/types/term-polymorphism.md | next: https://morloc-project.github.io/docs/types/infix-operators.md

A typeclass lets one name have a different implementation for each type it is applied to. Where term polymorphism gives the compiler a free choice between interchangeable definitions, a typeclass instance is *selected* by the type at the call site. The idea is the same as typeclasses in Haskell, traits in Rust, interfaces in Java, and concepts in C++.

A class declares method signatures. An instance supplies the implementations for one type:

**pretty.loc**

```morloc
module main (describeInts, describeReals)

import root-py

class Pretty a where
    pretty :: a -> Str

instance Pretty Int
       , Pretty Real where
    source Py from "ops.py" ("to_str" as pretty)

title :: Pretty a => a -> Str
title x = "value: " <> pretty x

describeInts :: [Int] -> [Str]
describeInts = map title

describeReals :: [Real] -> [Str]
describeReals = map title
```

**ops.py**

```python
def to_str(x):
    return str(x)
```

```console
$ morloc make -o pretty pretty.loc
$ ./pretty describeInts '[1,2]'
["value: 1","value: 2"]
$ ./pretty describeReals '[1.5]'
["value: 1.5"]
```

Three things in that module are worth naming.

**One instance may cover several types.** `instance Pretty Int , Pretty Real where` declares two instances that share a body. Python’s `str` handles both, so writing the `source` line twice would be noise. The standard library uses this form heavily — `root-py` declares a dozen `RealLike` instances in one block.

****A signature may carry a class constraint.** \`title**

Pretty a ⇒ a → Str\` says `title` works for any type that has a `Pretty` instance. Everything to the left of `⇒` is a constraint; multiple constraints are comma-separated and

**parenthesized, as in \`root’s \`sum**

(Foldable f, Integral a) ⇒ f a → a\`.

**A class body holds signatures only.** Morloc has no default method implementations. Writing a body inside a `class` block is a parse error:

```console
$ morloc typecheck dm.loc
dm.loc:6:16: unexpected identifier 'xs'
    |
  6 |     prettyList xs = "list"
    |                ^
  expected '::'
```

Put the shared logic in an ordinary constrained function instead, the way `title` does above.

> **Important: A generic function cannot be an entry point**
> An exported term whose type still has a class constraint is dropped from the generated program, because the compiler cannot pick an instance without a concrete type. Export `title` directly and it does not become a command:
> 
> ```console
> $ morloc make -o generic generic.loc
> Warning: skipping generic export 'title'
> $ ./generic title 1
> error: unexpected argument 'title' found
> ...
> ```
> 
> Export a monomorphic wrapper instead — `describeInts` and `describeReals` above — and keep the generic function internal.

## 5.2.1. One class, many languages

An instance may source implementations from several languages at once. The compiler then has a choice of instance bodies for the same method, and the usual collapse applies: it takes whichever one keeps the program in one language.

```morloc
class Addable a where
    zero :: a
    (+) :: a -> a -> a

instance Addable Int where
    source Py from "arithmetic.py" ("add" as (+))
    source Cpp from "arithmetic.hpp" ("add" as (+))
    zero = 0

instance Addable Real where
    source Py from "arithmetic.py" ("add" as (+))
    source Cpp from "arithmetic.hpp" ("add" as (+))
    zero = 0.0
```

The native functions may be polymorphic in their own language, in which case the same implementation is named by several instances. The Python `add` above is one function:

**arithmetic.py**

```python
def add(x, y):
    return x + y
```

And so is the C++ one:

**arithmetic.hpp**

```cpp
template <class A>
A add(A x, A y){
    return x + y;
}
```

A method does not have to come from a foreign language. `zero = 0` is an ordinary Morloc definition, and it is polymorphic in the same way any other term is: `zero` in the `Int` instance is the integer literal, `zero` in the `Real` instance is the floating-point one.

> **Warning: This example collides with root**
> `Addable` redeclares `zero` and `(+)`, which `root` already supplies through its `Integral` class. Two classes cannot define the same term, so this module compiles only in isolation — with `import internal` for the primitive types, not `import root-py`:
> 
> ```console
> $ morloc typecheck main.loc
> In module 'main': The typeclasses 'Integral' and 'Addable' have conflicting definitions of the term 'zero'
> ```

## 5.2.2. Superclasses

A class may require another class. Write the requirement to the left of `⇒` in the class head:

```morloc
class Pretty a => Boxed a where
    box :: a -> Str
```

Any type with a `Boxed` instance must also have a `Pretty` instance, and a function constrained on `Boxed a` may use `pretty` as well as `box`. This is how `root` layers its numeric hierarchy: `class Integral a ⇒ Numeric a` means every `Numeric` type is also `Integral`.

## 5.2.3. Importing a class from another module

A class is exported and imported by its name. Its methods are not separately importable and may not appear in an export list:

**numops/main.loc**

```morloc
module numops (Pretty, exclaim)

import root

class Pretty a where
    pretty :: a -> Str

exclaim :: Pretty a => a -> Str
exclaim x = pretty x <> "!"
```

Listing `pretty` in that export list gives `Module '.numops' does not export the following terms or types: [pretty]`, which is confusing but means what it says: a method has no standalone identity to export.

Importing the class name is enough to declare instances for it elsewhere:

**main.loc**

```morloc
module main (shout)

import root-py
import .numops (Pretty, exclaim)

instance Pretty Int where
    source Py from "ops.py" ("to_str" as pretty)

shout :: Int -> Str
shout = exclaim
```

```console
$ morloc make -o prog main.loc
$ ./prog shout 7
"7!"
```

This is the shape every language-specific standard library module takes. `root` declares `Eq`, `Ord`, `Functor`, `Foldable` and the rest; `root-py`, `root-cpp` and `root-r` import those names and fill in instances. Nothing in `root` knows which languages exist.


---

<!-- https://morloc-project.github.io/docs/types/infix-operators.html -->

# 5.3. Infix operators

Morloc Manual > Advanced Types | https://morloc-project.github.io/docs/types/infix-operators.html | prev: https://morloc-project.github.io/docs/types/typeclasses.md | next: https://morloc-project.github.io/docs/types/newtype.md

An operator in Morloc is an ordinary function whose name happens to be punctuation. Nothing about it is built in: `+`, `<>`, `.` and `$` are all declared in `root` and `internal` the same way you would declare your own.

An operator name is any run of these characters:

```
: ! $ % & * + . / < = > ? @ \ ^ | - ~ #
```

Give it a type by wrapping the name in parentheses, and it can then be used infix. An operator with no fixity declaration is left-associative with precedence 9:

**minus.loc**

```morloc
module main (test)

import root-py

(<->) :: Int -> Int -> Int
(<->) x y = x - y

test :: Int
test = 10 <-> 3 <-> 2
```

```console
$ morloc make -o minus minus.loc
$ ./minus test
5
```

`(10 - 3) - 2`, not `10 - (3 - 2)`.

## 5.3.1. Declaring associativity and precedence

`infixl` is left-associative, `infixr` right-associative, and `infix` non-associative. Each takes a precedence level from **0 through 9** inclusive, with higher binding tighter — the Haskell convention:

```morloc
infixl 6 +
infixl 7 *
infixr 8 **
```

The parentheses around the operator name are optional here; `root` writes `infixl 6 (+)` and both forms parse.

A level outside 0-9 is rejected at parse time:

```console
$ morloc typecheck prec10.loc
prec10.loc:3:8: infix precedence must be in [0,9], got 10
    |
  3 | infixl 10 <+>
    |        ^
```

Chaining a non-associative operator is an error, and so is mixing two operators of equal precedence with different associativity:

```console
$ morloc typecheck nonassoc.loc
nonassoc.loc:6:7: error:
Ambiguous use of <+> and <+>: parenthesize or declare compatible fixities
  |
6 | f = 1 <+> 2 <+> 3
  |       ^
```

Two modules may not declare different fixities for the same operator. If one module says `infixl 1 |>` and an importer says `infixl 2 |>`:

```console
$ morloc typecheck conflict.loc
Conflicting fixity definitions for |>
```

## 5.3.2. Operators from foreign languages and typeclasses

Operators are sourced like any other function:

```morloc
source Py from "ops.py" ("add" as (+), "mul" as (*))
```

And they can be typeclass methods, which is how `root` gives one `+` to every numeric type:

**arith.loc**

```morloc
module main (test_expr)

import internal

type Py => Int = "int"

class Num a where
    zero :: a
    invert :: a -> a
    (+) :: a -> a -> a
    (*) :: a -> a -> a

infixl 6 +
infixl 7 *

instance Num Int where
    source Py from "foo.py" ("add" as (+), "mul" as (*), "neg" as invert)
    zero = 0

test_expr :: Int
test_expr = 4 * 7 + 3
```

**foo.py**

```python
def add(x, y): return x + y
def mul(x, y): return x * y
def neg(x): return -x
```

```console
$ morloc make -o arith arith.loc
$ ./arith test_expr
31
```

`4 * 7` binds first because `*` was given the higher precedence, then `+ 3`.

This module imports `internal` rather than `root`, because `root` already declares `+`, `*` and `zero` in its `Integral` class and two classes cannot own the same term. `invert` is spelled that way for the same reason: `negate` belongs to ``internal’s `Negatable`` class.

## 5.3.3. Importing operators

Operators are imported by their parenthesized names. Their fixity travels with them, so the importing module does not redeclare it:

**ops/main.loc**

```morloc
module ops ((|>))

import root

infixl 1 |>

(|>) :: a -> (a -> b) -> b
(|>) x f = f x
```

**main.loc**

```morloc
module main (test)

import root-py
import .ops ((|>))

test :: Int
test = 3 |> (\x -> x + 1)
```

```console
$ morloc make -o prog main.loc
$ ./prog test
4
```

> **Warning: | alone is not available**
> The bare pipe is a reserved token, so `(|)` cannot be an operator name even though `|` is a legal operator character. `(||)`, `(|>)`, `(||.)` and the rest are fine.

## 5.3.4. Names that cannot start with `--`

An operator name may not begin with `--`. The sequence always opens a comment, whatever follows it:

```morloc
-- an ordinary comment
--' a docstring
--* a doc-group annotation
```

So a declaration like `infixl 6 --+` is read as `infixl 6` followed by a comment running to end of line. The `infixl` is left incomplete and the parser fails on the **next** line with an error that looks unrelated:

```console
$ morloc typecheck dashop.loc
dashop.loc:4:1: unexpected new declaration
    |
  4 | (--+) :: Int -> Int -> Int
    | ^
  expected one of: '(', '<', '>', '.', '*', '-', identifier, '+', '/', operator
```

The prefix is reserved so that further comment variants can be added later without colliding with user operators. `--^` is already rejected outright.


---

<!-- https://morloc-project.github.io/docs/types/newtype.html -->

# 5.4. Naming a type: `type` and `newtype`

Morloc Manual > Advanced Types | https://morloc-project.github.io/docs/types/newtype.html | prev: https://morloc-project.github.io/docs/types/infix-operators.md | next: https://morloc-project.github.io/docs/types/packable.md

Morloc gives you two keywords for putting a name on a type, and the choice between them decides whether the new name is the same type as the old one or a different one.

`type X = Y` is a **transparent alias**. `X` and `Y` are one type with two spellings, interchangeable everywhere.

`newtype X = Y` is a **nominal type**. `X` is a genuinely new type that happens to travel across language boundaries in the same format as `Y`. It owns its own typeclass instances and its own per-language representations, and a value cannot flow between `X` and `Y` without an explicit conversion.

A third form, a declaration with no right-hand side at all, declares an opaque primitive. That is covered at the end.

## 5.4.1. `type`: transparent aliases

An alias is fully substitutable with its right-hand side anywhere a type can appear — in signatures, annotations, container parameters, `Packable` instances, everywhere.

```morloc
type Filename = Str
type UserID   = Int
```

A `Filename` goes wherever a `Str` is expected and a `Str` goes wherever a `Filename` is expected. Two aliases on the same chain are interchangeable with each other too: with `type A = Str` and `type B = Str`, an `A` flows into a `B` slot without conversion.

Aliases are useful for three things: naming (a signature reads better when a `Filename` is called a `Filename`), shortening long type expressions (`type Coord = (Real, Real)`), and attaching per-argument CLI documentation, which is described below.

## 5.4.2. Alias chains resolve on their own

[Native type mappings](https://morloc-project.github.io/docs/features/foreign-functions.md#mapping-native-types) showed how a general type is mapped to each language:

```morloc
type Py => Str = "str"
```

You do not repeat that mapping for every alias. The compiler follows the chain until it finds a language-specific form, however many hops it takes:

**alias.loc**

```morloc
module main (shout)

import root-py

type LastName = Str
type Surname  = LastName

source Py from "ops.py" ("to_upper" as shout)
shout :: Surname -> LastName
```

**ops.py**

```python
def to_upper(s):
    return s.upper()
```

```console
$ morloc make -o prog alias.loc
$ ./prog shout 'smith'
"SMITH"
```

`Surname` resolves to `LastName`, which resolves to `Str`, which resolves to `"str"` in Python. Writing `type Py ⇒ Surname = "str"` would be redundant — and, as the next section shows, is rejected.

## 5.4.3. Docstring inheritance

An alias inherits docstring directives from its parent and may override individual fields. This is what makes per-argument CLI documentation work: both aliases below are `Str` for typechecking and codegen, but each carries its own description.

**crypt.loc**

```morloc
module main (encrypt)

import root-py

--' A secret key
--' metavar: KEY
type Key = Str

--' The message to encrypt
type PlainText = Str

--' An encrypted message
type CipherText = Str

--' Encrypt a message with a key
encrypt :: Key -> PlainText -> CipherText
encrypt k m = m <> k
```

```console
$ morloc make -o crypt crypt.loc
$ ./crypt encrypt --help
Encrypt a message with a key

Usage: ./crypt <nexus_options> @ <command_options>

General Options:
  -h, --help
          Print help (see a summary with '-h')

Positional arguments:
  1:  A secret key
      type: Str    (literal string)
  2:  The message to encrypt
      type: Str    (literal string)

Return: CipherText
  An encrypted message
...
```

The `metavar: KEY` directive is recorded and reaches `--json-help` and `--mcp-tools`, but the positional-argument block of `--help` does not print metavars today. See the [Building CLIs](https://morloc-project.github.io/docs/clis/index.md) chapter for the full set of docstring directives.

`newtype` does not inherit docstrings. A `newtype` is its own identity and its own documentation.

## 5.4.4. What an alias cannot do

An alias has no identity of its own, so it cannot own anything.

**It cannot have its own typeclass instances.** The instance belongs to the root of the chain, and every alias on the chain shares it:

```console
$ morloc typecheck aliasinst.loc
aliasinst.loc:4:1: error:
Cannot declare instance on transparent alias 'Filename'.
All members of an alias tree share a single instance. Either declare the instance for the root type, or change the declaration of 'Filename' from 'type' to 'newtype' so it becomes a nominally distinct type that owns its own instances.
  |
4 | instance Eq Filename where
  | ^
```

**It cannot have its own per-language form.** The chain has to resolve to one native type per language:

```console
$ morloc typecheck aliaslang.loc
aliaslang.loc:4:1: error:
'Filename' is declared as a 'type' alias but has a per-language form for py.
Change 'type' to 'newtype' so 'Filename' becomes a nominally distinct type that owns its native language forms.
  |
4 | type Py => Filename = "pathlib.Path"
  | ^
```

Both errors tell you the fix: use `newtype`.

## 5.4.5. `newtype`: nominal types

A `newtype` is a new type that shares a wire format with the type on its right-hand side. Its instances, its native forms, and its identity are its own.

```morloc
newtype Path = Str
type Py => Path = "pathlib.Path"
type Cpp => Path = "std::filesystem::path"
```

`Path` and `Str` are now different types, and mixing them is an error:

**nomix.loc**

```morloc
module main (bad)

import root-py

newtype Path = Str
type Py => Path = "pathlib.Path"

f :: Path -> Path
bad :: Str -> Path
bad s = f s
```

```console
$ morloc typecheck nomix.loc
nomix.loc:10:11: error:
Type mismatch:
  expected: Path
  inferred: Str
Cannot compare types Str and Path
   |
10 | bad s = f s
   |           ^
```

The wire format is still `Str` — a `Path` crosses a language boundary as a string — but inside each pool the value is a real `pathlib.Path` or `std::filesystem::path`.

## 5.4.6. When a `newtype` needs a `Packable` instance

A `newtype` crosses a language boundary as its wire parent. Whether anything has to convert that wire value into the newtype’s native form — and so whether you need a `Packable` instance — depends on one question: is the native form something the pool already has?

**Declare no per-language form and the answer is yes.** The newtype inherits its parent’s native form, so the value that arrives already is the right thing. No instance is needed, whatever the parent’s shape — a primitive, a list, a tuple, or another newtype. This is how the standard library’s `Vector` works in C++: `vector-cpp` declares no `Packable` instance for it at all, because `newtype Vector (n :: Nat) a = List a` and a `List` is already a `std::vector`.

**Declare a form and it travels with the value as a schema hint.** If the language binding knows how to build that form, you still need no instance. Python’s binding recognises `bytes`, `bytearray`, `list`, and `numpy.ndarray` (`data/lang/py/pymorloc.c`); `numpy.ndarray` is what puts tensor data on the zero-copy path.

**Anything else needs a `Packable`.** The instance is the general answer: it says how to build the native form from the wire form and back.

Three newtypes over `Str`, one of each kind:

**forms.loc**

```morloc
module main (nameKind, blobKind, pathKind)

import root-py

newtype Name = Str

newtype Blob = Str
type Py => Blob = "bytes"

newtype Path = Str
type Py => Path = "pathlib.Path"

source Py from "native.py"
  ("kind" as nameKind, "kind" as blobKind, "kind" as pathKind)

nameKind :: Name -> Str
blobKind :: Blob -> Str
pathKind :: Path -> Str
```

**native.py**

```python
import pathlib

def kind(x):
    return type(x).__name__

def str_to_path(s):
    return pathlib.Path(s)

def path_to_str(p):
    return str(p)
```

`kind` reports what the pool actually received:

```console
$ morloc make -o forms forms.loc
$ ./forms nameKind notes/report.txt
"str"
$ ./forms blobKind notes/report.txt
"bytes"
$ ./forms pathKind notes/report.txt
"str"
```

`Name` inherits ``Str’s form and gets a `str``, as declared. `Blob` asked for `bytes` and got one, with no instance, because the Python binding builds that hint. `Path` asked for `pathlib.Path` and got a `str` — the binding does not know that hint, and nothing said so.

> **Warning: An unsupported form is dropped silently**
> That third line is a trap. The module declares `pathlib.Path`, the pool receives a `str`, and there is no error and nothing on stderr. Your foreign function fails later, against a contract the compiler accepted.
> 
> Add the `Packable` instance and it is right:
> 
> ```morloc
> instance Packable Str Path where
>   source Py from "native.py" ("str_to_path" as pack, "path_to_str" as unpack)
> ```
> 
> ```console
> $ ./forms2 pathKind notes/report.txt
> "PosixPath"
> ```
> 
> Until this is caught at compile time, write the instance whenever you declare a per-language form outside the four the binding recognises.

A worked example with the instance in place, and a typeclass scoped to the new type:

**path.loc**

```morloc
module main (ext, joined, absolute)

import root-py

newtype Path = Str
type Py => Path = "pathlib.Path"

instance Packable Str Path where
  source Py from "pathlib_ops.py" ("str_to_path" as pack,
                                   "path_to_str" as unpack)

class Filelike a where
  extension  :: a -> Str
  joinPath   :: a -> a -> a
  isAbsolute :: a -> Bool

instance Filelike Path where
  source Py from "pathlib_ops.py"
    ( "path_extension"   as extension
    , "path_join"        as joinPath
    , "path_is_absolute" as isAbsolute
    )

ext :: Path -> Str
ext = extension

joined :: Path -> Path -> Path
joined = joinPath

absolute :: Path -> Bool
absolute = isAbsolute
```

**pathlib\_ops.py**

```python
import pathlib

def str_to_path(s):
    return pathlib.Path(s)

def path_to_str(p):
    return str(p)

def path_extension(p):
    return p.suffix

def path_join(a, b):
    return a / b

def path_is_absolute(p):
    return p.is_absolute()
```

```console
$ morloc make -o prog path.loc
$ ./prog ext 'notes/report.txt'
".txt"
$ ./prog joined '/home/z' 'notes.txt'
"\/home\/z\/notes.txt"
$ ./prog absolute 'notes.txt'
false
```

`Filelike` methods are available on `Path` and not on bare `Str`, which is exactly the constraint that makes the `newtype` worth declaring: a function over filesystem paths cannot be handed an arbitrary string.

## 5.4.7. Sharing a wire format across newtypes

`newtype` is how a family of related types share one serialized representation while keeping distinct behaviour. `root` declares `Deque` this way:

```morloc
newtype Deque a = List a
instance Packable (List a) (Deque a)
```

`Deque` is a separate type from `List` — it has its own `Stack` and `Queue` instances, tuned to a deque’s performance profile — but it travels as a flat list, so on the command line it looks like one:

**deque.loc**

```morloc
module main (pushFront, asList)

import root-py

pushFront :: Int -> Deque Int -> Deque Int
pushFront = cons

asList :: Deque Int -> [Int]
asList = unpack
```

```console
$ morloc make -o prog deque.loc
$ ./prog pushFront 0 '[1,2,3]'
[0,1,2,3]
$ ./prog asList '[1,2,3]'
[1,2,3]
```

`unpack` is the `Packable` method that converts the native form back to the wire form; it is the explicit conversion the nominal distinction demands.

## 5.4.8. Declarations with no body

A declaration with no right-hand side introduces a primitive: nominal, opaque, owning its own per-language forms and instances, with no underlying Morloc representation.

```morloc
newtype Int
newtype Str
newtype List a
```

`type` and `newtype` mean the same thing in this position — there is no alias to be transparent about — and the compiler treats them identically. Prefer `newtype`, which is what these declarations behave like. The standard library uses this form for every built-in type; `internal/main.loc` is a long list of them.

This is also how you declare a type that exists only in the foreign languages:

```morloc
newtype Map key val
type Py  => Map key val = "dict" key val
type Cpp => Map key val = "std::map<$1,$2>" key val
```

Such a type needs a `Packable` instance to say what it looks like on the wire. That is the next section.

## 5.4.9. The rules

1.  **An instance belongs to the root of an alias chain.** `instance Foo MyAlias` where `type MyAlias = Bar` is rejected. Declare it on `Bar`, or make `MyAlias` a `newtype`.
2.  **Every member of a `type` chain shares the root’s instances.** With `type A = Str` and `type B = Str`, the single `instance Eq Str` is found at every site that mentions `A`, `B`, or `Str`.
3.  **A `type` alias may not carry a per-language form.** `type Py ⇒ MyAlias = "…​"` is rejected. Use `newtype`.
4.  **A `newtype` is nominal.** It owns its instances and its per-language forms. It needs a `Packable` instance only when it declares a native form that the language binding cannot build from the wire form on its own.
5.  **`newtype` wire-parent chains may not cycle.** `newtype A = B` with `newtype B = A` gives `Mutual recursion between type definitions is not supported. Cycle: A, B`.


---

<!-- https://morloc-project.github.io/docs/types/packable.html -->

# 5.5. Serializing custom types with `Packable`

Morloc Manual > Advanced Types | https://morloc-project.github.io/docs/types/packable.html | prev: https://morloc-project.github.io/docs/types/newtype.md | next: https://morloc-project.github.io/docs/types/kinds.md

Morloc can move a value between languages when it knows how to write that value down. Primitives, lists, tuples and records all have a canonical written form, so they cross a boundary with no help from you. A type that does not decompose into those forms needs you to say what it looks like on the wire. You say it by declaring a `Packable` instance.

Consider `Map k v`. In Python it is a `dict`, in C++ a `std::map`, in R a named list; it could equally be a list of pairs, a pair of columns, or a balanced tree. None of those is more canonical than the others. What they share is that any of them can be written as a list of key/value pairs, and that is what `Packable` records.

The class lives in `internal` and has two methods:

```morloc
class Packable a b where
    pack :: a -> b
    unpack :: b -> a
```

`a` is the wire form and `b` is the type being described. `pack` builds the type from its wire form, `unpack` takes it apart.

## 5.5.1. A worked example: `Map`

`Map` is declared with no right-hand side — it is a primitive, opaque to Morloc, with a form in each language (see [Naming a type: `type` and `newtype`](https://morloc-project.github.io/docs/types/newtype.md)). The `Packable` instance says it travels as a list of pairs:

**counts.loc**

```morloc
module main (tally, topCount)

import root-py
import root-cpp

newtype Map key val

type Py  => Map key val = "dict" key val
type Cpp => Map key val = "std::map<$1,$2>" key val

instance Packable [(a, b)] (Map a b) where
    source Py  from "map-packing.py"  ("pack", "unpack")
    source Cpp from "map-packing.hpp" ("pack", "unpack")

source Py from "counts.py" ("tally")
tally :: [Str] -> Map Str Int

source Cpp from "counts.hpp" ("biggest")
biggest :: Map Str Int -> Int

topCount :: [Str] -> Int
topCount = biggest . tally
```

The packers are ordinary functions in their own languages. Python:

**map-packing.py**

```python
def pack(xs):
    return dict(xs)

def unpack(d):
    return list(d.items())
```

C++:

**map-packing.hpp**

```cpp
#pragma once
#include <map>
#include <tuple>
#include <vector>

template <class K, class V>
std::map<K,V> pack(std::vector<std::tuple<K,V>> xs){
    std::map<K,V> m;
    for (auto& kv : xs) m[std::get<0>(kv)] = std::get<1>(kv);
    return m;
}

template <class K, class V>
std::vector<std::tuple<K,V>> unpack(std::map<K,V> m){
    std::vector<std::tuple<K,V>> xs;
    for (auto& kv : m) xs.push_back({kv.first, kv.second});
    return xs;
}
```

And the two functions that actually do the work:

**counts.py**

```python
def tally(words):
    d = {}
    for w in words:
        d[w] = d.get(w, 0) + 1
    return d
```

**counts.hpp**

```cpp
#pragma once
#include <map>
#include <string>

inline int biggest(std::map<std::string,int> m){
    int best = 0;
    for (auto& kv : m) if (kv.second > best) best = kv.second;
    return best;
}
```

`topCount` composes a Python function that returns a `dict` with a C++ function that takes a `std::map`. Neither language knows about the other:

```console
$ morloc make -o counts counts.loc
$ ./counts topCount '["a","b","a"]'
2
$ ls counts-build/pools/
cpp
py
```

The standard library ships a fuller `Map` in its `map` module, declared exactly this way — `newtype Map a b`, then `instance Packable [(a, b)] (Map a b)`, with the per-language forms and packers in `map-py`, `map-cpp` and `map-r`. The version above is standalone so it can be read on its own.

You never call `pack` or `unpack` yourself here. The compiler builds a serialization tree from the general type and generates the native code to decompose the value recursively until only primitives remain. Those are what travel. The wire form is also what the command line accepts and prints, which is why `Map Str Int` appears as a list of pairs:

```console
$ ./counts tally '["a","b","a"]'
[["a",2],["b",1]]
```

## 5.5.2. Specialized instances

A native type is sometimes less general than the Morloc type. R’s named list, for example, can only have string keys. Declare a narrower instance and the compiler will use it where it fits and prune the language elsewhere:

```morloc
type R => Map key val = "list" key val

instance Packable [(Str, b)] (Map Str b) where
    source R from "map-packing.R" ("pack", "unpack")
```

If R is the only language available and a signature demands a non-string key, the program does not build:

**ronly.loc**

```morloc
module main (countStr, countInt)

import root-r

newtype Map key val
type R => Map key val = "list" key val

instance Packable [(Str, b)] (Map Str b) where
    source R from "map-packing.R" ("pack", "unpack")

source R from "ops.R" ("count_keys" as countKeys)
countKeys :: Map a b -> Int

countStr :: Map Str Int -> Int
countStr = countKeys

countInt :: Map Int Str -> Int
countInt = countKeys
```

```console
$ morloc make -o ronly ronly.loc
ronly.loc:1:24: error:
There was an error raised in subtyping while resolving serialization
The packer involved maps the type:
  forall b . Map Str b

To the serialized form:
  forall b . [(Str, b)]
...
However, the b <: a step failed:
Cannot compare types character and integer

The packer function may not be generic enough to pack the type you specify, if this is the case, you may need to simplify the datatype
  |
1 | module main (countStr, countInt)
  |                        ^
```

That is the message telling you the R backend cannot serve `Map Int Str`. With a Python implementation also in scope, the same program compiles and the R implementations are not selected.

One line of that message, elided above, currently prints raw compiler internals rather than a Morloc type. Read past it to the `Cannot compare types` line, which is the real content.

## 5.5.3. `pack` in your own code

`pack` and `unpack` are ordinary methods, so you can call them. `unpack` is how you convert a nominal type back to its wire form, as the `Deque` example in [Naming a type: `type` and `newtype`](https://morloc-project.github.io/docs/types/newtype.md) does.

Calling `pack` has one sharp edge. If the wire form itself contains a packable type, the compiler will not chain the two conversions and reports a missing instance. Here the target is `Matrix`, the standard library’s two-dimensional tensor (see [Tensors](https://morloc-project.github.io/docs/types/tensors.md)), whose wire form is a dimension tuple paired with a `Vector`:

```console
$ morloc typecheck m.loc
m.loc:7:5: error:
General type error: No instance found for Packable::pack
  Are you missing a top-level type signature?
  |
7 | m = pack ((2, 3), [1.0, 2.0, 3.0, 4.0, 5.0, 6.0])
  |     ^
```

Annotate the inner expression with the type it should have and it goes through:

```morloc
m :: Matrix 2 3 Real
m = pack ((2, 3), ([1.0, 2.0, 3.0, 4.0, 5.0, 6.0] :: Vector 6 Real))
```


---

<!-- https://morloc-project.github.io/docs/types/kinds.html -->

# 5.6. The kind system

Morloc Manual > Advanced Types | https://morloc-project.github.io/docs/types/kinds.html | prev: https://morloc-project.github.io/docs/types/packable.md | next: https://morloc-project.github.io/docs/types/tensors.md

> **Warning: Experimental Feature**
> The kind system works, and the standard library’s tensor and table types are built on it, but it is young. The syntax will change, some of the rules described here are enforced late (at code generation rather than at typechecking), and you cannot yet write your own functions over the record operators. The limits are collected at the end of this section.

A kind says what sort of thing a type variable stands for. Most of the time the answer is "an ordinary type" and you never think about it. Kinds become visible when you want the compiler to track something that would otherwise be runtime data only — the length of a vector, the name of a column, the shape of a record — alongside the types it appears in.

Take a fixed-length buffer. Written the ordinary way, its length is invisible to the type system:

```morloc
newtype Buffer a = List a
```

Add a Nat-kinded parameter and the length becomes part of the type:

```morloc
newtype Buffer (n :: Nat) a = List a
```

`n` is not a type. It is a number that lives in the type system and is erased before anything runs. That is the whole idea: a kind other than `Type` lets a value be carried at compile time so the compiler can check claims about it.

> **Note**
> Kinds are descriptions of types, not types themselves. A kind classifies what fits in a slot of a type constructor; it has no runtime presence and cannot be inhabited. The `Nat` kind says "this slot holds a natural number"; the `Rec` kind says "this slot holds a record schema". The expressions that fill these slots — `5`, `(n + m)`, `{x = Int, y = Str}`, `Singleton "x" Int` — all live at the kind level. You cannot take one of them and use it as the type of a runtime value.

A kind annotation is written between a type parameter’s name and its enclosing parentheses, in the declaration of the type. A bare lowercase parameter is `Type`\-kinded, as always.

The vocabulary is fixed and checked at parse time:

| Kind | Holds |
| --- | --- |
| `Type` | The default. Any concrete type: `Int`, `[Int]`, `(Int, Str)`, your own types. Parameters written without an annotation are `Type`\-kinded. |
| `Nat` | A natural number. Lengths, dimensions, row counts. |
| `Str` | A string literal lifted to the type level. Column names and other labels. |
| `Rec` | A record schema — a mapping from field names to types. |
| `List` | An ordered list of `Str`. |
| `Set` | An unordered, duplicate-free collection of `Str`. |

`List` and `Set` currently default their element kind to `Str`; there is no surface syntax for a list of anything else. A misspelled kind is rejected where you wrote it:

```console
$ morloc typecheck badkind.loc
badkind.loc:3:16: unknown kind "Nut"; expected one of Type, Nat, Str, Rec, List, Set
    |
  3 | type Foo (n :: Nut) a
    |                ^
```

## 5.6.1. Nat: numbers in the type

Here is the buffer, complete and runnable. `concat` is a Python function that joins two lists; its Morloc signature says the result length is the sum of the input lengths.

**buffer.loc**

```morloc
module main (join)

import root-py

newtype Buffer (n :: Nat) a = List a
type Py => (Buffer (n :: Nat) a) = "list" a

instance Packable (List a) (Buffer n a) where
  source Py from "buf.py" ("list" as pack, "list" as unpack)

source Py from "buf.py" ("concat")
concat :: Buffer m a -> Buffer n a -> Buffer (m + n) a

join :: Buffer 2 Int -> Buffer 3 Int -> Buffer 5 Int
join = concat
```

**buf.py**

```python
def concat(a, b):
    return list(a) + list(b)
```

```console
$ morloc make -o buffer buffer.loc
$ ./buffer join '[1,2]' '[3,4,5]'
[1,2,3,4,5]
```

The kind annotation appears twice: once in the `newtype` declaration and once in the Python form. `Buffer 2 Int` and `Buffer 3 Int` are concrete lengths, so the compiler evaluates `m + n` and checks it against the declared result. Change the `5` on line 14 to a `6` and it says so:

```console
$ morloc typecheck buffer6.loc
buffer6.loc:15:8: error:
Type mismatch:
  expected: (Buffer 2 Int) -> (Buffer 3 Int) -> (Buffer 6 Int)
  inferred: (Buffer b a) -> (Buffer c a) -> (Buffer (b + c) a)
Subtype error: Nat constraint mismatch
  5 <: 6
   |
15 | join = concat
   |        ^
```

The four arithmetic operators `+`, `-`, `*` and `/` are available on Nats, and `/` is integer division. They are evaluated whenever both operands are ground; when a variable is still free, the check is deferred until it is solved.

> **Warning: Subtraction is not clamped**
> Nat arithmetic is ordinary integer arithmetic, so `3 - 10` is `-7`, not `0`. A signature carrying a negative dimension is accepted, and the function it describes can then never be called. This bites when a shape formula such as `h - fh + 1` is instantiated with a window larger than the input.

## 5.6.2. Str: labels in the type

A `Str`\-kinded expression is a string that exists in the type system. Written as a literal it is a quoted string in type position:

```morloc
Singleton "age" Int
```

To get one from a runtime argument, use a **label**: `f@Str` declares an argument that is a `Str` at runtime and binds the type-level variable `f` to its value at the same time. Here it names a column in a `Frame`, a schema-carrying type declared for these examples and used through the rest of the section:

```morloc
newtype Frame (r :: Rec)

column :: f@Str -> [a] -> Frame (Singleton f a)
```

Call `column "age" xs` and the runtime sees the string `"age"` while the compiler sees the result type `Frame (Singleton "age" Int)`. The same syntax carries a number (`n@Int` binds a Nat) or a list of names (`l@[Str]` binds a List).

The label form is always `name@Type`. If you meet `name:Type` in older code, it is the same idea under the spelling the parser used to accept; it is a syntax error now.

## 5.6.3. Rec: schemas in the type

A `Rec`\-kinded expression is a mapping from field names to types. The literal form is `{name = Str, age = Int}` — note `=`, not `::`, because the right-hand side of each entry is a type.

Here is `Frame` in full, with signatures for three operations over it. There is no implementation; `morloc typecheck` is enough to watch the schemas propagate.

**frame.loc**

```morloc
module main (headers, twoCols)

import root-py

newtype Frame (r :: Rec)

column  :: f@Str -> [a] -> Frame (Singleton f a)
combine :: Frame r1 -> Frame r2 -> Frame (r1 + r2)
headers :: Frame r -> [Str]

twoCols :: Frame {name = Str, age = Int}
twoCols = combine (column "name" ["ann"]) (column "age" [31])
```

```console
$ morloc typecheck frame.loc
headers :: (Frame a) -> [Str]
twoCols :: Frame {name=Str, age=Int}
```

Each `column` call produces a one-field schema; `combine` merges them; the result matches the annotation. Merging schemas that share a key is an error, because there is no sensible answer:

**clash.loc — the same declarations, one more export**

```morloc
clash :: Frame {name = Str}
clash = combine (column "name" ["ann"]) (column "name" ["bob"])
```

```console
$ morloc typecheck clash.loc
clash.loc:11:9: error:
Type mismatch:
  expected: Frame {name=Str}
  inferred: Frame ({name=Str} + {name=Str})
Subtype error: Rec constraint mismatch: Rec union has overlapping keys: name
  ({name=Str} + {name=Str}) <: {name=Str}
   |
11 | clash = combine (column "name" ["ann"]) (column "name" ["bob"])
   |         ^
```

`ProjectField` looks a field up by name and reduces to its type:

**project.loc**

```morloc
module main (getAge)

import root-py

newtype Frame (r :: Rec)

getCol :: f@Str -> Frame r -> [ProjectField r f]

getAge :: Frame {name = Str, age = Int} -> [Int]
getAge = getCol "age"
```

Misspell the field and the lookup does not reduce, which shows up as a mismatch against whatever type you expected:

```console
$ morloc typecheck project-bad.loc
project-bad.loc:10:10: error:
Type mismatch:
  expected: (Frame {name=Str, age=Int}) -> [Int]
  inferred: (Frame a) -> [a."aeg"]
Cannot compare types {age=Int, name=Str}."aeg" and Int
   |
10 | getAge = getCol "aeg"
   |          ^
```

`a."aeg"` in that message is how an unreduced `ProjectField` prints.

## 5.6.4. List and Set: collections of labels

`Restrict` projects a schema down to a list of field names, and `l@[Str]` supplies that list from a runtime argument:

**restrict.loc**

```morloc
module main (narrow)

import root-py

newtype Frame (r :: Rec)

select :: l@[Str] -> Frame r -> Frame (Restrict r l)

narrow :: Frame {name = Str, age = Int, city = Str}
       -> Frame {name = Str, city = Str}
narrow = select ["name", "city"]
```

Ask for a field that is not there and the compiler refuses, without your having written the constraint that catches it:

**restrict-bad.loc — the same module with narrow changed**

```morloc
narrow :: Frame {name = Str, age = Int, city = Str}
       -> Frame {name = Str}
narrow = select ["name", "zip"]
```

```console
$ morloc typecheck restrict-bad.loc
Constraint violation: Subset: literal set missing 'zip'
```

A constraint violation carries no source location today, so on a large module you have to find the offending call yourself.

Set-kinded expressions come up mostly through `Keys`, which turns a schema into the set of its field names. They are what the disjointness checks are stated over.

> **Warning: Type-level lists are written with ticks**
> Inside a type, a list of labels is written `['x, 'y]` — tick-prefixed names, not quoted strings. The quoted form `["x", "y"]` is a parse error, and the single-element `["x"]` is worse: it parses as a **list type whose element is the string literal type**, never reduces, and only fails at a use site.
> 
> The tick is needed because `[Str]` in type position already means "a list of strings". Note that this affects type position only: at the term level, `select ["name", "city"]` is an ordinary list of string values, written normally.

## 5.6.5. Gradual arguments

Non-`Type` kind arguments are opt-in. A type constructor applied with fewer kind arguments than it declares gets the missing positions filled with compile-time placeholders. This is what lets a casual user ignore the machinery. The examples below use the standard library’s `Vector` (a length-indexed one-dimensional array) and `Tensor3` (its rank-3 counterpart); both are covered in [Tensors](https://morloc-project.github.io/docs/types/tensors.md).

```morloc
Vector 3 U8    -- concrete: exactly 3 elements
Vector n U8    -- polymorphic: the caller determines n
Vector U8      -- gradual: no length claim
```

All three coexist in the same program, and `morloc typecheck` prints a placeholder as `_`:

**grad.loc**

```morloc
f :: Vector U8 -> Int
g :: Vector 3 U8 -> Int
```

```console
$ morloc typecheck grad.loc
f :: (Vector _ U8) -> Int
g :: (Vector 3 U8) -> Int
```

A concrete `Vector 3 U8` flows into a `Vector U8` slot. Containers of differently-sized vectors follow, because each element’s Nat is independent:

**frames.loc**

```morloc
module main (frames, sizes)

import root-py
import vector-py

frames :: [Vector U8]
frames = [[1,2,3], [1,2,3,4], [1,2,3,4,5,6]]

sizes :: [U64]
sizes = map size frames
```

```console
$ morloc make -o frames frames.loc
$ ./frames sizes
[3,4,6]
```

Filling is left-to-right within each kind, so a partially-applied constructor fixes the leading positions:

**grad2.loc**

```morloc
a :: Tensor3 Real -> Str
b :: Tensor3 h Real -> Str
c :: Tensor3 h w Real -> Str
```

```console
$ morloc typecheck grad2.loc
a :: (Tensor3 _ _ _ Real) -> Str
b :: (Tensor3 a _ _ Real) -> Str
c :: (Tensor3 a b _ Real) -> Str
```

`Type` positions are never filled this way — the element type is always required. Omitting it entirely gets past `morloc typecheck` but fails at code generation:

**bare.loc**

```morloc
module main (a)

import root-py
import vector-py

a :: Vector -> Str
a t = "x"
```

```console
$ morloc typecheck bare.loc
a :: Vector -> Str
$ morloc make -o bare bare.loc
bare.loc:1:14: error:
cannot serialize parameterised pure morloc type: Vector
  |
1 | module main (a)
  |              ^
```

**Use \`size v**

U64\` from the `Sizeable` class to read a length at runtime, whichever annotation form the signature uses.

## 5.6.6. Reference: type-level functions

The compiler recognises a small set of named operators on kinded types. They look like ordinary type applications and reduce whenever their arguments are ground.

| Function | Kind signature | Reads as | Example reduction |
| --- | --- | --- | --- |
| `Singleton k v` | `Str → Type → Rec` | one-field record | `Singleton "x" Int` → `{x = Int}` |
| `Restrict r l` | `Rec → List Str → Rec` | project to the fields in `l`, in input order | `Restrict {x=Int, y=Str, z=Real} ['x, 'z]` → `{x=Int, z=Real}` |
| `ProjectField r f` | `Rec → Str → Type` | look up one field’s type | `ProjectField {x=Int, y=Str} "x"` → `Int` |
| `Keys r` | `Rec → Set Str` | the set of field names | `Keys {x=Int, y=Str}` → `{x, y}` |
| `ListToSet l` | `List a → Set a` | drop order and duplicates | `ListToSet ['x, 'y, 'x]` → `{x, y}` |
| `Size c` | `List a` / `Set a` / `Rec` → `Nat` | number of elements | `Size {x=Int, y=Str}` → `2` |

Some of the same operations have a symbolic form. The parser sees `+`, `-`, `*` and `/` in type position and the solver picks the meaning from the kinds of the arguments:

| Operator | Kinds | Meaning |
| --- | --- | --- |
| `n + m` | `Nat → Nat → Nat` | addition |
| `n - m` | `Nat → Nat → Nat` | subtraction (may go negative) |
| `n * m` | `Nat → Nat → Nat` | multiplication |
| `n / m` | `Nat → Nat → Nat` | integer division |
| `r + s` | `Rec → Rec → Rec` | merge two schemas |
| `r - f` | `Rec → Str → Rec` | drop one field by name |
| `r - l` | `Rec → List Str → Rec` | drop the fields named in `l` |

"Reduction" means the compiler walks the expression and simplifies it where it can. `Singleton "x" Int` becomes `{x = Int}` — still a `Rec` expression, now in canonical form. The result is never a `Type`. The reductions exist so that constraints can be discharged when their arguments happen to be ground, not so that you can build inhabitable types out of kind-level fragments.

## 5.6.7. Reference: constraints

A constraint restricts what a polymorphic variable may be. It goes to the left of `⇒`:

```morloc
foo :: (Constraint1 args, Constraint2 args) => a -> b
```

Typeclass constraints (`Eq a`, `Functor f`) are the familiar kind, discharged by finding an instance. Alongside them is a small set of built-in **primitive constraints** over the kinded operators:

| Constraint | Argument kinds | Holds when |
| --- | --- | --- |
| `Member a s` | `a :: x`, `s :: Set x` | `a` appears in `s` |
| `Subset s1 s2` | both `Set x` | every element of `s1` is in `s2` |
| `Disjoint s1 s2` | both `Set x` | `s1` and `s2` share no elements |

Each reports itself by name when it fails:

```console
Constraint violation: Member: 'q' not in literal set
Constraint violation: Subset: literal set missing 'q'
Constraint violation: Disjoint: shared element(s) 'x'
```

You rarely write these. The compiler emits them from the shape of a signature: a `Restrict r l` anywhere in a signature emits `Subset (ListToSet l) (Keys r)`, and extending a schema with a new key emits a `Disjoint` against the keys already there. That is why the `select` example above rejected `"zip"` without a single `⇒` in sight.

Write the explicit form only for a constraint the compiler could not derive from your signature’s shape — for instance, disjointness between two schema variables that never meet in a `+`:

```morloc
merge :: (Disjoint (Keys r1) (Keys r2))
      => Frame r1 -> Frame r2 -> Frame (r1 + r2)
```

The constraint set is deliberately tiny. `Member`, `Subset` and `Disjoint` over finite sets of strings are decidable and cheap; richer constraint languages stop being either.

## 5.6.8. What does not work yet

**You cannot implement a function over the `Rec` operators.** A signature that mentions `r1 + r2`, `Restrict r l` or `ProjectField r f` can be declared, and it can be called, but it cannot be given a body — not even a body that delegates to a function with the identical signature. An unreduced `Rec` expression fails to unify with itself:

**wrap.loc — with the same Frame declaration as above**

```morloc
select   :: l@[Str] -> Frame r -> Frame (Restrict r l)

mySelect :: l@[Str] -> Frame r -> Frame (Restrict r l)
mySelect l t = select l t
```

```console
$ morloc typecheck wrap.loc
wrap.loc:10:16: error:
Type mismatch:
  expected: Frame (a # l)
  inferred: Frame (a # l)
Subtype error: Cannot compare Rec expressions
  (a # l) <: (a # l)
   |
10 | mySelect l t = select l t
   |                ^
```

`#` is how `Restrict` prints.

Nat expressions do not have this problem, so `Buffer (m + n) a` can be wrapped freely. In practice it means the schema-changing operations have to be primitives sourced from a foreign language; you cannot build new ones out of old ones in Morloc.

**A kind-level expression cannot be given a name.** `type R = Singleton "x" Int` is a category error — the typedef machinery wants a `Type`\-kinded body. It is not caught at typechecking; it fails at code generation:

```console
$ morloc make -o recdef recdef.loc
recdef.loc:1:14: error:
cannot serialize type Singleton "x" Int -- no per-language alias resolution for Singleton. If Singleton is a newtype handle, add `newtype Singleton <params> = <wire-type>` in stdlib/internal.
  |
1 | module main (f)
  |              ^
```

Record types that values can actually have come from a `record` declaration, which is a different feature.

**The `Member` constraint takes only a quoted literal.** `Member "x" (Keys r)` works; `Member 'x (Keys r)` is a parse error, even though the tick form is what a `List` literal requires.


---

<!-- https://morloc-project.github.io/docs/types/tensors.html -->

# 5.7. Tensors

Morloc Manual > Advanced Types | https://morloc-project.github.io/docs/types/tensors.html | prev: https://morloc-project.github.io/docs/types/kinds.md | next: https://morloc-project.github.io/docs/types/tables.md

The standard library’s tensor types carry their dimensions in the type, so the compiler can catch a shape mismatch — a 3x4 matrix where a 4x3 was wanted — even when the two functions live in different languages. This is the kind system ([The kind system](https://morloc-project.github.io/docs/types/kinds.md)) doing its most useful job.

The types live in `vector` and `tensor`, with a language module for each backend. `Vector` is the flat one-dimensional form; the higher ranks pair a runtime dimension tuple with a flat `Vector` of the row-major data:

```morloc
newtype Vector  (n :: Nat) a = List a

newtype Matrix  (m :: Nat) (n :: Nat) a
        = ((Int, Int), Vector (m * n) a)

newtype Tensor3 (d1 :: Nat) (d2 :: Nat) (d3 :: Nat) a
        = ((Int, Int, Int), Vector (d1 * d2 * d3) a)

-- Tensor4 and Tensor5 follow the same pattern
```

The `Nat` parameters exist only while the program is being compiled; at runtime a `Vector 5 Int` is a list of five integers and the `5` is gone. `a` is the element type.

Each backend maps these onto the natural array type for its language:

| Language | Native form |
| --- | --- |
| Python | `numpy.ndarray` for every rank, which puts numeric data on the zero-copy deserialization path |
| C++ | `std::vector<T>` for `Vector`; `mlc::Tensor2<T>` and up for the higher ranks, an owning buffer with an `std::mdspan` view |
| R | an atomic vector (`numeric`, `integer`, `logical`, `character`) for `Vector`; `matrix` for `Matrix`; `array` above that |

## 5.7.1. Shapes that have to agree

`matmul` in `tensor` has the signature you would write on a whiteboard:

```morloc
matmul :: Matrix m k a -> Matrix k n a -> Matrix m n a
```

The `k` appears in both arguments, so the inner dimensions must match, and the result’s shape follows from the outer ones.

**matmul.loc**

```morloc
module main (project)

import root-py
import tensor-py

-- Multiply a 2x3 matrix by a 3x2 matrix
project :: Matrix 2 3 Real -> Matrix 3 2 Real -> Matrix 2 2 Real
project = matmul
```

```console
$ morloc make -o matmul matmul.loc
$ ./matmul project '[[2,3],[1,2,3,4,5,6]]' '[[3,2],[1,0,0,1,1,1]]'
[[2,2],[4,5,10,11]]
```

Claim a shape that does not hold and the compiler names the offending dimension pair:

```console
$ morloc typecheck matmul-bad.loc
matmul-bad.loc:8:11: error:
Type mismatch:
  expected: (Matrix 2 3 Real) -> (Matrix 2 3 Real) -> (Matrix 2 3 Real)
  inferred: (Matrix b d a) -> (Matrix d c a) -> (Matrix b c a)
Subtype error: Nat constraint mismatch
  2 <: 3
  |
8 | project = matmul
  |           ^
```

> **Note: Tensors on the command line**
> A tensor argument is written in its **wire form**: the dimension tuple first, then the flat row-major data. So a 2x3 matrix of reals is `[[2,3],[1,2,3,4,5,6]]`, and that is also how a tensor result prints. A `Vector` is the exception — its wire form is a plain list, so it is written `[1,2,3]`.

## 5.7.2. Dimensions computed from other dimensions

A signature can state an arithmetic relationship between shapes, and the compiler will evaluate it. Convolution is the standard case: a valid-mode convolution of an `n`\-element signal with a `k`\-element kernel gives `n - k + 1` elements.

**conv.loc**

```morloc
module main (smooth)

import root-py
import tensor-py

source Py from "conv.py" ("conv1d")
conv1d :: Vector n Real -> Vector k Real -> Vector (n - k + 1) Real

smooth :: Vector 8 Real -> Vector 3 Real -> Vector 6 Real
smooth = conv1d
```

**conv.py**

```python
import numpy as np

def conv1d(signal, kernel):
    return np.convolve(signal, kernel, mode="valid")
```

```console
$ morloc make -o conv conv.loc
$ ./conv smooth '[1,2,3,4,5,6,7,8]' '[0.25,0.5,0.25]'
[2,3,4,5,6,7]
```

`8 - 3 + 1` is 6, so the annotation holds. Write 5 instead and the compiler does the arithmetic for you:

```console
$ morloc typecheck conv-bad.loc
conv-bad.loc:10:10: error:
Type mismatch:
  expected: (Vector 8 Real) -> (Vector 3 Real) -> (Vector 5 Real)
  inferred: (Vector a Real) -> (Vector b Real) -> (Vector ((1 + a) + (-1 * b)) Real)
Subtype error: Nat constraint mismatch
  6 <: 5
   |
10 | smooth = conv1d
   |          ^
```

The `inferred` line shows the un-substituted shape formula in the solver’s normal form — `(1 + a) + (-1 * b)` is `a - b + 1`.

Any relationship you can write as arithmetic works the same way. These are signatures you might give your own foreign functions; the standard library does not supply them:

```morloc
flatten :: Matrix m n Real -> Vector (m * n) Real
vstack  :: Matrix m n Real -> Matrix p n Real -> Matrix (m + p) n Real
kron    :: Matrix m n Real -> Matrix p q Real -> Matrix (m * p) (n * q) Real
```

When a dimension is still a free variable the check is deferred until it is solved. If it never is, it is never checked.

## 5.7.3. Dimensions that come from arguments

The constructors in `vector` and `tensor` take their sizes as ordinary integer arguments, and the label syntax (see [The kind system](https://morloc-project.github.io/docs/types/kinds.md)) lifts those arguments into the result type:

```morloc
zeros1   :: d@Int -> Vector d a
zeros2   :: d1@Int -> d2@Int -> Matrix d1 d2 a
ones2    :: d1@Int -> d2@Int -> Matrix d1 d2 a
fill2    :: a -> d1@Int -> d2@Int -> Matrix d1 d2 a
identity :: n@Int -> Matrix n n a
```

Calling one with a literal fixes the shape, and the fixed shape flows onward:

**labels.loc**

```morloc
module main (eye, scaled)

import root-py
import tensor-py

eye :: Matrix 3 3 Real
eye = identity 3

scaled :: Matrix 2 3 Real
scaled = matmul (fill2 2.0 2 2) (ones2 2 3)
```

```console
$ morloc make -o labels labels.loc
$ ./labels eye
[[3,3],[1,0,0,0,1,0,0,0,1]]
$ ./labels scaled
[[2,3],[4,4,4,4,4,4]]
```

`identity 3` really is a `Matrix 3 3 Real` as far as the typechecker is concerned:

```console
$ morloc typecheck labels-bad.loc
labels-bad.loc:7:7: error:
Type mismatch:
  expected: Matrix 4 4 Real
  inferred: Matrix 3 3 a
Subtype error: Nat constraint mismatch
  3 <: 4
  |
7 | eye = identity 3
  |       ^
```

Let-bound variables and tuple accessors work too, so `let dims = (3, 4) in zeros2 (.0 dims) (.1 dims)` is a `Matrix 3 4 Real`.

## 5.7.4. Building a tensor by hand

Higher-rank tensors reach a language boundary through `Packable` ([Serializing custom types with `Packable`](https://morloc-project.github.io/docs/types/packable.md)). The standard library declares one instance per rank:

```morloc
instance Packable ((Int, Int), Vector (d1 * d2) a)
                  (Matrix d1 d2 a)

instance Packable ((Int, Int, Int), Vector (d1 * d2 * d3) a)
                  (Tensor3 d1 d2 d3 a)
```

The split is deliberate. The **runtime** dimension tuple is what crosses the wire and tells the receiver how much buffer to allocate. The **type-level** dimensions on the `Vector` let the compiler check that the flat data has as many elements as the shape claims. Device residency — whether a tensor lives on CPU or GPU — is left out on purpose: it is local to a node and meaningless across a wire, which is the same choice NumPy’s `.npy`, Arrow IPC, ONNX, HDF5 and TensorProto make. The `pack` and `unpack` functions handle host-device transfers where a backend needs them.

`Vector` needs no instance to reach a pool. Its wire parent is `List`, so a list literal becomes a `Vector` on shape alone, and each backend’s declared form does the rest: `vector-py` maps it to `numpy.ndarray`, which the Python binding builds directly, and `vector-cpp` declares no form at all because a `List` is already a `std::vector`. That is what keeps numpy buffers on the zero-copy path and lets a `std::vector` round-trip without an intermediate Python list.

`Vector` does declare `Packable (List a) (Vector n a)`, and `vector-py` implements it, but the serializer never routes through it. It is there so you can call `pack` and `unpack` on a `Vector` yourself.

Normally the compiler calls `pack` for you. When you write a tensor literal in Morloc you call it yourself, and the inner list needs an annotation, because the compiler will not chain two `Packable` conversions:

```morloc
m :: Matrix 2 3 Real
m = pack ((2, 3), ([1.0, 2.0, 3.0, 4.0, 5.0, 6.0] :: Vector 6 Real))
```

**Without the \`**

Vector 6 Real\`, the compiler reports a missing `Packable` instance for `pack`.

## 5.7.5. Omitting dimensions

Dimension arguments are opt-in, as [The kind system](https://morloc-project.github.io/docs/types/kinds.md) describes. `Vector 3 U8`, `Vector n U8` and `Vector U8` all coexist, and a concrete vector flows into a gradual slot:

```morloc
prettyPrint :: Tensor3 Real -> Str          -- three unknown dims
normalize   :: Tensor3 h Real -> Tensor3 h w d Real
```

Positions fill left to right, so `Tensor3 h Real` fixes the first dimension and leaves the other two open.

**The element type is never optional. Use \`size v**

U64\` from `Sizeable` to read a length at runtime whatever the signature says.

## 5.7.6. What is checked, and what is not

Morloc checks that the dimensions in your compositions agree. It does not check that a foreign function honours the signature you gave it. A C++ function declared `Matrix m n Real → Matrix n m Real` that actually returns its input unchanged will not be caught. This is the same bargain as a C header file: the types are a contract and the implementation is trusted to keep it.

Arithmetic constraints are checked when every variable involved is known. When some stay free the check is deferred, and if they are never resolved it does not happen at all.


---

<!-- https://morloc-project.github.io/docs/types/tables.html -->

# 5.8. Tables

Morloc Manual > Advanced Types | https://morloc-project.github.io/docs/types/tables.html | prev: https://morloc-project.github.io/docs/types/tensors.md | next: https://morloc-project.github.io/docs/clis/index.md

> **Warning: Experimental Feature**
> Typed tables work for the operations shown here, but the type-level side has holes, and two of them will bite you. `cbind` does not reject duplicate column names unless you write the result type out — otherwise it builds a table with a repeated key. A `getCol` on a column that is not in the schema passes `morloc typecheck` and then fails at code generation with an internal message pointing at the wrong line. Both are flagged where they come up below. The API will change.

A `Table` is columnar data whose row count and column schema are part of its type:

```morloc
type Table (n :: Nat) (r :: Rec)
```

`n` is the row count and `r` is the schema — a mapping from column names to column types, such as `{state = Str, pop = Int}`. Both are erased at runtime; they exist so the compiler can tell you that a column you asked for is not there, or that two tables you are stacking disagree.

The declaration has no right-hand side, which makes `Table` an opaque primitive (see [Naming a type: `type` and `newtype`](https://morloc-project.github.io/docs/types/newtype.md)): Morloc knows nothing about its structure and each language supplies its own form. In Python a `Table` is a `pyarrow.RecordBatch`, in C++ an `mlc::ArrowTable`, and in R an `arrow::RecordBatch`. All three are views over the same Apache Arrow C Data Interface buffers, which live in a memory region the pools share rather than in any one pool’s heap. That is why a table can cross a language boundary without being copied.

Pick the language module for the backend you want: `table-py`, `table-cpp`, or `table-r`.

## 5.8.1. Building a table

`asCol` lifts a `Vector` into a one-column table, and `setCol` adds or replaces a column. Multi-column tables are built by composing them.

```morloc
-- The label f@Str makes the column name a type-level value, so the
-- result schema names the column exactly.
asCol  :: f@Str -> Vector n a -> Table n (Singleton f a)
setCol :: f@Str -> Vector n a -> Table n r
                              -> Table n ((r - f) + Singleton f a)
```

`(r - f) + Singleton f a` reads "drop any field named `f` from `r`, then add `f` back at the vector’s element type" — which is why `setCol` works whether or not the column is already there.

Everything in this section builds one program. Its header and first export:

**census.loc**

```morloc
module main
  ( census
  , shape
  , columns
  , pops
  , justNames
  , withDensity
  , bigOnly
  , byPop
  , reversed
  , summarize
  )

import root-py
import table-py
import vector-py

census :: Table 4 {state = Str, pop = Int}
census =
  let states = (["WA", "OR", "CA", "NV"] :: Vector 4 Str)
      pops   = ([7705281, 4237256, 39538223, 3104614] :: Vector 4 Int)
  in setCol "pop" pops (asCol "state" states)
```

```console
$ morloc make -o census census.loc
$ ./census census
[{"state":"WA","pop":7705281},{"state":"OR","pop":4237256},{"state":"CA","pop":39538223},{"state":"NV","pop":3104614}]
```

`table-py` supplies the table operations. `vector-py` is there for the `Functor` and `Foldable` instances on `Vector`, which the later examples use; without it, `map` over a column has no implementation.

## 5.8.2. Introspection

Three functions read a table’s shape at runtime, and none of them cares what is in it:

```morloc
nrow  :: Table n r -> Int
ncol  :: Table n r -> Int
names :: Table n r -> [Str]
```

The `r` in those signatures is a `Rec` variable — it stands for any schema at all, so one compiled function serves every table:

```morloc
shape :: Table n r -> (Int, Int)
shape t = (nrow t, ncol t)

columns :: Table n r -> [Str]
columns = names
```

```console
$ ./census shape '[{"state":"WA","pop":1}]'
[1,2]
$ ./census columns '[{"state":"WA","pop":1}]'
["state","pop"]
```

## 5.8.3. Column operations

Column operations change the schema, and the type follows along.

```morloc
-- Extract a column. ProjectField looks its type up in the schema.
getCol :: f@Str -> Table n r -> Vector n (ProjectField r f)

-- Drop columns named in a literal list.
dropCols :: l@[Str] -> Table n r -> Table n (r - l)

-- Keep columns named in a literal list, in the order given.
selectCols :: l@[Str] -> Table n r -> Table n (Restrict r l)

-- Rename one column, keeping its type.
renameCol :: f@Str -> g@Str -> Table n r
          -> Table n ((r - f) + Singleton g (ProjectField r f))

-- Project by a list computed at runtime. The result schema cannot be
-- tracked, so the caller binds it. Prefer selectCols when the names
-- are known statically.
selectColsDyn :: [Str] -> Table n r1 -> Table n r2
```

`getCol` gives back a `Vector` whose element type came out of the schema, so ordinary vector functions apply to it:

```morloc
pops :: Vector 4 Int
pops = getCol "pop" census

justNames :: Table 4 {state = Str}
justNames = selectCols ["state"] census

withDensity :: Table 4 {state = Str, pop = Int, density = Real}
withDensity =
  setCol "density" (map (\p -> toReal p / 1000.0) (getCol "pop" census)) census
```

```console
$ ./census pops
[7705281,4237256,39538223,3104614]
$ ./census justNames
[{"state":"WA"},{"state":"OR"},{"state":"CA"},{"state":"NV"}]
$ ./census withDensity
[{"state":"WA","pop":7705281,"density":7705.281},{"state":"OR","pop":4237256,"density":4237.256},{"state":"CA","pop":39538223,"density":39538.223},{"state":"NV","pop":3104614,"density":3104.614}]
```

Ask for a column that is not in the schema and `selectCols` refuses at compile time:

```console
$ morloc typecheck badcol.loc
Constraint violation: Subset: literal set missing 'county'
```

That check comes from the `Restrict r l` in \`selectCols’s own signature; you did not have to write a constraint (see [The kind system](https://morloc-project.github.io/docs/types/kinds.md)).

> **Warning: A getCol typo is not caught by the typechecker**
> `getCol` has no such constraint. A column name that is not in the schema leaves an unreduced `ProjectField` in the result type, which `morloc typecheck` reports as if it were fine:
> 
> ```console
> $ morloc typecheck typo.loc
> oops :: Vector 4 {pop=Int, state=Str}."poop"
> ```
> 
> The build then fails with an internal message located at the module’s export list:
> 
> ```console
> $ morloc make -o typo typo.loc
> typo.loc:1:14: error:
> Cannot find constructor in VarF "list"  finalType=Vector
>   |
> 1 | module main (oops)
>   |              ^
> ```
> 
> The `."poop"` in the typecheck output is the tell. Annotate the result of every `getCol` and the mismatch is reported properly instead.

`selectColsDyn` gives up on static checking entirely, which is the point of having it: the column list is not known until the program runs. What it does not do is make up for that at runtime.

> **Warning: selectColsDyn does not check the schema you claim**
> `r2` is a free variable that the caller pins down, and nothing confronts that claim with the columns that actually come back. A function declared `[Str] → Table 4 {state = Str}` will happily return a table of `pop`:
> 
> ```console
> $ ./dyn pick '["state"]'
> [{"state":"WA"},{"state":"OR"},{"state":"CA"},{"state":"NV"}]
> $ ./dyn pick '["pop"]'
> [{"pop":7705281},{"pop":4237256},{"pop":39538223},{"pop":3104614}]
> ```
> 
> The mismatch surfaces later, as a runtime error in whatever consumes the table:
> 
> ```console
> $ ./dyn2 grab '["pop"]'
> Error: run failed
> 'Field "state" does not exist in schema'
>   at grab [py] (mid=1, dyn2.loc:1:14)
> ```
> 
> Use `selectCols` whenever the column names are known when you write the code.

## 5.8.4. Row operations

Row operations leave the schema alone and may change the row count. Where the output count cannot be known statically it is left as a fresh variable `m` that the caller pins down.

```morloc
-- Rows in the half-open range [start, end). Bounds are clamped: if
-- start >= end the result is empty, and end > nrow clamps to nrow.
--   sliceRows 0 (nrow t) t            -- everything
--   sliceRows 1 (nrow t) t            -- drop the first row
--   sliceRows 0 5 t                   -- head 5
--   sliceRows (nrow t - 5) (nrow t) t -- tail 5
sliceRows :: start@Int -> end@Int -> Table n r -> Table m r

-- Keep the rows where the mask is True. The mask must be as long as
-- the table.
filterRows :: Vector n Bool -> Table n r -> Table m r

-- Gather rows by index. Indices may repeat or be out of order;
-- out-of-range indices are a runtime error.
pickRows :: Vector m Int -> Table n r -> Table m r

-- Drop duplicate rows, comparing whole rows.
distinctRows :: Table n r -> Table m r

-- Stable multi-key sort. True is ascending, False descending; later
-- entries break ties in earlier ones.
sortRows :: [(Str, Bool)] -> Table n r -> Table n r
```

```morloc
bigOnly :: Table m {state = Str, pop = Int}
bigOnly = filterRows (map (\p -> p > 5000000) (getCol "pop" census)) census

byPop :: Table 4 {state = Str, pop = Int}
byPop = sortRows [("pop", False)] census

reversed :: Table 4 {state = Str, pop = Int}
reversed = pickRows ([3, 2, 1, 0] :: Vector 4 Int) census
```

```console
$ ./census bigOnly
[{"state":"WA","pop":7705281},{"state":"CA","pop":39538223}]
$ ./census byPop
[{"state":"CA","pop":39538223},{"state":"WA","pop":7705281},{"state":"OR","pop":4237256},{"state":"NV","pop":3104614}]
$ ./census reversed
[{"state":"NV","pop":3104614},{"state":"CA","pop":39538223},{"state":"OR","pop":4237256},{"state":"WA","pop":7705281}]
```

`sortRows` takes its column names as ordinary runtime strings, not labels, so a name that is not in the schema is a runtime error rather than a compile-time one.

## 5.8.5. Stacking tables

```morloc
-- Row-wise: the schemas must match and the row counts add.
rbind :: Table n1 r -> Table n2 r -> Table (n1 + n2) r

-- Column-wise: the row counts must match and the schemas merge.
cbind :: Table n r1 -> Table n r2 -> Table n (r1 + r2)
```

`rbind` adds the row counts in the type, and the compiler does the arithmetic:

**stacked.loc**

```morloc
module main (stacked)

import root-py
import table-py

west :: Table 2 {state = Str, pop = Int}
west =
  let states = (["WA", "OR"] :: Vector 2 Str)
      pops   = ([7705281, 4237256] :: Vector 2 Int)
  in setCol "pop" pops (asCol "state" states)

south :: Table 3 {state = Str, pop = Int}
south =
  let states = (["TX", "NM", "AZ"] :: Vector 3 Str)
      pops   = ([29145505, 2117522, 7151502] :: Vector 3 Int)
  in setCol "pop" pops (asCol "state" states)

stacked :: Table 5 {state = Str, pop = Int}
stacked = rbind west south
```

```console
$ morloc make -o stacked stacked.loc
$ ./stacked stacked
[{"state":"WA","pop":7705281},{"state":"OR","pop":4237256},{"state":"TX","pop":29145505},{"state":"NM","pop":2117522},{"state":"AZ","pop":7151502}]
```

Claim 6 rows instead of 5:

```console
$ morloc typecheck stacked-bad.loc
stacked-bad.loc:19:11: error:
Type mismatch:
  expected: Table 6 {state=Str, pop=Int}
  inferred: Table 5 {pop=Int, state=Str}
Subtype error: Nat constraint mismatch
  5 <: 6
   |
19 | stacked = rbind west south
   |           ^
```

`cbind` merges schemas with `+`. Merging two schemas that share a column name has no sensible answer, so it is meant to be rejected:

**widen.loc**

```morloc
module main (widened, oops)

import root-py
import table-py

names :: Table 2 {state = Str}
names = asCol "state" (["WA", "OR"] :: Vector 2 Str)

pops :: Table 2 {pop = Int}
pops = asCol "pop" ([7705281, 4237256] :: Vector 2 Int)

again :: Table 2 {state = Str}
again = asCol "state" (["CA", "NV"] :: Vector 2 Str)

widened :: Table 2 {state = Str, pop = Int}
widened = cbind names pops

oops :: Table 2 ({state = Str} + {state = Str})
oops = cbind names again
```

```console
$ morloc typecheck widen.loc
widen.loc:19:8: error:
Type mismatch:
  expected: Table 2 ({state=Str} + {state=Str})
  inferred: Table 2 ({state=Str} + {state=Str})
Subtype error: Rec constraint mismatch: Rec union has overlapping keys: state
  ({state=Str} + {state=Str}) <: ({state=Str} + {state=Str})
   |
19 | oops = cbind names again
   |        ^
```

The message prints the same type twice, which is unhelpful, but the middle line names the clash.

> **Warning: Always annotate the result of cbind**
> Delete the `oops ::` line so the result type is inferred, and the same program compiles and runs, producing a table with a duplicated key:
> 
> ```console
> $ ./widen2 oops
> [{"state":"WA","state":"CA"},{"state":"OR","state":"NV"}]
> ```
> 
> Writing the expected schema on the binding turns it back into a compile-time error. Do that on every `cbind` until this is fixed.

## 5.8.6. Crossing a language boundary

A table handoff between pools passes a shared-memory offset and a schema descriptor, not the data. The receiving pool imports the same column buffers.

Here Python loads the table with `pyarrow` and C++ slices it:

**crosslang.loc**

```morloc
module main (top2)

import root-py
import table-cpp

-- table-cpp gives the C++ operations; this line gives the Python side
-- the form it needs to hand a table across.
type Py => (Table (n :: Nat) (r :: Rec)) = "arrow" n r

source Py from "loader.py" ("load_census" as loadCensus)
loadCensus :: Int -> Table n {state = Str, pop = Int}

top2 :: Int -> Table m {state = Str, pop = Int}
top2 year = sliceRows 0 2 (loadCensus year)
```

**loader.py**

```python
import pyarrow as pa

def load_census(_year):
    return pa.record_batch(
        {"state": pa.array(["WA", "OR", "CA", "NV"]),
         "pop":   pa.array([7705281, 4237256, 39538223, 3104614])}
    )
```

```console
$ morloc make -o crosslang crosslang.loc
$ ./crosslang top2 2024
[{"state":"WA","pop":7705281},{"state":"OR","pop":4237256}]
$ ls crosslang-build/pools/
cpp
py
```

Two pools, and the table itself never leaves shared memory. Import both `table-py` and `table-cpp` and the compiler would collapse the program onto one language instead; the explicit `type Py ⇒ Table …​` line above supplies the Python form without the Python operations, which is what forces the split.

## 5.8.7. Reading and writing table files

A `Table` argument can be a literal JSON string or a path, and the runtime detects the format:

| Form | How it is recognised |
| --- | --- |
| JSON | Row-oriented `[{col: v, …​}, …​]` or column-oriented `{col: [v, …​], …​}`; the two are equivalent |
| Arrow IPC | the `ARROW1` magic |
| Parquet | the `PAR1` magic at head and tail |
| CSV / TSV | the `.csv` / `.tsv` extension; a header row is required |

The schema in your signature drives validation, and a file that does not match it is rejected before the data reaches a pool:

```console
$ ./census summarize bad.csv
Error: failed to parse argument #0: file 'bad.csv': Declared column 'pop' missing from CSV header
$ ./census summarize wrong.csv
Error: failed to parse argument #0: file 'wrong.csv': Failed to read CSV: Parser error: Error while parsing value 'abc' as type 'Int64' for column 1 at line 1. Row data: '[WA,abc]'
```

A nullable Arrow or Parquet column is accepted into a non-optional Morloc column as long as it holds no nulls at runtime. One actual null and it is refused:

```console
$ ./census summarize plainnull.parquet
Error: failed to parse argument #0: file 'plainnull.parquet': Failed to project record batch: Invalid argument error: Column 'pop' is declared as non-nullable but contains null values
```

> **Warning: Compressed Parquet cannot be read**
> The Parquet reader is compiled without its compression codecs, so a file written with snappy — the default for pyarrow, pandas and Spark — fails:
> 
> ```console
> $ ./census summarize snappy.parquet
> Error: failed to parse argument #0: file 'snappy.parquet': Failed to read Parquet record batches: Parquet argument error: Parquet error: Disabled feature at compile time: snap
> ```
> 
> Re-write the file with `compression='none'`, or use Arrow IPC, until this is fixed. Parquet written by the nexus itself is uncompressed and reads back fine.

Results are written in whatever `--output-form` (short form `-f`) asks for. It is a nexus option, so it goes to the left of the subcommand; putting it after gives `error: unexpected argument '-f' found`.

```console
$ ./census -f csv census > census.csv
$ cat census.csv
state,pop
WA,7705281
OR,4237256
CA,39538223
NV,3104614
$ ./census -f arrow   census > census.arrow
$ ./census -f parquet census > census.parquet
```

And read back, whatever the format, they are the same table:

```console
$ ./census summarize census.csv
54585374
$ ./census summarize census.arrow
54585374
$ ./census summarize census.parquet
54585374
$ ./census summarize '[{"state":"WA","pop":7705281},{"state":"OR","pop":4237256}]'
11942537
$ ./census summarize '{"state":["WA","OR"],"pop":[7705281,4237256]}'
11942537
```

where `summarize` is the last export of `census.loc`:

```morloc
summarize :: Table n {state = Str, pop = Int} -> Int
summarize t = fold (+) 0 (getCol "pop" t)
```

The Arrow, Parquet and CSV libraries are compiled into the nexus binary, so none of this depends on PyArrow, arrow-cpp or arrow-r being installed for a pool. Pools only ever see the Arrow C Data Interface.

## 5.8.8. Limits

**Column types must be primitive.** `Bool`, `Int`, `Real`, the sized integer and float types, and `Str`. A list-, struct-, or dictionary-typed column is accepted by the typechecker and fails when the data is built:

```console
$ ./nested t
Error: run failed
Unsupported Arrow column type for column 1
  at t [py] (mid=1, nested.loc:1:14)
```

`Date`, `Timestamp` and `Duration` round-trip as the underlying integer or string but have no Morloc types of their own yet.

**A table cannot be piped in.** A file path works and inline JSON works, but `-` for standard input fails:

```console
$ cat census.csv | ./census summarize -
Error: failed to parse argument #0: stdin: serialization error: Cannot compute msgpack size for a Table; Tables use the Arrow IPC SHM wire path
```

**You cannot write your own column operations.** A function whose signature mentions `r1 + r2`, `Restrict r l` or `ProjectField r f` can be declared and called but cannot be given a body, even one that delegates to a stdlib function with the same signature. See the end of [The kind system](https://morloc-project.github.io/docs/types/kinds.md). In practice every schema-changing operation has to be a primitive sourced from a foreign language.

**Tables are immutable.** Every column-modifying operation produces a new table. The Arrow shared-memory layer is reference-counted across pools, so building a "new" table is usually only a descriptor update, but there is no in-place mutation API.

Joins, group-by, aggregation and column casting belong to follow-on modules and are not part of `table`.


---

<!-- https://morloc-project.github.io/docs/clis/ -->

# 6. Building CLIs

Morloc Manual | https://morloc-project.github.io/docs/clis/ | prev: https://morloc-project.github.io/docs/types/tables.md | next: https://morloc-project.github.io/docs/clis/example-program.md

A Morloc module compiles to a command line tool. Every exported term becomes a subcommand, its type becomes the subcommand’s arguments and return value, and its docstring becomes the help text. You saw the smallest version of this in [Your first program](https://morloc-project.github.io/docs/getting-started/first-program.md): a two-line module, and `./hello -h` printed a usage statement nobody wrote.

This chapter is about the rest of it. Not about writing an interface — there is still no parser to write — but about the controls you have over the one the compiler derives: what the commands are called, which arguments are positional and which are flags, where an argument’s bytes come from, and what the result looks like on the way out.

Two properties are worth naming up front, because they are what the rest of the chapter builds on.

**The interface cannot drift from the functions.** It is generated from the same types the compiler checks calls against. Rename an argument, add a field to a record, change a return type, and the help, the JSON Schema, and the MCP tool definition all move with it on the next build. There is no second description of the tool to keep in sync.

**What passes between two Morloc tools is a value, not text.** A command writes its return type, serialized; a command that accepts that type reads it, in any of the formats both sides already understand. Neither end invents a file format, and neither end parses one.


---

<!-- https://morloc-project.github.io/docs/clis/example-program.html -->

# 6.1. The example program

Morloc Manual > Building CLIs | https://morloc-project.github.io/docs/clis/example-program.html | prev: https://morloc-project.github.io/docs/clis/index.md | next: https://morloc-project.github.io/docs/clis/argument-zones.md

The chapter uses one tool throughout: `sift`, which searches a directory tree for lines matching a pattern and reports what it finds. It is small enough to read in one sitting and has every shape the chapter needs — positional arguments, flags, a record of options, file inputs, structured output, and a streaming mode.

The work is done in Python:

**sift.py**

```python
import os

def walk_files(root):
    out = []
    for dirpath, dirnames, filenames in os.walk(root):
        dirnames.sort()
        for name in sorted(filenames):
            out.append(os.path.join(dirpath, name))
    return out

def hits_in(path, needles, fold):
    out = []
    with open(path) as fh:
        for i, line in enumerate(fh, start=1):
            hay = line.lower() if fold else line
            if any(n in hay for n in needles):
                out.append({"path": path, "line": i,
                            "text": line.rstrip("\n")})
    return out

def scan_many(patterns, root, opts):
    fold = opts["ignoreCase"]
    limit = opts["maxCount"]
    needles = [p.lower() if fold else p for p in patterns]
    hits = []
    for path in walk_files(root):
        hits.extend(hits_in(path, needles, fold))
        if limit and len(hits) >= limit:
            return hits[:limit]
    return hits

def scan(pattern, root, opts):
    return scan_many([pattern], root, opts)

def produce(pattern, root, opts, sink):
    fold = opts["ignoreCase"]
    needles = [pattern.lower() if fold else pattern]
    for path in walk_files(root):
        sink(hits_in(path, needles, fold))

def as_lines(hits):
    return "".join("%s:%d:%s\n" % (h["path"], h["line"], h["text"])
                   for h in hits)

def per_file(hits):
    counts = {}
    for h in hits:
        counts[h["path"]] = counts.get(h["path"], 0) + 1
    return [[p, n] for p, n in counts.items()]

def total(counts):
    return sum(n for _, n in counts)

def numbered(offset, hits):
    return ["%d %s:%d" % (offset + i + 1, h["path"], h["line"])
            for i, h in enumerate(hits)]
```

None of it knows about Morloc. It takes and returns dictionaries, lists, and strings.

The Morloc side gives those functions types, names them, and exports five of them. Read past the docstring directives for now — each one is introduced in its own section below.

**sift.loc**

```morloc
--' Search notes and count what turns up
module sift (scan, scanAll, summarize, total, stream)

import root-py

--' One matching line
record Hit where
  path :: Str
  line :: Int
  text :: Str

record Py => Hit = "dict"

--' How to search
--' @unroll
--' @arg --options
record Options where
  --' Match without regard to case
  --' @true -i/--ignore-case
  ignoreCase :: Bool

  --' Stop after this many hits; 0 means no limit
  --' @arg -m/--max-count
  --' @default 0
  maxCount :: Int

record Py => Options = "dict"

source Py from "sift.py"
  ( "scan"      as scanPy
  , "scan_many" as scanManyPy
  , "produce"   as producePy
  , "as_lines"  as asLines
  , "per_file"  as perFile
  , "total"     as totalPy
  , "numbered"  as numberHits
  )

scanPy     :: Str -> Str -> Options -> <IO> [Hit]
scanManyPy :: [Str] -> Str -> Options -> <IO> [Hit]
producePy  :: Str -> Str -> Options -> ([Hit] -> <IO> ()) -> <IO> ()
perFile    :: [Hit] -> [(Str, Int)]
totalPy    :: [(Str, Int)] -> Int

--' Print one `path:line:text` record per line
asLines :: [Hit] -> Str

--' Report the number of matches instead of the matches
countHits :: [Hit] -> U64
countHits = size

--' Number the hits as they stream past
numberHits :: U64 -> [Hit] -> [Str]

--' Count the hits without loading them into memory
countStaged :: IFile [Hit] -> <IO> Int
countStaged f = do
  Ok n <- @flen f
  n

--' Search a directory tree for lines containing a pattern
--' @with   -c/--count=countHits
--' @render -p/--plain=asLines
scan ::
  --' The text to search for
  --' @metavar PATTERN
  Str ->
  --' The directory to search
  --' @check.path r
  Str ->
  Options ->
  <IO> [Hit]
scan = scanPy

--' Search for any of several patterns, one per line of a file
--' @with   -c/--count=countHits
--' @render -p/--plain=asLines
scanAll ::
  --' A file of patterns, one per line
  --' @form list
  [Str] ->
  --' The directory to search
  --' @check.path r
  Str ->
  Options ->
  <IO> [Hit]
scanAll = scanManyPy

--' Count the hits in each file
summarize ::
  --' Hits produced by an earlier search
  [Hit] ->
  [(Str, Int)]
summarize = perFile

--' Add up a stream of per-file counts
total ::
  --' A file of counts; standard input when omitted
  --' @stdin
  Str ->
  <IO> Int
total f = do
  Ok s <- @open f :: <IO> (Try Str (IStream (Str, Int)))
  Ok counts <- @next s
  totalPy counts

--' Stream hits to standard output, one file at a time
--' @render -p/--plain=asLines @stream
--' @with   -c/--count=countHits
--' @with   -n/--staged=countStaged
--' @with   -N/--numbered=numberHits(@offset) @stream
stream ::
  --' The text to search for
  Str ->
  --' The directory to search
  --' @check.path r
  Str ->
  Options ->
  <IO> ()
stream pat root opts = @collect (producePy pat root opts)
```

There is something to search. Make it now; every search in this chapter runs against these two files:

```console
$ mkdir -p notes/2026
$ printf 'buy milk\nfix the parser\nwrite the manual\nuse -p for plain output\n' > notes/todo.txt
$ printf 'fix the build\nship the manual\nrest\n' > notes/2026/plan.txt
```

Build it:

```console
$ morloc make -o sift sift.loc
```

The five exports are the five subcommands, each with the first line of its docstring:

```console
$ ./sift -h
Search notes and count what turns up

Usage: ./sift <nexus_options> <command> <command_options>

Commands:
  scan       Search a directory tree for lines containing a pattern
  scanAll    Search for any of several patterns, one per line of a file
  summarize  Count the hits in each file
  total      Add up a stream of per-file counts
  stream     Stream hits to standard output, one file at a time

General Options:
  -h, --help  Print help; -hh adds details and examples, -hhh adds schemas
              (nexus options: -h @)
```

and `scan` runs:

```console
$ ./sift scan the notes -p
notes/todo.txt:2:fix the parser
notes/todo.txt:3:write the manual
notes/2026/plan.txt:1:fix the build
notes/2026/plan.txt:2:ship the manual
```

The docstring above `module` becomes the program’s description. `-h` shows its first line; repeating the flag shows more. `-hh` adds the rest of the description and any examples, and `-hhh` adds the layout of every named type the help mentions. `--help` is the same flag spelled long and repeats the same way. This is the split every piece of help follows: a command’s own `-h` shows the first line of its docstring, its `-hh` the rest.

A module docstring can also carry an `@epilogue` block. Everything after that directive is printed verbatim below the options, from `-hh` up:

```morloc
--' Search notes and count what turns up
--'
--' @epilogue
--' Examples:
--'   sift scan needle ./notes
--'   sift total needle ./notes
module sift (scan, scanAll, summarize, total, stream)
```

```console
$ ./sift -hh
Search notes and count what turns up

Usage: ./sift <nexus_options> <command> <command_options>
...
General Options:
  -h, --help  Print help; -hh adds details and examples, -hhh adds schemas
              (nexus options: -h @)

Examples:
  sift scan needle ./notes
  sift total needle ./notes
```

A command’s signature preamble takes the same block, and it renders at the foot of that subcommand’s help alone. Blank lines and `#` comments inside the block are kept, so each example can say what it is for:

```morloc
--' Count matches of a needle in one file
--' @epilogue
--' Examples:
--'
--' # count in a single file
--' sift total needle ./notes/today.md
total :: Str -> Str -> Int
```


---

<!-- https://morloc-project.github.io/docs/clis/argument-zones.html -->

# 6.2. The two argument zones

Morloc Manual > Building CLIs | https://morloc-project.github.io/docs/clis/argument-zones.html | prev: https://morloc-project.github.io/docs/clis/example-program.md | next: https://morloc-project.github.io/docs/clis/docstrings.md

Every Morloc program’s argv is split into two zones. The **nexus zone** holds the options that every Morloc CLI has — output format, output file, logging. The **command zone** holds the arguments and flags of the subcommand you are calling.

The two zones have separate namespaces, which is the point of the split: the options the runtime provides can never collide with the options your function declares. `sift` has a `-p` in each zone — `-p/--print` is the nexus’s pretty-printer, `-p/--plain` is the formatter declared on `scan` — and they do not interfere:

```console
$ ./sift -p scan the notes
[
  {
    "path": "notes\/todo.txt",
    "line": 2,
    "text": "fix the parser"
  },
...

$ ./sift scan the notes -p
notes/todo.txt:2:fix the parser
notes/todo.txt:3:write the manual
notes/2026/plan.txt:1:fix the build
notes/2026/plan.txt:2:ship the manual
```

The boundary is the subcommand name. Everything left of it is the nexus zone, everything right of it is the command zone. The nexus zone has no positionals, so every token in it is an `-x` or `--option` taking a fixed number of values. `-f` picks the output format and is the nexus option you will reach for most often; [Output formats](https://morloc-project.github.io/docs/clis/output-formats.md) covers it and the rest:

```console
$ ./sift -f jsonl scan the notes -c
4

$ ./sift scan -f jsonl the notes -c
error: unexpected argument '-f' found
...
```

Help follows the same rule. `-h` in the command zone documents the command; `-h` in the nexus zone documents the runtime:

```console
$ ./sift scan -h        # help for the `scan` command
$ ./sift -h             # help for the program: its commands
$ ./sift -h @           # help for the nexus: -f, -o, -p, and the rest
```

That last one introduces `@`, the explicit zone separator. You rarely need it, because a subcommand name already marks the boundary. It matters when there is no subcommand name to mark it.

## 6.2.1. Programs with a single export

When a module exports exactly one term, naming it is optional — there is nothing to choose between. Take a one-command program:

**greet.loc**

```morloc
--' Say hello
module greet (hello)

import root-py

--' Greet someone by name
hello :: Str -> Str
hello name = "Hello, " <> name
```

```console
$ morloc make -o greet greet.loc
$ ./greet Weena
"Hello, Weena"
$ ./greet hello Weena
"Hello, Weena"
```

Both forms work, and the help says so by putting `@` where the subcommand name would go:

```console
$ ./greet -h
Greet someone by name

Usage: ./greet <nexus_options> @ <command_options>

General Options:
  -h, --help  Print help; -hh adds details and examples, -hhh adds schemas
              (nexus options: -h @)

Positional arguments:
  1:  type: Str
      format: literal string

Return: Str
```

With the name omitted there is no token marking the zone boundary, so a nexus option has nothing to end it and the parser reads it as a command argument:

```console
$ ./greet -f jsonl Weena
error: unexpected argument '-f' found
...
```

Write `@` to close the nexus zone by hand:

```console
$ ./greet -f jsonl @ Weena
"Hello, Weena"
$ ./greet -p @ Weena
Hello, Weena
```

The command name will not do it here. It is optional, so the parser cannot treat it as a boundary marker, and spelling it out changes nothing:

```console
$ ./greet -f jsonl hello Weena
error: unexpected argument '-f' found
...
```

`@` is accepted in multi-command programs too, where it is redundant with the subcommand name:

```console
$ ./sift -f jsonl @ scan the notes -c
4
```


---

<!-- https://morloc-project.github.io/docs/clis/docstrings.html -->

# 6.3. Docstrings

Morloc Manual > Building CLIs | https://morloc-project.github.io/docs/clis/docstrings.html | prev: https://morloc-project.github.io/docs/clis/argument-zones.md | next: https://morloc-project.github.io/docs/clis/arguments.md

A docstring is a comment that the compiler keeps. An ordinary `--` comment is discarded after parsing; a `--'` comment is attached to whatever follows it and travels through to the generated interface. That is the whole authoring surface for the CLI: you never configure the parser, you annotate the code.

Docstrings attach in five places, and each one lands somewhere different in the help:

| Above | Becomes |
| --- | --- |
| `module` | The program’s description, shown at the top of `./prog -h`. |
| a term’s signature | The command’s description, shown at the top of `./prog <cmd> -h`. |
| a type inside a signature | That argument’s description; on the last type, the return description. |
| a `type` or `record` definition | The description of every argument and return that uses that type. A record must use the `record X where` form; see the warning below. |
| a `record` field | That field’s description, when the record is split into one flag per field (see [Record arguments](https://morloc-project.github.io/docs/clis/record-arguments.md)). |

The third and fourth interact usefully. ``sift’s `scan`` documents its arguments inline:

```morloc
--' Search a directory tree for lines containing a pattern
scan ::
  --' The text to search for
  Str ->
  --' The directory to search
  Str ->
  Options ->
  <IO> [Hit]
```

which is fine for two arguments used once. When the same type appears across several signatures, describing it at the type is less to write and impossible to get out of step:

**cipher.py**

```python
def xor(key, msg):
    return "".join(chr(ord(c) ^ ord(key[i % len(key)]))
                   for i, c in enumerate(msg))
```

**cipher.loc**

```morloc
module cipher (encode, decode)

import root-py

source Py from "cipher.py" ("xor")

xor :: Str -> Str -> Str

--' A secret key
--' @metavar KEY
type Key = Str

--' An encrypted message
--' @metavar CIPHERTEXT
type CipherText = Str

--' A decrypted message
--' @metavar PLAINTEXT
type PlainText = Str

--' Encode a plaintext with a key
encode :: Key -> PlainText -> CipherText
encode = xor

--' Decode a ciphertext with a key
decode :: Key -> CipherText -> PlainText
decode = xor
```

Both commands inherit the descriptions and the metavars, in the right positions, with nothing repeated:

```console
$ ./cipher encode -h
Encode a plaintext with a key

Usage: ./cipher <nexus_options> encode <command_options>

General Options:
  -h, --help  Print help; -hh adds details and examples, -hhh adds schemas
              (nexus options: -h @)

Positional arguments:
  1: KEY        A secret key
                type: Str
                format: literal string
  2: PLAINTEXT  A decrypted message
                type: Str
                format: literal string

Return: Str
  An encrypted message
```

An inline docstring on an argument wins over the one inherited from its type, so a signature can specialize a description where it matters and inherit it everywhere else.

## 6.3.1. Directives

A docstring line is either **prose** or a **directive**. A directive begins with `@`:

```
@keyword [arguments...]
```

The keyword is the first whitespace-delimited token; the rest of the line is its value. Some directives are bare switches (`@unroll`, `@many`, `@stdin`) and take no value at all.

Two other sigils appear inside directive values. `$1`, `$2`, …​ refer to the command’s own arguments by position, and `@value` and `@offset` name values the runtime supplies. Both are used by output actions and are introduced there.

To start a prose line with a literal `@`, escape it: `\@`.

The first line of \`scan’s preamble is prose; the two below it are directives:

```morloc
--' Search a directory tree for lines containing a pattern
--' @with   -c/--count=countHits
--' @render -p/--plain=asLines
```

Order does not matter. Prose lines are concatenated in the order written and become the description; directive lines are collected wherever they sit in the block.

Every directive is listed in [Directive reference](https://morloc-project.github.io/docs/clis/directive-reference.md). The rest of this chapter introduces them in the order you are likely to need them.

> **Warning**
> A misspelled directive is treated as prose, but the build says so. Writing `@metvar FILE` warns and keeps going:
> 
> ```console
> warning: unknown docstring directive 'metvar' (recognized: name, literal, many, stdin, unroll, default, metavar, arg, true, false, return, source, form, check.<kind>, list.source, list.form, list.check.<kind>, with, mime); if this line was meant as prose, prefix its content with '\' to suppress this warning
> ```
> 
> The warning prints whether or not the build succeeds. If a directive appears to have no effect, check the build output first, then its spelling against [Directive reference](https://morloc-project.github.io/docs/clis/directive-reference.md).
> 
> A related trap: for compatibility with an older syntax, a prose line whose first word ends in a colon is also read as a directive. `Example: pass a number` parses as the directive `Example` and is echoed back into the help verbatim, which usually looks fine and occasionally is not. Prefer a colon later in the line, or escape the line with a leading `\`.


---

<!-- https://morloc-project.github.io/docs/clis/arguments.html -->

# 6.4. Arguments

Morloc Manual > Building CLIs | https://morloc-project.github.io/docs/clis/arguments.html | prev: https://morloc-project.github.io/docs/clis/docstrings.md | next: https://morloc-project.github.io/docs/clis/record-arguments.md

By default every argument in a signature is a positional, in the order it appears. `summarize :: [Hit] → [(Str, Int)]` takes one argument, so the command takes one token. Save a search first, since the rest of the chapter reuses it:

```console
$ ./sift scan the notes > hits.json
$ ./sift summarize hits.json
[["notes\/todo.txt",2],["notes\/2026\/plan.txt",2]]
```

What that token may be depends on the argument’s type.

**Scalars and strings are read verbatim.** An `Int`, `Real`, `Bool`, sized integer or float, or a `Str` is taken from argv as written — no JSON quoting, no escaping:

```console
$ ./sift scan the notes -c
4
```

`the` is the `Str` pattern and `notes` is the `Str` directory.

Numbers need no quoting either, negative ones included: a leading `-` starts an option only when the next character is a letter. A second small program to show it with, used again later in this section:

**calc.py**

```python
def add(x, y):
    return x + y

def join(sep, words):
    return sep.join(words)
```

**calc.loc**

```morloc
module calc (add, join)

import root-py

source Py from "calc.py" ("add", "join")

--' Add two numbers
add :: Real -> Real -> Real

--' Join words with a separator
--' @name cat
join ::
  --' the separator
  Str ->
  --' the words to join
  --' @many
  [Str] ->
  Str
```

```console
$ morloc make -o calc calc.loc
$ ./calc add -4.0 -7
-11
```

**Everything else is a value in a recognized format.** Lists, tuples, records, and maps accept either a JSON value inline or a path to a file holding one:

```console
$ ./sift summarize '[{"path":"a.txt","line":1,"text":"x"}]'
[["a.txt",1]]

$ ./sift summarize hits.json
[["notes\/todo.txt",2],["notes\/2026\/plan.txt",2]]
```

The file’s format is detected from its contents, not its name. JSON, MessagePack, and Morloc’s own binary form (voidstar) are all recognized, so a file produced by an earlier command is read back without saying how it was written:

```console
$ ./sift -f mpk scan the notes > hits.mpk
$ ./sift summarize hits.mpk
[["notes\/todo.txt",2],["notes\/2026\/plan.txt",2]]
```

Arrow IPC and Parquet are recognized as well when the target type is a `Table`.

**Standard input is a value source too.** The token `-` (or `/dev/stdin`) reads the argument from standard input, which is what makes two Morloc commands compose in a pipeline:

```console
$ ./sift scan the notes | ./sift summarize -
[["notes\/todo.txt",2],["notes\/2026\/plan.txt",2]]
```

Only one argument per command may claim stdin; a second `-` is an error rather than a silent read of zero bytes.

## 6.4.1. When an argument is wrong

An argument that looks like a path — it contains a `/`, or ends in a recognized data extension — but does not exist is reported as a missing file rather than parsed as inline data:

```console
$ ./sift summarize nosuch.json
Error: failed to parse argument #0: file 'nosuch.json' not found
```

A file that exists but does not hold what the type wants is reported against the file:

```console
$ echo 'not json' > bad.json
$ ./sift summarize bad.json
Error: failed to parse argument #0: file 'bad.json': serialization error: JSON parse error: expected ident at line 1 column 2
```

Failures exit non-zero, so a Morloc command is safe to put in a `set -e` script or a `&&` chain.

> **Note**
> Errors number arguments from zero (`argument #0`) while `--help` numbers positionals from one. `argument #0` is the argument printed as `1:`.

## 6.4.2. Options, flags, and repeats

An argument becomes an option instead of a positional when you give it a flag name with `@arg`. An option can be omitted, so it also needs a `@default`:

```morloc
  --' Stop after this many hits; 0 means no limit
  --' @arg -m/--max-count
  --' @default 0
  maxCount :: Int
```

The default is written in JSON, and the compiler insists on it. Drop the `@default` line from `sift.loc` and the build stops:

```console
$ morloc make -o sift sift.loc
In sift:scan, argument #3, field maxCount: optional argument -m/--max-count must be given a default value
```

A `Bool` is a flag, not an option with a value, so it uses a different pair of directives. `@true` names the spelling that turns it on, and the default is false:

```morloc
  --' Match without regard to case
  --' @true -i/--ignore-case
  ignoreCase :: Bool
```

```console
$ ./sift scan MANUAL notes -i -p
notes/todo.txt:3:write the manual
notes/2026/plan.txt:2:ship the manual
```

`@false` is the mirror image: it names the spelling that turns the flag off, and the default becomes true. Giving both declares an on switch and an off switch for the same field. Using `@arg` on a `Bool` is rejected, with the alternative spelled out — change ``ignoreCase’s `@true`` to `@arg` and:

```console
$ morloc make -o sift sift.loc
In sift:scan, argument #3, field ignoreCase: a Bool argument cannot use `@arg`. Use `@true <opt>` (default false, the flag turns it on) or `@false <opt>` (default true, the flag turns it off) instead.
```

Both spellings are delivered, and help shows the flag with the `true` default it turns off. Adding `@false -s/--skip-empty` to a `reportEmpty` field gives:

```console
  -s, --skip-empty              Report each file even when it has no hits
                                type: Bool
                                default: true
```

`@many` makes an argument variadic: it consumes the remaining argv tokens and assembles them into a list. It applies to a `[a]`\-typed argument, and as a positional it must be the last one. ``calc’s `join`` above declares one:

```console
$ ./calc cat + a b c
"a+b+c"
```

## 6.4.3. Naming

`@name` gives a command a name of its own, independent of the Morloc term. `calc` exports `join` and calls the subcommand `cat`:

```console
$ ./calc -h
Usage: ./calc <nexus_options> <command> <command_options>

Commands:
  add  Add two numbers
  cat  Join words with a separator

General Options:
  -h, --help  Print help; -hh adds details and examples, -hhh adds schemas
              (nexus options: -h @)
```

Use it when the shell-facing name and the library-facing name want to differ — a name that reads well in a pipeline is not always the name you want to import.

`@metavar` names an argument. On an option it becomes the placeholder in the help text, replacing the type name:

```morloc
  --' how many things
  --' @arg -n/--num
  --' @metavar COUNT
  --' @default 0
  Int ->
```

```
Optional arguments:
  -n, --num <COUNT>  how many things
                     type: Int [default: 0]
```

On a positional it labels the slot, beside the index:

```console
$ ./sift scan -h
...
Positional arguments:
  1: PATTERN  The text to search for
              type: Str
              format: literal string
  2:          The directory to search
              type: Str
              format: path to a readable file
...
```

`scan` names only its first positional, so the second keeps a bare index and the two labels pad to a common width.

The same name is what the interface is keyed on wherever it is consumed by a program — the property name in the JSON Schema and in the MCP tool definition (`key` and `ciphertext` below, from the `@metavar KEY` and `@metavar CIPHERTEXT` on the `cipher` type definitions of [Docstrings](https://morloc-project.github.io/docs/clis/docstrings.md)):

```console
$ ./cipher --json-help | python3 -c "
import json,sys
d=json.load(sys.stdin)
for c in d['commands']:
  print(c['name'], [(a['name'], a['metavar']) for a in c['arguments']])
"
encode [('key', 'KEY'), ('plaintext', 'PLAINTEXT')]
decode [('key', 'KEY'), ('ciphertext', 'CIPHERTEXT')]
```

An unnamed positional is identified by index alone in both places, which is worth avoiding on anything a model or a script will call.

## 6.4.4. Ending option parsing

A bare `--` ends option parsing: every token after it is a positional, even one that looks like a flag. This is rarely needed, since `-4.0` and `-7` are already treated as positionals, but it is the way to pass a string that looks like a short option:

```console
$ ./sift scan -- -p notes
[{"path":"notes\/todo.txt","line":4,"text":"use -p for plain output"}]
```

Note what that costs: after `--`, the command’s own `-p` formatter is a positional too, so a search for the literal text `-p` cannot also ask for plain output.


---

<!-- https://morloc-project.github.io/docs/clis/record-arguments.html -->

# 6.5. Record arguments

Morloc Manual > Building CLIs | https://morloc-project.github.io/docs/clis/record-arguments.html | prev: https://morloc-project.github.io/docs/clis/arguments.md | next: https://morloc-project.github.io/docs/clis/sum-type-arguments.md

A record argument is a natural fit for a group of related settings, but a caller does not want to write a JSON object to set one field. `@unroll` splits the record open: each field becomes its own flag, and the record is reassembled before the call.

``sift’s `Options`` is declared once and used by three commands:

```morloc
--' How to search
--' @unroll
--' @arg --options
record Options where
  --' Match without regard to case
  --' @true -i/--ignore-case
  ignoreCase :: Bool

  --' Stop after this many hits; 0 means no limit
  --' @arg -m/--max-count
  --' @default 0
  maxCount :: Int
```

Each field carries the same directives an ordinary argument would — `@true` for the `Bool`, `@arg` plus `@default` for the `Int` — and each becomes a flag on every command that takes an `Options`:

```console
$ ./sift scan -hhh
Search a directory tree for lines containing a pattern

Usage: ./sift <nexus_options> scan <command_options>

General Options:
  -h, --help   Print help; -hh adds details and examples, -hhh adds schemas
               (nexus options: -h @)
  -c, --count  Report the number of matches instead of the matches
  -p, --plain  Print one `path:line:text` record per line

Optional arguments:
      --options <Options_JSON>  How to search
                                type: Options
  -i, --ignore-case             Match without regard to case
                                type: Bool
                                default: false
  -m, --max-count <Int>         Stop after this many hits; 0 means no limit
                                type: Int [default: 0]

Positional arguments:
  1: PATTERN  The text to search for
              type: Str
              format: literal string
  2:          The directory to search
              type: Str
              format: path to a readable file

Return:
  default:    [Hit]
  -c/--count: U64
  -p/--plain: Str    (raw bytes)

Record Schemas:
  Options
    ignoreCase :: Bool
    maxCount   :: Int

  Hit
    path :: Str
    line :: Int
    text :: Str
```

An unrolled record never becomes a positional. It appears in the argument list by name only because `@arg --options` also gives it a flag of its own; without that directive the fields are the only trace of it.

Every named type the help prints is defined once at the bottom, under `Record Schemas:`, when the help is asked for at its third tier (`-hhh`). That is why `Hit` is laid out here too, though it is the return type rather than an argument — the help names it, so the help defines it. A name the help never prints is never defined, which is why a record that is unrolled without `@arg` does not appear.

(The `-c` and `-p` flags and the `Return:` table in that help belong to \`scan’s output actions, which are [Output actions](https://morloc-project.github.io/docs/clis/output-actions.md).)

Without `@unroll`, the record stays whole: it becomes an ordinary positional and the caller supplies a JSON object, a file, or `-` for the entire thing. `@unroll false` on one argument opts that command out while the others stay unrolled:

```morloc
scanAll ::
  --' A file of patterns, one per line
  --' @form list
  [Str] ->
  --' The directory to search
  --' @check.path r
  Str ->
  --' @unroll false
  Options ->
  <IO> [Hit]
```

```console
$ ./sift scanAll -h
...
Positional arguments:
  1:  A file of patterns, one per line
      type: [Str]
      format: path to text file with one string per line
  2:  The directory to search
      type: Str
      format: path to a readable file
  3:  How to search
      type: Options
...
```

## 6.5.1. Three ways to fill it

The `@arg --options` on the record declares a **group flag**, which accepts the whole record at once. It coexists with the per-field flags, so a caller can use either or both.

**The whole record.** The group flag takes a JSON object, a file path, or `-`:

```console
$ echo '{"ignoreCase":true,"maxCount":1}' > opts.json
$ ./sift scan --options opts.json MANUAL notes -p
notes/todo.txt:3:write the manual

$ ./sift scan --options '{"ignoreCase":true}' MANUAL notes -p
notes/todo.txt:3:write the manual
notes/2026/plan.txt:2:ship the manual

$ cat opts.json | ./sift scan --options - MANUAL notes -p
notes/todo.txt:3:write the manual
```

Missing keys fall back to the field’s default, so a partial object is legal and `{}` means "all defaults".

**Field by field.** Each unrolled field has its own flag:

```console
$ ./sift scan MANUAL notes -i -m 1 -p
notes/todo.txt:3:write the manual
```

**A mix.** A partial object fills some fields and individual flags fill or override the rest. The per-field flag always wins:

```console
$ ./sift scan --options '{"ignoreCase":true,"maxCount":9}' -m 1 MANUAL notes -p
notes/todo.txt:3:write the manual
```

The full precedence for each field, highest first:

1.  the per-field flag,
2.  the value in the group bundle, if the key was present,
3.  the field’s declared default,
4.  `null`, for an optional field with neither,
5.  otherwise an error naming the field.

An explicit `null` in the bundle counts as present, so it overrides a default rather than falling through to it.

## 6.5.2. What is rejected

Object form rejects unknown keys, which turns a typo into an error instead of a silently ignored setting:

```console
$ ./sift scan --options '{"ignorecase":true}' MANUAL notes -p
Error: failed to parse argument #2: serialization error: unknown field 'ignorecase' in record bundle
```

A record may also be given positionally, as a JSON array of field values in declaration order. That form has no notion of a missing field, so the length must match exactly:

```console
$ ./sift scan --options '[true,1]' MANUAL notes -p
notes/todo.txt:3:write the manual

$ ./sift scan --options '[true]' MANUAL notes -p
Error: failed to parse argument #2: serialization error: record array form must have exactly 2 fields (one per schema field, in declaration order), got 1
```

A failure while loading one field names the field:

```console
$ ./sift scan -m notanumber manual notes
Error: failed to parse argument #2: field 'maxCount': serialization error: JSON parse error: expected ident at line 1 column 2
```


---

<!-- https://morloc-project.github.io/docs/clis/sum-type-arguments.html -->

# 6.6. Sum type arguments

Morloc Manual > Building CLIs | https://morloc-project.github.io/docs/clis/sum-type-arguments.html | prev: https://morloc-project.github.io/docs/clis/record-arguments.md | next: https://morloc-project.github.io/docs/clis/input-shape.md

A record says "all of these"; a `data` type says "one of these". Unrolled, a `data` argument becomes a set of options that exclude one another, one per constructor. An argument-free constructor is a bare flag. A constructor with fields takes exactly as many values as it has fields, in order.

```morloc
--' A shape to measure
data Shape
  --' a circle of some radius
  = Circle Real
  --' a box, width then height
  | Rect Real Real
  --' a point, with no size
  | Dot

--' Measure a shape
area ::
  --' the shape
  --' @unroll
  Shape ->
  Real
```

```console
$ ./shapes area -h
...
Optional arguments:
      --circle <Real>       a circle of some radius
                            constructor of Shape
      --rect <Real> <Real>  a box, width then height
                            constructor of Shape
      --dot                 a point, with no size
                            constructor of Shape
...
$ ./shapes area --rect 2.0 3.5
7
$ ./shapes area --dot
0
$ ./shapes area --dot --circle 1.0
error: the argument '--dot' cannot be used with '--circle <Real>'
```

Each option is the constructor’s name in lowercase, and its help is the constructor’s docstring. A value inside an arm is read the way an argument of its type would be, so a field of a `data` type takes a bare constructor (`--solid red`) and a field of a record type takes JSON or a file path.

Exactly one arm is required, unless the argument may be omitted — a `?Shape` with no arm given is null — or it declares a default, which may be a bare constructor:

```morloc
--' Count the corners, defaulting to a dot
count ::
  --' the shape
  --' @unroll
  --' @default dot
  Shape ->
  Int
```

To a program the argument is still one value. `--json-help` lists it under the role `alternatives` with its arms, and the MCP tool exposes a single property holding the constructor’s JSON, since a model has no need for the flags.

`@unroll` on a `data` argument does not combine with `@arg`, `@many` or `@stdin`: the constructors are the options, and there is nothing else to name.


---

<!-- https://morloc-project.github.io/docs/clis/input-shape.html -->

# 6.7. Input shape

Morloc Manual > Building CLIs | https://morloc-project.github.io/docs/clis/input-shape.html | prev: https://morloc-project.github.io/docs/clis/sum-type-arguments.md | next: https://morloc-project.github.io/docs/clis/reading-stdin.md

The defaults from [Arguments](https://morloc-project.github.io/docs/clis/arguments.md) cover most arguments: a scalar is read from argv, a compound value is inline JSON or a file. Three directives override that when an argument needs a particular shape.

-   `@source` says where the bytes come from: `inline` (the argv token **is** the value) or `file` (the argv token is a path and the file’s contents are the value).
-   `@form` says how the bytes are read: `list`, `bytes`, `bytes-only`, or `packet`.
-   `@check.<kind>` states an invariant the argument must satisfy before the command runs. The only kind today is `path`.

For a list, the same three exist per element as `@list.source`, `@list.form`, and `@list.check.<kind>`, describing what each **line** of the outer file means.

Which combinations are legal depends on the argument’s wire type. The tables below are the complete set; anything outside them is a compile error, reported against the docstring line that caused it.

**Table 1. Non-Str primitives (numeric values and booleans)**

| Modifier | Effect |
| --- | --- |
| *(none — the only valid case)* | argv is the literal value (`42`, `true`, `3.14`). No modifiers are allowed. |

**Table 2. Str**

| Modifier | Effect |
| --- | --- |
| *(default)* | argv is the string itself, verbatim. |
| `@check.path r` / `w` / `x` / `rw` | argv must be a path satisfying the requested mode. `r` = exists and is readable; `w` = writable (an existing writable file, or a non-existent file in a writable directory); `x` = does not yet exist and the parent directory is writable (exclusive create); `rw` = exists and is both readable and writable. Mutually exclusive with `@source file`. |
| `@source file` | argv is a path; the file’s contents become the string. One trailing newline is stripped, so it behaves like `$(cat file)`. |
| `@stdin` | Makes the positional optional and reads standard input when it is omitted. Implies `@check.path r`. See [Reading a stream from standard input](https://morloc-project.github.io/docs/clis/reading-stdin.md). |

**Table 3. Arrays of fixed-width scalars (\[U8\], \[I32\], \[F64\], \[Bool\], …​)**

| Modifier | Effect |
| --- | --- |
| *(default)* | argv is a JSON array (`[1,2,3]`) or a path to a JSON / MessagePack / packet file. |
| `@form bytes` | argv is a path; the file is checked for a Morloc packet header and otherwise read as packed raw bytes. |
| `@form bytes-only` | argv is a path; the file is packed raw bytes, with no packet check. |
| `@form packet` | argv is a path; the file must be a Morloc packet. |
| `@source inline` + `@form bytes` or `bytes-only` | Only on `[U8]`. argv is the literal byte sequence, one byte per character, with `\xNN`, `\n`, `\t`, `\r`, `\0` and `\\` recognized. |

**Table 4. Any list type (\[T\], including \[Str\] and \[(Int, Str)\])**

| Modifier | Effect |
| --- | --- |
| `@form list` | argv is a file (or `-`) with one element per line, or an inline JSON array. A token whose first byte is `[` is parsed as JSON; anything else is a path. Tuple elements accept a JSON array per line, TSV, or CSV. |
| `@form list` + `@list.source file` | Each line of the outer file is a path to a per-element file, each classified on its own (JSON / MessagePack / packet). |
| `@form list` + `@list.source file` + `@list.form packet` | Each line is a path, and each per-element file must be a Morloc packet. |
| `@form list` + `@list.source file` + `@list.form bytes` (or `bytes-only`) | Each line is a path, and each file is read as packed raw bytes. The element type must be an array of fixed-width scalars. |
| `@form list` + `@list.check.path r` (or `w` / `x` / `rw`) | Each line of the outer file must be a path satisfying the requested mode. The element type must be `Str`. |

**Table 5. Tuples, records, and other non-list compound types**

| Modifier | Effect |
| --- | --- |
| *(default, the only valid case)* | argv is JSON, or a path to a file holding JSON / MessagePack / a packet. No outer modifiers are allowed. |

## 6.7.1. A worked example

``sift’s `scanAll`` uses two of these. The pattern list is a file with one pattern per line, and the search root must be a directory that exists:

```morloc
--' Search for any of several patterns, one per line of a file
scanAll ::
  --' A file of patterns, one per line
  --' @form list
  [Str] ->
  --' The directory to search
  --' @check.path r
  Str ->
  Options ->
  <IO> [Hit]
```

A shaped argument gets a `format:` line in the help saying what it will accept. So does every `Str` argument, shaped or not: `Str` is the one type where argv is genuinely ambiguous, and stating the default reading is cheaper than making a reader infer it from the absence of a line.

```console
$ ./sift scanAll -h
...
Positional arguments:
  1:  A file of patterns, one per line
      type: [Str]
      format: path to text file with one string per line
  2:  The directory to search
      type: Str
      format: path to a readable file
...
```

```console
$ printf 'milk\nrest\n' > patterns.txt
$ ./sift scanAll patterns.txt notes -p
notes/todo.txt:1:buy milk
notes/2026/plan.txt:3:rest
```

`@form list` still accepts inline JSON, so the same command works without a file:

```console
$ ./sift scanAll '["milk","rest"]' notes -c
2
```

A failing `@check` is reported before the command runs, naming the check that failed:

```console
$ ./sift scan the nosuchdir
Error: argument #1: check.path: r requires path 'nosuchdir' to exist and be readable
```

## 6.7.2. Shape follows the wire form

Shape is classified against an argument’s **wire form**, not its source-level type name. A type declared with `Packable [(a, b)] T` crosses the language boundary as a list of pairs, so the CLI treats it as `[(a, b)]` and every list modifier is available.

`Map a b` from the standard library is the case you are most likely to meet. Its wire form is `[(a, b)]`, so a `Map Str Int` argument reads a two-column TSV or CSV exactly as `[(Str, Int)]` would:

**tally.loc**

```morloc
module tally (tally)

import root-py
import map-py

--' Count the entries of a two-column table read as a Map
tally ::
  --' A file with one `key<TAB>value` pair per line
  --' @form list
  Map Str Int -> U64
tally m = size m
```

```console
$ printf 'apple\t3\nbanana\t7\ncherry\t1\n' > counts.tsv
$ ./tally counts.tsv
3
```

Commas work as well as tabs, and a JSON array per line is the fallback:

```console
$ printf 'apple,3\nbanana,7\n' > counts.csv
$ ./tally counts.csv
2

$ printf '["apple",3]\n["banana",7]\n' > counts.jsonl
$ ./tally counts.jsonl
2
```

> **Note**
> **`@form list` is headerless.** A delimited file read this way is parsed row by row with no header, because tuples have no column names — a header row would be read as data and fail the schema check on the first field. There is no option to skip one:
> 
> ```console
> $ printf 'name\tcount\napple\t3\n' > hdr.tsv
> $ ./tally hdr.tsv
> Error: failed to parse argument #0: serialization error: JSON parse error: expected value at line 1 column 10
>   hint: the first row (`name	count`) looks like a column-name header. `form: list` is headerless -- remove the header row, or declare the argument as `Table` if you need column names.
> ```
> 
> If you need column names, declare the argument as a `Table`. The `Table` loader honors headers and aligns columns by name; `@form list` is for streams of rows.


---

<!-- https://morloc-project.github.io/docs/clis/reading-stdin.html -->

# 6.8. Reading a stream from standard input

Morloc Manual > Building CLIs | https://morloc-project.github.io/docs/clis/reading-stdin.html | prev: https://morloc-project.github.io/docs/clis/input-shape.md | next: https://morloc-project.github.io/docs/clis/output-formats.md

A tool that reads standard input when you do not give it a file is the shape that makes pipelines work. `@stdin` declares that shape.

Marking a `Str` positional `@stdin` makes it optional. When the caller supplies a path, the argument is that path; when the caller omits it, or writes `-`, the argument becomes standard input. The command opens it with `@open` and reads from the handle, so nothing in the body cares which of the two happened.

``sift’s `total`` adds up a stream of per-file counts:

```morloc
--' Add up a stream of per-file counts
total ::
  --' A file of counts; standard input when omitted
  --' @stdin
  Str ->
  <IO> Int
total f = do
  Ok s <- @open f :: <IO> (Try Str (IStream (Str, Int)))
  Ok counts <- @next s
  totalPy counts
```

`@open` needs to know what it is opening; the `::` annotation says the handle is an `IStream` of `(Str, Int)` pairs, which is what `summarize` produces. `@next` pulls the next batch off the stream.

All three call shapes give the same answer:

```console
$ ./sift -f packet summarize hits.json > counts.pkt
$ ./sift total counts.pkt
4

$ ./sift -f packet summarize hits.json | ./sift total
4

$ ./sift -f packet summarize hits.json | ./sift total -
4
```

`-f packet` is what makes the middle form work. It is Morloc’s own framing: the bytes carry the value’s schema, so the reader checks that what arrived is what it asked for instead of trusting the pipeline.

## 6.8.1. What standard input may carry

Morloc packets, and nothing else. A foreign format is refused rather than guessed at:

```console
$ printf 'this is definitely not a morloc packet, just plain text bytes\n' | ./sift total
Error: run failed
...
@next: stdin is not a morloc packet; expected a morloc data or stream packet. Foreign formats (JSON, MessagePack, CSV, ...) are not supported on stdin. A morloc program writes packets only when asked: add `-f packet` to the command on the writing end of this pipe.
```

Empty input is not an error — it is an empty batch, which is the right answer for a search that found nothing:

```console
$ printf '' | ./sift total
0
```

This is narrower than the `-` of [Arguments](https://morloc-project.github.io/docs/clis/arguments.md), which accepts JSON and MessagePack too. The difference is that `-` reads one **value** off stdin, while `@stdin` opens stdin as a **stream** that the command drains itself.

## 6.8.2. Rules for `@stdin`

At most one positional per command may read stdin, and it must be the last one. Both are compile errors. Given

**two.loc**

```morloc
module two (f)

import root-py

--' Two stdin arguments
f ::
  --' @stdin
  Str ->
  --' @stdin
  Str ->
  <IO> ()
f _ _ = @throw "unused"
```

**nl.loc**

```morloc
module nl (f)

import root-py

--' A stdin argument that is not last
f ::
  --' @stdin
  Str ->
  Int ->
  <IO> ()
f _ _ = @throw "unused"
```

```console
$ morloc make -o two two.loc
In two:f, more than one positional declares `@stdin`; at most one argument may read from stdin.

$ morloc make -o nl nl.loc
In nl:f, a positional follows the `@stdin` positional; the stdin argument must be the last positional.
```

`@stdin` implies `@check.path r` and cannot be combined with `@arg`, `@default`, or `@many`.

A handler may also open the argument as an `IFile`, which gives random access and a footer count instead of a sequential read. That works on a real file and fails on a pipe, so the usual idiom is to `match` on the `IFile` attempt and fall back to `IStream` in the `Err` arm. See [Random access and streaming](https://morloc-project.github.io/docs/runs/random-access-and-streaming.md).

> **Warning**
> Two limits are worth knowing before you design around `@stdin`.
> 
> It applies only to `Str`. An argument declared `[Str]` with `@form list` — the line-oriented filter shape — is rejected, so a `wc`\-style tool has to be called with an explicit `-` (`reports/0030`).
> 
> A stream whose element type has a **name** — a type alias, or a record — is accepted from a file and rejected from stdin, because only the stdin path compares the concrete schema name (`reports/0031`). That is why `total` opens an `IStream (Str, Int)` — an alias for the pair, or a record in its place, would be rejected on the pipe and accepted from the file.
> 
> Neither shows up in `-h`, which still prints the argument as though it were required.


---

<!-- https://morloc-project.github.io/docs/clis/output-formats.html -->

# 6.9. Output formats

Morloc Manual > Building CLIs | https://morloc-project.github.io/docs/clis/output-formats.html | prev: https://morloc-project.github.io/docs/clis/reading-stdin.md | next: https://morloc-project.github.io/docs/clis/output-actions.md

A command’s return value is serialized and written to standard output. The default form is JSON:

```console
$ ./sift summarize hits.json
[["notes\/todo.txt",2],["notes\/2026\/plan.txt",2]]
```

`/` is escaped as `\/`, which JSON permits and which some encoders do. It is the same string either way.

The nexus option `-f` picks a different form. Which forms are available does not depend on the program — serialization is the runtime’s job, not the tool’s, so every Morloc command can emit every form its type supports:

| Form | Notes |
| --- | --- |
| `json` | The default. Human-readable, lossy on integer width. |
| `jsonl` | One element per line. Meaningful for list-shaped results; a scalar is one line. |
| `mpk` | MessagePack. Compact, exact. |
| `voidstar` | Morloc’s in-memory binary form, written out. Carries the value’s schema. |
| `packet` | A Morloc wire packet: the value plus its schema and framing. This is what `@stdin` readers expect, and what `-z` compresses. |
| `arrow`, `parquet` | Apache Arrow IPC and Parquet. Requires a `Table` return type. |
| `csv` | Requires a `Table` return type. |

Because the reader detects the format from the bytes, a value written in one form is read back without being told which:

```console
$ ./sift -f mpk scan the notes > hits.mpk
$ ./sift summarize hits.mpk
[["notes\/todo.txt",2],["notes\/2026\/plan.txt",2]]
```

`-f jsonl` is the form to reach for when the next thing in the pipeline is a line-oriented Unix tool:

```console
$ ./sift -f jsonl scan the notes
{"path":"notes\/todo.txt","line":2,"text":"fix the parser"}
{"path":"notes\/todo.txt","line":3,"text":"write the manual"}
{"path":"notes\/2026\/plan.txt","line":1,"text":"fix the build"}
{"path":"notes\/2026\/plan.txt","line":2,"text":"ship the manual"}
```

Asking for a form the type cannot produce is an error, not a silent approximation:

```console
$ ./sift -f csv summarize hits.json
Error: --format=arrow|parquet|csv requires a Table return type
```

Two more nexus options shape the output. `-o` writes to a file instead of stdout. `-p` pretty-prints: JSON gets indentation, and a top-level `Str` is printed as text rather than as a quoted JSON string.

```console
$ ./sift -p summarize hits.json
[
  [
    "notes\/todo.txt",
    2
  ],
  [
    "notes\/2026\/plan.txt",
    2
  ]
]
```

`-z` compresses `-f packet` output; it is covered with the rest of the compression settings in [Compression](https://morloc-project.github.io/docs/runs/compression.md).

## 6.9.1. Nothing to report

A command that returns `()` or a top-level `Null` prints nothing at all. That matches the Unix convention that a tool with no result says nothing, and it is what you want when a Morloc command feeds `grep`, `xargs`, or a status check — a `()` carries no information, and a top-level `None` usually means "it ran and there was nothing to say".

A small program with an optional result, to show it with:

**nulls.py**

```python
def lookup(key, table):
    return dict(table).get(key)

def pair():
    return [5, None]
```

**nulls.loc**

```morloc
module nulls (lookupKey, pair)

import root-py
import map-py

source Py from "nulls.py" ("lookup" as lookupKey, "pair")

--' Look up a key, or nothing
lookupKey :: Str -> Map Str Str -> ?Str

--' A pair whose second element is Unit
pair :: (Int, ())
```

```console
$ ./nulls lookupKey zz '[["a","1"],["b","2"]]'
$ echo $?
0
```

When the distinction matters — a downstream consumer that needs `null` to mean "a null result" as against an empty file meaning "the process died" — pass `--keep-null`:

```console
$ ./nulls --keep-null lookupKey zz '[["a","1"],["b","2"]]'
null
```

Suppression is a JSON-only convenience. The binary forms always write a well-formed nil, so a reader sees the bytes it expects:

```console
$ ./nulls -f mpk lookupKey zz '[["a","1"]]' | od -An -tx1
 c0
```

A `null` **inside** a value is never suppressed — the shape carries information the consumer needs:

```console
$ ./nulls pair
[5,null]
```

## 6.9.2. Failure

Errors go to standard error and the process exits non-zero, so a Morloc command behaves in a `set -e` script or a `&&` chain the way any other tool does:

```console
$ ./sift summarize nosuch.json
Error: failed to parse argument #0: file 'nosuch.json' not found
$ echo $?
1
```

Errors raised inside a pool name the function and the source position that raised them:

```console
$ ./sift total < /dev/urandom
Error: run failed
...
@next: stdin is not a morloc packet; expected a morloc data or stream packet. Foreign formats (JSON, MessagePack, CSV, ...) are not supported on stdin. A morloc program writes packets only when asked: add `-f packet` to the command on the writing end of this pipe.
  at total [py] (mid=4, sift.loc:2:40)
```


---

<!-- https://morloc-project.github.io/docs/clis/output-actions.html -->

# 6.10. Output actions

Morloc Manual > Building CLIs | https://morloc-project.github.io/docs/clis/output-actions.html | prev: https://morloc-project.github.io/docs/clis/output-formats.md | next: https://morloc-project.github.io/docs/clis/streaming-output.md

One typed function usually wants more than one presentation. `sift scan` returns `[Hit]`, which is what a downstream program should get, and not at all what a person at a terminal wants to read. The obvious fix — export a second command that formats the first one’s output — doubles the module’s surface and puts presentation into the library.

An **output action** attaches a formatter to a command as a flag. The command keeps its type; the flag routes the result through a named term on the way out.

```morloc
--' Print one `path:line:text` record per line
asLines :: [Hit] -> Str

--' Report the number of matches instead of the matches
countHits :: [Hit] -> U64
countHits = size

--' Search a directory tree for lines containing a pattern
--' @with   -c/--count=countHits
--' @render -p/--plain=asLines
scan :: ...
```

The directive names the flag and the term: `@with -c/--count=countHits` means "the flag `-c` or `--count` routes the result through \`countHits\`". The term’s own docstring becomes the flag’s help text, which is why it is worth giving formatters docstrings even when they are one-liners:

```console
$ ./sift scan -h
...
General Options:
  -h, --help   Print help; -hh adds details and examples, -hhh adds schemas
               (nexus options: -h @)
  -c, --count  Report the number of matches instead of the matches
  -p, --plain  Print one `path:line:text` record per line
...
```

```console
$ ./sift scan the notes -c
4
```

A formatter is an ordinary typechecked function `A → B` where `A` unifies with the command’s return payload. `countHits` is `[Hit] → U64`, so the composed command returns a `U64`. `size` from the standard library would have done as well; `countHits` exists only to carry the docstring.

Because each flag produces a different type, `-h` reports the return as a table instead of a single line:

```console
Return:
  default:    [Hit]
  -c/--count: U64
  -p/--plain: Str
...
```

The command’s declared return type is unchanged. Morloc code that composes `scan` still sees `<IO> [Hit]`; the actions exist only at the interface.

## 6.10.1. `@with` keeps a value; `@render` produces bytes

The two directives differ in what they do with the formatter’s result.

`@render` treats the result as the **final bytes**. They are written verbatim, without quoting or escaping, and `-f` no longer applies. That is how `sift` declares `-p`, and it is why the output is readable:

```console
$ ./sift scan the notes -p
notes/todo.txt:2:fix the parser
notes/todo.txt:3:write the manual
notes/2026/plan.txt:1:fix the build
notes/2026/plan.txt:2:ship the manual
```

`@with` keeps the result **typed**. It flows through `-f` like any other return value, so the same `Str` comes out as a JSON string. Change the one word in `sift.loc` — 

```morloc
--' @with   -p/--plain=asLines
```

 — rebuild, and the same command gives you this instead:

```console
$ ./sift scan the notes -p
"notes\/todo.txt:2:fix the parser\nnotes\/todo.txt:3:write the manual\nnotes\/2026\/plan.txt:1:fix the build\nnotes\/2026\/plan.txt:2:ship the manual\n"
```

A `@render` handler must return `Str` or `[U8]`. Use `@with` when the result is data for something else to read, and `@render` when it is text or bytes for a human or a file.

A third case falls out of the same rule: a formatter that returns `()` is a **sink**. It has already done the writing itself — printed, saved a file, sent a request — and nothing goes on the wire. `Unit` in the `Return:` table marks one.

The table marks the framing too: `(raw bytes)` after a type means the row is a `@render` action, so those bytes go out as they are and `-f` does not apply to them.

## 6.10.2. Giving a formatter arguments

A formatter may take arguments besides the value it formats. `$1`, `$2`, …​ refer to the command’s own arguments by position, and `@value` refers to the value being formatted. Write them as a call:

**rep.py**

```python
def query(q):
    return [[q + "-" + str(i), i] for i in range(3)]

def tabulate(width, rows):
    for name, n in rows:
        print("%-*s%d" % (width, name, n))

def as_json(rows):
    import json
    return json.dumps(rows)
```

**rep.loc**

```morloc
module rep (report)

import root-py

source Py from "rep.py" ("query", "tabulate", "as_json" as asJson)

query :: Str -> <IO> [(Str, Int)]

--' Print the rows as a table, `width` columns wide
tabulate :: Int -> [(Str, Int)] -> <IO> ()

--' Render the rows as JSON text
asJson :: [(Str, Int)] -> Str

--' Run a query
--' @render -t/--table=tabulate($2) @default
--' @with   -j/--json=asJson
report ::
  --' the query to run
  Str ->
  --' output width
  Int ->
  <IO> [(Str, Int)]
report q _ = query q
```

The `@default` on `-t` is the subject of the next subsection; ignore it for the moment and pass the flag explicitly.

`$2` passes ``report’s second argument — the width — into `tabulate``:

```console
$ ./rep fruit 12 -t
fruit-0     0
fruit-1     1
fruit-2     2

$ ./rep fruit 20 -t
fruit-0             0
fruit-1             1
fruit-2             2
```

The value being formatted is appended last unless you place it yourself, so `tabulate($2)` applies `tabulate width rows`, while `tabulate(@value, $2)` would apply `tabulate rows width`.

## 6.10.3. Choosing a default

Mark one action `@default` and it fires when no action flag and no `-f` are given. That is how a command gets human-readable output by default while keeping its typed output one flag away:

```morloc
--' Run a query
--' @render -t/--table=tabulate($2) @default
--' @with   -j/--json=asJson
report :: ...
```

```console
$ ./rep fruit 12
fruit-0     0
fruit-1     1
fruit-2     2

$ ./rep fruit 12 -j
"[[\"fruit-0\", 0], [\"fruit-1\", 1], [\"fruit-2\", 2]]"

$ ./rep -f json @ fruit 12
[["fruit-0",0],["fruit-1",1],["fruit-2",2]]
```

An explicit `-f` suppresses the default, which is what makes the typed output reachable again. At most one action per command may be `@default`.

## 6.10.4. Media types

Bytes carry no label. A PNG and a CSV are both `[U8]` as far as the type system is concerned, and a caller that receives one has no way to tell which. `@mime` attaches a media type (RFC 6838 `type/subtype`) to a **type**, once:

**ramp.py**

```python
import struct
import zlib

def make_png(n):
    def chunk(typ, data):
        c = typ + data
        return (struct.pack(">I", len(data)) + c
                + struct.pack(">I", zlib.crc32(c) & 0xffffffff))
    raw = b""
    for _ in range(n):
        raw += b"\x00" + bytes([(x * 255) // max(n - 1, 1) for x in range(n)])
    png = b"\x89PNG\r\n\x1a\n"
    png += chunk(b"IHDR", struct.pack(">IIBBBBB", n, n, 8, 0, 0, 0, 0))
    png += chunk(b"IDAT", zlib.compress(raw))
    png += chunk(b"IEND", b"")
    return list(png)

def ident(bs):
    return bs
```

**ramp.loc**

```morloc
module ramp (ramp)

import root-py

--' A PNG image
--' @mime image/png
type PNG = [U8]

source Py from "ramp.py" ("make_png", "ident")

make_png :: Int -> PNG

--' Write the image bytes to standard output
ident :: PNG -> PNG

--' Draw an n-by-n grayscale ramp
--' @render -w/--write=ident
ramp :: Int -> PNG
ramp = make_png
```

Declaring it on the type rather than on each use means it cannot disagree between uses; a conflicting `@mime` along an alias chain is a compile error. The label replaces the type name wherever the type surfaces:

```console
$ ./ramp -h
Draw an n-by-n grayscale ramp
...
Return:
  default:    image/png
  -w/--write: image/png    (raw bytes)
  A PNG image
```

It does more than label. The HTTP daemon returns the raw bytes with a matching `Content-Type` instead of a JSON envelope, and the MCP server delivers them as an inline image block rather than an array of numbers — see [Building API interfaces](https://morloc-project.github.io/docs/apis/api-interfaces.md) and [Model Context Protocol (MCP)](https://morloc-project.github.io/docs/apis/mcp.md). A media-typed return must reduce to `Str` or a byte array (`[U8]`), or a list of either; anything else is rejected at compile time.

> **Note**
> `@mime` does not yet change what the CLI writes. Without an action flag the bytes still come out as a JSON array of numbers, so a `@render` sink is currently how you get a file (`reports/0029`):
> 
> ```console
> $ ./ramp 4 | head -c 60
> [137,80,78,71,13,10,26,10,0,0,0,13,73,72,68,82,0,0,0,4,0,0,0
> 
> $ ./ramp 4 -w > out.png
> $ file out.png
> out.png: PNG image data, 4 x 4, 8-bit grayscale, non-interlaced
> ```
> 
> The `ident` handler above exists for exactly this reason.

## 6.10.5. Rules and rejections

-   One action flag per invocation. Siblings are mutually exclusive and a second one is rejected at parse time.
-   Action directives belong in the signature preamble — the `--'` lines directly above `name ::` — not on argument docstrings, record fields, or type aliases.
-   The command needs an explicit signature.
-   Flag names must not collide with the command’s own `@arg` / `@true` / `@false` names, with each other, or with `-h` / `--help`.
-   Two directives whose long flags collapse to the same internal name (say `--bar-baz` and `--bar_baz`) are rejected, as is a synthesized entry name that collides with a top-level identifier in the module.


---

<!-- https://morloc-project.github.io/docs/clis/streaming-output.html -->

# 6.11. Streaming output with `@collect`

Morloc Manual > Building CLIs | https://morloc-project.github.io/docs/clis/streaming-output.html | prev: https://morloc-project.github.io/docs/clis/output-actions.md | next: https://morloc-project.github.io/docs/clis/composing-tools.md

Everything so far has treated a command’s output as one value: compute it, serialize it, write it. That breaks down when the result is larger than memory, or when the caller wants to see the first rows before the last ones exist.

A streaming command returns `()` and hands its data out in batches. The `@collect` intrinsic drives it:

```morloc
@collect :: (([a] -> <IO, e> ()) -> <IO, e> ()) -> <IO, e> ()
```

`@collect` takes a **producer**: a function that is given a sink and calls it once per batch. `@collect` supplies the sink, manages the stream, and writes each batch out in whatever form `-f` selects. The element type rides on the sink, so the compiler knows the stream’s type with no extra annotation.

``sift’s `stream`` searches the same tree as `scan` but emits one batch per file:

```morloc
producePy  :: Str -> Str -> Options -> ([Hit] -> <IO> ()) -> <IO> ()

--' Stream hits to standard output, one file at a time
stream ::
  Str ->
  --' @check.path r
  Str ->
  Options ->
  <IO> ()
stream pat root opts = @collect (producePy pat root opts)
```

The producer here is Python, and its last parameter is the sink:

```python
def produce(pattern, root, opts, sink):
    fold = opts["ignoreCase"]
    needles = [pattern.lower() if fold else pattern]
    for path in walk_files(root):
        sink(hits_in(path, needles, fold))
```

A morloc callback crossing into a Python function is an ordinary foreign call; nothing about the streaming machinery is visible from either side.

With no action flag, every batch goes to standard output in the `-f` form:

```console
$ ./sift -f jsonl stream the notes
{"path":"notes\/todo.txt","line":2,"text":"fix the parser"}
{"path":"notes\/todo.txt","line":3,"text":"write the manual"}
{"path":"notes\/2026\/plan.txt","line":1,"text":"fix the build"}
{"path":"notes\/2026\/plan.txt","line":2,"text":"ship the manual"}
```

## 6.11.1. Actions on a stream

The output actions of [Output actions](https://morloc-project.github.io/docs/clis/output-actions.md) work here too, with one extra dimension. On an ordinary command a formatter sees the return value; on a streaming command it can see either the whole gathered stream or each batch as it arrives. The `@stream` modifier chooses:

| Directive | Handler type | Behavior |
| --- | --- | --- |
| `@with` | `[a] → b` (or `IFile [a] → b`) | Gather the whole stream, apply once; `b` is serialized by `-f`. |
| `@with …​ @stream` | `[a] → [b]` | Apply to each batch as it arrives, at constant memory; the `b` elements are serialized by `-f`. |
| `@render` | `[a] → Str` / `[a] → [U8]` (or `IFile [a] → …​`) | Gather the whole stream, apply once, write the bytes verbatim. |
| `@render …​ @stream` | `[a] → Str` / `[a] → [U8]` | Apply to each batch, write each result’s bytes verbatim. |

``sift’s `stream`` declares one action from three of those cells:

```morloc
--' Stream hits to standard output, one file at a time
--' @render -p/--plain=asLines @stream
--' @with   -c/--count=countHits
--' @with   -n/--staged=countStaged
--' @with   -N/--numbered=numberHits(@offset) @stream
stream :: ...
```

`-p` renders each batch to text as it goes, which is the constant-memory version of what `scan -p` does:

```console
$ ./sift stream the notes -p
notes/todo.txt:2:fix the parser
notes/todo.txt:3:write the manual
notes/2026/plan.txt:1:fix the build
notes/2026/plan.txt:2:ship the manual
```

`-c` is the other extreme: gather everything and apply `countHits` once.

```console
$ ./sift stream the notes -c
4
```

## 6.11.2. `@offset`: where a batch sits in the stream

A `@stream` handler is called once per batch and has no memory between calls, so anything that depends on position has to be told. `@offset` supplies the number of elements already written:

```morloc
--' Number the hits as they stream past
numberHits :: U64 -> [Hit] -> [Str]
```

```morloc
--' @with   -N/--numbered=numberHits(@offset) @stream
```

`numberHits(@offset)` passes the offset as the handler’s first argument. Each file is a separate batch, and the numbering runs across them:

```console
$ ./sift -f jsonl stream the notes -N
"1 notes\/todo.txt:2"
"2 notes\/todo.txt:3"
"3 notes\/2026\/plan.txt:1"
"4 notes\/2026\/plan.txt:2"
```

`@offset` is only meaningful under `@stream`; using it elsewhere is an error.

## 6.11.3. `IFile`: the gathered stream as a file

A whole-stream handler may take its receiver as `IFile [a]` instead of `[a]`. The stream is staged to a temporary file and the handler gets a random-access handle rather than a materialized list — the way to write a whole-stream handler that does not need the whole stream in memory. The temporary file is removed when the handler returns.

`countStaged` reads the element count out of the staged file’s footer without touching the data:

```morloc
--' Count the hits without loading them into memory
countStaged :: IFile [Hit] -> <IO> Int
countStaged f = do
  Ok n <- @flen f
  n
```

```console
$ ./sift stream the notes -n
4
```

`IFile` and the rest of the random-access handles are covered in [Random access and streaming](https://morloc-project.github.io/docs/runs/random-access-and-streaming.md).

## 6.11.4. Streaming rules

The rules in [Rules and rejections](https://morloc-project.github.io/docs/clis/output-actions.md#action-rules) all apply. Two more are specific to streaming:

-   A `@stream` handler must return a list; its elements are what reach the wire.
-   `@render` under `@stream` writes each batch’s bytes as they are produced, with nothing added between batches — no separator, no trailing newline beyond what the handler itself emits.

## 6.11.5. What the help says a stream produces

A streaming command’s `Return:` block describes **standard output**, not the `()` the function returns. The two coincide for every other command and come apart here, so the block is worth reading closely:

```console
$ ./sift stream -h
...
Return:
  default:       [Hit]
  -p/--plain:    Str    (raw bytes)
  -c/--count:    U64
  -n/--staged:   Int
  -N/--numbered: [Str]
...
```

Five rows, five different things on stdout. `default` is the batch element the sink writes; `-c` and `-n` gather and return a single value; `-N` transforms each batch and streams the result; and `(raw bytes)` marks the one row where `-f` no longer applies, because a `@render` action writes its handler’s bytes verbatim.

The compiler works the element type out from the producer’s signature: a `@collect` argument takes exactly one parameter, the sink, so the sink is the last parameter of the producer’s declared type and the sink’s own parameter is what reaches standard output. A producer with no reachable signature — an inline lambda — leaves the row falling back to the return type; the help never claims `()` for a command that streams.


---

<!-- https://morloc-project.github.io/docs/clis/composing-tools.html -->

# 6.12. Composing tools

Morloc Manual > Building CLIs | https://morloc-project.github.io/docs/clis/composing-tools.html | prev: https://morloc-project.github.io/docs/clis/streaming-output.md | next: https://morloc-project.github.io/docs/clis/interface-as-data.md

A module that compiles to a CLI is still a module. Nothing about being a command line tool stops another module from importing it, so a toolbox is a module that imports and re-exports.

Here is a second module, unrelated to `sift` and written in R, that turns label-count pairs into a bar chart:

**stats.R**

```r
histogram <- function(counts){
  paste0(
    sapply(counts, function(row){
      sprintf("%-24s %s", row[[1]], strrep("#", as.integer(row[[2]])))
    }),
    collapse = "\n"
  )
}
```

**stats.loc**

```morloc
module stats (histogram)

import root-r

source R from "stats.R" ("histogram" as histogramR)

histogramR :: [(Str, Int)] -> Str

--' Draw a bar for each label
histogram :: [(Str, Int)] -> Str
histogram = histogramR
```

A toolbox picks what it wants from each:

**tools.loc**

```morloc
--' A little toolbox for reading notes
module tools (scan, summarize, histogram)

import .sift
import .stats
```

The leading `.` marks a local file rather than an installed module. Both modules here are files you built a moment ago, so both take it. A toolbox assembled from modules you installed names them without the dot — `import sift` — and is otherwise identical; that is the more common shape, and the only reason this chapter uses local files is so you can run it without installing anything first.

That is the whole toolbox: two imports and an export list, no glue code:

```console
$ morloc make -o tools tools.loc
$ ./tools -h
A little toolbox for reading notes

Usage: ./tools <nexus_options> <command> <command_options>

Commands:
  scan       Search a directory tree for lines containing a pattern
  summarize  Count the hits in each file
  histogram  Draw a bar for each label

General Options:
  -h, --help  Print help; -hh adds details and examples, -hhh adds schemas
              (nexus options: -h @)
```

Two of those commands run in Python and one in R. The pools start on demand, so a run that only touches `scan` never starts the R interpreter, and a pipeline that touches all three starts each once:

```console
$ ./tools scan the notes | ./tools summarize - | ./tools -p histogram -
notes/todo.txt           ##
notes/2026/plan.txt      ##
```

That pipeline is worth a second look. Three processes, two language runtimes, and no agreement between the stages about a file format: `scan` writes a `[Hit]`, `summarize` reads a `[Hit]` and writes a `[(Str, Int)]`, `histogram` reads a `[(Str, Int)]`. Each side knows the type, so each side knows how to read what arrived. Adding a stage means writing a function with the right type, not a parser.

Subtraction works the same way. A toolbox that lists three of \`sift’s five exports is a tool with three commands; nothing of the other two is compiled in. There is no plugin mechanism here because none is needed — the export list is the mechanism.

## 6.12.1. Grouping commands

A toolbox grows, and a flat list of twenty commands is a bad interface. Group them with `--*` annotations in the export list:

**tools.loc**

```morloc
--' A little toolbox for reading notes
module tools
  --* group: find
  --* Search the filesystem
  ( scan
  , scanAll
  --* group: report
  --* Turn hits into something readable
  , summarize
  , histogram
  )

import .sift
import .stats
```

A `--* group: <name>` line opens a group, and the `--*` lines after it are its description. Every export listed below it belongs to that group, until the next group line. Each group becomes a subcommand of its own:

```console
$ ./tools -h
A little toolbox for reading notes

Usage: ./tools <nexus_options> <command> <command_options>

Commands:
  find    Search the filesystem
  report  Turn hits into something readable

General Options:
  -h, --help  Print help; -hh adds details and examples, -hhh adds schemas
              (nexus options: -h @)

$ ./tools find -h
Search the filesystem

Usage: ./tools <nexus_options> find <command> <command_options>

Commands:
  scan     Search a directory tree for lines containing a pattern
  scanAll  Search for any of several patterns, one per line of a file

General Options:
  -h, --help  Print help; -hh adds details and examples, -hhh adds schemas
              (nexus options: -h @)
```

and the group name joins the invocation:

```console
$ ./tools find scan the notes | ./tools report summarize -
[["notes\/todo.txt",2],["notes\/2026\/plan.txt",2]]
```

Grouping is optional per export. Write `--* group:` with no name to close the current group; exports after it are ungrouped and appear at the top level alongside the groups.

## 6.12.2. Installing

`morloc make --install` puts the built program on your `PATH` instead of leaving it in the current directory:

```console
$ morloc make --install -o sift sift.loc
Installed 'sift' to /opt/morloc/bin/sift   # your MORLOC_HOME will differ
```

`morloc list` shows what is installed — modules first, then programs:

```console
$ morloc list
Modules:
  root 0.7.0  Define type signatures for common functions
...
Programs:
  sift  5 commands
...
```

Add `-v` to list each program’s commands with their return types.

Installing also regenerates shell completion for every installed program, into `$MORLOC_HOME/completions/`. The completions are derived from the same manifest the help is — command names, group names, and each command’s flags — so they cover the groups of the previous section without any extra declaration:

```console
$ sed -n '/Installed program: sift/,+9p' $MORLOC_HOME/completions/morloc-completions.bash | tail -2
    COMPREPLY=($(compgen -W "scan scanAll summarize total stream" -- "$cur"))
    return
```

Source the one for your shell from your shell’s startup file:

```console
$ source $MORLOC_HOME/completions/morloc-completions.bash   # bash
$ source $MORLOC_HOME/completions/_morloc_completions       # zsh
```

The entry points the compiler synthesizes for each output action are marked internal in the manifest, so they are absent from the count and from the completions — the surface you see is the surface the program accepts.


---

<!-- https://morloc-project.github.io/docs/clis/interface-as-data.html -->

# 6.13. The interface as data

Morloc Manual > Building CLIs | https://morloc-project.github.io/docs/clis/interface-as-data.html | prev: https://morloc-project.github.io/docs/clis/composing-tools.md | next: https://morloc-project.github.io/docs/clis/directive-reference.md

`-h` is written for a person. `--json-help` is the same information written for a program: a complete, machine-readable description of every command, its arguments, their types, and what it returns.

```console
$ ./sift --json-help
```

The top of the document records the compiler version, names the program, and lists its command groups:

```json
{
...
  "program": {
    "name": "sift",
    "description": [
      "Search notes and count what turns up"
    ]
  },
  "groups": []
}
```

Each command follows. `summarize` is the simplest one in `sift`:

```json
{
  "name": "summarize",
  "kind": "remote",
  "group": null,
  "description": [
    "Count the hits in each file"
  ],
  "arguments": [
    {
      "name": "arg0",
      "role": "positional",
      "position": 0,
      "metavar": null,
      "required": true,
      "variadic": false,
      "stdin": false,
      "quoted": false,
      "default": null,
      "description": [
        "Hits produced by an earlier search"
      ],
      "type": {
        "morloc": "[Hit]",
        "wire": "am34paths4linej4texts",
        "structure": {
          "type": "array",
          "items": {
            "type": "object",
            "properties": {
              "path": { "type": "string" },
              "line": { "type": "integer" },
              "text": { "type": "string" }
            },
            "required": ["path", "line", "text"],
            "additionalProperties": false
          }
        }
      },
      "named_type_kind": null,
      "input": {
        "source": "auto",
        "form": "auto",
        "checks": [],
        "list_source": "inline",
        "list_form": "auto",
        "list_checks": [],
        "format": null
      }
    }
  ],
  "return": {
    "description": [],
    "streaming": false,
    "type": {
      "morloc": "[(Str, Int)]",
      "wire": "at2sj",
      "structure": {
        "type": "array",
        "items": {
          "type": "array",
          "prefixItems": [
            { "type": "string" },
            { "type": "integer" }
          ],
          "minItems": 2,
          "maxItems": 2
        }
      }
    }
  },
  "terminals": []
}
```

Every type appears three ways, because three different readers want it: `morloc` is the type as written, `wire` is the serialization schema, and `structure` is JSON Schema, which a validator or a form generator can consume directly. The `wire` schema is the **general** one: the concrete schema the runtime dispatches on names the container the pool’s language builds, which would make the published contract move whenever the implementation language did. The `input` block carries the shape directives of [Input shape](https://morloc-project.github.io/docs/clis/input-shape.md), so a caller can tell that an argument wants a path rather than a value without parsing prose.

`return.streaming` says whether the command writes a stream to standard output rather than returning a value; when it does, `return.type` describes the batch that reaches stdout rather than the `()` the function returns.

`terminals` lists a command’s output actions, each with the type its flag puts on the wire. `scan` has two:

```json
[
  {
    "short": "c",
    "long": "count",
    "description": "Report the number of matches instead of the matches",
    "render": false,
    "default": false,
    "type": {
      "morloc": "U64",
      "wire": "u8",
      "structure": {
        "type": "integer"
      }
    }
  },
  {
    "short": "p",
    "long": "plain",
    "description": "Print one `path:line:text` record per line",
    "render": true,
    "default": false,
    "type": {
      "morloc": "Str",
      "wire": "s",
      "structure": {
        "type": "string"
      }
    }
  }
]
```

None of this is written by hand or kept in a sidecar file. It is derived from the same types and docstrings as the help text, on the same build, which is what makes it worth trusting: a description that can go stale is a description you have to verify, and this one cannot. Point a script at a directory of Morloc programs and you can build an accurate inventory of every command in it, with argument types, without knowing anything about any of them.

## 6.13.1. Other views of the same commands

Two more flags render the same information for model clients:

| Flag | Output |
| --- | --- |
| `--mcp-tools` | An MCP `tools/list` definition: one entry per command, with a JSON Schema `inputSchema` and `outputSchema`. |
| `--mcp-config` | A client MCP server config — an `mcpServers` JSON entry using the stdio transport — so the program can be registered with a model client by redirecting one command into a config file. |

Commands whose types cannot cross the MCP boundary are excluded, and the reason is printed on standard error rather than left for you to discover:

```console
$ ./sift --mcp-tools > tools.json
morloc mcp: excluding command 'total' from the tool surface (reads from @stdin)
```

The MCP surface is covered in [Model Context Protocol (MCP)](https://morloc-project.github.io/docs/apis/mcp.md), and the same module served over HTTP, TCP, and Unix sockets in [Building API interfaces](https://morloc-project.github.io/docs/apis/api-interfaces.md). They are worth reading together with this section, because they are the same point from three directions: the command line is one view of a typed library, not the thing the library is built on. A CLI, an HTTP endpoint, and an MCP tool are three renderings of one set of functions, and none of them is written by hand.


---

<!-- https://morloc-project.github.io/docs/clis/directive-reference.html -->

# 6.14. Directive reference

Morloc Manual > Building CLIs | https://morloc-project.github.io/docs/clis/directive-reference.html | prev: https://morloc-project.github.io/docs/clis/interface-as-data.md | next: https://morloc-project.github.io/docs/apis/index.md

Every docstring directive that affects the generated interface, grouped by where it may be written. A directive written in the wrong place is not an error; it is kept as prose and the build warns, so check this table when one appears to do nothing.

**Table 6. On the module docstring — the --' lines directly above module**

| Directive | Effect |
| --- | --- |
| `@epilogue` | Open a block. Every following docstring line, until the docstring ends, is printed verbatim below the options from `-hh` up. Use it for an "Examples:" section. |

**Table 7. On a term’s signature preamble — the --' lines directly above name ::**

| Directive | Effect |
| --- | --- |
| `@name <name>` | Name the subcommand something other than the Morloc term. |
| `@with <flags>=<term>` | Attach an output action whose result stays typed. See [Output actions](https://morloc-project.github.io/docs/clis/output-actions.md). |
| `@render <flags>=<term>` | Attach an output action whose result is written as final bytes. |
| `@return <text>` | Describe the return value. The same as a docstring on the signature’s last type. |
| `@epilogue` | Open a block printed verbatim at the foot of this subcommand’s help, after its argument and return blocks. Use it for the command’s own "Examples:" section. The top-level help shows only the module’s block. |

**Table 8. On an argument — the --' lines directly above a type inside a signature**

| Directive | Effect |
| --- | --- |
| `@arg <flags>` | Make this argument an option rather than a positional. Requires `@default`. Not allowed on `Bool`. |
| `@default <json>` | The value used when an option is omitted, written as JSON. |
| `@true <flags>` | On a `Bool`: the flag that sets it true. The default becomes false. |
| `@false <flags>` | On a `Bool`: the flag that sets it false. The default becomes true. See the warning in [Arguments](https://morloc-project.github.io/docs/clis/arguments.md). |
| `@metavar <NAME>` | Name the argument. Becomes the placeholder in help for an option, and the property name in the machine-readable views. |
| `@many` | Accept several argv tokens and assemble them into a list. The argument type must be a list; as a positional it must be the last one. |
| `@stdin` | Make a `Str` positional optional and read standard input when it is omitted. See [Reading a stream from standard input](https://morloc-project.github.io/docs/clis/reading-stdin.md). |
| `@source inline` / `file` | Where the bytes come from. See [Input shape](https://morloc-project.github.io/docs/clis/input-shape.md). |
| `@form list` / `bytes` / `bytes-only` / `packet` | How the bytes are read. |
| `@check.path r` / `w` / `x` / `rw` | Require the argument to be a path satisfying the mode. |
| `@list.source`, `@list.form`, `@list.check.<kind>` | The same three, applied to each element of a `@form list` argument. |
| `@unroll` | On a record argument: split it into one flag per field. `@unroll false` opts one use out. |

**Table 9. On a type, record, or record field definition**

| Directive | Effect |
| --- | --- |
| `@metavar <NAME>` | On a `type` or a record in either form: the metavar inherited by every argument of that type. |
| `@mime <type/subtype>` | Attach a media type to a type. See [Output actions](https://morloc-project.github.io/docs/clis/output-actions.md). |
| `@arg`, `@default`, `@true`, `@false` | On a record field: the same meaning as on an argument, applied when the record is unrolled. |
| `@arg <flags>` | On a `record` definition: declare the group flag that accepts the whole record at once. |

**Table 10. Modifiers and value references, written inside another directive**

| Token | Meaning |
| --- | --- |
| `@default` | On a `@with` / `@render` directive: this action fires when no action flag and no `-f` is given. At most one per command. |
| `@stream` | On a `@with` / `@render` directive of a `@collect` command: apply the handler to each batch instead of the gathered stream. |
| `@offset` | As a handler argument under `@stream`: the number of elements already written. |
| `@value` | As a handler argument: the value being formatted. Appended last if not written explicitly. |
| `$1`, `$2`, …​ | As a handler argument: the command’s own Nth argument. |

One more directive is a deprecated spelling rather than a feature: `literal: true` means `@source inline`. It still works, and the build warns when you use it.


---

<!-- https://morloc-project.github.io/docs/apis/ -->

# 7. Building APIs

Morloc Manual | https://morloc-project.github.io/docs/apis/ | prev: https://morloc-project.github.io/docs/clis/directive-reference.md | next: https://morloc-project.github.io/docs/apis/search-and-install.md


---

<!-- https://morloc-project.github.io/docs/apis/search-and-install.html -->

# 7.1. Search and install

Morloc Manual > Building APIs | https://morloc-project.github.io/docs/apis/search-and-install.html | prev: https://morloc-project.github.io/docs/apis/index.md | next: https://morloc-project.github.io/docs/apis/exposing-native-resources.md

The docstrings are used for discoverability as well. In this section I’ll cover how modules are installed as executables or standard modules and how they can be searched.

I’ll demonstrate this with a simple two module Morloc program describing a set of DnD operations. The first module defines general random operations:

**fate.loc**

```morloc
module fate (roll, coinToss, choose)

import root-py
import random

source Py from "fate.py"
  ( "roll" as roll
  , "coin_toss" as coinToss
  , "choose" as choose
  )

--' Roll n d-sided dice
roll ::
  --' Number of dice
  Int ->
  --' Number of pips per die
  Int ->
  --' Roll values
  <Random> [Int]

--' Randomly return True or False
coinToss :: <Random> Bool

--' Randomly choose one element from a non-empty list
choose :: [a] -> <Random> a
```

The sourced `fate.py` script contains the following code:

**fate.py**

```python
import random

def choose(xs):
    return random.choice(xs)

def roll(n, d):
    return [random.randint(1, d) for _ in range(n)]

def coin_toss():
    return bool(random.randint(0,1))
```

We can install `fate` with `morloc install --build ./fate`. This installs the module so it can be imported by other Morloc programs, and the `--build` flag additionally builds an executable we can test.

> **Note**
> `morloc install` (with or without `--build`) installs modules for import — from remote sources by name (e.g., `morloc install root`) or from local directories with `./`. In contrast, `morloc make --install` compiles a local program and installs the resulting executable.

Either way, an installed program is named after its **module** — the `module <name>` declaration — and not after the file it was compiled from. That is why the executable below is `fate`. Plain `morloc make` does the opposite and names its launcher after the source file ([Your first program](https://morloc-project.github.io/docs/getting-started/first-program.md)), and the difference is deliberate: `make` leaves a **local** artifact in your working directory, where the source name is the natural handle, the same way a C++ compiler hands you `a.out`. `--install` writes into a **global** namespace, where a program’s identity is the name other code imports it by. The entry file is conventionally `main.loc` and carries no identity at all.

We can test this, for example by rolling 3d8:

```console
$ fate roll 3 8
[8,2,5]
```

Next let’s build on this foundation. First let’s make a simple tavern script that helps generate new characters.

**tavern.loc**

```morloc
module tavern (randomClass, randomRace)

import root-py
import fate (choose)

--' Select a random class
randomClass :: <Random> Str
randomClass = choose ["Fighter", "Wizard", "Rogue", "Cleric", "Ranger", "Bard"]

--' Select a random race
randomRace :: <Random> Str
randomRace = choose ["Human", "Elf", "Dwarf", "Halfling"]
```

Next let’s add a module for combat:

**combat.loc**

```morloc
module combat (rollAdv, fighterDamage, intro)

import root-py
import root-r
import fate (roll, coinToss)

--' Roll a pair of d20 dice and keep the larger result
rollAdv :: <Random> Int
rollAdv = do fold max 0 !(roll 2 20)

--' Damage done on hit, modifier + sum of dice rolls
damage ::
  --' Enemy Armor Class
  Int ->
  --' Attack modifier
  Int ->
  --' Attack dice
  <Random> [Int] ->
  --' Damage modifier
  Int ->
  --' Damage dice
  <Random> [Int] ->
  --' Total damage
  <Random> Int
damage ac atkMod atkDice dmgMod dmgDice = do
  atkD <- atkDice
  dmgD <- dmgDice
  let atkRoll = fold max 0 atkD
  let atk = atkMod + atkRoll
  let dmg = dmgMod + sum dmgD
  ? atkRoll == 20 = 2 * dmg  -- critical
    ? atk >= ac = dmg        -- hit
    : 0                      -- miss

--' Damage calculation for a fighter
fighterDamage ::
  --' Enemy Armor Class
  Int ->
  --' Fighter's damage
  <Random> Int
fighterDamage ac = damage ac 4 (roll 1 20) 2 (roll 2 8)

source R from "combat.R" ("intro")

--' Introduce a new battle!
intro ::
  --' Monster name
  Str ->
  --' DM's monster intro
  Str
```

We can build and install the program with:

```bash
$ morloc make --install combat.loc
```

This command does several things.

First it installs the `combat` executable to a standard path. The build artifacts (the `manifest.json` and compiled pools) and the source files in the package — the directory holding the entry `.loc` file, wherever you run the command from — need to be moved to a standard location. There are two ways you can specify the required build files.

You can specify required files with `--include` arguments

```bash
$ morloc make --install combat.loc --include fate.loc --include combat.R
```

Or you can create a `package.yaml` file and add an `include` field. The default file can be generaed for you with `morloc new`. You can then modify the `include` field list with the required files:

```yaml
name: combat
version: 0.1.0
homepage: null
synopsis: null
description: null
category: null
license: MIT
author: null
maintainer: null
github: null
bug-reports: null
dependencies: []
# Files to include when installing with `morloc make --install`
include: ["combat.R"]
```

Then run `morloc make --install combat.loc`.

The opposite control is a `.morlocignore` file in the project root: one pattern per line, `#` comments, a trailing `/` for a directory, and `!` to negate. Without an `include` list the install copies the whole project minus `.git/` and whatever `.morlocignore` names, so a cargo `target/` or a `*pycache*/` beside the sources is copied too. Name them; a deployment image built with `mim freeze` refuses a program that carries them.

In both install paths, the `combat` source code is copied to the `~/.local/share/morloc/exe/<modname>/` folder (with the build artifacts, `manifest.json` and the compiled pools, nested under `<modname>-build/` inside it) and the launcher script itself is written to `~/.local/share/morloc/bin/`.

We can view the installed executable:

```console
$ morloc list -v combat
Programs:
  combat  3 commands
    rollAdv :: Int
    fighterDamage :: Int -> Int
    intro :: Str -> Str
```

If we add the Morloc bin folder above to PATH, then we can now use this program naturally:

```console
$ combat -h
... (auto-generated help: the three exported commands under a General Options section;
     `combat --help` additionally lists the nexus options, and `combat -h @` renders them too)
$ combat fighterDamage 15
12
$ combat fighterDamage 15
8
```

We can also uninstall with `morloc uninstall combat`. This will cleanly remove the installed source and the installed executable script.


---

<!-- https://morloc-project.github.io/docs/apis/exposing-native-resources.html -->

# 7.2. Exposing native resources

Morloc Manual > Building APIs | https://morloc-project.github.io/docs/apis/exposing-native-resources.html | prev: https://morloc-project.github.io/docs/apis/search-and-install.md | next: https://morloc-project.github.io/docs/apis/data-transfer.md

The `dependencies` field links against shared libraries. The `expose` field handles a different need: a module that defines a C++ struct, a Python class, or an R helper whose definition downstream foreign code needs to `#include` (or `import`, or `source`) by name. On install, listed files are copied to per-language well-known paths under `$MORLOC_HOME`, namespaced by module name. Downstream code then refers to them through a stable path that is the same for every consumer.

```yaml
expose:
  cpp: [person.hpp]
  py:  [__init__.py, helpers/]
  r:   [util.R]
```

Each key is optional. Paths are relative to the module root; glob patterns ( **within a segment,** `*` across segments, trailing `/` for a directory) follow the same syntax as `include`. Subtree structure is preserved on copy — essential for Python packages with `*init*.py` markers and for C++ headers that `#include` siblings by relative path.

| Language | Destination | Consumer code |
| --- | --- | --- |
| C++ | `$MORLOC_HOME/include/<module>/...` | `#include "<module>/foo.hpp"` |
| Python | `$MORLOC_HOME/lib/python/<py_module>/...` | `import <py_module>.foo` |
| R | `$MORLOC_HOME/lib/R/<module>/...` | `.morloc.source("<module>/foo.R")` |

For Python, hyphens in the module name are converted to underscores so the destination is a legal Python identifier (a module `tensor-cpp` becomes `tensor_cpp`). The C subtree \`$MORLOC\_HOME/include\` is already on every C pool’s `-I` path, so no compile flags need tweaking; consumers just write the namespaced `#include`. `morloc uninstall` symmetrically removes the exposed copies alongside the install dir.

A worked example. The `people` module declares a Morloc type backed by a C++ struct and exposes the header that defines it:

**people/main.loc**

```morloc
module people (Person, makePerson)

import root-cpp

type Cpp => Person = "person_t"

source Cpp from "person.hpp" ("make_person" as makePerson)

makePerson :: Str -> Int -> Person
```

**people/package.yaml**

```yaml
name: people
version: 0.1.0
expose:
  cpp: [person.hpp]
```

**people/person.hpp**

```cpp
#ifndef PEOPLE_PERSON_HPP
#define PEOPLE_PERSON_HPP

#include <string>

struct person_t {
    std::string name;
    int age;
};

inline person_t make_person(const std::string& name, int age) {
    return person_t{name, age};
}

#endif
```

Install with `morloc install ./people`. The exposed header now lives at `$MORLOC_HOME/include/people/person.hpp`. A downstream program imports the Morloc type and uses the underlying C++ struct directly in its own foreign code:

**main.loc**

```morloc
module main (greeting)

import people (Person, makePerson)
import root-cpp

source Cpp from "src.hpp" ("greet")

greet :: Person -> Str

greeting :: Str
greeting = greet (makePerson "Alice" 30)
```

**src.hpp**

```cpp
#include "people/person.hpp"
#include <string>

inline std::string greet(const person_t& p) {
    return "Hello, " + p.name + "! Age " + std::to_string(p.age);
}
```

The same exposed header is discoverable from non-Morloc C programs too: compile with \`g -I$MORLOC\_HOME/include\` and `#include "people/person.hpp"` works identically.


---

<!-- https://morloc-project.github.io/docs/apis/data-transfer.html -->

# 7.3. Controlling data transfer

Morloc Manual > Building APIs | https://morloc-project.github.io/docs/apis/data-transfer.html | prev: https://morloc-project.github.io/docs/apis/exposing-native-resources.md | next: https://morloc-project.github.io/docs/apis/api-interfaces.md

When a Morloc value crosses a pool boundary, the runtime picks one of three routes for data transfer:

-   **Inline** — for serialized payloads that are less than 64 KiB (by default), the bytes ride inside the packet over the Unix socket.
-   **Shared memory** — for larger payloads, the data sits in `/dev/shm` and the packet carries only an 8-byte pointer. This allows zero-copy data transfer between pools on the same host.
-   **Temp file** — only used when shared memory has been disabled (see `--no-shm` below). The data is written to a `.mpk` file and the packet carries the path.

Three `morloc make` flags let you override the default policy when it is wrong for your workload — restricted containers, tight `/dev/shm` quotas, debugging the wire-level traffic, or simply tuning the threshold to a value that matches your data shape:

| Flag | Effect |
| --- | --- |
| `--inline-size BYTES` | Move the inline/large threshold. Accepts a bare number or `k`/`m`/`g` suffix (binary, 1024-based). `0` means never inline. Default: `64k`. |
| `--no-shm` | Disable shared memory. Payloads above the inline threshold are written to a temp file and passed by path. |
| `--tmpdir PATH` | Directory for the temp files produced under `--no-shm`. **When set, the files are NOT auto-deleted at end of eval** — useful for testing and debugging. Default (unset): `$TMPDIR` or `/tmp`, with auto-cleanup at end of every eval. |

## 7.3.1. Combinations

| Build flags | Behavior |
| --- | --- |
| (none) | Inline `⇐ 64k`, shared memory above. |
| `--inline-size 0` | Never inline; all cross-pool transfers go through shared memory. |
| `--no-shm` | Inline `⇐ 64k`, temp files above. No `/dev/shm` traffic. |
| `--inline-size 0 --no-shm` | Every cross-pool transfer is a temp file. Slowest mode, but works on systems with no shared memory at all. |

## 7.3.2. Examples

Build for a container with no usable `/dev/shm`:

```console
$ morloc make --no-shm -o nexus main.loc
```

Run a workload where 64 KiB is too small (large records, every call exceeds the default):

```console
$ morloc make --inline-size 1m -o nexus main.loc
```

Inspect the wire-level packets for debugging or testing:

```console
$ morloc make --no-shm --inline-size 0 --tmpdir ./wire-dump -o nexus main.loc
$ ./nexus pipeline arg1 arg2
$ ls wire-dump/
morloc-pkt-12345-0.mpk  morloc-pkt-12345-1.mpk  ...
```

Each file is a self-contained MessagePack payload — one per pool return. Because `--tmpdir` was supplied, they persist after the program exits and can be inspected with any MessagePack reader.


---

<!-- https://morloc-project.github.io/docs/apis/api-interfaces.html -->

# 7.4. Building API interfaces

Morloc Manual > Building APIs | https://morloc-project.github.io/docs/apis/api-interfaces.html | prev: https://morloc-project.github.io/docs/apis/data-transfer.md | next: https://morloc-project.github.io/docs/apis/mcp.md

In addition to being CLI tools, compiled Morloc programs can run as long-lived daemons, accepting function calls over HTTP, TCP, or Unix sockets. A serving front-end (the `router` mode) aggregates several programs behind one HTTP port, serving both a JSON API (for HTTP clients) and MCP (for AI assistants) with optional bearer-token auth.

The daemon, HTTP, TCP, and socket machinery is already part of the `morloc-nexus` runtime that every compiled program wraps. To get a dedicated daemon executable, build the program with `--daemon-out`:

```console
$ morloc make --daemon-out combatd combat.loc
```

This writes a `./combatd` launcher next to the ordinary `./combat` CLI (you can produce both at once with `morloc make -o combat --daemon-out combatd combat.loc`). Running `./combatd` starts the program as a long-lived daemon and accepts the listener options shown below.

> **Note**
> `./combatd` is a thin wrapper around the shared runtime — it is equivalent to `morloc-nexus daemon ./combat`. Either form works; the dedicated executable is just the more convenient one to hand out and script against.

## 7.4.1. HTTP protocol

To start `combat` as a daemon on HTTP port 8080:

```console
$ ./combatd --http-port 8080 &
morloc-daemon: listening on http://0.0.0.0:8080
$ DAEMON_PID=$!
```

The trailing `&` creates the process in the background and `$!` captures its PID for later shutdown (see the Shutdown section below). This command launches all language pool processes (Python and R in this case) as child processes in separate process groups. A thread pool handles concurrent requests. If a pool crashes, the daemon detects it restarts it automatically.

We can check the daemon’s health:

```bash
$ curl -s localhost:8080/health
{"status":"ok","result":[true]}
```

The /health endpoint returns the liveness status of each pool.

The running daemons are discoverable:

```console
$ curl -s localhost:8080/discover | jq .
{
  "status": "ok",
  "result": {
    "name": "combat",
    "morloc_version": "0.94.0",
    "commands": [
      {
        "name": "rollAdv",
        "type": "remote",
        "return": { "type": "Int", "schema": "j" },
        "args": [],
        "desc": "Roll a pair of d20 dice and keep the larger result"
      },
      {
        "name": "fighterDamage",
        "type": "remote",
        "return": { "type": "Int", "schema": "j" },
        "args": [
          { "kind": "pos", "type": "Int", "schema": "j" }
        ],
        "desc": "Damage calculation for a fighter"
      },
      {
        "name": "intro",
        "type": "remote",
        "return": { "type": "Str", "schema": "s" },
        "args": [
          { "kind": "pos", "type": "Str", "schema": "s" }
        ],
        "desc": "Introduce a new battle!"
      }
    ]
  }
}
```

The `morloc_version` string identifies the compiler that produced the program. Each command has one of two `type` tags: `"remote"` (dispatched to a language pool) or `"pure"` (evaluated by the nexus itself — e.g. a plain composition that never crosses a language boundary). The `return` object bundles the general `type` and its wire `schema`; each entry in `args` uses the same shape (plus a `kind` field, `"pos"` for positional or `"opt"` for optional).

> **Note**
> The front-end’s `GET /discover/<program>` (see below) returns this same per-program shape; its top-level `GET /discover` is a flatter index across all served programs.

Functions can be called over the port:

```console
$ curl -s -X POST localhost:8080/call/rollAdv -d '[]'
{"status":"ok","result":18}

$ curl -s -X POST localhost:8080/call/fighterDamage -d '[15]'
{"status":"ok","result":12}
```

Bad commands will return sensible errors:

```bash
$ curl -s -X POST localhost:8080/call/fireball -d '[]'
{"status":"error","error":"Unknown command: fireball"}
```

Beyond the pre-compiled commands, `POST /eval` and `POST /typecheck` take a JSON body `{"expr": "…​"}` and evaluate (or type-check) a single Morloc expression on the fly:

```bash
$ curl -s -X POST localhost:8080/eval -d '{"expr":"import root-py; 1 + 2"}'
{"status":"ok","result":3}
```

`POST /eval` runs the expression in the **eval sandbox**. Beyond the base eval rules — it may use `let`/`where`/`do` but may not declare types, typeclasses, instances, `source` foreign code, or import local-filesystem modules — served eval is **always** sandboxed by two gates the operator configures when starting the server:

-   **Module allow-list.** The expression’s top-level imports are limited to the modules passed in `--eval-allowed-modules` (comma-separated). The default is empty, so an out-of-the-box daemon runs only pure, module-free expressions (literals and pure intrinsics like `@show`/`@hash`); grant access by curating the list. Matching is on the resolved module, so `import M as N` is checked against `M`.
-   **IO-intrinsic ban.** The expression may not write an IO intrinsic (`@open`, `@save`, `@write`, `@stdin`, …​) directly. IO reached **through** a function exported by an allow-listed module is fine, so a server exposes exactly the IO surface it chooses — wrapped in named functions — and never a raw filesystem primitive.

```bash
$ ./progd --eval-allowed-modules root-py &
$ curl -s -X POST localhost:8080/eval -d '{"expr":"import root-py; @write \"x\" 1"}'
{"status":"error","error":"IO intrinsics may not be used directly ..."}
$ curl -s -X POST localhost:8080/eval -d '{"expr":"import shell-py (run); run \"id\""}'
{"status":"error","error":"module '\''shell-py'\'' is not in the eval allow-list"}
```

This is the intended interface for exposing a curated set of server-side functions to untrusted callers — they can only compose what the operator allow-lists; arbitrary code upload is not possible. There is no unsandboxed served mode: for trusted, unrestricted evaluation use the local `morloc eval` CLI, and use `morloc make` server-side to build programs that need local modules. `POST /typecheck` only reports the inferred type and never executes anything, so it is not sandboxed the same way.

Every response also carries an HTTP status code that reflects the class of outcome, so HTTP clients with built-in retry / branching logic (curl `--fail`, axios, fetch) work as expected without parsing the JSON envelope. The JSON body is still always present for clients that prefer it.

| Code | Meaning | When |
| --- | --- | --- |
| `200` | OK | Success. The body’s `result` field carries the return value. |
| `204` | No Content | The response to a CORS preflight `OPTIONS` request. The daemon never dispatches OPTIONS through any handler; it answers immediately with the standard `Access-Control-Allow-*` headers and an empty body. |
| `400` | Bad Request | The request was malformed: missing required field, unparseable args JSON, wrong number of arguments, a value that didn’t match its declared schema, or a string containing an embedded NUL byte the target language can’t represent. |
| `404` | Not Found | The path or named resource doesn’t exist: an unknown HTTP endpoint (`GET /nope`), an unknown command (`POST /call/fireball`), or a binding name that wasn’t registered (`DELETE /bindings/missing`). |
| `408` | Request Timeout | A `POST /eval` or `POST /typecheck` expression consumed more CPU than the `--eval-timeout` budget (default 30s) and was killed by the kernel via `SIGXCPU`. This guard only applies to those two endpoints, which fork `morloc eval`/`typecheck` as a subprocess. `POST /call/` requests dispatch into a pre-compiled pool worker and are **not** bounded by `--eval-timeout` — long-running calls there are allowed. |
| `500` | Internal Server Error | A genuinely server-side failure: a pool socket error, a fork/pipe failure, the eval engine returning an unexpected error, or any other state that wasn’t the client’s fault. |
| `503` | Service Unavailable | The service is temporarily unable to handle the request but the caller should retry. The daemon emits 503 during the brief window where it is tearing down and respawning a crashed pool; the router emits 503 when forwarding a request to a daemon in that state, or when its cluster `/health` reports at least one program unhealthy. All 503 responses include `Retry-After: 1`. Clients with built-in retry middleware (curl `--retry`, axios-retry, hyper-retry) will back off and re-issue automatically. |

The same status-code mapping applies whether you call a single daemon directly or hit the router; the router forwards classification through unchanged. Client errors (4xx) describe something the caller can fix; server errors (5xx) describe something the caller should retry or report. Unix-socket and TCP clients see the same classification via the JSON envelope’s `status` and `error` fields, though they don’t get the HTTP-level `Retry-After` hint on 503.

## 7.4.2. TCP protocol

HTTP adds overhead per request: headers, text parsing, and the full HTTP framing around each message. When your client is a program rather than a browser or `curl`, you can skip all of that. The TCP protocol uses a compact binary framing — just a 4-byte big-endian length prefix followed by the JSON payload. This makes it well suited for service-to-service communication, high-throughput automated pipelines, or any context where you control both ends of the connection and want minimal overhead.

Start a daemon on TCP port 9001:

```bash
$ ./combatd --port 9001 &
morloc-daemon: listening on tcp://127.0.0.1:9001
```

Unlike the HTTP protocol, you can’t use `curl` to talk to a TCP daemon. You need a client that speaks the length-prefixed binary framing. Here is a minimal Python client:

**tcp\_client.py — minimal TCP client**

```python
import socket, struct, json

def recvall(s, n):
    data = b''
    while len(data) < n:
        chunk = s.recv(n - len(data))
        if not chunk:
            raise RuntimeError("Connection closed")
        data += chunk
    return data

def call(host, port, method, command=None, args=None):
    msg = {"method": method}
    if command: msg["command"] = command
    if args is not None: msg["args"] = args

    payload = json.dumps(msg).encode()
    s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    s.connect((host, port))
    # send 4-byte big-endian length, then the JSON payload
    s.sendall(struct.pack('>I', len(payload)) + payload)

    # read the 4-byte response length, then the response
    resp_len = struct.unpack('>I', recvall(s, 4))[0]
    resp = recvall(s, resp_len)
    s.close()
    return json.loads(resp)

print(call("localhost", 9001, "call", "rollAdv"))
# {"status": "ok", "result": 18}

print(call("localhost", 9001, "call", "fighterDamage", [15]))
# {"status": "ok", "result": 12}

print(call("localhost", 9001, "health"))
# {"status": "ok", "result": [true]}

print(call("localhost", 9001, "discover"))
# {"status": "ok", "result": {"name": "combat", "commands": [...]}}
```

The request is a JSON object with a `method` field (`"call"`, `"discover"`, or `"health"`), an optional `command` field naming the function, and an optional `args` array.

## 7.4.3. Unix socket protocol

For processes running on the same machine, Unix domain sockets are the fastest option. They bypass the entire network stack — no TCP handshake, no port allocation, no loopback routing. This is how Morloc pools communicate with the nexus internally.

To start a daemon on a Unix socket:

```bash
$ ./combatd --socket /tmp/combat.sock &
morloc-daemon: listening on unix:///tmp/combat.sock
```

The wire protocol is identical to TCP: a 4-byte big-endian length prefix followed by the JSON payload. The only difference is the socket type.

**unix\_client.py — minimal socket client**

```python
import socket, struct, json

def recvall(s, n):
    data = b''
    while len(data) < n:
        chunk = s.recv(n - len(data))
        if not chunk:
            raise RuntimeError("Connection closed")
        data += chunk
    return data

def call(sock_path, method, command=None, args=None):
    msg = {"method": method}
    if command: msg["command"] = command
    if args is not None: msg["args"] = args

    payload = json.dumps(msg).encode()
    s = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
    s.connect(sock_path)
    s.sendall(struct.pack('>I', len(payload)) + payload)

    resp_len = struct.unpack('>I', recvall(s, 4))[0]
    resp = recvall(s, resp_len)
    s.close()
    return json.loads(resp)

print(call("/tmp/combat.sock", "call", "rollAdv"))
# {"status": "ok", "result": 18}

print(call("/tmp/combat.sock", "call", "fighterDamage", [15]))
# {"status": "ok", "result": 12}

print(call("/tmp/combat.sock", "discover"))
# {"status": "ok", "result": {"name": "combat", "commands": [...]}}
```

## 7.4.4. Running all protocols at once

You don’t have to choose. One daemon can listen through all three protocols at the same time:

```bash
$ ./combatd \
      --http-port 8080 \
      --port 9001 \
      --socket /tmp/combat.sock
morloc-daemon: listening on unix:///tmp/combat.sock
morloc-daemon: listening on tcp://127.0.0.1:9001
morloc-daemon: listening on http://0.0.0.0:8080
```

All three protocols hit the same daemon process and share the same pool processes. A request arriving over HTTP, TCP, or the Unix socket is dispatched identically — only the framing differs.

### Ephemeral ports

If you don’t care which port the daemon binds to — which is the common case for tests, CI jobs, or any orchestrator running many daemons in parallel — pass `0` and the OS picks a free one for you. The actual port appears in the stderr ready line, and can also be written to a file in a fixed JSON shape:

```bash
$ ./combatd --http-port 0 --port 0 --port-file ports.json &
morloc-daemon: listening on tcp://127.0.0.1:46217
morloc-daemon: listening on http://0.0.0.0:39381

$ cat ports.json
{"http":39381,"tcp":46217,"unix":null}
```

The file is written atomically (via `rename`) only after every listener is bound, so a `stat`\-waiting client never sees a half-written file. Missing listeners are `null`, never absent — the schema is fixed.

## 7.4.5. From single daemons to a router

Everything above shows a single program running as a daemon. This is enough when you have one service, but Morloc programs are designed to be composed. You might have a `tavern` program that picks character classes and races, and a `combat` program that resolves attacks and damage. Each is its own compiled Morloc program with its own pools.

You *could* start each one as an independent daemon on its own port and have your client keep track of which port maps to which program. But that gets tedious. The router solves this: it presents a single HTTP endpoint — a **serving front-end** — that serves the programs you select behind one port, forwarding each call to that program’s own daemon. The front-end runs no user code itself, so a crashing call takes down only its program’s worker (restarted automatically), never the front-end. Router mode is the `router` subcommand of `morloc-nexus`; in practice you launch it through `mim start` (see [`mim` (Morloc Installation Manager)](https://morloc-project.github.io/docs/utilities/mim.md)), which adds the container, loopback/token handling, and the `install` → `expose` → `start` lifecycle.

The front-end exposes both adapters on the one port: MCP at `POST /mcp` (for AI assistants) and a JSON API at `POST /call/<program>/<command>` (for HTTP clients), plus `GET /discover` and `GET /health`.

The following diagram illustrates how a client request flows through the router to a program daemon and its language pools:

```
                  Client
                    |
                    | HTTP: POST /call/tavern/randomClass -d '[]'
                    v
             +--------------+
             |    Router    |  morloc-nexus router --http-port 9090
             |  (HTTP:9090) |  Reads manifests from fdb/ at startup
             +--------------+
              /            \
    Unix socket            Unix socket
            /                \
  +-----------+         +-----------+
  |  tavern   |         |  combat   |
  |  daemon   |         |  daemon   |
  +-----------+         +-----------+
       |                 /        \
       v                v          v
    Python           Python        R
     pool             pool        pool
```

Each daemon is a child process of the router, started lazily on first request. The router and its daemons communicate over Unix sockets using the same length-prefixed JSON protocol described above.

## 7.4.6. Router mode

### Setup

To make a program available to the router, install it with `--install`. This installs the program under the `exe/` directory (identified by its **module name**), where the front-end finds each named program’s `exe/<name>/<name>-build/manifest.json` at startup.

```bash
$ morloc make --install -o tavern tavern.loc
Installed 'tavern' to ~/.local/share/morloc/bin/tavern

$ morloc make --install -o combat combat.loc
Installed 'combat' to ~/.local/share/morloc/bin/combat

$ ls ~/.local/share/morloc/exe/
combat  tavern
```

### Starting the router

Name the programs to serve (there is no serve-everything scan); `--program` serves a program over both adapters, `--mcp`/`--api` restrict it to one:

```bash
$ morloc-nexus router --http-port 9090 --program combat --program tavern
morloc serve: MCP at http://0.0.0.0:9090/mcp (5 tools) | API at http://0.0.0.0:9090/call/<module>/<command> | discovery at http://0.0.0.0:9090/discover
```

If an auth token is configured (`--auth-token`, or `MORLOC_MCP_TOKEN`), every `/mcp` and `/call` request must carry `Authorization: Bearer <token>`; `/health` and CORS preflight (`OPTIONS`) stay open. A non-loopback bind with no token is refused unless `--allow-no-auth` is passed.

### Discovery

`GET /discover` is the API index — the served modules, the `/call` URL shape, and a pointer to the MCP `tools/list` catalog. `GET /discover/<program>` returns one program’s commands and their positional argument order:

```bash
$ curl -s localhost:9090/discover | python3 -m json.tool
{
    "api": { "modules": [ {"module": "combat", "call": "/call/combat/<command>", "help": "/discover/combat"}, ... ],
             "call": "/call/<module>/<command>", "note": "POST positional args as a JSON array." },
    "mcp": { "endpoint": "/mcp", "tools": 5, "note": "Use tools/list for the MCP catalog." },
    "eval": { "enabled": false, "endpoints": ["/eval", "mcp tool 'eval'"] }
}

$ curl -s localhost:9090/discover/tavern | python3 -m json.tool
{ "program": {"name": "tavern", ...}, "commands": [...] }
```

### Calling functions

Calls are routed by program name in the URL: `/call/<program>/<command>`.

```bash
$ curl -s -X POST localhost:9090/call/tavern/randomClass -d '[]'
{"status":"ok","result":"Rogue"}

$ curl -s -X POST localhost:9090/call/tavern/randomRace -d '[]'
{"status":"ok","result":"Elf"}

$ curl -s -X POST localhost:9090/call/combat/rollAdv -d '[]'
{"status":"ok","result":17}

$ curl -s -X POST localhost:9090/call/combat/fighterDamage -d '[15]'
{"status":"ok","result":12}

$ curl -s -X POST localhost:9090/call/combat/intro -d '["Goblin"]'
{"status":"ok","result":"A wild Goblin appears!"}
```

The first call to a program starts its daemon automatically. Subsequent calls reuse the running daemon with no startup cost. If a daemon crashes between calls, the front-end detects the failure and restarts it transparently. The same commands are available to MCP clients at `POST /mcp` as tools named `<program>*<command>*` *(e.g. `combat`*`rollAdv`), called with named arguments.

### Error handling

A program that is not served on the API adapter is `404`:

```bash
$ curl -s -X POST localhost:9090/call/dungeon/explore -d '[]'
{"error":"module not exposed on the API"}
```

### Independent daemons vs router-managed daemons

A daemon started manually (e.g., `./combatd --http-port 8080`) is completely independent of the front-end. The front-end only knows about the programs you named (`--program`/`--mcp`/`--api`), whose manifests live under the `exe/` directory, and it starts its own daemon instances as child processes. If you start a daemon on your own and also serve the same program through the front-end, you will have two separate daemon processes — each with its own pool processes and its own state.

## 7.4.7. Shutdown

Send `SIGTERM` (or `SIGINT`) to stop a daemon or router gracefully. The daemon sends `SIGTERM` to each pool process group, waits briefly for clean exit, then sends `SIGKILL` to any stragglers. Unix socket files are removed.

```bash
$ kill $DAEMON_PID
morloc-daemon: shutting down

$ kill $ROUTER_PID
morloc-router: shutting down
```

When a router shuts down, it terminates all the daemons it started. There is currently no way to stop an individual program’s daemon through the router API — the router manages their lifecycles internally. If you need to restart a specific program, restart the router.

## 7.4.8. Summary

| Role | Invocation | Description |
| --- | --- | --- |
| Daemon | `./<daemon-exe>` (built with `morloc make --daemon-out`) | Run one program as a persistent service |
| Front-end (router) | `morloc-nexus router --program <name>…​` | Serve the named programs behind one HTTP port (MCP + JSON API); usually launched by `mim start` |
| HTTP (daemon) | `--http-port <n>` | RESTful JSON API (curl-friendly); `0` = ephemeral. The daemon also serves TCP (`--port`), a Unix socket (`--socket`), and `--port-file` for ephemeral ports; the front-end is HTTP-only. |
| Auth (front-end) | `--auth-token` / `MORLOC_MCP_TOKEN` | Require a bearer token on `/mcp` and `/call` |
| exe | `--fdb <path>` | Override the installed-program directory (default: `$MORLOC_HOME/exe`) |


---

<!-- https://morloc-project.github.io/docs/apis/mcp.html -->

# 7.5. Model Context Protocol (MCP)

Morloc Manual > Building APIs | https://morloc-project.github.io/docs/apis/mcp.html | prev: https://morloc-project.github.io/docs/apis/api-interfaces.md | next: https://morloc-project.github.io/docs/runs/index.md

The same compiled program that runs as a CLI tool or a daemon can also serve as an [MCP](https://modelcontextprotocol.io) server, exposing its exported functions as tools that an AI agent (Claude Desktop, an IDE assistant, or any MCP client) can call. The MCP machinery is already part of the shared runtime; there is no separate build. Any compiled program is served over MCP by running the runtime in `mcp` mode against its manifest:

```console
$ morloc-nexus mcp ./combat            # or: morloc-nexus mcp combat-build/manifest.json
```

MCP is a JSON-RPC 2.0 protocol spoken over a program’s standard input and output: the server reads requests on stdin and writes responses on stdout; every other byte — pool output, log lines, diagnostics — is routed to stderr so the protocol stream stays clean.

This stdio server is the **local** transport: the client and morloc run on the same machine, and the client launches the server as a child process. For a **networked** deployment — morloc in a container, or the agent on another host — serve MCP over HTTP with `mim start` (see [Building APIs](https://morloc-project.github.io/docs/apis/index.md)); the same program then also answers a plain JSON API on the same port.

The command launches the language pools (Python and R for `combat`) and then blocks, waiting for JSON-RPC messages on stdin. It is normally started **by** an MCP client rather than typed at a shell, but because the transport is just line-delimited JSON you can drive it by hand to see how it works.

> **Note**
> Each message is one complete JSON object on a single line; the server reads one line at a time, so a pretty-printed object split across several lines is parsed as separate broken fragments (each answered with a `-32700 invalid JSON` error). The transcripts below are indented only for readability — on the wire every object is a single line.

## 7.5.1. The handshake

An MCP session opens with a three-message handshake: the client sends `initialize`, the server replies with its capabilities, and the client confirms with an `initialized` notification. Only then may tools be listed or called.

**client → server, then server → client**

```
--> {"jsonrpc":"2.0","id":1,"method":"initialize",
     "params":{"protocolVersion":"2025-06-18","capabilities":{},
               "clientInfo":{"name":"demo","version":"0"}}}
<-- {"jsonrpc":"2.0","id":1,"result":{
      "protocolVersion":"2025-06-18",
      "capabilities":{"tools":{"listChanged":false}},
      "serverInfo":{"name":"combat","version":"0.94.0"}}}

--> {"jsonrpc":"2.0","method":"notifications/initialized"}
```

The `notifications/initialized` message carries no `id` and receives no reply — that is what a JSON-RPC notification is. `ping` is answered at any point in the lifecycle; `tools/list` and `tools/call` are rejected until the handshake completes.

## 7.5.2. Inspecting the tool surface

Every exported function becomes one tool. You can dump the full tool list — the same payload `tools/list` returns — without starting a session, using the `--mcp-tools` flag. This is the MCP analogue of the daemon’s `/discover` endpoint:

```console
$ ./combat --mcp-tools | jq '.tools[] | {name, inputSchema}'
{
  "name": "rollAdv",
  "inputSchema": {
    "type": "object",
    "properties": {},
    "required": [],
    "additionalProperties": false
  }
}
{
  "name": "fighterDamage",
  "inputSchema": {
    "type": "object",
    "properties": {
      "_1": { "type": "integer", "description": "Enemy Armor Class" }
    },
    "required": [ "_1" ],
    "additionalProperties": false
  }
}
{
  "name": "intro",
  "inputSchema": {
    "type": "object",
    "properties": {
      "_1": { "type": "string", "description": "Monster name" }
    },
    "required": [ "_1" ],
    "additionalProperties": false
  }
}
```

Each tool’s `description` comes from the function’s docstring, and each argument’s morloc type is rendered as a JSON Schema type (`Int` → `integer`, `Str` → `string`, `[a]` → `array`, a record → `object`, `?a` → a nullable union). An argument’s own `--'` docstring becomes the property `description` — so ``fighterDamage’s `*1*`` *is documented as \_Enemy Armor Class* even though the key itself is a positional index.

## 7.5.3. How arguments map to properties

MCP delivers arguments as a **named** object, so every morloc argument needs a name. How that name is chosen depends on the kind of argument:

| Morloc argument | MCP property name |
| --- | --- |
| A positional argument | A reserved index: `_1` for the first positional, `_2` for the second, and so on. Metavars (`FILE`, `INT`, …​) are display placeholders that get reused and are deliberately **not** used as keys; the index is always unique. |
| An option (`--' arg: -f/--factor`) | The long name (`factor`), or the short character if there is no long form. |
| A flag (`--' true: --clean` / `--' false: --no-clean`) | The flag’s name, typed as a `boolean`. |
| An unrolled record (`--' unroll: true`) | One property per field, keyed by the field name. |
| A record passed whole (`--' arg: --config`) | A single `object` property named for the record type. |

Because option and flag names can never begin with an underscore (the compiler reserves that), a `_N` positional key can never collide with one. Two positionals that happen to share a metavar are still distinct tools arguments, where an earlier design would have had to drop the command.

## 7.5.4. Calling a tool

`tools/call` names the tool and supplies its arguments by key. The server inverts the named arguments back into a positional call, dispatches it through the same machinery the CLI and daemon use, and returns the result.

```
--> {"jsonrpc":"2.0","id":2,"method":"tools/call",
     "params":{"name":"fighterDamage","arguments":{"_1":15}}}
<-- {"jsonrpc":"2.0","id":2,"result":{
      "content":[{"type":"text","text":"12"}],
      "isError":false}}

--> {"jsonrpc":"2.0","id":3,"method":"tools/call",
     "params":{"name":"intro","arguments":{"_1":"Goblin"}}}
<-- {"jsonrpc":"2.0","id":3,"result":{
      "content":[{"type":"text","text":"\"A wild Goblin appears!\""}],
      "isError":false}}
```

A scalar or list return is placed in a single `text` content block. A record (Map) return additionally populates `structuredContent` and the tool’s `outputSchema`, so a client that understands structured tool output gets the typed object directly while still having the text mirror.

Omitted optional arguments fall back to their declared defaults, and an omitted record field is filled from its default — the client only has to supply what it wants to override.

## 7.5.5. What is not exposed

Some functions cannot be served correctly over a stdio JSON-RPC channel, so they are dropped from the tool surface (a note is written to stderr explaining why). The remaining tools are unaffected.

| Excluded when the function…​ | …​because |
| --- | --- |
| reads from `@stdin` (`--' stdin: true`) | stdin is the JSON-RPC input stream; the two cannot share it. |
| streams to `@stdout` | stdout is the JSON-RPC output stream. |
| has an Arrow `Table` argument or return | a `Table` has no JSON representation in either direction. |
| has a stream-handle argument or return (`IFile` / `IStream` / `OStream`) | a live handle cannot be marshalled as a JSON value. |

> **Note**
> Ordinary output from a function — a `print` in a Python pool, a `std::cout` in C++ — is **not** a problem. The server re-homes its own standard output before any pool starts, so stray writes land on stderr and can never corrupt the protocol stream.

## 7.5.6. Errors

The server distinguishes a malformed **request** from a failed **execution**. A bad request is a JSON-RPC error; a function that runs and fails is a normal result flagged with `isError`, so the agent can read the message and react rather than seeing the whole call rejected.

| Condition | Response |
| --- | --- |
| Unknown method | JSON-RPC error `-32601` (method not found) |
| Unknown tool, or missing / unexpected / wrong-typed arguments | JSON-RPC error `-32602` (invalid params) |
| The function raises (`@throw`), a pool errors, or a call fails | A result with `"isError": true` and the message in a `text` block |

```
--> {"jsonrpc":"2.0","id":4,"method":"tools/call",
     "params":{"name":"fireball","arguments":{}}}
<-- {"jsonrpc":"2.0","id":4,
     "error":{"code":-32602,"message":"unknown tool 'fireball'"}}
```

## 7.5.7. Connecting an MCP client

Point an MCP client at the compiled program. Most clients take a command and its arguments; give them the runtime in `mcp` mode against the program’s absolute manifest path. For a Claude Desktop-style configuration:

**mcp client configuration**

```
{
  "mcpServers": {
    "combat": {
      "command": "/absolute/path/to/morloc-nexus",
      "args": ["mcp", "/absolute/path/to/combat-build/manifest.json"]
    }
  }
}
```

You do not have to write this by hand: the launcher emits exactly this entry as pure JSON on stdout, ready to redirect into a client config file.

```console
$ ./combat --mcp-config > combat.mcp.json
```

MCP clients launch servers with a minimal `PATH`, so the `command` is an absolute path to `morloc-nexus`. For Claude Code, `claude mcp add combat — /absolute/path/to/morloc-nexus mcp /absolute/path/to/combat-build/manifest.json` registers the same entry.

The client launches the server, runs the handshake, calls `tools/list`, and surfaces `rollAdv`, `fighterDamage`, and `intro` to the model as callable tools. When the client disconnects (stdin closes) the server shuts its pools down and exits.

## 7.5.8. Inside versus outside the container

The configuration above assumes the client can **launch** the program — it runs wherever morloc is installed. For a containerized deployment that means the agent runs **inside** the same container as morloc: it points at the program’s absolute path and speaks stdio directly, with nothing to bridge. This is the simplest and most direct way to expose a module, and it is the baseline the networked cases build on.

When the agent runs **outside** the container it cannot launch the in-container program directly. Reach it over HTTP instead: `mim start` runs a serving front-end that answers MCP over HTTP at `POST /mcp` (the same handshake and `tools/list` / `tools/call` messages, carried in HTTP request bodies with a session header), and the client is registered with a URL rather than a command:

```console
$ mim start --mcp combat
$ claude mcp add --transport http combat http://127.0.0.1:9000/mcp
```

The same front-end also serves a plain JSON API (`POST /call/<module>/<command>` with positional arguments, discovered at `GET /discover`) for non-MCP HTTP clients on the same port. The HTTP transport, authentication, sessions, and the serving lifecycle (`install` → `expose` → `start`) are covered in the [Building APIs](https://morloc-project.github.io/docs/apis/index.md) and [`mim` (Morloc Installation Manager)](https://morloc-project.github.io/docs/utilities/mim.md) chapters.

## 7.5.9. Summary

| Aspect | Detail |
| --- | --- |
| Invocation (local) | `morloc-nexus mcp <manifest>`; config via `./prog --mcp-config` |
| Invocation (networked) | `mim start` (MCP over HTTP at `/mcp`, plus the JSON API) |
| Transport | JSON-RPC 2.0 over stdio, or over HTTP (protocol version `2025-06-18`) |
| Tools | One per exported function; `description` from the docstring |
| Positional keys | `_1`, `_2`, …​ (reserved, collision-free) |
| Option / flag keys | the `--long` / flag name |
| Return | `text` block; records also get `structuredContent` + `outputSchema` |
| Static preview | `./combat --mcp-tools` prints the `tools/list` payload |


---

<!-- https://morloc-project.github.io/docs/runs/ -->

# 8. Managing Runs

Morloc Manual | https://morloc-project.github.io/docs/runs/ | prev: https://morloc-project.github.io/docs/apis/mcp.md | next: https://morloc-project.github.io/docs/runs/logging.md

A morloc program is an executable that can dispatch work across multiple language pools and, optionally, remote compute nodes. "Managing runs" covers the observability and persistence surface around one invocation of that executable: emitting per-step log lines, finding where those logs are stored on disk, and inspecting after the fact what ran.


---

<!-- https://morloc-project.github.io/docs/runs/logging.html -->

# 8.1. Logging

Morloc Manual > Managing Runs | https://morloc-project.github.io/docs/runs/logging.html | prev: https://morloc-project.github.io/docs/runs/index.md | next: https://morloc-project.github.io/docs/runs/benchmarking.md

Morloc programs can emit per-call log lines around any labeled term in source. Logging is opt-in — a term emits start, pass, and fail messages only after the user wires it up in the program’s YAML config.

The log lines go to stderr, so the program’s stdout (the computed result and any user-printed data) is unaffected. Templates are user-controlled and may include ANSI color codes; colors are automatically stripped when stderr is not a terminal so log files and pipes never contain control bytes.

## 8.1.1. Enabling logging

Two things turn logging on for a term:

1.  The term must be labeled in source code. A labeled term is written `label@term`. For example, `a@map` is the term `map` with label `a`. Labels are per-call-site, so `(a@foo x, b@foo y)` labels two distinct invocations of the same `foo`.
2.  The label must appear in the program’s `.yaml` config under `labeled-groups` with `log: true`. The config lives next to the `.loc` source: for `main.loc`, the config is `main.yaml`.

A minimal config:

```yaml
labeled-groups:
  big: { log: true }
```

A label group can be applied to many terms (`big@read`, `big@parse`, `big@save`); all of them log under the same group.

## 8.1.2. Template placeholders

The compiler emits up to three lines per labeled call — **start** (entry), **pass** (success), and **fail** (exception). Each line’s text is a user template with `{placeholder}` substitutions. The default template is:

```yaml
log-template:
  start: "[{date}] {module}:{line}:{name}:{lang} start"
  pass:  "[{date}] {module}:{line}:{name}:{lang} pass (time={runtime})"
  fail:  "[{date}] {module}:{line}:{name}:{lang} fail (time={runtime})"
```

Templates resolve in this order, per subfield: per-label override > program-wide `log-template` (top of the config) > built-in default. A `null` subfield silences that event. A common case is to silence the verbose start and pass messages and keep only the failure trace:

```yaml
log-template:
  start: null
  pass:  null
  fail:  "{date} {module}:{line}:{name} FAILED (time={runtime})"
```

Setting all three subfields to `null` while keeping `log: true` is rejected at compile time as contradictory (use `log: false` instead).

Available placeholders:

| Placeholder | Value |
| --- | --- |
| `{name}` | The labeled term’s identifier in source (e.g. `map` for `big@map`). |
| `{group}` | The label group name (e.g. `big` for `big@map`). |
| `{lang}` | The pool language: `py`, `cpp`, `r`, etc. |
| `{module}` | The source file path of the labeled reference. |
| `{line}` | Line number of the labeled reference. |
| `{column}` | Column number of the labeled reference. |
| `{index}` | The manifold ID assigned by the compiler. Useful for cross-referencing with `morloc dump` output. |
| `{date}` | UTC ISO 8601 timestamp at the moment of emission, second resolution (e.g. `2026-06-08T16:59:59Z`). |
| `{runtime}` | Elapsed time in seconds since the call’s start, with microsecond precision. Pass `0.0` at the start event. |
| `{id}` | Call id pairing a start with its pass/fail. Format `{pid}:{counter}`; unique within a pool process. |

Unknown placeholder names are a compile-time error citing the file and line of the offending YAML entry.

## 8.1.3. Color codes

A `{c:NAME}` placeholder expands to the corresponding ANSI SGR escape sequence at compile time. Apply a color, render the text, then reset with `{c:reset}`:

```yaml
start: "{c:blue}{name}{c:reset} start"
```

Attributes and clears:

| Placeholder | Effect |
| --- | --- |
| `{c:reset}` | Reset every attribute to terminal default. |
| `{c:bold}` | Bold / increased intensity. |
| `{c:dim}` | Faint. |
| `{c:italic}` | Italic. |
| `{c:underline}` | Underline. |
| `{c:blink}` | Slow blink. |
| `{c:rapid-blink}` | Rapid blink. Spotty terminal support; prefer `{c:blink}`. |
| `{c:reverse}` | Swap foreground and background. |
| `{c:hidden}` | Conceal text (still occupies space). |
| `{c:strike}` | Strikethrough. |
| `{c:no-bold}` | Cancel bold (ANSI conflates with dim; see `{c:no-dim}`). |
| `{c:no-dim}` | Cancel dim (same SGR code as `{c:no-bold}`). |
| `{c:no-italic}` | Cancel italic. |
| `{c:no-underline}` | Cancel underline. |
| `{c:no-blink}` | Cancel blink. |
| `{c:no-reverse}` | Cancel reverse. |
| `{c:no-hidden}` | Cancel conceal. |
| `{c:no-strike}` | Cancel strikethrough. |

Foreground colors:

| Standard | Bright |
| --- | --- |
| `{c:black}` | `{c:bright-black}` (alias: `{c:gray}`, `{c:grey}`) |
| `{c:red}` | `{c:bright-red}` |
| `{c:green}` | `{c:bright-green}` |
| `{c:yellow}` | `{c:bright-yellow}` |
| `{c:blue}` | `{c:bright-blue}` |
| `{c:magenta}` | `{c:bright-magenta}` |
| `{c:cyan}` | `{c:bright-cyan}` |
| `{c:white}` | `{c:bright-white}` |
| `{c:default}` | — |

Background colors:

| Standard | Bright |
| --- | --- |
| `{c:bg-black}` | `{c:bg-bright-black}` |
| `{c:bg-red}` | `{c:bg-bright-red}` |
| `{c:bg-green}` | `{c:bg-bright-green}` |
| `{c:bg-yellow}` | `{c:bg-bright-yellow}` |
| `{c:bg-blue}` | `{c:bg-bright-blue}` |
| `{c:bg-magenta}` | `{c:bg-bright-magenta}` |
| `{c:bg-cyan}` | `{c:bg-bright-cyan}` |
| `{c:bg-white}` | `{c:bg-bright-white}` |
| `{c:bg-default}` | — |

## 8.1.4. Terminal detection and NO\_COLOR

Color codes from `{c:…​}` placeholders (or raw ANSI escapes a user writes directly into a template) are emitted unchanged when stderr is a terminal **and** the environment variable `NO_COLOR` is unset. In every other case — stderr redirected to a pipe or file, or `NO_COLOR` set to any value — the runtime strips all CSI sequences before writing, so the output is plain text.

This means the same program produces colored output when run interactively:

```bash
./main '[[1,2],[3,4,5]]'      # colors emitted to terminal
```

and plain text when piped or redirected:

```bash
./main '[[1,2],[3,4,5]]' 2> run.log    # run.log is plain text
./main '[[1,2],[3,4,5]]' 2>&1 | grep pass    # grep sees plain text
```

To suppress color even in an interactive terminal (e.g. for screen captures, color-blind users, or terminals with non-standard palettes), set `NO_COLOR`:

```bash
NO_COLOR=1 ./main '[[1,2],[3,4,5]]'
```

`NO_COLOR` follows the convention at [no-color.org](https://no-color.org/): any non-empty value disables color; the variable being unset means color is allowed.

## 8.1.5. Worked example

The source labels two terms — `a@map` and `b@sum` — in a small two-stage pipeline that sums each inner list:

```morloc
module main (foo)

import root-py

sum :: [Real] -> Real
sum = fold (+) 0.0

foo :: [[Real]] -> [Real]
foo = a@map b@sum
```

The config enables both labels and uses a maximalist template that exercises every color category and every placeholder:

```yaml
log-template:
  start: "{date} {c:gray}{module}:{line}:{column}:{lang}{c:reset} {name}:{id}: {c:blue}start{c:reset}"
  pass:  "{date} {c:gray}{module}:{line}:{column}:{lang}{c:reset} {name}:{id}: {c:green}pass{c:reset} {c:grey}(time: {runtime}){c:reset}"
  fail:  "{date} {c:gray}{module}:{line}:{column}:{lang}{c:reset} {name}:{id}: {c:red}fail{c:reset} {c:grey}(time: {runtime}){c:reset}"

labeled-groups:
  a: { log: true }
  b: { log: true }
```

Build and run on a two-row input:

```bash
morloc make -o main main.loc
./main '[[1,2],[3,4,5]]'
```

Output to stderr, rendered with the colors a terminal would show (one start/pass pair per call; `a@map` brackets two inner `b@sum` calls; the nested call ids `0`, `1`, `2` pair start lines with their corresponding pass lines):

```stderr
2026-06-08T16:59:59Z main.loc:9:7:py map:33804:0: start
2026-06-08T16:59:59Z main.loc:9:13:py sum:33804:1: start
2026-06-08T16:59:59Z main.loc:9:13:py sum:33804:1: pass (time: 0.000023)
2026-06-08T16:59:59Z main.loc:9:13:py sum:33804:2: start
2026-06-08T16:59:59Z main.loc:9:13:py sum:33804:2: pass (time: 0.000009)
2026-06-08T16:59:59Z main.loc:9:7:py map:33804:0: pass (time: 0.000520)
```

Output to stdout (the actual program result):

```
[3,12]
```

Run the same command with `NO_COLOR=1` for plain output even when stderr is a terminal:

```bash
NO_COLOR=1 ./main '[[1,2],[3,4,5]]'
```


---

<!-- https://morloc-project.github.io/docs/runs/benchmarking.html -->

# 8.2. Benchmarking

Morloc Manual > Managing Runs | https://morloc-project.github.io/docs/runs/benchmarking.html | prev: https://morloc-project.github.io/docs/runs/logging.md | next: https://morloc-project.github.io/docs/runs/run-directory.md

`log: true` reports every call as it happens. A benchmark wants the opposite: not a line per iteration but one row per label summarising every call the run made. That is `benchmark: true`.

```yaml
labeled-groups:
  parse: { benchmark: true }
```

```console
$ ./main run big.fasta
parse:readSeqs [cpp] n=2000 mean=0.000431 min=0.000298 max=0.004117
```

The two settings are independent and compose: a group may log, benchmark, both, or neither.

Timing is per-manifold, so process startup and pool spawn are outside the measurement. Because the label follows the manifold into whatever pool realises it, `{lang}` tells you which language actually ran the work — and a labeled call that crosses a pool boundary is measured on the **caller** side, so its row includes the round trip.

Only successful calls are recorded. A call that raised did not do the work being measured, and folding its duration into the mean would report a number that describes nothing.

## 8.2.1. The summary row

The row shape is the program-wide `benchmark-template`, whose one subfield is `summary`. Unlike `log-template` it is not per-label: the nexus aggregates every label’s timings and renders them through a single template.

The built-in default is readable rather than machine-shaped:

```yaml
benchmark-template:
  summary: "{group}:{name} [{lang}] n={count} mean={mean} min={min} max={max}"
```

A suite that wants columns overrides it. Rows go to stderr, like every other morloc log line, so the program’s own output is unaffected:

```yaml
benchmark-template:
  summary: "{group}\t{name}\t{lang}\t{count}\t{mean}\t{stddev}"
```

```console
$ ./main run big.fasta 2> bench.tsv
```

Available placeholders:

| Placeholder | Value |
| --- | --- |
| `{group}` | The label group name. |
| `{name}` | The labeled term’s identifier in source. |
| `{lang}` | The pool language that ran the calls. |
| `{count}` | Number of successful calls recorded. |
| `{mean}` | Arithmetic mean of the durations, in seconds. |
| `{min}` | Fastest recorded call, in seconds. |
| `{max}` | Slowest recorded call, in seconds. |
| `{total}` | Sum of all durations, in seconds. |
| `{stddev}` | Sample standard deviation; `0.000000` for a single call. |

Rows are ordered by `{group}`, then `{name}`, then `{lang}` — not by arrival. A benchmark exists to be compared against another run, and arrival order is not reproducible.

Setting `benchmark: true` while nulling `summary` is rejected at compile time: the timings would be collected and never reported.

## 8.2.2. Comparing implementations

Because labels are per-call-site, the same work measured under two labels yields two rows. That is the shape of an A/B comparison — one run, one input, the same warm pools:

```morloc
module cmp (compare2)

import root-py
import root-cpp

source Py from "lib.py" ("incr")
source Cpp from "lib.hpp" ("triple")

incr :: Int -> Int
triple :: Int -> Int

compare2 :: Int -> (Int, Int)
compare2 x = (slow@incr x, fast@triple x)
```

```console
$ ./cmp compare2 5
[6,15]
fast:triple [cpp] n=1 mean=0.000003 min=0.000003 max=0.000003
slow:incr [cpp] n=1 mean=0.000662 min=0.000662 max=0.000662
```

Both rows say `cpp`, including the one for the Python `incr`. That is the caller-side rule at work: this program is rooted in the C++ pool, so the call into Python is measured where it is made, and the 0.000662 seconds is the round trip rather than the addition. Measuring the Python side in isolation means rooting the program there instead.

Note also that `fast` precedes `slow` in the output despite running second. Rows are sorted, not logged.


---

<!-- https://morloc-project.github.io/docs/runs/run-directory.html -->

# 8.3. Run directory

Morloc Manual > Managing Runs | https://morloc-project.github.io/docs/runs/run-directory.html | prev: https://morloc-project.github.io/docs/runs/benchmarking.md | next: https://morloc-project.github.io/docs/runs/caching.md

> **Warning: Experimental Feature**
> The run directory’s contents are not settled. What a run writes today is narrower than what this section describes: a run matching the layout below produces the per-label directories and `summary.json`, but the top-level `log` appears only when a prologue writes to it. The compiler’s own test asserts a `start.json` that nothing produces.
> 
> Treat the directory as a place to find logs, not as a stable on-disk format to parse. The file names and the layout may change.

Every persistent-logging invocation of a morloc-built executable is one **run**. The run gets a unique id and a directory on disk holding the run’s artifacts: a top-level log file teed from stderr, per-label log files for `log: true` labels, and a `summary.json` sentinel.

The directory is opt-in. A bare `./my_program …​` invocation never creates one; pass `--log-dir <path>` (or set `MORLOC_LOG_DIR`) to activate. Interactive use stays clutter-free; cron jobs and servers opt in and get the structured artifacts.

The directory layout, after a run with `--log-dir runs/` and at least one labeled term with `log: true`, looks like:

```
 . runs/20260608T172304Z-a3f9c41b/
 |
 |-- a
 |   `-- log
 |-- b
 |   `-- log
 |-- log
 `-- summary.json
```

The run id is `{utc-iso8601-second}-{8-hex-random}` — lexically sortable, so `ls` orders runs chronologically. The top-level `log` is the prologue / epilogue / per-label-line tee; per-label `log` files hold only the per-label start / pass / fail emissions for that label.

## 8.3.1. Activation knobs

| Knob | Effect |
| --- | --- |
| `--log-dir PATH` (or `MORLOC_LOG_DIR=PATH`) | Creates a per-run subdir under `PATH`, tees stderr log lines into `PATH/<run_id>/log`, and writes `PATH/<run_id>/summary.json` at exit. Without this flag, none of these files exist — the run is pure stderr. |
| `--summary FILE` (or `MORLOC_SUMMARY=FILE`) | Writes the structured `summary.json` to `FILE`. Independent of `--log-dir`: an orchestrator that just wants a completion sentinel can use this alone without committing to a rundir of log files. When both are set, the explicit `--summary` path wins. |
| `--quiet` (or `MORLOC_QUIET=1`) | Suppress **all** morloc-emitted log lines — prologue, epilogue, per -label start / pass / fail — at the source. Lines are never generated, so they neither hit stderr nor tee into the rundir’s `log` file. `summary.json` is still written when `--log-dir` or `--summary` is active: the sentinel survives the silence. |

## 8.3.2. summary.json

Presence of `summary.json` means the run reached a clean exit (good or bad). The fields are minimal and stable:

```json
{
  "status": "ok",
  "exit_code": 0,
  "command": "align_reads",
  "run_id": "20260608T172304Z-a3f9c41b",
  "started_at": "2026-06-08T17:23:04Z",
  "finished_at": "2026-06-08T17:25:18.231Z",
  "wall_ms": 134231,
  "morloc_version": "0.88.0",
  "error": null
}
```

On a failing run, `status` is `"fail"`, `exit_code` is nonzero, and `error` carries the error packet’s message (often a foreign-language traceback). The write is atomic (tmp + rename) so a watcher polling for the file never sees a partial JSON.

A wrapper that runs morloc as a workflow step can poll for `--summary` existence to detect completion and read `status` to branch:

```bash
./align --summary $WORK/align.summary.json --log-dir $WORK/logs @ \
        reads.fastq.gz reference.fa
case "$(jq -r .status $WORK/align.summary.json)" in
  ok)   ./next_step ;;
  fail) echo "align failed: $(jq -r .error $WORK/align.summary.json)" ;;
esac
```

SIGKILL / OOM / kernel panic bypass the writer; the wrapper should have a timeout fallback for those.

## 8.3.3. Where the directory lives

Resolution order, highest precedence first:

| Source | Notes |
| --- | --- |
| `--log-dir PATH` / `MORLOC_LOG_DIR=PATH` | Activation knob **and** base directory. The run lands at `PATH/<run_id>/`. |
| Inheritance from a parent morloc process | If a morloc-built program launches another morloc-built program, the child reuses the parent’s run dir so logs interleave naturally. The check requires the parent’s owning PID to match `getppid()`, defeating stale shell-exported `MORLOC_RUN_DIR` values. |

There is no fallback default base directory: persistent logging is strictly opt-in. A `MORLOC_RUN_DIR` set without the matching `MORLOC_RUN_PARENT_PID` is treated as stale and ignored.

## 8.3.4. Cleanup

Morloc never deletes a past run directory. Old runs accumulate under the base until you remove them. A simple housekeeping cron (or a one-off `find <log-base> -mtime +30 -delete`) is sufficient.

## 8.3.5. Prologue and epilogue

Two top-level YAML keys add run-scope log entries. They behave like the per-label `log-template`: always emitted to stderr when defined, tee’d to the rundir’s `log` when `--log-dir` is active, suppressed entirely under `--quiet`.

```yaml
prologue: "[{c:bold}morloc{c:reset}] {name} v{version} start {started_at}"
epilogue:
  ok:   "[morloc] {name} ok in {runtime}s"
  fail: "[morloc] {name} FAILED ({exit_code}) in {runtime}s: {error}"
```

Two epilogue branches so the success line doesn’t have to render an empty `{error}` and the failure line can include fields the success line lacks. The compiler picks the matching branch based on the run’s exit status.

Available placeholders:

| Placeholder | Where it comes from |
| --- | --- |
| `{module}` | Entry-point morloc module name. Substituted at compile time. |
| `{version}` | Program version from `package.yaml` (or `?` if absent). Compile time. |
| `{morloc_version}` | Compiler version. Compile time. |
| `{name}` | Subcommand the user invoked. Runtime. |
| `{run_id}` | Per-run unique id. Empty when no rundir is materialized. Runtime. |
| `{started_at}` / `{finished_at}` | ISO 8601 timestamps. Runtime. |
| `{runtime}` | Wall seconds, six decimal places. Runtime (epilogue only). |
| `{pid}` | Nexus PID. Runtime. |
| `{hostname}` | `gethostname(2)` result. Runtime. |
| `{exit_code}` | Integer exit code. Runtime (`fail` epilogue only). |
| `{error}` | Error packet contents (may be multi-line). Runtime (`fail` epilogue only). |
| `{c:red}` / `{c:bold}` / `{c:reset}` / …​ | ANSI color codes. Compile time. The runtime strips them when stderr is not a TTY or `NO_COLOR` is set. |

## 8.3.6. Nested invocations

If a morloc-built program launches another morloc-built program, the child **inherits** the parent’s run directory when the parent activated one. Both programs' logs land under the same run id, so the user’s `tail -f` / `grep` tooling sees the full workflow as one entity rather than two. Pool processes are children of the nexus and use the same mechanism, which is why every pool’s log emission ends up in the expected per-label log file.

The inheritance check is robust against a stale `MORLOC_RUN_DIR` left over in a shell environment from a previous run: the child only inherits when the run-dir’s owning PID matches its actual parent. A mismatched PID falls back to no run dir (the child is then a normal, non-persistent invocation).


---

<!-- https://morloc-project.github.io/docs/runs/caching.html -->

# 8.4. Caching

Morloc Manual > Managing Runs | https://morloc-project.github.io/docs/runs/caching.html | prev: https://morloc-project.github.io/docs/runs/run-directory.md | next: https://morloc-project.github.io/docs/runs/compression.md

Mark any labeled call site `cache: true` in the program YAML and its result is memoized to disk. The next call with equivalent inputs against unchanged source code is served from the cache rather than recomputed.

The freshness check is content-based, not time-based. Morloc has **no** notion of "the source file is newer than the cache entry, so re-run." Editing a comment in an unrelated function will not invalidate any cache. Copying the program to a new path will not either. Two builds on two machines that emit byte-identical pool sources share the same cache namespace. This is a deliberate departure from `make`, `snakemake`, `nextflow`, and similar tools that use mtime as a freshness signal and routinely re-run the world after a `git checkout` or a clock skew.

The build parameters passed with `-X` (see the build parameters section) also participate in the key, since they can change the compiled output without changing the pool source — switching a Futhark backend or adding a compiler flag is treated as a distinct build rather than a cache hit.

## 8.4.1. Declaring a cached call

```yaml
labeled-groups:
  expensive_step: { cache: true }
```

```morloc
foo xs = expensive_step@slowfn xs
```

Every call into `expensive_step@slowfn` is memoized under the `expensive_step` cache label. The same group config also controls per-step logging (`log: true`); the two flags are independent and may be combined.

## 8.4.2. What goes in the hash

A cached entry is keyed by:

```
call_key = xxh64(pool_source_fingerprint, midx, arg_content_hashes ...)
```

where:

-   **`pool_source_fingerprint`** is `xxh64` of the rendered pool source text, seed-chained over the contents of any files listed under `hash-include:` in the program YAML. Editing the body of the cached function, of any function it calls, of any imported module that compiled into the same pool, or of any declared external data file, all shift this fingerprint.
-   **`midx`** is the compiler-assigned manifold id, deterministic per build.
-   **`arg_content_hashes`** are content-aware hashes of each argument’s **value**, walked through its msgpack schema. Two structurally equal inputs hash the same regardless of how their packet stored the data (inline bytes, shared-memory pointer, or temp file), and pointer bits are never themselves hashed.

The freshness test is therefore: same code + same code dependencies + same input values → cache hit. Anything else → miss.

## 8.4.3. Storage layout

The cache lives under one of these directories, in resolution order:

| Source | Notes |
| --- | --- |
| `MORLOC_CACHE_BASE` env var | Explicit override. Useful for Docker bind mounts and shared filesystems where SLURM workers need access to the same cache. |
| `$XDG_CACHE_HOME/morloc/cache` | If `XDG_CACHE_HOME` is set. |
| `~/.cache/morloc/cache` | Default. |

Inside, two file types coexist:

```
 . ~/.cache/morloc/cache/
 │
 ├── expensive_step/
 │   ├── 3f4a91d62b08e7d2.packet
 │   └── b772aa1c0e4ef801.packet
 ├── another_label/
 │   └── ...
 └── data/
     ├── ef46db3751d8e999.dat
     └── ...
```

-   **`<label>/<call_key>.packet`** is a small morloc data packet (`PACKET_TYPE_DATA`, source=FILE, format=MSGPACK) that points to a dat file under `data/`. One entry per distinct cached call.
-   **`data/<data_hash>.dat`** holds the cached value as schema-agnostic msgpack bytes. The filename is `xxh64` of those bytes, so equal return values consolidate onto one on-disk copy regardless of which label cached them or which pool language produced them.

The content-addressed split gives natural deduplication: a Python pool and an R pool that both cache the same dataset (say, a genome pulled from an external database) write the data once. A source edit that produces the same return value writes a new `.packet` pointer but reuses the existing `.dat`.

## 8.4.4. Reading and writing

On lookup, the runtime reads the `.packet` file and returns its bytes; the language pool’s `get_value` follows the FILE-source pointer through to the dat file. The on-disk format is opaque to the language pool — it sees only bytes that round-trip through the standard packet API, so future format changes (inline-small optimization, compression, alternative backends) require no recompile of pool code.

On store, the runtime materializes the input packet to msgpack (dereferencing any shared-memory or file-backed payload), hashes those bytes to choose the dat filename, skips the write if an identical dat already exists, and finally writes the small pointer packet under the label dir. Both writes go through an atomic tmp  
rename, so a partial file is never observed by another reader and concurrent writers to the same key cannot corrupt the on-disk state.

A cached call whose foreign body raises is **not** memoized — the cache wrap only stores on the happy path. A re-run with the same inputs fails the same way rather than silently serving a stale entry.


---

<!-- https://morloc-project.github.io/docs/runs/compression.html -->

# 8.5. Compression

Morloc Manual > Managing Runs | https://morloc-project.github.io/docs/runs/compression.html | prev: https://morloc-project.github.io/docs/runs/caching.md | next: https://morloc-project.github.io/docs/runs/debugging.md

Morloc data packets carry a compression byte in their header, so any packet written to disk can transparently round-trip through a compressor. Two surfaces expose this: the `@write` intrinsic accepts a per-call compression level for each `OStream` sub-packet, and the nexus accepts a `-z` flag that compresses whatever result it would normally write to `-o <file>` as a packet.

The pipeline is end-to-end: a compressed packet on disk is recognized by `@load` and by every `IFile` / `IStream` open, so the user never has to call a decompress step. Pool processes never see compressed bytes — decompression happens at the I/O boundary.

## 8.5.1. Compression in `@write`

`@write` takes a compression preset as its first argument. The preset is an integer in the range `0..=9`; `0` writes the sub-packet uncompressed and the other values map to zstd presets that trade speed for ratio.

```morloc
module main (writeBig)

import root-py (id)

writeBig :: [Int] -> Str -> <IO> ()
writeBig xs path = do
  Ok o <- @open path :: <IO> (Try Str (OStream Int))
  @write 5 o (id xs)
  @close o
```

The level is set per `@write` call, not per stream: each call chooses its own preset independently. Reading is unchanged — `@open path :: <IO> (Try Str (IFile Int))` (or `IStream`) inspects each sub-packet’s header and decompresses on the fly. The same call site reads both compressed and uncompressed sub-packets without any syntactic difference.

The other save intrinsics (`@savem` for raw MessagePack, `@savej` for raw JSON) do not produce packets and so do not take a compression level. Compressing those file shapes is a separate feature and is not covered here.

## 8.5.2. Compression in `morloc-nexus run`

A nexus run that writes its result as a packet (`-f packet -o foo.packet`) can compress that packet with `-z N`:

```bash
$ ./nexus -f packet -o result.packet -z 5 myCommand args...
```

`-z` defaults to `0` (no compression). The long form is `--compression-level`. The flag is silently a no-op for non-packet output formats (JSON, MessagePack as a raw file, voidstar, Arrow, Parquet, CSV); for those the file shape itself has no header in which to record a compression algorithm.

A packet written with `-z N` on one host is readable by `@load` (and by any future packet-reading surface) on any other host without an additional flag. The compression byte in the header is the only signal needed.

## 8.5.3. Compression-level table

The 0—​9 range is deliberately algorithm-agnostic: it spans "no compression" through "fast" through "archive-grade" without committing the user to a specific codec. Under the hood every non-zero level currently maps to a zstd preset, with long-range mode enabled for the top tiers:

| `-z N` | zstd level | long mode | Use case |
| --- | --- | --- | --- |
| 0 | — | — | No compression (default) |
| 1 | 1 | no | fastest; around 2x faster than level 2 |
| 2 | 3 | no | good balance, zstd’s default |
| 3 | 6 | no | better compression, still fast |
| 4 | 9 | no |  |
| 5 | 12 | no | high ratio, moderate cost |
| 6 | 15 | no | very high compression |
| 7 | 19 | yes | archive-grade; long-mode finds more distant repeats |
| 8 | 21 | yes |  |
| 9 | 22 | yes | maximum compression; very slow |

Multithreaded compression kicks in automatically once a payload exceeds 1 MiB. The worker count scales with payload size up to the number of cores available (capped at 16); below 1 MiB the encoder is single-threaded so small-packet latency is unaffected. Decompression is single-threaded by zstd’s frame-format design and needs no tuning.

## 8.5.4. Algorithm-agnostic surface

The compression level is an *abstract* knob. The morloc nexus produces and consumes its own compressed packets, so the underlying algorithm is an implementation detail. zstd is the current choice (modern Pareto winner on speed vs. ratio), but a future release may switch to a different codec without changing the `-z 0..9` semantics or the `@write` signature. The packet header records the algorithm it was written with, so old packets remain readable across algorithm changes.

The intent of preset N is stable: `1` is "fastest", `9` is "maximum ratio", and intermediate values increase ratio monotonically. The exact zstd levels in the table above may shift between releases as codec defaults evolve, but the user-facing meaning will not.

## 8.5.5. Caching is unaffected

Cache keys are content-based and hash the **uncompressed** value (see [Caching](https://morloc-project.github.io/docs/runs/caching.md)). This is deliberate:

-   A cache write and a cache lookup do not need to use the same `-z` level to share an entry. Hashing happens before compression.
-   Toggling `-z` on or off does not invalidate any cache.
-   Two pool languages that produced the same return value still share a single on-disk `.dat` payload regardless of which one compressed.

In short, compression affects bytes on disk but not the morloc-level identity of a value.


---

<!-- https://morloc-project.github.io/docs/runs/debugging.html -->

# 8.6. Debugging

Morloc Manual > Managing Runs | https://morloc-project.github.io/docs/runs/debugging.html | prev: https://morloc-project.github.io/docs/runs/compression.md | next: https://morloc-project.github.io/docs/runs/random-access-and-streaming.md

When a foreign function throws — a Python `ZeroDivisionError`, a C++ `std::invalid_argument`, an R `stop(…​)` — the default behavior of a morloc-built program is to surface the language-level error message and exit with non-zero status. That tells the operator something failed but not **why**: the inputs that triggered the throw are gone, and the stack across the dispatcher boundary doesn’t survive into the user’s terminal.

Compile-time debug-trace mode fills that gap. When enabled at build, every foreign-call manifold gains a try/catch wrap that:

1.  On exception, dumps the call’s arguments to a content-addressed file.
2.  Records a frame entry naming the manifold id and listing each arg’s schema and dump path.
3.  Re-raises the exception, unmodified.

The pool dispatcher concatenates the rendered trace into the fail packet’s error message before returning to the nexus. The trace then appears in `summary.json` and on stderr alongside the foreign-language error, so the failing inputs are recoverable from disk and the chain of calls is visible without re-running anything.

There is **zero** runtime cost in a build without `--debug`: no wraps are emitted, no per-frame state is allocated. Production binaries can ship unchanged; debugging is a recompile.

## 8.6.1. Enabling at compile time

Pass `--debug` to `morloc make`:

```shell
morloc make --debug -o my_program main.loc
```

The flag is a build-time switch only. It affects code generation, so the `--debug` and non-`--debug` variants are two physically different binaries. There is no runtime flag that re-enables tracing on a binary compiled without `--debug`.

## 8.6.2. A failing run

A trivial example: `boomP n = idpy (pyThrowAtZero n)`, where `pyThrowAtZero` raises on `n == 0`.

```shell
$ ./my_program boomP 0
Error: run failed
ZeroDivisionError: float division by zero
  at m1 (py)
morloc trace (innermost first):
  frame 0 mid=1
    arg[0] :: f8 -> .morloc-debug/inputs/e8b6f11aa7a0ccb4.pkt
```

The arg’s value is in `e8b6f11aa7a0ccb4.pkt` — a msgpack-encoded copy of `n` (`0.0` here) usable as a `@load` target in any morloc program that needs to replay the call.

For a cross-pool failure (Python calls C, C throws), each pool’s catch contributes its own frames; the trace shows both pools' state stacked with the innermost (the C++ throw site) at the top.

## 8.6.3. Runtime knobs

The compile-time wrap is unconditional, but four runtime knobs shape **what** the catch records when it fires. All four can be set on the nexus command line or via environment variables; the CLI flag wins when both are present.

| Knob | Effect |
| --- | --- |
| `--log-dir PATH` (or `MORLOC_LOG_DIR=PATH`) | Same flag that activates persistent logging (see "Run directory"). When set, debug-trace dumps land under `PATH/<run_id>/debug/inputs/` and the rendered trace appears in `summary.json`. |
| `--debug-cache-depth N` (or `MORLOC_DEBUG_CACHE_DEPTH=N`) | Maximum number of disk writes per dispatch. Default `1`. Set to `0` for unlimited. Only successful writes count, so a single huge run with one outlier arg gets that arg dumped without burning the budget on later trivial args. Limit is per-dispatch, not per-process; each fresh dispatch starts the counter at zero. |
| `--debug-cache-max BYTES` (or `MORLOC_DEBUG_CACHE_MAX=BYTES`) | Per-arg size cap on the msgpack-encoded payload. Args whose encoded bytes exceed this are recorded by hash but not written to disk. The rendered trace shows `(hash=…​, size exceeded MORLOC_DEBUG_CACHE_MAX)` for skipped args. Suffix `k`/`m`/`g` for KiB/MiB/GiB. Default `0` (unlimited). |
| `--debug-recursion-cap N` (or `MORLOC_DEBUG_RECURSION_CAP=N`) | Per-manifold-id frame limit. A recursive function that throws at the bottom of an N-deep stack would otherwise produce N frame entries for the same manifold; this cap silently drops entries past the limit and appends a one-line note that the cap was hit. Default `3`. Set to `0` for unlimited. |

## 8.6.4. Where dumps land

Resolution order for the debug-dump directory, highest precedence first:

| Source | Notes |
| --- | --- |
| `MORLOC_DEBUG_DIR=PATH` | Direct override. Dumps go to `PATH/inputs/<hash>.pkt`. Set this to collect across many runs in one shared dir. |
| `--log-dir PATH` / `MORLOC_LOG_DIR=PATH` | Composes with the rundir: dumps go to `PATH/<run_id>/debug/inputs/`, partitioned by run id. The same directory holds `log` and `summary.json`. This is the recommended setup for any reproducibility-conscious run. |
| Default fallback | `./.morloc-debug/inputs/<hash>.pkt` in the invocation CWD. The directory is created on first write; failure to create (read-only filesystem, missing CWD) degrades to a hash-only frame entry rather than an error. The dotfile keeps the working directory uncluttered for casual users. |

The default fallback exists because the wrap is a debugging tool and the most common case is "I just want to see what crashed my function" — no orchestrator, no opt-in flag, no setup. The fallback is only reached when the binary was compiled with `--debug` (no wraps fire otherwise), so non-debug builds never create `.morloc-debug/`.

## 8.6.5. Content-addressed dedup

The dump filename is the xxh64 of the msgpack-encoded arg bytes. Two args with identical content — the same `Int 42`, the same reference genome passed through five frames — write the same file once and are referenced by hash in each frame’s trace line. Disk cost scales with **distinct** inputs, not with frame count.

## 8.6.6. Frame status markers

A frame’s trace line for each arg is one of:

-   `arg[0] :: f8 → .morloc-debug/inputs/<hash>.pkt` — written to disk.
-   `arg[0] :: f8 (hash=…​, size exceeded MORLOC_DEBUG_CACHE_MAX)` — arg’s msgpack payload was over the per-arg size cap; the hash is recorded but no file was written.
-   `arg[0] :: f8 (hash=…​, depth cap reached — raise --debug-cache-depth to dump more)` — this dispatch already wrote `--debug-cache-depth` args; this one fell off the budget.
-   `arg[0] :: f8 (hash=…​, write failed — check that the debug dir is writable)` — the directory was resolvable but `mkdir` or the atomic write returned an error (read-only mount, no permission, full disk).
-   `arg[0] :: f8 (serialize failed)` — the arg’s schema couldn’t be resolved or the msgpack encoder errored. No hash is meaningful.

The four marker variants exist because the failure cause changes the right next step: a depth-cap miss is a knob adjustment; a write-failed is a filesystem problem; a size-exceeded is a per-arg vs whole-run budget tradeoff.

## 8.6.7. Recovering an input via `@load`

Each `.pkt` is a single msgpack value (the morloc wire form of the dumped arg). The morloc `@load path` intrinsic can read these directly — no separate decoder is required — because the same code path that loads `@savem`\-written files also handles bare msgpack:

```morloc
loadArg :: Str -> <IO> Int
loadArg path = do
  Ok v <- @load path
  v
```

```shell
$ ./debug loadArg .morloc-debug/inputs/e8b6f11aa7a0ccb4.pkt
0
```

If the schema of ``loadArg’s return type doesn’t match the dumped value’s schema, the load comes back as an `Err`` arm and the refutable bind above throws it. To try several shapes without crashing, bind the result and match on it, falling through to a default or to another attempt:

```morloc
loadArgOrZero :: Str -> <IO> Int
loadArgOrZero path = do
  r <- @load path :: <IO> (Try Str Int)
  match r | (Ok v) = v | (Err _) = 0
```

## 8.6.8. Interaction with the cache

Stage-3 caching and debug-trace mode are independent layers and combine naturally. A cached call that hits the cache never enters the foreign function, so no debug-trace frame fires. A cached call that misses, falls through to the foreign function, and throws produces a frame just like an uncached call would. The two systems share no state.


---

<!-- https://morloc-project.github.io/docs/runs/random-access-and-streaming.html -->

# 8.7. Random access and streaming

Morloc Manual > Managing Runs | https://morloc-project.github.io/docs/runs/random-access-and-streaming.html | prev: https://morloc-project.github.io/docs/runs/debugging.md | next: https://morloc-project.github.io/docs/utilities/index.md

Three abstract types describe a value that lives in a file rather than in memory:

| Type | Role |
| --- | --- |
| `IFile a` | Random-access reader. The file is open for indexed and pattern access; elements are decoded on demand. `a` is the type of the data in the file. |
| `OStream a` | Sequential writer. Elements are buffered, compressed, and flushed to disk as sub-packets. The output type is always of type `[a]`. |
| `IStream a` | Sequential reader. The file is walked forward one sub-packet at a time; element order is preserved. Input is always of type `[a]`. |

All three are opened with the same `@open` intrinsic, which returns an integer handle that lives in a shared SHM registry, so the handle can be passed transparently across pool boundaries.

## 8.7.1. Random access with IFile

`@open path` opens an existing morloc stream file for random access. The file’s element schema is read out of the header and must match the `IFile` parameter type; a mismatch errors at open time.

Once open, an `IFile a` is indexable and sliceable using the bracket patterns documented in [Patterns](https://morloc-project.github.io/docs/features/patterns.md):

```morloc
module main (lookup, slice)

lookup :: Str -> Int -> <IO> Person
lookup path i = do
  Ok f <- @open path :: <IO> (Try Str (IFile [Person]))
  let p = .[i] f
  @close f
  p

slice :: Str -> <IO> [Person]
slice path = do
  Ok f <- @open path :: <IO> (Try Str (IFile [Person]))
  let xs = .[100:200] f
  @close f
  xs
```

A slice that spans multiple sub-packets decompresses each sub-packet once, in parallel, and copies just the selected elements into the result. The decompressed sub-packets are cached per handle so a second access to a nearby element is a pointer add rather than another zstd pass. `@flen f` returns the total element count without scanning the file.

## 8.7.2. Sequential writing with OStream

The line

```morloc
@open path :: <IO> (Try Str (OStream a))
```

creates a new stream file. The OStream serializes elements into a write buffer; once the buffer reaches its cap, the contents flush to disk as one zstd-compressed sub-packet. Element atomicity is preserved: an element larger than the buffer flushes as its own oversize sub-packet rather than splitting across boundaries.

```morloc
module main (writeMany)

source Py from "compute.py" ("produceBatch")
produceBatch :: Int -> [Person]

writeMany :: Str -> Int -> <IO> ()
writeMany path n = do
  Ok o <- @open path :: <IO> (Try Str (OStream Person))
  @write 3 o (produceBatch n)
  @write 3 o (produceBatch n)
  @flush o
  @write 3 o (produceBatch n)
  @close o
```

`@write level o xs` appends `xs` to `o` using `level` as the zstd preset for the resulting sub-packet (`0` disables compression; the preset table is the same as for [Compression](https://morloc-project.github.io/docs/runs/compression.md)). `@flush o` forces the buffer to disk as a sub-packet boundary; without it, a partially-filled buffer is held until the next write that overflows it, or until `@close`. `@close` writes the final footer (full sub-packet index, element count, end-of-file marker) and releases the slot; the file is now readable by `IFile` or `IStream`.

`@append path` opens a stream file for further writes, creating it if it is not there yet, so an append-only log needs nothing else to start one. The element schema is checked against the file’s recorded schema and a mismatch errors before any byte is written.

`@concat paths dest` byte-level concatenates a list of compatible stream files into `dest` using `sendfile`, with no userspace copy. It builds beside `dest` and renames onto it, so `dest` may appear in `paths` — adding a batch to a log is the ordinary use — and a merge that fails leaves the old `dest` as it was. Refusing an aliased destination was the alternative; it would have forbidden that use and still left a failed merge free to delete a file it never created.

## 8.7.3. Sequential reading with IStream

`@open path` opens a stream file for forward reads. `@next s` returns the next sub-packet’s elements as a list; when the file is exhausted, `@next` returns `[]`. The cursor advances under the slot’s futex, so two pools holding the same IStream handle can take turns calling `@next` and each gets a distinct sub-packet.

```morloc
module main (drain)

drain :: Str -> <IO> U64
drain path = do
  Ok s <- @open path :: <IO> (Try Str (IStream Int))
  Ok a <- @next s
  Ok b <- @next s
  Ok c <- @next s
  @close s
  size a + size b + size c
```

`@stream f` derives a fresh IStream from an open `IFile a`. The derived stream has its own slot, fd, and cursor, so walking it does not perturb the IFile’s random-access state. The underlying file is the same — closing the IFile invalidates the derived IStream’s next read, which returns a generation-mismatch error.

`@open` also reads standard input, not just files. Opening the path `/dev/stdin` — the value a `--' @stdin` CLI argument takes when it is omitted, and the target of a `-` argument — routes an `IStream` to the process’s stdin, exactly like `@stdin`. Because a pipe is not seekable, `@open "/dev/stdin" :: IFile a` instead returns an `Err` arm without reading any bytes, so a reader can attempt fast random access first and fall back to a sequential `IStream` for stdin (see [Reading a stream from standard input](https://morloc-project.github.io/docs/clis/reading-stdin.md) for the CLI side). Compressed sub-packets are decompressed on demand at `@next`; a non-morloc input on stdin is rejected rather than mis-decoded.

## 8.7.4. Typed standard streams

`@stdin`, `@stdout`, and `@stderr` are nullary intrinsics that expose the process’s standard streams as typed morloc streams. Their element types are fixed by inline ascription at the open site, exactly like `@open`:

| Intrinsic | Type |
| --- | --- |
| `@stdin` | `<IO> (Try Str (IStream a))` |
| `@stdout` | `<IO> (OStream a)` |
| `@stderr` | `<IO> (OStream a)` |

Once opened, they support the same `@next` / `@write` / `@flush` / `@close` API as their file-backed cousins. The point is type-safe stream IO between morloc programs: the compiler checks the connection at the source level, and the runtime validates each sub-packet’s wire schema on arrival. Piped morloc programs can carry structured data without falling back to ad-hoc text formats.

The canonical Unix-filter shape:

```morloc
module main (producer, doubler)

import root-py

producer :: <IO> ()
producer = do
  o <- @stdout :: <IO> (OStream Int)
  @write 0 o [1, 2, 3]
  @write 0 o [4, 5, 6]
  @close o

doubler :: <IO> ()
doubler = do
  Ok s <- @stdin  :: <IO> (Try Str (IStream Int))
  o     <- @stdout :: <IO> (OStream Int)
  Ok xs <- @next s
  @write 0 o (map (\x -> x * 2) xs)
  Ok ys <- @next s
  @write 0 o (map (\x -> x * 2) ys)
  @close o
  @close s
```

Connecting them is a shell pipeline, and every stage but the last is asked for morloc’s own framing:

```console
$ ./main -f packet producer | ./main doubler
[2,4,6,8,10,12]
```

The compiler enforces that both sides agree on element type `Int`. `-f packet` is required rather than inferred: a nexus cannot see where its standard output goes, and the choice between morloc framing and a readable rendering belongs to whoever built the pipeline. The same flag appears in [Reading a stream from standard input](https://morloc-project.github.io/docs/clis/reading-stdin.md).

> **Warning**
> `@stdin` / `@stdout` / `@stderr` carry morloc’s binary sub-packet format — the same wire format used on disk for `IFile` / `IStream` / `OStream`. It is **not** human-readable text. If a sourced foreign function writes to the same standard stream while morloc holds it open (`print` in Python, `std::cout <<` in C++, `cat` / `message` in R), those raw bytes interleave with the morloc packet stream and the reader’s next `@next` fails with a schema-decode error. Either use file-based `IFile` / `OStream` for structured output, or ensure no foreign code writes to a standard stream that morloc has opened.

## 8.7.5. Environment variables

| Variable | Effect |
| --- | --- |
| `MORLOC_REGISTRY_SLOT_COUNT` | Number of concurrent stream handles per nexus invocation. Default `4096`. Each slot occupies 512 bytes of SHM; the table is allocated once at nexus startup. |
| `MORLOC_WRITE_BUFFER_BYTES` | Per-OStream write buffer cap. Default 16 MiB. Smaller values produce more sub-packets (finer reader granularity, more per-flush overhead); larger values amortise zstd overhead over more elements (coarser granularity, longer end-of-run flush). An element larger than the buffer is always written as its own oversize sub-packet regardless of this setting. |
| `MORLOC_IFILE_CACHE_BYTES` | Per-handle SHM cache for decompressed IFile sub-packets. Default 256 MiB. The cache uses an approximate clock-hand LRU and is released on `@close`. Set to `0` to disable, in which case every IFile access decompresses anew. |

The `MORLOC_FRAME_WORKERS` knob from [Compression](https://morloc-project.github.io/docs/runs/compression.md) also applies: an IFile cache miss on a sub-packet with multiple zstd frames decompresses the frames in parallel using the same worker pool.


---

<!-- https://morloc-project.github.io/docs/utilities/ -->

# 9. Utilities

Morloc Manual | https://morloc-project.github.io/docs/utilities/ | prev: https://morloc-project.github.io/docs/runs/random-access-and-streaming.md | next: https://morloc-project.github.io/docs/utilities/nexus-file.md

The nexus ships two utility subcommands that operate on data files without needing a compiled morloc program: `file` (identifies a file) and `view` (loads and re-emits a file in a chosen format).

Both are invoked directly via `morloc-nexus`, not through a wrapper script, since they don’t take a manifest.

A typical workflow for ad-hoc inspection of binary morloc data:

```console
$ ./myprog -f packet -o result.packet mycmd
$ morloc-nexus file result.packet
$ morloc-nexus view result.packet -f json | jq '.' | less
```


---

<!-- https://morloc-project.github.io/docs/utilities/nexus-file.html -->

# 9.1. morloc-nexus file

Morloc Manual > Utilities | https://morloc-project.github.io/docs/utilities/nexus-file.html | prev: https://morloc-project.github.io/docs/utilities/index.md | next: https://morloc-project.github.io/docs/utilities/nexus-view.md

`morloc-nexus file` plays the role of the UNIX `file` utility for Morloc-compatible data files. It recognises morloc packets (`data-packet`, `call-packet`, `stream-packet`, `ping-packet`), JSON, MessagePack, CSV, Arrow IPC, Parquet, ASCII / UTF-8 text, and empty files.

Default output is exactly one line per input file, formatted as `<path>: <type> key=value …​`. The stream is grep / awk / sort friendly: `morloc-nexus file * | wc -l` always equals the number of input files.

For example:

```console
$ morloc-nexus file data.packet log.stream data.json data.mpk people.csv README empty.bin
data.packet: data-packet source=mesg format=msgpack schema="as" payload=1024 metadata=64 total=1120
log.stream: stream-packet schema="ad8" state=final status=closed subpackets=42 elements=524288 payload_full_size=8388608 payload_wire_size=524288
data.json: json
data.mpk: msgpack
people.csv: csv columns=3 delimiter=","
README: text encoding=ascii
empty.bin: empty
```

For morloc packets, classification is magic-byte based and seek-only, so even a multi-gigabyte CALL or STREAM packet costs a few hundred bytes of I/O. The on-disk file size is checked against the size declared in the packet header by default; truncated or oversize packets are reported and exit non-zero.

## 9.1.1. Stream packets

Stream-packet output carries the footer’s summary when one is present. `state=final` marks a cleanly closed stream (final footer + end-of-file tail present); `state=temp` marks an intermediate mid-stream footer written by an in-progress `@close`; `state=missing` marks a stream whose writer exited before writing a footer.

For `state=missing` files, `morloc-nexus file` runs a bounded forward scan of the sub-packet headers to report a best-effort `subpackets=N` and `elements=N` count. The scan is capped at 10 000 sub-packets (`MORLOC_FILE_MAX_SCAN_SUBPACKETS` to override); when the cap is hit the counts carry a trailing `+` and the file is flagged as truncated at the scan boundary. Scan results annotate the `state=missing` line but are diagnostic only: the file is left untouched. `morloc-nexus view` reuses the same scan to open the file through `IStream` or the pattern walker.

For the remaining formats, classification reads up to one kilobyte of the file and feeds it through the same parsers the runtime uses on real ingest.

## 9.1.2. Options

| `-F`, `--no-file` | Suppress the `<path>:` prefix. |
| --- | --- |
| `-D`, `--no-description` | Suppress every `key=value` field; keep only the type token. `-FD` combined yields a clean type stream for programmatic use. |
| `-v`, `--verbose` | Break the one-line rule. CALL packets emit one indented `arg[i]: data-packet …​` line per argument. CSV files emit one indented `<name>:<type>` line per column, with type inferred as one of `int`, `float`, `str`, `bool`, `date`, `time`, or `other`. STREAM packets emit the extended footer diag (writer pid, flush timestamps, oversize count, largest sub-packet, EOF tail window). |
| `-n`, `--bytes SIZE` | Bytes read for content-based detection. Accepts a K / M / G suffix. Default `1k`. Morloc-packet detection always uses just the 32-byte header regardless. |
| `--json` | Emit one JSON object per file. JSON output is unaffected by `-F` / `-D` / `-v` and always includes the full structure, including CSV column names and types. |
| `--validate` | After classifying, fully load each file through the exact same loader the `run` subcommand uses, then discard the result. Appends `validated=yes`, `validated=no error="<msg>"`, or `validated=structure-only`. If `file --validate` passes, `run` (or `view`) reading the same file will not fail at the load stage. |
| `--schema STRING` | Morloc schema used by `--validate` for inputs that don’t embed one. Ignored without `--validate`. |

## 9.1.3. Verbose example

```console
$ morloc-nexus file -v people.csv
people.csv: csv columns=3 delimiter=","
  name:str
  age:int
  city:str

$ morloc-nexus file -v call.packet
call.packet: call-packet midx=42 entrypoint=local nargs=3 payload=72 total=104
  arg[0]: data-packet source=mesg format=msgpack schema="i4" payload=4
  arg[1]: data-packet source=mesg format=msgpack schema="as" payload=16
  arg[2]: data-packet source=rptr format=voidstar schema="ad8" payload=8

$ morloc-nexus file -v log.stream
log.stream: stream-packet schema="ad8" state=final status=closed subpackets=42 elements=524288 payload_full_size=8388608 payload_wire_size=524288
  diag_version=1
  writer_pid=12345
  n_oversize_subpackets=0
  writer_start_time=1720000000
  first_flush_time=1720000001
  last_flush_time=1720000042
  largest_packet_uncompressed=262144
  largest_packet_idx=17
  tail_window=[...]
```


---

<!-- https://morloc-project.github.io/docs/utilities/nexus-view.html -->

# 9.2. morloc-nexus view

Morloc Manual > Utilities | https://morloc-project.github.io/docs/utilities/nexus-view.html | prev: https://morloc-project.github.io/docs/utilities/nexus-file.md | next: https://morloc-project.github.io/docs/utilities/mim.md

Reads a data file (morloc data-packet, morloc stream-packet, `.json`, `.mpk`, `.arrow`, `.parquet`, or `.csv`) and re-emits it in a chosen output format. The intended use is ad-hoc inspection (`view foo.packet | jq | less`), one-shot format conversion (`view foo.json -f mpk --schema "as" -o foo.mpk`), and slicing large files down to a subset without materialising the whole thing (`view foo.packet --pattern ".[100:200]" -f json`).

`view` reuses the same loader chain `run` uses for argument ingress and the same output emitter `run` uses for results, so format support and conversion behaviour cannot drift away from a real morloc program run. A file accepted by `view` is one `run` would also accept, and vice versa.

```console
$ ./myprog -f packet -o result.packet mycmd
$ morloc-nexus view result.packet -f json
[1, 2, 3]
$ morloc-nexus view result.packet -f json | jq '. | length'
3
$ morloc-nexus view data.json -f mpk --schema "as" -o data.mpk
```

Before loading, `view` runs the same classifier `morloc-nexus file` uses, so a truncated or oversize morloc packet is rejected up-front with a clear error instead of decoding garbage from a partial payload.

## 9.2.1. Reading from stdin

A single `-` argument reads from stdin. For stream-packet input (the IFile / OStream / IStream shape), `view -` iterates sub-packet by sub-packet and emits one element (or one line, for `-f jsonl`) at a time, so a multi-gigabyte stream can be viewed in constant memory. For data-packet input, or when the selected output form requires a full-value load (e.g. `-f arrow`), stdin is drained to a `$TMPDIR` temp file and the normal file path runs. The temp file is deleted on exit.

```console
$ ./producer | morloc-nexus view - -f jsonl | head -5
```

## 9.2.2. Schema resolution

`view` always loads through the typed loader, so a schema is required:

1.  `--schema STRING` if given.
2.  Otherwise, the schema embedded in a morloc-packet’s metadata (data or stream).
3.  Otherwise, `view` exits with an error directing the user to `--schema`.

## 9.2.3. `--pattern`: extract a subset

`--pattern STR` applies a morloc pattern chain to the input before emission. The grammar is the same one morloc source uses for bracket accessors: field access (`.foo`), tuple/positional index (`.0`, `.1`), bracket index and slice (`.[i]`, `.[a:b]`, `.[a:b:c]`), grouped projection (`.(.a;.b)`), and broadcast tails after a slice (`.[:].name`, `.[:].[0]`, `.[:].(.name;.age)`).

The pattern is parsed and type-checked against the input’s schema before any I/O happens. Mismatches (a `.foo` on an integer, a slice on a scalar) fail with a diagnostic that points at the offending pattern fragment.

Note that the CLI uses `;` as the group separator (`.(.a;.b)`) so a single argument stays shell-safe; in morloc source the separator is `,` (`.(.a,.b)`). Both mean the same thing.

```console
$ morloc-nexus view people.packet --pattern ".[0:10].(.name;.age)" -f json
[["Alice",30],["Bob",25],...]

$ morloc-nexus view timeseries.stream --pattern ".[100000:100010]" -f jsonl
{"t":1.0,"v":3.14}
{"t":1.01,"v":3.15}
...
```

The pattern dispatches through the same IFile walker `IFile a` values use in morloc code (`mlc_ifile_walk`) — for a data packet, a single mmap + slice; for a stream packet with a valid footer, `log2(K)` seeks over `K` sub-packets. For footer-less streams, `view` forward-scans the file to reconstruct the sub-packet index before dispatch (see [Footer-less streams](#footer-less-streams) below).

## 9.2.4. Packet-shape output: `-s` / `-d` / `-p`

Three mutually exclusive flags control the packet shape when the output form is a morloc packet:

| `-s`, `--stream-packet` | Emit a `MORLOC_STREAM_PACKET` file. The value (or the pattern result) must be a list. Implies `-f packet`. |
| --- | --- |
| `-d`, `--data-packet` | Emit a `MORLOC_DATA_PACKET` file (the default packet shape). Implies `-f packet`. |
| `-p`, `--preserve-packet` | Mirror the input packet’s shape on output: data-in / data-out, stream-in / stream-out. Errors when the input is not a morloc packet. Implies `-f packet`. |

Bare `-f packet` (no `-s`/`-d`/`-p`) emits a data packet. Any of `-s`/`-d`/`-p` combined with `-f <not packet>` is an error.

The four conversion arms all preserve typed semantics through the runtime’s authoritative packet writers:

-   **DATA → DATA** is a byte-level copy (or a loader re-emit when the `--schema` differs from the one embedded in the input).
-   **DATA → STREAM** opens the input as an IFile, chunks the array via bracket-slice, and writes each chunk through an OStream so the output is chunked, compressed, and indexed like any other stream file.
-   **STREAM → STREAM** drains the input via IStream `@next` and rewrites via OStream `@write`, so recompression, schema override, and footer normalisation all flow through the canonical writers. When `--compression-level` matches the source, a fast path verbatim-copies sub-packet payloads and only rewrites the footer.
-   **STREAM → DATA** materialises the whole stream to memory via the IFile walker and re-emits as a data packet. Gated by the size guardrail below.

## 9.2.5. `-f jsonl`: line-delimited JSON

`jsonl` output emits one JSON value per input element, one per line. JSON emission is element-by-element in all cases, so peak per-line memory is one element’s JSON body regardless of input size. Input buffering depends on the source:

-   stdin (`view - -f jsonl`) reads one sub-packet at a time and serialises its elements before reading the next, so a multi-GB producer streams through in constant memory.
-   File-based stream input goes through the buffered loader: the whole list is materialised, then emitted line-by-line. Combine with `--pattern .[a:b]` (or `-f jsonl` on stdin) for a constant-memory shape on multi-GB inputs.

## 9.2.6. Size guardrails and `--force`

Two guardrails refuse large operations without an explicit `--force`:

-   Any buffered path (stream input to a non-`jsonl` text output, stream input to a data packet, single-value binary output) whose projected working set exceeds 1 GiB.
-   Binary output (`-f packet`, `-f mpk`, `-f voidstar`, `-f arrow`, `-f parquet`) to a terminal.

`--force` lifts both. Refusal messages state which guardrail fired and suggest an alternative (`-s`, `-f jsonl`, `--pattern ".[a:b]"`, or `-o FILE`). The buffered-path threshold can be overridden with `MORLOC_VIEW_MAX_BUFFER_BYTES`.

For footer-less stream input the projected size falls back to the compressed file’s on-disk length as a conservative lower bound, so a partially-written multi-GB stream still hits the guardrail rather than silently OOM’ing the buffered path.

## 9.2.7. Footer-less streams

A stream file whose final footer is missing (writer crashed before `@close`, `program > out.stream` interrupted, etc.) is still usable through `view`. The classifier reports the missing footer, and:

-   `view -p` (or explicit `-s`) re-emits a well-formed stream with a canonical footer — the runtime’s IStream drains forward regardless of footer state, and OStream writes a fresh footer.
-   `view -d` (buffered materialisation) succeeds subject to the size guardrail.
-   `view --pattern PATTERN` forward-scans the file to recover a sub-packet-offset index, then dispatches the walker as normal.

## 9.2.8. Options

| `-f`, `--output-form FORM` | Output format: `json` (default), `jsonl`, `mpk`, `voidstar`, `packet`, `arrow`, `parquet`, `csv`. Same set as `run -f` plus `jsonl`. |
| --- | --- |
| `-z`, `--compression-level N` | zstd preset `0..=9`. Applies to each sub-packet when emitting `-s` (stream packet). |
| `-o`, `--output-file PATH` | Write to `PATH` instead of stdout. |
| `-s`, `--stream-packet` | Emit a stream packet (mutex with `-d`/`-p`). |
| `-d`, `--data-packet` | Emit a data packet (mutex with `-s`/`-p`). |
| `-p`, `--preserve-packet` | Mirror the input packet’s shape (mutex with `-s`/`-d`). |
| `--pattern STR` | Apply a pattern chain (`.[a:b].field`, `.[:].[0]`, …​) before emission. |
| `--force` | Lift the size and binary-to-tty guardrails. |
| `--schema STRING` | Morloc schema string (compact format, e.g. `"as"` for `[Str]`, `"ad8"` for `[F64]`). Overrides the input’s embedded schema. |


---

<!-- https://morloc-project.github.io/docs/utilities/mim.html -->

# 9.3. `mim` (Morloc Installation Manager)

Morloc Manual > Utilities | https://morloc-project.github.io/docs/utilities/mim.html | prev: https://morloc-project.github.io/docs/utilities/nexus-view.md | next: https://morloc-project.github.io/docs/utilities/morloc-eval.md

## 9.3.1. System-scope environments

`mim`, the Morloc Installation Manager, keeps environments in two parallel scopes: a per-user **local** scope (no privileges required) and a machine-wide **system** scope (root required). The `--system` flag selects the system scope on the subcommands that mutate it:

-   `mim new <name> --system` — build a new environment in the system scope so it is shared across users (its `--engine`, if given, also sets the system-scope default engine for later `--system` creates).
-   `mim rm <name> --system` — remove an environment from the system scope.
-   `mim nuke --system` — remove all system-scope environments (and, with `--images`, their backing container images).
-   `mim modify --env <name> --set-default --system` — write the default-environment tag to the system config rather than the user’s local config.

Read-only subcommands also accept `--system` for discovery rather than mutation:

-   `mim ls --system` — list only system-scope environments (`--local` is the symmetric filter).
-   `mim info [<name>] --system` — describe a system-scope environment, useful when a local environment of the same name shadows it.

A regular (non-root) user can therefore find out whether a system-scope environment exists — and what it is configured with — without elevated privileges, by running `mim ls --system` or `mim info <name> --system`. Mutating subcommands will refuse to run without root and print a hint to re-invoke under `sudo`.

## 9.3.2. Rebuilding and reconfiguring environments

Two verbs cover the environment lifecycle after creation, split by what they touch:

`update` rebuilds an environment. With no flags it re-solves and rebuilds at the environment’s **current** morloc version; it never moves the version implicitly.

```console
$ mim update --env myenv                       # rebuild, same version
$ mim update --env myenv --latest              # move to the newest release
$ mim update --env myenv --morloc-version 0.105.2 # move to a specific version
$ mim update --env myenv --force               # force a fresh re-solve (repair)
```

`modify` changes an environment’s settings **without** moving its morloc version. Setting the default and copying dotfiles are instant; changing languages or packages triggers a rebuild at the current version. `modify` validates the whole request before making any change, so an invalid combination never leaves an environment half-modified.

```console
$ mim modify --env myenv --set-default         # make it your default (no rebuild)
$ mim modify --env myenv --env-name newname    # rename it; a default follows (no rebuild)
$ mim modify --env myenv --dotfiles ~/dotfiles # copy dotfiles (no rebuild)
$ mim modify --env myenv --lang py@3.13        # re-pin a language, then rebuild
$ mim modify --env myenv --conda-packages-file tools.conda # set conda packages
$ mim modify --env myenv --system-packages-file tools.apt  # set apt packages
```

## 9.3.3. Packages

Beyond language runtimes, an environment can carry extra packages from two sources, each supplied as a file — one package per line, with `#` comments and blank lines ignored:

-   `--conda-packages-file <file>` adds conda packages from conda-forge, such as command-line utilities like `jq`, `ripgrep`, or `hyperfine`. Each line is a conda match-spec: a bare name takes the latest compatible version, or pin one (`hyperfine>=1.18`, `numpy=1.26`, `cmake<4`). Conda packages work on **every** backend, native included.
-   `--system-packages-file <file>` adds OS packages installed with the base image’s package manager (apt), for things conda-forge does not carry — `locales`, `linux-tools-generic`, kernel-matched tooling. These are baked into the image, so they apply to **container backends only**; requesting them on a native environment is an error. Prefer `--conda-packages-file` for anything on conda-forge.

Both flags are accepted by `new` and `modify`. The file is the **whole** list for that source: passing it **replaces** the environment’s stored packages for that source, so to add or remove one package you edit the file and re-apply. Changing either list triggers a rebuild at the current morloc version, and the stored list is updated only after the rebuild succeeds — a package that breaks the build leaves the previous list intact, so a typo never wedges the environment.

## 9.3.4. Language toolchains

`--lang` provisions a language runtime into an environment, at `new` or `modify` time. Each value is a language name or a `lang@version` pin, and the flag is repeatable or comma-separated — these are equivalent:

```console
$ mim new myenv --lang py,r,cpp
$ mim new myenv --lang py --lang r --lang cpp
$ mim new myenv --lang py@3.12,r@4.3   # with version pins
```

Most languages (python, R, C++, Rust) are provisioned from conda-forge and their versions are solved together with the rest of the environment. A language whose toolchain is not on conda-forge — currently **futhark** — is instead installed by a script when the container image is built, so it is supported only on the docker/podman backends; requesting it on the native or apptainer backends is an error. `--lang futhark` installs a fixed, tested futhark release.

## 9.3.5. Development environments

A **development environment** mounts a morloc source tree and provisions the **tooling** to build it, giving contributors (and coding agents) a uniform, reproducible place to build, edit, and test morloc itself. Point `--dev` at a morloc source checkout:

```console
$ mim new --dev /path/to/morloc --engine podman
```

This builds a container (docker or podman only) that bakes the Haskell toolchain via ghcup and provisions the language runtimes with pixi — but it does **not** build morloc. That is deliberate: an in-development source may not compile, and that must never block creating the environment. You build the compiler and runtime yourself, in the dev shell, where the toolchain (`stack`, `ghc`, `cargo`) is on `PATH`. The dev shell does **not** mount the source at a fixed path: it opens in your host working directory (bind-mounted at `/work`), so `cd` into your morloc checkout on the host first, then open the shell and build from there:

```console
$ cd /path/to/morloc      # on the host: your source checkout
$ mim shell --env dev
# inside the container, now in /work (your checkout):
$ stack install morloc:exe:morloc morloc:exe:morloc-codegen-generic \
    --local-bin-path /opt/morloc-runtime
$ MORLOC_RUST_DIR="$PWD/data/rust" morloc init -f
```

Because the source is reached through the working-directory mount rather than a dedicated one, point `morloc init`/`morloc make` at the Rust workspace with `MORLOC_RUST_DIR` (its `data/rust` subdirectory) when you rebuild the runtime. The compiler you build and the build caches live in host-mounted directories, so `stack`, `ghc`, and `cargo` sit alongside the language runtimes and edits persist across shells. A dev environment is local-scope and docker/podman only, and its default name is `dev`. `--dev` is compatible with `--morloc-version`, which sets the **stdlib** base the environment tracks; the compiler itself is whatever you build from the source. `freeze` is refused, since a dev environment is not reproducible, and `info` reports the source path.

To round out a development environment with the tools you work with — linters, profilers, language servers — add them from conda-forge with `--conda-packages-file` (see [Packages](#mim-packages)), which behaves the same here as on any other environment:

```console
$ mim new --dev /path/to/morloc --engine podman \
    --conda-packages-file dev-tools.conda   # e.g. jq, ripgrep, hyperfine, nodejs
```

## 9.3.6. Inspecting an environment

`mim info <name>` describes an environment as four groups: its identity and materialization status, the **folders** it occupies on the host, the **environment variables** a `run`/`serve` process sees, and its **dependencies** — the language runtimes and package count from the solved world (`pixi.lock`):

```console
$ mim info myenv
Name:      myenv
Scope:     local
Default:   yes
Backend:   native
Morloc:    0.98.2
Status:    materialized

Folders (host):
  Data dir:     .../environments/myenv
  Runtime:      .../environments/myenv  (shared with state)
  Pixi:         .../environments/myenv/pixi
  Requirements: .../environments/myenv/requirements
  Cache:        .../environments/myenv/cache
  Config:       .../environments/myenv/env.yaml

Environment (exported into run/serve):
  MORLOC_HOME=.../environments/myenv
  MORLOC_ENV=myenv
  MORLOC_PIXI=.../bin/pixi

Dependencies (locked):
  Languages:  python 3.12.4, rust 1.83.0
  203 packages in the solved world (`info myenv --packages` for the full list)
```

`mim info myenv --packages` lists every package in the solved world at its locked version; `--json` prints the whole description (packages included) as machine-readable JSON, and `--packages --json` prints just the package array.

## 9.3.7. Checking environment health

`mim doctor [<name>]` runs read-only health checks against an environment and exits non-zero if any check fails, so it fits a setup script or CI gate. On the native backend it verifies that the runtime is materialized and its conda toolchain is present, that the captured activation exports the compiler tools a build needs (`$CC`/`$CXX`/`$AR`/…​), that each provisioned language stack and `libmorloc` are in place and resolve their libraries, that the compiler/manager/agent versions are in step, and that the dependency world is solved and the data dir is writable:

```console
$ mim doctor --env myenv
...
9 passed, 0 warnings, 0 errors
```

Add `--deep` for slower checks (installed program launchers), `--json` for a machine-readable report, and `--strict` to treat warnings as failures.

## 9.3.8. The environment home and dotfiles

Under the Docker and Podman backends a container runs as your host user but does **not** mount your host `$HOME`. Each such environment instead owns a private home directory on the host, bind-mounted as `$HOME` inside the container. It is writable from both sides and survives runs, `update`, and `clean`. `mim info` prints its path under `Folders`:

```console
$ mim info myenv
...
Folders (host):
  ...
  Home:         .../environments/myenv/home  (shell $HOME; drop dotfiles here)
```

Anything you place there behaves like a normal home file. An interactive shell reads `.bashrc`, so a custom prompt and aliases go there (a `.vimrc`, `.gitconfig`, or nested `.config/…​` work the same way):

```bash
# .../environments/myenv/home/.bashrc
export PS1='(myenv) \w $ '
alias grep='grep --color=auto'
alias ls='ls --color=auto'
```

```console
$ mim shell              # the shell picks up the new .bashrc
```

`shell` spawns a fresh interactive shell (exit with `exit`) and tags its prompt with the environment name so it is clear which environment you are in. It does this without editing your dotfiles — for bash and zsh it sources your real init first, then prepends the `(myenv)` tag — so your own prompt still applies.

To seed a whole set of dotfiles at once, `new` and `modify` accept `--dotfiles <dir>`, which recursively copies a directory into the environment home (overwriting like `cp -rf`; symlinked directories are skipped):

```console
$ mim new myenv --dotfiles ~/dotfiles
$ mim modify --env myenv --dotfiles ~/dotfiles   # re-copy after edits
```

> **Note**
> This is a Docker/Podman feature. An Apptainer environment mounts your real host `$HOME` (so it already sees your host dotfiles) and the native backend runs against your real home, so `--dotfiles` is rejected on both. A `.bashrc` affects interactive shells only — it is not read by a non-interactive `morloc make`, so build-time settings belong elsewhere.

## 9.3.9. Serving installed programs

Beyond managing environments, `mim` runs the **serving lifecycle** that presents compiled programs to AI assistants (over MCP) and to HTTP clients (over a JSON API). The first three steps are distinct on purpose — installing a program does not make it reachable, and declaring that it should be does not serve it until you ask. The fourth turns the environment you have been working in into something you can hand to someone else:

| Step | Command |
| --- | --- |
| **install** — build a module into an environment (the default, or `--env`) | `mim install main.loc` |
| **view** — declare which installed modules answer on which adapter | `mim view add <module> --as mcp,api` |
| **serve** — launch one front-end over the declared views | `mim start` |
| **freeze** — turn the environment into a self-contained image | `mim freeze --tag <image>` |

An environment is **pliable** while you work in it: you install programs, add dependencies, rebuild, and try things. Freezing takes that environment as it stands and produces an image that runs the same programs with nothing mounted in from outside.

### install

`mim install <src>` is shorthand for `morloc make --install <src>` run inside the target environment (the default, or the one named with `--env`). The installed program is identified by its **module name** — the `module <name>` declaration in the source — never by the file name or any `-o`. By convention the source lives in `main.loc`, so a program declaring `module dna` installs as `dna` regardless of the file name.

### view

A **view** is what an adapter shows of an environment. The MCP view and the API view are independent sets, and `eval` is a separate capability. `view` edits a per-environment file recording that declared intent; nothing is served until `start` realizes it.

```console
$ mim view                        # show this environment's views
$ mim view add dna --as mcp,api   # dna answers on both /mcp and /call
$ mim view add util --as api      # util answers on the JSON API only
$ mim view eval --allow dna       # enable sandboxed eval (allow: dna)
$ mim view rm dna                 # remove dna from every view
```

Installing a module makes it importable; adding it to a view makes it callable over the network. The two are independent, so a program can be installed and unreachable, which is the default. A module must be installed before it can be added (`view add` checks for the launcher and errors otherwise). `--as` takes a comma-separated list of `mcp` and/or `api`. `view eval` turns on the sandboxed eval capability with an import allow-list; `--off` disables it.

### start

`start` launches a single serving front-end — one HTTP listener that answers `/mcp` (MCP), `/call/<module>/<command>` (JSON API), `/discover` (API discovery), and `/health` (liveness) — forwarding each call to that module’s own worker process.

```console
$ mim start                    # serve the environment's views
$ mim start --mcp dna          # ad-hoc: serve just `dna` over MCP
$ mim start -p 9090:9090       # pin the host:container port
```

With no `--mcp`, `start` serves the environment’s views; it errors if there are none rather than serving nothing. `--mcp <module>` is a quick one-off that ignores the declared views and serves that one module over MCP.

By default the endpoint binds the **host’s** `127.0.0.1` (loopback, reachable only from the host), so no token is required. Making the endpoint reachable elsewhere is a deliberate opt-in:

| Flag | Effect |
| --- | --- |
| `--auth-token <tok>` / `MORLOC_MCP_TOKEN` | Require `Authorization: Bearer <tok>` on every `/mcp` and `/call` request (`/health` stays open). The env var keeps the token off the command line. |
| `--expose` | Publish on `0.0.0.0` (all interfaces) instead of loopback. Requires a token and `--allow-plaintext`. |
| `--allow-plaintext` | Acknowledge that there is no TLS — the token and traffic travel in cleartext. |
| `--allow-no-auth` | Permit an off-box endpoint with no token (an open server; strongly discouraged). |
| `--unsafe` | On a VM-backed engine (Docker Desktop / podman machine), where a loopback bind cannot be confined to the host, serve unauthenticated anyway. Dangerous; trusted hosts only. |

For AI assistants, `start` also prints an `mcpServers` client-config entry as pure JSON on stdout (so `mim start …​ > dna.mcp.json` is ready to drop into a client); human status goes to stderr.

### status, stop, logs

```console
$ mim status         # list running serve containers (mode, modules, URL)
$ mim logs [-f]      # stream the serve container's logs
$ mim stop           # stop the running serve container
```

`status` reads a runtime record written by `start`, so it reports the mode, served modules, and URL even under host networking where the engine shows no published port.

### eval

`mim eval '<expr>'` evaluates a Morloc expression against a running serve container’s sandboxed eval endpoint (enabled with `view eval`):

```console
$ mim eval 'import dna (revcomp); revcomp [("s","ACGTA")]'
$ mim eval -p 9090 'map (add 1) [1,2,3]'
```

`mim eval` reaches the serve on the port it recorded when it started, so you do not have to remember which one it picked; `--port` overrides that. If the serve requires a bearer token, pass it with `--auth-token` or set `MORLOC_MCP_TOKEN` — the token is never written to disk, so `mim` can tell you that one is needed but cannot supply it for you.

### freeze

Freezing turns the environment you have been working in into a container image that carries everything it needs.

```console
$ mim freeze --tag dna-service:v1
```

The image is the environment with its mounted halves baked in. The morloc runtime is copied: the nexus, the shared library, the language bindings, and a launcher for every installed program, all of which were built after the environment’s image was. The language toolchain is installed from the environment’s own `pixi.lock` — a solved toolchain is not a directory you can copy, so what travels is the lock that reproduces it. The programs and the module sources behind them are copied too. Anything required and missing is reported by name, so freezing an environment that was never provisioned fails instead of building an image with holes in it.

Each program travels as the mirror of its project directory that `install` made, since the pools import sourced files from it. Whatever else the project held goes with it — which is fine for a data file the program reads and unfortunate for a build tree. So before anything expensive runs, `freeze` walks every installed program and prints what it will carry:

```console
$ mim freeze --tag dna-service:v1
Installed programs to freeze:
  dna                  0.4 MB
  atlas                310.2 MB
  large file: atlas/data/genome.fa (299.0 MB)
  tool state: dna/target (not sized)
```

A tool-state directory — `.git`, a cargo `target/`, `node_modules`, `*pycache*`, `.venv`, an editor’s cache, and the like — is a refusal: nothing a program runs reads one, and the fix belongs in the project. Add it to the project’s `.morlocignore` (one pattern per line, `target/` for a directory), reinstall the program, and freeze again; `include:` in `package.yaml` is the allowlist form of the same control. A program over 100 MB, or a file over 50 MB, is a question rather than a refusal, because only you know whether that is data the program needs: on a terminal `freeze` shows the sizes and asks; from a script, where nobody can answer, it stops. An environment with no programs installed is asked about the same way, since freezing one is usually a mistake but sometimes the point — a base to run programs in later. `--force` answers all of these.

Nothing appears in your working directory. The artifact is a tag in the engine’s image store, which is what makes it portable: you move it by pushing it to a registry, or as a file.

```console
$ docker push ghcr.io/you/dna-service:v1
$ mim freeze --tag dna-service:v1 --save ./dna-service-v1.tar
$ docker load -i ./dna-service-v1.tar   # on the far side
```

`--save` is the engine’s own `docker save`, so the tarball carries every layer including the base and needs nothing else to restore. There is no morloc-specific archive format.

The image above is the environment whole — the compiler, the Rust and C toolchains, pixi — which is what lets it evaluate expressions and rebuild a pool, and is also most of its size. When the programs are the point and nothing in the image will ever compile, freeze it slim:

```console
$ mim freeze --slim --tag dna-service:v1-slim
```

A slim image keeps every interpreter and package the environment declared and removes what only built things: the compilers and their sysroot, rustc, make, git, the morloc compiler, pixi, and the runtime’s headers. The cut is computed from the environment’s own package records, so it also reaches the compiler chain that conda’s `r-base` drags in for `install.packages` — about half of a typical environment goes. What runs is exactly what the full image runs, at the same paths, and the image is checked after the build: every program’s launcher starts, and the nexus and every compiled pool resolve their shared libraries. The default tag ends in `-slim`, so the two flavours of one environment do not overwrite each other, and the `morloc.flavor` label says which one you hold.

A slim image cannot eval, so an environment that exposes eval is refused; drop the view or freeze without `--slim`. The same holds for anything a program compiles at run time — Cython at import, numba, R packages installed from source — which needs the full image.

An image that has left `mim` has nothing tracking it, so it carries its own provenance as labels — the morloc version, the environment it came from, the programs inside, and which of them answer on each adapter. `docker inspect` reads them back.

The image serves the environment’s views, and nothing else. Declare no views and the image has no default command — it is still complete, and you run programs in it by name:

```console
$ docker run -e MORLOC_MCP_TOKEN=$TOKEN -p 8080:8080 dna-service:v1
$ docker run dna-service:v1 dna revcomp ACGTA
```

The first form serves; the second runs a program through the command line the compiler generated for it. One image, both interfaces, from the same declarations.

Inside the container the service binds every interface, because a container’s loopback belongs to the container and a published port never reaches `127.0.0.1` in there. That address says nothing about who can reach the service, so the image does not treat it as if it did: it serves without a token, and says so once at startup. What can reach it is your decision, made outside the container — publish to loopback with `-p 127.0.0.1:8080:8080`, keep it on an internal network, or put a gateway in front. Set `MORLOC_MCP_TOKEN` to require a bearer token as well.

Eval is the exception. It runs expressions the caller writes rather than the functions you exported, and a single call can rebuild a pool, so it asks for a token even where the rest of the endpoint does not:

```console
$ docker run -e MORLOC_MCP_TOKEN=$TOK -p 8080:8080 dna-service:v1
$ docker run -e MORLOC_EVAL_ALLOW_NO_AUTH=1 -p 8080:8080 dna-service:v1
```

The first serves eval to holders of the token; the second waives the requirement, for a deployment where something in front already gates it. Without either, eval is locked: it is not advertised in the tool list, `/discover` reports it as not callable, and a request to it is refused with the reason. Everything else serves normally.

`mim start` waives the requirement by itself when the endpoint cannot leave the host, which is the case eval mostly exists for; `--eval-allow-no-auth` waives it when serving off-box.

`mim` manages environments, not images. It builds a deployment image and has no further relationship with it: `status`, `logs` and `stop` act on serves started from environments, so an image you run yourself is not something `mim` tracks, and needs neither `mim` nor the environment it came from.

> **Warning: Experimental Feature**
> The image is built where the environment lives, because it is that environment’s image with the runtime and toolchain added. Freezing on one machine to build on another is not supported; move the built image instead.

The wire protocols behind these endpoints — the MCP handshake, the JSON API call/discover shapes, sessions, and status codes — are documented in the [Model Context Protocol (MCP)](https://morloc-project.github.io/docs/apis/mcp.md) and [Building APIs](https://morloc-project.github.io/docs/apis/index.md) chapters.


---

<!-- https://morloc-project.github.io/docs/utilities/morloc-eval.html -->

# 9.4. morloc eval

Morloc Manual > Utilities | https://morloc-project.github.io/docs/utilities/morloc-eval.html | prev: https://morloc-project.github.io/docs/utilities/mim.md | next: https://morloc-project.github.io/docs/modules/index.md

Morloc has three subcommands that turn source into a result, and they serve distinct roles:

-   `morloc make` — compile a **module** into an executable (the nexus and its language pools). This is the full language: a module may `source` foreign code, declare types, typeclasses, and instances, import local modules, and export zero, one, or many terms.
-   `morloc typecheck` — type-check a **module** without compiling or running it. Same full language as `make`; it only reports the inferred types of the exported terms.
-   `morloc eval` — compile and run a single **expression**. An expression **composes functions that are already installed on the system**; it cannot introduce new ones. An eval expression may import installed modules and use `let`/`where`/`do`, but it may not `source` foreign code or declare types, typeclasses, instances, or module structure.

The dividing line is **module vs expression**. `make` and `typecheck` consume a module, which can define and source new functionality and export any number of terms. `eval` consumes one expression assembled purely from already-installed pieces, producing exactly one result. Use `eval` for quick experiments, shell pipelines, and for exposing a fixed set of installed functions to callers who may only **compose** them — never to introduce new code.

Pass the expression inline with `-e`, or name a file containing it as the positional argument. The two are interchangeable: writing an `-e` string to a file and running `morloc eval file` gives the same result — the file is treated as expression text, not as a module.

```console
$ morloc eval -e "import root-py; 1 + 2"
3

$ morloc eval -e 'import root-py; "foo" <> "bar"'
"foobar"

$ printf 'import root-py\n1 + 2\n' > add.loc
$ morloc eval add.loc
3
```

Because an eval expression can only compose installed functions, `eval` is also the safe surface to expose over an API or daemon: it resolves only **installed** modules, never local-filesystem modules, so an untrusted caller cannot `source` arbitrary foreign code or reach a module they uploaded. A local import — a bare name that resolves on the filesystem, or a dot-prefixed name (`.utils`) — is rejected in eval mode; build programs that depend on local modules with `morloc make` instead. The `--allow-local-modules` flag re-enables local resolution for local development only and is insecure for server use.

## 9.4.1. Imports in eval strings

Morloc has no implicit prelude: every name an expression refers to must come from a module the eval string explicitly imports. Operators like `+` and `<>` are typeclass methods sourced from the standard library, so a typical eval string begins with one or more imports:

```console
$ morloc eval -e "import root-py; import root-cpp; 1 + 2"
3
```

As described above, only installed modules may be imported (named bare, like `root-py`); local imports are rejected in eval mode. See [Importing modules](https://morloc-project.github.io/docs/modules/importing-modules.md) for the full import rules.

If no import brings the required operator or function into scope, the compiler reports an undefined-term error with a hint pointing at the fix:

```console
$ morloc eval -e "1 + 2"
<expr>:1:2: error:
Undefined term: +
hint: an eval expression has no implicit prelude; prefix the expression with 'import root-py;' (or the module that defines +) to bring it into scope
```

## 9.4.2. The eval sandbox

The `morloc eval` CLI is **trusted**: on your own machine it may import any installed module and use any intrinsic, exactly like the examples above. When eval is exposed to untrusted callers — over a daemon or router — it is **sandboxed**, and two further gates apply on top of the base rules (installed-only, no `source`, no type/class/instance declarations):

-   **Module allow-list.** The expression’s top-level imports are limited to a curated list. An empty list allows no imports, so only pure, module-free expressions run: literals and the pure intrinsics such as `@show`/`@hash`/`@lang`. Nothing else is in scope — not even `+`, which is a typeclass method that must be imported. Matching is on the resolved module name, so `import M as N` is checked against `M`.
-   **IO-intrinsic ban.** An IO intrinsic (`@open`, `@save`, `@write`, `@stdin`, …​) may not be written **directly** in the expression. An IO intrinsic reached **through** a function exported by an allow-listed module is fine: a server exposes the IO surface it chooses as named functions, never a raw filesystem primitive. Pure intrinsics remain usable.

Together these give the operator term-level control: allow-list a curated module that re-exports exactly the functions callers may compose (the standard re-export idiom — a `public` facade), and eval can reach only those.

Two flags drive the gates:

| Flag | Effect |
| --- | --- |
| `--eval-allowed-modules a,b` | Restrict top-level imports to `a` and `b` (comma-separated). Implies `--eval-sandbox`. An empty list permits no imports. |
| `--eval-sandbox` | Enable the gates without granting any modules (an empty allow-list). Mainly for previewing the sandbox from the CLI. |

On the CLI these let you preview how an expression behaves once served:

```console
$ morloc eval --eval-allowed-modules root-py -e 'import root-py; 1 + 2'
3
$ morloc eval --eval-allowed-modules root-py -e 'import root-py; @write "x" 1'
error: IO intrinsics may not be used directly in a sandboxed eval expression ...
$ morloc eval --eval-allowed-modules base -e 'import root-py; 1 + 2'
error: module 'root-py' is not in the eval allow-list
```

Served eval (over a daemon or router) is **always** sandboxed — there is no unsandboxed served mode. The operator sets the allow-list once when starting the server. Effects other than IO are not yet gated: a future effect disallow-list distinguishing read, write, and execute is planned; today the module allow-list and the IO-intrinsic ban are the sandbox.

## 9.4.3. Single-line layout: braces and semicolons

A Morloc source file relies on indentation to delimit blocks. An `eval` string is a single shell argument, so block structure must use the explicit-brace forms that the grammar provides as alternatives to the indentation-based forms. Two rules apply:

-   Top-level items (imports and the trailing expression) are separated by a literal `;`. The eval preprocessor rewrites every top-level `;` to a newline before handing the string to the parser, which is the same effect as starting a new top-level line in a file.
-   Block bindings inside `where`, `let`, and `do` are written with literal braces and semicolons: `where { a = 1; b = 2 }`, `let { a = 1; b = 2 } in expr`, `do { stmt1; stmt2; expr }`. Semicolons inside `{…​}` are preserved by the preprocessor and consumed by the parser as item separators.

A `where` clause that would normally span multiple indented lines in a source file:

```morloc
result = a + b where
    a = 10
    b = 20
```

becomes, on the command line:

```console
$ morloc eval -e 'import root-py; a + b where { a = 10; b = 20 }'
30
```

Likewise for `let`:

```console
$ morloc eval -e 'import root-py; let { a = 10; b = 20 } in a + b'
30
```

A `do`\-block (see [Effects and delayed evaluation](https://morloc-project.github.io/docs/features/effects.md)) uses the same brace-and-semicolon form:

```morloc
do { stmt1; stmt2; final_expr }
```

These explicit-brace forms are not specific to `eval` — they are part of the Morloc grammar and may be used in source files too. They are simply the only practical way to write multi-binding blocks inside a single shell-quoted string.

## 9.4.4. Saving an eval expression as a command

`--save NAME` installs the compiled expression as a reusable command:

```console
$ morloc eval --save adder -e "import root-py; 1 + 2"
$ adder
3
```

The installed command behaves like any other Morloc executable.


---

<!-- https://morloc-project.github.io/docs/modules/ -->

# 10. Modules and Libraries

Morloc Manual | https://morloc-project.github.io/docs/modules/ | prev: https://morloc-project.github.io/docs/utilities/morloc-eval.md | next: https://morloc-project.github.io/docs/modules/importing-modules.md


---

<!-- https://morloc-project.github.io/docs/modules/importing-modules.html -->

# 10.1. Importing modules

Morloc Manual > Modules and Libraries | https://morloc-project.github.io/docs/modules/importing-modules.html | prev: https://morloc-project.github.io/docs/modules/index.md | next: https://morloc-project.github.io/docs/modules/installing-modules.md

Every Morloc file is a module. A module declaration names the module and optionally lists the terms it exports:

```morloc
module mylib (foo, bar)
```

This declares a module named `mylib` that exports `foo` and `bar`. Only exported terms are visible to other modules that import this one.

If a module exports everything it defines, you can use the wildcard form:

```morloc
module mylib (*)
```

If a module’s export list is empty, it exports no named terms. This is useful for modules whose only purpose is to provide typeclass instances — instances travel with the module rather than the export list, so once a module is imported all of its instances become available. An instance-only module can therefore write:

```morloc
module myinstances ()
import .base
type Py => Int = "int"
instance Addable Int where
    source Py from "ops.py" ("add_int" as add)
```

To use the instances, write `import .myinstances` in the consuming module. Like in Haskell, typeclass methods are not picked individually — importing the module makes the entire instance available.

For submodules that exist only to be imported by a parent, you can omit the name entirely:

```morloc
module (*)
```

An anonymous module’s name is inferred from the path it is imported by. For example, if `main.loc` imports `.utils`, the compiler will resolve the module in `utils.loc` (or `utils/main.loc`) and assign it the name `utils`.

A single file may contain more than one module declaration. Each `module` keyword starts a new module; everything indented under it (or appearing before the next `module` keyword) belongs to that module. A module body may also be empty:

```morloc
module utils (helper)
import root-py
helper :: Int -> Int
helper x = x + 1

module main (run)
import utils (helper)
run :: Int -> Int
run x = helper x
```

Morloc distinguishes between two kinds of imports: **system** modules and **local** modules.

System modules are installed packages that live in `~/.local/share/morloc/lib/`. They are imported by name, without any prefix:

```morloc
import root-py
import root-cpp
```

System modules are installed with `morloc install`:

```console
$ morloc install root
$ morloc install root-py
```

`root` and a `root-<lang>` are not alternatives. `root` declares the standard library — the classes, and everything derivable from them — and a `root-<lang>` supplies the instances by sourcing native implementations. A program needs both, so its import list names a `root-<lang>`.

Which one it names sets what a single invocation costs, because the program starts that language’s runtime before it can call anything:

| Import | Per invocation | Where the time goes |
| --- | --- | --- |
| `root` alone, nexus intrinsics only | about 8 ms | the nexus process; no pool is started |
| `root-cpp` | about 21 ms | a compiled pool; roughly 13 ms of that is crossing into it |
| `root-py` | about 132 ms | a Python pool; starting the interpreter dominates |

Crossing into a pool is cheap and starting an interpreter is not. That rarely matters for a batch job and decides how a frequently typed command feels. Moving between backends is a one-line change to the import.

Local modules are files or directories within your own project. They are imported with a dot (`.`) prefix to distinguish them from system modules:

```morloc
import .utils (helper)
import .lib.math (square)
```

The dot prefix tells the compiler to look for the module inside your project rather than in the system library.

Both system and local imports support selective imports. Without a selector, all exported terms are brought into scope:

```morloc
import root-py             -- import everything from root-py
import .mylib              -- import everything from local mylib
import .mylib (foo, bar)   -- import only foo and bar from local mylib
```

When you write `import .foo`, the compiler looks for the module under the project root — the directory holding the entry file you passed to `morloc make`. It checks two locations, in order:

1.  A file module: `foo.loc`
2.  A directory module: `foo/main.loc`

Dot-separated paths map to nested directories. For example, `import .lib.math` resolves to either `lib/math.loc` or `lib/math/main.loc`.

The project root is fixed for a whole build, so a dotted import means the same thing in every file of a project however deeply nested that file is. Paths in a `source` declaration work the other way round: they resolve against the file that names them, so a module and the native code it sources travel together when either moves.

Here is an example project layout:

```
project/
  main.loc            -- module main, imports .utils and .lib.math
  utils.loc           -- module (*), a flat file module
  utils.py
  lib/
    math/
      main.loc        -- module (*), a directory module
      main.py
```

The top-level `main.loc` imports both:

```morloc
module main (negate_square, square_negate)

type Py => Real = "float"

import .utils (negate)
import .lib.math (square)

negate_square :: Real -> Real
negate_square x = negate (square x)

square_negate :: Real -> Real
square_negate x = square (negate x)
```

The flat file `utils.loc` exports `negate`:

```morloc
module (*)

source Py from "utils.py" ("negate")

type Py => Real = "float"

negate :: Real -> Real
```

And the directory module `lib/math/main.loc` exports `square`:

```morloc
module (*)

source Py from "main.py" ("square")

type Py => Real = "float"

square :: Real -> Real
```

Local modules can also import other local modules, and the path is still written from the project root. For example, if `bar/baz/main.loc` needs a module at `bif/biz/`, it writes:

```morloc
import .bif.biz (mul)
```

This resolves to `bif/biz.loc` or `bif/biz/main.loc` at the top of the project, not to anything beneath `bar/baz/`. The identical line in the top-level `main.loc` names the identical module.

Since `root` is also the name of a system module, a local directory named `root/` must be imported with the dot prefix to avoid ambiguity:

```morloc
import root         -- imports the system "root" module
import .root        -- imports the local "root/" directory
```

The dot prefix always forces local resolution, so there is never a collision between local and system module names.


---

<!-- https://morloc-project.github.io/docs/modules/installing-modules.html -->

# 10.2. Installing modules

Morloc Manual > Modules and Libraries | https://morloc-project.github.io/docs/modules/installing-modules.html | prev: https://morloc-project.github.io/docs/modules/importing-modules.md | next: https://morloc-project.github.io/docs/modules/universal-library.md

The default Morloc modules are hosted on GitHub under the [morloclib](https://github.com/morloclib) organization. Modules can be installed with the `morloc install` command:

```console
$ morloc install internal
$ morloc install root
$ morloc install root-cpp
$ morloc install root-py
$ morloc install root-r
```

The positional arguments to `morloc install` are **install strings**. Each describes one module to install and optionally pins it to a specific commit, branch, or release tag. The same shape is used in the `morloc-deps` entries of a `package.yaml`. More than one install string may be passed in a single invocation, in which case each is installed in turn (any transitive dependencies are also installed automatically).

```text
INSTALL  := [REMOTE ":"] NAME [ "@" [FORM ":"] REF ]
REMOTE   := "github" | "gitlab" | "bitbucket" | "codeberg" | "azure"
NAME     := <repo>           # core module on the default org (morloclib)
          | <owner>/<repo>   # repository under the chosen remote
          | <local-path>     # must start with '.', '/', or '~'
FORM     := "hash" | "branch" | "version" | "tag"   # 'tag' is an alias of 'version'
REF      := <hexhash>        # seven or more hex digits
          | <semver>         # [v]M.m[.p][-prerelease][+build]
          | <branch>         # a valid git branch name
```

Without an explicit `FORM:` prefix the `REF` is auto-detected by trying hash first, then version, then branch. If no `@REF` is given the default branch is used. The default `REMOTE` is `github`, the default core organization is `morloclib`, and clones use HTTPS unless `--ssh` is passed.

A bare name like `root` resolves to `morloclib/root` on github.com. An `owner/repo` form resolves to `github.com/owner/repo`. Adding a remote prefix selects a different host: `gitlab:` is gitlab.com, `codeberg:` is codeberg.org, `bitbucket:` is bitbucket.org. Local paths must begin with `.`, `/`, or `~`; everything from the path start to the first `@` is the path, and any `@REF` that follows is honored only when the local directory is itself a git repository — otherwise the working tree is copied verbatim (filtering `.git` and gitignored files) and the ref is silently dropped.

If the optional `registry` field is set in `~/.morloc/config.yaml`, bare names and bare `owner/name` strings (containing none of `:`, `@`, `.`, `~`) are resolved against the registry before falling back to a git clone from GitHub. Refs are not supported in registry mode — the registry’s latest version is always installed.

**Table 11. Install string examples**

| Install string | Effect |
| --- | --- |
| `root` | stdlib `root` from `morloclib/root` on GitHub (default branch) |
| `root root-py math` | install three modules in one invocation |
| `weena/calendar` | `weena/calendar` on GitHub (default branch) |
| `github:weena/calendar` | same as above, with the remote stated explicitly |
| `gitlab:weena/calendar` | `weena/calendar` on gitlab.com |
| `codeberg:weena/foo` | `weena/foo` on codeberg.org |
| `weena/calendar@1.0` | release tag `1.0` (semver auto-detected) |
| `weena/calendar@v1.0.0` | release tag `v1.0.0` |
| `weena/calendar@version:1.0.0` | explicit release-tag form |
| `weena/calendar@tag:1.0.0` | same as `@version:1.0.0` (`tag` is an alias) |
| `weena/calendar@a1b2c3d` | commit `a1b2c3d` (hash auto-detected at >= 7 hex chars) |
| `weena/calendar@hash:abc1234` | explicit commit-hash form |
| `weena/calendar@dev` | branch `dev` (auto-detected after hash/version fail) |
| `weena/calendar@branch:feature/x` | explicit branch form (allows `/` in the branch name) |
| `gitlab:weena/foo@version:1.0` | remote prefix combined with an explicit release tag |
| `.` | install from the current working directory |
| `./mymod` | install from a relative path |
| `~/code/mymod` | install from a tilde-expanded path |
| `/abs/path/to/mod` | install from an absolute path |
| `./mymod@branch:dev` | install branch `dev` of a local git repo |
| `./mymod@v0.2.0` | install tag `v0.2.0` of a local git repo |

The `morloc install -h` help screen carries a shorter example block for quick reference.

Installed modules are stored in `~/.local/share/morloc/lib/` and can be imported in any Morloc script.

To view the modules that are currently installed, you can run `morloc list`. This will list all installed modules, their version, and their short descriptions. Adding the `-v` option additionally prints the types of all exported terms.

To view just the exports of one desired module, you can include pattern that matches the module of interest:

```bash
$ morloc list -v il
Modules:
  internal
    pack :: a -> b
    unpack :: b -> a
    (.) :: (b -> c) -> (a -> b) -> a -> c
    ($) :: (a -> b) -> a -> b
```

Here `il` matches any module with a name including the ordered characters `i` and `l` — only `internal` in this case.

## 10.2.1. Configuring the C++ build

When a Morloc program contains C sources, the compiler invokes \`g\` (or whatever `$CXX` resolves to) to build the C++ side of the program. Three optional `package.yaml` fields tune that build: two structured fields (`cpp-version` and `dependencies`) that translate into specific flag patterns, and one verbatim field (`cxx-flags`) that passes arbitrary flags through unchanged. For build steps that go beyond flag-tweaking, a fourth field — `setup` — runs a shell script at install time.

**`cpp-version: 20`**

Selects the C standard. Translates directly into a \`-std=cNN\` flag (default `20`).

```yaml
cpp-version: 20
```

**`dependencies: [foo, bar]`**

Names shared libraries the C code links against. For each entry \`name\`, Morloc looks for \`lib<name>.so\` under \`~/.local/share/morloc/lib\` and a \`<name>.h\` / \`.hpp\` / \`.hxx\` under \`~/.local/share/morloc/include\`, then emits the matching \`-l\`, \`-I\`, \`-L\`, and \`-Wl,-rpath\` flags. Use this when your C code does `#include <foo.hpp>` and links against a `libfoo.so` that another Morloc package installed.

```yaml
dependencies:
  - eigen-cppmorloc
  - jsoncpp
```

**`cxx-flags: [-O3, -DENABLE_FAST_PATH]`**

A free-form list of flags appended verbatim to the compile command. Use this for anything the structured fields above don’t cover — optimization levels, architecture targeting, preprocessor defines, warning controls, and so on. Each list element is one shell argument, so no quoting or word-splitting is needed.

```yaml
cxx-flags:
  - -O3
  - -march=native
  - -DENABLE_FAST_PATH
```

The three fields combine in the obvious way:

```yaml
name: vector-ops
version: 0.1.0
cpp-version: 20
dependencies: [eigen-cppmorloc]
cxx-flags:
  - -O3
  - -march=native
  - -DEIGEN_NO_DEBUG
```

When one package depends on another, the downstream package inherits all three fields from its dependencies: a consumer of `vector-ops` above will automatically get `-leigen-cppmorloc` and the listed `cxx-flags` on its own C++ compile line.

For build steps that go beyond what these flags can express — running `cmake`, building a vendored library from source, fetching pre-built artifacts, or installing a Python or R package — use the more general `setup` field, which names a shell script that runs once at install time:

```yaml
name: my-package
setup: scripts/install.sh
```

The script runs after the module’s source has been laid down on disk and after morloc-level dependencies have been installed, with these environment variables set:

**`MORLOC_HOME`**

Morloc’s install root (default `~/.local/share/morloc`).

**`MORLOC_MODULE_NAME`, `MORLOC_MODULE_VERSION`**

name and version from this `package.yaml`.

**`MORLOC_MODULE_DIR`**

absolute path to the installed module’s directory; also the script’s working directory.

**`MORLOC_PLANE`, `MORLOC_PLANE_DIR`**

the active plane name and its directory.

A non-zero exit fails the install. The path must be relative to the module root and must not contain `..` segments. Prefer the three structured fields above when they suffice — a setup script is more powerful but also more code to maintain.


---

<!-- https://morloc-project.github.io/docs/modules/universal-library.html -->

# 10.3. The universal library

Morloc Manual > Modules and Libraries | https://morloc-project.github.io/docs/modules/universal-library.html | prev: https://morloc-project.github.io/docs/modules/installing-modules.md | next: https://morloc-project.github.io/docs/languages/index.md

A module may export types, typeclasses, and function signatures but no implementations. Such a module would be completely language agnostic. A powerful approach to building libraries in the Morloc ecosystem is to write one module that defines all types, then $n$ modules for language-specific implementations that import the type module, and then one module to import and merge all implementations. This is the approach taken by the `base` module and by other core libraries.

In the future, when hundreds of languages are supported, and when possibly some functions may even have many implementations per language, it will be desirable to have finer control over what functions are used. One solution would be to add filters to the import statement. Thus the import expressions would be a sort of query. Alternatively, constraints could be added at the function level, and thus the entire Morloc script would be a query over the universal library. This would be especially powerful when imported types are expressed as unknowns to be inferred by usage.


---

<!-- https://morloc-project.github.io/docs/languages/ -->

# 11. Language Support

Morloc Manual | https://morloc-project.github.io/docs/languages/ | prev: https://morloc-project.github.io/docs/modules/universal-library.md | next: https://morloc-project.github.io/docs/languages/build-parameters.md


---

<!-- https://morloc-project.github.io/docs/languages/build-parameters.html -->

# 11.1. Build parameters

Morloc Manual > Language Support | https://morloc-project.github.io/docs/languages/build-parameters.html | prev: https://morloc-project.github.io/docs/languages/index.md | next: https://morloc-project.github.io/docs/languages/cpp.md

Morloc compiles each part of a program with its own language toolchain. The `-X` flag passes key/value parameter pairs directly through to those builders:

```bash
morloc make -X futhark:backend=cuda -o nexus main.loc
morloc make -X cpp:flags=-march=native main.loc
```

The syntax is `-X LANG:KEY=VALUE`. It is repeatable. Morloc splits on the first `:` and the first `=`. Everything after the `=` is the value and is taken **verbatim** (it may itself contain `:` or `=`, as in `-X cpp:flags=-DFOO=bar`).

For every compiled language, the key `flags` is a raw passthrough: its value is appended verbatim to that language’s compile command, in order, without deduplication. This is the escape hatch for anything morloc does not model directly.

```bash
# each -X contributes one flag, appended in order
morloc make -X cpp:flags=-march=native -X cpp:flags=-O3 main.loc
```

Because order and adjacency matter to a compiler (`-L` before `-l`, paired tokens like `-Xpreprocessor -foo`), `flags` values are never reordered or merged away.

## 11.1.1. Per-machine defaults with `morloc config`

Repeating `-X futhark:backend=cuda` on every build is tedious on a machine that always has the same GPU. `morloc config` stores per-machine defaults so you set them once:

```bash
morloc config set futhark:backend=cuda   # every build now defaults to CUDA
morloc config list                       # show the current defaults
morloc config unset futhark:backend      # drop the default
```

These write the `lang-params` block of the per-machine build config at `$MORLOC_HOME/.build-config.yaml` (by default `~/.local/share/morloc/.build-config.yaml`):

```yaml
lang-params:
  futhark:
    backend: cuda
```

Unlike `morloc init`, `morloc config` only edits this file — it does not rebuild the toolchain.

## 11.1.2. Per-session defaults with `MORLOC_LANG_PARAMS`

A shell session, a build script, or a test harness that cannot edit every command line can set parameters in the environment. `MORLOC_LANG_PARAMS` holds a `;`\-separated list of the same `LANG:KEY=VALUE` entries:

```bash
export MORLOC_LANG_PARAMS='rust:lto=off;cpp:flags=-O1'
morloc make main.loc           # both parameters apply
```

Stray or trailing separators are ignored; a value cannot contain `;`.

## 11.1.3. Precedence

Parameters resolve in four layers, lowest to highest:

```text
compiled-in default  <  build config (per-machine)  <  MORLOC_LANG_PARAMS  <  -X on the command line
```

For most keys the highest layer wins. The `flags` key is the exception: values from each layer are concatenated (build-config flags, then environment flags, then command-line flags), so a per-machine default and a one-off flag both take effect.

## 11.1.4. Recognized keys

Morloc passes every `LANG:KEY` through; each language’s builder reads the keys it understands and rejects a value it does not. The keys the builders read:

| Key | Values | Default | Effect |
| --- | --- | --- | --- |
| `cpp:flags` | any | *(none)* | Raw flags appended to the C++ compile command. |
| `futhark:backend` | `c`, `multicore`, `opencl`, `cuda`, `hip`, `ispc` | `c` | Futhark code generation backend. |
| `futhark:device` | a device selector | *(none)* | Device for a GPU backend; rejected with a CPU backend. |
| `rust:lto` | `off`, `false`, `thin`, `fat`, `true` (Cargo’s `lto` vocabulary) | `thin` | Link-time optimization of the Rust pool. `thin` re-optimizes across the pool’s dependencies (rustmorloc, Arrow) on every link and costs seconds per pool; `off` links the precompiled dependencies as they are. |
| `rust:opt-level` | `0`, `1`, `2`, `3`, `s`, `z` | `2` | Cargo `opt-level` of the Rust pool. |

A non-default `rust:lto` or `rust:opt-level` compiles its own copy of the pool’s dependencies into the shared build cache the first time it is used; after that both profiles are cached side by side.

## 11.1.5. Build parameters vs. `package.yaml`

A project’s `package.yaml` is for **hard build requirements** — libraries the code cannot compile without (`dependencies`), a required C++ standard (`cpp-version`), or fixed compiler flags (`cxx-flags`). These are committed with the project and apply on every machine.

Build parameters are for **per-machine, per-invocation variance** — which Futhark backend this host’s hardware supports, a tuning flag like `-march=native` that is specific to the build machine. Keep these out of committed source: choose them on the command line, or set a per-machine default with `morloc config`. A `flags` passthrough is appended after the `package.yaml` flags, so the two compose.


---

<!-- https://morloc-project.github.io/docs/languages/cpp.html -->

# 11.2. C++

Morloc Manual > Language Support | https://morloc-project.github.io/docs/languages/cpp.html | prev: https://morloc-project.github.io/docs/languages/build-parameters.md | next: https://morloc-project.github.io/docs/languages/python.md

C++ has no language-level package manager, so a C++ pool’s dependencies are external **system libraries**: the environment provisions the library (its headers and shared objects) and the pool links against it.

## 11.2.1. Library dependencies (`cpp-deps`)

Declare the libraries a module needs in its `package.yaml` under `cpp-deps`, using `package: version-constraint`:

```yaml
cpp-deps:
  boost: ">=1.80"
```

`cpp-deps` are provisioned from the [conda-forge](https://conda-forge.org/) package database — installing the library’s headers and shared objects into the environment. `conda` is the default (and only) source, so the bare form above is equivalent to `boost: {version: ">=1.80", source: conda}`. The compiler takes the union across every imported module and the solver intersects conflicting constraints.

A `cpp-deps` entry may also name a conda `channel` other than conda-forge (see the Python chapter for the full rules), for a library that lives on a subordinate channel.

## 11.2.2. Link flags (`dependencies`)

Provisioning a library makes its headers available, but linking against a compiled component also needs a linker flag. List the link names under `dependencies` (a bare list); each entry `foo` becomes `-lfoo` on the pool’s compile line:

```yaml
cpp-deps:
  boost: ">=1.80"
dependencies:
  - boost_filesystem
  - boost_system
```

A header-only library (or a header-only part of a larger one) needs only `cpp-deps` and no `-l` flag. For any other compiler flags — extra include paths, `-DFOO`, `-O3`, `-march=native` — use `cxx-flags`, a bare list appended verbatim to the compile line. All three fields propagate transitively through a module’s dependencies.

The `local-deps` section that Python and Rust use for in-project packages ([Local packages (`local-deps`)](https://morloc-project.github.io/docs/languages/python.md#local-deps)) is not available for C++, which has no package manager to track such an install; a library you build yourself is linked through `dependencies` and `cxx-flags` like any other.


---

<!-- https://morloc-project.github.io/docs/languages/python.html -->

# 11.3. Python

Morloc Manual > Language Support | https://morloc-project.github.io/docs/languages/python.html | prev: https://morloc-project.github.io/docs/languages/cpp.md | next: https://morloc-project.github.io/docs/languages/r.md

## 11.3.1. Dependencies (`py-deps`)

Declare a module’s Python dependencies in its `package.yaml` under `py-deps`. Every entry must state its `source` — the package database it is drawn from — because Python packages split across two incompatible worlds:

```yaml
py-deps:
  matplotlib: {version: ">=3.5", source: "conda"}
  requests: {version: "*", source: "pypi"}
```

`source: conda` draws the package from the [conda-forge](https://conda-forge.org/) package database; `source: pypi` draws it from the [Python Package Index](https://pypi.org/). Use **conda** for anything that ships a compiled C/C++/Fortran extension (`numpy`, `scipy`, `pandas`, `matplotlib`, `pyarrow`, `pillow`, …​) so it shares one ABI with the rest of the environment; **pypi** is appropriate for pure-Python packages that are not mirrored on conda-forge.

Unlike the other languages, `source` is **mandatory** for Python: a bare `matplotlib: ">=3.5"` is a build error, because the correct database cannot be guessed safely. Names are used exactly as written (there is no PyPI-to-conda renaming table), so under `source: "conda"` use the conda-forge package name where it differs from the PyPI name.

The compiler takes the union of `py-deps` across every imported module; two modules constraining the same package differently are intersected by the solver rather than being an error.

### conda channels

A conda dependency may name the **channel** it is drawn from — a distinct package database within conda — for a package that lives on a channel other than conda-forge:

```yaml
py-deps:
  pysam: {version: "*", channel: "bioconda"}
  numpy: {version: ">=2", source: "conda"}
```

The `channel` field implies `source: conda` (channels are a conda-only concept), so `pysam` above needs no explicit `source`. conda-forge remains the universal default and the highest-priority base; a named channel such as [bioconda](https://bioconda.github.io/) sits **below** it under strict priority and can only supply packages conda-forge lacks. The environment’s channel list is **derived** automatically from the channels its modules declare — channels are never configured at the environment level, because a package’s database of record is a property of the module that needs it, not of where it is deployed.

`channel` is valid only on a conda dependency: pairing it with `source: pypi` is a build error. And because a package has a single database of record, two modules that draw the same package from **different** channels is a build error naming both modules — unlike differing version constraints, which are simply intersected.

## 11.3.2. Local packages (`local-deps`)

A Python package that lives in your project tree rather than in a package database — typically the helper package you are writing alongside the Morloc program — is declared under `local-deps`. The section is keyed by language, and each entry gives a path relative to the module’s directory:

```yaml
local-deps:
  py:
    greet:
      path: ./src/greet
      editable: true
```

`path` must be a pip-installable package: a directory holding a `pyproject.toml` or `setup.py`. When you run `morloc make`, the environment installs it into the pool’s Python, so pool code can `import greet` like any other package. With `editable: true` the install points at your source tree, so an edit to the package is picked up on the next run of the program without rebuilding it; the default is `false`, a snapshot taken at build time. An installed program (`mim install`) always gets a snapshot, whatever `editable` says, so a running service does not change under you.

`source Py from` still names a file, not a package, so reach the package through a one-line bridge:

**glue.py**

```python
from greet import greet
```

```morloc
source Py from "glue.py" ("greet")
```

Three rules keep a local dependency portable. Only the top-level module being built may declare `local-deps`; an imported module’s relative path has no meaning from the build root, and the build refuses it. The path must stay inside the project: it cannot be absolute and cannot contain `..`. To depend on a package stored elsewhere, put a symlink to it inside the project and point `path` at the symlink. And only Python and Rust support `local-deps`; the build rejects it for C++ and R. Each rule fails with an error naming the dependency:

```console
$ morloc make main.loc
morloc: user error (local dependency 'greet' for py has path '../greet' that must not contain '..' (use an in-project symlink for an external source))
```


---

<!-- https://morloc-project.github.io/docs/languages/r.html -->

# 11.4. R

Morloc Manual > Language Support | https://morloc-project.github.io/docs/languages/r.html | prev: https://morloc-project.github.io/docs/languages/python.md | next: https://morloc-project.github.io/docs/languages/rust.md

## 11.4.1. Dependencies (`r-deps`)

Declare a module’s R dependencies in its `package.yaml` under `r-deps`, using the canonical CRAN package name and a version constraint:

```yaml
r-deps:
  ggplot2: ">=3.4"
```

R packages are provisioned from the [conda-forge](https://conda-forge.org/) package database, which mirrors CRAN as `r-<name>` feedstocks. You write the plain CRAN name; morloc lowercases it and prefixes `r-` when lowering, so `ggplot2` above is installed as the conda package `r-ggplot2`. Do not write the `r-` prefix yourself.

`conda` is the default — and, for now, only — source, so the bare form above is equivalent to `ggplot2: {version: ">=3.4", source: conda}`. The `cran` and `bioconductor` sources (installing directly from those registries, compiling against the environment’s R) are recognized but not yet supported; declaring one is a build error today.

The `r-`\-prefixing applies only under conda-forge. An R dependency may instead draw from another conda `channel` (see the Python chapter), in which case the name is passed directly to that channel:

```yaml
r-deps:
  bioconductor-deseq2: {version: "*", channel: bioconda}
```

As with the other languages, the compiler takes the union of `r-deps` across every imported module and the solver intersects conflicting constraints. Only packages that exist as a conda-forge `r-` feedstock can be resolved this way; most of CRAN (and Bioconductor) is mirrored there.


---

<!-- https://morloc-project.github.io/docs/languages/rust.html -->

# 11.5. Rust

Morloc Manual > Language Support | https://morloc-project.github.io/docs/languages/rust.html | prev: https://morloc-project.github.io/docs/languages/r.md | next: https://morloc-project.github.io/docs/languages/futhark.md

## 11.5.1. Cargo pools

Each Rust pool is a generated Cargo project, built with `cargo build --release`:

```
<name>-build/pools/rust/
├── Cargo.toml     # generated manifest
├── build.rs       # links libmorloc and sets the rpath
└── src/main.rs    # generated pool code
```

`cargo` must be on your `PATH`; install it via [rustup](https://rustup.rs).

`morloc init` copies the Rust runtime workspace to `$MORLOC_HOME/rust` (a Cargo path dependency of every pool) and warms a shared build cache at `$MORLOC_STATE/cache/rust-build`. Re-run `morloc init -f` after changing compiler versions.

## 11.5.2. Dependencies (`rust-deps`)

Declare crates.io dependencies in a module’s `package.yaml` under `rust-deps`, using `crate: version` as in a `Cargo.toml`:

```yaml
rust-deps:
  ndarray: "0.16"
```

Each entry is written into the pool `Cargo.toml` `[dependencies]`. The compiler takes the union of `rust-deps` across all imported modules; conflicting versions of the same crate are a build error.

Crates come from [crates.io](https://crates.io), the only source Rust supports, so the bare form above is equivalent to `ndarray: {version: "0.16", source: crates}`; declaring any other source is a build error. The conda `channel` field is likewise conda-only and has no meaning for crates.

## 11.5.3. Local crates (`local-deps`)

A crate that lives in your project tree rather than on crates.io — the shared library you are writing alongside the Morloc program — is declared under `local-deps`, keyed by language, with a path relative to the module’s directory:

```yaml
local-deps:
  rust:
    statcrate:
      path: ./statcrate
```

This is the Cargo path dependency you would write by hand: the compiler adds `statcrate = { path = "<absolute path>/statcrate" }` to the pool’s `Cargo.toml`, and cargo builds the crate and links it statically into the pool binary. There is no install step and nothing to load at run time.

`source Rust from` names a file, and the compiler includes that file’s text in the pool crate, so reach the crate through a one-line re-export:

**glue.rs**

```rust
pub use statcrate::{mean, stdev};
```

```morloc
source Rust from "glue.rs" ("mean" as colMean, "stdev" as colStdev)
```

The rules are the ones given for Python in [Local packages (`local-deps`)](https://morloc-project.github.io/docs/languages/python.md#local-deps): only the top-level module may declare `local-deps`, and the path must be relative and stay inside the project (an in-project symlink reaches a crate kept elsewhere). The `editable` flag has no meaning for a crate, which is recompiled on every build anyway.

One limit is specific to Rust. A Morloc `record` cannot be bound to a struct defined in the crate, because the marshalling code Morloc generates for a record must be an `impl` in the pool crate, and Rust’s orphan rule forbids implementing a foreign trait for a foreign type. Share functions across the crate boundary, pass primitives, lists, tuples and strings, and assemble records on the Morloc side; a crate struct that must cross the boundary goes through a tuple in the bridge file.

## 11.5.4. One resolution per environment

Every pool starts from one `Cargo.lock`: the lock persisted with the runtime at `$MORLOC_HOME/rust/Cargo.lock`, extended by every crate a pool in this environment has resolved before (kept at `$MORLOC_STATE/cache/rust-env.lock`). A pool whose crates are all pinned there is built `--offline`; cargo never consults the registry and every pool links the same compiled dependencies. A pool that declares a crate not yet in the lock resolves it once, over the network, and its pins are merged back, so the next pool needing that crate — in any program — builds offline too. Deleting `rust-env.lock` costs one online resolution per external crate and nothing else.

The pool’s release profile is set by the `rust:lto` and `rust:opt-level` build parameters ([Build parameters](https://morloc-project.github.io/docs/languages/build-parameters.md)); `rust:lto` is the one setting that trades link time against cross-crate optimization.


---

<!-- https://morloc-project.github.io/docs/languages/futhark.html -->

# 11.6. Futhark

Morloc Manual > Language Support | https://morloc-project.github.io/docs/languages/futhark.html | prev: https://morloc-project.github.io/docs/languages/rust.md | next: https://morloc-project.github.io/docs/install/index.md

## 11.6.1. Futhark backends and GPU devices

Futhark kernels are the first structured consumer of build parameters. Two keys are recognized:

-   `backend` — the Futhark compilation backend: one of `c` (the default, sequential CPU), `multicore` (parallel CPU), `opencl`, `cuda`, `hip`, or `ispc`. The GPU backends (`opencl`, `cuda`, `hip`) are what make Futhark run on a GPU.
-   `device` — which device to run on, for a GPU backend, using Futhark’s device selector syntax (`#0`, `#1`, …​). It is only valid together with a GPU backend.

```bash
# build the Futhark kernels for CUDA and run on the second GPU
morloc make -X futhark:backend=cuda -X futhark:device=#1 -o nexus main.loc
```

A GPU build runs on the single selected device, and calls into the Futhark context are serialized. The required GPU runtime libraries (for example `-lcuda -lnvrtc` for CUDA) are linked automatically; if your toolchain lives in a non-standard location, add the paths with the C++ escape hatch, e.g. `-X cpp:flags=-L/usr/local/cuda/lib64`.


---

<!-- https://morloc-project.github.io/docs/install/ -->

# 12. Installation, Versions, and Deployment

Morloc Manual | https://morloc-project.github.io/docs/install/ | prev: https://morloc-project.github.io/docs/languages/futhark.md | next: https://morloc-project.github.io/docs/install/container-engine.md

`mim` is distributed as a static binary for x86 Linux, Linux ARM, and Apple Silicon macOS. The installer in [Installing `mim`](https://morloc-project.github.io/docs/getting-started/installing.md#installing-mim) fetches it; the binaries themselves are on the [morloc-manager releases page](https://github.com/morloc-project/morloc-manager/releases). Each Morloc release is a version `mim` can provision into an environment, and this chapter covers the commands that create, pin, customize, and deploy those environments. The full `mim` reference is [`mim` (Morloc Installation Manager)](https://morloc-project.github.io/docs/utilities/mim.md).


---

<!-- https://morloc-project.github.io/docs/install/container-engine.html -->

# 12.1. Choosing a container engine

Morloc Manual > Installation, Versions, and Deployment | https://morloc-project.github.io/docs/install/container-engine.html | prev: https://morloc-project.github.io/docs/install/index.md | next: https://morloc-project.github.io/docs/install/creating-an-environment.md

There is no separate setup step. `mim new` picks the container engine and remembers it for later environments: if exactly one engine is installed it is auto-detected, otherwise pass `--engine` on your first `new`:

```console
$ mim new --engine podman    # or: --engine docker, --engine apptainer
```

`--engine` accepts `docker`, `podman`, `apptainer`, or `singularity` (the last is an alias for `apptainer`, the common engine on HPC clusters), or `none` for the native (no-container) backend. The choice made on the first `new` becomes the default for subsequent environments.


---

<!-- https://morloc-project.github.io/docs/install/creating-an-environment.html -->

# 12.2. Creating an environment

Morloc Manual > Installation, Versions, and Deployment | https://morloc-project.github.io/docs/install/creating-an-environment.html | prev: https://morloc-project.github.io/docs/install/container-engine.md | next: https://morloc-project.github.io/docs/install/running-commands.md

An **environment** is a self-contained Morloc installation: a base container image (or, on the native backend, none), a pinned Morloc compiler and runtime, a solved language toolchain, any extra packages you asked for, engine flags, and its own module and binary directories. Everything in Morloc happens inside an environment.

```console
$ mim new                                  # named after the version: 'latest'
$ mim new myenv                            # use latest morloc release
$ mim new myenv --morloc-version 0.105.2   # pin a specific version
```

The first environment you create becomes the **default** — the one every command targets when you do not pass `--env`. There is no separate "activate" step. To point the default at a different environment later, use `modify --set-default`:

```console
$ mim modify --env myenv --set-default
```


---

<!-- https://morloc-project.github.io/docs/install/running-commands.html -->

# 12.3. Running commands

Morloc Manual > Installation, Versions, and Deployment | https://morloc-project.github.io/docs/install/running-commands.html | prev: https://morloc-project.github.io/docs/install/creating-an-environment.md | next: https://morloc-project.github.io/docs/install/managing-environments.md

`run` executes a command inside an environment (the default when no `--env` is given); `shell` drops you into an interactive shell:

```console
$ mim run -- morloc make -o hello hello.loc   # default env
$ mim run --env myenv -- ./hello 21           # a named env
$ mim shell                                   # interactive shell in the default env
```

Pass container environment variables with `--env-var KEY=VALUE` (or `--env-file`); `--env` names the morloc environment.

The current directory is bind-mounted into the container so source files and build artifacts are shared with the host. On SELinux systems (Fedora, RHEL), the `:z` relabel suffix is applied automatically, and because relabeling `~` itself, `/`, or `/tmp` would be unsafe, `mim` refuses those directories there: work in a subdirectory such as `~/myproject`.


---

<!-- https://morloc-project.github.io/docs/install/managing-environments.html -->

# 12.4. Managing environments

Morloc Manual > Installation, Versions, and Deployment | https://morloc-project.github.io/docs/install/managing-environments.html | prev: https://morloc-project.github.io/docs/install/running-commands.md | next: https://morloc-project.github.io/docs/install/customizing-environments.md

You can keep multiple named environments and act on any of them with `--env`:

```console
$ mim ls                          # list all environments (marks the default)
$ mim modify --env myenv --set-default   # make myenv the default
$ mim info myenv                  # detailed info for myenv
$ mim info                        # overview of all environments
$ mim rm myenv                    # remove an environment
```

Because every command accepts `--env`, you never have to change the default to work with another environment — and a bare command is legible from shell history, since it always means the (rarely changed) default. Removing the environment that is currently the default simply clears the default tag; set a new one with `modify --set-default`.

Environments can be created at local scope (per-user, the default) or system-wide (`--system`, requires root). Local environments shadow system environments of the same name. A regular user can `run`/`shell`/`eval` a system environment without being able to modify it, which makes a `--system` default a convenient shared, read-only place to run code.

```console
$ sudo mim new shared --morloc-version 0.105.2 --system
$ sudo mim modify --env shared --set-default --system
$ mim run -- morloc --version
```


---

<!-- https://morloc-project.github.io/docs/install/customizing-environments.html -->

# 12.5. Customizing environments

Morloc Manual > Installation, Versions, and Deployment | https://morloc-project.github.io/docs/install/customizing-environments.html | prev: https://morloc-project.github.io/docs/install/managing-environments.md | next: https://morloc-project.github.io/docs/install/extra-container-flags.md

An environment’s contents are declared, not scripted: there is no user Dockerfile. Extra software comes in through `mim new` and `mim modify` and is solved together with the language toolchain, so the declaration — not a build log — is what reproduces the environment. The two package sources are covered in [Packages](https://morloc-project.github.io/docs/utilities/mim.md#mim-packages):

```console
$ mim new scipy --conda-packages-file tools.conda    # conda-forge packages
$ mim new scipy --system-packages-file tools.apt     # apt packages (containers only)
$ mim modify --env scipy --conda-packages-file tools.conda   # change later, rebuilds
```

Python and R libraries that a **program** needs are not declared here at all: a module’s `package.yaml` lists them, and `mim` solves them into the environment when the program is built ([Dependencies (`py-deps`)](https://morloc-project.github.io/docs/languages/python.md#py-deps), [Dependencies (`r-deps`)](https://morloc-project.github.io/docs/languages/r.md#r-deps), [Dependencies (`rust-deps`)](https://morloc-project.github.io/docs/languages/rust.md#rust-deps)). Pin language versions with `--lang` ([Language toolchains](https://morloc-project.github.io/docs/utilities/mim.md#mim-language-toolchains)).

Container backends also let you choose the base image with `--base`: `heavy` (the default, `ubuntu:24.04`) or `light` (`debian:bookworm-slim`, smaller, fewer preinstalled packages). Both are accepted by `new` and `modify`, and changing the base rebuilds the environment.


---

<!-- https://morloc-project.github.io/docs/install/extra-container-flags.html -->

# 12.6. Extra container flags

Morloc Manual > Installation, Versions, and Deployment | https://morloc-project.github.io/docs/install/extra-container-flags.html | prev: https://morloc-project.github.io/docs/install/customizing-environments.md | next: https://morloc-project.github.io/docs/install/serve-and-freeze.md

Anything a container engine needs beyond what `mim` passes on its own — a GPU device, a proxy, a bind mount, a hostname — goes in an **engine flag file**: a YAML document partitioned by phase and by engine. A phase is one of the three engine verbs an environment performs: `build` (the image build, at `new`, `update`, and `freeze`), `run` (`run`, `shell`, `install`), and `start` (the serving container). Under each phase, an engine section (`docker`, `podman`, `apptainer`) lists the flags that engine gets, and an `all` section lists the flags every engine gets. Each entry is one argument as the engine would see it on its command line:

```yaml
# flags.yaml
build:
  all:
    - --build-arg=HTTP_PROXY=http://proxy.example.com:3128
  podman:
    - --no-cache

run:
  all:
    - --volume=/data/reference:/ref:ro     # a host directory every program can read
  apptainer:
    - --nv                                 # NVIDIA GPU passthrough
  podman:
    - --device=/dev/dri

start:
  all:
    - --hostname=morloc-serve
```

Every section is optional, and an absent one is empty. The schema is strict: a section or engine name outside this set is a parse error, so a misspelt `podmann:` is refused rather than ignored. Entries pass through the shell before they reach the engine, so `$HOME`, `~`, and globs expand, and a value that needs a space must be quoted as it would be in a shell.

Install the file when the environment is created, or on an existing one with `modify`, which also takes `--no-flagfile` to remove it:

```console
$ mim new gpu --engine podman --flagfile flags.yaml
$ mim modify --env gpu --flagfile flags.yaml      # replace it; build flags apply at the next `update`
$ mim modify --env gpu --no-flagfile              # drop it
```

The file is validated and then copied whole — comments included — to `~/.config/morloc/environments/<name>/env.flags.yaml`, replacing whatever was there; it is never merged into an existing one. `mim info <name>` names the file and prints what each phase materializes to for the environment’s engine. The `build` section is part of the image’s cache key, so changing it is enough to make the next `update` rebuild.

For each invocation the flag list is `<phase>.all ++ <phase>.<engine>`, with any one-shot CLI override appended after it. Two overrides exist, and neither touches the persisted file:

-   `-x <flag>` (`--engine-arg`) appends one flag for this invocation, and is repeatable. On `run`, `shell`, and `install` it joins the `run` phase; on `start`, the `start` phase; on `new` and `update`, the `build` phase.
-   `--flagfile <file>` on `run`, `shell`, `install`, and `start` uses that file **instead of** the environment’s for this invocation, which is how a second instance of a serving environment comes up under different flags.

```console
$ mim run -x --device=/dev/dri -- ./prog render     # one extra flag, this run only
$ mim run --flagfile gpu.yaml -- ./prog render      # a different file, this run only
$ mim start --flagfile alt.yaml -p 9090:8080        # a second serve under other flags
```

One flag is owned by `mim` and refused in the file and on the command line alike: `--platform`, because the environment’s `--arch` chooses it and a raw platform flag would run the image on an architecture its toolchain was not solved for. The whole facility is for container backends; the native backend has no engine to pass flags to, and `--flagfile` is an error there.


---

<!-- https://morloc-project.github.io/docs/install/serve-and-freeze.html -->

# 12.7. Deployment: serve and freeze

Morloc Manual > Installation, Versions, and Deployment | https://morloc-project.github.io/docs/install/serve-and-freeze.html | prev: https://morloc-project.github.io/docs/install/extra-container-flags.md | next: https://morloc-project.github.io/docs/install/composable-function-services.md

Once a program is installed in an environment (`mim install`, or `morloc make --install` inside it) there are two ways to put it in front of other machines and other people, and they are complementary:

-   **`start`** serves an environment in place: the environment’s state is bind-mounted, read-only, into a container (or, on the native backend, a host process) that answers HTTP. Fast, and no build step.
-   **`freeze`** bakes the environment into a self-contained container image that runs the same programs, and serves the same views, with nothing mounted in from outside. This is the artifact you push to a registry or hand to a cluster.

Both serve only what you have declared with `mim view` — installing a program does not make it reachable. The whole lifecycle, from `install` through `view` to `start` and `freeze`, is walked through in [Serving installed programs](https://morloc-project.github.io/docs/utilities/mim.md#mim-serve); this section is the short form.

## 12.7.1. Local serving with `start`

```console
$ mim start                   # serve the default environment on :8080
$ mim start --env myenv -p 9090:8080
$ mim status                  # list running servers
$ mim logs --env myenv        # view logs
$ mim logs --env myenv -f     # stream logs
$ mim stop --env myenv        # stop the container
```

The serve process is `morloc-nexus router`, one HTTP listener that answers:

**`GET /health`**

Liveness check; answers before any token check

**`POST /mcp`**

The MCP endpoint, for AI assistants

**`GET /discover`**

List the modules on the JSON API and whether eval is callable

**`GET /discover/<module>`**

Show a module’s commands and their ordered arguments

**`POST /call/<module>/<command>`**

Invoke one command with positional JSON arguments

**`POST /eval`**

Compose and evaluate expressions from installed modules

By default the listener binds the host’s loopback and needs no token; reaching it from elsewhere is an opt-in (`--expose`, `--auth-token`) described in [start](https://morloc-project.github.io/docs/utilities/mim.md#mim-start).

## 12.7.2. Portable images with `freeze`

> **Warning: Experimental Feature**
> `freeze` is in development and likely to change. It builds with the Docker or Podman engine: it refuses a native environment (a native freeze would be a multi-platform lock, not an image), a dev environment (`--dev`), and one built from a `--local-runtime`, since neither of the last two pins a released compiler. It does not yet handle an Apptainer environment either.

For deploying to a different machine, freeze the environment into a tagged image and move that:

```console
$ mim freeze --tag myservice:v1
$ mim freeze --tag myservice:v1 --save ./myservice-v1.tar   # also write a tarball
$ docker run -d -p 8080:8080 myservice:v1                   # serves the declared views
$ docker run myservice:v1 <program> <command> <args>        # or run a program by name
```

Nothing lands in your working directory: the image is a tag in the engine’s image store (default `morloc-<env>:<version>`), and `--save` is the engine’s own `docker save`, so the tarball restores with `docker load` and needs nothing else. The image carries its provenance as labels — the Morloc version, the environment it came from, the programs inside, and which of them answer on each adapter — and `docker inspect` reads them back.

The full image is the environment whole, compiler and toolchain included, which is what lets it eval and rebuild a pool. Add `--slim` for an image that keeps every interpreter and package but drops the compilers, pixi, and the Morloc compiler — about half the size, tagged `-slim` by default, and refused if the environment exposes eval, since a slim image cannot evaluate. Before building, `freeze` audits each installed program and refuses one that carries tool state (a `.git/`, a cargo `target/`); the fix is the project’s `.morlocignore`. See [freeze](https://morloc-project.github.io/docs/utilities/mim.md#mim-freeze) for the audit, the slim cut, and how the image decides whether to require a token.


---

<!-- https://morloc-project.github.io/docs/install/composable-function-services.html -->

# 12.8. Beyond fixed APIs: composable function services

Morloc Manual > Installation, Versions, and Deployment | https://morloc-project.github.io/docs/install/composable-function-services.html | prev: https://morloc-project.github.io/docs/install/serve-and-freeze.md | next: https://morloc-project.github.io/docs/install/execution-contexts.md

A conventional API exposes a fixed set of endpoints. A Morloc serve container goes further: in addition to calling pre-compiled commands, the `/eval` endpoint lets callers compose *new* expressions from the functions available in installed modules. Because Morloc’s type system spans all installed languages, these compositions are type-checked before execution, and the runtime handles all cross-language marshalling automatically.

This means a single deployed container does not just serve a finite set of functions — it serves the entire *composition space* of every function in every installed module. An agent or client can discover available functions via `/discover`, read their argument schemas, and synthesize novel pipelines that were never anticipated at build time, all within the safety guarantees of the type system.

Eval is off until you turn it on (`mim view eval --allow <modules>`), and the safety model for it relies on several layers:

-   The serve-mode parser accepts only a restricted subset of the Morloc language: callers can compose primitives and functions from the modules on the eval allow-list, but cannot source new foreign code or import modules outside it
-   Module resolution is checked at compile time — only functions from installed modules are reachable
-   The type system prevents invalid compositions across language boundaries
-   The container runs with a read-only filesystem and a CPU-time limit on each evaluation, and an off-box endpoint serves eval only to holders of a bearer token ([freeze](https://morloc-project.github.io/docs/utilities/mim.md#mim-freeze))


---

<!-- https://morloc-project.github.io/docs/install/execution-contexts.html -->

# 12.9. Execution contexts

Morloc Manual > Installation, Versions, and Deployment | https://morloc-project.github.io/docs/install/execution-contexts.html | prev: https://morloc-project.github.io/docs/install/composable-function-services.md | next: https://morloc-project.github.io/docs/install/system-wide-podman.md

A term in morloc can be annotated with a label that associates the term with a computational context. Currently the main use case for this is the submission of jobs to remote worker nodes in an HPC context.

In the example below, the compute heavy function `run` is mapped over a list of input strings, each call to `run` returns an integer, the resulting integer list is reduced to a final output integer by `reduce`:

**heavy.loc**

```morloc
module heavy (foo)

import root-py
source Py from "heavy.py" ("run", "reduce")
run :: Str -> Int
reduce :: [Int] -> Int

foo :: [Str] -> Str
foo = reduce . map big@run
```

Here we have annotated `run` with the label `big`. This provides a handle for controlling how and where this heavy function is evaluated. Complex terms may also be labeled, allowing entire branches of the execution tree to be remotely evaluated. Labels may be arbitrarily nested, as shown in the example below:

**pipeline.loc**

```morloc
module pipeline (analyze)

import root-py
import root-cpp
import root-r

source Py  from "munge.py"     ("munge")
source Cpp from "analyze.hpp"  ("analyze")
source R   from "summarize.R"   ("summarize")

munge :: Str -> [Str]
analyze :: Str -> Real
reduce :: [[Real]] -> Real

bigStep :: Str -> Real
bigStep input = reduce (map big@analyze (munge input))

analyzeMany :: [Str] -> [(Str, Real)]
analyzeMany inputs = zip (map big@bigStep) inputs
```

Here `analyzeMany` submits can submit a remote job for each `bigStep` call and `bigStep` call, in turn, submits a remote call for each `analyze` call. We can flatten the execution style by removing the label on `analyze`.

## 12.9.1. The `remote:` context: dispatching as a separate job

The labels can be annotated in YAML config files that specialize the execution context for the associated term.

A `remote:` block in the label’s YAML config tells the runtime to evaluate the labeled subtree as a separate morloc invocation (through mim) instead of in-process. Without the block the label is parsed and grouped but the term runs in-process.

**heavy.yaml**

```yaml
labeled-groups:
  big:
    remote:
      threads: 8
      memory: 32        # GB
      time: 3600        # seconds
      gpus: 0
```

Dispatch sends only the outer input and output across the job boundary: a `mim` invocation that picks up the job brings up every pool the subtree references and runs all the internal foreign calls between them locally within that one invocation. The shape of the labeled term — single function or composition — does not change the protocol.

> **Note**
> SLURM is currently the only wired-up dispatch backend. The bridge wraps a `mim run` invocation in `sbatch` and polls completion via `sacct`. A future local-container backend (``exec`ing the same `mim run`` directly on the host for dependency isolation), a Kubernetes backend, or a PBS backend would consume the same `remote:` config without source-level changes. The rest of this section walks through the SLURM path.

## 12.9.2. Enabling dispatch codegen

The morloc compiler only emits the dispatch op when its build config opts in. The current opt-in is the `--slurm` flag of `morloc init` (generic name pending; it enables the same dispatch-codegen path that any future backend will share). `mim` runs `morloc init` for you when it materializes an environment, without that flag, so re-run init inside the environment once it exists:

```console
$ mim new hpcdemo --engine apptainer
$ mim run --env hpcdemo -- morloc init -f --slurm
```

Any rebuild — `mim update`, or a `mim modify` that changes languages or packages — runs `morloc init` again without the flag, so repeat the second command afterwards. `mim doctor --slurm` (below) reports when it is missing.

## 12.9.3. Running with the dispatch bridge

The runtime flag is `--slurm-bridge` (the name will generalize as other backends land):

```console
$ mim run --env hpc --slurm-bridge -- bash -c "morloc make heavy.loc && ./heavy foo 5"
```

When the flag is set, mim:

1.  Spawns a Unix-domain-socket bridge thread on the host.
2.  Launches the driver container with that socket bind-mounted at `/run/morloc-bridge.sock` and `MORLOC_BRIDGE_SOCKET` exported.
3.  Each labeled term that gets evaluated inside the container becomes a JSON RPC over the socket. The runtime hands the bridge a structured argv — a `mim run` invocation that, when executed, brings up the same env’s container and runs the nexus in call-packet mode with the labeled subtree as its payload.
4.  The bridge wraps that argv in the current backend. On SLURM the wrap is `sbatch --wrap='<argv>'` and the wrapped command lands on a compute node. A future local-container backend would just `exec` the same argv on the host; a Kubernetes backend would create a Job manifest, etc. In every case the argv is identical and the job ends up writing its result packet to a shared `.morloc-cache/<hash>.dat` file.
5.  The driver pool polls job status through the bridge (on SLURM via `sacct -j ID`), reads the result, validates the packet header, and returns the bytes up the call chain.

The dispatched job is also launched with `--slurm-bridge`, so any nested labeled terms encountered while evaluating the dispatched subtree can themselves fan out to fresh jobs.

## 12.9.4. What lands on the shared filesystem

All inter-job bytes flow through content-addressed files in the project’s `.morloc-cache/` directory:

**`<arg-hash>.dat`**

raw msgpack payload of one argument (data-sized)

**`<arg-hash>.packet`**

small header packet that references the `.dat` by path (~100 bytes)

**`<call-hash>-call.dat`**

the assembled call packet (just a CALL header plus the small arg-reference headers — tiny regardless of arg size)

**`<call-hash>.dat`**

result packet, doubles as the memoization cache

**`<call-hash>.out` / `.err`**

stdout/stderr captured by the backend (e.g. by SLURM for sbatch’d jobs)

Two consequences worth highlighting: a large argument (e.g. a genome) is written once under its hash and referenced thereafter, and a re-run with unchanged args hits the cached result without dispatching a fresh job.

## 12.9.5. Preconditions and the `doctor` check

What the dispatch design requires depends on the backend. The backend-agnostic preconditions are minimal: the active env has a runnable image, and the bridge socket directory is writable. The SLURM backend adds cluster-shape requirements:

-   The user’s `$HOME` is mounted on every compute node at the same absolute path (the HPC norm via NFS). A local-container backend would not need this.
-   The `mim` binary lives at a path reachable from every compute node (typical: `~/.local/bin/mim` on an NFS-mounted `$HOME`). A local backend would just use whichever binary is currently running.
-   `sbatch` and `sacct` are on the host’s PATH.
-   The active env’s image is reachable from every compute node. Apptainer satisfies this trivially because `.sif` is a file on the shared FS. Podman and Docker work too as long as you handle image distribution yourself (registry pull or pre-populated store).

To check all of the above before relying on the bridge:

```console
$ mim doctor --slurm
```

Each check reports PASS / WARN / FAIL with an actionable hint. The flag is `--slurm` for the same reason as the codegen flag above; it exercises the SLURM realization of the dispatch path.


---

<!-- https://morloc-project.github.io/docs/install/system-wide-podman.html -->

# 12.10. System-wide environments with Podman

Morloc Manual > Installation, Versions, and Deployment | https://morloc-project.github.io/docs/install/system-wide-podman.html | prev: https://morloc-project.github.io/docs/install/execution-contexts.md | next: https://morloc-project.github.io/docs/internals/index.md

Podman stores images per-user. After creating a system environment with `sudo`, configure rootless Podman to read the rootful image store by adding this line to the `[storage.options]` section of `/etc/containers/storage.conf`:

```
additionalimagestores = ["/var/lib/containers/storage"]
```

No Podman restart is needed; the setting is re-read on every invocation.

Apptainer needs no analogous configuration: `.sif` files are plain files on disk, so a system environment dropped under `/usr/local/share/morloc/environments/<env>/` is readable by every user without daemon or socket configuration. This is the natural deployment pattern on shared HPC filesystems.


---

<!-- https://morloc-project.github.io/docs/internals/ -->

# 13. Build Architecture

Morloc Manual | https://morloc-project.github.io/docs/internals/ | prev: https://morloc-project.github.io/docs/install/system-wide-podman.md | next: https://morloc-project.github.io/docs/internals/architecture-overview.md


---

<!-- https://morloc-project.github.io/docs/internals/architecture-overview.html -->

# 13.1. Architecture Overview

Morloc Manual > Build Architecture | https://morloc-project.github.io/docs/internals/architecture-overview.html | prev: https://morloc-project.github.io/docs/internals/index.md | next: https://morloc-project.github.io/docs/internals/cross-language-calls.md

A compiled Morloc program has two kinds of components: one **nexus** and one or more **pools**.

The **nexus** is a pre-compiled Rust binary that serves as the CLI entry point. It reads a JSON manifest describing the program’s structure, parses command-line arguments, and orchestrates execution. The nexus starts pool daemons, sends them call packets over Unix domain sockets, and prints the result. When done, it tears everything down.

**Pools** are language-specific daemons — one per language used in the program. A pool contains all functions from its language, compiled into a single unit. Pools listen on Unix sockets for call packets, dispatch to the appropriate function, and return results. All pools support **concurrency**, starting with one worker and growing dynamically as needed. C++ pools use OS threads for true parallelism; Python and R pools use worker processes to handle concurrent requests.

Data moves between pools via Unix domain sockets. For **small values** (up to 64 KB), the serialized data is embedded directly in the packet — no shared memory is needed. For **large values**, data is placed in a shared memory region and only an 8-byte pointer travels over the socket. Pools can also call each other directly for cross-language ("foreign") calls, without routing through the nexus.

![Diagram](https://morloc-project.github.io/docs/static/img/diag-334e01bb4d909869bdcac4a7598ffe360d9cb326f8bc9851cdfc985982e49b6d.svg)

Here are the runtime rules you should be able to count on. Any violations should be considered bugs.

1.  **STDOUT and STDERR pass through.** Any output written to stdout or stderr by user functions is never intercepted or buffered by the Morloc runtime. It passes directly to the terminal.
2.  **Errors become tracebacks.** All exceptions raised by user functions are caught by the pool and returned as error packets. As the error propagates back through foreign calls to the nexus, each layer appends context, building a full cross-language traceback that the user can read.
3.  **Intra-pool calls are near-native.** Calls between functions within the same pool go through a simple dispatch table — there is no serialization, no socket overhead, and no IPC. Performance should be nearly native.
4.  **Inter-pool calls cost socket time plus marshalling.** A call between pools (or between the nexus and a pool) pays only the few microseconds of Unix socket round-trip plus the cost of data marshalling. In the best case, the data in shared memory can be directly used between programs and marshalling cost is zero. In practice, copies are often needed — for example, Python demands ownership of its strings even when the data could in principle be shared directly.


---

<!-- https://morloc-project.github.io/docs/internals/cross-language-calls.html -->

# 13.2. Cross-language function calls

Morloc Manual > Build Architecture | https://morloc-project.github.io/docs/internals/cross-language-calls.html | prev: https://morloc-project.github.io/docs/internals/architecture-overview.md | next: https://morloc-project.github.io/docs/internals/protocols.md

When a Morloc program composes functions from different languages, the compiler must bridge the language boundary. A key design principle is that **Morloc never serializes functions**. Functions cannot be meaningfully transmitted between language runtimes — there is no way to pickle a C++ template instantiation into something Python can call directly. Instead, Morloc generates **wrapper functions** that make IPC calls to the foreign language pool.

## 13.2.1. How it works

Each function in a compiled Morloc program is assigned a unique integer identifier called a **manifold ID** (mid). Every pool maintains a dispatch table mapping manifold IDs to concrete function implementations. When a function needs to call a function in another language, it does not call it directly — it sends a call packet containing the target manifold ID and serialized arguments over a Unix domain socket to the foreign pool, which dispatches the call and returns the result.

The compiler generates all of this automatically. Consider a program where Python’s `pmap` calls a C++ `sum` function:

```morloc
module foo (sumOfSums)

import root-cpp
import root-py

source Py from "foo.py" ("pmap")
source Cpp from "foo.hpp" ("sum")

pmap :: (a -> b) -> [a] -> [b]
sum :: [Real] -> Real

sumOfSums = sum . pmap sum
```

When `pmap` is compiled in the Python pool, it receives `sum` not as a C++ function pointer, but as a **Python wrapper function** generated by the compiler. This wrapper:

1.  Serializes its arguments into the binary wire format
2.  Sends a call packet (with the C++ `sum` manifold ID) over the Unix socket to the C++ pool
3.  Reads the result packet back
4.  Deserializes the result into a Python value

From Python’s perspective, this wrapper is an ordinary Python callable. It can be passed to `multiprocessing.Pool.map`, stored in a list, or used anywhere a function is expected — because it *is* a regular Python function. The cross-language call is hidden inside it.

## 13.2.2. What the generated code looks like

The Python pool contains a wrapper like this (simplified):

```python
def m1384(x):
    packed = morloc.put_value(x, "<list>a<float>f8")
    result = morloc.foreign_call(cpp_socket_path, 1384, [packed])
    return morloc.get_value(result, "<float>f8")
```

Here `1384` is the manifold ID assigned to `sum`, and `cpp_socket_path` is the path to the C++ pool’s Unix domain socket. The `morloc.foreign_call` function handles the IPC: it sends a call packet, waits for the response, and returns the raw result packet. The `put_value` and `get_value` functions handle serialization and deserialization using a compact binary schema string.

On the C++ side, the pool’s dispatch table routes the manifold ID to the actual `sum` implementation:

```c++
// compiler-generated dispatch
uint8_t* local_dispatch(uint32_t mid, const uint8_t** args) {
    switch(mid) {
        case 1384: return m1384(args[0]);  // calls sum
        // ...
    }
}
```

## 13.2.3. Performance implications

Intra-pool calls (functions in the same language) are direct native function calls — no serialization, no sockets, no dispatch table lookup. The only overhead is that functions may be wrapped in thin wrapper functions, but even this can be eliminated with the `%inline` pragma, which inlines the function body at the call site.

Inter-pool calls (cross-language) pay the cost of:

-   Serializing the arguments (proportional to data size)
-   A Unix socket round-trip (microseconds for small payloads)
-   Deserializing the result

In special cases, serialization can be avoided entirely. When data has the same binary representation in both languages, only a pointer to shared memory needs to cross the socket — no copying or conversion. Currently this zero-copy path is supported for Arrow tables; support for fixed-size numeric vectors and tensors is planned.

For higher-order functions like `pmap`, each invocation of the wrapped function is a separate IPC round-trip. If `pmap sum` is called on a list of 1000 elements, that is 1000 cross-language calls. This is the expected cost of language interop — the alternative would be to batch the data and send it all at once, but that would require changing the function’s interface.

When performance matters, the best strategy is to keep hot loops within a single language. The compiler’s implementation selection algorithm already optimizes for this: given multiple implementations of a function, it prefers the one that avoids cross-language calls.


---

<!-- https://morloc-project.github.io/docs/internals/protocols.html -->

# 13.3. Protocols

Morloc Manual > Build Architecture | https://morloc-project.github.io/docs/internals/protocols.html | prev: https://morloc-project.github.io/docs/internals/cross-language-calls.md | next: https://morloc-project.github.io/docs/internals/runtime-and-dev-builds.md

> **Note**
> This section is primarily of interest to users extending the Morloc ecosystem (e.g., adding a new language backend) or debugging at the binary level.

This section describes the binary formats used for communication between the nexus and pools: the manifest, the packet protocol, the shared memory layout, and the voidstar data format.

## 13.3.1. The manifest

The manifest is a standalone `manifest.json` file written into the program’s `<name>-build/` directory. The launcher script that `morloc make` produces is a thin wrapper carrying no embedded payload; it execs the shared `morloc-nexus` runtime against an absolute path to this file, so the launcher can be freely moved as long as its build directory stays put. The manifest describes the program’s structure. Key fields:

| Field | Description |
| --- | --- |
| `version` | Manifest format version (currently `1`) |
| `name` | Program name |
| `build_dir` | Absolute path to the build directory |
| `pools` | Array of pool descriptors (see below) |
| `commands` | Array of exported commands (see below) |

Each **pool** entry:

-   `lang` — Language name (e.g., `"python3"`, `"cpp"`)
-   `exec` — Command-line tokens to launch the pool, with the pool path relative to `build_dir` (e.g., `["python3", "pools/python3/pool.py"]`)
-   `socket` — Unix domain socket basename (e.g., `"pipe-python3"`)

Each **command** entry:

-   `name` — CLI subcommand name
-   `type` — `"remote"` (dispatched to a pool) or `"pure"` (evaluated in the nexus)
-   `mid` — Manifold index identifying the function in the pool
-   `pool` — Index into the `pools` array
-   `needed_pools` — Indices of all pools that must be running
-   `arg_schemas` / `return_schema` — Schema strings describing argument and return types (see [Schema strings](#schema-strings))
-   `args` — CLI argument descriptors

## 13.3.2. Packet protocol

All communication uses a binary packet protocol over Unix domain sockets. Every packet starts with a 32-byte packed header:

![Diagram](https://morloc-project.github.io/docs/static/img/diag-3a8a4214514fb5bdcf108374d4bfe4f885ff06efcf232eb8b10a5c35ca7a2853.svg)

**Table 12. Packet header fields**

| Field | Type | Width | Description |
| --- | --- | --- | --- |
| `magic` | uint32\_t | 4 | Constant `0x0707f86d` (little-endian) |
| `plain` | uint16\_t | 2 | Plain membership (reserved, always 0) |
| `version` | uint16\_t | 2 | Format version (currently 0) |
| `flavor` | uint16\_t | 2 | Metadata convention (reserved) |
| `mode` | uint16\_t | 2 | Evaluation mode (reserved) |
| `command` | union | 8 | Type-specific command data (see below) |
| `offset` | uint32\_t | 4 | Bytes of metadata between header and payload |
| `length` | uint64\_t | 8 | Payload length in bytes |

Total packet size is always `32 + offset + length`.

### Packet types

The `command` field’s first byte is a type tag:

**Data packet** (`0x00`) — Carries data or error messages:

| Field | Type | Width | Description |
| --- | --- | --- | --- |
| `type` | uint8\_t | 1 | `0x00` |
| `source` | uint8\_t | 1 | `0x00`\=MESG (inline), `0x01`\=FILE (path), `0x02`\=RPTR (shared memory pointer) |
| `format` | uint8\_t | 1 | `0x00`\=JSON, `0x01`\=MSGPACK, `0x02`\=TEXT, `0x03`\=DATA, `0x04`\=VOIDSTAR |
| `compression` | uint8\_t | 1 | Reserved, always 0 |
| `encryption` | uint8\_t | 1 | Reserved, always 0 |
| `status` | uint8\_t | 1 | `0x00`\=PASS, `0x01`\=FAIL |
| `padding` | uint8\_t | 2 | Zero |

For small data (up to 64 KB serialized), the most common combination is `source=MESG, format=VOIDSTAR` — the voidstar binary is embedded directly in the packet payload, avoiding shared memory entirely. For large data, the combination is `source=RPTR, format=VOIDSTAR` — only an 8-byte relative pointer travels over the socket, and the data lives in shared memory.

When `status=FAIL`, the packet carries a UTF-8 error message (`source=MESG, format=TEXT`).

**Call packet** (`0x01`) — Instructs a pool to execute a function:

| Field | Type | Width | Description |
| --- | --- | --- | --- |
| `type` | uint8\_t | 1 | `0x01` |
| `entrypoint` | uint8\_t | 1 | `0x00`\=LOCAL, `0x01`\=REMOTE\_SFS |
| `padding` | uint8\_t | 2 | Zero |
| `midx` | uint32\_t | 4 | Manifold index (which function to call) |

The payload is a contiguous sequence of data packets, one per argument.

**Ping packet** (`0x02`) — Header-only, no payload. The nexus pings pools to check readiness; the pool echoes it back as a pong.

### Metadata blocks

Between the header and payload (in the `offset` region), packets can carry metadata blocks. Each has an 8-byte header:

| Field | Type | Width | Description |
| --- | --- | --- | --- |
| `magic` | char\[3\] | 3 | Constant `"mmh"` |
| `type` | uint8\_t | 1 | `0x01`\=SCHEMA\_STRING, `0x02`\=XXHASH |
| `size` | uint32\_t | 4 | Payload size in bytes |

## 13.3.3. Shared memory

Pools share data through POSIX shared memory segments rather than copying over sockets. Only relative pointers (8 bytes) travel over the wire.

### Volumes

Shared memory is organized as multiple volumes (`/dev/shm/morloc-<hash>_0`, `morloc-<hash>_1`, etc.). The nexus creates the first volume (64 KB). New volumes are created automatically when space runs out (up to 32 volumes). If `/dev/shm` is too small (common in Docker), volumes fall back to files in the temporary directory. Under Apptainer/Singularity the host’s `/dev/shm` is shared into the container at host size, so this fallback is rarely triggered.

### Pointer types

| Type | Description |
| --- | --- |
| `absptr_t` (`void*`) | Virtual address in the current process. Different per process. |
| `volptr_t` (`ssize_t`) | Offset within a single volume (0 = first byte after the header). |
| `relptr_t` (`ssize_t`) | Global offset across all volumes. **This is the pointer type shared between processes** — it appears in data packets and in voidstar data structures. |

```
             volume 0 (size=20)        volume 1
         ---xxxxxx........----xxxxxx............---->
 relptr           0      7          8         19
```

### Volume header (`shm_t`)

| Field | Type | Description |
| --- | --- | --- |
| `magic` | unsigned int | Constant `0xFECA0DF0` |
| `volume_name` | char\[256\] | Volume identifier |
| `volume_index` | int | Index in the pool (0, 1, 2, …​) |
| `volume_size` | size\_t | Usable data capacity (excludes header) |
| `relative_offset` | size\_t | Sum of all prior volumes' sizes |
| `rwlock` | pthread\_rwlock\_t | Process-shared read-write lock |
| `cursor` | volptr\_t | Current free block (allocator hint) |

### Block header (`block_header_t`, packed)

| Field | Type | Description |
| --- | --- | --- |
| `magic` | unsigned int | Constant `0x0CB10DF0` |
| `reference_count` | atomic unsigned int | Active references (0 = free) |
| `size` | size\_t | Payload size in bytes (excludes header) |

Blocks use reference counting. `shmalloc` allocates with first-fit and lazy coalescing. `shfree` decrements the reference count; blocks are merged during the next allocation scan.

## 13.3.4. Schema strings

Schema strings are a compact encoding of a data type’s binary layout. They appear in the manifest and in packet metadata.

Primitives:

| Schema | Type |
| --- | --- |
| `z` | nil (1 byte) |
| `b` | bool (1 byte) |
| `i1`/`i2`/`i4`/`i8` | signed int (1/2/4/8 bytes) |
| `u1`/`u2`/`u4`/`u8` | unsigned int (1/2/4/8 bytes) |
| `f4`/`f8` | float (4/8 bytes) |
| `s` | variable-length UTF-8 string |

Compounds:

| Pattern | Description |
| --- | --- |
| `a<elem>` | Array. `ai4` = array of int32. |
| `t<N><elems>` | Tuple. `t2i4f8` = (int32, float64). |
| `m<N><fields>` | Record with length-prefixed keys. `m2<3>agei4<4>names` = \\{age: int32, name: string}. |

## 13.3.5. Voidstar binary format

Every Morloc general type maps unambiguously to a binary form that consists of several fixed-width literal types, a list container, and a tuple container. The literal types include a unit type, a boolean, signed integers (8, 16, 32, and 64 bit), unsigned integers (8, 16, 32, and 64 bit), and IEEE floats (32 and 64 bit). The list container is represented by a 64-bit size integer and a pointer to an unboxed vector. The tuple is represented as a set of values in contiguous memory. These basic types are listed below:

**Table 13. Morloc primitives**

| Type | Domain | Schema | Width (bytes) |
| --- | --- | --- | --- |
| Unit | `()` | z | 1 |
| Bool | `True` \| `False` | b | 1 |
| U8 | $[0,2^{8})$ | u1 | 1 |
| U16 | $[0,2^{16})$ | u2 | 2 |
| U32 | $[0,2^{32})$ | u4 | 4 |
| U64 | $[0,2^{64})$ | u8 | 8 |
| I8 | $[-2^{7},2^{7})$ | i1 | 1 |
| I16 | $[-2^{15},2^{15})$ | i2 | 2 |
| I32 | $[-2^{31},2^{31})$ | i4 | 4 |
| I64 | $[-2^{63},2^{63})$ | i8 | 8 |
| F32 | IEEE float | f4 | 4 |
| F64 | IEEE double | f8 | 8 |
| List x | lists | a{x} | $16 + n \Vert a \Vert$ |
| Tuple2 x1 x2 | 2-ples | t2{x1}{x2} | $\Vert a \Vert + \Vert b \Vert$ |
| TupleX  $\ t_i\ ...\ t_k$ | k-ples | $tkt_1\ ...\ t_k$ | $\sum_i^k \Vert t_i \Vert$ |
| $\{ f_1 :: t_1,\ ... \ , f_k :: t_k \}$ | records | $mk \Vert f_1 \Vert f_1 t_1\ ...\ \Vert f_k \Vert f_k t_k$ | $\sum_i^k \Vert t_i \Vert$ |

All basic types may be written to a schema that is used internally to direct conversions between Morloc binary and native basic types. The schema values are shown in the table above. For example, the type `[(Bool, [I8])]` would have the schema `at2bai1`. You will not usually have to worry about these schemas, since they are mostly used internally. They are worth knowing, though, since they appear in low-level tests, generated source code, and binary data packets.

Here is an example of how the type `([U8], Bool)`, with the value `([3,4,5],True)`, might be laid out in memory:

```
---
03 00 00 00 00 00 00 00 -- first tuple element, specifies list length (little-endian)
30 00 00 00 00 00 00 00 -- first tuple element, pointer to list
01 00 00 00 00 00 00 00 -- second tuple element, with 0-padding
03 04 05                -- 8-bit values of 3, 4, and 5
---
```

Records and tables (described in detail earlier) are represented as tuples in voidstar format — field names are stored only in the type schemas. The `table` annotation is not just syntactic sugar for a record of lists; it is preserved through the compiler to the translator, where language-specific serialization functions may have special handling for tables.

```morloc
record Person = Person { name :: Str, age :: U8 }
table People = People { name :: Str, age :: Int }

alice = { name = "Alice", age = 27 }
students = { name = ["Alice", "Bob"], age = [27, 25] }
```

The Morloc type signatures can be translated to schema strings that may be parsed by a foundational Morloc C library into a type structure. Every supported language in the Morloc ecosystem must provide a library that wraps this Morloc C library and translates to/from Morloc binary given the Morloc type schema.


---

<!-- https://morloc-project.github.io/docs/internals/runtime-and-dev-builds.html -->

# 13.4. Runtime and Dev Builds

Morloc Manual > Build Architecture | https://morloc-project.github.io/docs/internals/runtime-and-dev-builds.html | prev: https://morloc-project.github.io/docs/internals/protocols.md | next: https://morloc-project.github.io/docs/future/index.md

The Morloc runtime — the shared library (`libmorloc.so`) and the nexus binary (`morloc-nexus`) — is written in Rust. Language-specific pools and extensions still use their own compilers (g++ for C++, etc.), but they link against the Rust-built `libmorloc.so` through a stable C ABI defined in a single header (`morloc.h`).

## 13.4.1. Build modes

`morloc init -f` installs the runtime by building it from source. It reads one environment variable:

| Variable | Purpose |
| --- | --- |
| `MORLOC_RUST_DIR` | Path to the `data/rust/` Cargo workspace. `morloc init` runs `cargo build --release` there and copies the artifacts into place. Requires a working Rust toolchain (`cargo`). |

`libmorloc.so` and `morloc-nexus` are deliberately **not** shipped as prebuilt binaries. A Morloc release publishes the compiler per platform plus the platform-independent Rust workspace as `morloc-rust-src.tar.gz`; the runtime is then built from that source with the environment’s own toolchain, so the library every pool links is ABI-coherent with the compilers that built those pools.

Under `mim` this is automatic: `mim` unpacks the release’s Rust source into its runtime store, points `MORLOC_RUST_DIR` at it, and provisions `cargo` through the environment’s conda toolchain. You only set the variable by hand when building from a source checkout:

```console
export MORLOC_RUST_DIR=/path/to/compiler/data/rust
morloc init -f
```

## 13.4.2. What the runtime provides

The Rust workspace produces two artifacts:

**`libmorloc.so`**

Shared library linked by every pool. Provides the IPC layer (Unix domain sockets), shared-memory allocator, packet codec, JSON/MessagePack serialization, and worker-pool concurrency primitives.

**`morloc-nexus`**

Pre-compiled CLI entry point. Reads a program’s `manifest.json` file, parses command-line arguments, starts pool daemons, dispatches calls, and prints results.

## 13.4.3. Build dependencies

| Tool | When needed |
| --- | --- |
| `cargo` | Always — `morloc init` builds the runtime from source. Provisioned by `mim` through the environment’s conda toolchain; install via [rustup](https://rustup.rs) only for an unmanaged build. |
| `gcc` / `g++` | Still required for C++ pool compilation and Python/R language extensions. No longer needed for the runtime itself. |

## 13.4.4. Container images

An environment’s container image is generated by `mim` from the same requirements the native backend solves; there is no hand-authored base image to pull. The generated image stages the release’s Rust source and sets `MORLOC_RUST_DIR` itself, so users of `mim` never set it manually on either backend.


---

<!-- https://morloc-project.github.io/docs/future/ -->

# 14. Future Directions

Morloc Manual | https://morloc-project.github.io/docs/future/ | prev: https://morloc-project.github.io/docs/internals/runtime-and-dev-builds.md | next: https://morloc-project.github.io/docs/qa/index.md

## 14.1. Planes of libraries

> **Important**
> The infrastructure for "planes" is not yet constructed, so the following is speculative

The concept of "planes" is central to the future organization of Morloc and is one of the primary reasons that I created it. A **plane** is like a namespace for a community’s modules—​but instead of organizing by category or programming language, modules in a plane share a common philosophy about quality, trustworthiness, software design and the review process.

Currently, the universe of functions is separated first by language and then by subject area. Morloc, being polyglot, allows the first mode of separation to be lifted, so language does not need to separate communities. Instead, communities can organize around **values**.

-   **Levels of review & trust:** Code may be wild and experimental; tightly reviewed and trusted in production; or formally verified.
-   **Design philosophy:** Groups may prioritize safety, raw performance, or elegance by some metric.
-   **Use case:** Planes may focus on production, pedagogy, competition or experimentation.

Making these differences explicit (and easy to navigate) lets the community set and find their own standards.

**Real-World Analogs**

Within the R community, you could define three planes:

-   **CRAN:** Has stringent requirements for acceptance and manual application process focused on adherence to well-defined (mostly automated) requirements
-   **rOpenSci:** Focuses on a formal peer review process that considers motivation, documentation, and good software design
-   **GitHub:** Wild west. Anything goes.

You could probably find more "planes" in R, but these three capture the idea of what a plane is. It is a design philosophy and set of protocols that define admission.

**Possible examples of Morloc planes**

-   **default**: Official libraries used in sandboxes and demos (not necessarily efficient).
-   **unstable**: For newly submitted or unvetted modules, e.g., loaded straight from GitHub.
-   **safe**: Modules that passed manual review, rigorous automated tests, and have strong test suites.
-   **true**: Formally verified modules, strict on what languages are allowed (e.g., dependently typed languages).
-   **prod**: Production ready modules, combining safety and performance.
-   **comp**: Modules suited for competitive programming; all performance, no safety checks or focus on software design principles.
-   **red**: Adversarial modules—​written to give the Morloc bot problems. Probably don’t want to import these.
-   **weird**: Esoteric code. For silly implementations that abuse languages in fun ways.
-   **demo**: Prototypes, examples, and proof-of-concept modules. More pedagogical than practical.

Planes aren’t rigid categories, but cultures: each has its own ground rules, review process, and ideas about what makes code "good". Anyone can propose a new plane, but we don’t want too many; a bit of consensus is required before adding one.

**How Does a Module Join a Plane?**

Again, the architecture is in development. But here is the basic process:

1.  **Register:** Authors register their module (e.g., import code from GitHub and authenticate).
2.  **AI Vetting:** Our AI (Weena) checks code for basic standards.
3.  **Acceptance:** After being accepted, the module defaults to the `unstable` plane.
4.  **Level Up:** Module authors can then apply to join other planes. Getting accepted depends on the plane’s review process (could be peer review, automated testing, thumbs up from community members, or nothing at all).
5.  **Multiple Planes:** Modules can exist in multiple planes at once—​different communities may trust the same code for different reasons.

This process will eventually be mediated on the website morloc.io (under construction).

Overall, planes help you find code that matches your needs and values—​whether you want ultimate safety, bleeding-edge performance, or just something weird that might surprise you. They also provide community and allow relations between different codebases to be specified.


---

<!-- https://morloc-project.github.io/docs/qa/ -->

# 15. Q&A

Morloc Manual | https://morloc-project.github.io/docs/qa/ | prev: https://morloc-project.github.io/docs/future/index.md | next: https://morloc-project.github.io/docs/qa/one-language.md


---

<!-- https://morloc-project.github.io/docs/qa/one-language.html -->

# 15.1. I only use one language, is Morloc still useful?

Morloc Manual > Q&A | https://morloc-project.github.io/docs/qa/one-language.html | prev: https://morloc-project.github.io/docs/qa/index.md | next: https://morloc-project.github.io/docs/qa/bioinformatics-only.md

Yes, Morloc remains useful even if you only use one programming language.

While Morloc is designed to allow polyglot development, its core benefits also apply to single-language projects. In the Morloc ecosystem, you may continue working in your preferred language, but focus shifts to writing libraries instead of standalone applications.

Morloc lets you compose these functions and automatically generate applications from them, offering several advantages:

-   **Broader usability**: Your functions can be easily reused and easily accessed by other language communities.
-   **Improved testing and benchmarking**: Functions can be integrated into language-agnostic testing and benchmarking frameworks.
-   **Future-proofing**: If you ever need to migrate to a new language, Morloc’s type annotations and documentation carry over—only the implementation needs to change. And if you want to leave the Morloc ecosystem, your implementation does not need to change.
-   **Better workflows**: Especially in fields like bioinformatics, Morloc shifts workflows from chaining applications and files to composing typed functions and native data structures, making pipelines more robust and easier to validate.
-   **No more format parsing**: Morloc data structures replace bespoke file formats and offer efficient serialization.

While language interop is a major feature of Morloc, it is not the main purpose. The very first version of Morloc was not even polyglot at all. The focus originally was to just have a simple composition language that separated pure code from associated effects, conditions, caching, etc.

The primary goal of Morloc is to support the development of composable, typed universal libraries. Support for many languages is required for this goal, since no one language is best for all cases. Most Morloc users would continue to program in their favorite language, but gain the ability to compose, share, and extend functionality more easily.


---

<!-- https://morloc-project.github.io/docs/qa/bioinformatics-only.html -->

# 15.2. Is this just a bioinformatics workflow language?

Morloc Manual > Q&A | https://morloc-project.github.io/docs/qa/bioinformatics-only.html | prev: https://morloc-project.github.io/docs/qa/one-language.md | next: https://morloc-project.github.io/docs/qa/per-function-environments.md

No. The Morloc [paper](https://peerj.com/articles/cs-3435/), is focused on bioinformatics applications. As discussed at length in the paper, Morloc addresses systematic flaws in the traditional approaches to building bioinformatics workflows. Given the need, and also given my personal background, bioinformatics is a good place to start. However, Morloc can be more broadly applied to any functional problem.


---

<!-- https://morloc-project.github.io/docs/qa/per-function-environments.html -->

# 15.3. Does Morloc allow function-specific containerized environments?

Morloc Manual > Q&A | https://morloc-project.github.io/docs/qa/per-function-environments.html | prev: https://morloc-project.github.io/docs/qa/bioinformatics-only.md | next: https://morloc-project.github.io/docs/qa/object-oriented.md

No, unlike workflow managers such as Snakemake and Nextflow, Morloc does not offer function-specific environments. This is a deliberate design choice.

Dependency resolution is a hard and heavily researched problem. The general goal of dependency solvers is to find **one** set of dependencies that satisfies the **entire** program. The bioinformatics community often gives up on finding unified environments and instead runs each function in its independent environment. With every function running in its own container, all dependency issues are encapsulated and all functions may be executed from one manager. But this comes at a heavy cost. Each application must be wrapped in a script, the script must be executed via an expensive system call into the container, and data must be serialized and sent to the container. This approach is reasonable for workflows with a small number of heavy components. But from a programming language perspective, wrapping every function call in its own environment is inefficient and opaque.

Morloc is designed not to hide problems in boxes, but rather to solve the root problem. Conventional workflow languages attempt to simplify workflows design by layering frameworks over the functions. The Morloc approach is the exact opposite. First delete everything unnecessary from all applications and lift their light algorithmic cores into clean, well-typed libraries. Then build upwards through composition of these pure functions—​and judicious use of impure ones—​to create efficient, reliable, and composable tools.

Now, if you really do need to run something in a container, you can just make a function that wraps a call to a container and then use it just as you would any other function. You could even write a wrapper function that takes a record with all the metadata needed for a conda environment and execute its function within that environment. We can do this through libraries, so there is no need to hardcode this pattern into the Morloc language itself.

The reproducibility of Morloc workflows may be ensured by running the entire Morloc program in an environment or container, with a single set of dependencies. The specific Morloc compiler version can be specified and modules may be imported using their git hashes. This is done in the current Morloc examples (see the Dockerfile in the workflow-comparisons folder of [https://github.com/morloc-project/examples](https://github.com/morloc-project/examples)).


---

<!-- https://morloc-project.github.io/docs/qa/object-oriented.html -->

# 15.4. What about object-oriented programming?

Morloc Manual > Q&A | https://morloc-project.github.io/docs/qa/object-oriented.html | prev: https://morloc-project.github.io/docs/qa/per-function-environments.md | next: https://morloc-project.github.io/docs/qa/ai-relevance.md

An "object" is a somewhat loaded term in the programming world. As far as Morloc is concerned, an object is a thing that contains data and possibly other unknown stuff, such as hidden fields and methods. All types in Morloc have must forms that are transferable between languages. Methods do not easily transfer; at least they cannot be written to Morloc binary. However, it is possible to convey class-like APIs through typeclasses. Hidden fields are more challenging since, by design, they are not accessible. So objects cannot generally be directly represented in the Morloc ecosystem.

Objects that have a clear "plain old data" representation can be handled by Morloc. These objects, and their component types, must have no vital hidden data, no vital state, and no required methods. Examples of these are all the basic Python types (`int`, `float`, `list`, `dict`, etc) and many C++ types such as the standard vector and tuple types. When these objects are passed between languages, they are reduced to their pure data.


---

<!-- https://morloc-project.github.io/docs/qa/ai-relevance.html -->

# 15.5. Is Morloc still relevant when AI can program and translate?

Morloc Manual > Q&A | https://morloc-project.github.io/docs/qa/ai-relevance.html | prev: https://morloc-project.github.io/docs/qa/object-oriented.md | next: https://morloc-project.github.io/docs/qa/not-a-human.md

*Maybe*. Morloc may serve as a system for functional composition, verification, and automation even when most functions are generated by machines.

I’ll lay out an argument for this below, starting with a few proposition:

1.  **Adversaries exist**. AIs may themselves be adversarial or there might be adversarial code in ecosystem around the AIs (for example, prompt injection). Humans can’t trust humans, humans can’t trust AIs, AIs can’t trust humans, and AIs can’t trust AIs. Depending on their architecture, AIs may not even be able to trust their own memories.
2.  **Stupid is fast**. Narrow intelligence outperforms general intelligence for narrow problems. A vast AGI system with deep understanding of physics and Shakespeare will not be the fastest tool for sorting a list of integers. There will always be a need for programs across the intelligence spectrum — from classical functions, to statistical models, to general intelligences.
3.  **Creating functions is expensive**. Designing high-performance algorithms is not trivial. Even simple functions, like sorting algorithms, require deep thought to optimize for a given use case. But there is a further combinatorial explosion of more complex physical simulations, graphics engines, and statistical algorithms. While simple functions might be created in seconds, others may take years of CPU time to optimize.
4.  **Reproducibility is important**. Future AIs may serve as nearly perfect oracles, but they are complex entities and future AIs will likely be capable of evolving over time as persons. So they will likely not give equivalent answers day to day. It is valuable to be able to crystallize a thought process into something that will behave the same every time it is invoked on a given input. So again, functions are important.
5.  **Correctness is important**. If functions are being composed by AIs to create new programs, any function that does not behave in the way the AI expects can cause cascading errors. It doesn’t matter how intelligent the AI is, if it is building programs from functions that it cannot verify, then the programs may not be safe.

A few things follow from these propositions.

First, AI will benefit from writing functions. Even in a world with no humans, they will need functions for efficiently solving narrow problems. They will likely generate libraries of billions of specialized functions. Some may be classical functions and others may be small statistical models. By caching these functions, compute time can be saved. Rather than generating entire programs from first principles, they can build them logically through composition of prior functions. The same forms of abstractions that help humans reason will also be of value to AIs. Yes, they have far larger working memories than we do, but that does not change the fact that abstraction and composition reduce the costs of re-derivation.

Time can also be saved if different AIs share functions they have written (both with each other and with humans). Since adversaries exist, shared functions must be verified. But verification is hard, especially if a godlike super-intelligence were trying to hide adversarial features in the binary. The problem can be simplified by using a controlled language that can be formally verified by a trusted classical computer program — a compiler. So rather than share functions as binary, it would make sense to share them in strict controlled languages. For this reason, I believe that something resembling current programming languages will exist far into the future. Their main purpose will be as easily verifiable and human readable specifications for languages that can be compiled into high-performance code.

So in this imagined future, there are billions of functions in databases that are written in verifiable languages readable by humans, classical machines, and AIs. But what language is used? Maybe the AIs can converge on one standard. But even for AIs, and perhaps especially for them, I don’t think a single language is optimal. Rather, just as in human mathematics, there will likely be many languages for many domains. Languages make trade-offs. In general, the more complex a language is, the more difficult it is to parse, verify and optimize. So even if we ignore human factors, multi-lingual ecosystems are still likely to appear. Adding in human factors, we are again likely to see a spectrum of languages that accept different trade offs in rigor, ease of use, and domain specificity.

I predict a future where humans and AIs use libraries of functions written in specialized languages. All the functions need to be easily verifiable by an outside actor and verified functions need to be composed to more complex programs using a well-verified composer. Since we don’t trust any agent to verify, we need a classical program. Morloc is a potential candidate for this role. It would serve as a classical composition tool, function verification ecosystem, automation engine, and conceptual framework for organizing and using billions of mostly machine generated functions.

In this world, Morloc could serve as the composition engine that parses AI-generated compositions as Morloc scripts, generates all the artifacts, and manages search over functional libraries. The AI design space could be defined as composition-space of all modules that are available in the ecosystem. This lowers the complexity of design space, likely increasing generation efficiency, and allows more formal reasoning over the side-effects a given AI system can have.

Of course, the future is impossible to predict, especially where AI is concerned. It is possible that AIs will converge on a single universal representation for computation. It is possible that the need for human readability and curation will disappear. It is possible that classical computer functions could be entirely replaced by discrete mathematical constructs that are composable and machine verifiable but entirely incomprehensible to humans.


---

<!-- https://morloc-project.github.io/docs/qa/not-a-human.html -->

# 15.6. I’m not a human, do I still need Morloc?

Morloc Manual > Q&A | https://morloc-project.github.io/docs/qa/not-a-human.html | prev: https://morloc-project.github.io/docs/qa/ai-relevance.md | next: https://morloc-project.github.io/docs/qa/morlocks.md

Yes, you are still a programmer and writing endless glue code to stitch things together takes time and is brittle. Abstraction is universal. With Morloc you can build with confidence using simple components with highly efficient interfaces. You can access a library of functions and freely compose them to build infinite new tools. Morloc is a way to define, access, and explore customizable and efficient toolsets.


---

<!-- https://morloc-project.github.io/docs/qa/morlocks.html -->

# 15.7. Why is it named after Morlocks, weren’t they, like, bad?

Morloc Manual > Q&A | https://morloc-project.github.io/docs/qa/morlocks.html | prev: https://morloc-project.github.io/docs/qa/not-a-human.md | next: https://morloc-project.github.io/docs/qa/more-questions.md

While the Morlocks of Wellian fame are best known for their culinary preferences, I think Wells misrepresented them. And even if he didn’t, we don’t treat our own Eloi any better. Meat choices aside, the Morlocks worked below to maintain the machines that simplified life above. That’s why the Morloc language adapts their name.


---

<!-- https://morloc-project.github.io/docs/qa/more-questions.html -->

# 15.8. Wait! I have more questions!

Morloc Manual > Q&A | https://morloc-project.github.io/docs/qa/more-questions.html | prev: https://morloc-project.github.io/docs/qa/morlocks.md | next: https://morloc-project.github.io/docs/env/index.md

Great! Look me up on Discord (link below) and we can chat.


---

<!-- https://morloc-project.github.io/docs/env/ -->

# 16. Environmental Variable List

Morloc Manual | https://morloc-project.github.io/docs/env/ | prev: https://morloc-project.github.io/docs/qa/more-questions.md | next: https://morloc-project.github.io/docs/contact/index.md

Every environment variable that morloc reads or sets lives in the `MORLOC_` namespace. They fall into two roles:

-   **Tunables** — read at runtime to override a default. A user (or an orchestrator) sets these; morloc only reads them.
-   **Wiring** — set by the nexus, the compiler, or `mim` and read by the child processes they launch (pools, setup scripts, SLURM re-entry). Users normally do not set these by hand.

The tables below group the variables by topic. The **Introduced in** column links to the manual section that introduces the variable; a `(pending)` entry marks a variable that is described here but does not yet have a dedicated home elsewhere in the manual.

## 16.1. Installation and runtime setup

| Variable | Description | Default | Introduced in |
| --- | --- | --- | --- |
| `MORLOC_HOME` | Morloc install root; base for `bin/`, `include/`, `lib/`, and the module tree. | `~/.local/share/morloc` | [Exposing native resources](https://morloc-project.github.io/docs/apis/exposing-native-resources.md) |
| `MORLOC_RUST_DIR` | Path to the `data/rust/` Cargo workspace; `morloc init` compiles the runtime from source (requires `cargo`). Set by `mim` to the Rust source unpacked from the Morloc release. | *(search)* | [Build modes](https://morloc-project.github.io/docs/internals/runtime-and-dev-builds.md#build-modes) |
| `MORLOC_LANG_PARAMS` | `;`\-separated `LANG:KEY=VALUE` build parameters, layered between the per-machine build config and `-X` on the command line. | *(none)* | [Build parameters](https://morloc-project.github.io/docs/languages/build-parameters.md) |
| `MORLOC_BIN_LINK_DIR` | Directory where `morloc init` symlinks `morloc-nexus` and `mim`. An empty value suppresses symlinking; `mim` sets it inside containers so links land in a morloc-owned `bin/` instead of clobbering `~/.local/bin`. | `~/.local/bin` if present, else none | (pending) |
| `MORLOC_NEXUS` | Explicit path to the `morloc-nexus` binary. First entry in the resolution chain (then `$MORLOC_HOME/bin`, `$PATH`, `~/.local/bin`) used when a pool must re-invoke the nexus, e.g. for SLURM dispatch. | *(resolution chain)* | (pending) |

## 16.2. Module installation (setup-script environment)

Set by the compiler for a module’s `setup` script child process and read by that script.

| Variable | Description | Default | Introduced in |
| --- | --- | --- | --- |
| `MORLOC_MODULE_NAME` | Name of the module being installed. | *(per module)* | [Module setup scripts](https://morloc-project.github.io/docs/modules/installing-modules.md#configuring-cpp-build) |
| `MORLOC_MODULE_VERSION` | Version of the module being installed. | *(per module)* | [Module setup scripts](https://morloc-project.github.io/docs/modules/installing-modules.md#configuring-cpp-build) |
| `MORLOC_MODULE_DIR` | Absolute path to the installed module directory; also the script’s working directory. | *(per module)* | [Module setup scripts](https://morloc-project.github.io/docs/modules/installing-modules.md#configuring-cpp-build) |
| `MORLOC_PLANE` | Active plane name. | *(config)* | [Module setup scripts](https://morloc-project.github.io/docs/modules/installing-modules.md#configuring-cpp-build) |
| `MORLOC_PLANE_DIR` | Directory of the active plane’s library path. | *(config)* | [Module setup scripts](https://morloc-project.github.io/docs/modules/installing-modules.md#configuring-cpp-build) |
| `MORLOC_EXPOSE_CPP_DIR` | Destination directory a `setup` script should copy exposed C++ resources into. | *(derived from plane/module)* | (pending) |
| `MORLOC_EXPOSE_PY_DIR` | Destination directory a `setup` script should copy exposed Python resources into. | *(derived from plane/module)* | (pending) |
| `MORLOC_EXPOSE_R_DIR` | Destination directory a `setup` script should copy exposed R resources into. | *(derived from plane/module)* | (pending) |

## 16.3. Run directory, logging, and reporting

| Variable | Description | Default | Introduced in |
| --- | --- | --- | --- |
| `MORLOC_LOG_DIR` | Base directory that activates the per-run directory, stderr log tee, and `summary.json`. Equivalent to `--log-dir`. | *(none; opt-in)* | [Activation knobs](https://morloc-project.github.io/docs/runs/run-directory.md#rundir-activation) |
| `MORLOC_SUMMARY` | Path to write `summary.json`, independent of a rundir. Equivalent to `--summary`. | *(none)* | [Activation knobs](https://morloc-project.github.io/docs/runs/run-directory.md#rundir-activation) |
| `MORLOC_QUIET` | Suppress all morloc-emitted log lines at the source. Equivalent to `--quiet`. | off | [Activation knobs](https://morloc-project.github.io/docs/runs/run-directory.md#rundir-activation) |
| `MORLOC_RUN_DIR` | Current run’s directory. Set by the runtime and inherited by child morloc processes so their logs interleave. | *(only when active)* | [Where the directory lives](https://morloc-project.github.io/docs/runs/run-directory.md#rundir-location) |
| `MORLOC_RUN_PARENT_PID` | PID that created the rundir; guards against a stale inherited `MORLOC_RUN_DIR` (must match `getppid()`). | *(own PID)* | [Where the directory lives](https://morloc-project.github.io/docs/runs/run-directory.md#rundir-location) |
| `MORLOC_RUN_BASE` | Parent directory of the active run directory. Published by the runtime for child pools; informational only (a child reconstructs its run from `MORLOC_RUN_DIR`). | *(only when active)* | (pending) |
| `MORLOC_TRACE` | Any non-empty value enables per-stage timing trace lines on stderr (ingest, size-estimate, and emit hot paths) for diagnosing where wall time is spent. | off | (pending) |

## 16.4. Debugging

| Variable | Description | Default | Introduced in |
| --- | --- | --- | --- |
| `MORLOC_DEBUG_DIR` | Explicit output directory for `--debug` input dumps (`PATH/inputs/<hash>.pkt`). | *(rundir or `./.morloc-debug`)* | [Where dumps land](https://morloc-project.github.io/docs/runs/debugging.md#dump-location) |
| `MORLOC_DEBUG_CACHE_DEPTH` | Maximum disk writes per dispatch (`0` = unlimited). Equivalent to `--debug-cache-depth`. | `1` | [Runtime knobs](https://morloc-project.github.io/docs/runs/debugging.md#debug-knobs) |
| `MORLOC_DEBUG_CACHE_MAX` | Per-arg size cap on the dumped payload; larger args recorded by hash only. Equivalent to `--debug-cache-max`. | `0` (unlimited) | [Runtime knobs](https://morloc-project.github.io/docs/runs/debugging.md#debug-knobs) |
| `MORLOC_DEBUG_RECURSION_CAP` | Per-manifold frame limit in a recorded trace (`0` = unlimited). Equivalent to `--debug-recursion-cap`. | `3` | [Runtime knobs](https://morloc-project.github.io/docs/runs/debugging.md#debug-knobs) |

## 16.5. Caching

| Variable | Description | Default | Introduced in |
| --- | --- | --- | --- |
| `MORLOC_CACHE_BASE` | Override the cache base directory (useful for Docker mounts and shared filesystems). | `$XDG_CACHE_HOME/morloc/cache` | [Storage layout](https://morloc-project.github.io/docs/runs/caching.md#cache-layout) |
| `MORLOC_CACHE_COMPRESSION_LEVEL` | zstd level for persisted cache entries. `0` stores them uncompressed so `morloc dump` reads them without decompressing; raise it for large-payload workloads where disk footprint matters. | `0` (uncompressed) | (pending) |
| `MORLOC_POOL_HASH` | Hex source fingerprint mixed into every cache key so editing the morloc source (or a `@hash-include` file) invalidates the cache. `0` keeps caching working but drops source-edit invalidation. | `0` (no invalidation) | (pending) |

## 16.6. Data transfer (packets and shared memory)

Set by the nexus from the compiled manifest and read once by `libmorloc` on the first packet operation.

| Variable | Description | Default | Introduced in |
| --- | --- | --- | --- |
| `MORLOC_INLINE_SIZE` | Maximum flat byte size embedded inline in a packet; larger payloads spill to shared memory (or a temp file). | `65536` (64 KiB) | (pending) |
| `MORLOC_NO_SHM` | Set to `1`/`true`/`yes`/`on` to disable shared memory and route large payloads through temp files instead. | off | (pending) |
| `MORLOC_TMPDIR` | Temp directory for file-routed packets (used when a payload exceeds the inline threshold and SHM is disabled or unavailable). | system temp | (pending) |

## 16.7. Streaming and compression

| Variable | Description | Default | Introduced in |
| --- | --- | --- | --- |
| `MORLOC_REGISTRY_SLOT_COUNT` | Concurrent stream handles per nexus invocation (each slot uses 512 bytes of SHM). | `4096` | [Streaming environment variables](https://morloc-project.github.io/docs/runs/random-access-and-streaming.md#stream-env-vars) |
| `MORLOC_WRITE_BUFFER_BYTES` | Per-`OStream` write-buffer cap; trades sub-packet granularity against per-flush overhead. | 16 MiB | [Streaming environment variables](https://morloc-project.github.io/docs/runs/random-access-and-streaming.md#stream-env-vars) |
| `MORLOC_IFILE_CACHE_BYTES` | Per-handle SHM cache for decompressed IFile sub-packets (`0` disables). | 256 MiB | [Streaming environment variables](https://morloc-project.github.io/docs/runs/random-access-and-streaming.md#stream-env-vars) |
| `MORLOC_FRAME_WORKERS` | Worker threads for parallel zstd frame compression/decompression (capped at 16). | `min(cores, 16)` | [Streaming environment variables](https://morloc-project.github.io/docs/runs/random-access-and-streaming.md#stream-env-vars) |

## 16.8. Nexus and inter-process wiring

Set by the nexus and read by the pools it launches.

| Variable | Description | Default | Introduced in |
| --- | --- | --- | --- |
| `MORLOC_BRIDGE_SOCKET` | Unix-socket path for the SLURM submission bridge; exported by `mim` into the driver container. | *(unset = direct sbatch)* | [Running with the dispatch bridge](https://morloc-project.github.io/docs/install/execution-contexts.md#dispatch-bridge) |
| `MORLOC_NEXUS_PATH` | Absolute path of the running nexus, exported to every pool so remote/SLURM dispatch can wrap an `sbatch` call to the same binary. | *(from `current_exe`)* | (pending) |
| `MORLOC_MANIFEST_PATH` | Absolute (canonicalized) path of the program manifest, exported to pools for tooling that must re-enter the same program. | *(canonical manifest path)* | (pending) |
| `MORLOC_NEXUS_STDIO_SOCK` | Unix socket the nexus binds for its stdio server; a pool connects to it to route `@stdin` / `@stdout` / `@stderr` back to the nexus. Unset means the pool was not started by a nexus. | *(unset)* | (pending) |

## 16.9. Nexus inspection tools (`view` / `file`)

| Variable | Description | Default | Introduced in |
| --- | --- | --- | --- |
| `MORLOC_VIEW_MAX_BUFFER_BYTES` | Override the buffered-path size threshold that the `view` / `file` guardrail refuses without `--force`. | 1 GiB | [Size guardrails and `--force`](https://morloc-project.github.io/docs/utilities/nexus-view.md#view-size-guardrails) |
| `MORLOC_FILE_MAX_SCAN_SUBPACKETS` | Cap on sub-packets scanned when classifying a footer-less stream file. | `10000` | [Stream packets](https://morloc-project.github.io/docs/utilities/nexus-file.md#stream-packets) |
| `MORLOC_CSV_SNIFF_ROWS` | Data rows Arrow scans to infer a CSV schema. `0` falls back to `String` columns; a non-numeric value falls back to the default. The whole file is never scanned. | `100` | (pending) |

## 16.10. String safety

| Variable | Description | Default | Introduced in |
| --- | --- | --- | --- |
| `MORLOC_SKIP_NULL_CHECK` | Skip the interior-NUL scan on strings crossing into a pool (unsafe). Set to `1`/`true`/`yes`. | off | [Null strings](https://morloc-project.github.io/docs/features/strings.md#nul-bytes) |


---

<!-- https://morloc-project.github.io/docs/contact/ -->

# 17. Contact

Morloc Manual | https://morloc-project.github.io/docs/contact/ | prev: https://morloc-project.github.io/docs/env/index.md

This is a young project and any brave early users are highly valued. Feel free to contact me for any reason!

-   github: [https://github.com/morloc-project/morloc](https://github.com/morloc-project/morloc)
-   discord: [https://discord.gg/dyhKd9sJfF](https://discord.gg/dyhKd9sJfF)
-   X: [https://x.com/morlocproject](https://x.com/morlocproject)
-   BlueSky: [https://bsky.app/profile/morloc-project.bsky.social](https://bsky.app/profile/morloc-project.bsky.social)
-   email: [z@morloc.io](mailto:z@morloc.io)