13.3. Process lifecycle

A run creates processes, a private directory and shared memory, and must remove all of them however it ends: normally, by error, by signal, or by being killed outright. This section specifies how a run starts its pools, how it knows they are ready, how it stops them, and what it guarantees about cleanup and exit status.

Startup. Before it starts any pool, the nexus:

  1. Creates the run directory, a fresh directory under /tmp readable only by the user, and writes an owner record into it (the nexus’s pid, process start time, boot id and pid namespace). It always lives under /tmp so a later run can find it; the owner record tells a dead run from a live one.

  2. Removes the run directories and shared memory of earlier runs whose owner is dead and whose pools have all ended.

  3. Creates the run’s shared memory: the first volume and the stream registry (Shared memory layout).

  4. Installs its signal handlers.

It then starts each pool the request needs (Runtime overview):

  1. It starts a pin: a tiny process that leads a new process group, ignores termination signals, and exits only when the nexus closes a pipe to it. The pin holds the group’s id, so the id cannot be reissued to an unrelated process while the nexus may still signal the group.

  2. It records the group in the run directory, then starts the pool inside that group. The pool’s command line is the manifest’s command for it followed by three arguments: its socket path, the run directory, and the shared-memory base name. The pool inherits the nexus’s standard input, output and error unchanged.

  3. It pings each pool on its socket until one answers with a ping packet (Packet protocol). Retries back off exponentially. If the pool process exits first, startup fails at once and reports its exit status.

The lifeline. The nexus holds the only write end of a pipe; every pool receives the read end. The kernel closes the write end when the nexus exits for any reason, including SIGKILL, so a pool always learns that its nexus is gone. It then ends its own process group (SIGTERM, 200 ms, SIGKILL) and starts a detached sweeper that removes the run’s shared memory and directory once every recorded group has ended. Without the lifeline, a killed nexus would leave pools and shared memory behind indefinitely.

Shutdown. At the end of a run, or on SIGINT, SIGTERM or SIGHUP, the nexus stops its pools: SIGTERM to every pool group, up to 200 ms for the pool processes to exit, then SIGKILL to every group. It then removes the names of the shared-memory objects and the run directory. It removes names but never unmaps memory at exit, because a background thread may still be reading it. When a pool process exits on its own, the nexus kills the rest of its group, so a pool’s worker processes never outlive it.

Cleanup on every path:

How the run ends Who removes shared memory and the run directory

Normal exit, error, SIGINT/SIGTERM/SIGHUP

The nexus, before it exits.

Crash signal (SIGSEGV, SIGBUS, SIGFPE, SIGABRT)

The nexus’s crash handler removes the shared memory, then the signal takes its default action.

Internal error (panic)

The nexus kills its pool groups and removes what it registered before exiting.

SIGKILL or power loss of the nexus

Each pool’s sweeper, after its lifeline closes; failing that, the next run’s startup sweep.

A long-running daemon additionally recovers from a pool crash by stopping every pool, waiting for running requests, discarding the shared memory and starting fresh; that is specified with the daemon in Serving.

Exit status of a one-shot run:

Status Meaning

0

Success.

1

The call failed: an error raised by user code, a pool that died, an argument that could not be read, or a runtime error.

2

Usage error: an unknown command or option.

70

Internal error: an invariant inside Morloc itself was broken (a panic). Only a panic exits with 70.

128 + N

Ended by signal N (130 for SIGINT, 143 for SIGTERM).

141

Standard output was a closed pipe (128 + SIGPIPE), as when the output is piped to head.