6.8. Reading a stream from standard input

A tool that reads standard input when you do not give it a file is the shape that makes pipelines work. @stdin declares that shape.

Marking a Str positional @stdin makes it optional. When the caller supplies a path, the argument is that path; when the caller omits it, or writes -, the argument becomes standard input. The command opens it with @open and reads from the handle, so nothing in the body cares which of the two happened.

sift’s `total adds up a stream of per-file counts:

--' Add up a stream of per-file counts
total ::
  --' A file of counts; standard input when omitted
  --' @stdin
  Str ->
  <IO> Int
total f = do
  Ok s <- @open f :: <IO> (Try Str (IStream (Str, Int)))
  Ok counts <- @next s
  totalPy counts

@open needs to know what it is opening; the :: annotation says the handle is an IStream of (Str, Int) pairs, which is what summarize produces. @next pulls the next batch off the stream.

All three call shapes give the same answer:

$ ./sift -f packet summarize hits.json > counts.pkt
$ ./sift total counts.pkt
4

$ ./sift -f packet summarize hits.json | ./sift total
4

$ ./sift -f packet summarize hits.json | ./sift total -
4

-f packet is what makes the middle form work. It is Morloc’s own framing: the bytes carry the value’s schema, so the reader checks that what arrived is what it asked for instead of trusting the pipeline.

6.8.1. What standard input may carry

Morloc packets, and nothing else. A foreign format is refused rather than guessed at:

$ printf 'this is definitely not a morloc packet, just plain text bytes\n' | ./sift total
Error: run failed
...
@next: stdin is not a morloc packet; expected a morloc data or stream packet. Foreign formats (JSON, MessagePack, CSV, ...) are not supported on stdin. A morloc program writes packets only when asked: add `-f packet` to the command on the writing end of this pipe.

Empty input is not an error — it is an empty batch, which is the right answer for a search that found nothing:

$ printf '' | ./sift total
0

This is narrower than the - of Arguments, which accepts JSON and MessagePack too. The difference is that - reads one value off stdin, while @stdin opens stdin as a stream that the command drains itself.

6.8.2. Rules for @stdin

At most one positional per command may read stdin, and it must be the last one. Both are compile errors. Given

two.loc
module two (f)

import root-py

--' Two stdin arguments
f ::
  --' @stdin
  Str ->
  --' @stdin
  Str ->
  <IO> ()
f _ _ = @throw "unused"
nl.loc
module nl (f)

import root-py

--' A stdin argument that is not last
f ::
  --' @stdin
  Str ->
  Int ->
  <IO> ()
f _ _ = @throw "unused"
$ morloc make -o two two.loc
In two:f, more than one positional declares `@stdin`; at most one argument may read from stdin.

$ morloc make -o nl nl.loc
In nl:f, a positional follows the `@stdin` positional; the stdin argument must be the last positional.

@stdin implies @check.path r and cannot be combined with @arg, @default, or @many.

A handler may also open the argument as an IFile, which gives random access and a footer count instead of a sequential read. That works on a real file and fails on a pipe, so the usual idiom is to match on the IFile attempt and fall back to IStream in the Err arm. See Random access and streaming.

Warning

Two limits are worth knowing before you design around @stdin.

It applies only to Str. An argument declared [Str] with @form list — the line-oriented filter shape — is rejected, so a wc-style tool has to be called with an explicit - (reports/0030).

A stream whose element type has a name — a type alias, or a record — is accepted from a file and rejected from stdin, because only the stdin path compares the concrete schema name (reports/0031). That is why total opens an IStream (Str, Int) — an alias for the pair, or a record in its place, would be rejected on the pipe and accepted from the file.

Neither shows up in -h, which still prints the argument as though it were required.