ALPHABET-GRID(1)                                 ALPHABET-GRID(1)

     NAME
          grid - peer-to-peer data distribution

     SYNOPSIS
          load alphabet
          typeset /grid
          type /grid/endpoint

     DESCRIPTION
          Grid is a typeset for alphabet (see sh-alphabet(1)) which
          enables allows direct interconnection of remote and local
          processing components. It defines one new type, endpoint,
          which represents a place in the network to which two parties
          can connect and exchange data.

          In the following descriptions, if a type is not endpoint or
          a type defined in the root typeset (see alphabet-main(1)),
          it is assumed to be of type /string.

          Modules currently provided within the /grid typeset include:

          farm [-lnkavA] endpoint addr tasktype [arg...] -> endpoint
                    Farm connects to a grid labour exchange (see
                    scheduler(4)) at addr, starts a new job of type
                    workflow, and passes all the data read from
                    endpoint to be processed by the currently avail-
                    able labour.  The data is split into records, each
                    one of which will be processed on a worker node by
                    tasktype, with its associated arguments. Other
                    than -A, which specifies unauthenticated access to
                    the scheduler, the various options are all passed
                    verbatim to workflow: -l causes it to split its
                    input on newline-separated records; -n specifies
                    that no record separation is necessary on output;
                    -k specifies that intermediate data for failed
                    tasks should be kept around; -a specifies that
                    intermediate data for all tasks should be kept
                    around, and -v specifies that workflow should pro-
                    duce a wordy description of what it is doing.

          local endpoint -> /fd
                    Local reads everything from endpoint, and writes
                    it to fd.

          remote [-a addr] fd -> endpoint
                    Remote is the inverse of local: it reads data from
                    fd and writes it to a newly created endpoint.  If
                    -a is given, addr specifies the network address of
                    an endpoint server on which to create the new end-
                    point.

     ALPHABET-GRID(1)                                 ALPHABET-GRID(1)

          rexec [-A] endpoint addr cmd -> endpoint
                    Rexec connects to a remote execution server at
                    addr (unauthenticated if -A is specified), and
                    arranges to execute the alphabet expression cmd
                    there. The expression should be compatible with
                    usage fd -> fd.  Data from the argument endpoint
                    will be piped through this expression, and made
                    available as the resulting endpoint endpoint.

     EXAMPLES
          The examples below that a local endpoint is available, and
          the following alphabet declarations:
               load alphabet
               typeset /grid
               type /string /endpoint /fd
               import /grid/local /grid/remote
               autoconvert fd endpoint remote
          Set up a rendering pipeline:
               -{/read /tmp/somedata |
                    remote |
                    rexec tcp!node1!rexec "{(/fd); /filter $1 "{os render_stage1}} |
                    rexec tcp!node2!rexec "{(/fd); /filter $1 "{os render_stage2}} |
                    /create /tmp/somedata.result
               }

     SOURCE
          /appl/alphabet/grid.b, /appl/alphabet/gridtypes.b
          /appl/cmd/grid/*.b

     SEE ALSO
          sh-alphabet(1), alphabet-main(1), sh(1)