AOE(3)                                                     AOE(3)

     NAME
          aoe - ATA-over-Ethernet (AoE) interface

     SYNOPSIS
          bind -a #æ /dev
          /dev/aoe/ctl
          /dev/aoe/log
          /dev/aoe/shelf.slot/config
          /dev/aoe/shelf.slot/ctl
          /dev/aoe/shelf.slot/devlink/0
          ...
          /dev/aoe/shelf.slot/devlink/i
          /dev/aoe/shelf.slot/ident
          ...

     DESCRIPTION
          The AoE (ATA-over-Ethernet) interface serves a three-level
          directory providing control and access to AoE targets.  The
          interface provided is primarily intended for low-level con-
          trol of the AoE initiator.  See sdaoe(3) for the standard
          interface.

        Top-level files
          In order to access AoE targets, one or more Ethernet con-
          trollers need to be bound to the AoE initiator.  By default,
          the system starts with no interfaces bound.  For automatic
          binding of interfaces on boot, the aoeif configuration vari-
          able is set in plan9.ini(8). Ethernet interfaces are speci-
          fied as ethern, not as #ln.  To bind the first and second
          Ethernet devices on boot, add

               aoeif=ether0 ether1

          To bind ether1 to a running system:

               % echo bind '#l1/ether1' >/dev/aoe/ctl

          And to unbind it

               % echo unbind '#l1/ether1' >/dev/aoe/ctl

          When an interface is unbound, targets depending on that
          interface are removed.

          Each local interface is called a netlink. The mapping of AoE
          targets to netlinks is called a devlink. Each devlink may
          see multiple interfaces per target.  For example, if the
          local machine has one Ethernet address bound and the target
          has two interfaces on the same Ethernet segment, this will
          result in one netlink and one devlink with two Ethernet

     AOE(3)                                                     AOE(3)

          addresses.  AoE frames are sent in round-robin fashion.
          Each successive frame is sent on the next address available
          on the next available devlink (local interface).

          Normally the initiator automatically discovers and adds new
          device directories on startup.  New devices are not added
          except as new interfaces are bound to the initiator.  Sev-
          eral messages can be written to /dev/aoe/ctl which alter
          this behavior:

          autodiscover toggle
               If toggle is absent, the state of autodiscover is tog-
               gled.  If it is the string on, it is turned on.  Any
               other string turns autodisover off.  This option is not
               useful after Ethernet devices have been bound.

          discover shelf.slot
               Attempt to find the named target on all bound inter-
               faces.

          remove shelf.slot
               The converse of discover: remove the named target if it
               exists.

          rediscover toggle
               Allow or disallow rediscovery.  This allows for auto-
               matic discovery of new targets.  Unfortunately, it also
               allows automatic modification or loss of existing tar-
               gets.  This option is considered dangerous.

          Reading /dev/aoe/ctl returns a list of colon-separated lines
          with keywords and their values:

          debug
          autodiscover
          rediscover  Returns the current state of the variable named
                      by the keyword.  Writing the variable's name to
                      the control file toggles the state of that vari-
                      able.

          ifn path    Path to nth bound Ethernet device.

          ifn ea      Ethernet address of this device.

          ifn flag    A flag of ``Up'' indicates that this interface
                      is available.

          ifn lostjumbo
                      Number of consecutive lost jumbograms.

          ifn datamtu Incorrect and unused.

     AOE(3)                                                     AOE(3)

        Target subdirectories
          Once configured, each AoE target is accessed via files in
          the directory named for its shelf and slot.  For example,
          shelf 42, slot 0 would be accessed through the path
          `/dev/aoe/42.0'.  The ident file contains the read-only,
          verbatim result of the identify unit ATA command.  The
          config file contains the target's AoE configuration string.
          Writing to this file sets the targets configuration string.

          Reading a target's ctl file returns a list of colon-
          separated lines with the following keywords and values:

          state     ``Up'' or ``down''.

          nopen     Number of clients using this target.

          nout      Number of outstanding AoE frames.

          nmaxout   Maximum number of outstanding frames allowed.

          nframes   Maximum number of outstanding frames.  Nframes is
                    greater than nmaxout when the initiator is reduc-
                    ing the number of in-flight frames due to packet
                    loss.  It is assumed that packet loss is due to an
                    overwhelmed target and not poor network condi-
                    tions.

          maxbcount Maximum number of data bytes per AoE frame.  Using
                    standard frames, maxbcount is 1024 or two sectors.
                    AoE ATA headers are 36 bytes.

          model
          serial
          firmware  The respective fields from the ATA identify unit
                    command.

          flag      List of flags useful for debugging.  The flag
                    jumbo indicates that jumbo frames are accepted,
                    not that they are being used.  Maxbcount should be
                    consulted for this purpose.

          Writing to the ctl file, the following commands may be
          issued:

          failio    fail outstanding i/o.

          identify  send an ata `identify device' command to the tar-
                    get.

          maxbno n  set the maximum number of block sent per packet.

     AOE(3)                                                     AOE(3)

          mtu n     set the maximum number of bytes (including header)
                    sent per packet.

          nofail    never fail this target.  This is useful if your
                    root device is on this target.

          setsize n with no arguments, reset the device size to the
                    size claimed.  Otherwise, assume the device is the
                    given size.

          The data file may be read or written like a normal file
          except that reads and writes to this file are converted to
          AoE commands to the target, so transfers should be 512 or
          1024 bytes long (or a larger multiple of 512 iff jumbo pack-
          ets are in use).  The size of this file is the usable size
          of the target.

          The devlink directory contains one file for each interface
          the target was discovered on.  The files are numbers from 0
          to n and contain a list of colon-separated lines with key-
          words and their values:

          addr      A space-separated list of the target's Ethernet
                    addresses visible from this interface.

          npkt      The number of frames sent on this interface.

          resent    The number of frames re-sent.  Frames are re-sent
                    when they have been outstanding twice the RTT
                    average.

          flag      ``Up'' when the netlink is up.

          rttavg
          mintimer  Minimum timer and RTT average as per Congestion
                    Avoidance and Control.

          nl path   Path of the Ethernet device.

          nl ea     Ethernet address of the local Ethernet device.

          nl flag   ``Up'' if the local interface is up.

          nl lostjumbo
                    Number of consecutive jumbograms lost.

          nl datamtu
                    Unused.

     SOURCE
          /sys/src/9/port/devaoe.c

     AOE(3)                                                     AOE(3)

     SEE ALSO
          sd(3), sdaoe(3), aoesrv(8), snoopy(8)
          http://support.coraid.com/documents/AoEr11.txt
          Van Jacobson and Michael J. Karels, ``Congestion Avoidance
          and Control'', ACM Computer Communication Review; Proceed-
          ings of the Sigcomm '88 Symposium in Stanford, CA, August,
          1988.

     BUGS
          There is no raw file for executing arbitrary commands.

          This is a fairly primitive interface; sdaoe(3) is usually
          more suitable.