Response

The sequence of channel responses the server sends, what ends it, and what a finish with nothing before it means.

[0][filetree frame, postcard …]      the snapshot, or one change
[1][error JSON …]                    last
  • The sequence. The first channel response is the snapshot the proxy sent, or an error. Zero or more channel responses follow, each one change or a further snapshot, each relayed as the proxy sent it. An error, when the server sends one, is the last channel response. The channel response finish follows the last channel response: the server sends it when the proxy finishes the tree scope. A channel response finish that no channel response precedes states that the proxy did not serve the watch.
  • The filetree frame. After the byte 0, the payload is the filetree frame the proxy sent, verbatim, running to the end of the payload, encoded as the postcard reading of Notation provides. The server does not read it. The frame is one of four variants, and its discriminant is a varint: 0 is Snapshot, followed by children, a varint count followed by that many nodes; 1 is Inserted, followed by path, a varint count followed by that many strings, followed by node, one node; 2 is Modified, followed by path and node in the same forms; 3 is Removed, followed by path.
  • The node. A node is one of three variants, and its discriminant is a varint: 0 is File, followed by name, a string, size, an optional u64, created_at, an optional u64, and modified_at, an optional u64; 1 is Directory, followed by name, created_at, modified_at, changes, one byte 0 or 1, and children, a varint count followed by that many nodes; 2 is Symlink, followed by name, path, a varint count followed by that many strings, created_at and modified_at. An optional u64 is the byte 0, or the byte 1 followed by a varint. name is the name of the node in its directory and is never a path. size is the length of the file in bytes, absent when the proxy could not read it. created_at and modified_at are seconds since 1970-01-01T00:00:00Z, unsigned, absent when the filesystem records no such time. children is never absent; an empty directory carries an empty list. A Symlink is the link itself, never followed: its path is the target of the link as components relative to the root of the container, and is present whether or not a node exists at that path.
  • Paths. Every path is a sequence of components from the root of the container, each component the name of a node. The root of the container has no path and no node; a snapshot carries the entries of the root. The last component of the path of an Inserted or Modified frame equals the name of its node.
  • The snapshot. The first channel response of every filetree the proxy serves is a Snapshot carrying the whole tree of the container at that time, less what is left out. A further Snapshot is sent whenever the proxy has lost track of changes. Every Snapshot replaces the tree whole.
  • A change. After a snapshot, one channel response is sent for each change to the filesystem of the container, as the change occurs. A node that comes into existence at a path that held no node is sent as Inserted at that path, with the node complete, a directory with its whole subtree. A node that changes in place is sent as Modified at its path, with the node’s complete new value, a directory with its whole subtree; the value replaces the node and is not merged into it. A node that ceases to exist is sent as Removed at its path; a directory takes everything beneath it, and no channel response is sent for a descendant. A node that is relocated, whether renamed within its directory or moved elsewhere in the tree, is sent as Removed at the path it left, followed by Inserted at the path it arrived at, with the inserted node complete. No channel response states a relocation as one event, and no frame of this specification does so.
  • Order and repetition. The channel responses for changes are sent in the order in which the changes occurred. A channel response for one change may be sent more than once, and a Snapshot may be sent at any time. Every channel response places or clears exactly one place in the tree, and no channel response depends on the tree the client holds.
  • A directory not watched. A Directory whose changes is 0 is one whose contents the proxy does not watch: no channel response is sent for a change beneath it until the next Snapshot, and its children are the entries the proxy found when it read the directory.
  • The tree the client holds. A client that holds the tree applies each channel response as follows. A Snapshot replaces the tree with its children. An Inserted or a Modified places its node at its path, replacing a node of that name in that directory when one is there. A Removed removes the node at its path and everything beneath it. A channel response whose path names a parent that is absent from the tree, or that is not a directory, has no effect on the tree; the next Snapshot corrects it.
  • The error. After the byte 1, exactly one JSON value in the form defined on the volumes::list response page. The server sends it when the proxy sent its error, or when the server could not carry the watch.

The channel response is defined by diverge-sdk-rs/src/provider/endpoints/containers/tools/run/server/channel_response/filetree/frame.rs:

//! What a server's channel response frame carries on a filetree
//! channel.

use std::fmt;

use crate::wire::decode::Decode;
use crate::wire::encode::{Encode, Writer};
use crate::shared::error::Error;
use crate::shared::filetree;

/// One change on the container's filesystem, or the news that the
/// tree cannot be watched.
///
/// A payload leads with one byte saying which — `0` for
/// [`Filetree`](Self::Filetree), `1` for [`Error`](Self::Error) — and
/// the rest is that variant's own bytes.
///
/// # How it ends
///
/// | the channel ends with | means |
/// |-----------------------|-------|
/// | frames, then a finish | the watch ended — the scope did, or the container |
/// | an [`Error`](Self::Error), then a finish | the tree could not be watched, or is no longer |
///
/// A [`Snapshot`](crate::shared::filetree::response::Frame::Snapshot)
/// comes first and may come again — see
/// [`shared::filetree`](crate::shared::filetree) for when.
#[derive(Debug, Clone, PartialEq)]
pub enum Frame {
    /// One change on the container's filesystem. Tag `0`.
    ///
    /// A [`filetree`](crate::shared::filetree) stream over the
    /// container's own root — one snapshot, then one frame per change
    /// — which is the stream the proxy's
    /// [`filesystem::tree`](crate::container_proxy::outside::endpoints::filesystem::tree)
    /// carries, relayed.
    Filetree(filetree::response::Frame),
    /// A failure. Tag `1`.
    ///
    /// See [`shared::error::Error`](crate::shared::error::Error) for
    /// why it says so little.
    Error(Error),
}

/// Tag for [`Frame::Filetree`].
const FILETREE: u8 = 0;

/// Tag for [`Frame::Error`].
const ERROR: u8 = 1;

/// Two variants, two encodings, and neither converted into the
/// other's: a filetree frame is postcard's and is handed to postcard;
/// an error is JSON, because a [`serde_json::Value`] cannot come back
/// out of postcard at all.
impl Encode for Frame {
    /// One failure per half, and they are different libraries'.
    type Error = FrameEncodeError;

    // Spelled out rather than `Self::Error`: this enum has a variant
    // called `Error`, so the associated type is ambiguous by that name.
    fn encode(
        &self,
        out: &mut Writer<'_>,
    ) -> Result<(), FrameEncodeError> {
        match self {
            Frame::Filetree(frame) => {
                out.extend_from_slice(&[FILETREE]);
                frame.encode(out).map_err(FrameEncodeError::Filetree)
            }
            Frame::Error(error) => {
                out.extend_from_slice(&[ERROR]);
                error.encode(out).map_err(FrameEncodeError::Error)
            }
        }
    }
}

/// A filetree response that could not be written.
#[derive(Debug)]
pub enum FrameEncodeError {
    /// The filetree frame did not serialize.
    Filetree(postcard::Error),
    /// The error did not serialize.
    Error(serde_json::Error),
}

impl fmt::Display for FrameEncodeError {
    fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
        match self {
            FrameEncodeError::Filetree(error) => {
                write!(f, "filetree frame did not serialize: {error}")
            }
            FrameEncodeError::Error(error) => {
                write!(f, "filetree error did not serialize: {error}")
            }
        }
    }
}

impl std::error::Error for FrameEncodeError {
    fn source(&self) -> Option<&(dyn std::error::Error + 'static)> {
        match self {
            FrameEncodeError::Filetree(error) => Some(error),
            FrameEncodeError::Error(error) => Some(error),
        }
    }
}

impl Decode<'_> for Frame {
    /// Four ways to fail, and each names which half failed.
    type Error = FrameError;

    // Spelled out for the same reason as `encode` above.
    fn decode(bytes: &[u8]) -> Result<Self, FrameError> {
        let (tag, rest) = bytes.split_first().ok_or(FrameError::Empty)?;
        match *tag {
            FILETREE => filetree::response::Frame::decode(rest)
                .map(Frame::Filetree)
                .map_err(FrameError::Filetree),
            ERROR => {
                Error::decode(rest).map(Frame::Error).map_err(FrameError::Error)
            }
            tag => Err(FrameError::UnknownTag(tag)),
        }
    }
}

/// A filetree response frame that could not be read.
#[derive(Debug)]
pub enum FrameError {
    /// No bytes at all, so not even a tag.
    Empty,
    /// A tag that is neither of this frame's two.
    UnknownTag(u8),
    /// The filetree frame did not decode.
    Filetree(postcard::Error),
    /// The error did not parse.
    Error(serde_json::Error),
}

impl fmt::Display for FrameError {
    fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
        match self {
            FrameError::Empty => {
                f.write_str("filetree response frame is empty")
            }
            FrameError::UnknownTag(tag) => {
                write!(f, "unknown filetree response frame tag {tag}")
            }
            FrameError::Filetree(error) => {
                write!(f, "filetree frame did not decode: {error}")
            }
            FrameError::Error(error) => {
                write!(f, "filetree error did not parse: {error}")
            }
        }
    }
}

impl std::error::Error for FrameError {
    fn source(&self) -> Option<&(dyn std::error::Error + 'static)> {
        match self {
            FrameError::Filetree(error) => Some(error),
            FrameError::Error(error) => Some(error),
            FrameError::Empty | FrameError::UnknownTag(_) => None,
        }
    }
}

and by diverge-sdk-rs/src/shared/filetree/response/frame.rs:

//! What a filetree response frame carries.

use serde::{Deserialize, Serialize};

use super::Node;
use crate::wire::decode::Decode;
use crate::wire::encode::{Encode, Writer};

/// One change on a filetree stream.
///
/// [`Snapshot`](Frame::Snapshot) establishes the tree. Every other
/// variant names exactly one node and says what became of it: it
/// appeared ([`Inserted`](Frame::Inserted)), changed in place
/// ([`Modified`](Frame::Modified)), or ceased to exist
/// ([`Removed`](Frame::Removed)).
///
/// Insertion and modification are distinguished because a consumer
/// usually wants to treat them differently — a newly appeared file is
/// not the same news as an existing one being written to. The
/// distinction is INFORMATION, not a constraint: a consumer with no
/// use for it may treat the two identically, and doing so is what
/// keeps the fold tolerant of replay and reordering.
///
/// A delta that carries a node carries its COMPLETE value, never a
/// patch against a value the consumer is assumed to hold. That is what
/// makes replaying an already-applied frame harmless, and therefore
/// what makes at-least-once delivery safe: every variant overwrites
/// or clears one place in the tree, and none reads the tree first.
///
/// # A rename is two frames
///
/// There is no move. A node renamed — within one directory or across
/// the tree — is reported as what happened on disk:
/// [`Removed`](Frame::Removed) at the path it left and
/// [`Inserted`](Frame::Inserted) at the path it arrived at, the
/// inserted node complete, a directory with its whole subtree. The
/// pairing a filesystem offers for the two halves is not reliable
/// enough to promise a consumer, and the tree is right without it;
/// what a consumer loses is only the knowledge that the two were one
/// node.
///
/// Every `path` in every variant is a component vector relative to the
/// filetree root — one meaning of "path" throughout, matching
/// [`Node::Symlink`]'s.
///
/// # Variant ORDER is part of the wire format
///
/// Serialized in serde's default representation, which writes the
/// variant's INDEX rather than its name — the same arrangement
/// [`Node`] uses, and for the same reason: an index needs no lookahead
/// to read, which is what makes it encodable in a format with no
/// self-description.
///
/// It is also a constraint. Reordering these variants, or inserting
/// one among them, silently changes what existing bytes mean. New
/// variants go on the END; nowhere else is a compatible change.
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
pub enum Frame {
    /// The whole tree: the root's entries, recursively.
    ///
    /// The first frame of every stream, and the only frame that may
    /// come again: a source that lost track of the tree — a watch
    /// whose event queue overflowed — sends a fresh one rather than
    /// deltas it cannot know. Each replaces the tree whole.
    Snapshot {
        /// The root's entries. The root itself is not among them and
        /// is not described — see [`Root`](super::Root).
        children: Vec<Node>,
    },
    /// A node came into existence at a path that held nothing.
    ///
    /// Also how a node arrives by rename, from elsewhere in the tree
    /// or from outside it: from this path's point of view nothing was
    /// relocated, something appeared.
    Inserted {
        /// Where the node appeared. The last element equals `node`'s
        /// `name`.
        ///
        /// Components rather than a joined string: a path is a
        /// sequence, and joining it would invent a separator that then
        /// has to be escaped out of names that contain it.
        path: Vec<String>,
        /// The node's complete value. A directory carries its whole
        /// subtree.
        node: Node,
    },
    /// A node that already existed changed, staying where it was.
    Modified {
        /// The node's path, unchanged by this frame. The last element
        /// equals `node`'s `name`.
        path: Vec<String>,
        /// The node's complete new value, replacing the old one. A
        /// directory carries its whole subtree, so this replaces rather
        /// than merges.
        node: Node,
    },
    /// A node ceased to exist. A directory takes its whole subtree with
    /// it — no per-descendant removals follow.
    ///
    /// Also how a node leaves by rename — to elsewhere in the tree,
    /// where an [`Inserted`](Frame::Inserted) reports its arrival, or
    /// out of it.
    Removed {
        /// The vanished node's path.
        path: Vec<String>,
    },
}

/// Postcard, where the rest of the crate is JSON.
///
/// A filetree stream is the one thing here that is both high-volume
/// and free to choose: it relays nothing, so no byte of it has to
/// survive a round trip unchanged, and nothing downstream reads it as
/// text. What it is instead is spammy — one frame per changed node,
/// indefinitely — so the envelope is worth minimizing.
///
/// Postcard drops field names entirely and varint-encodes every length
/// and integer, which puts a small delta within a couple of bytes of
/// the information it actually carries. `Removed` naming
/// `src/main.rs` is fourteen bytes, ten of them the two strings
/// themselves.
impl Encode for Frame {
    /// Postcard's own failure. It has few ways to happen when writing
    /// — a buffer that will not take bytes, mostly — since everything
    /// here is a shape it can always represent.
    type Error = postcard::Error;

    fn encode(&self, out: &mut Writer<'_>) -> Result<(), Self::Error> {
        postcard::to_io(self, &mut *out)?;
        Ok(())
    }
}

impl Decode<'_> for Frame {
    /// Postcard's own failure.
    type Error = postcard::Error;

    fn decode(bytes: &[u8]) -> Result<Self, Self::Error> {
        postcard::from_bytes(bytes)
    }
}

and by diverge-sdk-rs/src/shared/filetree/response/node.rs:

//! One node of the filesystem tree.

use serde::{Deserialize, Serialize};

/// One node of the filesystem tree.
///
/// A [`Directory`](Node::Directory) carries its children inline;
/// [`File`](Node::File) and [`Symlink`](Node::Symlink) are leaves. A
/// symlink is the link ITSELF and is never followed, so a dangling or
/// looping link is a leaf rather than an error or an infinite tree.
///
/// Every variant carries `name` — the basename, never a path — plus
/// `created_at` and `modified_at`. A directory alone carries
/// `changes`: whether what happens beneath it will be reported, which
/// is a question only a directory can answer, because only a
/// directory is watched — see the field.
///
/// # Times are unsigned seconds
///
/// The same representation
/// [`Volume::created`](crate::provider::endpoints::volumes::list::server::response::Volume::created)
/// uses, and unsigned for the same reason: nothing a provider offers
/// predates 1970, and a signed field's negative half would exist to
/// represent a state that never occurs. `Option` here is about
/// availability rather than sign — a filesystem that records no birth
/// time has nothing to report, which is not the same as reporting a
/// time before the epoch.
///
/// # Variant ORDER is part of the wire format
///
/// This enum is serialized in serde's default representation, which
/// writes the variant's INDEX rather than its name. That is what makes
/// it encodable at all in a format with no self-description — there is
/// no name to look up and no lookahead to do — and it is also a
/// constraint: reordering these variants, or inserting one among them,
/// silently changes what existing bytes mean.
///
/// New variants go on the END. Nowhere else is a compatible change.
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
pub enum Node {
    /// A regular file.
    File {
        /// Basename of this file.
        name: String,
        /// Size in bytes. `None` when the stat could not be read.
        size: Option<u64>,
        /// Creation time (unix seconds), when the filesystem records a
        /// birth time. `None` where unsupported — this is display
        /// metadata and is never load-bearing.
        created_at: Option<u64>,
        /// Last-modified time (unix seconds). `None` when the stat
        /// could not be read.
        modified_at: Option<u64>,
    },
    /// A directory, carrying its entries.
    Directory {
        /// Basename of this directory. The watched root has no node of
        /// its own — see [`Root`](super::Root).
        name: String,
        /// Creation time (unix seconds), when the filesystem records a
        /// birth time.
        created_at: Option<u64>,
        /// Last-modified time (unix seconds). A directory's mtime
        /// tracks entry add/remove, not changes within its children.
        modified_at: Option<u64>,
        /// Whether changes beneath this directory stream.
        ///
        /// `true` is the ordinary case: what happens under it arrives
        /// as deltas. `false` is a directory the source could not
        /// watch — a watch limit reached, a corner it was refused —
        /// and with it everything beneath, files included: what is
        /// here is what the walk found, nothing under it will be
        /// reported until the next [`Snapshot`](super::Frame::Snapshot),
        /// and a consumer should treat it as possibly stale. Only a
        /// directory carries this, because only a directory is
        /// watched: a file's changes are its parent's to report, so a
        /// file cannot fail to be watched on its own.
        changes: bool,
        /// This directory's entries. An empty directory carries an
        /// empty list — this field is never absent, so a consumer never
        /// has to distinguish "no children" from "children unknown".
        children: Vec<Node>,
    },
    /// A symbolic link — the link itself, never its target.
    Symlink {
        /// Basename of this link.
        name: String,
        /// The link's target, as path components ALWAYS RELATIVE TO
        /// THE FILETREE ROOT — the same frame of reference as the
        /// `path` carried by [`Frame::Inserted`](super::Frame::Inserted),
        /// [`Frame::Modified`](super::Frame::Modified) and
        /// [`Frame::Removed`](super::Frame::Removed). Every path in
        /// this API means the same thing, so a consumer walks a link's
        /// target down from the snapshot's child list exactly as it
        /// walks a frame's path, with no separate rule for links.
        ///
        /// Addressable is not the same as resolved: the link is still
        /// never followed, and the components may name a node that
        /// does not exist — an ordinary dangling link.
        ///
        /// Always present. A link whose contents could not be read is
        /// not a symlink node with a missing target; it is a failure,
        /// and is reported as one. That is unrelated to dangling: a
        /// link pointing at nothing still reports its components
        /// perfectly well, because reading a link never touches its
        /// target.
        path: Vec<String>,
        /// Creation time (unix seconds), when the filesystem records a
        /// birth time.
        created_at: Option<u64>,
        /// Last-modified time (unix seconds).
        modified_at: Option<u64>,
    },
}

// Visible throughout `response` — but no wider. These exist so
// [`Root`](super::Root)'s fold can walk and edit a tree; they are not
// part of the specification and nothing outside this module should
// depend on them.
impl Node {
    /// This node's basename.
    pub(super) fn name(&self) -> &str {
        match self {
            Node::File { name, .. }
            | Node::Directory { name, .. }
            | Node::Symlink { name, .. } => name,
        }
    }

    /// This node's entries, mutably — `None` for anything but a
    /// directory.
    pub(super) fn children_mut(&mut self) -> Option<&mut Vec<Node>> {
        match self {
            Node::Directory { children, .. } => Some(children),
            _ => None,
        }
    }
}