Response
The server sends a snapshot of the tree, one response per change to it, a further snapshot whenever it has lost track of changes, and an error as its last response when the watch fails; each response is the byte 0 followed by a filetree frame in the postcard wire format, or the byte 1 followed by an error as one JSON value; the response finish follows the last response.
The server sends zero or more responses on the scope, and the response finish follows the last of them. No frame bearing the scope follows the finish.
[0][filetree frame, postcard …] the snapshot, or one change
[1][error JSON …] last
- The sequence. The first response is a snapshot, or an error. Zero or more responses follow the first, each of which is one change or a further snapshot. An error, when the server sends one, is the last response. The response finish follows the last response. A response finish that no response precedes states that the request was not served, as Endpoints provides.
- The filetree frame. A payload whose first byte is
0carries a filetree frame after that byte, running to the end of the payload, encoded as the postcard reading of Notation provides. The frame is one of four variants, and its discriminant is a varint:0isSnapshot, followed bychildren, a varint count followed by that many nodes;1isInserted, followed bypath, a varint count followed by that many strings, followed bynode, one node;2isModified, followed bypathandnodein the same forms;3isRemoved, followed bypath. - The node. A node is one of three variants, and its discriminant
is a varint:
0isFile, followed byname, a string,size, an optionalu64,created_at, an optionalu64, andmodified_at, an optionalu64;1isDirectory, followed byname,created_at,modified_at,changes, one byte0or1, andchildren, a varint count followed by that many nodes;2isSymlink, followed byname,path, a varint count followed by that many strings,created_atandmodified_at. An optionalu64is the byte0, or the byte1followed by a varint.nameis the name of the node in its directory and is never a path.sizeis the length of the file in bytes, absent when the server could not read it.created_atandmodified_atare seconds since1970-01-01T00:00:00Z, unsigned, absent when the filesystem records no such time.childrenis never absent; an empty directory carries an empty list. ASymlinkis the link itself, never followed: itspathis the target of the link as components relative to the root of the volume, and is present whether or not a node exists at that path. - Paths. Every
pathis a sequence of components from the root of the volume, each component thenameof a node. The root of the volume has no path and no node; a snapshot carries the entries of the root. The last component of thepathof anInsertedorModifiedframe equals thenameof itsnode. - The snapshot. The first response of every watch the server can
serve is a
Snapshotcarrying the whole tree of the volume at that time. The server sends a furtherSnapshotwhenever it has lost track of changes to the tree. EverySnapshotreplaces the tree whole. - A change. After a snapshot, the server sends one response for
each change to the filesystem of the volume, as the change occurs.
A node that comes into existence at a path that held no node is
sent as
Insertedat that path, with the node complete, a directory with its whole subtree. A node that changes in place is sent asModifiedat its path, with the node’s complete new value, a directory with its whole subtree; the value replaces the node and is not merged into it. A node that ceases to exist is sent asRemovedat its path; a directory takes everything beneath it, and the server sends no response for a descendant. A node that is relocated, whether renamed within its directory, moved elsewhere in the tree, or moved out of or into the volume, is sent asRemovedat the path it left, followed byInsertedat the path it arrived at, with the inserted node complete. The server sends no response that states a relocation as one event, and no frame of this specification does so. - Order and repetition. The server sends the responses for
changes in the order in which the changes occurred. The server may
send a response for one change more than once, and it may send a
Snapshotat any time. Every response places or clears exactly one place in the tree, and no response depends on the tree the client holds. - A directory not watched. A
Directorywhosechangesis0is one whose contents the server does not watch: the server sends no response for a change beneath it until the nextSnapshot, and itschildrenare the entries the server found when it read the directory. - The tree the client holds. A client that holds the tree applies
each response as follows. A
Snapshotreplaces the tree with itschildren. AnInsertedor aModifiedplaces itsnodeat itspath, replacing a node of thatnamein that directory when one is there. ARemovedremoves the node at itspathand everything beneath it. A response whosepathnames a parent that is absent from the tree, or that is not a directory, has no effect on the tree; the nextSnapshotcorrects it. - The error. A payload whose first byte is
1carries an error after that byte, running to the end of the payload. The error is exactly one JSON value, in the form defined on the volumes::list response page. This revision prescribes nothing about its content. The error states that the server no longer keeps the watch: it is the last response, and the tree the client holds is not kept current after it. - Malformed. A payload with no byte, a payload whose first byte
is neither
0nor1, a payload whose bytes after0do not decode as a filetree frame, and a payload whose bytes after1are not one JSON value are malformed.
The response frame is defined by diverge-provider-sdk/src/endpoints/volumes/watch/server/response/frame.rs:
//! One change on the watched tree, or a failure to keep watching.
use std::fmt;
use crate::decode::Decode;
use crate::encode::{Encode, Writer};
use crate::shared::error::Error;
use crate::shared::filetree;
/// One change on the watched tree, or the news that there will not be
/// another.
///
/// A payload leads with one byte saying which — `0` for
/// [`Filetree`](Self::Filetree), `1` for [`Error`](Self::Error) — and
/// the rest is that variant's own bytes.
///
/// # The failure is this endpoint's, the tree is not
///
/// [`Filetree`](Self::Filetree) carries
/// [`filetree::response::Frame`](crate::shared::filetree::response::Frame),
/// which is what a change to a tree looks like anywhere — a
/// [`container`](crate::endpoints::containers) reports one over its
/// own filesystem using the same type. Defining it
/// again here would be two definitions of one thing waiting to
/// disagree, and
/// [`Root::update`](crate::shared::filetree::response::Root::update)
/// would be where they did.
///
/// The [`Error`](Self::Error) beside it is not shared, because what
/// can go wrong belongs to the exchange: a volume that went away under
/// a watch is this endpoint's problem, and a container that stopped is
/// the laboratory's.
///
/// # A watch is the one stream that does not expect to end
///
/// The others answer a question and finish. This one runs for as long
/// as the caller holds the scope, so an [`Error`](Self::Error) here is
/// the interesting case rather than the edge one — it is how a caller
/// learns that a tree it has been folding is no longer being kept up
/// to date, which it could not otherwise tell from quiet.
#[derive(Debug, Clone, PartialEq)]
pub enum Frame {
/// One change on the tree. Tag `0`.
///
/// The snapshot that establishes it, or one node inserted,
/// modified, moved or removed. See
/// [`filetree::response::Frame`](crate::shared::filetree::response::Frame)
/// for the variants and for what makes the stream replay-safe.
Filetree(filetree::response::Frame),
/// A failure. Tag `1`.
///
/// See [`shared::error::Error`](crate::shared::error::Error) for
/// why it says so little.
Error(Error),
}
/// Tag for [`Frame::Filetree`].
const FILETREE: u8 = 0;
/// Tag for [`Frame::Error`].
const ERROR: u8 = 1;
/// Two variants, two formats, and the tag chooses between them. A
/// filetree frame is postcard's and is handed to postcard; an
/// [`Error`](Frame::Error) is a [`serde_json::Value`], which
/// deserializes through `deserialize_any` and so cannot come back out
/// of a format with no self-description.
impl Encode for Frame {
/// One failure per half, and they are different libraries'.
type Error = FrameEncodeError;
// Spelled out rather than `Self::Error`: this enum has a variant
// called `Error`, so the associated type is ambiguous by that name.
fn encode(
&self,
out: &mut Writer<'_>,
) -> Result<(), FrameEncodeError> {
match self {
Frame::Filetree(frame) => {
out.extend_from_slice(&[FILETREE]);
frame.encode(out).map_err(FrameEncodeError::Filetree)
}
Frame::Error(error) => {
out.extend_from_slice(&[ERROR]);
error.encode(out).map_err(FrameEncodeError::Error)
}
}
}
}
impl Decode<'_> for Frame {
/// Four ways to fail, and each names which half failed.
type Error = FrameDecodeError;
// Spelled out for the same reason as `encode` above.
fn decode(bytes: &[u8]) -> Result<Self, FrameDecodeError> {
let (tag, rest) = bytes.split_first().ok_or(FrameDecodeError::Empty)?;
match *tag {
FILETREE => filetree::response::Frame::decode(rest)
.map(Frame::Filetree)
.map_err(FrameDecodeError::Filetree),
ERROR => Error::decode(rest)
.map(Frame::Error)
.map_err(FrameDecodeError::Error),
tag => Err(FrameDecodeError::UnknownTag(tag)),
}
}
}
/// A watch response that could not be written.
#[derive(Debug)]
pub enum FrameEncodeError {
/// The filetree frame did not serialize.
Filetree(postcard::Error),
/// The error did not serialize.
Error(serde_json::Error),
}
impl fmt::Display for FrameEncodeError {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
match self {
FrameEncodeError::Filetree(error) => {
write!(f, "filetree frame did not serialize: {error}")
}
FrameEncodeError::Error(error) => {
write!(f, "watch error did not serialize: {error}")
}
}
}
}
impl std::error::Error for FrameEncodeError {
fn source(&self) -> Option<&(dyn std::error::Error + 'static)> {
match self {
FrameEncodeError::Filetree(error) => Some(error),
FrameEncodeError::Error(error) => Some(error),
}
}
}
/// A watch response that could not be read.
#[derive(Debug)]
pub enum FrameDecodeError {
/// No bytes at all, so not even a tag.
Empty,
/// A tag that is neither of this frame's two.
UnknownTag(u8),
/// The filetree frame did not decode.
Filetree(postcard::Error),
/// The error did not parse.
Error(serde_json::Error),
}
impl fmt::Display for FrameDecodeError {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
match self {
FrameDecodeError::Empty => {
f.write_str("watch response frame is empty")
}
FrameDecodeError::UnknownTag(tag) => {
write!(f, "unknown watch response frame tag {tag}")
}
FrameDecodeError::Filetree(error) => {
write!(f, "filetree frame did not decode: {error}")
}
FrameDecodeError::Error(error) => {
write!(f, "watch error did not parse: {error}")
}
}
}
}
impl std::error::Error for FrameDecodeError {
fn source(&self) -> Option<&(dyn std::error::Error + 'static)> {
match self {
FrameDecodeError::Filetree(error) => Some(error),
FrameDecodeError::Error(error) => Some(error),
FrameDecodeError::Empty | FrameDecodeError::UnknownTag(_) => None,
}
}
}
The filetree frame is defined by diverge-provider-sdk/src/shared/filetree/response/frame.rs:
//! What a filetree response frame carries.
use serde::{Deserialize, Serialize};
use super::Node;
use crate::decode::Decode;
use crate::encode::{Encode, Writer};
/// One change on a filetree stream.
///
/// [`Snapshot`](Frame::Snapshot) establishes the tree. Every other
/// variant names exactly one node and says what became of it: it
/// appeared ([`Inserted`](Frame::Inserted)), changed in place
/// ([`Modified`](Frame::Modified)), or ceased to exist
/// ([`Removed`](Frame::Removed)).
///
/// Insertion and modification are distinguished because a consumer
/// usually wants to treat them differently — a newly appeared file is
/// not the same news as an existing one being written to. The
/// distinction is INFORMATION, not a constraint: a consumer with no
/// use for it may treat the two identically, and doing so is what
/// keeps the fold tolerant of replay and reordering.
///
/// A delta that carries a node carries its COMPLETE value, never a
/// patch against a value the consumer is assumed to hold. That is what
/// makes replaying an already-applied frame harmless, and therefore
/// what makes at-least-once delivery safe: every variant overwrites
/// or clears one place in the tree, and none reads the tree first.
///
/// # A rename is two frames
///
/// There is no move. A node renamed — within one directory or across
/// the tree — is reported as what happened on disk:
/// [`Removed`](Frame::Removed) at the path it left and
/// [`Inserted`](Frame::Inserted) at the path it arrived at, the
/// inserted node complete, a directory with its whole subtree. The
/// pairing a filesystem offers for the two halves is not reliable
/// enough to promise a consumer, and the tree is right without it;
/// what a consumer loses is only the knowledge that the two were one
/// node.
///
/// Every `path` in every variant is a component vector relative to the
/// filetree root — one meaning of "path" throughout, matching
/// [`Node::Symlink`]'s.
///
/// # Variant ORDER is part of the wire format
///
/// Serialized in serde's default representation, which writes the
/// variant's INDEX rather than its name — the same arrangement
/// [`Node`] uses, and for the same reason: an index needs no lookahead
/// to read, which is what makes it encodable in a format with no
/// self-description.
///
/// It is also a constraint. Reordering these variants, or inserting
/// one among them, silently changes what existing bytes mean. New
/// variants go on the END; nowhere else is a compatible change.
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
pub enum Frame {
/// The whole tree: the root's entries, recursively.
///
/// The first frame of every stream, and the only frame that may
/// come again: a source that lost track of the tree — a watch
/// whose event queue overflowed — sends a fresh one rather than
/// deltas it cannot know. Each replaces the tree whole.
Snapshot {
/// The root's entries. The root itself is not among them and
/// is not described — see [`Root`](super::Root).
children: Vec<Node>,
},
/// A node came into existence at a path that held nothing.
///
/// Also how a node arrives by rename, from elsewhere in the tree
/// or from outside it: from this path's point of view nothing was
/// relocated, something appeared.
Inserted {
/// Where the node appeared. The last element equals `node`'s
/// `name`.
///
/// Components rather than a joined string: a path is a
/// sequence, and joining it would invent a separator that then
/// has to be escaped out of names that contain it.
path: Vec<String>,
/// The node's complete value. A directory carries its whole
/// subtree.
node: Node,
},
/// A node that already existed changed, staying where it was.
Modified {
/// The node's path, unchanged by this frame. The last element
/// equals `node`'s `name`.
path: Vec<String>,
/// The node's complete new value, replacing the old one. A
/// directory carries its whole subtree, so this replaces rather
/// than merges.
node: Node,
},
/// A node ceased to exist. A directory takes its whole subtree with
/// it — no per-descendant removals follow.
///
/// Also how a node leaves by rename — to elsewhere in the tree,
/// where an [`Inserted`](Frame::Inserted) reports its arrival, or
/// out of it.
Removed {
/// The vanished node's path.
path: Vec<String>,
},
}
/// Postcard, where the rest of the crate is JSON.
///
/// A filetree stream is the one thing here that is both high-volume
/// and free to choose: it relays nothing, so no byte of it has to
/// survive a round trip unchanged, and nothing downstream reads it as
/// text. What it is instead is spammy — one frame per changed node,
/// indefinitely — so the envelope is worth minimizing.
///
/// Postcard drops field names entirely and varint-encodes every length
/// and integer, which puts a small delta within a couple of bytes of
/// the information it actually carries. `Removed` naming
/// `src/main.rs` is fourteen bytes, ten of them the two strings
/// themselves.
impl Encode for Frame {
/// Postcard's own failure. It has few ways to happen when writing
/// — a buffer that will not take bytes, mostly — since everything
/// here is a shape it can always represent.
type Error = postcard::Error;
fn encode(&self, out: &mut Writer<'_>) -> Result<(), Self::Error> {
postcard::to_io(self, &mut *out)?;
Ok(())
}
}
impl Decode<'_> for Frame {
/// Postcard's own failure.
type Error = postcard::Error;
fn decode(bytes: &[u8]) -> Result<Self, Self::Error> {
postcard::from_bytes(bytes)
}
}
The node is defined by diverge-provider-sdk/src/shared/filetree/response/node.rs:
//! One node of the filesystem tree.
use serde::{Deserialize, Serialize};
/// One node of the filesystem tree.
///
/// A [`Directory`](Node::Directory) carries its children inline;
/// [`File`](Node::File) and [`Symlink`](Node::Symlink) are leaves. A
/// symlink is the link ITSELF and is never followed, so a dangling or
/// looping link is a leaf rather than an error or an infinite tree.
///
/// Every variant carries `name` — the basename, never a path — plus
/// `created_at` and `modified_at`. A directory alone carries
/// `changes`: whether what happens beneath it will be reported, which
/// is a question only a directory can answer, because only a
/// directory is watched — see the field.
///
/// # Times are unsigned seconds
///
/// The same representation
/// [`Volume::created`](crate::endpoints::volumes::list::server::response::Volume::created)
/// uses, and unsigned for the same reason: nothing a provider offers
/// predates 1970, and a signed field's negative half would exist to
/// represent a state that never occurs. `Option` here is about
/// availability rather than sign — a filesystem that records no birth
/// time has nothing to report, which is not the same as reporting a
/// time before the epoch.
///
/// # Variant ORDER is part of the wire format
///
/// This enum is serialized in serde's default representation, which
/// writes the variant's INDEX rather than its name. That is what makes
/// it encodable at all in a format with no self-description — there is
/// no name to look up and no lookahead to do — and it is also a
/// constraint: reordering these variants, or inserting one among them,
/// silently changes what existing bytes mean.
///
/// New variants go on the END. Nowhere else is a compatible change.
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
pub enum Node {
/// A regular file.
File {
/// Basename of this file.
name: String,
/// Size in bytes. `None` when the stat could not be read.
size: Option<u64>,
/// Creation time (unix seconds), when the filesystem records a
/// birth time. `None` where unsupported — this is display
/// metadata and is never load-bearing.
created_at: Option<u64>,
/// Last-modified time (unix seconds). `None` when the stat
/// could not be read.
modified_at: Option<u64>,
},
/// A directory, carrying its entries.
Directory {
/// Basename of this directory. The watched root has no node of
/// its own — see [`Root`](super::Root).
name: String,
/// Creation time (unix seconds), when the filesystem records a
/// birth time.
created_at: Option<u64>,
/// Last-modified time (unix seconds). A directory's mtime
/// tracks entry add/remove, not changes within its children.
modified_at: Option<u64>,
/// Whether changes beneath this directory stream.
///
/// `true` is the ordinary case: what happens under it arrives
/// as deltas. `false` is a directory the source could not
/// watch — a watch limit reached, a corner it was refused —
/// and with it everything beneath, files included: what is
/// here is what the walk found, nothing under it will be
/// reported until the next [`Snapshot`](super::Frame::Snapshot),
/// and a consumer should treat it as possibly stale. Only a
/// directory carries this, because only a directory is
/// watched: a file's changes are its parent's to report, so a
/// file cannot fail to be watched on its own.
changes: bool,
/// This directory's entries. An empty directory carries an
/// empty list — this field is never absent, so a consumer never
/// has to distinguish "no children" from "children unknown".
children: Vec<Node>,
},
/// A symbolic link — the link itself, never its target.
Symlink {
/// Basename of this link.
name: String,
/// The link's target, as path components ALWAYS RELATIVE TO
/// THE FILETREE ROOT — the same frame of reference as the
/// `path` carried by [`Frame::Inserted`](super::Frame::Inserted),
/// [`Frame::Modified`](super::Frame::Modified) and
/// [`Frame::Removed`](super::Frame::Removed). Every path in
/// this API means the same thing, so a consumer walks a link's
/// target down from the snapshot's child list exactly as it
/// walks a frame's path, with no separate rule for links.
///
/// Addressable is not the same as resolved: the link is still
/// never followed, and the components may name a node that
/// does not exist — an ordinary dangling link.
///
/// Always present. A link whose contents could not be read is
/// not a symlink node with a missing target; it is a failure,
/// and is reported as one. That is unrelated to dangling: a
/// link pointing at nothing still reports its components
/// perfectly well, because reading a link never touches its
/// target.
path: Vec<String>,
/// Creation time (unix seconds), when the filesystem records a
/// birth time.
created_at: Option<u64>,
/// Last-modified time (unix seconds).
modified_at: Option<u64>,
},
}
// Visible throughout `response` — but no wider. These exist so
// [`Root`](super::Root)'s fold can walk and edit a tree; they are not
// part of the specification and nothing outside this module should
// depend on them.
impl Node {
/// This node's basename.
pub(super) fn name(&self) -> &str {
match self {
Node::File { name, .. }
| Node::Directory { name, .. }
| Node::Symlink { name, .. } => name,
}
}
/// This node's entries, mutably — `None` for anything but a
/// directory.
pub(super) fn children_mut(&mut self) -> Option<&mut Vec<Node>> {
match self {
Node::Directory { children, .. } => Some(children),
_ => None,
}
}
}