/filesystem/tree
The filesystem of the container, watched: the server sends exactly one message naming the paths to leave out; the proxy sends a snapshot, one message per change, a snapshot again after lost events, or one error as its last message.
The server opens /filesystem/tree and sends exactly one message,
the request defined below as JSON, naming the paths the tree leaves
out. The server sends no further message; a further binary message
from the server ends the connection. The proxy sends one message per
event for as long as the path lives.
server → proxy: [request JSON] once
proxy → server: [0][postcard-encoded filetree frame …] per event
[1][message …] last- What the tree leaves out. The tree leaves out every path the
request names, each given as components from the root of the
container, and the paths
/proc,/sysand/dev. A path left out is absent from every snapshot, and the proxy reports no change under it. The proxy disregards an empty path in the request. - Not served. A close before the request, and a request that does not decode, cause the proxy to close the connection with no message before the close.
- A snapshot first. The first message on every opening is a
snapshot of the whole tree, with first byte
0. Every later message with first byte0is one change, sent as it occurs. - A snapshot again. A watch that has lost events sends a fresh snapshot on the same path, and the fresh snapshot replaces the tree whole.
- One error, last. A message with first byte
1, followed by a message, states that the watch could not exist. It is the last message the proxy sends before the close. A part of the tree that the proxy could not watch is not an error; the tree itself reports it, with thechangesfield of its directory set tofalse. - Every opening starts whole. No opening resumes a previous one. An opening after an ending, clean or abrupt, begins with a snapshot.
- The encoding. After the first byte
0, the message is the filetree response frame defined below, encoded in the postcard wire format. The order of variants is the discriminant on the wire.
The message is defined by
diverge-provider-sdk/src/container_proxy/filesystem/tree/request/request.rs:
//! What the tree leaves out.
use serde::{Deserialize, Serialize};
use crate::decode::Decode;
use crate::encode::{Encode, Writer};
/// The first message on `/filesystem/tree`, from the server: the
/// paths the tree does not contain.
///
/// The server names the MOUNTS here — every one it placed in the
/// container, volume and identity and FUSE alike, whose watch would
/// cost the walk and report what the caller already holds. `/proc`,
/// `/sys` and `/dev` are the proxy's own and are never listed. Each
/// path is components from the container's root, the shape every
/// path in this crate takes; an empty one is dropped rather than read
/// as the root. An ignored path does not exist as far as the stream
/// is concerned: absent from the snapshot, never watched, an event
/// under it dropped.
#[derive(Debug, Clone, PartialEq, Eq, Hash, Serialize, Deserialize, Default)]
pub struct Request {
/// The paths to leave out, each as components from the root.
pub ignore: Vec<Vec<String>>,
}
impl Encode for Request {
/// The ordinary JSON failure.
type Error = serde_json::Error;
fn encode(&self, out: &mut Writer<'_>) -> Result<(), Self::Error> {
serde_json::to_writer(out, self)
}
}
impl Decode<'_> for Request {
/// The ordinary JSON failure.
type Error = serde_json::Error;
fn decode(bytes: &[u8]) -> Result<Self, Self::Error> {
serde_json::from_slice(bytes)
}
}The messages the proxy sends are defined by
diverge-provider-sdk/src/container_proxy/filesystem/tree/response/frame.rs:
//! One event of the tree, or why there will be no more.
use super::FrameError;
use crate::decode::Decode as _;
use crate::encode::{Encode, Writer};
use crate::shared::filetree;
/// One message on `/filesystem/tree`.
///
/// ```text
/// [kind: u8][postcard frame… | message…]
/// ```
///
/// The first on a connection is a
/// [`Snapshot`](filetree::response::Frame::Snapshot) behind kind `0`;
/// every one after is a delta, or a snapshot again when the watch
/// lost events; and an [`Error`](Self::Error) is the last, when the
/// watch could not exist at all.
#[derive(Debug, Clone, PartialEq)]
pub enum Frame<'a> {
/// Kind `0`. One filetree event, as
/// [`shared::filetree`](crate::shared::filetree) encodes it.
Filetree(filetree::response::Frame),
/// Kind `1`. The watch could not be made, the root could not be
/// watched, or the walk died — and this says why, for a reader
/// rather than a program. The last message before the close;
/// nothing follows it, and the server starts over when it likes.
Error(&'a str),
}
impl Encode for Frame<'_> {
/// postcard's own failure, from the half that has one; a kind
/// byte and a message's bytes cannot fail.
type Error = postcard::Error;
fn encode(&self, out: &mut Writer<'_>) -> Result<(), postcard::Error> {
match self {
Frame::Filetree(frame) => {
out.extend_from_slice(&[0]);
frame.encode(out)
}
Frame::Error(message) => {
out.extend_from_slice(&[1]);
out.extend_from_slice(message.as_bytes());
Ok(())
}
}
}
}
impl<'a> Frame<'a> {
/// Decode one message. The error message borrows from `bytes`.
pub fn decode(bytes: &'a [u8]) -> Result<Self, FrameError> {
let (kind, rest) = bytes.split_first().ok_or(FrameError::Empty)?;
match *kind {
0 => filetree::response::Frame::decode(rest)
.map(Frame::Filetree)
.map_err(FrameError::Filetree),
1 => std::str::from_utf8(rest)
.map(Frame::Error)
.map_err(|_| FrameError::MessageUtf8),
other => Err(FrameError::UnknownKind(other)),
}
}
}The ways in which such a message fails to decode are defined by
diverge-provider-sdk/src/container_proxy/filesystem/tree/response/error.rs:
//! Why a message on `/filesystem/tree` could not be decoded.
use std::error;
use std::fmt;
/// A filetree message that could not be read.
#[derive(Debug)]
pub enum FrameError {
/// No bytes at all, so not even a kind.
Empty,
/// A kind this answer does not define.
UnknownKind(u8),
/// The postcard payload would not decode as a filetree frame.
Filetree(postcard::Error),
/// An error message that is not UTF-8.
MessageUtf8,
}
impl fmt::Display for FrameError {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
match self {
FrameError::Empty => f.write_str("filetree message is empty"),
FrameError::UnknownKind(kind) => {
write!(f, "unknown filetree message kind {kind}")
}
FrameError::Filetree(error) => {
write!(f, "filetree frame did not decode: {error}")
}
FrameError::MessageUtf8 => {
f.write_str("filetree error message is not utf-8")
}
}
}
}
impl error::Error for FrameError {
fn source(&self) -> Option<&(dyn error::Error + 'static)> {
match self {
FrameError::Filetree(error) => Some(error),
FrameError::Empty
| FrameError::UnknownKind(_)
| FrameError::MessageUtf8 => None,
}
}
}The filetree frame after the first byte 0 is defined by
diverge-provider-sdk/src/shared/filetree/response/frame.rs:
//! What a filetree response frame carries.
use serde::{Deserialize, Serialize};
use super::Node;
use crate::decode::Decode;
use crate::encode::{Encode, Writer};
/// One change on a filetree stream.
///
/// [`Snapshot`](Frame::Snapshot) establishes the tree. Every other
/// variant names exactly one node and says what became of it: it
/// appeared ([`Inserted`](Frame::Inserted)), changed in place
/// ([`Modified`](Frame::Modified)), or ceased to exist
/// ([`Removed`](Frame::Removed)).
///
/// Insertion and modification are distinguished because a consumer
/// usually wants to treat them differently — a newly appeared file is
/// not the same news as an existing one being written to. The
/// distinction is INFORMATION, not a constraint: a consumer with no
/// use for it may treat the two identically, and doing so is what
/// keeps the fold tolerant of replay and reordering.
///
/// A delta that carries a node carries its COMPLETE value, never a
/// patch against a value the consumer is assumed to hold. That is what
/// makes replaying an already-applied frame harmless, and therefore
/// what makes at-least-once delivery safe: every variant overwrites
/// or clears one place in the tree, and none reads the tree first.
///
/// # A rename is two frames
///
/// There is no move. A node renamed — within one directory or across
/// the tree — is reported as what happened on disk:
/// [`Removed`](Frame::Removed) at the path it left and
/// [`Inserted`](Frame::Inserted) at the path it arrived at, the
/// inserted node complete, a directory with its whole subtree. The
/// pairing a filesystem offers for the two halves is not reliable
/// enough to promise a consumer, and the tree is right without it;
/// what a consumer loses is only the knowledge that the two were one
/// node.
///
/// Every `path` in every variant is a component vector relative to the
/// filetree root — one meaning of "path" throughout, matching
/// [`Node::Symlink`]'s.
///
/// # Variant ORDER is part of the wire format
///
/// Serialized in serde's default representation, which writes the
/// variant's INDEX rather than its name — the same arrangement
/// [`Node`] uses, and for the same reason: an index needs no lookahead
/// to read, which is what makes it encodable in a format with no
/// self-description.
///
/// It is also a constraint. Reordering these variants, or inserting
/// one among them, silently changes what existing bytes mean. New
/// variants go on the END; nowhere else is a compatible change.
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
pub enum Frame {
/// The whole tree: the root's entries, recursively.
///
/// The first frame of every stream, and the only frame that may
/// come again: a source that lost track of the tree — a watch
/// whose event queue overflowed — sends a fresh one rather than
/// deltas it cannot know. Each replaces the tree whole.
Snapshot {
/// The root's entries. The root itself is not among them and
/// is not described — see [`Root`](super::Root).
children: Vec<Node>,
},
/// A node came into existence at a path that held nothing.
///
/// Also how a node arrives by rename, from elsewhere in the tree
/// or from outside it: from this path's point of view nothing was
/// relocated, something appeared.
Inserted {
/// Where the node appeared. The last element equals `node`'s
/// `name`.
///
/// Components rather than a joined string: a path is a
/// sequence, and joining it would invent a separator that then
/// has to be escaped out of names that contain it.
path: Vec<String>,
/// The node's complete value. A directory carries its whole
/// subtree.
node: Node,
},
/// A node that already existed changed, staying where it was.
Modified {
/// The node's path, unchanged by this frame. The last element
/// equals `node`'s `name`.
path: Vec<String>,
/// The node's complete new value, replacing the old one. A
/// directory carries its whole subtree, so this replaces rather
/// than merges.
node: Node,
},
/// A node ceased to exist. A directory takes its whole subtree with
/// it — no per-descendant removals follow.
///
/// Also how a node leaves by rename — to elsewhere in the tree,
/// where an [`Inserted`](Frame::Inserted) reports its arrival, or
/// out of it.
Removed {
/// The vanished node's path.
path: Vec<String>,
},
}
/// Postcard, where the rest of the crate is JSON.
///
/// A filetree stream is the one thing here that is both high-volume
/// and free to choose: it relays nothing, so no byte of it has to
/// survive a round trip unchanged, and nothing downstream reads it as
/// text. What it is instead is spammy — one frame per changed node,
/// indefinitely — so the envelope is worth minimizing.
///
/// Postcard drops field names entirely and varint-encodes every length
/// and integer, which puts a small delta within a couple of bytes of
/// the information it actually carries. `Removed` naming
/// `src/main.rs` is fourteen bytes, ten of them the two strings
/// themselves.
impl Encode for Frame {
/// Postcard's own failure. It has few ways to happen when writing
/// — a buffer that will not take bytes, mostly — since everything
/// here is a shape it can always represent.
type Error = postcard::Error;
fn encode(&self, out: &mut Writer<'_>) -> Result<(), Self::Error> {
postcard::to_io(self, &mut *out)?;
Ok(())
}
}
impl Decode<'_> for Frame {
/// Postcard's own failure.
type Error = postcard::Error;
fn decode(bytes: &[u8]) -> Result<Self, Self::Error> {
postcard::from_bytes(bytes)
}
}Its nodes are defined by diverge-provider-sdk/src/shared/filetree/response/node.rs:
//! One node of the filesystem tree.
use serde::{Deserialize, Serialize};
/// One node of the filesystem tree.
///
/// A [`Directory`](Node::Directory) carries its children inline;
/// [`File`](Node::File) and [`Symlink`](Node::Symlink) are leaves. A
/// symlink is the link ITSELF and is never followed, so a dangling or
/// looping link is a leaf rather than an error or an infinite tree.
///
/// Every variant carries `name` — the basename, never a path — plus
/// `created_at` and `modified_at`. A directory alone carries
/// `changes`: whether what happens beneath it will be reported, which
/// is a question only a directory can answer, because only a
/// directory is watched — see the field.
///
/// # Times are unsigned seconds
///
/// The same representation
/// [`Volume::created`](crate::endpoints::volumes::list::server::response::Volume::created)
/// uses, and unsigned for the same reason: nothing a provider offers
/// predates 1970, and a signed field's negative half would exist to
/// represent a state that never occurs. `Option` here is about
/// availability rather than sign — a filesystem that records no birth
/// time has nothing to report, which is not the same as reporting a
/// time before the epoch.
///
/// # Variant ORDER is part of the wire format
///
/// This enum is serialized in serde's default representation, which
/// writes the variant's INDEX rather than its name. That is what makes
/// it encodable at all in a format with no self-description — there is
/// no name to look up and no lookahead to do — and it is also a
/// constraint: reordering these variants, or inserting one among them,
/// silently changes what existing bytes mean.
///
/// New variants go on the END. Nowhere else is a compatible change.
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
pub enum Node {
/// A regular file.
File {
/// Basename of this file.
name: String,
/// Size in bytes. `None` when the stat could not be read.
size: Option<u64>,
/// Creation time (unix seconds), when the filesystem records a
/// birth time. `None` where unsupported — this is display
/// metadata and is never load-bearing.
created_at: Option<u64>,
/// Last-modified time (unix seconds). `None` when the stat
/// could not be read.
modified_at: Option<u64>,
},
/// A directory, carrying its entries.
Directory {
/// Basename of this directory. The watched root has no node of
/// its own — see [`Root`](super::Root).
name: String,
/// Creation time (unix seconds), when the filesystem records a
/// birth time.
created_at: Option<u64>,
/// Last-modified time (unix seconds). A directory's mtime
/// tracks entry add/remove, not changes within its children.
modified_at: Option<u64>,
/// Whether changes beneath this directory stream.
///
/// `true` is the ordinary case: what happens under it arrives
/// as deltas. `false` is a directory the source could not
/// watch — a watch limit reached, a corner it was refused —
/// and with it everything beneath, files included: what is
/// here is what the walk found, nothing under it will be
/// reported until the next [`Snapshot`](super::Frame::Snapshot),
/// and a consumer should treat it as possibly stale. Only a
/// directory carries this, because only a directory is
/// watched: a file's changes are its parent's to report, so a
/// file cannot fail to be watched on its own.
changes: bool,
/// This directory's entries. An empty directory carries an
/// empty list — this field is never absent, so a consumer never
/// has to distinguish "no children" from "children unknown".
children: Vec<Node>,
},
/// A symbolic link — the link itself, never its target.
Symlink {
/// Basename of this link.
name: String,
/// The link's target, as path components ALWAYS RELATIVE TO
/// THE FILETREE ROOT — the same frame of reference as the
/// `path` carried by [`Frame::Inserted`](super::Frame::Inserted),
/// [`Frame::Modified`](super::Frame::Modified) and
/// [`Frame::Removed`](super::Frame::Removed). Every path in
/// this API means the same thing, so a consumer walks a link's
/// target down from the snapshot's child list exactly as it
/// walks a frame's path, with no separate rule for links.
///
/// Addressable is not the same as resolved: the link is still
/// never followed, and the components may name a node that
/// does not exist — an ordinary dangling link.
///
/// Always present. A link whose contents could not be read is
/// not a symlink node with a missing target; it is a failure,
/// and is reported as one. That is unrelated to dangling: a
/// link pointing at nothing still reports its components
/// perfectly well, because reading a link never touches its
/// target.
path: Vec<String>,
/// Creation time (unix seconds), when the filesystem records a
/// birth time.
created_at: Option<u64>,
/// Last-modified time (unix seconds).
modified_at: Option<u64>,
},
}
// Visible throughout `response` — but no wider. These exist so
// [`Root`](super::Root)'s fold can walk and edit a tree; they are not
// part of the specification and nothing outside this module should
// depend on them.
impl Node {
/// This node's basename.
pub(super) fn name(&self) -> &str {
match self {
Node::File { name, .. }
| Node::Directory { name, .. }
| Node::Symlink { name, .. } => name,
}
}
/// This node's entries, mutably — `None` for anything but a
/// directory.
pub(super) fn children_mut(&mut self) -> Option<&mut Vec<Node>> {
match self {
Node::Directory { children, .. } => Some(children),
_ => None,
}
}
}