# /filesystem/tree > The filesystem of the container, watched: the server sends exactly one message naming the paths to leave out; the proxy sends a snapshot, one message per change, a snapshot again after lost events, or one error as its last message. Canonical: https://provider.diverge.network/2.3.0/proxy/filesystem/tree/ Specification revision: 2.3.0 The server opens `/filesystem/tree` and sends exactly one message, the request defined below as JSON, naming the paths the tree leaves out. The server sends no further message; a further binary message from the server ends the connection. The proxy sends one message per event for as long as the path lives. ```text server → proxy: [request JSON] once proxy → server: [0][postcard-encoded filetree frame …] per event [1][message …] last ``` - **What the tree leaves out.** The tree leaves out every path the request names, each given as components from the root of the container, and the paths `/proc`, `/sys` and `/dev`. A path left out is absent from every snapshot, and the proxy reports no change under it. The proxy disregards an empty path in the request. - **Not served.** A close before the request, and a request that does not decode, cause the proxy to close the connection with no message before the close. - **A snapshot first.** The first message on every opening is a snapshot of the whole tree, with first byte `0`. Every later message with first byte `0` is one change, sent as it occurs. - **A snapshot again.** A watch that has lost events sends a fresh snapshot on the same path, and the fresh snapshot replaces the tree whole. - **One error, last.** A message with first byte `1`, followed by a message, states that the watch could not exist. It is the last message the proxy sends before the close. A part of the tree that the proxy could not watch is not an error; the tree itself reports it, with the `changes` field of its directory set to `false`. - **Every opening starts whole.** No opening resumes a previous one. An opening after an ending, clean or abrupt, begins with a snapshot. - **The encoding.** After the first byte `0`, the message is the filetree response frame defined below, encoded in the [postcard wire format](https://postcard.jamesmunns.com/wire-format). The order of variants is the discriminant on the wire. The message is defined by `diverge-provider-sdk/src/container_proxy/filesystem/tree/request/request.rs`: ```rust //! What the tree leaves out. use serde::{Deserialize, Serialize}; use crate::decode::Decode; use crate::encode::{Encode, Writer}; /// The first message on `/filesystem/tree`, from the server: the /// paths the tree does not contain. /// /// The server names the MOUNTS here — every one it placed in the /// container, volume and identity and FUSE alike, whose watch would /// cost the walk and report what the caller already holds. `/proc`, /// `/sys` and `/dev` are the proxy's own and are never listed. Each /// path is components from the container's root, the shape every /// path in this crate takes; an empty one is dropped rather than read /// as the root. An ignored path does not exist as far as the stream /// is concerned: absent from the snapshot, never watched, an event /// under it dropped. #[derive(Debug, Clone, PartialEq, Eq, Hash, Serialize, Deserialize, Default)] pub struct Request { /// The paths to leave out, each as components from the root. pub ignore: Vec>, } impl Encode for Request { /// The ordinary JSON failure. type Error = serde_json::Error; fn encode(&self, out: &mut Writer<'_>) -> Result<(), Self::Error> { serde_json::to_writer(out, self) } } impl Decode<'_> for Request { /// The ordinary JSON failure. type Error = serde_json::Error; fn decode(bytes: &[u8]) -> Result { serde_json::from_slice(bytes) } } ``` The messages the proxy sends are defined by `diverge-provider-sdk/src/container_proxy/filesystem/tree/response/frame.rs`: ```rust //! One event of the tree, or why there will be no more. use super::FrameError; use crate::decode::Decode as _; use crate::encode::{Encode, Writer}; use crate::shared::filetree; /// One message on `/filesystem/tree`. /// /// ```text /// [kind: u8][postcard frame… | message…] /// ``` /// /// The first on a connection is a /// [`Snapshot`](filetree::response::Frame::Snapshot) behind kind `0`; /// every one after is a delta, or a snapshot again when the watch /// lost events; and an [`Error`](Self::Error) is the last, when the /// watch could not exist at all. #[derive(Debug, Clone, PartialEq)] pub enum Frame<'a> { /// Kind `0`. One filetree event, as /// [`shared::filetree`](crate::shared::filetree) encodes it. Filetree(filetree::response::Frame), /// Kind `1`. The watch could not be made, the root could not be /// watched, or the walk died — and this says why, for a reader /// rather than a program. The last message before the close; /// nothing follows it, and the server starts over when it likes. Error(&'a str), } impl Encode for Frame<'_> { /// postcard's own failure, from the half that has one; a kind /// byte and a message's bytes cannot fail. type Error = postcard::Error; fn encode(&self, out: &mut Writer<'_>) -> Result<(), postcard::Error> { match self { Frame::Filetree(frame) => { out.extend_from_slice(&[0]); frame.encode(out) } Frame::Error(message) => { out.extend_from_slice(&[1]); out.extend_from_slice(message.as_bytes()); Ok(()) } } } } impl<'a> Frame<'a> { /// Decode one message. The error message borrows from `bytes`. pub fn decode(bytes: &'a [u8]) -> Result { let (kind, rest) = bytes.split_first().ok_or(FrameError::Empty)?; match *kind { 0 => filetree::response::Frame::decode(rest) .map(Frame::Filetree) .map_err(FrameError::Filetree), 1 => std::str::from_utf8(rest) .map(Frame::Error) .map_err(|_| FrameError::MessageUtf8), other => Err(FrameError::UnknownKind(other)), } } } ``` The ways in which such a message fails to decode are defined by `diverge-provider-sdk/src/container_proxy/filesystem/tree/response/error.rs`: ```rust //! Why a message on `/filesystem/tree` could not be decoded. use std::error; use std::fmt; /// A filetree message that could not be read. #[derive(Debug)] pub enum FrameError { /// No bytes at all, so not even a kind. Empty, /// A kind this answer does not define. UnknownKind(u8), /// The postcard payload would not decode as a filetree frame. Filetree(postcard::Error), /// An error message that is not UTF-8. MessageUtf8, } impl fmt::Display for FrameError { fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { match self { FrameError::Empty => f.write_str("filetree message is empty"), FrameError::UnknownKind(kind) => { write!(f, "unknown filetree message kind {kind}") } FrameError::Filetree(error) => { write!(f, "filetree frame did not decode: {error}") } FrameError::MessageUtf8 => { f.write_str("filetree error message is not utf-8") } } } } impl error::Error for FrameError { fn source(&self) -> Option<&(dyn error::Error + 'static)> { match self { FrameError::Filetree(error) => Some(error), FrameError::Empty | FrameError::UnknownKind(_) | FrameError::MessageUtf8 => None, } } } ``` The filetree frame after the first byte `0` is defined by `diverge-provider-sdk/src/shared/filetree/response/frame.rs`: ```rust //! What a filetree response frame carries. use serde::{Deserialize, Serialize}; use super::Node; use crate::decode::Decode; use crate::encode::{Encode, Writer}; /// One change on a filetree stream. /// /// [`Snapshot`](Frame::Snapshot) establishes the tree. Every other /// variant names exactly one node and says what became of it: it /// appeared ([`Inserted`](Frame::Inserted)), changed in place /// ([`Modified`](Frame::Modified)), or ceased to exist /// ([`Removed`](Frame::Removed)). /// /// Insertion and modification are distinguished because a consumer /// usually wants to treat them differently — a newly appeared file is /// not the same news as an existing one being written to. The /// distinction is INFORMATION, not a constraint: a consumer with no /// use for it may treat the two identically, and doing so is what /// keeps the fold tolerant of replay and reordering. /// /// A delta that carries a node carries its COMPLETE value, never a /// patch against a value the consumer is assumed to hold. That is what /// makes replaying an already-applied frame harmless, and therefore /// what makes at-least-once delivery safe: every variant overwrites /// or clears one place in the tree, and none reads the tree first. /// /// # A rename is two frames /// /// There is no move. A node renamed — within one directory or across /// the tree — is reported as what happened on disk: /// [`Removed`](Frame::Removed) at the path it left and /// [`Inserted`](Frame::Inserted) at the path it arrived at, the /// inserted node complete, a directory with its whole subtree. The /// pairing a filesystem offers for the two halves is not reliable /// enough to promise a consumer, and the tree is right without it; /// what a consumer loses is only the knowledge that the two were one /// node. /// /// Every `path` in every variant is a component vector relative to the /// filetree root — one meaning of "path" throughout, matching /// [`Node::Symlink`]'s. /// /// # Variant ORDER is part of the wire format /// /// Serialized in serde's default representation, which writes the /// variant's INDEX rather than its name — the same arrangement /// [`Node`] uses, and for the same reason: an index needs no lookahead /// to read, which is what makes it encodable in a format with no /// self-description. /// /// It is also a constraint. Reordering these variants, or inserting /// one among them, silently changes what existing bytes mean. New /// variants go on the END; nowhere else is a compatible change. #[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)] pub enum Frame { /// The whole tree: the root's entries, recursively. /// /// The first frame of every stream, and the only frame that may /// come again: a source that lost track of the tree — a watch /// whose event queue overflowed — sends a fresh one rather than /// deltas it cannot know. Each replaces the tree whole. Snapshot { /// The root's entries. The root itself is not among them and /// is not described — see [`Root`](super::Root). children: Vec, }, /// A node came into existence at a path that held nothing. /// /// Also how a node arrives by rename, from elsewhere in the tree /// or from outside it: from this path's point of view nothing was /// relocated, something appeared. Inserted { /// Where the node appeared. The last element equals `node`'s /// `name`. /// /// Components rather than a joined string: a path is a /// sequence, and joining it would invent a separator that then /// has to be escaped out of names that contain it. path: Vec, /// The node's complete value. A directory carries its whole /// subtree. node: Node, }, /// A node that already existed changed, staying where it was. Modified { /// The node's path, unchanged by this frame. The last element /// equals `node`'s `name`. path: Vec, /// The node's complete new value, replacing the old one. A /// directory carries its whole subtree, so this replaces rather /// than merges. node: Node, }, /// A node ceased to exist. A directory takes its whole subtree with /// it — no per-descendant removals follow. /// /// Also how a node leaves by rename — to elsewhere in the tree, /// where an [`Inserted`](Frame::Inserted) reports its arrival, or /// out of it. Removed { /// The vanished node's path. path: Vec, }, } /// Postcard, where the rest of the crate is JSON. /// /// A filetree stream is the one thing here that is both high-volume /// and free to choose: it relays nothing, so no byte of it has to /// survive a round trip unchanged, and nothing downstream reads it as /// text. What it is instead is spammy — one frame per changed node, /// indefinitely — so the envelope is worth minimizing. /// /// Postcard drops field names entirely and varint-encodes every length /// and integer, which puts a small delta within a couple of bytes of /// the information it actually carries. `Removed` naming /// `src/main.rs` is fourteen bytes, ten of them the two strings /// themselves. impl Encode for Frame { /// Postcard's own failure. It has few ways to happen when writing /// — a buffer that will not take bytes, mostly — since everything /// here is a shape it can always represent. type Error = postcard::Error; fn encode(&self, out: &mut Writer<'_>) -> Result<(), Self::Error> { postcard::to_io(self, &mut *out)?; Ok(()) } } impl Decode<'_> for Frame { /// Postcard's own failure. type Error = postcard::Error; fn decode(bytes: &[u8]) -> Result { postcard::from_bytes(bytes) } } ``` Its nodes are defined by `diverge-provider-sdk/src/shared/filetree/response/node.rs`: ```rust //! One node of the filesystem tree. use serde::{Deserialize, Serialize}; /// One node of the filesystem tree. /// /// A [`Directory`](Node::Directory) carries its children inline; /// [`File`](Node::File) and [`Symlink`](Node::Symlink) are leaves. A /// symlink is the link ITSELF and is never followed, so a dangling or /// looping link is a leaf rather than an error or an infinite tree. /// /// Every variant carries `name` — the basename, never a path — plus /// `created_at` and `modified_at`. A directory alone carries /// `changes`: whether what happens beneath it will be reported, which /// is a question only a directory can answer, because only a /// directory is watched — see the field. /// /// # Times are unsigned seconds /// /// The same representation /// [`Volume::created`](crate::endpoints::volumes::list::server::response::Volume::created) /// uses, and unsigned for the same reason: nothing a provider offers /// predates 1970, and a signed field's negative half would exist to /// represent a state that never occurs. `Option` here is about /// availability rather than sign — a filesystem that records no birth /// time has nothing to report, which is not the same as reporting a /// time before the epoch. /// /// # Variant ORDER is part of the wire format /// /// This enum is serialized in serde's default representation, which /// writes the variant's INDEX rather than its name. That is what makes /// it encodable at all in a format with no self-description — there is /// no name to look up and no lookahead to do — and it is also a /// constraint: reordering these variants, or inserting one among them, /// silently changes what existing bytes mean. /// /// New variants go on the END. Nowhere else is a compatible change. #[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)] pub enum Node { /// A regular file. File { /// Basename of this file. name: String, /// Size in bytes. `None` when the stat could not be read. size: Option, /// Creation time (unix seconds), when the filesystem records a /// birth time. `None` where unsupported — this is display /// metadata and is never load-bearing. created_at: Option, /// Last-modified time (unix seconds). `None` when the stat /// could not be read. modified_at: Option, }, /// A directory, carrying its entries. Directory { /// Basename of this directory. The watched root has no node of /// its own — see [`Root`](super::Root). name: String, /// Creation time (unix seconds), when the filesystem records a /// birth time. created_at: Option, /// Last-modified time (unix seconds). A directory's mtime /// tracks entry add/remove, not changes within its children. modified_at: Option, /// Whether changes beneath this directory stream. /// /// `true` is the ordinary case: what happens under it arrives /// as deltas. `false` is a directory the source could not /// watch — a watch limit reached, a corner it was refused — /// and with it everything beneath, files included: what is /// here is what the walk found, nothing under it will be /// reported until the next [`Snapshot`](super::Frame::Snapshot), /// and a consumer should treat it as possibly stale. Only a /// directory carries this, because only a directory is /// watched: a file's changes are its parent's to report, so a /// file cannot fail to be watched on its own. changes: bool, /// This directory's entries. An empty directory carries an /// empty list — this field is never absent, so a consumer never /// has to distinguish "no children" from "children unknown". children: Vec, }, /// A symbolic link — the link itself, never its target. Symlink { /// Basename of this link. name: String, /// The link's target, as path components ALWAYS RELATIVE TO /// THE FILETREE ROOT — the same frame of reference as the /// `path` carried by [`Frame::Inserted`](super::Frame::Inserted), /// [`Frame::Modified`](super::Frame::Modified) and /// [`Frame::Removed`](super::Frame::Removed). Every path in /// this API means the same thing, so a consumer walks a link's /// target down from the snapshot's child list exactly as it /// walks a frame's path, with no separate rule for links. /// /// Addressable is not the same as resolved: the link is still /// never followed, and the components may name a node that /// does not exist — an ordinary dangling link. /// /// Always present. A link whose contents could not be read is /// not a symlink node with a missing target; it is a failure, /// and is reported as one. That is unrelated to dangling: a /// link pointing at nothing still reports its components /// perfectly well, because reading a link never touches its /// target. path: Vec, /// Creation time (unix seconds), when the filesystem records a /// birth time. created_at: Option, /// Last-modified time (unix seconds). modified_at: Option, }, } // Visible throughout `response` — but no wider. These exist so // [`Root`](super::Root)'s fold can walk and edit a tree; they are not // part of the specification and nothing outside this module should // depend on them. impl Node { /// This node's basename. pub(super) fn name(&self) -> &str { match self { Node::File { name, .. } | Node::Directory { name, .. } | Node::Symlink { name, .. } => name, } } /// This node's entries, mutably — `None` for anything but a /// directory. pub(super) fn children_mut(&mut self) -> Option<&mut Vec> { match self { Node::Directory { children, .. } => Some(children), _ => None, } } } ```