// Moved from docs/src/architecture-internals/MESSAGING.adoc to docs/working/scratch/src/architecture-internals/MESSAGING.adoc on 2026-06-16 (docs reorg Phase 2) == 1. Overview :toc: left :toc-title: Contents :toclevels: 3 xref:../README.adoc[← Back to Documentation Index] This document defines the *thermodynamic model for the StarshipOS Messaging Field*, a formal framework describing the behavior of message propagation and processing as a physical system. The messaging subsystem is not treated as a passive bus. Instead, it is conceptualized as a *field* in which discrete *messages* behave analogously to quanta (photons), and *words* (the fundamental executable units of the StarshipOS runtime) behave as *particles* or loci of interaction. This model provides a unified basis for: * *Prioritization & Scheduling* * *Credit Allocation and Flow Control* * *Thermal Regulation of System Load* * *Entropy-Driven Policy and ML Feedback* * *Instrumentation of Message Dynamics* ''''' == 2. Conceptual Model === 2.1 Message Field The *Message Field* is the logical substrate mediating all communication within StarshipOS. It consists of: * *Endpoints*: Boundaries between word spaces and the field. * *Channels*: Directed pathways through which messages propagate (virtual channels, pub/sub topics, RPC routes). * *Field Properties*: Dynamic parameters such as message density, propagation latency, and entropy. The field is *not centralized*. It emerges from the coordinated behavior of `+sf_msg-srv+` (broker/router), `+sf_msg-drv+` (ABI provider), and `+libsfmsg+` (participant library), operating over shared memory rings and control IPC. ''''' === 2.2 Messages Messages are modeled as *energy quanta* with the following core properties: [cols=",",options="header",] |=== |Property |Description |Energy |Encodes priority, TTL, QoS, and thermodynamic state |Momentum |Routing vector through the field |Cross Section |Probability of absorption by a word |Entropy |Thermodynamic measure of the flow’s state space |=== Messages can be *absorbed*, *emitted*, or *scattered* by words and services, analogous to photon–atom interactions. ''''' === 2.3 Words as Particles Each *word* is treated as a *particle* in the field — a locus of interaction rather than an isolated routine. Words have intrinsic properties: [cols=",",options="header",] |=== |Property |Meaning |Mass |Computational cost or complexity |Charge |Degree of side-effect or state mutation |Cross-Section |Message types and classes to which the word responds |=== When a message reaches a word, the interaction may *excite* the word (trigger execution), *alter its state*, or __ __pass through unabsorbed**. Executed words may **emit new messages**, either deterministically or in a burst (stimulated emission). ''''' == 3. Mathematical Formulation === 3.1 Maxwellian Entropy The system adopts a *Maxwell–Boltzmann entropy model*, treating each endpoint/channel as a thermodynamic micro-system. For a given flow ( f ): {empty}[ S_f(t) = k ( 1 + )] Where: * ( ^2_\{t}(f) ): Variance of inter-arrival intervals * ( Q_\{}(f) ): Normalized queue occupancy (0–1) * ( U_\{}(f) ): Ratio of credits consumed to granted * ( R_\{}(f) ): Number of delivery paths (fan-out) * ( T_\{}(f) ): Normalized TTL of messages in the flow * ( k ): Scaling constant (typically 1) ( S_f ) represents the *thermodynamic entropy* of the flow. High entropy indicates unpredictable, high-energy flows approaching saturation. Low entropy indicates stable, structured behavior. === 3.2 System Temperature and Pressure * *Temperature (Θ)* is the average energy of messages within the field, weighted by density. * *Pressure (Π)* represents backlog intensity: [ = _f Q_\{}(f) U_\{}(f)] * *Entropy (Σ)* is the aggregate system entropy: [ = _f S_f] These quantities provide real-time metrics for system health and are suitable as inputs for control algorithms. ''''' == 4. Header and Data Structure Extensions Messages carry *thermodynamic metadata* in addition to conventional routing and control information. === 4.1 Thermodynamic Header Extension [source,c] ---- struct sfm_hdr_thermo { uint16_t energy_q8; // Encoded priority/TTL/entropy composite (fixed point) uint16_t entropy_mx; // Maxwellian entropy (Q8.8 fixed-point) uint16_t temp_hint; // Optional producer-side temperature hint uint16_t reserved; }; ---- * `+energy_q8+` is a composite scalar derived from QoS class, TTL, and policy weighting. * `+entropy_mx+` is computed at the sender or by the broker to characterize flow state. * `+temp_hint+` allows producers to indicate expected burstiness or instability. === 4.2 Endpoint State Endpoints maintain thermodynamic state variables: [source,c] ---- struct sfm_endpoint_state { float entropy_current; float temperature; float pressure; float credit_utilization; uint64_t last_refresh_tsc; }; ---- This state is updated continuously during message transfer and scheduling. ''''' == 5. Scheduler and Credit System Integration === 5.1 Credits as Energy Quanta Credits represent the *available energy budget* for message emission along a channel. * *High-entropy flows* receive smaller, more frequent credit refreshes (tight regulation). * *Low-entropy flows* receive larger, batched credits (looser regulation). Credit recycling policies (immediate, piggyback, or batched) are selected dynamically based on the flow’s entropy and temperature. ''''' === 5.2 Thermodynamic Scheduling Scheduling priority is a function of QoS class, deadline, and entropy: {empty}[ P_\{sched} = f(, , S_f)] * Low-entropy, real-time flows are scheduled deterministically. * High-entropy, bulk flows are throttled or coalesced. * Aging and anti-starvation mechanisms apply within entropy bands. This approach prevents high-entropy flows from destabilizing the system, while ensuring low-entropy flows achieve predictable latency. ''''' == 6. Thermodynamic Monitoring and Control Loop A *control loop* runs in `+sf_msg-srv+`, maintaining field stability: [arabic] . *Sampling* — Entropy, temperature, and pressure are sampled periodically from endpoints and channels. . *Aggregation* — System-wide Σ, Θ, Π are computed. . *Policy Evaluation* — Control laws or ML bandits adjust credit windows, scheduling weights, and routing based on sampled state. . *Actuation* — The scheduler and credit allocator apply the updated parameters. This loop can operate using fixed thresholds or adaptive policies. ML components treat entropy as the *order parameter * for optimization. ''''' == 7. Implications for Forth and StarshipOS Runtime In StarshipOS, *words are the fundamental execution loci*. By modeling them as particles interacting via message photons, the runtime gains: * *A unified abstraction for computation and communication* * *Fine-grained control over execution dynamics through entropy* * *A natural basis for prioritization without ad-hoc heuristics* * *Thermodynamic instrumentation for debugging and analysis* This model integrates cleanly with the existing Forth execution model, since words are already explicit entities with well-defined entry points. ''''' == 8. Future Work [arabic] . *Formal Policy Language* Define a compact language for specifying entropy-based routing and scheduling policies. . *Distributed Thermodynamic Fields* Extend the model to multi-node topologies, where message photons propagate across network transports. . *Entropy-Driven GC & Memory Tiering* Couple message entropy metrics with VM memory placement decisions. . *Visualization* Develop tooling to visualize message flow as a dynamic field, aiding in debugging and optimization. ''''' == 9. APPEXDIX .... /* * StarshipOS - Messaging Thermodynamics * File: include/sfm_thermo.h * Status: Draft / Reference Only * * This header defines the public structures and APIs for the * Maxwellian entropy model applied to StarshipOS messaging flows. * It is intended as a design reference, not production code. * * Language: C99 * License: CC0 1.0 / Public Domain */ #ifndef SFM_THERMO_H #define SFM_THERMO_H #include #include #ifdef __cplusplus extern "C" { #endif /* ============================================================= * Fixed-point helpers (Q8.8 and Q16.16 formats) * =========================================================== */ /** Convert float → Q8.8 fixed point. */ static inline uint16_t sfm_fp_to_q8_8(float x) { if (x <= 0.0f) return 0u; float v = x * 256.0f; if (v > 65535.0f) v = 65535.0f; return (uint16_t)(v + 0.5f); } /** Convert Q8.8 fixed point → float. */ static inline float sfm_q8_8_to_fp(uint16_t q) { return ((float)q) / 256.0f; } /** Convert float → Q16.16 fixed point. */ static inline uint32_t sfm_fp_to_q16_16(float x) { if (x <= 0.0f) return 0u; double v = (double)x * 65536.0; if (v > 4294967295.0) v = 4294967295.0; return (uint32_t)(v + 0.5); } /** Convert Q16.16 fixed point → float. */ static inline float sfm_q16_16_to_fp(uint32_t q) { return ((float)q) / 65536.0f; } /* ============================================================= * Thermodynamic Header Extension * =========================================================== */ /** * Additional header fields attached to each message. * These encode thermodynamic properties derived from the * flow's state and scheduling context. * * All fields are fixed-point Q8.8 unless otherwise stated. */ struct sfm_hdr_thermo { uint16_t energy_q8; /**< Composite of priority, TTL, entropy. */ uint16_t entropy_mx; /**< Maxwellian entropy estimate (per-flow). */ uint16_t temp_hint; /**< Optional producer hint for burstiness. */ uint16_t reserved; /**< Align to 8 bytes. */ }; /* ============================================================= * Per-flow Thermodynamic State * =========================================================== */ /** * Credit refresh policy hints returned to allocator. */ typedef enum { SFM_CREDIT_IMMEDIATE = 0, /**< Refresh credits on every recycle. */ SFM_CREDIT_PIGGYBACK = 1, /**< Return credits with next outbound. */ SFM_CREDIT_BATCHED = 2 /**< Batch refresh until watermark. */ } sfm_credit_mode_t; /** * Per-flow thermodynamic state, maintained by sf_msg-srv. * All quantities are continuously updated during operation. */ typedef struct { /* --- Inter-arrival statistics (Welford) --- */ uint64_t last_arrival_us; double iat_mean; double iat_M2; uint64_t iat_count; /* --- Queue occupancy EWMA --- */ double q_occ_ewma; double q_alpha; /* --- Credit utilization EWMA --- */ uint64_t credits_granted_total; uint64_t credits_consumed_total; double credit_util_ewma; double credit_alpha; /* --- Fanout & TTL tracking --- */ double fanout_ewma; double ttl_ms_ewma; /* --- Derived thermodynamic quantities --- */ double entropy_mx; /**< Maxwellian entropy S_f */ double temperature; /**< Average energy proxy Θ */ double pressure; /**< Backlog pressure proxy Π */ /* --- Policy thresholds --- */ double wm_low; double wm_high; double entropy_cool; double entropy_hot; /* --- Last update timestamp --- */ uint64_t last_update_us; } sfm_flow_state_t; /** * Snapshot of field-wide thermodynamic state. * Useful for monitoring and control loops. */ typedef struct { double system_entropy_sum; double system_temperature_sum; double system_pressure_sum; uint64_t flows_count; } sfm_field_snapshot_t; /** * Credit allocation decision returned to the scheduler. */ typedef struct { sfm_credit_mode_t mode; /**< Suggested refresh mode. */ uint32_t grant_quanta; /**< Number of descriptors to grant. */ uint8_t qos_hint; /**< Optional QoS class hint (255 = no change). */ } sfm_thermo_decision_t; /* ============================================================= * API Prototypes * =========================================================== */ /** * Initialize a flow state structure with sane defaults. * * @param st Pointer to the state structure. * @param q_alpha EWMA alpha for queue occupancy. * @param credit_alpha EWMA alpha for credit utilization. * @param wm_low Low watermark for batching (0–1). * @param wm_high High watermark for backpressure (0–1). * @param entropy_cool Entropy threshold for loosening control. * @param entropy_hot Entropy threshold for tightening control. */ void sfm_thermo_init_flow(sfm_flow_state_t *st, double q_alpha, double credit_alpha, double wm_low, double wm_high, double entropy_cool, double entropy_hot); /** * Update flow state when a message is enqueued or processed. * * @param st Flow state. * @param now_us Current monotonic timestamp in microseconds. * @param q_norm Current normalized queue occupancy [0..1]. * @param credits_g Delta of credits granted since last update. * @param credits_c Delta of credits consumed since last update. * @param fanout Observed fanout for this message. * @param ttl_ms Message TTL in milliseconds. */ void sfm_thermo_on_enq(sfm_flow_state_t *st, uint64_t now_us, double q_norm, uint32_t credits_g, uint32_t credits_c, double fanout, double ttl_ms); /** * Sample the current state and compute a scheduling / credit * allocation decision. This is invoked periodically or on events. * * @param st Flow state. * @param batch_watermark Recycle count watermark for batching. * @param max_grant Safety cap on credit grants. * @return Suggested decision structure. */ sfm_thermo_decision_t sfm_thermo_sample_and_decide(sfm_flow_state_t *st, uint32_t batch_watermark, uint32_t max_grant); /** * Aggregate a set of flow states into a single field snapshot. * * @param flows Array of flow states. * @param n Number of elements in `flows`. * @param out Output snapshot structure. */ void sfm_thermo_aggregate(const sfm_flow_state_t *flows, size_t n, sfm_field_snapshot_t *out); /** * Populate a thermodynamic header extension for a message. * * @param st Flow state. * @param h Pointer to header extension struct. * @param energy_q8 Precomputed composite energy (Q8.8). */ static inline void sfm_thermo_fill_hdr(const sfm_flow_state_t *st, struct sfm_hdr_thermo *h, uint16_t energy_q8) { if (!st || !h) return; h->energy_q8 = energy_q8; h->entropy_mx = sfm_fp_to_q8_8((float)st->entropy_mx); h->temp_hint = sfm_fp_to_q8_8((float)st->temperature); h->reserved = 0; } #ifdef __cplusplus } #endif #endif /* SFM_THERMO_H */ .... == 10. References * Maxwell, J.C., _Illustrations of the Dynamical Theory of Gases_, Phil. Mag. 1860. * Boltzmann, L., _Weitere Studien über das Wärmegleichgewicht unter Gasmolekülen_, 1872. * StarshipOS Internal Messaging Architecture Specifications. ''''' *End of Document* '''''