PhoenixSwarm user documentation

Pierre Aubert


This is the user documentation. The developer documentation is here. The page of the project is here.

The project PhoenixSwarm aims to easily transform programs into daemons which can run on distributed environments such as Kubernetes, Slurm or HT Condor. More specifically, PhoenixSwarm provides:

  • A daemon model: any program can be turned into a Daemon --- an autonomous process with its own address and port --- able to send and receive typed data to and from other daemons of the swarm.
  • A data-push communication pattern: each daemon pushes data directly to the desired receiver. A daemon can handle as many data types as needed; an internal scheduler automatically dispatches incoming data to the appropriate processing method based on the data type name.
  • Flexible backends: the communication layer (socket) and the clock are abstracted behind a backend system, allowing developers to choose the most suitable technology for their environment (Unix Socket, ZMQ, NetMQ, RabbitMQ, etc.). Three operating modes are supported:
    • Normal mode (NO_MOCK): uses the official production backend.
    • Mock mode (MOCK): replays a recorded sequence of data, enabling unit tests without network dependency or latency.
    • Record mode (MOCK_RECORD): runs the official backend while recording all exchanges for future replay.
  • A statistics system: each daemon automatically accumulates computing and communication statistics (event count, min, max, average, rate, latency histograms). These are periodically sent to a dedicated StatDaemonManager, which centralises the statistics of the entire swarm.
  • Configurable via TOML/YAML/JSON: the whole swarm topology (daemon addresses, ports, timeouts, send/receive flags, statistics parameters) is described in a single configuration file, making deployment on distributed schedulers straightforward.
  • A testable architecture: the mock backend allows full unit testing of daemon workflows --- including data exchange sequences and timing --- without requiring a real network infrastructure.