Can /diagnostics carry a fault, or only a status?
We spent a year building a diagnostic layer for ROS 2 (ros2_medkit, earlier thread: What's missing from ROS 2 diagnostics (and what we built) ) and the question we get most (including from a ROSCon reviewer) is why we did not extend diagnostic_msgs, diagnostic_updater and the aggregator instead.
Here is the reasoning with the message definitions side by side, so the specifics can be argued with.
TL;DR: a status is information, a fault is a model you can query and act on, and the second does not fall out of the first by adding fields or more context.
What /diagnostics is, and does well
REP 107 (Tully Foote, November 2010) wrote down the diagnostics system Willow Garage had already built: a /diagnostics topic carrying DiagnosticArray of DiagnosticStatus, and an aggregator with analyzers. It names three things the system is for:
- a quick look at normal operation
- detailed debug information
- and long-term logs for history.
Its best practices are robot_monitor on screen while the robot runs, and a rosbag record of /diagnostics in the bringup launch. The REP defined three levels, OK, Warn and Error. The message today:
byte OK=0
byte WARN=1
byte ERROR=2
byte STALE=3
byte level
string name # "a description of the test/component reporting"
string message # "a description of the status"
string hardware_id # "a hardware unique string"
KeyValue[] values
diagnostic_updater publishes on a period (1s by default)
diagnostic_aggregator groups statuses by name rules (prefix, substring, regex) into /diagnostics_agg, publishes the worst level on /diagnostics_toplevel_state, and marks an item STALE when it stops updating within the analyzer timeout.
rqt_robot_monitor keeps the last 30 arrays in memory.
diagnostic_remote_logging sends statuses to InfluxDB.
diagnostic_analysis, the ROS 1 tool for recorded diagnostics, is “Not ported to ROS2 yet” in the ros2 README.
That is a live status stream. It works well for a person watching a screen, and we did NOT change any of it.
Where a status stream is not enough
Identity. A status is keyed by who reports it (name, “LiDAR scan0”). It can say what broke in message, hardware_id or a key-value, and people use those fields that way, but that is text. Nothing in the stack reads it as a fault identity. The aggregator keeps one status per name, the latest, and overwrites it with the next one. So the stack has no thing called “the fault”. I mean nothing knows when it started, how many times it came back, whether it is still active, or whether anyone acknowledged it.
rqt keeps the last 30 arrays, so a blip at night is gone by morning. People have asked for this before. In January 2018 someone on ROS Answers wanted a latched log of warnings and errors (“high level system log, latching diagnostic warnings/error”, question 280273). Nobody answered, and the asker wrote that they would probably build their own collector node.
Address. Say you have the rosbag from the three seconds around the failure. Where do you put it? hardware_id is just a free text. The aggregator tree is for display in rqt and a tag for the InfluxDB exporter, not a place where you can attach a recording.
Also, much of what breaks in a robot is not a ROS node: the MCU in a motor driver, a sensor with its own firmware. These appear in /diagnostics only if the driver author wrote a diagnostic task for them, and when the driver goes quiet the aggregator can only say STALE.
Operations and state. You cannot ask the stream about a fault. Is it still active? acknowledge it, change the configuration behind it, give me the recording. What the aggregator holds in RAM dies with the process. A bag of /diagnostics or the InfluxDB export survives a restart, but those are stored samples, not state. To know what is active now, you read the samples and work it out yourself. Whatever answers those questions has to be a separate service beside the ROS graph, one that holds fault records with an address and verbs. At that point you are no longer extending a message. Extending the aggregator ends in the same place. The aggregator does one thing: statuses in, one combined status out, with a fast path for watchdogs. Add fault ids, a database and an API and it is no longer an aggregator. It is a fault manager.
What we built beside it
ros2_medkit_msgs/Fault:
string fault_code # e.g. MOTOR_OVERHEAT
uint8 severity # INFO WARN ERROR CRITICAL
string description # the third argument of report()
builtin_interfaces/Time first_occurred
builtin_interfaces/Time last_occurred
builtin_interfaces/Time last_passed
uint32 occurrence_count # counted on edges: a cleared fault that comes back adds one
string status # PREFAILED PREPASSED CONFIRMED HEALED CLEARED
string[] reporting_sources
The lifecycle is an AUTOSAR DEM-style debounce counter. FAILED events move it towards CONFIRMED, PASSED events move it towards HEALED. HEALED means the counter reached the healing threshold. CLEARED means a human acknowledged the fault through the clear service. Both kinds of record stay in the database.
Defaults worth knowing before you compare
(click for more details)
On CONFIRMED the fault manager takes a freeze frame of the reporting node’s topics and with recording on, it writes its ring buffer to an MCAP bag. Each fault is linked by its reporting sources to an entity in a SOVD (ISO 17978) tree: Areas, Components, Apps, Functions. A function carries its own /faults, so “obstacle avoidance is degraded” maps to the app that is containing the fault. A device that never speaks DDS becomes an external component from the manifest, and a protocol plugin (OPC UA in the open repo) turns its alarms into faults on it. All of it is served over REST: list and stream faults (SSE), read one, clear it, read the entity’s data, PUT a configuration, download the bag from bulk-data, etc. A laptop, a fleet server or a tool can do that without joining the DDS graph.
And /diagnostics stays an input
ros2_medkit_diagnostic_bridge subscribes to /diagnostics and feeds the fault manager:
| DiagnosticStatus level |
Fault event |
| OK |
PASSED, moves the fault towards HEALED when healing is enabled |
| WARN |
fault, severity WARN |
| ERROR |
fault, severity ERROR |
| STALE |
fault, severity CRITICAL (see caveats) |
An existing diagnostic_updater gains a fault code, a lifecycle and a history. The node does not change.
Bridge details and caveats
(click for more details)
How the existing tools fit
(click for more details)
The question to diagnostics users
Where should the line be? Our answer: DiagnosticStatus stays the health stream, and the fault record, its entity and the REST API get agreed on separately. That is what we built. Should the fault record become a shared ROS message one day? That is the question for this group. The ROSCon talk goes through the trade-offs.
Repo: ros2_medkit
Sources
(click for more details)
9 posts - 2 participants
Read full topic
ASL ETHZ,
still with no plans to remove ROS 
(for those who have already this package installed, it will auto-update, so no worries there).
Platinum Sponsor
Gold Sponsors
Silver Sponsors
Bronze Sponsors
Startup Alley Sponsors
ROS Meetup Cartagena 2026 is coming!
Napoleón de la Rosa Echávez Auditorium — Universidad Tecnológica de Bolívar
November 7, 2026
8:00 am - 5:00 PM
Register yourself as a participant, or as an speaker in the following LinkTree: 
