Hermes Agent Deep Cuts: The Status Command Says No Traces, the Exporter Builds Spans

A Hermes gateway can export health signals to your observability stack without exporting a single prompt, tool argument, or session transcript. The default is off, and enabling the exporter does not make the gateway a conversation tracing firehose. There is a documentation wrinkle: hermes monitoring status says there are no traces, while the OTLP exporter turns approved monitoring events into OpenTelemetry spans. That mismatch matters when you are deciding what data crosses the boundary.

The event boundary is the feature

Monitoring has two pieces: a gateway health exporter and an OTLP destination. The health side derives metrics and a small set of operational events from gateway runtime state. The exporter maps those events to OpenTelemetry signals and sends them to an endpoint you configure. There is no shipped destination.

The implementation is deliberately narrower than ordinary application tracing. For each event kind, the OTLP mapper allows only named fields. Gateway health can report bounded state such as active agent count, platform count, gateway busy state, version, and supervision mode. Diagnostic events carry a classified error, subsystem, platform, severity, or transition state. Other fields never become span attributes just because they happened to be present in the source event.

The producer does not forward prompts, messages, tool arguments or results, session history, audit records, or product analytics. A bounded redaction pass also runs on string values that are allowed through. Resource labels have their own allowlist, and the install identifier is transformed into an opaque hash before export.

But do not read the status command’s final line literally. It says, “No prompts, messages, tool args/results, usage analytics, or traces.” The installed exporter calls tracer.start_span(...) for each accepted event and sends those spans over OTLP. Those are operational event spans, not conversation traces. The content boundary is narrow, but the word “traces” in the status output is too broad for what the implementation does.

That is a sensible contract for fleet health: answer whether the service is up, busy, or reporting a platform failure, without turning your telemetry collector into a second conversation store.

Enable it without putting credentials in YAML

The exact config keys are in Hermes’s defaults. Start by configuring the OTLP endpoint and explicitly enabling both layers:

monitoring:
  export:
    otlp:
      enabled: true
      endpoint: "https://otel.example.net/v1/traces"
      headers_env:
        Authorization: HERMES_OTLP_AUTHORIZATION
  gateway_health_export:
    enabled: true
    metrics_enabled: true
    diagnostic_events_enabled: true
    warning_error_events_enabled: true
    export_interval_seconds: 60
    logs_export_interval_seconds: 5
    resource_attributes:
      service.name: hermes-gateway
      deployment.environment.name: production

headers_env maps header names to environment variable names, not secret values. Put the credential in the gateway service environment as HERMES_OTLP_AUTHORIZATION. The exporter resolves that variable at export time, skips a missing variable, and does not log or persist its value. Do not paste the secret into config.yaml.

The OpenTelemetry SDK is an optional dependency. The exporter loads it lazily. If it is absent, enabling the settings does not conjure an exporter into existence. Install the optional extra using the package manager for your Hermes installation, then restart the gateway so startup initializes the exporter. Hermes’s source names the extra as hermes-agent[otlp]; package-manager-managed installations may require their own supported dependency workflow.

The two enabled flags are separate switches. The endpoint can be enabled while health export remains disabled, or health export can be enabled without a usable OTLP destination. Check both rather than inferring one from the other.

What can go wrong quietly

The exporter subscribes to an in-process emitter. Emission is isolated from the gateway’s main work: a monitoring subscriber failure is logged locally and dropped instead of being raised into the caller. That’s the right availability tradeoff for telemetry. It also means the gateway can keep working while your collector receives nothing.

The status command is a configuration and dependency check, not a delivery receipt:

hermes monitoring status

On this machine it reports:

Gateway monitoring
  Health export:  disabled (monitoring.gateway_health_export.enabled)
  OTLP endpoint:  not configured (monitoring.export.otlp)
  OTel SDK:       not installed (optional extra: hermes-agent[otlp])

  Scope: gateway service health + redacted diagnostics only.
  No prompts, messages, tool args/results, usage analytics, or traces.

A configured endpoint can still be unreachable, credentials can be absent, or the receiver can reject the payload. The command does not perform an end-to-end probe. Its output tells you what Hermes sees locally, not what the collector accepted.

There is a second distinction worth keeping straight: the same OTLP plumbing supports spans, metrics, and diagnostic logs, but their event sets and enablement are not interchangeable. Turning off diagnostic events does not mean that health metrics are disabled. Conversely, receiving metrics does not prove diagnostic events are permitted or being exported.

Verify the path, not just the switch

First inspect the effective local posture:

hermes monitoring status

Then check the gateway’s startup log after restarting it. The startup path invokes the health exporter initializer, and a missing SDK produces a warning rather than silently making the feature available. Finally, check the collector for the Hermes resource and expected metric or event names. The source-level integration test starts a local OTLP collector, enables metrics, flushes the provider, and asserts receipt of hermes.gateway.up; that is the right shape for a real deployment smoke test too.

Keep the test receiver outside the production process and use a credential with only the permissions needed to write telemetry. Avoid sending a raw debug bundle as a substitute for monitoring: that is a different feature with a different data boundary.

The practical rule is simple: hermes monitoring status verifies local configuration. A collector-side signal verifies delivery. You need both before calling the pipeline healthy.

Sources

• Hermes CLI: monitoring command
• hermes_cli/subcommands/monitoring.py
• hermes_cli/main_agent_cmds.py: cmd_monitoring
• hermes_cli/config_defaults.py: monitoring defaults
• agent/monitoring/gateway_health.py
• agent/monitoring/gateway_health_export.py
• agent/monitoring/otlp_exporter.py
• OTLP exporter integration test