Skip to content
STIRUP

TECHNOLOGY INTELLIGENCE

Understand what's next.

Stromboli AI SRE Agent Integrates with Fly.io Using Read-Only Tokens

Stromboli's integration with Fly.io uses a read-only organization token to access metrics and machine lists without deployment permissions, ensuring secure and limited access for incident management.

Share
AI-generated conceptual illustration accompanying: Stromboli AI SRE Agent Integrates with Fly.io Using Read-Only Tokens
AI-generated editorial illustration of the article topic; not a photograph, product screenshot or measured result.

Integration with Fly.io and Read-Only Access

The Stromboli open-source AI SRE agent integrates with Fly.io using a read-only organization token, which grants access to metrics, machine lists, and catalog discovery without requiring full deployment permissions. According to Hacker News Show, a read-only token is sufficient for these purposes, and it cannot be used to deploy, restart, or access secrets. To set up the integration, users can generate a read-only token with the command fly tokens create readonly <org-slug>, and then add the Fly.io connector to Stromboli using the token inline. As noted in Hacker News Show, Stromboli only stores aggregated evidence from each investigation, ensuring that sensitive data is not retained. This approach aligns with the principle of minimizing data exposure while still enabling effective incident mitigation through AI-driven analysis.

Service Catalog Management and Configuration

Stromboli's service catalog management enables teams to define and organize services with detailed metadata, including owners, dependencies, and monitoring configurations. Users can discover services using commands like stromboli catalog discover, then manually set attributes such as owners and dependencies with stromboli catalog set. For bulk operations, the catalog can be exported to a YAML file, edited with additional details like runbooks and metrics, and then imported back into Stromboli using stromboli catalog import. The catalog is stored in Stromboli's database and includes fields such as service name, description, owners, tier, logs, deploys, metrics, runbooks, monitors, and runtime environments. This structured approach ensures that all service-related information is centralized and easily accessible for incident response and operational management.

Incident Investigation and Mitigation Features

Stromboli's incident investigation and mitigation features are designed to streamline the on-call engineer's workflow by focusing on actionable insights. The tool collects critical information that an on-call engineer would typically examine first, such as service dependencies, blast radius, log error patterns, and per-instance metrics like memory, CPU, and request rate. Stromboli also compares these metrics against a baseline from the preceding time window, helping identify anomalies quickly. Additionally, it surfaces recent deploys or commits, runbooks, and similar past investigations to provide context. For mitigation, Stromboli provides precise commands, instructions on verifying success, and options to undo actions, along with risk classifications such as safe, reversible, or destructive. This approach ensures that engineers can respond to incidents with confidence, reducing resolution time and minimizing the impact of outages.

Slack Integration and Permissions

The Stromboli SRE agent integrates with Slack using the chat:write scope, as defined in deploy/slack/manifest.json. This minimal permission allows the app to post messages directly to channels, facilitating real-time incident communication. To enable interactive buttons, the Signing Secret must be configured in the app's Basic Information settings. Optional permissions can be set via the SLACK_APPROVERS environment variable, which restricts the ability to click Apply or Reject buttons to specific users identified by their Slack member IDs. Verification of Slack integration can be done using the stromboli investigate --description "testing Slack" command, which should result in a message appearing in the designated channel. The integration is described in the report section and is currently read-only, relying on external API tools rather than LLM-based processing.

Metric and Log Connector Setup

The Metric and Log Connector Setup in Stromboli requires careful configuration to ensure proper integration with various monitoring tools. For example, when connecting to Fly.io, the setup involves using the fly tokens create command to generate a read-only token, which is then used with the FlyV1 scheme. Similarly, for Prometheus-compatible systems, the connector add prom command is used with the appropriate URL and authentication parameters. Google Cloud Monitoring also requires specific parameters, such as project= and credentials_json=@key.json. These configurations are essential for Stromboli to access and process metrics effectively. According to the documentation, Fly.io automatically collects machine metrics into a managed Prometheus instance, eliminating the need for additional deployment. Each connector setup must be tailored to the specific tool being used, ensuring compatibility and secure access.

Sources and reporting notes

Based on published source material. No independent hands-on testing is claimed.

Read more