Deploy Correlation

Incidents are more actionable when they answer "what changed?" alongside "what broke?". palena-sre enriches every Incident with deploy and PR context from three independent layers. Each layer degrades independently — any combination can be absent without breaking the others.

Correlation is additive. It never blocks detection or investigation; it runs as a separate controller on Incident creation and writes results to Incident.status.correlation as they arrive.

Layers

Incident detected on workload payments
  │
  ▼
Layer 1: Kubernetes deploy correlation  (always available)
  └─► Deployment/payments rolled out 3m ago, revision 144
  │
  ▼
Layer 2: ArgoCD correlation  (when configured)
  └─► ArgoCD App payments synced to revision abc123 at 10:04
  │
  ▼
Layer 3: GitHub correlation  (when configured)
  └─► PR #4821 "Update memory limits" merged at 10:03 by alice

Layer 1 — Kubernetes deploys

Requires nothing beyond what's already running in the detector. On Incident creation the controller looks up the workload's Deployment or StatefulSet and reads:

• status.conditions[type=Progressing] — last transition time
• metadata.annotations["deployment.kubernetes.io/revision"]

If the last rollout happened within the correlation window (default 30m) before the Incident's firstSeen, it emits:

correlation:
  k8s:
    recentRollout:
      revision: "144"
      deployedAt: "2026-04-20T10:04:12Z"
      age: "3m"

Layer 2 — ArgoCD

Enable with an ArgoCDIntegration ConfigMap in the operator namespace:

apiVersion: v1
kind: ConfigMap
metadata:
  name: palena-sre-argocd-integration
  namespace: palena-sre-system
data:
  endpoint: "https://argocd.example.com"
  tokenSecretRef: "argocd-api-token"
  tokenSecretKey: "token"
  appLabelKey: "argocd.argoproj.io/instance"
  defaultAppNamespace: "argocd"

Workload → Application resolution. If the workload carries the appLabelKey label, the controller reverse-looks-up the Application by that label. Otherwise it falls back to a namespace + name heuristic — label-based discovery is the supported approach.

Output.

correlation:
  argocd:
    application: "payments"
    syncRevision: "abc123def456"
    syncedAt: "2026-04-20T10:04:00Z"
    repoURL: "https://github.com/example/payments"
    targetBranch: "main"
    recentRevisions:
      - revision: "abc123"
        timestamp: "2026-04-20T10:04:00Z"
      - revision: "prev456"
        timestamp: "2026-04-19T14:30:00Z"

Layer 3 — GitHub

Enable with a GitHubIntegration ConfigMap:

apiVersion: v1
kind: ConfigMap
metadata:
  name: palena-sre-github-integration
  namespace: palena-sre-system
data:
  apiEndpoint: "https://api.github.com"   # or a GitHub Enterprise URL
  tokenSecretRef: "github-api-token"
  tokenSecretKey: "token"
  defaultRepoOwner: ""

Resolution.

1. Derive repoURL from the ArgoCD layer or from the workload annotation palena.ai/repo-url.
2. Query commits between the previous sync revision and the current one.
3. Resolve each commit to its associated PR via the GitHub Commits API.
4. Pull PR metadata: title, author, reviewers, files changed, merged-at.

Rate limit awareness. The controller tracks the X-RateLimit-Remaining header. When remaining < 100 it skips correlation for the current Incident and emits a CorrelationGitHubRateLimited condition. PR metadata is cached for an hour — PRs don't change after merge.

Output.

correlation:
  github:
    recentPRs:
      - number: 4821
        title: "Update memory limits for workers"
        author: "alice"
        mergedAt: "2026-04-20T10:03:50Z"
        commitSHA: "abc123"
        filesChanged:
          - "internal/worker/worker.go"
          - "k8s/deploy.yaml"
        url: "https://github.com/example/payments/pull/4821"

GitLab support follows the same pattern but is a later-release feature. v1 ships GitHub correlation only.

Feeding the investigator

Correlation output is part of the evidence bundle the investigator uses. When the bundle is built (see Investigation), the dispatcher reads Incident.status.correlation and formats it for the prompt:

RECENT CHANGES:
- Deploy 3m ago (ArgoCD sync to revision abc123)
- PR #4821 "Update memory limits for workers" merged 4m ago by alice
- Files changed: internal/worker/worker.go, k8s/deploy.yaml

INCIDENT SIGNAL: CrashLoopBackOff with OOMKilled

The LLM now has strong signal to correlate the incident with the specific PR.

Privacy

GitHub API responses can include PII. The controller filters before writing:

• Keeps author.login (username)
• Drops author.email unless GitHubIntegration.includeAuthorEmail is set
• Drops commit messages that match patterns flagged by the leak detector

What's out of scope

• Function-level or taint-flow correlation — not something this operator tries to do
• Source code fetching of changed functions — not needed; the LLM reasons over PR metadata and file names
• Correlating against in-progress PRs (pre-merge) — a later release
• Automatic fix-PR generation — part of the post-v1 autoremediation arc