Skip to content

Installation ​

palena-sre ships in three distribution formats: a Helm chart, an OLM bundle for OperatorHub, and raw Kustomize manifests. All three install the same CRDs and the same workload Deployments; the choice is purely about how you prefer to manage Kubernetes add-ons.

This page covers the Helm path. For OpenShift OperatorHub installs, use the OLM bundle — the prerequisites and the CRD surface are identical.

Requirements ​

ResourceMinimum
Kubernetes1.28+
Helm3.12+
Architecturelinux/amd64 or linux/arm64
Nodesmetrics-server recommended (auto-discovered when present)

Supported distributions: AKS, GKE, OpenShift 4.14+, kubeadm, k3s.

Presets ​

The chart ships four values presets. Pick the closest to your environment and override from there:

PresetIntent
values.yamlGeneric Kubernetes defaults
values-dev.yamlSingle replica, permissive, no leak detection
values-production.yamlStandard production: leak detection off, IncidentTrace emission on
values-regulated.yamlAir-gap friendly, leak detection on, strict resource limits
values-openshift.yamlOpenShift overrides — UBI images, restricted-v2 SCC compatibility

Standard install ​

bash
helm install palena-sre oci://ghcr.io/palenaai/charts/palena-sre \
  --namespace palena-sre-system \
  --create-namespace

Verify the install:

bash
kubectl -n palena-sre-system get deploy
kubectl get crds | grep palena.ai

You should see the operator, detector, and investigator Deployments and nine CRDs in the palena.ai group.

Production install ​

bash
helm install palena-sre oci://ghcr.io/palenaai/charts/palena-sre \
  --namespace palena-sre-system --create-namespace \
  -f values-production.yaml

The production preset sets:

  • Two operator replicas with leader election
  • Stricter resource limits
  • Leak detector workload disabled (CRD + controller still installed)
  • IncidentTrace retention of 180 days
  • PodMonitor scraping off by default — opt-in if you run Prometheus

Regulated / air-gapped install ​

Air-gap is a first-class mode. The operator makes zero outbound calls by default; everything it talks to is configured by you.

  1. Mirror images. Copy the operator images and optional Drain3 sidecar image from ghcr.io/palenaai/* to your internal registry using skopeo copy.
  2. Download the chart as a tarball. helm pull the chart once from an internet-connected host and ship the tarball into the restricted environment.
  3. Install with the regulated preset.
bash
helm install palena-sre ./palena-sre-0.1.0.tgz \
  --namespace palena-sre-system --create-namespace \
  -f values-regulated.yaml \
  --set global.imageRegistry=registry.internal.corp

The regulated preset enables leak detection and assumes you have classifier and NER endpoints ready (see Leak detection). If you don't, override:

bash
--set leakDetector.enabled=false

The installation works behind a deny-all-egress NetworkPolicy. The operator's only required network peer is the Kubernetes API.

OpenShift ​

bash
helm install palena-sre oci://ghcr.io/palenaai/charts/palena-sre \
  --namespace palena-sre-system --create-namespace \
  -f values-openshift.yaml

The OpenShift overlay:

  • Uses UBI base images (ubi9-minimal, ubi9/python-311 for the Drain3 sidecar)
  • Runs every workload under restricted-v2 SCC — no elevated SCCs requested
  • Uses disk-backed emptyDir, not memory
  • Ships no Routes or Ingresses — create them yourself

For OperatorHub-style installs, the OLM bundle is the preferred path and produces the same topology.

Key Helm values ​

Common overrides — see the chart's README for the full set.

yaml
global:
  imageRegistry: ""           # e.g. registry.internal.corp
  imagePullSecrets: []
  namespace: palena-sre-system

operator:
  replicas: 2
  leaderElection: true

detector:
  enabled: true
  replicas: 1
  drain3Sidecar:
    enabled: true             # required for log pattern drift detection

investigator:
  enabled: true
  replicas: 1
  scaleToZero: false          # enable only if KEDA is installed

leakDetector:
  enabled: false              # default off — enable when classifier + NER are ready
  classifierEndpointRef:
    name: ""
  nerEndpointRef:
    name: ""

integrations:
  prometheus: { enabled: false, endpoint: "" }
  loki:       { enabled: false, endpoint: "" }
  argocd:     { enabled: false, endpoint: "", tokenSecretRef: "" }
  github:     { enabled: false, apiEndpoint: "https://api.github.com", tokenSecretRef: "" }

podMonitor:
  enabled: false              # opt-in; requires kube-prometheus-stack
networkPolicy:
  enabled: false              # opt-in

Post-install configuration ​

The operator installs inert — it watches nothing until you create an SREWatch. The minimum viable configuration is:

  1. An LLMEndpoint (only required if you want LLM-driven investigation).
  2. A DetectionPolicy listing which detectors to run.
  3. An InvestigationPolicy pointing at the LLMEndpoint and defining budget.
  4. An SREWatch selecting workloads and referencing the policies.

Optionally:

  • NotificationChannel CRDs for Slack, Teams, PagerDuty, webhook, or email.
  • Runbook CRDs to tailor investigation prompts for specific workloads.
  • ArgoCD and GitHub integration ConfigMaps for deploy correlation.

Full examples in the Quick Start. Full spec in CRDs.

Upgrading ​

Minor versions are backward compatible — CRD changes are additive within a minor series and reconcilers handle prior-minor resources gracefully.

bash
helm upgrade palena-sre oci://ghcr.io/palenaai/charts/palena-sre \
  --namespace palena-sre-system

Major versions may require manual CRD migration; follow the release notes. OLM users get upgrades through the OperatorHub subscription model.

Uninstalling ​

bash
helm uninstall palena-sre --namespace palena-sre-system

Helm leaves the CRDs in place on purpose — uninstalling the operator should not destroy incident history. To fully remove:

bash
kubectl get crd -o name | grep palena.ai | xargs kubectl delete

Observability ​

The operator exposes a Prometheus /metrics endpoint on port 8443 per workload. A PodMonitor template ships with the chart (opt-in). Example dashboards and alert rules are in the repo under hack/dashboards/ and hack/alerts/ — not installed by default.

Relevant metrics:

  • palena_sre_incident_phase_transitions_total{from,to}
  • palena_sre_incident_reconcile_duration_seconds
  • palena_sre_incident_in_phase{phase}
  • palena_sre_investigation_budget_usage{policy,counter}
  • palena_sre_incidents_open

Out of scope ​

  • Windows nodes — Linux only
  • Kubernetes < 1.28
  • Helm v2
  • Rancher marketplace listing — planned for a later release

Released under the Apache 2.0 License.