How OSI Digital built an event-driven, least-privilege CI/CD platform on AKS with KEDA, Karpenter, rootless BuildKit, Azure Workload Identity and JFrog Artifactory.
Teams that outgrow multi-tenant cloud runners (private networking, data sovereignty, or audits such as SOC 2 Type II, ISO 27001, PCI-DSS and FedRAMP) usually fall back to static, self-hosted build VMs, which carry three hidden costs:
OSI Digital replaced it with an event-driven, zero-trust platform on Azure Kubernetes Service (AKS):
--once), unregisters and terminates: a clean, isolated workspace per job.Result: Zero Standing Privileges (ZSP) today, and a roadmap to verifiable supply-chain provenance (SLSA/SBOM).
Cost: Spot capacity up to 90% below on-demand list pricing, above a small always-warm floor. Speed (DORA lead time): pull-request bursts get pods within one polling interval plus node start time, up to 20 replicas per toolchain and a 128-vCPU NodePool limit; beyond that, jobs queue. Security: no perpetual tokens, stored passwords or Docker sockets. Governance: a new toolchain is a ~30-line values manifest and a pull request, not a VM image-baking pipeline.
FinOps baselining: savings depend on pipeline concurrency, job duration variance and legacy VM sizing; OSI Digital establishes an empirical FinOps baseline before migration to quantify net ROI.
The legacy estate ran pipelines on long-running, self-hosted VMs, with four anti-patterns common to mature engineering organizations:
Static VM pools sized for peak Monday-morning load ran underutilized in business hours and idle overnight and at weekends.
Persistent workers reused filesystems, caches and temp directories, so leftover artifacts caused intermittent, non-reproducible failures that cost significant triage time.
Image builds used rootful Docker-in-Docker with privileged: true (a host-escape vector), and unpinned toolchains let unvetted package updates cause silent drift between releases.
Workers registered with organization-wide PATs stored in cleartext on disk, so any compromised job running untrusted pull-request code could exfiltrate access to repositories and registries.
The platform decouples build triggering from compute: it watches Azure DevOps queue depth, provisions isolated compute on demand, enforces least privilege during the job, and reclaims capacity when it ends.
Java, .NET, Helm).--once), the pod and its volumes are destroyed, and Karpenter consolidates empty nodes after a 5-minute cooldown (outside the business-hours disruption freeze).1. Control Plane: KEDA talks only to the Azure DevOps queue, using a read-only Entra identity with no execution permissions in the pool.
2. Execution Plane: agent pods run as non-root UID 1001 with drop: ["ALL"] and default seccomp profiles, with no access to the host Docker daemon or the Kubernetes API (automountServiceAccountToken: false). The short-lived Entra registration token stays in process memory, never on disk, but same-UID steps can still read the agent's pool-scoped credential under /azp and the (still exchangeable) projected Workload Identity token file: run untrusted pull-request builds in a separate agent pool with a minimally privileged identity.
3. Container Compilation: rootless BuildKit accepts builds only over mutual TLS (mTLS); Kubernetes NetworkPolicies limit its egress to private container registries and the JFrog Artifactory virtual repositories that Dockerfile package installs resolve through.
4. Supply Chain: all third-party packages (Maven, npm, NuGet, PyPI) resolve only through authenticated JFrog Artifactory virtual repositories with automated vulnerability scanning.
+---------------------------- Azure DevOps --------------------------------+
| Pipeline Job Queued (Demands: [Java]) |
+------------------+-------------------------------------^----------------+
| Queue Polled every 30s | Agent Registers via
| (KEDA Identity: Pool Read-Only) | Ephemeral Token
+------------------v------------------- AKS -------------+----------------+
| |
| KEDA --ScaledJob--> [ Agent Pod: --once ] --mTLS--> BuildKit |
| | | (Rootless, |
| | | On-Demand) |
| Pending Pod | | Dependencies | |
| Karpenter <-----------------+ | | Push/Pull |
| | Spot / On-Demand VM | | |
| v | | |
| [ Build-Agent NodePool ] | | |
| + Pre-Pull DaemonSet | | |
| + Warm Pause Pod (Priority -1) | | |
+----------------------------------------|--------------------|-----------+
v v
JFrog Artifactory Private Container Registry
(Maven/npm/NuGet/PyPI) (Azure Private Endpoint)
Instead of one monolithic, multi-gigabyte image with every toolchain, the fleet is nine independent agent charts on one centralized Helm library chart, which enforces mandatory security contexts, Workload Identity wiring, volume configurations and KEDA trigger templates. Each chart belongs to one of three workload classes (standard, light, heavy) or the stateful singleton, which standardizes its scheduling, capability routing and compute allocation. Values compose in three layers: toolchain values, workload-class defaults and an environment overlay:
| Workload Class | Toolchains & Runtimes | Routing Demands | Kubernetes Primitive | Compute & Node Sizing Node sizes; per-pod requests are set per chart (e.g. 1 vCPU / 1.5 GiB for jdk17) |
|---|---|---|---|---|
| standard General Compilers |
Java (JDK 17/21, Maven), .NET (SDK 10 LTS), Node.js (v22 LTS) | Java, DotNet, Node |
Ephemeral ScaledJob 1:1 pod lifecycle ( --once) |
Azure Spot First (D/E/F series nodes, 4–8 vCPU / 8–16 GiB; on-demand fallback) |
| light Automation & IaC |
Terraform / Azure CLI, Helm / kubectl, Python 3.11+, SQL DACPAC | Terraform, Helm, Python, SQL |
Ephemeral ScaledJob Rapid sub-minute burst |
Azure Spot First (High bin-packing density on 4 vCPU / 8–16 GiB nodes) |
| heavy E2E Browser Suites |
Selenium, Headless Chromium & Firefox test suites | Selenium |
Warm ScaledObject Persistent listener pool (30m cooldown) |
Dedicated On-Demand (8–16 vCPU / 32+ GiB nodes; zero Spot eviction risk) |
| stateful Singleton Scanner |
OWASP Dependency-Check (NVD vulnerability database) | OWASP |
Dedicated StatefulSet Singleton runner, daily sidecar sync |
On-Demand + Managed Disk (Persistent retained disk for multi-GB DB) |
# 1. charts/agents/jdk17/values.yaml — toolchain-specific specification
workloadClass: standard
agent:
name: jdk17
image:
repository: <acr-registry>.azurecr.io/build-agents-jdk17
tag: v1.2.0@sha256:<digest>
capabilities: # Primary capability matches Azure DevOps pipeline demands
- Java
- JDK17
- Maven
- Docker # Docker/BuildKit = buildctl client to the remote BuildKit daemon
- BuildKit
resources:
requests: { cpu: 1000m, memory: 1500Mi } # Burst capacity without CPU throttling limits
# 2. charts/agents/_classes/standard.yaml — workload scheduling & scaling class
keda:
scalingMode: ScaledJob # Strict 1:1 job-to-pod lifecycle
minReplicaCount: 1 # Pre-warmed listener registered to eliminate pod bootstrap delay
pollingInterval: 30 # 30s evaluation window prevents downstream REST throttling
scheduling:
protectFromDisruption: true # karpenter.sh/do-not-disrupt on all running build pods
nodeSelector: { workload: build-agent }
# 3. envs/agents/jdk17.yaml — environment-specific authentication bindings
workloadIdentity:
enabled: true
clientId: "<managed-identity-client-id>"
tenantId: "<entra-tenant-id>"
buildkit:
tls: { enabled: true }
Seven of the nine toolchains run as KEDA ScaledJobs: a matching queued job starts a Kubernetes Job whose pod runs the Azure Pipelines agent with --once, deregisters and terminates; Kubernetes then destroys the pod and its ephemeral storage. backoffLimit: 0 (no automatic pod retries on build failures) and a hard 2-hour activeDeadlineSeconds stop failed or hung builds from leaving runaway orphan processes.
Architectural Exceptions: two workloads deliberately avoid ScaledJobs:
preStop drain hook and a long terminationGracePeriodSeconds are configured.apiVersion: keda.sh/v1alpha1
kind: ScaledJob
metadata:
name: build-agent-jdk17
namespace: build-agents
spec:
pollingInterval: 30
minReplicaCount: 1 # Pre-warmed listener; immediately replenished on execution
maxReplicaCount: 20 # Per-toolchain blast radius ceiling
scalingStrategy:
strategy: accurate # Direct queue depth matching
successfulJobsHistoryLimit: 5
failedJobsHistoryLimit: 20 # Retain failed job pods for post-mortem forensics
jobTargetRef:
parallelism: 1
completions: 1
backoffLimit: 0 # Prevent automatic pod retries on build failures
activeDeadlineSeconds: 7200 # Hard 2-hour execution boundary
ttlSecondsAfterFinished: 600 # Automated pod and volume cleanup after 10 minutes
template:
metadata:
annotations:
karpenter.sh/do-not-disrupt: "true"
labels:
azure.workload.identity/use: "true"
spec:
restartPolicy: Never
serviceAccountName: build-agent-jdk17
automountServiceAccountToken: false # No Kubernetes API token; Workload Identity projects its own
nodeSelector: { workload: build-agent }
tolerations: # Required: the NodePool taints nodes workload=build-agent:NoSchedule
- { key: workload, operator: Equal, value: build-agent, effect: NoSchedule }
securityContext:
runAsNonRoot: true
runAsUser: 1001
seccompProfile: { type: RuntimeDefault }
containers:
- name: agent
image: <acr-registry>.azurecr.io/build-agents-jdk17@sha256:<digest>
env:
- { name: AZP_URL, value: "https://dev.azure.com/<org>" }
- { name: AZP_POOL, value: "<pool-name>" }
resources:
requests: { cpu: 1000m, memory: 1500Mi } # Karpenter sizes nodes from requests
securityContext:
allowPrivilegeEscalation: false
capabilities: { drop: ["ALL"] }
# readOnlyRootFilesystem: true # rolling out; needs emptyDir mounts for /azp and /tmp
args: ["--once"] # Execute single job, unregister, and terminate
triggers:
- type: azure-pipelines
metadata:
poolID: "<pool-id>"
organizationURLFromEnv: AZP_URL
targetPipelinesQueueLength: "1"
activationTargetPipelinesQueueLength: "0"
demands: "Java" # Primary capability demand match
requireAllDemands: "true"
authenticationRef:
name: keda-trigger-auth
---
apiVersion: keda.sh/v1alpha1
kind: TriggerAuthentication
metadata:
name: keda-trigger-auth
namespace: build-agents
spec:
podIdentity:
provider: azure-workload # Read-only OIDC queue interrogation; zero static PAT
identityId: "<keda-identity-client-id>"
KEDA matches each pipeline's declared demands against the scaler's demand list. Multi-capability matching causes Cartesian scaling bugs when business units share a queue (see Incident 01), so each toolchain scaler matches exactly one routing capability (e.g., Java, DotNet, Helm). Utilities baked into every runner image (e.g., Docker, BuildKit, git) are advertised as Azure DevOps capabilities but never used as KEDA scaling criteria.
One single heterogeneous Karpenter NodePool, rather than fragmented node groups, spans D, E and F-series Azure VM families (4 to 32 vCPUs). Karpenter picks the most cost-effective SKU for pending agent pods' resource requests, prioritizing Azure Spot capacity.
Disruption Handling: karpenter.sh/do-not-disrupt: "true" on every ephemeral build pod blocks Karpenter's own voluntary disruption (consolidation, drift and expiry) while a job runs, but not Azure Spot evictions: when Azure reclaims a Spot VM the pod is lost and, with backoffLimit: 0, the run fails and must be retried. Spot therefore suits short, retry-tolerant jobs (use pipeline-level retry on infrastructure failure); long-running or preemption-sensitive workloads request On-Demand instances in the same shared pool via a nodeSelector. Consolidation is WhenEmpty with a 5-minute stabilization window, and a disruption budget blocks voluntary node churn during business hours (which also delays reclaiming empty nodes in that window). Budget schedules are evaluated in UTC, so set the cron to your engineering hours:
apiVersion: karpenter.sh/v1
kind: NodePool
metadata:
name: build-agents
spec:
template:
metadata:
labels:
workload: build-agent
spec:
nodeClassRef:
group: karpenter.azure.com
kind: AKSNodeClass
name: build-agents
taints:
- { key: workload, value: build-agent, effect: NoSchedule }
requirements:
- { key: kubernetes.io/arch, operator: In, values: ["amd64"] }
- { key: karpenter.sh/capacity-type, operator: In, values: ["spot", "on-demand"] }
- { key: karpenter.azure.com/sku-family, operator: In, values: ["D", "E", "F"] }
- { key: karpenter.azure.com/sku-cpu, operator: In, values: ["4", "8", "16", "32"] }
expireAfter: 168h # Weekly rolling node refresh
terminationGracePeriod: 24h # Safeguard active builds during node drains
limits:
cpu: "128" # Cluster blast-radius cap preventing runaway autoscaling
memory: 512Gi
disruption:
consolidationPolicy: WhenEmpty # Reclaim nodes strictly when all pods terminate
consolidateAfter: 5m
budgets:
# Schedules are evaluated in UTC: set the cron to your engineering hours.
# nodes: "0" blocks ALL voluntary disruption in the window, including reclaiming empty nodes.
- { nodes: "0", schedule: "0 8 * * MON-FRI", duration: 8h }
- { nodes: "20%" }
---
apiVersion: karpenter.azure.com/v1beta1
kind: AKSNodeClass
metadata:
name: build-agents
spec:
vnetSubnetID: <subnet-resource-id>
osDiskSizeGB: 512
imageFamily: Ubuntu2204
Ephemeral agents need start-up that rivals warm runners. Unoptimized, a Kubernetes pod takes 2 to 5 minutes (node provisioning, image layer pulls, toolchain downloads); four tiers keep the typical queue-to-agent-start wait at about 30 seconds or less. The balloon pod hides cold node provisioning only while its pre-warmed node has spare capacity:
| Cold-Start Overhead Driver | Unoptimized Latency | Architectural Remediation |
|---|---|---|
| Node Provisioning & VM Boot | 60–120 s | Negative-priority balloon pause pods hold a pre-warmed node; a real build pre-empts the pause pod immediately, triggering background node scale-out. |
| Container Image Pulling | 30–90 s | A system-node-critical pre-pull DaemonSet loads all agent images into each new node's local containerd cache immediately. |
| Agent Binary Installation | 10–20 s | The official Azure DevOps agent binary is baked into the container image; runtime external downloads are disabled. |
| Dependency Resolution & Restore | 15–60 s | JFrog Artifactory virtual repositories (a nearby private proxy cache) plus read-only, pre-seeded package caches on Azure Files; anything writable stays in the per-pod emptyDir. |
npm's cache (~/.npm) is content-addressed and integrity-checked, so sharing it is low risk. Maven (~/.m2/repository) and NuGet (~/.nuget/packages) caches are keyed by name and version and can hold mutable SNAPSHOTs and resolver metadata: a writable shared cache would let an untrusted pull-request build plant an artifact a later release build consumes, and concurrent writers over Azure Files risk corruption. So only read-only, pre-seeded caches are shared; writable caches are per-pod and discarded, and Artifactory provides proximity. Workspaces, build trees, temp directories and credential volumes are ephemeral emptyDir volumes destroyed with the pod.
Docker-in-Docker (DinD) needs privileged: true and host Docker socket mounts, violating security baselines, so all pipelines use a centralized rootless BuildKit daemon via the lightweight buildctl CLI:
npm install or apt-get resolve through Artifactory; the Helm chart denies open egress policies.
Every build agent service account is federated to an Azure Managed Identity via Azure Workload Identity; that identity is a Basic-access Azure DevOps user, and no Personal Access Token (PAT) is active in the organization. At pod initialization, the Kubernetes admission webhook projects an OIDC token to /var/run/secrets/azure/tokens/azure-identity-token.
A lightweight entrypoint exchanges it with Microsoft Entra ID (an OAuth 2.0 client-assertion JWT exchange using only curl and jq, no Azure CLI) for a short-lived token scoped to Azure DevOps (Resource ID: 499b84ac-1321-427f-aa17-267ca6975798). The token registers the agent once and is then unset; VSO_AGENT_IGNORE also keeps credential variables out of step environments. Deregistration re-acquires a fresh token, because Entra access tokens last roughly 60 to 90 minutes and a job can run for 2 hours. Steps running as the same UID can still read the projected token file (see the Execution Plane boundary), so protection comes from short lifetime and narrow scope, not from hiding the file:
#!/bin/bash
set -euo pipefail
AGENT_DIR=/azp
# 1. Workload Identity OIDC exchange (re-runnable: also used at teardown)
get_token() {
curl -sf --retry 3 -X POST \
"${AZURE_AUTHORITY_HOST%/}/${AZURE_TENANT_ID}/oauth2/v2.0/token" \
-d "grant_type=client_credentials" \
-d "client_id=${AZURE_CLIENT_ID}" \
-d "client_assertion_type=urn:ietf:params:oauth:client-assertion-type:jwt-bearer" \
--data-urlencode "client_assertion=$(cat "${AZURE_FEDERATED_TOKEN_FILE}")" \
-d "scope=499b84ac-1321-427f-aa17-267ca6975798/.default" \
| jq -r '.access_token // empty'
}
# Fail closed: never fall back to an ambient PAT
unset AZP_TOKEN
[ -f "${AZURE_FEDERATED_TOKEN_FILE:-}" ] || { echo "[FATAL] No federated token file" >&2; exit 1; }
TOKEN="$(get_token || true)"
[ -n "${TOKEN}" ] || { echo "[FATAL] Workload Identity token exchange failed" >&2; exit 1; }
# 2. Register, then drop the token from the process (the Entra token is never written to disk;
# the agent itself stores its pool-scoped registration credential under /azp)
# The agent's "PAT" auth scheme accepts the Entra access token; no PAT exists.
"${AGENT_DIR}/config.sh" --unattended --replace --agent "$(hostname)" \
--url "${AZP_URL}" --pool "${AZP_POOL}" --auth PAT --token "${TOKEN}"
unset TOKEN
export VSO_AGENT_IGNORE="AZP_TOKEN,AZP_TOKEN_FILE"
# 3. Deregister with a FRESH token: the registration token may expire during a 2h job
cleanup() {
local t
t="$(get_token || true)"
if [ -n "${t}" ]; then
"${AGENT_DIR}/config.sh" remove --unattended --auth PAT --token "${t}" \
|| echo "[WARN] deregistration failed; a stale agent may remain" >&2
else
echo "[WARN] could not obtain a token to deregister" >&2
fi
}
trap cleanup EXIT
trap 'exit 143' TERM
"${AGENT_DIR}/run.sh" "$@" & # pass --once
wait $!
Every base-image layer is sha256-pinned and all toolchain binaries are version-locked. Agents run as unprivileged UID 1001, drop all Linux kernel capabilities, and are moving to readOnlyRootFilesystem: true (rolling out; see the roadmap). Vertical Pod Autoscaler (VPA) runs only in Recommendation mode, profiling compiler resource use without disrupting active builds:
# Stage 1: Download the agent binary pinned to an exact release
FROM debian:bookworm-slim@sha256:<digest> AS agent-builder
ARG AGENT_VERSION=4.273.0
RUN apt-get update \
&& apt-get install -y --no-install-recommends ca-certificates curl \
&& mkdir -p /azp \
&& curl -fsSL "https://download.agent.dev.azure.com/agent/${AGENT_VERSION}/vsts-agent-linux-x64-${AGENT_VERSION}.tar.gz" \
| tar -xz -C /azp
# Stage 2: Hardened runtime container (pin apt package versions in production)
FROM ubuntu:24.04@sha256:<digest> AS runtime
RUN apt-get update \
&& apt-get install -y --no-install-recommends ca-certificates curl jq git openjdk-17-jdk-headless maven \
&& rm -rf /var/lib/apt/lists/*
# buildctl client only (no Docker daemon): builds run on the remote rootless BuildKit service
COPY --from=moby/buildkit:rootless@sha256:<digest> /usr/bin/buildctl /usr/local/bin/buildctl
COPY --from=agent-builder /azp /azp
RUN /azp/bin/installdependencies.sh \
&& groupadd -g 1001 agent && useradd -m -u 1001 -g agent agent \
&& chown -R agent:agent /azp
COPY --chown=agent:agent --chmod=0755 start.sh /azp/start.sh
USER 1001
ENV AGENT_ALLOW_RUNASROOT=0
ENTRYPOINT ["/azp/start.sh"]
Kyverno validates every resource admitted to the build namespace against zero-trust baselines, preventing configuration drift; two policies are enforced and two run in Audit mode ahead of enforcement:
| Kyverno Admission Policy | Enforced Operational Baseline | Admission Mode |
|---|---|---|
| Disallow Kernel Capabilities | Rejects pods that add capabilities such as NET_RAW, SYS_ADMIN or SYS_PTRACE. | Enforced |
| Mandate Image Digest Pinning | Blocks mutable tags such as :latest; requires explicit sha256 digests. | Audit |
| Require FinOps Workload Labeling | Rejects unlabelled workloads to keep FinOps chargeback accounting precise. | Enforced |
| Mandate Pod Disruption Budgets | Requires PDB declarations for all multi-replica shared infrastructure components. | Audit |
An Argo CD ApplicationSet continuously reconciles all nine toolchain charts from Git (GitOps) with automated prune and self-heal drift remediation. Sync-waves create the BuildKit mTLS certificates and KEDA TriggerAuthentication before any agent scaler starts, and CI contract tests run helm lint, helm-unittest and Checkov static analysis on all generated manifests before merge.
apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
name: build-agents
namespace: argocd
spec:
generators:
- list:
elements:
- { agent: jdk17, class: standard }
- { agent: dotnet10, class: standard }
- { agent: node22, class: standard }
- { agent: python, class: light }
- { agent: helm, class: light }
- { agent: terraform, class: light }
- { agent: sql, class: light }
- { agent: selenium, class: heavy }
- { agent: owasp, class: stateful }
template:
metadata:
name: 'agent-{{ .agent }}'
annotations:
argocd.argoproj.io/sync-wave: "1"
spec:
project: build-agents
source:
repoURL: <git-repo>
path: 'charts/agents/{{ .agent }}'
destination:
server: https://kubernetes.default.svc
namespace: build-agents
syncPolicy:
automated: { prune: true, selfHeal: true }
syncOptions: [ServerSideApply=true]
Found during scale validation, each incident came from valid components interacting under edge conditions, not overt misconfiguration, and lists the automated control added to prevent recurrence.
Queued jobs stalled in pending status while KEDA reported zero eligible work. After switching to disjunctive demand matching, one scaler spawned 28 pods within 120 seconds for jobs it lacked the toolchain to run.
The scaler demand list mirrored the image’s complete capability set. Conjunction (requireAllDemands: true) failed because pipelines requested only their primary toolchain; disjunction (requireAllDemands: false) scaled on generic shared tags such as Docker, intercepting jobs across the multi-tenant pool.
Each scaler now evaluates one atomic routing capability. A Helm templating bug that silently converted explicit booleans was fixed and locked down with automated unit contract tests.
Canary runners on Workload Identity registered and passed every pipeline suite, yet Entra ID audit logs showed zero OIDC token exchanges.
The shared baseline values manifest still injected a legacy PAT environment variable, which the bootstrap script preferred over federated exchange, so runners silently authenticated with the deprecated PAT.
The Helm library now prunes PAT environment variables whenever Workload Identity is active, and CI regression tests fail the build if static credentials appear in rendered manifests.
Karpenter created Azure VMs for pending pods, but the new nodes never registered with the Kubernetes API server, leaving pods Pending indefinitely.
The AKSNodeClass subnet lacked the User-Defined Route (UDR) table sending egress through the enterprise Next-Generation Firewall, so kubelets booted but could not reach the AKS API server or registry endpoints.
Subnet network security group and route-table bindings are now checked in automated pre-flight Terraform validation, ensuring routing parity across all auto-provisioned subnets.
All eight KEDA scalers (seven ScaledJobs and the Selenium ScaledObject) went NotReady simultaneously, halting cluster autoscaling.
Agent charts referenced an imperatively created TriggerAuthentication whose name diverged from the GitOps-managed specification; routine cleanup of untracked cluster resources deleted it, breaking every dependent scaler.
All TriggerAuthentication objects are now declared in the base platform release and ordered by Argo CD sync-waves, and manual cluster modifications are automatically reconciled.
backoffLimit: 0) and a hard 2-hour deadline prevent runaway orphan processes.:latest admission check is moving from Audit to Enforce.WhenEmpty consolidation and zero voluntary disruption in business hours (schedules in UTC).The ephemeral execution plane is in production; next come supply-chain verification and telemetry:
Self-hosted build infrastructure is among the most privileged yet least-examined parts of an enterprise technology stack. Making it ephemeral and event-driven turns a brittle collection of static machines into a resilient, cloud-native utility, improving economics and security posture at the same time.
The surest way to remove the security risk and capital waste of long-lived build infrastructure is to minimize it and keep whatever remains short-lived and tightly scoped. If your pipelines still run on static build VMs, OSI Digital's Platform Engineering practice can guide a phased, validated, low-risk modernization.
karpenter.sh/do-not-disrupt: "true", which blocks Karpenter's voluntary disruption but not Azure Spot evictions, so short jobs rely on pipeline-level retry. Interruption-sensitive workloads (browser test suites, the BuildKit daemon) select on-demand capacity via nodeSelector in the same NodePool.
drop: ["ALL"]. Rootless BuildKit over mTLS replaces privileged Docker-in-Docker, aligning with CIS Kubernetes benchmark controls apart from one documented seccomp exception.
Move your Azure DevOps pipelines to an on-demand, least-privilege, cost-optimized Kubernetes platform with OSI Digital's platform engineering practice.
Schedule an Architecture Strategy Session →