CVE-2026-77596 is a critical authorization failure in Kubeflow Pipelines that can let a low-privilege tenant alter shared pipeline definitions. In the wrong deployment, the next unpinned recurring run can execute an attacker-controlled pipeline version inside another tenant’s namespace, using that workload’s service account and access.
Operational priority: inventory every multi-user Kubeflow Pipelines deployment, upgrade the API server and matching manifests to 2.18.0 or later, then investigate existing shared versions and schedules. Installing the fix does not erase a version that may already have been uploaded.
What the Kubeflow vulnerability changes
The upstream advisory was published on October 28, 2026. It identifies Kubeflow Pipelines API server releases through 2.17.3 as affected and 2.18.0 as the first fixed release. The issue is tracked as CVE-2026-77596 and GHSA-rjr3-43j3-4q8g. The maintainer-assigned CVSS 3.1 score is 9.6; this article uses that published score rather than calculating a new one.
In multi-user mode, the HTTP multipart endpoints for uploading a pipeline or a new pipeline version delegated authorization to canUploadVersionedPipeline. When the resolved namespace was empty, the helper returned successfully before performing Kubernetes SubjectAccessReview. An empty namespace represents a shared, cluster-wide pipeline and was also the default when callers omitted the namespace parameter.
That gap matters because pipeline definitions contain container images, commands, arguments, volumes, parameters, and execution logic. A tenant who should not be allowed to maintain shared pipelines could create one or inject a new version into an existing shared entry. The advisory also notes that anonymous exploitation is possible when the API server is directly reachable without an authenticating proxy; that is a deployment condition, not a universal property of Kubeflow.
When cross-tenant code execution becomes possible
Unauthorized upload is the first stage. Automatic execution requires an enabled recurring run that follows the latest version of the targeted shared pipeline. When that schedule fires, Kubeflow resolves the newest ready version and constructs the workflow in the consuming tenant’s namespace. A malicious specification can therefore run with the victim workload’s service account and reach resources available to that identity.
This does not automatically mean cluster-admin access or every secret in a namespace is exposed. Impact follows the permissions, mounts, network paths, and credentials available to the affected workload. Runs pinned to a reviewed version, or runs containing an embedded specification, are not subject to latest-version substitution through this path.
Exposure checklist
- The deployment uses Kubeflow Pipelines in multi-user mode.
- The API server is version 2.17.3 or earlier.
- Shared pipelines exist or upload clients omit an explicit tenant namespace.
- Recurring runs follow the latest shared pipeline version instead of a pinned, reviewed version.
- The API is reachable by low-privilege tenants, or it is exposed without a reliable authenticating proxy.
Immediate response plan
1. Confirm the deployed API-server version
Start with deployment evidence, not an operator’s recollection. Capture the namespace, deployment name, image reference, digest, manifest source, multi-user configuration, and authentication path. The following read-only examples help locate likely API-server deployments; labels and names differ between distributions, so verify them before relying on the output.
kubectl get deployment -A
-o custom-columns='NAMESPACE:.metadata.namespace,NAME:.metadata.name,IMAGES:.spec.template.spec.containers[*].image'
| grep -E 'ml-pipeline|kfp-api-server'
kubectl -n kubeflow get deployment ml-pipeline
-o jsonpath='{.spec.template.spec.containers[*].image}{"n"}'Preserve the output in the incident or change record. Do not infer the running version only from a mutable image tag; record the digest when your runtime exposes it.
2. Contain risky shared-pipeline behavior
If immediate upgrading is impossible, the official advisory recommends setting REQUIRE_NAMESPACE_FOR_PIPELINES=true so empty-namespace operations are rejected. Treat this as temporary hardening: test it in staging because existing shared-upload workflows can break. Pause high-risk unpinned recurring runs, restrict access to the Pipelines API, and require tenant-owned uploads to specify their namespace explicitly.
Do not restore the bypass or grant blanket access to the Kubeflow system namespace just to keep older automation working. Designate a small shared-pipeline maintainer group and grant only the pipeline-write permission it needs.
3. Upgrade the server and matching manifests
Move to Kubeflow Pipelines 2.18.0 or later using the release and migration guidance for your distribution. Version 2.18.0 changes authorization and client behavior, so update the coordinated server components and manifests instead of replacing only one container. Validate UI uploads, SDK uploads with an explicit namespace, recurring runs, service-account permissions, and shared-pipeline maintenance after rollout.
The fix maps shared writes to the Kubeflow system namespace and runs the normal authorization check. You can validate the intended identity without performing an upload by using Kubernetes authorization review. Substitute the real service account and system namespace:
kubectl auth can-i create pipelines.pipelines.kubeflow.org
--as=system:serviceaccount:kubeflow:shared-pipeline-maintainer
-n kubeflow4. Investigate shared versions and recurring runs
Patching stops the missing authorization path, but it does not remove existing pipeline versions. Export or enumerate the shared catalog using your supported administrative workflow. Compare version creation times, uploader identities, specifications, container images, commands, parameters, source repositories, and expected change tickets. Review every schedule that follows the latest version, then pin it to a known-good version while the investigation proceeds.
Hunt API and proxy logs for POST /apis/v2beta1/pipelines/upload and POST /apis/v2beta1/pipelines/upload_version, especially requests with no namespace, unexpected identities, unusual source networks, or successful responses outside approved change windows. Correlate suspicious version creation with scheduled workflows, pods, image pulls, service-account token use, cloud audit logs, outbound network connections, and secret access. Missing historical logs are an evidence gap, not proof that exploitation did not occur.
Hands-on lab: build a remediation queue safely
This lab classifies an offline CSV inventory. It does not contact Kubernetes, upload pipelines, or reproduce the authorization bypass. Use synthetic data first, then replace it with approved inventory evidence.
Prerequisites
- Python 3.10 or later.
- A disposable local directory with no production credentials.
- CSV fields collected through your normal change or asset-management process.
Sample inventory
deployment,version,multi_user,api_externally_reachable,shared_pipelines,unpinned_recurring_runs,history_reviewed
prod-ml,2.17.3,true,false,true,true,false
stage-ml,2.16.1,true,true,true,false,false
dev-single,2.17.2,false,false,false,false,false
pilot-ml,2.18.0,true,false,true,false,false
secure-ml,2.18.1,true,false,true,false,true
legacy-ml,unknown,true,false,true,true,falseClassifier
#!/usr/bin/env python3
import csv, sys
FIXED = (2, 18, 0)
def yes(value):
return value.strip().lower() in {"1", "true", "yes", "y"}
def version(value):
parts = value.strip().lower().lstrip("v").split(".")
if len(parts) < 2 or not all(p.isdigit() for p in parts[:3]):
return None
nums = [int(p) for p in parts[:3]]
return tuple((nums + [0, 0])[:3])
def classify(row):
current = version(row["version"])
if current is None:
return "UNKNOWN_VERSION"
if current >= FIXED:
return "FIXED_AND_REVIEWED" if yes(row["history_reviewed"]) else "VERIFY_HISTORY"
if not yes(row["multi_user"]):
return "UPDATE_RECOMMENDED"
if yes(row["shared_pipelines"]) and yes(row["unpinned_recurring_runs"]):
return "CRITICAL_REVIEW"
if yes(row["shared_pipelines"]) or yes(row["api_externally_reachable"]):
return "PATCH_AND_AUDIT"
return "PATCH_REQUIRED"
with open(sys.argv[1], newline="", encoding="utf-8") as handle:
for row in csv.DictReader(handle):
print(f"{classify(row):20} {row['deployment']:18} {row['version']}")Run it with python3 kfp_inventory.py kfp-inventory.csv. Expected classifications for the sample are:
CRITICAL_REVIEW prod-ml 2.17.3
PATCH_AND_AUDIT stage-ml 2.16.1
UPDATE_RECOMMENDED dev-single 2.17.2
VERIFY_HISTORY pilot-ml 2.18.0
FIXED_AND_REVIEWED secure-ml 2.18.1
UNKNOWN_VERSION legacy-ml unknownTroubleshooting and cleanup
If the script reports missing columns, preserve the header exactly. Versions with vendor suffixes are intentionally classified as unknown; confirm their upstream base version instead of guessing. After review, delete the synthetic CSV and script or move them into your approved incident repository. Never place tokens, kubeconfigs, pipeline secrets, or raw production specifications in the lab file.
Long-term controls for multi-tenant MLOps
- Prefer tenant-owned private pipelines; reserve shared definitions for reviewed, centrally maintained artifacts.
- Pin production schedules to immutable, approved versions and require change review before repointing them.
- Apply least privilege to workflow service accounts, registry access, cloud identities, secrets, and network egress.
- Authenticate every API path, restrict direct API-server reachability, and retain upload and workflow audit logs.
- Continuously compare running image digests and pipeline specifications with signed or otherwise trusted sources.
For broader cluster preparation, see our Kubernetes node audit checklist. Teams operating generated-code interfaces should also review the Prefab UI sandbox escape response guide, because MLOps security depends on both orchestration boundaries and the code-generation layer.
Conclusion
CVE-2026-77596 turns a missing authorization check into a cross-tenant integrity problem for shared Kubeflow pipelines. Upgrade to 2.18.0 or later, but do not stop at deployment success. Review historical versions, pin recurring runs, validate the shared-writer role, and investigate workloads launched from unexpected definitions. The absence of a confirmed public intrusion report does not reduce the need to close a reproducible authorization gap.







