The Future of Kubernetes Admission Logic
A presentation at Container Days Conference in in Hamburg, Germany by Marcus Noble
The Future of Kubernetes Admission Logic ContainerDays September 11th 2025 🐘 @Marcus@k8s.social | 🌐 MarcusNoble.com | 🦋 @averagemarcus.bsky.social
Hi 👋 , I’m Marcus Noble! I’m a platform engineer at I run a monthly newsletter - CloudNative.Now 7+ years experience running Kubernetes in production environments. 🐘 @Marcus@k8s.social | 🌐 MarcusNoble.com | 🦋 @averagemarcus.bsky.social
Dynamic Admission Control ValidatingAdmission MutatingAdmission Webhook Webhook 🐘 @Marcus@k8s.social | 🌐 MarcusNoble.com | 🦋 @averagemarcus.bsky.social
Purpose / Use Cases Defaulting Policy Enforcement ● Injecting imagePullSecrets dynamically when pods are created ● Injecting sidecars into pods ● Injecting proxy environment variables into pods ● Prevent using latest image tag ● Require all pods to have resource limits set ● Block the use of deprecated Kubernetes APIs (e.g. batch/v1beta1) ● Require a PodDisruptionBudget ● Enforce a standard set of labels / annotations on all resources ● Replace all image registries with an inhouse container image proxy / cache ● Block nodes joining the cluster with known CVEs based on the kernel version (e.g. CVE-2022-0185) ● Inject Log4Shell mitigation env var into all pods (CVE-2021-44228) ● Block binding to the cluster-admin role Best Practices Problem Mitigation 🐘 @Marcus@k8s.social | 🌐 MarcusNoble.com | 🦋 @averagemarcus.bsky.social
Webhooks in Kubernetes are ✨ POWERFUL ✨ But with that power comes Taken from my talk about this at KCD UK 😱 RISK 😱 🐘 @Marcus@k8s.social | 🌐 MarcusNoble.com | 🦋 @averagemarcus.bsky.social
Wouldn’t it be great if we had a safer alternative? • Yes! Yes, it would! 🐘 @Marcus@k8s.social | 🌐 MarcusNoble.com | 🦋 @averagemarcus.bsky.social
Introducing ValidatingAdmissionPolicies Status: Alpha in v1.26, Beta in v1.28, GA in v1.30 Introduced in: KEP-3488 A declarative, in-process alternative to validating admission webhooks. 🐘 @Marcus@k8s.social | 🌐 MarcusNoble.com | 🦋 @averagemarcus.bsky.social
Introducing ValidatingAdmissionPolicies Status: Alpha in v1.26, Beta in v1.28, GA in v1.30 Introduced in: KEP-3488 A declarative, in-process alternative to validating admission webhooks. Kubernetes manifests 🐘 @Marcus@k8s.social | 🌐 MarcusNoble.com | 🦋 @averagemarcus.bsky.social
Introducing ValidatingAdmissionPolicies Status: Alpha in v1.26, Beta in v1.28, GA in v1.30 Introduced in: KEP-3488 api-server A declarative, in-process alternative to validating admission webhooks. Kubernetes manifests 🐘 @Marcus@k8s.social | 🌐 MarcusNoble.com | 🦋 @averagemarcus.bsky.social
Introducing ValidatingAdmissionPolicies Status: Alpha in v1.26, Beta in v1.28, GA in v1.30 Introduced in: KEP-3488 api-server A declarative, in-process alternative to validating admission webhooks. Kubernetes manifests More on this shortly Uses the Common Expression Language (CEL) for the policy language. 🐘 @Marcus@k8s.social | 🌐 MarcusNoble.com | 🦋 @averagemarcus.bsky.social
Introducing ValidatingAdmissionPolicies Status: Alpha in v1.26, Beta in v1.28, GA in v1.30 Introduced in: KEP-3488 api-server A declarative, in-process alternative to validating admission webhooks. Kubernetes manifests More on this shortly Uses the Common Expression Language (CEL) for the policy language. Consists of two main resources: • ValidatingAdmissionPolicy describes the policy logic • ValidatingAdmissionPolicyBinding links the above policy to the resources it applies to 🐘 @Marcus@k8s.social | 🌐 MarcusNoble.com | 🦋 @averagemarcus.bsky.social
Brief introduction to CEL • Victor gave a great introduction to CEL on Tuesday References: • https://kubernetes.io/docs/ reference/using-api/cel/ • https://github.com/google/ cel-go • https://playcel.undistro.io/ (CEL playground) • • • • Uses a similar syntax to the expressions in C-based languages, e.g. self.minReplicas <= self.replicas && self.replicas <= self.maxReplicas Designed to be embedded into other applications with a focus on “one-liners” of code. Small number of built in functions (e.g. split, has) Functions expanded through custom libraries (Kubernetes includes several of these) Already used by Kyverno, Tekton, Kubewarden, etc. 🐘 @Marcus@k8s.social | 🌐 MarcusNoble.com | 🦋 @averagemarcus.bsky.social
Example Blocking the use of the ‘latest’ image tag apiVersion: admissionregistration.k8s.io/v1 kind: ValidatingAdmissionPolicy metadata: name: “prevent-latest-image-tag” spec: failurePolicy: Fail matchConstraints: resourceRules: - apiGroups: [“apps”] apiVersions: [“v1”] operations: [“CREATE”, “UPDATE”] resources: [“deployments”, “daemonsets”, “statefulsets”] validations: - message: “The use of the ‘latest’ tag is not allowed” expression: | object.spec.template.spec.containers.all(container, !container.image.endsWith(“:latest”) && container.image.contains(“:”) ) && ( !has(object.spec.template.spec.initContainers) || object.spec.template.spec.initContainers.all(container, !container.image.endsWith(“:latest”) && container.image.contains(“:”) ) ) More examples: github.com/AverageMarcus/ common-admission-policies 🐘 @Marcus@k8s.social | 🌐 MarcusNoble.com | 🦋 @averagemarcus.bsky.social
Example Blocking the use of the ‘latest’ image tag apiVersion: admissionregistration.k8s.io/v1 kind: ValidatingAdmissionPolicy metadata: name: “prevent-latest-image-tag” spec: failurePolicy: Fail matchConstraints: resourceRules: - apiGroups: [“apps”] apiVersions: [“v1”] operations: [“CREATE”, “UPDATE”] resources: [“deployments”, “daemonsets”, “statefulsets”] validations: - message: “The use of the ‘latest’ tag is not allowed” expression: | object.spec.template.spec.containers.all(container, !container.image.endsWith(“:latest”) && container.image.contains(“:”) ) && ( !has(object.spec.template.spec.initContainers) || object.spec.template.spec.initContainers.all(container, !container.image.endsWith(“:latest”) && container.image.contains(“:”) ) ) The name of our policy. We’ll reference this in our ValidatingAdmission PolicyBinding 🐘 @Marcus@k8s.social | 🌐 MarcusNoble.com | 🦋 @averagemarcus.bsky.social
Example Blocking the use of the ‘latest’ image tag apiVersion: admissionregistration.k8s.io/v1 kind: ValidatingAdmissionPolicy metadata: name: “prevent-latest-image-tag” spec: failurePolicy: Fail matchConstraints: resourceRules: - apiGroups: [“apps”] apiVersions: [“v1”] operations: [“CREATE”, “UPDATE”] resources: [“deployments”, “daemonsets”, “statefulsets”] validations: - message: “The use of the ‘latest’ tag is not allowed” expression: | object.spec.template.spec.containers.all(container, !container.image.endsWith(“:latest”) && container.image.contains(“:”) ) && ( !has(object.spec.template.spec.initContainers) || object.spec.template.spec.initContainers.all(container, !container.image.endsWith(“:latest”) && container.image.contains(“:”) ) ) How we should handle mistakes / runtime errors in our CEL expressions. By default it will block the request. The alternative is Ignore if you want to allow requests on error. Note: This is when something errors, not when the validations fail. E.g. typo in CEL expression 🐘 @Marcus@k8s.social | 🌐 MarcusNoble.com | 🦋 @averagemarcus.bsky.social
Example Blocking the use of the ‘latest’ image tag apiVersion: admissionregistration.k8s.io/v1 kind: ValidatingAdmissionPolicy metadata: name: “prevent-latest-image-tag” spec: failurePolicy: Fail matchConstraints: resourceRules: - apiGroups: [“apps”] apiVersions: [“v1”] operations: [“CREATE”, “UPDATE”] resources: [“deployments”, “daemonsets”, “statefulsets”] validations: - message: “The use of the ‘latest’ tag is not allowed” expression: | object.spec.template.spec.containers.all(container, !container.image.endsWith(“:latest”) && container.image.contains(“:”) ) && ( !has(object.spec.template.spec.initContainers) || object.spec.template.spec.initContainers.all(container, !container.image.endsWith(“:latest”) && container.image.contains(“:”) ) ) We define what resources this policy applies to. Multiple resource types can be defined here but if they don’t have the same general API the CEL expression will quickly become very complex. Supported operations are: - CREATE - UPDATE - DELETE - CONNECT - * (meaning all the previous) 🐘 @Marcus@k8s.social | 🌐 MarcusNoble.com | 🦋 @averagemarcus.bsky.social
Example Blocking the use of the ‘latest’ image tag apiVersion: admissionregistration.k8s.io/v1 kind: ValidatingAdmissionPolicy metadata: name: “prevent-latest-image-tag” spec: failurePolicy: Fail matchConstraints: resourceRules: - apiGroups: [“apps”] apiVersions: [“v1”] operations: [“CREATE”, “UPDATE”] resources: [“deployments”, “daemonsets”, “statefulsets”] validations: - message: “The use of the ‘latest’ tag is not allowed” expression: | object.spec.template.spec.containers.all(container, !container.image.endsWith(“:latest”) && container.image.contains(“:”) ) && ( !has(object.spec.template.spec.initContainers) || object.spec.template.spec.initContainers.all(container, !container.image.endsWith(“:latest”) && container.image.contains(“:”) ) ) Validations define one or more conditions that must be met for the request to be allowed. The array is AND’d together and all must pass. 🐘 @Marcus@k8s.social | 🌐 MarcusNoble.com | 🦋 @averagemarcus.bsky.social
Example Blocking the use of the ‘latest’ image tag apiVersion: admissionregistration.k8s.io/v1 kind: ValidatingAdmissionPolicy metadata: name: “prevent-latest-image-tag” spec: failurePolicy: Fail matchConstraints: resourceRules: - apiGroups: [“apps”] apiVersions: [“v1”] operations: [“CREATE”, “UPDATE”] resources: [“deployments”, “daemonsets”, “statefulsets”] validations: - message: “The use of the ‘latest’ tag is not allowed” expression: | object.spec.template.spec.containers.all(container, !container.image.endsWith(“:latest”) && container.image.contains(“:”) ) && ( !has(object.spec.template.spec.initContainers) || object.spec.template.spec.initContainers.all(container, !container.image.endsWith(“:latest”) && container.image.contains(“:”) ) ) We’ll add a humanfriendly message to be shown when the API request has been blocked by this policy. If we don’t include this, a generic message that include the whole expression is shown instead. 🐘 @Marcus@k8s.social | 🌐 MarcusNoble.com | 🦋 @averagemarcus.bsky.social
Example Blocking the use of the ‘latest’ image tag apiVersion: admissionregistration.k8s.io/v1 kind: ValidatingAdmissionPolicy metadata: name: “prevent-latest-image-tag” spec: failurePolicy: Fail matchConstraints: resourceRules: - apiGroups: [“apps”] apiVersions: [“v1”] operations: [“CREATE”, “UPDATE”] resources: [“deployments”, “daemonsets”, “statefulsets”] validations: - message: “The use of the ‘latest’ tag is not allowed” expression: | object.spec.template.spec.containers.all(container, !container.image.endsWith(“:latest”) && container.image.contains(“:”) ) && ( !has(object.spec.template.spec.initContainers) || object.spec.template.spec.initContainers.all(container, !container.image.endsWith(“:latest”) && container.image.contains(“:”) ) ) Finally we have our expression. This is the expression of allowed resources, not those to block. (I keep getting caught out by this 😅 ) 🐘 @Marcus@k8s.social | 🌐 MarcusNoble.com | 🦋 @averagemarcus.bsky.social
Example Blocking the use of the ‘latest’ image tag apiVersion: admissionregistration.k8s.io/v1 kind: ValidatingAdmissionPolicy metadata: name: “prevent-latest-image-tag” spec: failurePolicy: Fail matchConstraints: resourceRules: - apiGroups: [“apps”] apiVersions: [“v1”] operations: [“CREATE”, “UPDATE”] resources: [“deployments”, “daemonsets”, “statefulsets”] validations: - message: “The use of the ‘latest’ tag is not allowed” expression: | object.spec.template.spec.containers.all(container, !container.image.endsWith(“:latest”) && container.image.contains(“:”) ) && ( !has(object.spec.template.spec.initContainers) || object.spec.template.spec.initContainers.all(container, !container.image.endsWith(“:latest”) && container.image.contains(“:”) ) ) object is the incoming resource from the API call. 🐘 @Marcus@k8s.social | 🌐 MarcusNoble.com | 🦋 @averagemarcus.bsky.social
Example Blocking the use of the ‘latest’ image tag apiVersion: admissionregistration.k8s.io/v1 kind: ValidatingAdmissionPolicy metadata: name: “prevent-latest-image-tag” spec: failurePolicy: Fail matchConstraints: resourceRules: - apiGroups: [“apps”] apiVersions: [“v1”] operations: [“CREATE”, “UPDATE”] resources: [“deployments”, “daemonsets”, “statefulsets”] validations: - message: “The use of the ‘latest’ tag is not allowed” expression: | object.spec.template.spec.containers.all(container, !container.image.endsWith(“:latest”) && container.image.contains(“:”) ) && ( !has(object.spec.template.spec.initContainers) || object.spec.template.spec.initContainers.all(container, !container.image.endsWith(“:latest”) && container.image.contains(“:”) ) ) First we check all containers don’t have the latest tag (or no tag specified). 🐘 @Marcus@k8s.social | 🌐 MarcusNoble.com | 🦋 @averagemarcus.bsky.social
Example Blocking the use of the ‘latest’ image tag apiVersion: admissionregistration.k8s.io/v1 kind: ValidatingAdmissionPolicy metadata: name: “prevent-latest-image-tag” spec: failurePolicy: Fail matchConstraints: resourceRules: - apiGroups: [“apps”] apiVersions: [“v1”] operations: [“CREATE”, “UPDATE”] resources: [“deployments”, “daemonsets”, “statefulsets”] validations: - message: “The use of the ‘latest’ tag is not allowed” expression: | object.spec.template.spec.containers.all(container, !container.image.endsWith(“:latest”) && container.image.contains(“:”) ) && ( !has(object.spec.template.spec.initContainers) || object.spec.template.spec.initContainers.all(container, !container.image.endsWith(“:latest”) && container.image.contains(“:”) ) ) Then we check if this resource defines initContainers and if so we check they also don’t use latest. 🐘 @Marcus@k8s.social | 🌐 MarcusNoble.com | 🦋 @averagemarcus.bsky.social
Example Blocking the use of the ‘latest’ image tag Our policy does nothing until we bind it to some conditions. apiVersion: admissionregistration.k8s.io/v1 kind: ValidatingAdmissionPolicyBinding metadata: name: “prevent-latest-image-tag” spec: policyName: “prevent-latest-image-tag” validationActions: [Deny] matchResources: {} 🐘 @Marcus@k8s.social | 🌐 MarcusNoble.com | 🦋 @averagemarcus.bsky.social
Example Blocking the use of the ‘latest’ image tag The name of the policy we have just created. apiVersion: admissionregistration.k8s.io/v1 kind: ValidatingAdmissionPolicyBinding metadata: name: “prevent-latest-image-tag” spec: policyName: “prevent-latest-image-tag” validationActions: [Deny] matchResources: {} 🐘 @Marcus@k8s.social | 🌐 MarcusNoble.com | 🦋 @averagemarcus.bsky.social
Example Blocking the use of the ‘latest’ image tag The action to take when a policy conditions aren’t met. apiVersion: admissionregistration.k8s.io/v1 kind: ValidatingAdmissionPolicyBinding metadata: name: “prevent-latest-image-tag” spec: policyName: “prevent-latest-image-tag” validationActions: [Deny] Note: This is an array as you can matchResources: {} specify both warn and audit together Available options are: Deny, Warn & Audit 🐘 @Marcus@k8s.social | 🌐 MarcusNoble.com | 🦋 @averagemarcus.bsky.social
Example Blocking the use of the ‘latest’ image tag We’re not defining any filtering so this policy applies cluster-wide. apiVersion: admissionregistration.k8s.io/v1 kind: ValidatingAdmissionPolicyBinding metadata: name: “prevent-latest-image-tag” spec: policyName: “prevent-latest-image-tag” validationActions: [Deny] matchResources: {} We could limit our policy to specific namespaces or labels, for example. 🐘 @Marcus@k8s.social | 🌐 MarcusNoble.com | 🦋 @averagemarcus.bsky.social
Example Blocking the use of the ‘latest’ image tag We’re not defining any filtering so this policy applies cluster-wide. apiVersion: admissionregistration.k8s.io/v1 kind: ValidatingAdmissionPolicyBinding metadata: name: “prevent-latest-image-tag” spec: policyName: “prevent-latest-image-tag” validationActions: [Deny] matchResources: namespaceSelector: matchLabels: environment: prod We could limit our policy to specific namespaces or labels, for example. 🐘 @Marcus@k8s.social | 🌐 MarcusNoble.com | 🦋 @averagemarcus.bsky.social
Example Blocking the use of the ‘latest’ image tag apiVersion: apps/v1 kind: Deployment metadata: name: nginx-blocked labels: app: nginx spec: replicas: 1 selector: matchLabels: app: nginx template: metadata: labels: app: nginx spec: containers: - name: nginx image: nginx:latest
kubectl apply -f deployment.yaml The deployments “nginx-blocked” is invalid: : ValidatingAdmissionPolicy ‘prevent-latest-image-tag’ with binding ‘prevent-latest-image-tag’ denied request: The use of the ‘latest’ tag is not allowed Our friendly message
⛔ Blocked! ⛔ 🐘 @Marcus@k8s.social | 🌐 MarcusNoble.com | 🦋 @averagemarcus.bsky.social
More advanced features 🐘 @Marcus@k8s.social | 🌐 MarcusNoble.com | 🦋 @averagemarcus.bsky.social
More advanced features • More context values - Along with object you also have oldObject, namespaceObject & request you can use in your expressions • matchConditions: - name: “exclude-kubelet-requests” expression: “!(“system:nodes” in request.userInfo.groups)” 🐘 @Marcus@k8s.social | 🌐 MarcusNoble.com | 🦋 @averagemarcus.bsky.social
More advanced features • • More context values - Along with object you also have oldObject, namespaceObject & request you can use in your expressions Parameters - make your policies configurable by allowing parameter resources to be applied by the binding (can use any resource type, even a CRD) apiVersion: admissionregistration.k8s… kind: ValidatingAdmissionPolicy metadata: name: “param-example” spec: paramKind: apiVersion: v1 kind: ConfigMap validations: - expression: object.spec.replicas <= params.data.maxReplicas apiVersion: admissionregistration.k… kind: ValidatingAdmissionPolicyBinding metadata: name: “param-example” spec: policyName: “param-example” paramRef: name: “my-parameters” namespace: “default” apiVersion: v1 kind: ConfigMap metadata: name: “my-parameters” namespace: “default” data: maxReplicas: 3 🐘 @Marcus@k8s.social | 🌐 MarcusNoble.com | 🦋 @averagemarcus.bsky.social
More advanced features • • • More context values - Along with object you also have oldObject, namespaceObject & request you can use in your expressions Parameters - make your policies configurable by allowing parameter resources to be applied by the binding (can use any resource type, even a CRD) Variables - reusable CEL expressions to simplify your validation expressions spec: variables: - name: teamLabel expression: “‘team’ in object.spec.metadata.labels ? object.spec.metadata.labels[‘team’] : ‘no-team’” validations: - expression: variables.teamLabel == ‘rel-eng’ 🐘 @Marcus@k8s.social | 🌐 MarcusNoble.com | 🦋 @averagemarcus.bsky.social
More advanced features • • • • More context values - Along with object you also have oldObject, namespaceObject & request you can use in your expressions Parameters - make your policies configurable by allowing parameter resources to be applied by the binding (can use any resource type, even a CRD) Variables - reusable CEL expressions to simplify your validation expressions Audit annotations - Add extra metadata to the audit logs, dynamic values using CEL auditAnnotations: - key: “replica-count” valueExpression: | “‘Deployment spec.replicas set to ’ + string(object.spec.replicas)” { “kind”: “Event”, “apiVersion”: “audit.k8s.io/v1”, “annotations”: { “demo-policy.example.com/replica-count”: “Deployment spec.replicas set to 128” … } 🐘 @Marcus@k8s.social | 🌐 MarcusNoble.com | 🦋 @averagemarcus.bsky.social
More advanced features • • • • • More context values - Along with object you also have oldObject, namespaceObject & request you can use in your expressions Parameters - make your policies configurable by allowing parameter resources to be applied by the binding (can use any resource type, even a CRD) Variables - reusable CEL expressions to simplify your validation expressions Audit annotations - Add extra metadata to the audit logs, dynamic values using CEL Message expressions - Leverage some CEL in your message to have dynamic validation messages messageExpression: “‘object.spec.replicas must be no greater than ’ + string(params.maxReplicas)” 🐘 @Marcus@k8s.social | 🌐 MarcusNoble.com | 🦋 @averagemarcus.bsky.social
So that’s validation covered, what about mutating? Well… 🐘 @Marcus@k8s.social | 🌐 MarcusNoble.com | 🦋 @averagemarcus.bsky.social
MutatingAdmissionPolicies Status: Alpha in v1.32, Beta in v1.34 Introduced in: KEP-3962 • • • • Follows the same idea as ValidatingAdmissionPolicies. Introduces MutatingAdmissionPolicy and MutatingAdmissionPolicyBinding Supports two patch strategies: ApplyConfiguration and JSONPatch Not yet enabled by default, you must enable the following: • MutatingAdmissionPolicy feature gate • admissionregistration.k8s.io/v1beta1 API Previously v1alpha1 in <= v1.33 kind: Cluster apiVersion: kind.x-k8s.io/v1alpha4 name: kind-cluster featureGates: “MutatingAdmissionPolicy”: true runtimeConfig: “admissionregistration.k8s.io/v1beta1”: true Create a Kind cluster with: kind create cluster —config kind-config.yaml 🐘 @Marcus@k8s.social | 🌐 MarcusNoble.com | 🦋 @averagemarcus.bsky.social
Example Injecting proxy values as env vars apiVersion: admissionregistration.k8s.io/v1beta1 kind: MutatingAdmissionPolicy metadata: name: “proxy-values” spec: failurePolicy: Fail reinvocationPolicy: IfNeeded matchConstraints: resourceRules: - apiGroups: [“”] apiVersions: [“v1”] operations: [“CREATE”] resources: [“pods”] mutations: - patchType: “ApplyConfiguration” applyConfiguration: expression: > Object{ spec: Object.spec{ containers: object.spec.containers.map(c, Object.spec.containers.item{ name: c.name, env: [ Object.spec.containers.env{ name: “HTTP_PROXY”, value: “http://proxy.proxy.svc:3128” } ] } ) } } 🐘 @Marcus@k8s.social | 🌐 MarcusNoble.com | 🦋 @averagemarcus.bsky.social
Example Injecting proxy values as env vars Same as ValidatingAdmissionPolicies apiVersion: admissionregistration.k8s.io/v1beta1 kind: MutatingAdmissionPolicy metadata: name: “proxy-values” spec: failurePolicy: Fail reinvocationPolicy: IfNeeded matchConstraints: resourceRules: - apiGroups: [“”] apiVersions: [“v1”] operations: [“CREATE”] resources: [“pods”] mutations: - patchType: “ApplyConfiguration” applyConfiguration: expression: > Object{ spec: Object.spec{ containers: object.spec.containers.map(c, Object.spec.containers.item{ name: c.name, env: [ Object.spec.containers.env{ name: “HTTP_PROXY”, value: “http://proxy.proxy.svc:3128” } ] } ) } } 🐘 @Marcus@k8s.social | 🌐 MarcusNoble.com | 🦋 @averagemarcus.bsky.social
Example Injecting proxy values as env vars Should the policy be re-applied if other mutations are made to the resource? Allowed values are Never and IfNeeded apiVersion: admissionregistration.k8s.io/v1beta1 kind: MutatingAdmissionPolicy metadata: name: “proxy-values” spec: failurePolicy: Fail reinvocationPolicy: IfNeeded matchConstraints: resourceRules: - apiGroups: [“”] apiVersions: [“v1”] operations: [“CREATE”] resources: [“pods”] mutations: - patchType: “ApplyConfiguration” applyConfiguration: expression: > Object{ spec: Object.spec{ containers: object.spec.containers.map(c, Object.spec.containers.item{ name: c.name, env: [ Object.spec.containers.env{ name: “HTTP_PROXY”, value: “http://proxy.proxy.svc:3128” } ] } ) } } 🐘 @Marcus@k8s.social | 🌐 MarcusNoble.com | 🦋 @averagemarcus.bsky.social
Example Injecting proxy values as env vars Replace validations with mutations apiVersion: admissionregistration.k8s.io/v1beta1 kind: MutatingAdmissionPolicy metadata: name: “proxy-values” spec: failurePolicy: Fail reinvocationPolicy: IfNeeded matchConstraints: resourceRules: - apiGroups: [“”] apiVersions: [“v1”] operations: [“CREATE”] resources: [“pods”] mutations: - patchType: “ApplyConfiguration” applyConfiguration: expression: > Object{ spec: Object.spec{ containers: object.spec.containers.map(c, Object.spec.containers.item{ name: c.name, env: [ Object.spec.containers.env{ name: “HTTP_PROXY”, value: “http://proxy.proxy.svc:3128” } ] } ) } } 🐘 @Marcus@k8s.social | 🌐 MarcusNoble.com | 🦋 @averagemarcus.bsky.social
Example Injecting proxy values as env vars Indicates the patch strategy Possible values: ● ApplyConfiguration ● JSONPatch apiVersion: admissionregistration.k8s.io/v1beta1 kind: MutatingAdmissionPolicy metadata: name: “proxy-values” spec: failurePolicy: Fail reinvocationPolicy: IfNeeded matchConstraints: resourceRules: - apiGroups: [“”] apiVersions: [“v1”] operations: [“CREATE”] resources: [“pods”] mutations: - patchType: “ApplyConfiguration” applyConfiguration: expression: > Object{ spec: Object.spec{ containers: object.spec.containers.map(c, Object.spec.containers.item{ name: c.name, env: [ Object.spec.containers.env{ name: “HTTP_PROXY”, value: “http://proxy.proxy.svc:3128” } ] } ) } } 🐘 @Marcus@k8s.social | 🌐 MarcusNoble.com | 🦋 @averagemarcus.bsky.social
Example Injecting proxy values as env vars Either applyConfiguration or jsonPatch depending on the patchType. apiVersion: admissionregistration.k8s.io/v1beta1 kind: MutatingAdmissionPolicy metadata: name: “proxy-values” spec: failurePolicy: Fail reinvocationPolicy: IfNeeded matchConstraints: resourceRules: - apiGroups: [“”] apiVersions: [“v1”] operations: [“CREATE”] resources: [“pods”] mutations: - patchType: “ApplyConfiguration” applyConfiguration: expression: > Object{ spec: Object.spec{ containers: object.spec.containers.map(c, Object.spec.containers.item{ name: c.name, env: [ Object.spec.containers.env{ name: “HTTP_PROXY”, value: “http://proxy.proxy.svc:3128” } ] } ) } } 🐘 @Marcus@k8s.social | 🌐 MarcusNoble.com | 🦋 @averagemarcus.bsky.social
Example Injecting proxy values as env vars Introduction of named types The Object refers to the type of the incoming resource (Pod in this example). Child fields can be used by using Object.[fieldname] and can go multiple levels down as see here with the Object.spec.containers.env apiVersion: admissionregistration.k8s.io/v1beta1 kind: MutatingAdmissionPolicy metadata: name: “proxy-values” spec: failurePolicy: Fail reinvocationPolicy: IfNeeded matchConstraints: resourceRules: - apiGroups: [“”] apiVersions: [“v1”] operations: [“CREATE”] resources: [“pods”] mutations: - patchType: “ApplyConfiguration” applyConfiguration: expression: > Object{ spec: Object.spec{ containers: object.spec.containers.map(c, Object.spec.containers.item{ name: c.name, env: [ Object.spec.containers.env{ name: “HTTP_PROXY”, value: “http://proxy.proxy.svc:3128” } ] } ) } } 🐘 @Marcus@k8s.social | 🌐 MarcusNoble.com | 🦋 @averagemarcus.bsky.social
Example Injecting proxy values as env vars Merge our proxy vars with any existing vars in all containers. As this is an ApplyConfiguration patch type the existing env vars on the object remain untouched unless the same name (HTTP_PROXY) is used, which will be overwritten. Note: We’re missing initContainers and ephermeralContainers here due to limited space. apiVersion: admissionregistration.k8s.io/v1beta1 kind: MutatingAdmissionPolicy metadata: name: “proxy-values” spec: failurePolicy: Fail reinvocationPolicy: IfNeeded matchConstraints: resourceRules: - apiGroups: [“”] apiVersions: [“v1”] operations: [“CREATE”] resources: [“pods”] mutations: - patchType: “ApplyConfiguration” applyConfiguration: expression: > Object{ spec: Object.spec{ containers: object.spec.containers.map(c, Object.spec.containers.item{ name: c.name, env: [ Object.spec.containers.env{ name: “HTTP_PROXY”, value: “http://proxy.proxy.svc:3128” } ] } ) } } 🐘 @Marcus@k8s.social | 🌐 MarcusNoble.com | 🦋 @averagemarcus.bsky.social
Example Injecting proxy values as env vars apiVersion: admissionregistration.k8s.io/v1beta1 kind: MutatingAdmissionPolicyBinding metadata: name: “proxy-values” spec: policyName: “proxy-values” matchResources: {} paramRef: null Nothing too surprising about our binding resource. All properties are the same as Validating except for the lack of a validationActions property. 🐘 @Marcus@k8s.social | 🌐 MarcusNoble.com | 🦋 @averagemarcus.bsky.social
Advanced features • • • • • More context values - Along with object you also have oldObject, namespaceObject & request you can use in your expressions Parameters - make your policies configurable by allowing parameter resources to be applied by the binding (can use any resource type, even a CRD) Variables - reusable CEL expressions to simplify your validation expressions Audit annotations - Add extra metadata to the audit logs, dynamic values using CEL Message expressions - Leverage some CEL in your message to have dynamic validation messages 🐘 @Marcus@k8s.social | 🌐 MarcusNoble.com | 🦋 @averagemarcus.bsky.social
What about the future beyond that? 🔮 🐘 @Marcus@k8s.social | 🌐 MarcusNoble.com | 🦋 @averagemarcus.bsky.social
- All these are speculative / wishful thinking right now • • • • • Generative policies - create new resources based on API requests Resource lookup - get the current state of other resources within the cluster Policy exceptions - disable policies under certain circumstances Policy reports - user friendly view of validation failures, etc. (Maybe OpenReports) More abstractions - e.g. Kyverno already has this with ValidatingPolicy that extends ValidatingAdmissionPolicy and MutatingPolicy that extends MutatingAdmissionPolicy Alpha in Kyverno v1.15 🐘 @Marcus@k8s.social | 🌐 MarcusNoble.com | 🦋 @averagemarcus.bsky.social
Summary • Time to start replacing those risky webhooks with in-process policies. • v1.30 for GA release of ValidatingAdmissionPolicies for safe validation logic. • v1.32 for alpha release of MutatingAdmissionPolicies. (Must be enabled) • v1.34 for beta release of MutatingAdmissionPolicies. (Must be enabled) • More abstractions, generative policies and API lookups hopefully coming in the future. • Examples of common use-case policies: github.com/AverageMarcus/common-admission-policies WIP, Contributions welcome 🐘 @Marcus@k8s.social | 🌐 MarcusNoble.com | 🦋 @averagemarcus.bsky.social
Slides and resources available at: https://go-get.link/containerdays25 Thoughts, comments and feedback: https://go-get.link/containerdays25-feedback Thank You! 🐘 @Marcus@k8s.social | 🌐 MarcusNoble.com | 🦋 @averagemarcus.bsky.social
Dynamic admission controllers have long played a pivotal role in enhancing the robustness and adaptability of clusters. For instance, ValidatingWebhookConfiguration empowers users to implement intricate and finely-tuned access controls beyond the capabilities of RBAC and MutatingWebhookConfiguration provides advanced defaulting logic for all resource types. However, this capability often comes at a price – the ease with which they can be misconfigured, potentially leading to cluster disruption and downtime.
Historically, we’ve accepted this fragility as an inevitable trade-off for greater control over our clusters. But that stops now!
Enter CEL-based, in-process Admission Policies!
In this talk we’ll take a look at what makes ValidatingAdmissionPolicies and MutatingAdmissionPolicies a safer choice for your admission logic, we will dive into the features and limitations and will also draw comparisons with their webhook-based alternatives, highlighting the problems they solve. Finally, we’ll walkthrough how you can begin leveraging them today and take a look at what might be coming in the future.
Resources
The following resources were mentioned during the presentation or are useful additional information.
-
KEP-3488: CEL for Admission Control
Proposal for CEL based admission control (ValidatingAdmissionPolicy)
-
CEL in Kubewarden
-
OpenReports
An open standard for policy engine reporting.
-
common-admission-policies repo
-
Kyverno Validating Policy
-
Kyverno Mutating Policy
-
Kubernetes: Admission Control
A fantastic tutorial covering built-in admission controllers, webhooks and policies.
-
[KEP 3488] Promote ValidatingAdmissionPolicy to GA
Issue tracking the promotion of ValidatingAdmissionPolicy to GA, currently targetting v1.30
-
Validating Admission Policy Documentation
The official documentation for using ValidatingAdmissionPolicy
-
KEP-3962: Mutating Admission Policies
Proposal introducing the concept of MutatingAdmissionPolicy
-
MutatingAdmissionPolicy implementation PR
The work in progress PR adding the functionality for MutatingAdmissionPolicy
-
Webhooks - What’s the worst that could happen?
My talk from Rejekts in Chicago about the dangers of admission controller webhooks
-
Common Expression Language in Kubernetes
The official Kubernetes documentation on CEL, including the custom functions available.
-
Kyverno - Using CEL expressions
The documentation for using CEL with Kyverno policies.
-
Tekton - CEL expressions
The documentation for using CEL in Tekton Triggers.
-
cel-go library
The CEL Go library
-
CEL Playground
An online, interactive tool for building and testing CEL expressions.
