Permissions and policies
Inside your namespace you have a lot of freedom, but not unlimited freedom. Two mechanisms draw the line, and they announce themselves differently:
- RBAC decides which resources you may touch at all. A violation ends in
Forbidden. - Platform rules decide whether a specific manifest is accepted. A violation ends with your
kubectl applybeing rejected.
What RBAC grants
"Full" below means get, list, watch, create, update, patch and delete. Where a row says something narrower, that is the whole grant.
| API group | Resources | Access |
|---|---|---|
apps | deployments, statefulsets, replicasets | Full |
apps | deployments/scale, statefulsets/scale | get, update, patch |
"" (core, objects) | services, configmaps, secrets, serviceaccounts, persistentvolumeclaims | Full |
"" (core, runtime) | pods, pods/log, pods/exec, pods/attach, pods/portforward, events, endpoints | Full |
"" (core, quotas) | resourcequotas, limitranges | Read-only |
events.k8s.io | events | Read-only |
batch | jobs, cronjobs | Full |
autoscaling | horizontalpodautoscalers | Full |
autoscaling.k8s.io | verticalpodautoscalers | Full |
policy | poddisruptionbudgets | Full |
gateway.networking.k8s.io | httproutes, grpcroutes | Full |
gateway.nginx.org | clientsettingspolicies | Full |
postgresql.cnpg.io | clusters, backups, scheduledbackups, poolers, databases, publications, subscriptions, imagecatalogs | Full |
k8s.mariadb.com | mariadbs, databases, users, grants, connections, backups, restores, sqljobs, maxscales | Full |
dragonflydb.io | dragonflies | Full |
keda.sh | scaledobjects, scaledjobs, triggerauthentications | Full |
metrics.k8s.io | pods | get, list |
All of it applies inside your namespace only.
Two consequences of the read-only rows are worth stating outright:
- You cannot change your own quota. You can read it, which is what
kubectl get resourcequotaandkubectl describe resourcequotaare for, but raising it is a plan change, see Billing and quotas. - You cannot change the namespace defaults applied to containers that declare no resources.
This table describes what you can do with kubectl. If you deploy by pointing your namespace at a Git repository instead, a few kinds are not accepted through that route even though they appear here, currently vertical pod autoscalers and client settings policies.
Everything else is denied
Anything not in the table is not allowed. That notably covers nodes, namespaces, cluster roles and CRD management. Those commands end in Forbidden, see Access and kubectl.
You cannot list the storage classes
kubectl get storageclass is denied as well, so you cannot discover the available classes from inside the cluster. There are two, and they are named and described under Storage.
Platform rules
On top of RBAC, the platform checks every manifest against a fixed set of rules. They are hard rules: a manifest that breaks one is rejected.
| Rule | What to do instead |
|---|---|
| Volume types defined inline in the pod are not accepted | Request storage through a PersistentVolumeClaim |
Services of type NodePort, LoadBalancer and ExternalName are not accepted, and neither is spec.externalIPs | Stay on ClusterIP and publish through an HTTPRoute |
You cannot write Certificate resources yourself | Let the certificate come from the annotation on your HTTPRoute |
| An HTTPRoute must declare at least one hostname | A route with none would attach as a catch-all on the shared gateway |
| Hostnames must be under your namespace subdomain, or added to your namespace in the portal | See Gateway API |
| A pod may not choose its own node, and may not carry a toleration that matches everything | Let the scheduler place it. A blanket operator: Exists toleration counts, so do not copy one in |
| A pod may not set a priority class of its own | The right one is applied for you |
A ClientSettingsPolicy may not raise the request body size above 512 MiB, or set it to zero | Set an explicit size within that bound |
| The HTTP scale-to-zero resources are managed for you | Use the itsh.dev/scale-to-zero annotation, see Autoscaling |
On the free tier, nfs-rwx is the only permitted storage class | See Free tier |
Sandboxed runtime
Your workloads run in a sandboxed runtime. It is applied automatically and you cannot opt out, and for your own workloads there is nothing to notice: it is set for you even if you say nothing.
It surfaces in one place. Managed PostgreSQL pods are the exception to it, and because kubectl exec is only permitted into sandboxed pods, you cannot get a shell in one. See Databases.
Inline volumes: use a PVC
Volume types defined directly in the pod are rejected. That explicitly includes an nfs volume in the pod spec:
# Rejected
spec:
volumes:
- name: data
nfs:
server: nfs.example.com
path: /export/dataIf you need shared storage, request it through a PVC on the nfs-rwx class, see Storage:
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: data
spec:
accessModes:
- ReadWriteMany
storageClassName: nfs-rwx
resources:
requests:
storage: 10GiNo NodePort, no LoadBalancer
The way out is exclusively an HTTPRoute on the shared gateway, see Gateway API. A service with type: LoadBalancer or type: NodePort is rejected; ClusterIP is the type you want.
Certificates come from the route
Creating a Certificate resource yourself is rejected. TLS comes from the cert-manager.io/cluster-issuer annotation on your HTTPRoute, without you doing anything else.
What a rejection looks like
When a manifest breaks one of the rules, kubectl apply answers roughly like this:
Error from server: error when creating "app.yaml": admission webhook denied the
request: <reason naming the rule that was broken>Two real examples are on Gateway API, for a hostname that is not yours, and on Databases, for exec into a PostgreSQL pod.
The exact wording depends on the rule. What matters is the difference from a syntax error: denied the request means your YAML is fine and a rule fired. There is nothing to fix in the manifest, you need a different approach, and the reason that follows tells you which rule it was.
What's next
- The API said no if something was refused
- Gateway API for the hostname rules in full