CoreDNS
DNS names for Services and Pods, how a Pod's resolv.conf is built, dnsPolicy and dnsConfig, the CoreDNS Corefile and its plugins, forwarding a private zone, and a step-by-step for debugging resolution.
Exam tasks: 3.6 (use CoreDNS), with 5.5 when names don't resolve
The decision: which name should the client use, which resolver answers it (CoreDNS's kubernetes plugin, a
forwarded upstream, or the node's resolver), and where do you change that?
What's running
| Piece | Name in a kubeadm cluster |
|---|---|
| Deployment | coredns in kube-system (Pods labelled k8s-app=kube-dns) |
| Service | kube-dns in kube-system. Its ClusterIP (often 10.96.0.10) is what Pods use as nameserver |
| Config | ConfigMap coredns, key Corefile |
| Kubelet side | clusterDNS and clusterDomain in /var/lib/kubelet/config.yaml |
Names you can resolve
| Record | Name | Answer |
|---|---|---|
| Service | pricing.sales.svc.cluster.local | The ClusterIP |
| Headless Service | ledger-peers.sales.svc.cluster.local | One A record per ready Pod |
| StatefulSet Pod behind headless Service | ledger-0.ledger-peers.sales.svc.cluster.local | That Pod's IP |
| Named port (SRV) | _http._tcp.pricing.sales.svc.cluster.local | Port number and target |
| Any Pod by IP | 10-42-1-14.sales.pod.cluster.local | 10.42.1.14 |
| ExternalName Service | billing-api.sales.svc.cluster.local | CNAME to the external name |
- A Pod gets the
<hostname>.<subdomain>.<ns>.svc.cluster.localname only when it setsspec.hostnameandspec.subdomain, and a headless Service with the same name as the subdomain exists in its namespace. StatefulSets do this for you throughserviceName. cluster.localis the default cluster domain. A task may use a different one, so checkclusterDomain.
How short names work
A Pod in namespace sales gets roughly this /etc/resolv.conf:
nameserver 10.96.0.10
search sales.svc.cluster.local svc.cluster.local cluster.local
options ndots:5pricingresolves via the first search domain (same namespace).pricing.opsresolves viasvc.cluster.local(another namespace).- With
ndots:5, any name with fewer than five dots is tried against the search list first, so external names likeapi.partner.netcost several failed lookups before the real one. A trailing dot (api.partner.net.) skips the search list.
Exam signal
"Reach Service X in namespace Y from a Pod in namespace Z" is answered with X.Y (or the full
X.Y.svc.cluster.local). Plain X only works from inside namespace Y.
dnsPolicy and dnsConfig
dnsPolicy | Resolver the Pod uses |
|---|---|
ClusterFirst (default) | CoreDNS. Non-cluster names are forwarded upstream by CoreDNS |
Default | The node's resolv.conf. Cluster names don't resolve. (Yes, Default isn't the default) |
ClusterFirstWithHostNet | CoreDNS for a hostNetwork: true Pod, which would otherwise fall back to Default |
None | Only what you put in dnsConfig |
spec:
dnsPolicy: None
dnsConfig:
nameservers: ["10.96.0.10"]
searches: ["sales.svc.cluster.local", "svc.cluster.local", "cluster.local"]
options:
- name: ndots
value: "2"For a few fixed names inside one Pod, spec.hostAliases adds entries to the Pod's /etc/hosts without touching DNS.
The Corefile
The kubeadm default, trimmed to the lines you'll touch (see k -n kube-system get cm coredns -o yaml for the rest):
.:53 { # server block: every zone, port 53
errors # log errors to stdout
kubernetes cluster.local in-addr.arpa ip6.arpa {
pods insecure # answer a-b-c-d.<ns>.pod names
fallthrough in-addr.arpa ip6.arpa # pass unknown reverse lookups on
}
forward . /etc/resolv.conf # non-cluster names go to the node's resolvers
cache 30
loop
reload # re-read the Corefile when the ConfigMap changes
}The full default also carries health, ready (probe endpoints), prometheus :9153 (metrics) and loadbalance
(shuffles A records).
| Plugin | Does |
|---|---|
kubernetes | Answers cluster names from the API. pods insecure enables the a-b-c-d.ns.pod records |
forward | Sends other queries upstream, here to the nameservers in the CoreDNS Pod's resolv.conf (the node's) |
cache | Caches answers for up to 30 seconds |
reload | Picks up Corefile changes without a restart (after the ConfigMap reaches the Pods) |
loop | Detects forwarding loops and stops CoreDNS (CrashLoopBackOff) rather than looping |
hosts, rewrite | Static records, and rewriting one name to another |
log | Logs every query. Add it temporarily when debugging |
Forwarding a private zone (stub domain)
Add a server block for the zone. Don't edit the forward . line, which would send all external lookups there.
corp.lan:53 {
errors
cache 30
forward . 10.20.0.53
}k -n kube-system edit configmap coredns # add the block next to .:53
k -n kube-system rollout restart deploy coredns # or wait for reload; restarting is faster and certain
k -n kube-system logs -l k8s-app=kube-dns --tail=20 # syntax errors show hereA typo in the Corefile
A broken Corefile can make CoreDNS crash on restart, and then every lookup in the cluster fails. After any
edit, watch the Pods come back Ready and read their logs before moving on. Keep a copy:
k -n kube-system get cm coredns -o yaml > coredns-backup.yaml.
Debugging a lookup
k run dns --rm -it --restart=Never --image=registry.k8s.io/e2e-test-images/jessie-dnsutils:1.3 -- \
nslookup pricing.sales
k exec -n sales deploy/web -- cat /etc/resolv.conf # right nameserver and search list?
k -n kube-system get pods -l k8s-app=kube-dns # Running and Ready?
k -n kube-system get endpointslices -l kubernetes.io/service-name=kube-dns
k -n kube-system logs -l k8s-app=kube-dns| Symptom | Likely cause |
|---|---|
Every name fails, connection timed out; no servers could be reached | CoreDNS Pods down, kube-dns Service has no endpoints, or an egress NetworkPolicy blocks port 53 |
| Cluster names fail, internet names work | Pod has dnsPolicy: Default, or wrong clusterDomain |
| Internet names fail, cluster names work | Upstream in forward unreachable, or the node's resolv.conf is wrong |
NXDOMAIN for a Service | Wrong namespace in the name, or the Service doesn't exist |
Scenarios
The search list starts with the Pod's own namespace, so a bare name only finds Services in web. inventory.stock
resolves via the svc.cluster.local search domain. Default would remove cluster DNS altogether. forward sends
queries to upstream servers, not to another namespace. A NetworkPolicy affects packets, not name resolution.
A zone-specific server block sends only build.internal queries to the company server. Changing the main forward line would send every external lookup there. dnsPolicy Default would break cluster names. hostAliases needs every hostname listed by hand in every Pod.
The loop plugin stops CoreDNS when queries come back to itself, typically because forward targets the kube-dns IP
or a node resolver such as 127.0.0.53 that points back into the cluster. Cache TTLs don't cause loops. A Service IP
change wouldn't produce a loop error, and the ConfigMap must be in kube-system to be mounted at all.
Drill
kubectl config use-context lab-dns
- Configure CoreDNS so names in
corp.lanare resolved by10.20.0.53, without changing how other names resolve. - In namespace
sales, find the IP thatpricing.sales.svc.cluster.localresolves to and write it to/opt/answers/pricing-ip.txt.
k -n kube-system get cm coredns -o yaml > /tmp/coredns-backup.yaml
k -n kube-system edit cm corednsAdd a second server block inside the Corefile key, after the closing brace of .:53:
corp.lan:53 {
errors
cache 30
forward . 10.20.0.53
}k -n kube-system rollout restart deploy coredns
k -n kube-system rollout status deploy coredns
k -n kube-system logs -l k8s-app=kube-dns --tail=10 # no plugin errors
k -n sales run dns --rm -i --restart=Never --image=registry.k8s.io/e2e-test-images/jessie-dnsutils:1.3 -- \
dig +short pricing.sales.svc.cluster.local > /opt/answers/pricing-ip.txt
cat /opt/answers/pricing-ip.txt
k -n sales get svc pricing -o jsonpath='{.spec.clusterIP}{"\n"}' # should match
k run dns2 --rm -it --restart=Never --image=registry.k8s.io/e2e-test-images/jessie-dnsutils:1.3 -- nslookup host1.corp.lanIf kubectl run adds a deletion message to the output file, write the IP from the get svc command instead
or trim the file afterwards.
Further reading
Ingress
Ingress resources and controllers, IngressClass and the default class, host and path rules, Exact vs Prefix vs ImplementationSpecific path types, TLS Secrets, and kubectl create ingress.
Domain 4 · Storage
10% of the exam. Creating PersistentVolumes, PersistentVolumeClaims and StorageClasses, getting a claim to bind, and mounting it into a Pod.