Istio Service Mesh

Istio is a service mesh for Kubernetes. This guide describes the installation in a Gardener shoot cluster on the noris Sovereign Cloud (nSC). The concrete configuration, for example for traffic management, mTLS, tracing, or canary releases, is up to you as a customer.

Expertise recommended

A service mesh adds another abstraction layer to the network. We recommend testing changes beforehand and familiarising yourself with Istio.

The guide follows the upstream tutorial Gardener yourself a Shoot with Istio, custom Domains, and Certificates.

Prerequisites

Network Configuration (Cilium)

For Istio to work with the Cilium CNI, two adjustments to the shoot’s network configuration are required. Add the following to your shoot declaration:

apiVersion: core.gardener.cloud/v1beta1
kind: Shoot
...
spec:
  networking:
    type: cilium
    providerConfig:
      apiVersion: cilium.networking.extensions.gardener.cloud/v1alpha1
      kind: NetworkConfig
      bpfSocketLBHostnsOnly:
        enabled: false
      cni:
        exclusive: false

Certificates and DNS

For incoming traffic, Istio can act as an ingress gateway. Alternatively, the Kubernetes Gateway API can be used, which Istio also supports. Both benefit from TLS certificates and DNS entries that Gardener manages via the shoot-cert-service and shoot-dns-service extensions. These are described in the Certificates section of the Gardener documentation. Issuance via Let’s Encrypt makes usage convenient and is recommended, but not strictly required. Set up both extensions as described there before proceeding with the Istio installation.

The issuer name assigned there (in the example custom-issuer) is referenced in the next step during the Istio installation.

Install Istio

Istio offers numerous configuration options. This guide configures only the ingress gateway functionality as an example. Further options are described in the Istio documentation.

Install istioctl following the official installation guide.

Then set the environment variables for your domain and issuer. The domain corresponds to a subdomain of your shoot cluster:

export domainname="istio-mesh.g5lxqfthcn.k8s.nsc02.noris.cloud"
export issuer="custom-issuer"

Install Istio with the default profile and annotate the ingress gateway service so that Gardener handles DNS and certificates:

cat <<EOF | istioctl install -y -f -
apiVersion: install.istio.io/v1alpha1
kind: IstioOperator
spec:
  profile: default
  components:
    ingressGateways:
    - name: istio-ingressgateway
      enabled: true
      k8s:
        serviceAnnotations:
          cert.gardener.cloud/issuer: "${issuer}"
          cert.gardener.cloud/secretname: wildcard-tls
          dns.gardener.cloud/class: garden
          dns.gardener.cloud/dnsnames: "${domainname}"
          dns.gardener.cloud/ttl: "120"
EOF

Certificate secret namespace

The certificate secret must reside in the same namespace as the gateway (istio-system). Placing it elsewhere is a common source of errors.

Verify the Installation

Check whether DNS entries and certificates have been created successfully:

kubectl -n istio-system describe service istio-ingressgateway

The events section should show messages like certificate ready and dns entry active. Certificate issuance via Let’s Encrypt takes about 70 seconds, so wait for this period to elapse before verifying.

Verify the DNS entry:

dig ${domainname}

Further steps, such as exposing an application via Istio Gateway and VirtualService resources, are described in the upstream tutorial under Ingress at Your Service.

For configuration issues, istioctl analyze can help.

Remove Istio

To remove Istio again:

istioctl uninstall --purge

Implementation Example: mTLS

The following example demonstrates mTLS between namespaces. It creates three namespaces: foo and bar with Istio injection, legacy without a sidecar. In foo, PeerAuthentication enforces STRICT mTLS.

Save the manifest as mtls-demo.yaml and apply it:

kubectl apply -f mtls-demo.yaml

---
# Namespace: foo (with Istio injection)
apiVersion: v1
kind: Namespace
metadata:
  name: foo
  labels:
    istio-injection: enabled
---
# Namespace: bar (with Istio injection)
apiVersion: v1
kind: Namespace
metadata:
  name: bar
  labels:
    istio-injection: enabled
---
# Namespace: legacy (without Istio injection)
apiVersion: v1
kind: Namespace
metadata:
  name: legacy
---
# web in foo
apiVersion: v1
kind: ServiceAccount
metadata:
  name: web
  namespace: foo
---
apiVersion: v1
kind: Service
metadata:
  name: web
  namespace: foo
spec:
  ports:
  - port: 80
    name: http
  selector:
    app: web
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: web
  namespace: foo
spec:
  replicas: 1
  selector:
    matchLabels:
      app: web
  template:
    metadata:
      labels:
        app: web
    spec:
      serviceAccountName: web
      containers:
      - name: nginx
        image: nginx:alpine
        ports:
        - containerPort: 80
---
# sleep in foo
apiVersion: v1
kind: ServiceAccount
metadata:
  name: sleep
  namespace: foo
---
apiVersion: v1
kind: Service
metadata:
  name: sleep
  namespace: foo
spec:
  ports:
  - port: 80
    name: http
  selector:
    app: sleep
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: sleep
  namespace: foo
spec:
  replicas: 1
  selector:
    matchLabels:
      app: sleep
  template:
    metadata:
      labels:
        app: sleep
    spec:
      serviceAccountName: sleep
      containers:
      - name: sleep
        image: curlimages/curl:latest
        command: ["/bin/sleep", "infinity"]
---
# sleep in bar
apiVersion: v1
kind: ServiceAccount
metadata:
  name: sleep
  namespace: bar
---
apiVersion: v1
kind: Service
metadata:
  name: sleep
  namespace: bar
spec:
  ports:
  - port: 80
    name: http
  selector:
    app: sleep
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: sleep
  namespace: bar
spec:
  replicas: 1
  selector:
    matchLabels:
      app: sleep
  template:
    metadata:
      labels:
        app: sleep
    spec:
      serviceAccountName: sleep
      containers:
      - name: sleep
        image: curlimages/curl:latest
        command: ["/bin/sleep", "infinity"]
---
# sleep in legacy (no sidecar)
apiVersion: v1
kind: ServiceAccount
metadata:
  name: sleep
  namespace: legacy
---
apiVersion: v1
kind: Service
metadata:
  name: sleep
  namespace: legacy
spec:
  ports:
  - port: 80
    name: http
  selector:
    app: sleep
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: sleep
  namespace: legacy
spec:
  replicas: 1
  selector:
    matchLabels:
      app: sleep
  template:
    metadata:
      labels:
        app: sleep
    spec:
      serviceAccountName: sleep
      containers:
      - name: sleep
        image: curlimages/curl:latest
        command: ["/bin/sleep", "infinity"]
---
# PeerAuthentication: STRICT mTLS in namespace foo
apiVersion: security.istio.io/v1beta1
kind: PeerAuthentication
metadata:
  name: default
  namespace: foo
spec:
  mtls:
    mode: STRICT

Tests

Test 1: mTLS from bar to foo, expecting 200:

kubectl exec -n bar deploy/sleep -- curl -s -o /dev/null -w "%{http_code}" http://web.foo.svc.cluster.local/

Test 2: Plaintext from legacy to foo is blocked by STRICT mTLS, expecting 000:

kubectl exec -n legacy deploy/sleep -- curl -s -o /dev/null -w "%{http_code}" http://web.foo.svc.cluster.local/

Test 3: Switch to PERMISSIVE, then plaintext works too:

kubectl patch peerauthentication default -n foo --type=json -p='[{"op":"replace","path":"/spec/mtls/mode","value":"PERMISSIVE"}]'

The curl command from test 2 should now return 200 instead of 000.