Kubernetes providers - Notes

Quick comments to the different Kubernetes providers or Kubernetes distributions.

K3s

Great for the first experiments with self-hosted Kubernetes. Low HW requirements, easy installation.
Contains LoadBalancer ServiceLB (former Klipper).
Contains Ingress Controller Traefik. It does not play well with Domino Verse (with a simple forwarding rule), so I replaced it with Ingress NGINX. (How to disable Traefik on K3s.)
The default StorageCless is local-path. It does not support volume expansion. You can deploy other storage provider that supports volume expansion, like Longhorn.

Good distro for hosting Kubernetes on-premises. Easy installation.
RKE2 deploys Ingress NGINX as an Ingress Controller.
No storage class is defined out of the box; you need to install your CSI. Good choice could be Longhorn because, created by the same company that maintains RKE2.
Load balancers are not provided. You can deploy one of the solutions designed for on-prem, like MetalLB.

Kubernetes managed service, by Amazon.
For a quick cluster creation, use eksctl tool.
EKS does not deploy CSI driver by default. YOu have to install it manually.
More info: Amazon EBS CSI driver
The default StorageCless is gp2. The storageClass is not enabled for expansion by default.
You can enable it with this command:
```
kubectl patch sc gp2 -p '{"allowVolumeExpansion": true}'
```
You can deploy EKS native Ingress Controller, or you can install Ingress NGINX controler.
AWS native load balancers expose services as a hostname. Point your custom domain names (alpha.space.demo) to these hostnames using CNAME records in your DNS zone.

Kubernetes managed service, by Microsoft.
Several StorageClesses created in the cluster, of of the box.
You can use managed-csi (default) or managed-csi-premium, for example.
All classes are enabled for Volume Expansion.
There is a simple Ingress controller, that could be enabled by adding addon “http_application_routing”. The add-on is being retired, so you should not use it. You can use a new native Web Application Routing as an ingress controller.
Or you can deploy a classic Ingress NGINX controller.
IMPORTANT: You need to add the proper annotation to the Ingress Controller (using /healthz for a health check, as described in the link above), otherwise Azure load balancer would think that backend server is not ready and will not route traffic to Domino HTTP.
Azure native load balancers expose services as an IP address. Point your custom domain names (alpha.space.demo) to these IPs using A records in your DNS zone.

Kubernetes managed service, by Microsoft.
Several StorageClesses created in the cluster, of of the box.
You can use standard-rwo (default), _standard or premium-rwo.
GKE offers its Ingress Controler (info), but I deployed Ingress NGINX which works fine.

A good alternative to big Kubernetes providers.
No Ingress Controller is installed by default; Linode suggests using Ingress NGINX.
CSI is deployed by Linode, storage classes: linode-block-storage-retain, linode-block-storage.
Load balancers are provided by Linode.