Kubernetes DiskPress触发测试

  1. 准备一个磁盘大小为100GB节点

  2. 进入该节点上的某个Pod，写入85GB大小的文件

    1
    2
    kubectl exec -it nginx-deployment-basic-599bb7fd55-pl7mf -- bash
    dd if=/dev/zero of=/tmp/test.log bs=100M count=850

  3. 当磁盘大小阈值超过90GB的时候，Pod被驱逐

    1
    2
    3
    4
    5
    # kubectl describe pod nginx-deployment-basic-599bb7fd55-pl7mf
    ...
    Events:
      Warning  Evicted    8m43s  kubelet            The node was low on resource: ephemeral-storage. Container nginx was using 74147524Ki, which exceeds its request of 0.
      Normal   Killing    8m43s  kubelet            Stopping container nginx

  4. 查看集群事件，发现Node 产生DiskPressure

    1
    2
    kubectl get event
    3m55s       Normal    NodeHasDiskPressure node/xxx Node xxxx status is now: NodeHasDiskPressure

  5. 短时间内，由于节点被自动打上了node.kubernetes.io/disk-pressure taint，新的Pod无法立即被调度到该节点上，需要等一小段时间，k8s清理掉存储并把node的taint去掉之后才能重新调度

1
3m11s       Warning   FailedScheduling          pod/nginx-deployment-basic-599bb7fd55-7zwvq    0/1 nodes are available: 1 node(s) had taint {node.kubernetes.io/disk-pressure: }, that the pod didn't tolerate.
  1. 查看节点的磁盘容量监控，Pod被驱逐后，容器数据也跟着一起删掉，磁盘容量恢复，节点正常

在这里插入图片描述