Kubernetes使用 OpenEBS 实现 Local PV 动态持久化存储

仅限开发测试使用

OpenEBS安装条件

# 安装 iscsi
yum install iscsi-initiator-utils -y

# 查看 InitiatorName 是否正常配置
cat /etc/iscsi/initiatorname.iscsi

# 启动查看状态
systemctl enable --now iscsid
systemctl start iscsid.service
systemctl status iscsid.service

openebs quickstart

安装方式

方式一 helm


helm repo add openebs https://openebs.github.io/charts
helm repo update
helm install openebs --namespace openebs openebs/openebs --create-namespace

方式二 kubectl

kubectl apply -f https://openebs.github.io/charts/openebs-operator.yaml

helm pull openebs/openebs
tar xf openebs-3.3.1.tgz 
cd openebs
# 修改存储目录
vim values.yaml
    66	  baseDir: "/data/openebs"
   108	  basePath: "/data/openebs/local"
   252	    path: "/data/openebs/sparse"
   406	  defaultStoragePath: "/data/openebs"

helm install openebs -n openebs . --create-namespace

查看

[root@ha openebs]# kubectl get storageclass
NAME               PROVISIONER        RECLAIMPOLICY   VOLUMEBINDINGMODE      ALLOWVOLUMEEXPANSION   AGE
openebs-device     openebs.io/local   Delete          WaitForFirstConsumer   false                  6m32s
openebs-hostpath   openebs.io/local   Delete          WaitForFirstConsumer   false                  6m32s

测试

接下来我们创建一个 PVC 资源对象，Pods 使用这个 PVC 就可以从 OpenEBS 动态 Local PV Provisioner 中请求 Hostpath Local PV 了。

创建pvc

kubectl apply -f https://openebs.github.io/charts/examples/local-hostpath/local-hostpath-pvc.yaml

# local-hostpath-pvc.yaml
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: local-hostpath-pvc
spec:
  storageClassName: openebs-hostpath
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 5Gi

[root@ha openebs]# kubectl get pvc
NAME                 STATUS    VOLUME   CAPACITY   ACCESS MODES   STORAGECLASS       AGE
local-hostpath-pvc   Pending                                      openebs-hostpath   5s

我们可以看到这个 PVC 的状态是 Pending，这是因为对应的 StorageClass 是延迟绑定模式，所以需要等到 Pod 消费这个 PVC 后才会去绑定，接下来我们去创建一个 Pod 来使用这个 PVC。

运行pvc pod

kubectl apply -f https://openebs.github.io/charts/examples/local-hostpath/local-hostpath-pod.yaml

# local-hostpath-pod.yaml
apiVersion: v1
kind: Pod
metadata:
  name: hello-local-hostpath-pod
spec:
  volumes:
  - name: local-storage
    persistentVolumeClaim:
      claimName: local-hostpath-pvc
  containers:
  - name: hello-container
    image: busybox
    command:
       - sh
       - -c
       - 'while true; do echo "`date` [`hostname`] Hello from OpenEBS Local PV." >> /mnt/store/greet.txt; sleep $(($RANDOM % 5 + 300)); done'
    volumeMounts:
    - mountPath: /mnt/store
      name: local-storage

直接创建这个 Pod：

[root@ha openebs]# kubectl get pods hello-local-hostpath-pod  
NAME                       READY   STATUS    RESTARTS   AGE
hello-local-hostpath-pod   1/1     Running   0          20s

[root@ha openebs]# kubectl get pvc
NAME                 STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS       AGE
local-hostpath-pvc   Bound    pvc-8ad71844-85d8-49e6-86bc-031d193e48c1   5G         RWO            openebs-hostpath   3m4s

查看PVC详情

[root@ha openebs]# kubectl get pv pvc-8ad71844-85d8-49e6-86bc-031d193e48c1 -o yaml
apiVersion: v1
kind: PersistentVolume
metadata:
  annotations:
    pv.kubernetes.io/provisioned-by: openebs.io/local
  creationTimestamp: "2022-11-20T07:23:40Z"
  finalizers:
  - kubernetes.io/pv-protection
  labels:
    openebs.io/cas-type: local-hostpath
  name: pvc-8ad71844-85d8-49e6-86bc-031d193e48c1
  resourceVersion: "143666"
  uid: 0f4b9c6b-408a-40b1-bca1-4430b028b66d
spec:
  accessModes:
  - ReadWriteOnce
  capacity:
    storage: 5G
  claimRef:
    apiVersion: v1
    kind: PersistentVolumeClaim
    name: local-hostpath-pvc
    namespace: default
    resourceVersion: "143626"
    uid: 8ad71844-85d8-49e6-86bc-031d193e48c1
  local:
    fsType: ""
    path: /data/openebs/local/pvc-8ad71844-85d8-49e6-86bc-031d193e48c1
  nodeAffinity:
    required:
      nodeSelectorTerms:
      - matchExpressions:
        - key: kubernetes.io/hostname
          operator: In
          values:
          - node2
  persistentVolumeReclaimPolicy: Delete
  storageClassName: openebs-hostpath
  volumeMode: Filesystem
status:
  phase: Bound

我们可以看到这个自动生成的 PV 和我们前面自己手动创建的 Local PV 基本上是一致的，和 node2 节点是亲和关系，本地数据目录位于 /data/openebs/local/pvc-8ad71844-85d8-49e6-86bc-031d193e48c1 下面。

接着我们来验证下 volume 数据，前往 node2 节点查看下上面的数据目录中的数据：

[root@node2 ~]# ls /data/openebs/local/pvc-8ad71844-85d8-49e6-86bc-031d193e48c1
greet.txt
[root@node2 ~]# cat /data/openebs/local/pvc-8ad71844-85d8-49e6-86bc-031d193e48c1/greet.txt
Sun Nov 20 07:23:44 UTC 2022 [hello-local-hostpath-pod] Hello from OpenEBS Local PV.

可以看到 Pod 容器中的数据已经持久化到 Local PV 对应的目录中去了。但是需要注意的是 StorageClass 默认的数据回收策略是 Delete，所以如果将 PVC 删掉后数据会自动删除，我们可以 Velero 这样的工具来进行备份还原。

使用 OpenEBS 实现 Local PV 动态持久化存储

Kubernetes使用 OpenEBS 实现 Local PV 动态持久化存储

Openebs

OpenEBS安装条件

安装方式

查看

测试

CATALOG

FEATURED TAGS

FRIENDS