8 min read
- Install Konvoy on AWS
- Default settings to create 3x t3.large master and 4x t3.xlarge worker instances (total 7)
- Install a demo environment on your laptop using Docker running Konvoy provision --provisioner=docker parameter (optional)
- CentOS 7
- jq
- AWS Command Line Interface (AWS CLI)
- kubectl v1.15.0 or newer
- Konvoy
- OpenEBS
- Elasticsearch, Kibana, Prometheus, Velero (or any other stateful workload)
$ sudo apt-get update && sudo apt-get install awscli jq -y
$ aws configure
kubectl: $ curl -LO https://storage.googleapis.com/kubernetes-release/release/$(curl -s https://storage.googleapis.com/kubernetes-release/release/stable.txt)/bin/linux/amd64/kubectl $ chmod +x ./kubectl $ sudo mv ./kubectl /usr/local/bin/kubectl
$ kubectl version --short Client Version: v1.15.0 Server Version: v1.15.0
$ tar -xf konvoy_v0.6.0_linux.tar.bz2
$ sudo mv ./konvoy_v0.6.0/* /usr/local/bin/
$ konvoy --version { "Version": "v0.6.0", "BuildDate": "Thu Jul 25 04:55:40 UTC 2019"
$ konvoy init
$ nano cluster.yaml --- kind: ClusterProvisioner apiVersion: konvoy.D2iQ.io/v1alpha1 metadata: name: ubuntu creationTimestamp: "2019-07-31T23:04:29.030542867Z" spec: provider: aws aws: region: us-west-2 availabilityZones: - us-west-2c tags: owner: ubuntu adminCIDRBlocks: - 0.0.0.0/0 nodePools: - name: worker count: 4 machine:rootVolumeSize: 80 rootVolumeType: gp2 imagefsVolumeEnabled: true imagefsVolumeSize: 160 imagefsVolumeType: gp2 type: t3.xlarge - name: control-plane controlPlane: true count: 3 machine: rootVolumeSize: 80 rootVolumeType: gp2 imagefsVolumeEnabled: true imagefsVolumeSize: 160 imagefsVolumeType: gp2 type: t3.large - name: bastion bastion: true count: 0 machine: rootVolumeSize: 10 rootVolumeType: gp2 type: t3.small sshCredentials: user: centos publicKeyFile: ubuntu-ssh.pub privateKeyFile: ubuntu-ssh.pem version: v0.6.0
... addons: configVersion: v0.0.47 addonsList: - name: awsebscsiprovisioner enabled: false - name: awsebsprovisioner enabled: false - name: dashboard enabled: true - name: dex enabled: true - name: dex-k8s-authenticator enabled: true - name: elasticsearch enabled: false - name: elasticsearchexporter enabled: false - name: fluentbit enabled: false - name: helm enabled: true - name: kibana enabled: false - name: kommander enabled: true - name: konvoy-ui enabled: true - name: localvolumeprovisioner enabled: false - name: opsportal enabled: true - name: prometheus enabled: false - name: prometheusadapter enabled: false - name: traefik enabled: true - name: traefik-forward-auth enabled: true - name: velero enabled: false version: v0.6.0
$ konvoy up
Kubernetes cluster and addons deployed successfully!Run `konvoy apply kubeconfig` to update kubectl credentials.Navigate to the URL below to access various services running in the cluster. https://a67190d830fd64957884d49fd036ea78-841155633.us-west-2.elb.amazonaws.com/ops/landing And login using the credentials below. Username: dazzling_swirles Password: DGwwyTFNCt6h2JSxTfqzD8PUWSLo1qPLCZJ82kQmMmaNctodWa3gd8W0O2SPFCIcIf the cluster was recently created, the dashboard and services may take a few minutes to be accessible.
$ konvoy apply kubeconfig
$ kubectl get nodes NAME STATUS ROLES AGE VERSION ip-10-0-129-203.us-west-2.compute.internal Ready <none> 4m49s v1.15.0 ip-10-0-129-216.us-west-2.compute.internal Ready <none> 4m49s v1.15.0 ip-10-0-130-203.us-west-2.compute.internal Ready <none> 4m49s v1.15.0 ip-10-0-131-148.us-west-2.compute.internal Ready <none> 4m49s v1.15.0 ip-10-0-194-238.us-west-2.compute.internal Ready master 7m8s v1.15.0 ip-10-0-194-76.us-west-2.compute.internal Ready master 5m27s v1.15.0 ip-10-0-195-6.us-west-2.compute.internal Ready master 6m31s v1.15.0
$ aws --region=us-west-2 ec2 describe-instances | grep ubuntu- ... "Value": "ubuntu-bb0c" In my case, my CLUSTER name is ubuntu-bb0c (as indicated in the “Value” field2). Replace the values for the first three variables with your settings and set the variables to be used in our script: export CLUSTER=ubuntu-bb0c # name of your cluster, export REGION=us-west-2 # region-name export KEY_FILE=ubuntu-ssh.pem # path to private key fileIPS=$(aws --region=$REGION ec2 describe-instances | jq --raw-output ".Reservations[].Instances[] | select((.Tags | length) > 0) | select(.Tags[].Value | test("$CLUSTER-worker")) | select(.State.Name | test("running")) | [.PublicIpAddress] | join(" ")")
for ip in $IPS; do echo $ip ssh -o StrictHostKeyChecking=no -i $KEY_FILE centos@$ip sudo yum install iscsi-initiator-utils -y ssh -o StrictHostKeyChecking=no -i $KEY_FILE centos@$ip sudo systemctl enable iscsid ssh -o StrictHostKeyChecking=no -i $KEY_FILE centos@$ip sudo systemctl start iscsid done
export DISK_SIZE=200
aws --region=$REGION ec2 describe-instances | jq --raw-output ".Reservations[].Instances[] | select((.Tags | length) > 0) | select(.Tags[].Value | test("$CLUSTER-worker")) | select(.State.Name | test("running")) | [.InstanceId, .Placement.AvailabilityZone] | "\(.[0]) \(.[1])"" | while read instance zone; do echo $instance $zone volume=$(aws --region=$REGION ec2 create-volume --size=$DISK_SIZE --volume-type gp2 --availability-zone=$zone --tag-specifications="ResourceType=volume,Tags=[{Key=string,Value=$CLUSTER}, {Key=owner,Value=michaelbeisiegel}]" | jq --raw-output .VolumeId) sleep 10 aws --region=$REGION ec2 attach-volume --device=/dev/xvdc --instance-id=$instance --volume-id=$volume done
wget https://openebs.github.io/charts/openebs-operator-1.1.0.yaml
… filterconfigs: – key: vendor-filter name: vendor filter state: true include: “” exclude: “CLOUDBYT,OpenEBS” – key: path-filter name: path filter state: true include: “” exclude: “loop,/dev/fd0,/dev/sr0,/dev/ram,/dev/dm-,/dev/md,/dev/nvme0n1,/dev/nvme1n1” …
kubectl apply -f openebs-operator-1.1.0.yaml
$ kubectl get blockdevices -n openebs NAME SIZE CLAIMSTATE STATUS AGE blockdevice-16b54eaf38720b5448005fbb3b2e803e 107374182400 Unclaimed Active 23s blockdevice-2403405be0d70c38b08ea6e29c5b0a2f 107374182400 Unclaimed Active 14s blockdevice-6d9b524c340bcb4dcecb69358054ec31 107374182400 Unclaimed Active 22s blockdevice-ecafd0c075fb4776f78bcde26ee2295f 107374182400 Unclaimed Active 23s
cat <<EOF | kubectl apply -f - kind: StoragePoolClaim apiVersion: openebs.io/v1alpha1 metadata: name: cstor-ebs-disk-pool annotations: cas.openebs.io/config: | - name: PoolResourceRequests value: |- memory: 2Gi - name: PoolResourceLimits value: |- memory: 4Gi spec: name: cstor-disk-pool type: disk poolSpec: poolType: striped blockDevices: blockDeviceList: - blockdevice-16b54eaf38720b5448005fbb3b2e803e - blockdevice-2403405be0d70c38b08ea6e29c5b0a2f - blockdevice-6d9b524c340bcb4dcecb69358054ec31 - blockdevice-ecafd0c075fb4776f78bcde26ee2295f EOF
cat <<EOF | kubectl apply -f - kind: StorageClass apiVersion: storage.k8s.io/v1 metadata: name: openebs-cstor-default annotations: openebs.io/cas-type: cstor cas.openebs.io/config: | - name: StoragePoolClaim value: "cstor-ebs-disk-pool" - name: ReplicaCount value: "3" storageclass.kubernetes.io/is-default-class: 'true' provisioner: openebs.io/provisioner-iscsi EOF
$ kubectl get sc NAME PROVISIONER AGE openebs-cstor-default (default) openebs.io/provisioner-iscsi 4s openebs-device openebs.io/local 26m openebs-hostpath openebs.io/local 26m openebs-jiva-default openebs.io/provisioner-iscsi 26m openebs-snapshot-promoter volumesnapshot.external-storage.k8s.io/snapshot-promoter 26m
... addons: configVersion: v0.0.47 addonsList: - name: awsebscsiprovisioner enabled: false - name: awsebsprovisioner enabled: false - name: dashboard enabled: true - name: dex enabled: true - name: dex-k8s-authenticator enabled: true - name: elasticsearch enabled: true - name: elasticsearchexporter enabled: true - name: fluentbit enabled: true - name: helm enabled: true - name: kibana enabled: true - name: kommander enabled: true - name: konvoy-ui enabled: true - name: localvolumeprovisioner enabled: false - name: opsportal enabled: true - name: prometheus enabled: true - name: prometheusadapter enabled: true - name: traefik enabled: true - name: traefik-forward-auth enabled: true - name: velero enabled: true
$ konvoy up
STAGE [Deploying Enabled Addons] helm [OK] dashboard [OK] opsportal [OK] traefik [OK] fluentbit [OK] kommander [OK] prometheus [OK] elasticsearch [OK] traefik-forward-auth [OK] konvoy-ui [OK] dex [OK] dex-k8s-authenticator [OK] elasticsearchexporter [OK] kibana [OK] prometheusadapter [OK] velero [OK]STAGE [Removing Disabled Addons] awsebscsiprovisioner [OK]Kubernetes cluster and addons deployed successfully! Run `konvoy apply kubeconfig` to update kubectl credentials. Navigate to the URL below to access various services running in the cluster. https://konvoy.us-west-2.elb.amazonaws.com/ops/landing And login using the credentials below. Username: dazzling_swirles Password: YourPasswordHere If the cluster was recently created, the dashboard and services may take a few minutes to be accessible.
$ kubectl get pv NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE pvc-0b6031e2-38c7-49e0-98b6-94ba9decda19 4Gi RWO Delete Bound kubeaddons/data-elasticsearch-kubeaddons-master-2 openebs-cstor-default 18m pvc-0fa74e6d-978a-47e6-961d-9ff6601702e6 10Gi RWO Delete Bound velero/data-minio-3 openebs-cstor-default 15m pvc-18cfe51f-0a2c-409a-ba89-dbd89ad1c9f5 10Gi RWO Delete Bound velero/data-minio-2 openebs-cstor-default 15m pvc-442dad69-56e5-44f1-8c43-cc90028c1d6d 10Gi RWO Delete Bound velero/data-minio-1 openebs-cstor-default 16m pvc-69b30e82-6e47-4061-9360-54591a3cb593 30Gi RWO Delete Bound kubeaddons/data-elasticsearch-kubeaddons-data-0 openebs-cstor-default 21m pvc-6d837a95-8607-4956-b3f5-bce4e42f5165 10Gi RWO Delete Bound velero/data-minio-0 openebs-cstor-default 16m pvc-83498bcb-6f7e-45b8-8d25-d6f7af46af9a 4Gi RWO Delete Bound kubeaddons/data-elasticsearch-kubeaddons-master-1 openebs-cstor-default 19m pvc-9df994f7-38af-4cb2-b0fc-2d314245a965 50Gi RWO Delete Bound kubeaddons/db-prometheus-prometheus-kubeaddons-prom-prometheus-0 openebs-cstor-default 21m pvc-f1c0e0ad-f03e-4212-81b5-f4b9e4b70f10 30Gi RWO Delete Bound kubeaddons/data-elasticsearch-kubeaddons-data-1 openebs-cstor-default 19m pvc-f92ca2d2-b808-4dc5-b148-362b93ef6d20 4Gi RWO Delete Bound kubeaddons/data-elasticsearch-kubeaddons-master-0 openebs-cstor-default 21m