Launching Kubernetes with Ansible roles

Launching Kubernetes is a tedious task and this problem can be solved if we can automate the task with configuration tools like ansible.

So I created an ansible collection which consists roles for provisioning the instances and then configuring the cluster. Now below is how I coded the roles

- name: install boto
pip:
name: boto
- name: create kube master
ec2:
key_name: "{{ key_name }}"
group: "{{ sec_grp }}"
instance_type: "{{ os_type }}"
image: "{{ ami_id }}"
wait: true
region: ap-south-1
exact_count: 1
count_tag:
Name: kube_master
instance_tags:
Name: kube_master
App: kube
register: kube_ec2_master
- name: create kube slaves
ec2:
key_name: "{{ key_name }}"
group: "{{ sec_grp }}"
instance_type: "{{ os_type }}"
image: "{{ ami_id }}"
wait: true
region: ap-south-1
exact_count: 2
count_tag:
Name: kube_slave
instance_tags:
Name: kube_slave
App: kube
register: kube_ec2_slave
  1. First task is to install boto library in the machine which we are using to provision the other instances (I executed this inside localhost).
  2. Then we need to create the Kubernetes Master with required tags and of count 1.
  3. Next, we crate Kubernetes Slave with count as 2 and the other variables will be passed as vars

Vars file

---
# vars file for provision
key_name: mypem11
sec_grp: ssh-only
ami_id: ami-0a9d27a9f4f5c0efc
os_type: t2.micro

Here I have used amazon Linux but using RedHat will be a problem because docker isn't available in rhel8.

Since the provisioning is completed, now we go to configuring all the Kubernetes nodes and from the collection, there is a role named common

- name: install container engine [docker] and iproute-tc
package:
name:
- docker
- iproute-tc
state: present
- name: start the container engine
service:
name: docker
state: started
enabled: yes
- name: Copy kubernetes repo file
copy:
src: kubernetes.repo
dest: /etc/yum.repos.d/
- name: Install kubectl, kubelet, kubeadm programs
yum:
name:
- kubectl
- kubelet
- kubeadm
state: present
disable_excludes: kubernetes
- name: change the cgroup driver to systemd
copy:
src: daemon.json
dest: /etc/docker/
notify: restart docker
- name: enable bridging in iptables
sysctl:
name: net.bridge.bridge-nf-call-iptables
value: '1'
- name: enable bridging in ip6tables
sysctl:
name: net.bridge.bridge-nf-call-ip6tables
value: '1'
  • Here, first the packages of docker and iproute-tc (used for ip table config) are installed and then the docker service is started and enabled
  • Now the Kubernetes repo file is copied from the controller node to the managed node
[kubernetes]
name=Kubernetes
baseurl=https://packages.cloud.google.com/yum/repos/kubernetes-el7-\$basearch
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://packages.cloud.google.com/yum/doc/yum-key.gpg https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg
exclude=kubelet kubeadm kubectl
  • Now we install the Kubernetes related software from the configured repo in before step and also disabling the excluded repos.
  • Now we need to change the cgroup driver to systemd and then we to restart the docker file if there is a change with handler notification
- name: restart docker
service:
name: docker
state: restarted
  • Now we need to enable the bridging in iptables of ipv4 and ipv6

Now we need to configure the master node

- name: init kubectl
command: "kubeadm init --pod-network-cidr={{ net_name }} --ignore-preflight-errors=NumCPU --ignore-preflight-errors=Mem"
ignore_errors: yes
- name: create .kube dir
file:
path: $HOME/.kube
state: directory
mode: 0755
- name: copy config
command: cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
- name: change ownership
shell: chown $(id -u):$(id -g) $HOME/.kube/config
- name: create token for connecting workernodes
command: 'kubeadm token create --print-join-command'
register: jointoken
- name: create flannel overlay network
shell: "curl https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml | sed 's@10.244.0.0/16@{{net_name}}@' > kube-flannel.yml; kubectl apply -f kube-flannel.yml"
  • First stem is to initialize the kubernetes cluster with kubeadm init command and the pod network is provided from the user input
  • After cluster creation, we need to copy the config file to access the endpoint of Kubernetes to the $HOME/.kube directory
  • After this, we need to create a token that can be used for the worker nodes to join the cluster.
  • Then we create a overlay network with the help of flannel. but we need to change the default network name to the user’s network name specified and we can achieve this with sed command.

Now in slave node all we need to do is to get the command output from the master node for the token so that the slave node can connect to the master

- name: run the join command
shell: "{{join_command}}"
ignore_errors: yes

Now we need to create a playbook to make this work

First we need to configure our dynamic inventory that I have already discussed in my previous articles.

plugin: amazon.aws.aws_ec2
regions:
- ap-south-1
keyed_groups:
- key: tags
prefix: tag
hostnames:
- ip-address

After we have the dynamic inventory, we need to configure our ansible.cfg file which is as below

[defaults]
inventory=$HOME/inventory/aws_ec2.yml
host_key_checking=False
- hosts: localhost
collections:
- smc181002.k8s
tasks:
- name: launch nodes
include_role:
name: provision
vars:
key_name: mypem11
sec_grp: allow-all
ami_id: ami-0bcf5425cdc1d8a85
os_type: t2.micro
- set_fact:
instances: "{{ kube_ec2_master.instances + kube_ec2_slave.instances }}"
- name: wait for ssh to start
wait_for:
host: "{{item.public_dns_name}}"
port: 22
state: started
loop: "{{instances}}"

Now that we provisioned the instances, we need to create the playbook to configure the nodes with the collection roles

- hosts: tag_App_kube
remote_user: ec2-user
become: yes
become_user: root
collections:
- smc181002.k8s
roles:
- common
- hosts: tag_Name_kube_master
remote_user: ec2-user
become: yes
become_user: root
collections:
- smc181002.k8s
tasks:
- name: run master role
include_role:
name: master
vars:
net_name: "10.240.0.0/16"
- set_fact:
join_command: "{{ jointoken }}"
- debug:
msg: "{{hostvars[groups['tag_Name_kube_master'][0]]['join_command']['stdout']}}"
- hosts: tag_Name_kube_slave
remote_user: ec2-user
become: yes
become_user: root
collections:
- smc181002.k8s
tasks:
- name: slave config
include_role:
name: slave
vars:
join_command: "{{hostvars[groups['tag_Name_kube_master'][0]]['join_command']['stdout']}}"
  • The tag_App_kube is the group which includes all the nodes of the cluster which have a common configuration and common role from smc181002.k8s is used
  • Next comes the master node configuration and we use the master role from smc181002.k8s and then we save the token command output to host vars by setting with set_facts module
  • Then we configure the slaves by passing the command that the nodes should run to join the kubernetes cluster from the host vars that is set in the master node config role.
  • And after this, we can run this by first adding the ssh key for the instances so that ansible can contact the instances in AWS.
eval `ssh-agent`;
ssh-add /path/to/pem_file
  • After running the playbook commands, we can now login to the master node and use kubectl commands from the root user just like we do in minikube.

Hope you enjoyed the article and found it useful

Ansible Galaxy — ansible collection link

https://github.com/smc181002/k8s/ — github code

smc181002/create_kube_cluster (github.com) — includes the playbook files too

Love DevOps and Web Development