Label (Space Separation): kubernetes Series

  • 1: High Availability of kubernetes master Nodes

  • 2: Configuring nginx server

  • Third: The keepalived high availability of LB configuring nginx

1: High Availability of kubernetes master Nodes

kubernetes multi-master cluster architecture

The high availability of kubernetes is mainly due to apiserver

deploy master01  IP Address: 192.168.100.12 

//Deploy the same services as Master 01 in 192.168.100.12

scp -r /opt/kubernetes/ root@192.168.100.12:/opt/

scp -r /opt/etcd/ root@192.168.100.12:/opt/

scp /usr/bin/kubectl root@192.168.20.12:/usr/bin/

scp /usr/lib/systemd/system/kube-* root@192.168.100.12:/usr/lib/systemd/system/

Modify configuration files 
cd /opt/kubernetes/cfg 

vim kube-apiserver 

---
--bind-address=192.168.100.12

--advertise-address=192.168.100.12

Start up service:

service kube-apiserver start

service kube-controller-manager start

service kube-scheduler start

login :
    192.168.20.12

kubectl get cs

Configuring load balancing nginx

nginx Server address: 192.168.100.15
nginx Of yum Package address

http://nginx.org/en/linux_packages.html

cd /etc/yum.repos.d/

vim  nginx.repo
---
[nginx-stable]
name=nginx stable repo
baseurl=http://nginx.org/packages/centos/$releasever/$basearch/
gpgcheck=1
enabled=1
gpgkey=https://nginx.org/keys/nginx_signing.key

[nginx-mainline]
name=nginx mainline repo
baseurl=http://nginx.org/packages/mainline/centos/$releasever/$basearch/
gpgcheck=1
enabled=0
gpgkey=https://nginx.org/keys/nginx_signing.key

---
yum install nignx 

cd /etc/nginx/
vim nginx.conf 

//Increase:
---
stream {
    log_format main "$remote_addr $upstream_addr $time_local $status";
    access_log /var/log/nginx/k8s-access.log main;
    upstream k8s-apiserver {

            server 192.168.100.11:6443;
            server 192.168.100.12:6443;

      }
    server {

           listen 192.168.100.15:6443;
           proxy_pass k8s-apiserver;

     }

}

···

modify node Nodal master Server pointing
login : 192.168.20.13 
cd /opt/kubernetes/cfg/

vim bootstrap.kubeconfig

---
server: https://192.168.100.11:6443 instead:

server: https://192.168.100.15:6443
---

vim kubelet.kubeconfig

----
server: https://192.168.100.11:6443 instead:

server: https://192.168.100.15:6443
----

vim kube-proxy.kubeconfig
----
server: https://192.168.100.11:6443 instead:

server: https://192.168.100.15:6443
----

scp bootstrap.kubeconfig kubelet.kubeconfig kube-proxy.kubeconfig root@192.168.100.14:/opt/kubernetes/cfg/

Restart kubelet and kube-proxy of node node 

service kubelet restart 
service kube-proxy restart 


Configuring tests with master nodes

kubectl get nodes 

Third: The keepalived high availability of LB configuring nginx

login 192.168.100.16 

//Install nginx server as above 192.168.100.15

//Amendment:

vim /etc/nginx/nginx.conf

----
stream {
    log_format main "$remote_addr $upstream_addr $time_local $status";
    access_log /var/log/nginx/k8s-access.log main;

    upstream k8s-apiserver {

            server 192.168.100.11:6443;
            server 192.168.100.12:6443;

      }
    server {

           listen 192.168.100.16:6443;
           proxy_pass k8s-apiserver;

     }

}
---

service nginx start 

To configure keepalive High availability 

yum install keepalived 

keepalived Configuration file

cd /etc/keepalived/

vim keepalived.conf 

---
! Configuration File for keepalived 

global_defs { 
   # Receiving mail address 
   notification_email { 
     acassen@firewall.loc 
     failover@firewall.loc 
     sysadmin@firewall.loc 
   } 
   # Mail Sending Address 
   notification_email_from Alexandre.Cassen@firewall.loc  
   smtp_server 127.0.0.1 
   smtp_connect_timeout 30 
   router_id NGINX_MASTER 
} 

vrrp_script check_nginx {
    script "/etc/keepalived/check_nginx.sh"
}

vrrp_instance VI_1 { 
    state MASTER 
    interface ens33
    virtual_router_id 51 # VRRP routing ID instance, each instance is unique 
    priority 100    # Priority, standby 192.168.100.16 server settings 90   
    advert_int 1    # Specify the VRRP Heart Packet Notification Interval, default 1 second 
    authentication { 
        auth_type PASS      
        auth_pass 1111 
    }  
    virtual_ipaddress { 
        192.168.100.70/24 
    } 
    track_script {
        check_nginx
    } 
}

----

//Configuration Check nginx Process Check
cd /etc/keepalived/

vim check_nginx.sh

---

#!/bin/bash

count=$(ps -ef |grep nginx |egrep -cv "grep|$$")

if [ "$count" -eq 0 ];then
    /etc/init.d/keepalived stop
fi

---
service keepalived start 

ps -ef |grep keepalived 

cd /etc/keepalived

scp check_nginx.sh keepalived.conf root@192.168.100.16:/etc/keepalived/
login: 192.168.100.16 

cd /etc/keepalived/

vim keepalived.conf
---
//Change priority 100 to priority 90

----

serivice keepalived start 

ps -ef |grep keepalived 

change node Nodal IP address

//Modify master server pointing of node node
login : 192.168.20.13 
cd /opt/kubernetes/cfg/

vim bootstrap.kubeconfig

---
server: https://192.168.100.15:6443 instead:

server: https://192.168.100.70:6443
---

vim kubelet.kubeconfig

----
server: https://192.168.100.15:6443 instead:

server: https://192.168.100.70:6443
----

vim kube-proxy.kubeconfig
----
server: https://192.168.100.15:6443 instead:

server: https://192.168.100.70:6443
----

scp bootstrap.kubeconfig kubelet.kubeconfig kube-proxy.kubeconfig root@192.168.100.14:/opt/kubernetes/cfg/
Restart kubelet and kube-proxy of node node 

service kubelet restart 
service kube-proxy restart 

login: 192.168.100.15
 Modify nginx configuration file

cd /etc/nginx/

vim nginx.conf 
----
Change linsten 192.168.100.15:6443 to 0.0.0.0:6443

----

service nginx restart 


login: 192.168.100.16
 Modify nginx configuration file

cd /etc/nginx/

vim nginx.conf 
----
Change linsten 192.168.100.16:6443 to 0.0.0.0:6443

----
service nginx restart 

Go to the master 01 node to see

kubectl get svc 

kubectl get svc,pods -n kube-system --all-namespaces 

Go to the master 02 node to see

kubectl get svc 

kubectl get svc,pods -n kube-system --all-namespaces