1. 首页
  2. 技术知识

Ansible部署K8s集群的方法

starterknow • 2023年11月1日 pm5:35 • 技术知识 • 阅读 1

目录

    检查网络：k8s-check.yaml检查k8s各主机的网络是否可达;检查k8s各主机操作系统版本是否达到要求；配置k8s集群dns解析： k8s-hosts-cfg.yaml配置yum源：k8s-yum-cfg.yaml时钟同步：k8s-time-sync.yaml禁用iptable、firewalld、NetworkManager服务禁用SElinux、swap：k8s-SE-swap-disable.yaml修改内核：k8s-kernel-cfg.yaml配置ipvs：k8s-ipvs-cfg.yaml安装docker：k8s-docker-install.yaml安装k8s组件[kubeadm\kubelet\kubectl]：k8s-install-kubepkgs.yaml安装集群镜像：k8s-APPs-images.yamlk8s集群初始化：k8s-cluster-init.yaml

环境：

主机 IP地址 组件
ansible 192.168.175.130 ansible
master 192.168.175.140 docker，kubectl，kubeadm，kubelet
node 192.168.175.141 docker，kubectl，kubeadm，kubelet
node 192.168.175.142 docker，kubectl，kubeadm，kubelet

检查及调试相关命令：

$ ansible-playbook -v k8s-time-sync.yaml –syntax-check

$ ansible-playbook -v k8s-*.yaml -C

$ ansible-playbook -v k8s-yum-cfg.yaml -C –start-at-task=”Clean origin dir” –step

$ ansible-playbook -v k8s-kernel-cfg.yaml –step主机inventory文件：

/root/ansible/hosts

[k8s_cluster]

master ansible_host=192.168.175.140

node1  ansible_host=192.168.175.141

node2  ansible_host=192.168.175.142

[k8s_cluster:vars]

ansible_port=22

ansible_user=root

ansible_password=hello123       


检查网络：k8s-check.yaml检查k8s各主机的网络是否可达;


检查k8s各主机操作系统版本是否达到要求；

– name: step01_check

  hosts: k8s_cluster

  gather_facts: no

  tasks:

    – name: check network

      shell:

        cmd: “ping -c 3 -m 2 {{ansible_host}}”

      delegate_to: localhost

    – name: get system version

      shell: cat /etc/system-release

      register: system_release

    – name: check system version

      vars:

        system_version: “{{ system_release.stdout | regex_search(‘([7-9].[0-9]+).*?’) }}”

        suitable_version: 7.5

      debug:

        msg: “{{ ‘The version of the operating system is ‘+ system_version +’, suitable!’ if (system_version | float >= suitable_version) else ‘The version of the operating system is unsuitable’ }}”调试命令：

$ ansible-playbook –ssh-extra-args ‘-o StrictHostKeyChecking=no’ -v -C k8s-check.yaml

$ ANSIBLE_HOST_KEY_CHECKING=False ansible-playbook -v -C k8s-check.yaml

$ ANSIBLE_HOST_KEY_CHECKING=False ansible-playbook -v k8s-check.yaml –start-at-task=”get system version”

    连接配置：k8s-conn-cfg.yaml在ansible服务器的/etc/hosts文件中添加k8s主机名解析配置生成密钥对，配置ansible免密登录到k8s各主机

– name: step02_conn_cfg

  hosts: k8s_cluster

  gather_facts: no

  vars_prompt:

    – name: RSA

      prompt: Generate RSA or not(Yes/No)?

      default: “no”

      private: no

    – name: password

      prompt: input your login password?

      default: “hello123”

  tasks:

    – name: Add DNS of k8s to ansible

      delegate_to: localhost

      lineinfile:

        path: /etc/hosts

        line: “{{ansible_host}}  {{inventory_hostname}}”

        backup: yes

    – name: Generate RSA

      run_once: true

      shell:

        cmd: ssh-keygen -t rsa -f ~/.ssh/id_rsa -N ”

        creates: /root/.ssh/id_rsa

      when: RSA | bool

    – name: Configure password free login

      shell: |

          /usr/bin/ssh-keyscan {{ ansible_host }} >> /root/.ssh/known_hosts 2> /dev/null

          /usr/bin/ssh-keyscan {{ inventory_hostname }} >> /root/.ssh/known_hosts 2> /dev/null

          /usr/bin/sshpass -p'{{ password }}’ ssh-copy-id root@{{ ansible_host }}

          #/usr/bin/sshpass -p'{{ password }}’ ssh-copy-id root@{{ inventory_hostname }}

    – name: Test ssh

      shell: hostname执行：

$ ansible-playbook k8s-conn-cfg.yaml

Generate RSA or not(Yes/No)? [no]: yes

input your login password? [hello123]:

PLAY [step02_conn_cfg] **********************************************************************************************************

TASK [Add DNS of k8s to ansible] ************************************************************************************************

ok: [master -> localhost]

ok: [node1 -> localhost]

ok: [node2 -> localhost]

TASK [Generate RSA] *************************************************************************************************************

changed: [master -> localhost]

TASK [Configure password free login] ********************************************************************************************

changed: [node1 -> localhost]

changed: [node2 -> localhost]

TASK [Test ssh] *****************************************************************************************************************

changed: [master]

changed: [node1]

changed: [node2]

PLAY RECAP **********************************************************************************************************************

master                     : ok=4    changed=3    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0

node1                      : ok=3    changed=2    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0

node2                      : ok=3    changed=2    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0


配置k8s集群dns解析： k8s-hosts-cfg.yaml

    设置主机名

    /etc/hosts文件中互相添加dns解析

– name: step03_cfg_host

  hosts: k8s_cluster

  gather_facts: no

  tasks:

    – name: set hostname

      hostname:

        name: “{{ inventory_hostname }}”

        use: systemd

    – name: Add dns to each other

      lineinfile:

        path: /etc/hosts

        backup: yes

        line: “{{item.value.ansible_host}}  {{item.key}}”

      loop: “{{ hostvars | dict2items }}”

      loop_control:

        label: “{{ item.key }} {{ item.value.ansible_host }}”执行：

$ ansible-playbook k8s-hosts-cfg.yaml

PLAY [step03_cfg_host] **********************************************************************************************************

TASK [set hostname] *************************************************************************************************************

ok: [master]

ok: [node1]

ok: [node2]

TASK [Add dns to each other] ****************************************************************************************************

ok: [node2] => (item=node1 192.168.175.141)

ok: [master] => (item=node1 192.168.175.141)

ok: [node1] => (item=node1 192.168.175.141)

ok: [node2] => (item=node2 192.168.175.142)

ok: [master] => (item=node2 192.168.175.142)

ok: [node1] => (item=node2 192.168.175.142)

ok: [node2] => (item=master 192.168.175.140)

ok: [master] => (item=master 192.168.175.140)

ok: [node1] => (item=master 192.168.175.140)

PLAY RECAP **********************************************************************************************************************

master                     : ok=2    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0

node1                      : ok=2    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0

node2                      : ok=2    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0


配置yum源：k8s-yum-cfg.yaml

– name: step04_yum_cfg

  hosts: k8s_cluster

  gather_facts: no

  tasks:

    – name: Create back-up directory

      file:

        path: /etc/yum.repos.d/org/

        state: directory

    – name: Back-up old Yum files

      shell:

        cmd: mv -f /etc/yum.repos.d/*.repo /etc/yum.repos.d/org/

        removes: /etc/yum.repos.d/org/

    – name: Add new Yum files

      copy:

        src: ./files_yum/

        dest: /etc/yum.repos.d/

    – name: Check yum.repos.d

        cmd: ls /etc/yum.repos.d/*


时钟同步：k8s-time-sync.yaml

– name: step05_time_sync

  hosts: k8s_cluster

  gather_facts: no

  tasks:

    – name: Start chronyd.service

      systemd:

        name: chronyd.service

        state: started

        enabled: yes

    – name: Modify time zone & clock

      shell: |

        cp -f /usr/share/zoneinfo/Asia/Shanghai /etc/localtime

        clock -w

        hwclock -w

    – name: Check time now

      command: date


禁用iptable、firewalld、NetworkManager服务

– name: step06_net_service

  hosts: k8s_cluster

  gather_facts: no

  tasks:

    – name: Stop some services for net

      systemd:

        name: “{{ item }}”

        state: stopped

        enabled: no

      loop:

        – firewalld

        – iptables

        – NetworkManager执行：

$ ansible-playbook -v k8s-net-service.yaml

… …

failed: [master] (item=iptables) => {

    “ansible_loop_var”: “item”,

    “changed”: false,

    “item”: “iptables”

}

MSG:

Could not find the requested service iptables: host

PLAY RECAP **********************************************************************************************************************

master                     : ok=0    changed=0    unreachable=0    failed=1    skipped=0    rescued=0    ignored=0

node1                      : ok=0    changed=0    unreachable=0    failed=1    skipped=0    rescued=0    ignored=0

node2                      : ok=0    changed=0    unreachable=0    failed=1    skipped=0    rescued=0    ignored=0


禁用SElinux、swap：k8s-SE-swap-disable.yaml

– name: step07_net_service

  hosts: k8s_cluster

  gather_facts: no

  tasks:

    – name: SElinux disabled

      lineinfile:

        path: /etc/selinux/config

        line: SELINUX=disabled

        regexp: ^SELINUX=

        state: present

        backup: yes

    – name: Swap disabled

        path: /etc/fstab

        line: ‘#\1’

        regexp: ‘(^/dev/mapper/centos-swap.*$)’

        backrefs: yes


修改内核：k8s-kernel-cfg.yaml

– name: step08_kernel_cfg

  hosts: k8s_cluster

  gather_facts: no

  tasks:

    – name: Create /etc/sysctl.d/kubernetes.conf

      copy:

        content: ”

        dest: /etc/sysctl.d/kubernetes.conf

        force: yes

    – name: Cfg bridge and ip_forward

      lineinfile:

        path: /etc/sysctl.d/kubernetes.conf

        line: “{{ item }}”

        state: present

      loop:

        – ‘net.bridge.bridge-nf-call-ip6tables = 1’

        – ‘net.bridge.bridge-nf-call-iptables = 1’

        – ‘net.ipv4.ip_forward = 1’

    – name: Load cfg

      shell:

        cmd: |

          sysctl -p

          modprobe br_netfilter

        removes: /etc/sysctl.d/kubernetes.conf

    – name: Check cfg

        cmd: ‘[ $(lsmod | grep br_netfilter | wc -l) -ge 2 ] && exit 0 || exit 3’执行：

$ ansible-playbook -v k8s-kernel-cfg.yaml –step

TASK [Check cfg] ****************************************************************************************************************

changed: [master] => {

    “changed”: true,

    “cmd”: “[ $(lsmod | grep br_netfilter | wc -l) -ge 2 ] && exit 0 || exit 3”,

    “delta”: “0:00:00.011574”,

    “end”: “2022-02-27 04:26:01.332896”,

    “rc”: 0,

    “start”: “2022-02-27 04:26:01.321322”

}

changed: [node2] => {

    “delta”: “0:00:00.016331”,

    “end”: “2022-02-27 04:26:01.351208”,

    “start”: “2022-02-27 04:26:01.334877”

changed: [node1] => {

    “delta”: “0:00:00.016923”,

    “end”: “2022-02-27 04:26:01.355983”,

    “start”: “2022-02-27 04:26:01.339060”

PLAY RECAP **********************************************************************************************************************

master                     : ok=4    changed=4    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0

node1                      : ok=4    changed=4    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0

node2                      : ok=4    changed=4    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0


配置ipvs：k8s-ipvs-cfg.yaml

– name: step09_ipvs_cfg

  hosts: k8s_cluster

  gather_facts: no

  tasks:

    – name: Install ipset and ipvsadm

      yum:

        name: “{{ item }}”

        state: present

      loop:

        – ipset

        – ipvsadm

    – name: Load modules

      shell: |

        modprobe — ip_vs

        modprobe — ip_vs_rr

        modprobe — ip_vs_wrr

        modprobe — ip_vs_sh

        modprobe — nf_conntrack_ipv4

    – name: Check cfg

      shell:

        cmd: ‘[ $(lsmod | grep -e -ip_vs -e nf_conntrack_ipv4 | wc -l) -ge 2 ] && exit 0 || exit 3’


安装docker：k8s-docker-install.yaml

– name: step10_docker_install

  hosts: k8s_cluster

  gather_facts: no

  tasks:

    – name: Install docker-ce

      yum:

        name: docker-ce-18.06.3.ce-3.el7

        state: present

    – name: Cfg docker

      copy:

        src: ./files_docker/daemon.json

        dest: /etc/docker/

    – name: Start docker

      systemd:

        name: docker.service

        state: started

        enabled: yes

    – name: Check docker version

      shell:

        cmd: docker –version


安装k8s组件[kubeadm\kubelet\kubectl]：k8s-install-kubepkgs.yaml

– name: step11_k8s_install_kubepkgs

  hosts: k8s_cluster

  gather_facts: no

  tasks:

    – name: Install k8s components

      yum:

        name: “{{ item }}”

        state: present

      loop:

        – kubeadm-1.17.4-0

        – kubelet-1.17.4-0

        – kubectl-1.17.4-0

    – name: Cfg k8s

      copy:

        src: ./files_k8s/kubelet

        dest: /etc/sysconfig/

        force: no

        backup: yes

    – name: Start kubelet

      systemd:

        name: kubelet.service

        state: started

        enabled: yes


安装集群镜像：k8s-apps-images.yaml

– name: step12_apps_images

  hosts: k8s_cluster

  gather_facts: no

  vars:

    apps:

      – kube-apiserver:v1.17.4

      – kube-controller-manager:v1.17.4

      – kube-scheduler:v1.17.4

      – kube-proxy:v1.17.4

      – pause:3.1

      – etcd:3.4.3-0

      – coredns:1.6.5

  vars_prompt:

      – name: cfg_python

        prompt: Do you need to install docker pkg for python(Yes/No)?

        default: “no”

        private: no

  tasks:

    – block:

        – name: Install python-pip

          yum:

            name: python-pip

            state: present

        – name: Install docker pkg for python

          shell:

            cmd: |

              pip install docker==4.4.4

              pip install websocket-client==0.32.0

            creates: /usr/lib/python2.7/site-packages/docker/

      when: cfg_python | bool

    – name: Pull images

      community.docker.docker_image:

        name: “registry.cn-hangzhou.aliyuncs.com/google_containers/{{ item }}”

        source: pull

      loop: “{{ apps }}”

    – name: Tag images

        repository: “k8s.gcr.io/{{ item }}”

        force_tag: yes

        source: local

    – name: Remove images for ali

        state: absent执行：

$ ansible-playbook k8s-apps-images.yaml

Do you need to install docker pkg for python(Yes/No)? [no]:

PLAY [step12_apps_images] *******************************************************************************************************

TASK [Install python-pip] *******************************************************************************************************

skipping: [node1]

skipping: [master]

skipping: [node2]

TASK [Install docker pkg for python] ********************************************************************************************

TASK [Pull images] **************************************************************************************************************

changed: [node1] => (item=kube-apiserver:v1.17.4)

changed: [node2] => (item=kube-apiserver:v1.17.4)

changed: [master] => (item=kube-apiserver:v1.17.4)

changed: [node1] => (item=kube-controller-manager:v1.17.4)

changed: [master] => (item=kube-controller-manager:v1.17.4)

changed: [node1] => (item=kube-scheduler:v1.17.4)

changed: [master] => (item=kube-scheduler:v1.17.4)

changed: [node1] => (item=kube-proxy:v1.17.4)

changed: [node2] => (item=kube-controller-manager:v1.17.4)

changed: [master] => (item=kube-proxy:v1.17.4)

changed: [node1] => (item=pause:3.1)

changed: [master] => (item=pause:3.1)

changed: [node2] => (item=kube-scheduler:v1.17.4)

changed: [node1] => (item=etcd:3.4.3-0)

changed: [master] => (item=etcd:3.4.3-0)

changed: [node2] => (item=kube-proxy:v1.17.4)

changed: [node1] => (item=coredns:1.6.5)

changed: [master] => (item=coredns:1.6.5)

changed: [node2] => (item=pause:3.1)

changed: [node2] => (item=etcd:3.4.3-0)

changed: [node2] => (item=coredns:1.6.5)

TASK [Tag images] ***************************************************************************************************************

ok: [node1] => (item=kube-apiserver:v1.17.4)

ok: [master] => (item=kube-apiserver:v1.17.4)

ok: [node2] => (item=kube-apiserver:v1.17.4)

ok: [node1] => (item=kube-controller-manager:v1.17.4)

ok: [master] => (item=kube-controller-manager:v1.17.4)

ok: [node2] => (item=kube-controller-manager:v1.17.4)

ok: [master] => (item=kube-scheduler:v1.17.4)

ok: [node1] => (item=kube-scheduler:v1.17.4)

ok: [node2] => (item=kube-scheduler:v1.17.4)

ok: [master] => (item=kube-proxy:v1.17.4)

ok: [node1] => (item=kube-proxy:v1.17.4)

ok: [node2] => (item=kube-proxy:v1.17.4)

ok: [master] => (item=pause:3.1)

ok: [node1] => (item=pause:3.1)

ok: [node2] => (item=pause:3.1)

ok: [master] => (item=etcd:3.4.3-0)

ok: [node1] => (item=etcd:3.4.3-0)

ok: [node2] => (item=etcd:3.4.3-0)

ok: [master] => (item=coredns:1.6.5)

ok: [node1] => (item=coredns:1.6.5)

ok: [node2] => (item=coredns:1.6.5)

TASK [Remove images for ali] ****************************************************************************************************

PLAY RECAP **********************************************************************************************************************

master                     : ok=3    changed=2    unreachable=0    failed=0    skipped=2    rescued=0    ignored=0

node1                      : ok=3    changed=2    unreachable=0    failed=0    skipped=2    rescued=0    ignored=0

node2                      : ok=3    changed=2    unreachable=0    failed=0    skipped=2    rescued=0    ignored=0


k8s集群初始化：k8s-cluster-init.yaml

– name: step13_cluster_init

  hosts: master

  gather_facts: no

  tasks:

    – block:

        – name: Kubeadm init

          shell:

            cmd:

              kubeadm init

              –apiserver-advertise-address={{ ansible_host }}

              –kubernetes-version=v1.17.4

              –service-cidr=10.96.0.0/12

              –pod-network-cidr=10.244.0.0/16

              –image-repository registry.aliyuncs.com/google_containers

        – name: Create /root/.kube

          file:

            path: /root/.kube/

            state: directory

            owner: root

            group: root

        – name: Copy /root/.kube/config

          copy:

            src: /etc/kubernetes/admin.conf

            dest: /root/.kube/config

            remote_src: yes

            backup: yes

        – name: Copy kube-flannel

            src: ./files_k8s/kube-flannel.yml

            dest: /root/

        – name: Apply kube-flannel

            cmd: kubectl apply -f /root/kube-flannel.yml

        – name: Get token

            cmd: kubeadm token create –print-join-command

          register: join_token

        – name: debug join_token

          debug:

            var: join_token.stdout到此这篇关于Ansible部署K8s集群的文章就介绍到这了,更多相关Ansible部署K8s集群内容请搜索共生网络以前的文章或继续浏览下面的相关文章希望大家以后多多支持共生网络！

相关文章：

  1. Kubernetes(K8S)容器集群管理环境完整部署详细教程-中篇
  2. Kubernetes(K8S)容器集群管理环境完整部署详细教程-上篇
  3. Kubernetes(K8S)容器集群管理环境完整部署详细教程-下篇
  4. Docker私有仓库Harbor介绍和部署方法详解

原创文章，作者：starterknow，如若转载，请注明出处：https://www.starterknow.com/105712.html

赞 (0)
starterknow
0
容器化技术架构jenkins docker k8s脚本浅析
« 上一篇 2023年11月1日 pm5:35
Nginx+Tomcat负载均衡及动静分离群集的实现
下一篇 » 2023年11月1日 pm5:44

相关推荐

联系我们