~/.config/systemd/user/openclaw-gateway.service ~/.config/systemd/user/openclaw-gateway.service.d/
这个文件一共 两个 play： Play 1（localhost）：只在控制机（你现在跑 ansible 的那台）做准备工作：生成/读取 SSH 公钥、写 ssh config、准备离线目录、尝试下载 k8s apt keyring（可选、失败不报错）。 Play 2（k8s_cluster）：对所有 k8s 节点做“真正的装机 + Kubernetes 初始化/加入”流程：系统基础配置、离线 apt repo 挂载、安装 containerd/k8s 组件、VIP 负载均衡（haproxy+keepalived，仅 master）、kubeadm init（首个 master）、其余 master join、worker join。Play 1：Prepare controller SSH key (localhost) 目标：让控制机具备 SSH key，并把公钥内容作为 fact 传给后续 play 使用。 关键点： 1.确保 ~/.ssh 存在 file 创建目录、权限 0700。 2.若缺失则生成 SSH key ssh-keygen -t rsa -b 4096 -N "" -f "{{ controller_ssh_key }}" creates: 保证幂等：文件存在则不再执行。 3.读取控制机公钥并 set_fact slurp 会把文件内容 base64 读回来；再 b64decode 变成文本。 存到 controller_pubkey，供后续节点写入 authorized_keys。 4.可选：写入控制机的 ~/.ssh/config 对 192.168.30.* 统一指定： 用户 root IdentityFile 使用 controller 生成的 key StrictHostKeyChecking no / UserKnownHostsFile /dev/null（方便自动化，但安全性降低） 这一步不影响节点配置，只是让你控制机 ssh 更省事。 5.确保 files/offline 目录存在 后续用到离线包、keyring 都放在 files/offline。 6.尝试下载 kubernetes apt keyring（最佳努力，不失败） failed_when: false + changed_when: false 有网就生成 kubernetes-apt-keyring.gpg，没网也继续往下跑。 随后 stat 检测是否存在，并设置 controller_has_k8s_keyring。 注意：你这里的离线 repo list 使用了 trusted=yes，所以 即使没有 keyring 也能装。但保留 keyring 逻辑可以让你以后切回在线源/或者取消 trusted=yes 更安全。Play 2：Bootstrap all k8s nodes（核心） 2.1 vars：整套集群“参数中心” ssh_user / ssh_user_home：依据 ansible_user 判断是 /root 还是 /home/<user>，用来给该用户写 authorized_keys。 VIP / keepalived / haproxy： apiserver_vip=192.168.30.58 apiserver_vip_port=16443（VIP 对外暴露端口） apiserver_bind_port=6443（kube-apiserver 实际监听端口） apiserver_vip_iface：用默认网卡名（拿 facts） keepalived_virtual_router_id/auth_pass：VRRP 参数 离线 repo： offline_tools_repo_tar、offline_containerd_repo_tar、offline_k8s_repo_tar、offline_lb_repo_tar 以及对应解压目录、apt list 文件路径 k8s 版本： k8s_pkg_version: 1.30.14-1.1 kubeadm_kubernetes_version: v1.30.14 kubeadm_image_repository：你用阿里镜像仓库，适合国内/离线镜像同步场景 containerd： containerd_sandbox_image 指定 pause 镜像 SystemdCgroup=true 内核模块：overlay、br_netfilter、ipvs 一套 + nf_conntrack 集群网络：pod/service 子网、domain、cri socket LB 节点选择逻辑（很关键）： lb_masters: "{{ (groups['k8s_cluster'] | select('search','master') | list | sort)[:2] }}" 从 inventory 里挑主机名含 master 的，排序后取前两个作为“做 VIP LB 的两台 master” init_master: "{{ lb_masters[0] }}"：第一台 master 作为 kubeadm init 节点 is_lb_master/is_init_master：根据当前 host 判断分支执行 注意：这个选择逻辑 强依赖你的 inventory 主机名里包含 master，且至少有 2 台；否则 haproxy 配置那里引用 lb_masters[1] 会出问题。 2.2 apt 源清理：避免离线环境 apt 卡死 注释 /etc/apt/sources.list 和 /etc/apt/sources.list.d/*.list 里 deb cdrom: 行 离线环境最常见问题就是 apt update 时尝试访问 cdrom 或不可达源导致报错/卡住，这里算是“保险丝”。 2.3 主机基础：hostname、/etc/hosts、SSH 信任 hostname 设置为 inventory_hostname 生成 k8s_hosts_block：把所有节点 IP + 主机名写进 blockinfile 写入 /etc/hosts（保证节点互相能用主机名解析） 写 authorized_keys： 给 ansible_user 和 root 都写入控制机公钥（让控制机免密登录节点） 给 root 再写入“所有节点之间互信”的 key（node<->node） 配 sshd drop-in： PermitRootLogin prohibit-password（允许 root 使用公钥登录） PasswordAuthentication no（禁用密码登录） 并触发 handler restart ssh 风险提示：如果你原本靠密码/其他方式登录，禁用密码可能把你锁在门外。好在你的流程先把公钥塞进去再禁用密码，逻辑上是对的，但仍建议谨慎在生产环境使用。 2.4 swap & 内核参数：Kubernetes 前置条件 swapoff -a + 注释 /etc/fstab 里 swap 行（避免重启恢复） 写 /etc/modules-load.d/k8s.conf 并 modprobe 写 /etc/sysctl.d/99-kubernetes-cri.conf 并 sysctl --system 包括桥接流量、ip_forward、nonlocal_bind（VIP 常用） 2.5 离线 apt repo：解压、自动定位 Packages.gz、写 file: 源 流程对每个 repo 都类似： 确保 /opt/offline-repos 存在 解压 tar.gz 到对应目录 find Packages.gz，取其所在目录当“repo root” 写 deb [trusted=yes] file:<repo_root> ./ apt update 刷缓存 安装 packages trusted=yes 让 apt 不校验签名，离线很好用，但安全性降低；如果你已经有 keyring/签名，也可以改为不 trusted 并正确配 key。 2.6 containerd：配置 CRI、启动服务 安装 containerd/runc（来自离线 repo） 写 /etc/containerd/config.toml sandbox_image 指定 pause snapshotter=overlayfs SystemdCgroup=true registry mirrors 里 docker.io 指向 registry-1（若离线环境不出网，拉 docker.io 仍会失败——通常你会提前把镜像导入，或搭私有 registry） systemd 启动并 enable 写 /etc/crictl.yaml 让 crictl 默认连 containerd 2.7 kubeadm/kubelet/kubectl：离线安装 + hold 可选复制 kubernetes apt keyring 到节点 /etc/apt/keyrings 安装固定版本 kubeadm/kubelet/kubectl + 依赖（kubernetes-cni、cri-tools…） apt-mark hold 锁版本 启动 kubelet（此时可能还会报错是正常的，直到 kubeadm init/join 完成） 2.8 VIP LB（仅两台 master）：haproxy + keepalived 两台 master 安装 haproxy、keepalived haproxy： 监听 *:16443 后端转发到两台 master 的 :6443 keepalived： check_haproxy.sh：只检查 haproxy 进程是否存在 两台都用 state BACKUP，用优先级决定谁抢到 VIP virtual_ipaddress 配 VIP/24 track_script 绑定健康检查 启动并 enable；等待本地 16443 端口起来 这套结构就是：VIP(16443) -> haproxy -> master(6443) kubeadm 的 controlPlaneEndpoint 指向 VIP:16443，所以集群内外都走 VIP。 2.9 kubeadm init（仅 init master） 先检查 /etc/kubernetes/admin.conf 是否存在，存在说明已经初始化过，避免重复 init 写 /root/kubeadm.yaml apiVersion: kubeadm.k8s.io/v1beta3（你已经标注了“修复2：v1beta4 -> v1beta3”） controlPlaneEndpoint: VIP:16443 imageRepository 指向阿里 apiServer.certSANs 包含 VIP、两台 master IP/hostname、localhost InitConfiguration 里： advertiseAddress 用本机 IP bindPort 用 6443 nodeRegistration 设置 cri socket 和 node-ip 执行 kubeadm init（带 --upload-certs，并忽略 SystemVerification、Swap） 把 admin.conf 拷贝到 /root/.kube/config 方便 kubectl 生成 join 命令： worker join command：kubeadm token create --print-join-command 控制面 join 需要 --control-plane --certificate-key <key> kubeadm init phase upload-certs --upload-certs 输出里抓 64 位 hex key 把 join 命令写成脚本 /root/join-worker.sh、/root/join-controlplane.sh 最关键的一步：把 join 命令通过 delegate_to: localhost + delegate_facts: true 变成“全局事实”，让后续其他节点能引用： hostvars['localhost'].global_join_worker hostvars['localhost'].global_join_cp 2.10 其余节点 join 先看 /etc/kubernetes/kubelet.conf 是否存在（存在说明已 join） 第二台 master（is_lb_master 且 not is_init_master）： 运行 global_join_cp 加入控制面 worker： 运行 global_join_worker handlers：服务重启 containerd、haproxy、keepalived、ssh 的 restart 都集中在 handlers 由上面 tasks 的 notify 触发，符合 ansible 最佳实践--- ############################################################################### # Play 1: 仅在控制机(localhost)执行 # 目的： # 1) 生成/准备控制机 SSH key（用于免密登录所有节点） # 2) 读取控制机公钥，保存为 fact，供后续 play 写入各节点 authorized_keys # 3) 准备离线目录与（可选）kubernetes apt keyring 文件 ############################################################################### - name: Prepare controller SSH key (localhost) hosts: localhost gather_facts: false tasks: # 确保控制机 ~/.ssh 目录存在 - name: Ensure ~/.ssh exists on controller ansible.builtin.file: path: "{{ lookup('env','HOME') + '/.ssh' }}" state: directory mode: "0700" # 若 controller_ssh_key 不存在则生成（幂等：creates 控制） - name: Generate SSH key on controller if missing ansible.builtin.command: > ssh-keygen -t rsa -b 4096 -N "" -f "{{ controller_ssh_key }}" args: creates: "{{ controller_ssh_key }}" # 读取控制机公钥（slurp 返回 base64） - name: Read controller public key ansible.builtin.slurp: src: "{{ controller_ssh_pub }}" register: controller_pubkey_raw # 将 base64 解码成文本形式的公钥，保存为 controller_pubkey（供后续 hostvars['localhost'] 引用） - name: Set controller_pubkey fact ansible.builtin.set_fact: controller_pubkey: "" # 可选：写入控制机 ~/.ssh/config，方便你从控制机 ssh 到 192.168.30.* 网段 # 注意：StrictHostKeyChecking no 会降低安全性，但便于自动化环境 - name: Ensure controller ssh config includes cluster rule (optional but recommended) ansible.builtin.blockinfile: path: "{{ lookup('env','HOME') + '/.ssh/config' }}" create: true mode: "0600" marker: "# {mark} ANSIBLE K8S CLUSTER SSH" block: | Host 192.168.30.* User root IdentityFile {{ controller_ssh_key }} IdentitiesOnly yes StrictHostKeyChecking no UserKnownHostsFile /dev/null # 确保项目的离线文件目录存在（tar.gz、keyring 等都在这里） - name: Ensure files/offline exists on controller ansible.builtin.file: path: "{{ playbook_dir }}/../files/offline" state: directory mode: "0755" # 尝试在线下载 kubernetes apt keyring（最佳努力：失败不报错） # 离线环境没网也没关系，你目录里已有 kubernetes-apt-keyring.gpg 的话同样可用 - name: Try to generate kubernetes apt keyring on controller if missing (best effort, no-fail) ansible.builtin.shell: | set -e curl -fsSL --connect-timeout 5 --max-time 20 https://pkgs.k8s.io/core:/stable:/v1.30/deb/Release.key \ | gpg --dearmor -o "{{ playbook_dir }}/../files/offline/kubernetes-apt-keyring.gpg" args: creates: "{{ playbook_dir }}/../files/offline/kubernetes-apt-keyring.gpg" changed_when: false failed_when: false # 检测 keyring 是否存在 - name: Check kubernetes apt keyring exists on controller ansible.builtin.stat: path: "{{ playbook_dir }}/../files/offline/kubernetes-apt-keyring.gpg" register: controller_k8s_keyring_stat # 设置一个布尔 fact，供后续 play 决定要不要复制 keyring 到节点 - name: Set controller_has_k8s_keyring fact ansible.builtin.set_fact: controller_has_k8s_keyring: "" ############################################################################### # Play 2: 对所有 k8s 节点执行（hosts: k8s_cluster） # 目的（大而全）： # - 系统基础：hostname、/etc/hosts、关闭 swap、内核模块与 sysctl # - SSH：控制机 -> 节点免密；节点之间 root 互信（node<->node） # - 离线安装：解压离线 repo，写 file: apt 源，apt 安装工具/容器运行时/k8s 组件 # - master VIP：haproxy + keepalived 提供 apiserver VIP 入口 # - kubeadm：init 首个 master；其余 master/worker join ############################################################################### - name: Bootstrap all k8s nodes (hostname, /etc/hosts, SSH trust, offline tools, kernel modules, containerd, k8s pkgs, swapoff, apiserver VIP LB, kubeadm init/join) hosts: k8s_cluster become: true gather_facts: true vars: # 当前 ansible 连接用户及其 home（用于写 authorized_keys） ssh_user: "{{ ansible_user }}" ssh_user_home: "{{ '/root' if ssh_user == 'root' else '/home/' ~ ssh_user }}" # apiserver VIP（对外入口），以及 VIP 对外端口与 apiserver 实际 bind 端口 apiserver_vip: "192.168.30.58" apiserver_vip_port: 16443 apiserver_bind_port: 6443 # keepalived 使用的网卡（默认取 facts 的默认网卡，否则 ens33） apiserver_vip_iface: "" keepalived_virtual_router_id: 51 keepalived_auth_pass: "k8sVIP@2025" # ------------------------- # 离线 repo：系统工具 # ------------------------- offline_tools_repo_tar: "{{ playbook_dir }}/../files/offline/os-tools-repo-ipvs.tar.gz" offline_tools_repo_dir: "/opt/offline-repos/os-tools-ipvs" offline_tools_repo_list: "/etc/apt/sources.list.d/offline-os-tools-ipvs.list" offline_tools_packages: - expect - wget - jq - psmisc - vim - net-tools - telnet - lvm2 - git - ntpdate - chrony - bind9-utils - rsync - unzip - ipvsadm - ipset - sysstat - conntrack # ------------------------- # 离线 repo：containerd # ------------------------- offline_containerd_repo_tar: "{{ playbook_dir }}/../files/offline/containerd-repo.tar.gz" offline_containerd_repo_dir: "/opt/offline-repos/containerd" offline_containerd_repo_list: "/etc/apt/sources.list.d/offline-containerd.list" offline_containerd_packages: - containerd - runc # ------------------------- # 离线 repo：haproxy/keepalived（仅 master 用） # ------------------------- offline_lb_repo_tar: "{{ playbook_dir }}/../files/offline/nginx-keepalived-repo.tar.gz" offline_lb_repo_dir: "/opt/offline-repos/nginx-keepalived" offline_lb_repo_list: "/etc/apt/sources.list.d/offline-nginx-keepalived.list" # ------------------------- # Kubernetes 版本与镜像仓库 # ------------------------- k8s_pkg_version: "1.30.14-1.1" kubeadm_kubernetes_version: "v1.30.14" kubeadm_image_repository: "registry.cn-hangzhou.aliyuncs.com/google_containers" # containerd pause 镜像（pod sandbox） containerd_sandbox_image: "registry.cn-hangzhou.aliyuncs.com/google_containers/pause:3.9" containerd_config: "/etc/containerd/config.toml" # ------------------------- # 离线 repo：Kubernetes apt 仓库 # ------------------------- offline_k8s_repo_tar: "{{ playbook_dir }}/../files/offline/k8s-repo-v1.30.14-1.1.tar.gz" offline_k8s_repo_dir: "/opt/offline-repos/k8s-v1.30.14-1.1" offline_k8s_repo_list: "/etc/apt/sources.list.d/offline-k8s-v1.30.14-1.1.list" # Kubernetes keyring（如果控制机存在，就复制到节点） offline_k8s_keyring_src: "{{ playbook_dir }}/../files/offline/kubernetes-apt-keyring.gpg" offline_k8s_keyring_dest: "/etc/apt/keyrings/kubernetes-apt-keyring.gpg" # k8s 组件及依赖（kubeadm/kubelet/kubectl 固定版本安装） offline_k8s_packages: - "kubeadm={{ k8s_pkg_version }}" - "kubelet={{ k8s_pkg_version }}" - "kubectl={{ k8s_pkg_version }}" - kubernetes-cni - cri-tools - socat - ebtables - ethtool - apt-transport-https # ipvs 与 k8s 常用模块 ipvs_modules: - ip_vs - ip_vs_rr - ip_vs_wrr - ip_vs_sh - nf_conntrack k8s_modules: - overlay - br_netfilter # 集群网络参数 pod_subnet: "10.244.0.0/16" service_subnet: "10.96.0.0/12" cluster_domain: "cluster.local" cri_socket: "unix:///run/containerd/containerd.sock" # 从 inventory 中挑选主机名包含 master 的节点，排序取前两台作为 LB master # 第 1 台同时作为 kubeadm init 节点 lb_masters: "{{ (groups['k8s_cluster'] | select('search','master') | list | sort)[:2] }}" is_lb_master: "{{ inventory_hostname in lb_masters }}" init_master: "{{ lb_masters[0] }}" is_init_master: "{{ inventory_hostname == init_master }}" tasks: # ------------------------- # apt 源清理：禁用 cdrom 源（离线环境常见坑） # ------------------------- - name: Disable CDROM apt source in /etc/apt/sources.list (comment deb cdrom:) ansible.builtin.replace: path: /etc/apt/sources.list regexp: '^deb\s+cdrom:' replace: '# deb cdrom:' failed_when: false - name: Find .list files under /etc/apt/sources.list.d ansible.builtin.find: paths: /etc/apt/sources.list.d patterns: "*.list" file_type: file register: apt_list_files failed_when: false - name: Disable CDROM apt source in sources.list.d files (comment deb cdrom:) ansible.builtin.replace: path: "{{ item.path }}" regexp: '^deb\s+cdrom:' replace: '# deb cdrom:' loop: "" failed_when: false # ------------------------- # 主机名与 hosts 解析：确保节点互相能解析主机名 # ------------------------- - name: Set hostname ansible.builtin.hostname: name: "{{ inventory_hostname }}" - name: Build hosts block for all cluster nodes ansible.builtin.set_fact: k8s_hosts_block: | {% for h in groups['k8s_cluster'] | sort %} {{ hostvars[h].ansible_default_ipv4.address }} {{ h }} {% endfor %} - name: Ensure /etc/hosts contains cluster nodes mapping ansible.builtin.blockinfile: path: /etc/hosts marker: "# {mark} ANSIBLE K8S CLUSTER HOSTS" block: "{{ k8s_hosts_block }}" # ------------------------- # SSH 免密：控制机 -> 节点（ansible_user 与 root） # ------------------------- - name: Ensure ansible user .ssh dir exists ansible.builtin.file: path: "{{ ssh_user_home }}/.ssh" state: directory mode: "0700" owner: "{{ ssh_user }}" group: "{{ ssh_user }}" - name: Add controller pubkey to ansible user authorized_keys ansible.builtin.lineinfile: path: "{{ ssh_user_home }}/.ssh/authorized_keys" create: true mode: "0600" owner: "{{ ssh_user }}" group: "{{ ssh_user }}" line: "" when: (hostvars['localhost'].controller_pubkey | default('')) | length > 0 - name: Ensure root .ssh dir exists ansible.builtin.file: path: /root/.ssh state: directory mode: "0700" - name: Add controller pubkey to root authorized_keys ansible.builtin.lineinfile: path: /root/.ssh/authorized_keys create: true mode: "0600" line: "" when: (hostvars['localhost'].controller_pubkey | default('')) | length > 0 # ------------------------- # SSHD 策略：允许 root 公钥登录，但不禁用密码登录 # ------------------------- - name: Ensure sshd drop-in dir exists ansible.builtin.file: path: /etc/ssh/sshd_config.d state: directory mode: "0755" - name: Allow root login with publickey (drop-in) and keep password login enabled ansible.builtin.copy: dest: /etc/ssh/sshd_config.d/99-ansible-rootlogin.conf mode: "0644" content: | PermitRootLogin prohibit-password PubkeyAuthentication yes PasswordAuthentication yes notify: Restart ssh # ------------------------- # 节点之间 root 互信：node <-> node # 思路：每个节点生成自己的 /root/.ssh/id_rsa，然后把所有节点的公钥汇总写到每台的 authorized_keys # ------------------------- - name: Generate node SSH key if missing ansible.builtin.command: ssh-keygen -t rsa -b 4096 -N "" -f /root/.ssh/id_rsa args: creates: /root/.ssh/id_rsa - name: Read node public key ansible.builtin.slurp: src: /root/.ssh/id_rsa.pub register: node_pubkey_raw - name: Set node_pubkey_text fact ansible.builtin.set_fact: node_pubkey_text: "" - name: Add all nodes keys to every node authorized_keys (node <-> node) ansible.builtin.lineinfile: path: /root/.ssh/authorized_keys create: true mode: "0600" line: "{{ hostvars[item].node_pubkey_text }}" loop: "" when: hostvars[item].node_pubkey_text is defined # ------------------------- # swap：k8s 要求关闭 swap # ------------------------- - name: Disable swap immediately ansible.builtin.command: swapoff -a changed_when: false failed_when: false - name: Comment swap in /etc/fstab ansible.builtin.replace: path: /etc/fstab regexp: '^(\s*[^#\n]+\s+[^ \n]+\s+swap\s+[^ \n]+.*)$' replace: '# \1' failed_when: false # ------------------------- # 内核模块与 sysctl：k8s + ipvs 常规前置 # ------------------------- - name: Ensure k8s modules-load file ansible.builtin.copy: dest: /etc/modules-load.d/k8s.conf mode: "0644" content: | overlay br_netfilter ip_vs ip_vs_rr ip_vs_wrr ip_vs_sh nf_conntrack - name: Modprobe required modules ansible.builtin.command: "modprobe {{ item }}" loop: "{{ k8s_modules + ipvs_modules }}" changed_when: false failed_when: false - name: Ensure sysctl for Kubernetes ansible.builtin.copy: dest: /etc/sysctl.d/99-kubernetes-cri.conf mode: "0644" content: | net.bridge.bridge-nf-call-iptables = 1 net.bridge.bridge-nf-call-ip6tables = 1 net.ipv4.ip_forward = 1 net.ipv4.ip_nonlocal_bind = 1 - name: Apply sysctl ansible.builtin.command: sysctl --system changed_when: false # ------------------------- # 离线 repo：目录准备 # ------------------------- - name: Ensure offline repos base dir exists ansible.builtin.file: path: /opt/offline-repos state: directory mode: "0755" - name: Ensure offline repo dirs exist ansible.builtin.file: path: "{{ item }}" state: directory mode: "0755" loop: - "{{ offline_tools_repo_dir }}" - "{{ offline_containerd_repo_dir }}" - "{{ offline_k8s_repo_dir }}" - "{{ offline_lb_repo_dir }}" # ------------------------- # 离线 repo：系统工具 repo（解压 -> 自动定位 Packages.gz -> 写 file: 源） # ------------------------- - name: Unpack offline tools repo ansible.builtin.unarchive: src: "{{ offline_tools_repo_tar }}" dest: "{{ offline_tools_repo_dir }}" - name: Find Packages.gz for offline tools repo (auto-detect repo root) ansible.builtin.find: paths: "{{ offline_tools_repo_dir }}" patterns: "Packages.gz" recurse: true register: tools_pkg_index - name: Set offline tools repo root ansible.builtin.set_fact: offline_tools_repo_root: "{{ (tools_pkg_index.files | first).path | dirname }}" when: (tools_pkg_index.matched | int) > 0 - name: Write offline tools apt source list ansible.builtin.copy: dest: "{{ offline_tools_repo_list }}" mode: "0644" content: | deb [trusted=yes] file: ./ # ------------------------- # 离线 repo：containerd repo（解压 -> 自动定位 Packages.gz -> 写 file: 源） # ------------------------- - name: Unpack offline containerd repo ansible.builtin.unarchive: src: "{{ offline_containerd_repo_tar }}" dest: "{{ offline_containerd_repo_dir }}" - name: Find Packages.gz for offline containerd repo (auto-detect repo root) ansible.builtin.find: paths: "{{ offline_containerd_repo_dir }}" patterns: "Packages.gz" recurse: true register: containerd_pkg_index - name: Set offline containerd repo root ansible.builtin.set_fact: offline_containerd_repo_root: "{{ (containerd_pkg_index.files | first).path | dirname }}" when: (containerd_pkg_index.matched | int) > 0 - name: Write offline containerd apt source list ansible.builtin.copy: dest: "{{ offline_containerd_repo_list }}" mode: "0644" content: | deb [trusted=yes] file: ./ # ------------------------- # 离线 repo：k8s repo（解压 -> 自动定位 Packages.gz -> 写 file: 源） # ------------------------- - name: Unpack offline kubernetes repo ansible.builtin.unarchive: src: "{{ offline_k8s_repo_tar }}" dest: "{{ offline_k8s_repo_dir }}" - name: Find Packages.gz for offline kubernetes repo (auto-detect repo root) ansible.builtin.find: paths: "{{ offline_k8s_repo_dir }}" patterns: "Packages.gz" recurse: true register: k8s_pkg_index - name: Set offline kubernetes repo root ansible.builtin.set_fact: offline_k8s_repo_root: "{{ (k8s_pkg_index.files | first).path | dirname }}" when: (k8s_pkg_index.matched | int) > 0 - name: Write offline kubernetes apt source list ansible.builtin.copy: dest: "{{ offline_k8s_repo_list }}" mode: "0644" content: | deb [trusted=yes] file: ./ # ------------------------- # 离线 repo：LB repo（仅 master，且 best effort） # ------------------------- - name: Unpack offline LB repo (masters only, best effort) ansible.builtin.unarchive: src: "{{ offline_lb_repo_tar }}" dest: "{{ offline_lb_repo_dir }}" when: is_lb_master failed_when: false - name: Find Packages.gz for offline LB repo (auto-detect repo root) ansible.builtin.find: paths: "{{ offline_lb_repo_dir }}" patterns: "Packages.gz" recurse: true register: lb_pkg_index when: is_lb_master failed_when: false - name: Set offline LB repo root ansible.builtin.set_fact: offline_lb_repo_root: "{{ (lb_pkg_index.files | first).path | dirname }}" when: - is_lb_master - lb_pkg_index is defined - (lb_pkg_index.matched | default(0) | int) > 0 - name: Write offline LB apt source list (masters only, best effort) ansible.builtin.copy: dest: "{{ offline_lb_repo_list }}" mode: "0644" content: | deb [trusted=yes] file: ./ when: is_lb_master failed_when: false # 配置完离线源后刷新 apt cache - name: Update apt cache after configuring offline repos ansible.builtin.apt: update_cache: true cache_valid_time: 3600 # 安装常用工具（失败不致命：可能某些包不在 repo 里） - name: Install common offline tools packages ansible.builtin.apt: name: "{{ offline_tools_packages }}" state: present update_cache: false failed_when: false # ------------------------- # containerd 安装与配置 # ------------------------- - name: Ensure containerd is installed ansible.builtin.apt: name: "{{ offline_containerd_packages }}" state: present update_cache: false # 写入 containerd 配置（包含 SystemdCgroup=true 等） - name: Write containerd config.toml ansible.builtin.copy: dest: "{{ containerd_config }}" mode: "0644" content: | version = 2 root = "/var/lib/containerd" state = "/run/containerd" [grpc] address = "/run/containerd/containerd.sock" [plugins."io.containerd.grpc.v1.cri"] sandbox_image = "{{ containerd_sandbox_image }}" [plugins."io.containerd.grpc.v1.cri".containerd] snapshotter = "overlayfs" default_runtime_name = "runc" [plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc] runtime_type = "io.containerd.runc.v2" [plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc.options] SystemdCgroup = true [plugins."io.containerd.grpc.v1.cri".registry.mirrors] [plugins."io.containerd.grpc.v1.cri".registry.mirrors."docker.io"] endpoint = ["https://registry-1.docker.io"] notify: Restart containerd - name: Enable & start containerd ansible.builtin.systemd: name: containerd enabled: true state: started # 配置 crictl 默认连接 containerd socket - name: Configure crictl ansible.builtin.copy: dest: /etc/crictl.yaml mode: "0644" content: | runtime-endpoint: unix:///run/containerd/containerd.sock image-endpoint: unix:///run/containerd/containerd.sock timeout: 10 debug: false # ------------------------- # k8s keyring（可选）与 k8s 组件安装 # ------------------------- - name: Ensure /etc/apt/keyrings exists ansible.builtin.file: path: /etc/apt/keyrings state: directory mode: "0755" - name: Copy kubernetes apt keyring if exists on controller ansible.builtin.copy: src: "{{ offline_k8s_keyring_src }}" dest: "{{ offline_k8s_keyring_dest }}" mode: "0644" when: hostvars['localhost'].controller_has_k8s_keyring | default(false) - name: Install kubeadm/kubelet/kubectl and deps ansible.builtin.apt: name: "{{ offline_k8s_packages }}" state: present update_cache: false # 锁定版本，避免被 apt upgrade 意外升级 - name: Hold kubeadm/kubelet/kubectl ansible.builtin.command: "apt-mark hold kubeadm kubelet kubectl" changed_when: false failed_when: false - name: Enable kubelet ansible.builtin.systemd: name: kubelet enabled: true state: started # ------------------------- # VIP LB：haproxy + keepalived（仅两台 master） # ------------------------- - name: Install haproxy and keepalived on masters ansible.builtin.apt: name: - haproxy - keepalived state: present update_cache: false when: is_lb_master # haproxy 将 VIP:16443 转发到 两台 master 的 6443 - name: Write haproxy config for apiserver VIP ansible.builtin.copy: dest: /etc/haproxy/haproxy.cfg mode: "0644" content: | global log /dev/log local0 log /dev/log local1 notice daemon maxconn 20000 defaults log global mode tcp option tcplog timeout connect 5s timeout client 1m timeout server 1m frontend kube-apiserver bind *:{{ apiserver_vip_port }} default_backend kube-apiserver backend kube-apiserver option tcp-check balance roundrobin server {{ lb_masters[0] }} {{ hostvars[lb_masters[0]].ansible_default_ipv4.address }}:{{ apiserver_bind_port }} check server {{ lb_masters[1] }} {{ hostvars[lb_masters[1]].ansible_default_ipv4.address }}:{{ apiserver_bind_port }} check when: is_lb_master notify: Restart haproxy # 修复点：只在 master 写 keepalived 脚本，并确保目录存在 - name: Ensure /etc/keepalived exists (masters only) ansible.builtin.file: path: /etc/keepalived state: directory mode: "0755" when: is_lb_master # keepalived 健康检查脚本：haproxy 进程存在即认为健康 - name: Write keepalived health check script (masters only) ansible.builtin.copy: dest: /etc/keepalived/check_haproxy.sh mode: "0755" content: | #!/usr/bin/env bash pgrep haproxy >/dev/null 2>&1 when: is_lb_master # keepalived VRRP：两台都 BACKUP，用 priority 决定谁持有 VIP - name: Write keepalived config ansible.builtin.copy: dest: /etc/keepalived/keepalived.conf mode: "0644" content: | global_defs { router_id {{ inventory_hostname }} } vrrp_script chk_haproxy { script "/etc/keepalived/check_haproxy.sh" interval 2 fall 2 rise 2 } vrrp_instance VI_1 { state BACKUP interface {{ apiserver_vip_iface }} virtual_router_id {{ keepalived_virtual_router_id }} priority {{ 150 if inventory_hostname == lb_masters[0] else 100 }} advert_int 1 authentication { auth_type PASS auth_pass {{ keepalived_auth_pass }} } virtual_ipaddress { {{ apiserver_vip }}/24 } track_script { chk_haproxy } } when: is_lb_master notify: Restart keepalived - name: Enable & start haproxy/keepalived ansible.builtin.systemd: name: "{{ item }}" enabled: true state: started loop: - haproxy - keepalived when: is_lb_master # 确认 haproxy 已经监听 VIP 端口（本地 127.0.0.1:16443） - name: Wait haproxy port listening on masters ansible.builtin.wait_for: host: "127.0.0.1" port: "{{ apiserver_vip_port }}" timeout: 30 when: is_lb_master # ------------------------- # kubeadm init（仅 init master） # ------------------------- - name: Check if cluster already initialized ansible.builtin.stat: path: /etc/kubernetes/admin.conf register: adminconf_stat when: is_init_master # 修复点：apiVersion 使用 v1beta3（与你的 kubeadm 版本匹配） - name: Write kubeadm config ansible.builtin.copy: dest: /root/kubeadm.yaml mode: "0644" content: | apiVersion: kubeadm.k8s.io/v1beta3 kind: ClusterConfiguration kubernetesVersion: "{{ kubeadm_kubernetes_version }}" imageRepository: "{{ kubeadm_image_repository }}" controlPlaneEndpoint: "{{ apiserver_vip }}:{{ apiserver_vip_port }}" networking: podSubnet: "{{ pod_subnet }}" serviceSubnet: "{{ service_subnet }}" dnsDomain: "{{ cluster_domain }}" apiServer: certSANs: - "{{ apiserver_vip }}" - "{{ hostvars[lb_masters[0]].ansible_default_ipv4.address }}" - "{{ hostvars[lb_masters[1]].ansible_default_ipv4.address }}" - "{{ lb_masters[0] }}" - "{{ lb_masters[1] }}" - "localhost" --- apiVersion: kubeadm.k8s.io/v1beta3 kind: InitConfiguration localAPIEndpoint: advertiseAddress: "{{ ansible_default_ipv4.address }}" bindPort: {{ apiserver_bind_port }} nodeRegistration: criSocket: "{{ cri_socket }}" kubeletExtraArgs: node-ip: "{{ ansible_default_ipv4.address }}" when: is_init_master and not adminconf_stat.stat.exists - name: Run kubeadm init ansible.builtin.command: argv: - kubeadm - init - "--config=/root/kubeadm.yaml" - "--upload-certs" - "--ignore-preflight-errors=SystemVerification" - "--ignore-preflight-errors=Swap" register: kubeadm_init_out when: is_init_master and not adminconf_stat.stat.exists failed_when: kubeadm_init_out.rc != 0 - name: Re-check admin.conf after kubeadm init ansible.builtin.stat: path: /etc/kubernetes/admin.conf register: adminconf_stat_after when: is_init_master - name: Ensure /root/.kube exists on init master ansible.builtin.file: path: /root/.kube state: directory mode: "0700" when: is_init_master # 让 init master 上 root 可直接 kubectl - name: Copy admin.conf to /root/.kube/config on init master ansible.builtin.copy: remote_src: true src: /etc/kubernetes/admin.conf dest: /root/.kube/config mode: "0600" when: is_init_master and (adminconf_stat_after.stat.exists | default(false)) # 生成 worker join 命令 - name: Generate worker join command (init master) ansible.builtin.command: argv: - kubeadm - token - create - "--print-join-command" register: join_worker_cmd_raw when: is_init_master and (adminconf_stat_after.stat.exists | default(false)) # 获取 control-plane join 需要的 certificate-key - name: Upload-certs and get certificate key (init master) ansible.builtin.command: argv: - kubeadm - init - phase - upload-certs - "--upload-certs" register: upload_certs_out when: is_init_master and (adminconf_stat_after.stat.exists | default(false)) - name: Extract certificate key ansible.builtin.set_fact: cert_key: "{{ (upload_certs_out.stdout_lines | select('match','^[0-9a-f]{64}$') | list | first) | default('') }}" when: is_init_master and (adminconf_stat_after.stat.exists | default(false)) # 拼出控制面 join 命令：在 worker join 命令基础上增加 --control-plane 与 --certificate-key - name: Build control-plane join command (init master) ansible.builtin.set_fact: join_cp_cmd: " --control-plane --certificate-key {{ cert_key }}" when: is_init_master and (adminconf_stat_after.stat.exists | default(false)) # 把 join 命令保存成脚本文件（便于人工排障/复用） - name: Save join commands to files (init master) ansible.builtin.copy: dest: "{{ item.path }}" mode: "0700" content: | #!/usr/bin/env bash set -e {{ item.cmd }} loop: - { path: "/root/join-worker.sh", cmd: "" } - { path: "/root/join-controlplane.sh", cmd: "" } when: is_init_master and (adminconf_stat_after.stat.exists | default(false)) # 关键：把 join 命令存成 localhost 的 delegate_facts，方便其它节点通过 hostvars['localhost'] 读取 - name: Set join commands as global facts on localhost ansible.builtin.set_fact: global_join_worker: "" global_join_cp: "" delegate_to: localhost delegate_facts: true run_once: true when: is_init_master and (adminconf_stat_after.stat.exists | default(false)) # ------------------------- # join（其余 master / worker） # ------------------------- - name: Check if node already joined ansible.builtin.stat: path: /etc/kubernetes/kubelet.conf register: kubeletconf_stat # 第二台 master 加入 control-plane（仅 lb master，且不是 init master） - name: Join second master as control-plane ansible.builtin.command: "{{ hostvars['localhost'].global_join_cp }}" when: - is_lb_master - not is_init_master - not kubeletconf_stat.stat.exists - hostvars['localhost'].global_join_cp is defined - (hostvars['localhost'].global_join_cp | length) > 0 # worker 加入集群（非 lb master 视为 worker） - name: Join workers ansible.builtin.command: "{{ hostvars['localhost'].global_join_worker }}" when: - (not is_lb_master) - not kubeletconf_stat.stat.exists - hostvars['localhost'].global_join_worker is defined - (hostvars['localhost'].global_join_worker | length) > 0 handlers: # containerd 配置变更后重启 - name: Restart containerd ansible.builtin.systemd: name: containerd state: restarted # haproxy 配置变更后重启 - name: Restart haproxy ansible.builtin.systemd: name: haproxy state: restarted # keepalived 配置变更后重启 - name: Restart keepalived ansible.builtin.systemd: name: keepalived state: restarted # sshd 配置变更后重启 - name: Restart ssh ansible.builtin.systemd: name: ssh state: restarted
一、先给虚拟机加上硬盘 二、在虚拟机中识别 2.1 扫描SCSI总线以添加新设备在Linux系统中，尤其是虚拟化环境（如VMware, KVM, VirtualBox），新添加的SCSI硬盘可以通过触发系统重新扫描SCSI主机总线来识别。#​​首先，查看当前已连接的SCSI主机总线编号​​： ls /sys/class/scsi_host/#​​强制系统重新扫描所有的SCSI主机总线​​： echo "- - -" | sudo tee /sys/class/scsi_host/host*/scan#验证新硬盘是否被识别​​： lsblk2.2 加入扩容整个流程分为四步：​​创建物理卷 -> 扩展卷组 -> 扩展逻辑卷 -> 扩展文件系统​​。#将新磁盘 /dev/sdb 初始化为 LVM 物理卷 (PV)​ sudo pvcreate /dev/sdb #这条命令相当于告诉 LVM：“这块新磁盘交给你来管理”。 # Physical volume "/dev/sdb" successfully created.#将新的物理卷扩展到名为 rl的卷组 (VG) 中 sudo vgextend rl /dev/sdb #这条命令相当于把新来的“员工”（PV /dev/sdb）分配到了现有的“部门”（VG rl）里。现在这个部门就有了额外的100G“人力”（空间）。 #Volume group "rl" successfully extended#查看卷组确认空间，并扩展逻辑卷 (LV)​ sudo vgs # VSize(总大小) 变大了，并且 VFree(剩余空间) 这里显示了加入的 ~100G 空间。#方案A：将所有空闲空间都给 root​ sudo lvextend -l +100%FREE /dev/mapper/rl-root​​#方案B：只扩展指定大小（例如 50G）​ sudo lvextend -L +50G /dev/mapper/rl-root
免责声明：本教程仅供学习使用，24小时内需删除。 需注意：在我国，未经电信主管部门批准使用非法定信道（如 OpenVPN）接入国际网络以及利用网络（包括非法接入国际网络后）进行传播违法信息、实施网络犯罪等违法活动均属于违法行为。一、OpenVPN简介OpenVPN 是一种开源的虚拟专用网络技术，利用加密功能在公共网络上构建安全通道。它具有安全性高、采用先进加密算法和多种认证方式保障数据安全；灵活性强，可适应各种网络环境且支持自定义配置；易于部署，安装和配置相对简便且有丰富文档和社区支持。正因如此，OpenVPN 广泛应用于远程办公，方便员工访问公司资源提高效率并确保安全便利；用于跨地域网络连接，助力多个分支机构实现数据共享与协同工作；还在保护隐私方面发挥重要作用，能隐藏用户真实 IP 地址，防止网络跟踪和监控。二、使用须知使用 OpenVPN 需注意合法性，确保目的合法且符合当地法律规定。要做好安全设置，如选强密码、保管好证书、及时更新。关注网络稳定性，尤其在公共网络。对于 CentOS 系统高于 7.x 以上版本：wget https://git.io/vpn -O openvpn-install.sh && bash openvpn-install.shCentOS 7.x 版本使用一下命令：git clone https://github.com/guoew/openvpn-install.git cd openvpn-install && bash openvpn-install.sh运行该脚本后，会提示要求输入一些参数： 本机IP：这个会默认带出来。 对外的公网IP：如果服务器是公网的或者是内网穿透映射的，就填写外部能访问到的IP。 协议：这里默认选择第一个UDP即可。 监听端口：可以使用默认的端口，当然为了安全性也可以使用其他端口。DNS解析：使用本机配置的DNS服务器即可。编辑/etc/openvpn/userfile.sh这个文件来添加OpenVPN的连接账户：vim /etc/openvpn/userfile.sh添加账户的格式如：用户名 密码[root@localhost ~]# cat /etc/openvpn/userfile.sh axing 123456 a1 123456然后将会生成客户端配置文件/root/client.ovpn将这个文件下载到本机，使用OpenVPN的客户端导入。客户端下载地址： https://openvpn.net/client/根据提示填入添加的账户和密码：然后点击CONNECT进行连接即可：然后启动sudo systemctl start openvpn-server@server sudo systemctl enable openvpn-server@server sudo systemctl status openvpn-server@server防火墙的配置sudo firewall-cmd --add-port=1194/udp --permanent sudo firewall-cmd --reload电脑端下载https://openvpn.net/community-downloads/免责声明：本教程仅供学习使用，24小时内需删除。 需注意：在我国，未经电信主管部门批准使用非法定信道（如 OpenVPN）接入国际网络以及利用网络（包括非法接入国际网络后）进行传播违法信息、实施网络犯罪等违法活动均属于违法行为。
一、项目介绍项目地址：https://github.com/cloudreve/Cloudreve Cloudreve 是出色的自托管文件管理和共享系统。它支持多种云存储提供商，提供灵活存储方案。文件管理功能强大，可上传、下载等操作。文件共享便捷，能设密码和有效期。对团队协作很有用，可精细权限管理和版本控制。还具良好扩展性，不断更新优化，是个人和团队的理想选择，带来高效、便捷、安全的体验。二、功能特点多云存储集成：无缝支持本地存储、远程存储，以及七牛云、阿里云 OSS、腾讯云 COS、又拍云、OneDrive 和 S3 兼容 API，满足多样化的存储需求。 高效上传/下载：提供直接传输的上传和下载功能，支持速度限制，确保资源的高效管理。离线下载能力：通过集成 Aria2，实现离线下载，并使用多个下载节点来优化负载分配。 全面文件操作：支持文件的压缩、解压及批量下载，简化文件管理流程。 WebDAV 支持：全面覆盖所有存储提供商，使文件管理更加便捷。 直观的用户交互：支持拖放上传文件或文件夹，并进行流式上传处理；用户可以通过简单的拖放操作来管理文件。 多用户和群组管理：提供多用户和群组支持，便于组织管理和权限分配。 灵活的共享机制：可以创建带有过期日期的文件和文件夹共享链接，确保共享的安全性和时效性。 丰富的在线预览和编辑：支持在线预览视频、图片、音频及 ePub 文件，并能在线编辑文本和 Office 文档。 个性化定制：提供主题颜色自定义、暗模式、PWA 应用、SPA 和国际化支持，提升用户体验。 一体化解决方案：所有功能开箱即用，简化部署和使用。三、部署教程下载适用于您目标机器操作系统、CPU架构的主程序，直接运行即可。 二进制下载地址： https://github.com/cloudreve/Cloudreve/releases# 解压程序包 tar -zxvf cloudreve_VERSION_OS_ARCH.tar.gz # 赋予执行权限 chmod +x ./cloudreve # 启动 Cloudreve ./cloudreve 浏览器输入ip或者域名:5251 等于密码是随机生成的 后台地址是admin 补充使用下面这个后台命令关了终端任务就断了 ./cloudreve & 解决方法： yum install -y screen screen ./cloudreve 按 Ctrl+A 然后 D 来分离会话，随时可以使用 screen -r 重新连接。 这时候关闭终端不会断了。
1. mkdir mysqlcluster-operator #创建一个名为 mysqlcluster-operator 的目录。 2. cd mysqlcluster-operator #进入刚刚创建的 mysqlcluster-operator 目录。 3. git init #初始化一个新的 Git 仓库，将当前目录标记为一个 Git 仓库以便进行版本控制。 4. touch README.md #创建一个名为 README.md 的文件，通常用于项目的介绍或说明。 5. git add README.md #将 README.md 文件添加到 Git 的暂存区，准备提交到仓库。 6. git commit -m "first commit" #提交暂存区中的文件，并添加提交信息 "first commit"。 7. git remote add origin https://gitee.com/axzys/mysqlcluster-operator.git #将远程仓库的地址（在 Gitee 上）与当前的本地仓库关联，命名为 origin。 8. git push -u origin "master" #将本地仓库的 master 分支推送到远程仓库的 origin，并设置跟踪（-u 参数表示在未来自动与这个远程分支同步）。mkdir webhook cd webhook git init touch README.md git add README.md git commit -m "first commit" git remote add origin https://gitee.com/axzys/webhook.git git push -u origin "master"[root@master01 mysqlcluster-operator]# git push -u origin "master" Username for 'https://gitee.com': 13143087657 Password for 'https://13143087657@gitee.com': Counting objects: 3, done. Delta compression using up to 8 threads. Compressing objects: 100% (2/2), done. Writing objects: 100% (3/3), 1.64 KiB | 0 bytes/s, done. Total 3 (delta 0), reused 0 (delta 0) remote: Powered by GITEE.COM [1.1.5] remote: Set trace flag 7e4992ae To https://gitee.com/axzys/mysqlcluster-operator.git * [new branch] master -> master Counting objects：统计需要推送的对象数目。 Compressing objects：压缩要推送的对象。 Writing objects：将对象写入远程仓库。 remote: Powered by GITEE.COM：表示已经成功连接到 Gitee 并完成操作。 [new branch] master -> master：本地的 master 分支已经成功推送到远程仓库的 master 分支。git add . #添加修改 git commit -m "Your commit message" #提交修改 git push origin master #推送到远程仓库 #推送到其他分支 git checkout -b <branch_name> #创建并切换到新分支 git push origin <branch_name> #推送到新分支 #回滚到之前的提交并推送（如果需要重置远程仓库的状态） git reset --hard abc1234 #回滚到某个提交（假设提交ID为 abc1234） git push origin master --force #强制推送到远程仓库
1.Roles基本概述 通过使用 roles，你可以将复杂的设置和操作流程封装在一个统一的文件夹下，进而实现代码的复用和模块化例如你要部署负载均衡、web服务器、nfs、数据库，那你可以创建四个role每个role都组织管理了好了各白需要的所有元素(包括任务、变量、handler、文件等)的目录结构。2.创建单独一个Role 一个完整的Role里包含的目录和文件可能较多，手动去创建所有这些目录和文件是一件比较烦人的事，好在可以使用ansible-galaxy init ROLE NAME命令来快速创建一个符合Role文件组织规范的框架。[root@lb workspace]# ansible-galaxy init first_role [root@lb workspace]# tree first_role/ first_role/ # 角色名称，或者叫项目名 ├── README.md ├── defaults # 默认的变量（优先级很低） │ └── main.yml ├── files # 存放文件，使用copy模块时自动获取 ├── handlers # 存放触发器的配置 │ └── main.yml ├── meta # 依赖的服务，执行该项目时先执行其他的项目 │ └── main.yml ├── tasks # 默认执行的playbook │ └── main.yml ├── templates # 存放jinja2模板，使用template模块时自动获取 ├── tests │ ├── inventory │ └── test.yml └── vars # 存放变量 └── main.yml3.一些关键介绍 Ansible Roles的依赖关系说明`roles`允许你再使用roles时自动引入其他的roles。role依赖关系存储在roles目录中meta/main.yml文件中。 例如：推送wordpress并解压，前提条件，必须要安装nginx和php，把服务跑起来，才能运行wordpress的页面，此时我们就可以在wordpress的roles中定义依赖nginx和php的roles[root@m01 roles]# vim /etc/ansible/roles/wordpress/meta/main.yml dependencies: - { role: nginx } - { role: php } 如果编写了meta目录下的main.yml文件，那么Ansible会自动先执行meta目录中main.yml文件中的dependencies文件，如上所示，就会先执行nginx和php的安装。Role中有两个地方可以定义变量:roles/xxx/defaults/main.yml:用于定义Role的默认变量roles/xxx/vars/main.yml:用于定义其它变量两个文件之间的区别在于，defaults/main.yml中定义的变量优先级低于vars/main.yml中定义的变量。事实上，defaults/main.yml中的变量优先级几乎是最低的，基本上其它任何地方定义的变量都可以覆盖它4.基于roles机制重构playbook主机IP身份m01192.168.110.110控制端web01192.168.110.97受控端db01192.168.110.163受控端nfs192.168.110.51受控端web02192.168.110.191受控端lb01192.168.110.138受控端配置主机清单[root@manager ~]# cat /etc/ansible/hosts [lb] lb01 ansible_ssh_pass='1' [nfs_server] nfs ansible_ssh_pass='1' [web_group] web01 ansible_ssh_pass='1' web02 ansible_ssh_pass='1' [db_server] db01 ansible_ssh_pass='1' [www:children] lb nfs_server web_group db_server配置hosts[root@manager ~]# cat /etc/hosts # 添加如下内容 192.168.110.138 lb01 192.168.110.51 nfs 192.168.110.91 web01 192.168.110.191 web02 192.168.110.163 db01创建项目及各个role# 1、创建项目目录，项目名就叫roles吧，好理解 mkdir /project mkdir /project/roles # 2、在roles目录下创建一系列的role cd /project/roles # 切换到工作目录下 ansible-galaxy init base # 基础优化role ansible-galaxy init nfs # 部署nfs服务的role ansible-galaxy init web # 部署web服务的role ansible-galaxy init mysql # 部署mysql服务的role ansible-galaxy init lb # 部署lb负载均衡服务的role ansible-galaxy init wordpress # 部署wordpress项目的rolebase role[root@m01 tasks]# cat /project/roles/base/tasks/main.yml # tasks file for base - name: Stop Selinux selinux: state: disabled - name: Create www Group group: name: www gid: 1666 state: present - name: Create www User user: name: www uid: 1666 group: www shell: /sbin/nologin create_home: false state: presentnfs role 准备文件[root@m01 tasks]# cat /project/roles/nfs/files/exports.txt /data 192.168.110.0/24(rw,sync,all_squash) 编写tasks[root@m01 tasks]# cat /project/roles/nfs/tasks/main.yml # tasks file for nfs - name: Install nfs-util yum: name: nfs* state: present - name: mkidr /data file: path: "/data" state: directory - name: Config nfs copy: src: "exports.txt" dest: /etc/exports - name: Start nfs-server systemd: name: nfs-server state: restarted enabled: trueweb role 部署两台web上的Nginx+php环境，并且挂载好nfs共享存储 （1）准备好包和配置文件 nginx的安装源文件 cd /project/roles/web/filescat > nginx.repo << "EOF" [nginx-stable] name=nginx stable repo baseurl=http://nginx.org/packages/centos/$releasever/$basearch/ gpgcheck=1 enabled=1 gpgkey=https://nginx.org/keys/nginx_signing.key module_hotfixes=true [nginx-mainline] name=nginx mainline repo baseurl=http://nginx.org/packages/mainline/centos/$releasever/$basearch/ gpgcheck=1 enabled=0 gpgkey=https://nginx.org/keys/nginx_signing.key module_hotfixes=true EOFnginx转发php-fpm的配置cat > myweb.conf << "EOF" server { listen 8181; server_name localhost; location / { root /usr/share/nginx/html; index index.php index.html; } location ~ \.php$ { fastcgi_pass 127.0.0.1:9000; fastcgi_param SCRIPT_FILENAME /usr/share/nginx/html$fastcgi_script_name; fastcgi_param HTTPS on; include fastcgi_params; } } EOF准备好php-fpm的配置文件cat > www.conf << "EOF" [www] user = www group = www listen = 127.0.0.1:9000 listen.allowed_clients = 127.0.0.1 pm = dynamic pm.max_children = 50 pm.start_servers = 5 pm.min_spare_servers = 5 pm.max_spare_servers = 35 slowlog = /var/opt/remi/php74/log/php-fpm/www-slow.log php_admin_value[error_log] = /var/opt/remi/php74/log/php-fpm/www-error.log php_admin_flag[log_errors] = on php_value[soap.wsdl_cache_dir] = /var/opt/remi/php74/lib/php/wsdlcache EOF（2）编写部署php-fpm+nginx的剧本 vi /project/roles/web/tasks/mian.yml# 1、安装php - name: Gather OS version command: cat /etc/redhat-release register: os_version - name: Extract major version from OS version set_fact: os_major_version: "{{ os_version.stdout.split()[3] | regex_replace('\\..*$', '') }}" - name: Extract minor version from OS version set_fact: os_minor_version: "{{ os_version.stdout.split()[3] | regex_replace('^[^.]*\\.', '') }}" - name: Install rpm package for CentOS 7.9 yum: name: http://rpms.remirepo.net/enterprise/remi-release-7.rpm state: latest when: os_major_version == '7' and os_minor_version == '9.2009' - name: Install rpm package for CentOS 9.3 yum: name: http://rpms.remirepo.net/enterprise/remi-release-9.3.rpm state: latest when: os_major_version == '9' and os_minor_version == '3' - name: Install php-fpm yum: name: - 'php74-php-pdo' - 'php74-php-mbstring' - 'php74-php-cli' - 'php74-php-fpm' - 'php74-php-mysqlnd' state: latest - name: Config php-fpm copy: src: www.conf dest: /etc/opt/remi/php74/php-fpm.d/www.conf notify: restart_php - name: Start php-fpm systemd: name: php74-php-fpm state: restarted enabled: true #2、安装nginx，配置nginx代理php-fpm - name: copy nginx.repo copy: src: nginx.repo dest: /etc/yum.repos.d/nginx.repo - name: Install nginx yum: name: nginx state: latest - name: Config nginx copy: src: myweb.conf dest: /etc/nginx/conf.d/myweb.conf notify: restart_nginx - name: Start nginx server systemd: name: nginx state: restarted enabled: true #3、配置所有web服务挂载nfs - name: 安装nfs yum: name: nfs-utils state: latest - name: 挂载 mount: path: /usr/share/nginx/html src: "{{ nfs_share_dir }}" fstype: nfs opts: defaults state: mounted配置触发器handler[root@m01 base]# cat /project/roles/web/handlers/main.yml # handlers file for web - name: restart_php systemd: name: php74-php-fpm state: restarted - name: restart_nginx systemd: name: nginx state: restarted 创建变量[root@m01 roles]# cat web/vars/main.yml nfs_share_dir: "192.168.110.51:/data"mysql role安装mysql_db模块ansible-galaxy collection install community.mysql编写剧本[root@m01 base]# cat /project/roles/web/handlers/main.yml # handlers file for web - name: restart_php systemd: name: php74-php-fpm state: restarted - name: restart_nginx systemd: name: nginx state: restarted [root@m01 base]# cat /project/roles/mysql/tasks/main.yml # tasks file for mysql - name: Download PyMySQL tar.gz get_url: url: https://files.pythonhosted.org/packages/44/39/6bcb83cae0095a31b6be4511707fdf2009d3e29903a55a0494d3a9a2fac0/PyMySQL-0.8.1.tar.gz dest: /tmp/PyMySQL-0.8.1.tar.gz - name: Extract PyMySQL tar.gz unarchive: src: /tmp/PyMySQL-0.8.1.tar.gz dest: /tmp/ remote_src: yes - name: Install PyMySQL command: cmd: "python setup.py install" chdir: "/tmp/PyMySQL-0.8.1" - name: clear mysql shell: "yum remove mysql* -y" ignore_errors: True - name: Install mariadb yum: name: mariadb* state: latest - name: init maridb shell: "rm -rf /var/lib/mysql/*" - name: Start mariadb systemd: name: mariadb state: restarted enabled: true - name: create database mysql_db: # root登录localhost不允许，需要用套接字登录 login_unix_socket: /var/lib/mysql/mysql.sock # 引用变量名必须加引号，否则报错 name: "{{ my_db.name }}" state: present encoding: "{{ my_db.encoding }}" - name: grant all on *.* to 'bob'@'192.168.110.%' identified by '12345'; mysql_user: login_unix_socket: /var/lib/mysql/mysql.sock # 引用变量名必须加引号，否则报错 name: "{{ my_user.name }}" host: "{{ my_user.host }}" password: "{{ my_user.password }}" priv: "{{ my_user.priv }}" state: present创建变量文件[root@m01 base]# cat /project/roles/mysql/vars/main.yml # vars file for mysql my_db: name: wordpress encoding: utf8mb4 my_user: name: bob host: 192.168.110.% password: 12345 priv: '*.*:ALL' lb role 准备nginx.repo cd /project/roles/lb/filescat > lb/files/nginx.repo << "EOF" [nginx-stable] name=nginx stable repo baseurl=http://nginx.org/packages/centos/$releasever/$basearch/ gpgcheck=1 enabled=1 gpgkey=https://nginx.org/keys/nginx_signing.key module_hotfixes=true [nginx-mainline] name=nginx mainline repo baseurl=http://nginx.org/packages/mainline/centos/$releasever/$basearch/ gpgcheck=1 enabled=0 gpgkey=https://nginx.org/keys/nginx_signing.key module_hotfixes=true EOF准备证书openssl genrsa -out server.key 2048 openssl req -new -x509 -days 3650 -key server.key -out server.crt -subj "/C=CH/ST=mykey/L=mykey/O=mykey/OU=mykey/CN=domain1/CN=www.egon.com/CN=domain3"放置证书到lb/files目录下[root@m01 roles]# mv server.* /project/roles/lb/files/准备配置文件[root@m01 base]# cat /project/roles/lb/files/nginx.conf user nginx; worker_processes auto; error_log /var/log/nginx/error.log; pid /run/nginx.pid; include /usr/share/nginx/modules/*.conf; events { worker_connections 1024; } http { upstream webserver { server 192.168.110.97:8181; server 192.168.110.191:8181; } server { listen 443 ssl; server_name www.egon.com 192.168.110.138; ssl_certificate /etc/nginx/ssl_key/server.crt; ssl_certificate_key /etc/nginx/ssl_key/server.key; location / { proxy_pass http://webserver; # 把真实的访问者ip发给后端web，后端web会据此来拼接静态文件的url地址以便让访问者浏览器发起二次请求 # 如果没有下面的这段内容，后端web会将静态资源的url地址拼成http://webserver/static/img/1.jpg的形式，导致访问者浏览器二次访问失败 proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header Host $http_host; proxy_next_upstream error timeout http_500 http_502 http_503 http_504 http_403 http_404; } } server { listen 80; server_name 192.168.110.138 www.egon.com; rewrite (.*) https://$server_name$1; } } 编写剧本[root@m01 base]# cat /project/roles/lb/tasks/main.yml # tasks file for lb - name: clear nginx shell: > yum remove nginx* -y ; rm -rf /etc/nginx /usr/share/nginx - name: copy nginx.repo copy: src: nginx.repo dest: /etc/yum.repos.d/nginx.repo - name: Install nginx yum: name: nginx state: latest - name: Create dir file: path: /etc/nginx/ssl_key state: directory - name: copy multiple files copy: src: "{{ item.src }}" dest: "{{ item.dest }}" with_items: - { src: 'nginx.conf', dest: '/etc/nginx/nginx.conf'} - { src: 'server.crt', dest: '/etc/nginx/ssl_key/server.crt'} - { src: 'server.key', dest: '/etc/nginx/ssl_key/server.key'} notify: restart_nginx - name: Start nginx server systemd: name: nginx state: restarted enabled: true 配置触发器[root@m01 base]# cat /project/roles/lb/handlers/main.yml # handlers file for lb - name: restart_nginx systemd: name: nginx state: restartedwordpress role 安装包cd /project/roles/wordpress/files wget https://wordpress.org/latest.zip准备好配置文件（配置上数据库相关信息）[root@m01 base]# cat /project/roles/wordpress/files/wp-config.php <?php define( 'DB_NAME', 'wordpress' ); define( 'DB_USER', 'bob' ); define( 'DB_PASSWORD', '12345' ); define( 'DB_HOST', '192.168.110.138' ); define( 'DB_CHARSET', 'utf8mb4' ); define( 'DB_COLLATE', '' ); define( 'AUTH_KEY', 'put your unique phrase here' ); define( 'SECURE_AUTH_KEY', 'put your unique phrase here' ); define( 'LOGGED_IN_KEY', 'put your unique phrase here' ); define( 'NONCE_KEY', 'put your unique phrase here' ); define( 'AUTH_SALT', 'put your unique phrase here' ); define( 'SECURE_AUTH_SALT', 'put your unique phrase here' ); define( 'LOGGED_IN_SALT', 'put your unique phrase here' ); define( 'NONCE_SALT', 'put your unique phrase here' ); $table_prefix = 'wp_'; define( 'WP_DEBUG', false ); if ( ! defined( 'ABSPATH' ) ) { define( 'ABSPATH', __DIR__ . '/' ); } require_once ABSPATH . 'wp-settings.php'; EOF 编写剧本（解压到nfs共享目录里就发布给了所有的web服务）[root@m01 base]# cat /project/roles/lb/handlers/main.yml # handlers file for lb - name: restart_nginx systemd: name: nginx state: restarted [root@m01 base]# cat /project/roles/wordpress/tasks/main.yml --- # tasks file for wordpress - name: mkdir /data copy: src: "latest.zip" dest: "/data" - name: install unzip yum: name: unzip state: present - name: 发布 shell: unzip /data/latest.zip -d /data - name: 传送配置 copy: src: "wp-config.php" dest: "/data/wordpress/wp-config.php"整合为一个playbook编写一个playbook剧本，剧本里面一如各个子role,运行的时候运行这一个playbook即可[root@m01 base]# cat /project/roles/run.yml - name: 优化部分 hosts: all roles: - base - name: 安装nfs hosts: nfs_server roles: - nfs - name: 安装web hosts: web_group roles: # 其实你可以将安装web进一步细分为nginx role与php role，留给你作业了 - web - name: 安装数据库 hosts: db_server roles: - mysql - name: 配置负载均衡和高可用 hosts: lb_server roles: - lb #- keepalived # 高可用role留给你来实现 - name: 发布wordpress项目 hosts: nfs_server roles: - wordpress补充：各个子role里的hosts其实不用指定，run.yml已经指定了ansible-playbook /project/roles/run.yml访问负载均衡：https://192.168.110.138/wordpress/
先下载好镜像下面这个是阿里的镜像站，可以自己选择版本https://mirrors.aliyun.com/centos/7.9.2009/isos/x86_64/?spm=a2c6h.25603864.0.0.18a9f5adJSpgfw1.测试环境准备两台CentOS7.9操作系统的服务器或虚拟机，host01作为服务端，host02作为客户端，要确保客户端必须可以访问到服务端； 准备好CentOS7.9的ISO文件：CentOS-7-x86_64-DVD-2009.iso 关闭防火墙和SELinux，开启防火墙的情况下需要开放80端口，因为下面我使用web软件对外提供服务。2.测试场景测试过程中分为以下两种情况： (1)仅将ISO文件中的安装包上传到服务器中； (2)需要自定义软件包，比如将ISO文件中的安装包和自己下载的软件包一起放在YUM源服务器中； 两种情况也可以理解成一种，实际上它们都是rpm包。3.前提条件需要在服务端安装web服务，这里使用Apache，也可以使用Nginx；安装Apache：yum install httpd -y如果服务端不能访问外面，需要先配置本地YUM源，本地YUM源的方法见本文末尾。启动服务：systemctl start httpd当然服务端不强制使用web服务，也可以使用ftp服务。4.仅使用ISO文件中的安装包这个需求是最简单的，因为ISO文件中自带软件包索引文件，不需要额外创建。4.1挂载ISO文件挂载ISO文件后，可以看到以下目录及文件：重点是Packages和repodata两个目录，KEY文件可以不使用。基本Packages包含所有软件包，repodata包含软件包的索引文件。4.2拷贝ISO中的软件包到服务端创建目录：mkdir /var/www/html/CentOS7.9拷贝文件：cp -a /mnt/Packages /var/www/html/CentOS7.9 cp -a /mnt/repodata /var/www/html/CentOS7.9http://192.168.110.51/CentOS7.9/4.3配置客户端测试[root@localhost yum.repos.d]# cat /etc/yum.repos.d/local.repo [Local_YUM] name=Local_YUM baseurl=http://192.168.110.51/CentOS7.9 enable=1 gpgcheck=0 [root@localhost yum.repos.d]# yum makecache 已加载插件：fastestmirrorLoading mirror speeds from cached hostfileLocal_YUM | 3.6 kB 00:00:00 Local_YUM/group_gz FAILED http://192.168.110.51/CentOS7.9/repodata/a4e2b46586aa556c3b6f814dad5b16db5a669984d66b68e873586cd7c7253301-c7-x86_64-comps.xml.gz: [Errno 14] HTTP Error 404 - Not Found 正在尝试其它镜像。To address this issue please refer to the below wiki article https://wiki.centos.org/yum-errorsIf above article doesn't help to resolve this issue please use https://bugs.centos.org/.Local_YUM/primary_db FAILED http://192.168.110.51/CentOS7.9/repodata/0e82d111931e402316702696ae4178e65673dc0f9c23a2bb012d192cb2625ea1-primary.sqlite.bz2: [Errno 14] HTTP Error 404 - Not Found 正在尝试其它镜像。Local_YUM/filelists_db FAILED http://192.168.110.51/CentOS7.9/repodata/60aa4673bf5fa63d1a7a640ef407b8e48fe5b3337720d844a29a53fec130d6da-filelists.sqlite.bz2: [Errno 14] HTTP Error 404 - Not Found 正在尝试其它镜像。Local_YUM/other_db FAILED http://192.168.110.51/CentOS7.9/repodata/9d6292fa12fd9ca472fafa8c0b348c3e1f139681aa066d41dc53b3563bcd1f53-other.sqlite.bz2: [Errno 14] HTTP Error 404 - Not Found 正在尝试其它镜像。Local_YUM/primary_db FAILED http://192.168.110.51/CentOS7.9/repodata/0e82d111931e402316702696ae4178e65673dc0f9c23a2bb012d192cb2625ea1-primary.sqlite.bz2: [Errno 14] HTTP Error 404 - Not Found 正在尝试其它镜像。Local_YUM/other_db FAILED http://192.168.110.51/CentOS7.9/repodata/9d6292fa12fd9ca472fafa8c0b348c3e1f139681aa066d41dc53b3563bcd1f53-other.sqlite.bz2: [Errno 14] HTTP Error 404 - Not Found 正在尝试其它镜像。Local_YUM/filelists_db FAILED http://192.168.110.51/CentOS7.9/repodata/60aa4673bf5fa63d1a7a640ef407b8e48fe5b3337720d844a29a53fec130d6da-filelists.sqlite.bz2: [Errno 14] HTTP Error 404 - Not Found 正在尝试其它镜像。http://192.168.110.51/CentOS7.9/repodata/0e82d111931e402316702696ae4178e65673dc0f9c23a2bb012d192cb2625ea1-primary.sqlite.bz2: [Errno 14] HTTP Error 404 - Not Found 正在尝试其它镜像。 One of the configured repositories failed (Local_YUM), and yum doesn't have enough cached data to continue. At this point the only safe thing yum can do is fail. There are a few ways to work "fix" this: 1. Contact the upstream for the repository and get them to fix the problem. 2. Reconfigure the baseurl/etc. for the repository, to point to a working upstream. This is most often useful if you are using a newer distribution release than is supported by the repository (and the packages for the previous distribution release still work). 3. Run the command with the repository temporarily disabled yum --disablerepo=Local_YUM ... 4. Disable the repository permanently, so yum won't use it by default. Yum will then just ignore the repository until you permanently enable it again or use --enablerepo for temporary usage: yum-config-manager --disable Local_YUM or subscription-manager repos --disable=Local_YUM 5. Configure the failing repository to be skipped, if it is unavailable. Note that yum will try to contact the repo. when it runs most commands, so will have to try and fail each time (and thus. yum will be be much slower). If it is a very temporary problem though, this is often a nice compromise: yum-config-manager --save --setopt=Local_YUM.skip_if_unavailable=truefailure: repodata/0e82d111931e402316702696ae4178e65673dc0f9c23a2bb012d192cb2625ea1-primary.sqlite.bz2 from Local_YUM: [Errno 256] No more mirrors to try.http://192.168.110.51/CentOS7.9/repodata/0e82d111931e402316702696ae4178e65673dc0f9c23a2bb012d192cb2625ea1-primary.sqlite.bz2: [Errno 14] HTTP Error 404 - Not Found如果报错的话repodata的元数据有问题，这问题是没有后缀手动生成了后缀。mv a4e2b46586aa556c3b6f814dad5b16db5a669984d66b68e873586cd7c7253301 a4e2b46586aa556c3b6f814dad5b16db5a669984d66b68e873586cd7c7253301-c7-x86_64-comps.xml.gz mv 0e82d111931e402316702696ae4178e65673dc0f9c23a2bb012d192cb2625ea1 0e82d111931e402316702696ae4178e65673dc0f9c23a2bb012d192cb2625ea1-primary.sqlite.bz2 mv 60aa4673bf5fa63d1a7a640ef407b8e48fe5b3337720d844a29a53fec130d6da 60aa4673bf5fa63d1a7a640ef407b8e48fe5b3337720d844a29a53fec130d6da-filelists.sqlite.bz2 mv 9d6292fa12fd9ca472fafa8c0b348c3e1f139681aa066d41dc53b3563bcd1f53 9d6292fa12fd9ca472fafa8c0b348c3e1f139681aa066d41dc53b3563bcd1f53-other.sqlite.bz2另一种方式是使用createrepo重新生成元数据:createrepo /path/to/your/local/repo4.3.1 配置YUM文件将原有的配置文件删除，新建一个：rm -rf /etc/yum.repos.d/* touch /etc/yum.repos.d/local.repo #注意文件名必须以repo结尾vi /etc/yum.repos.d/local.repo [Local_YUM] name=Local_YUM baseurl=http://192.168.110.51/CentOS7.9 enable=1 gpgcheck=04.3.2 更新YUM缓存yum clean all yum makecache4.3.3 安装软件包安装一个系统中未安装的软件包，这里以wget软件为例：rpm -qa | grep wget #查看wget软件包是否安装 yum install wget -y #安装wget5.添加自定义安装包到YUM服务器中5.1准备软件包比如需要安装数据库：MariaDB-server-10.11.7，此时YUM服务器是不存在这些软件包的，先把软件包上传到服务器中，如：MariaDB-server目录中包含了MariaDB-server-10.11.7和MariaDB-client-10.11.7所需要的依赖包，但只有rpm包还不能使用，需要为它们创建索引文件。5.2创建索引创建索引需要使用createrepo命令，默认情况下未安装该命令，安装createrepo：yum install createrepo -y比如说安装MariaDB先去官网将其复制并粘贴到 /etc/yum.repos.d 下的文件中（建议将文件命名为 MariaDB.repo 或类似名称）。 # MariaDB 11.4 CentOS repository list - created 2024-07-15 08:20 UTC # https://mariadb.org/download/ [mariadb] name = MariaDB # rpm.mariadb.org is a dynamic mirror if your preferred mirror goes offline. See https://mariadb.org/mirrorbits/ for details. # baseurl = https://rpm.mariadb.org/11.4/centos/$releasever/$basearch baseurl = https://mirror.rackspace.com/mariadb/yum/11.4/centos/$releasever/$basearch module_hotfixes = 1 # gpgkey = https://rpm.mariadb.org/RPM-GPG-KEY-MariaDB gpgkey = https://mirror.rackspace.com/mariadb/yum/RPM-GPG-KEY-MariaDB gpgcheck = 1sudo yum --downloadonly --downloaddir=/root/abc install MariaDB-server MariaDB-client 可以把安装包和依赖包缓存起来并且指定路径创建索引信息：createrepo -v /var/www/html/MariaDB-server因为之前配置ISO文件中的软件包不包含MariaDB 10.11.7版本，所以还无法安装成功。5.3.1配置YUM文件touch /etc/yum.repos.d/mariadb.repo vi /etc/yum.repos.d/mariadb.repo [Mariadb_YUM] name=Mariadb_YUM baseurl=http://192.168.110.51/MariaDB-server enable=1 gpgcheck=05.3.2更新YUM缓存yum install MariaDB-server-11.4.2 MariaDB-client-11.4.2已经可以安装成功了。6.总结至此，我们已经拥有一台在内网可以使用的YUM服务器了；以上也可以将所有rpm软件包放在同一个目录中，使用createrepo命令创建索引，以上为了区分配置了两个目录；重点操作：在客户端配置YUM文件中的baseurl时，路径为repodata目录所在的目录，比如repodata目录在MariaDB-server中，则写成：baseurl=http://192.168.110.51/MariaDB-server附录：(1)挂载ISO文件如果使用的是VMWare虚拟机管理工具，把iso文件挂载到光驱中，然后挂载，比如将镜像到mnt目录下：mount /dev/cdrom /mnt如果是其他方式，可以将iso文件上传到操作系统中，其后挂载：mount -o loop CentOS-7-x86_64-DVD-1810.iso /mnt(2)移除/备份系统当前YUM文件备份当前系统中的YUM源配置文件：mkdir /etc/yum.repos.d/bak && mv /etc/yum.repos.d/*.repo /etc/yum.repos.d/bak(3)创建并配置YUM文件vim /etc/yum.repos.d/localyum.repo增加内容如下：[CentOS7.6] name=CentOS7.6 Local YUMbaseurl=file:///mnt gpgcheck=0 enabled=1参数说明： 参数说明： [CentOS7.6] #名称，需自定义，确保在YUM配置文件中唯一 name=CentOS7.6 Local YUM #描述信息 baseurl=file:///mnt #设置 yum 源目录为mnt，根据挂载的目录而定 gpgcheck=0 #检查GPG-KEY，0为不检查，1为检查 enabled=1 #是否用该yum源，0为禁用，1为使用更新软件包索引：yum clean all yum makecache
drbd参考上一篇博客先明确主服务器的keepalivde的配置drbd关闭时执行的脚本 keepalivde关闭时执行的脚本 故障恢复主服务器自动抢回vip的脚本 脚本记得给执行权限从服务器的keepalivde的配置主服务器失效后，从服务器自动挂载，并且成为主服务器，vip也会跟随过来。 原主服务器正常后，主服务器自动降为从服务器，并且取消挂载，vip也会自动转移走。两台主机操作drbdadm status #确定drbd正常工作 dnf install -y keepalivedalpha节点操作/etc/keepalived/keepalived.conf! Configuration File for keepalived global_defs { notification_email { root@localhost } notification_email_from keepalived@localhost smtp_server 127.0.0.1 smtp_connect_timeout 30 router_id DRBD_HA_MASTER } vrrp_script chk_drbd { script "/etc/keepalived/check_drbd.sh" interval 5 } vrrp_instance VI_1 { state MASTER interface ens160 virtual_router_id 51 priority 150 advert_int 1 authentication { auth_type PASS auth_pass 1111 } track_script { chk_drbd } notify_stop /etc/keepalived/notify_stop.sh ### keepalived服务关闭执行的脚本 notify_master /etc/keepalived/notify_master.sh ### 重新拿回 master 的脚本 virtual_ipaddress { 192.168.110.88 ### vip 请选择一个没有在占用的IP地址 } }通过上面的配置文件，这里需要编写三个脚本，如下：/etc/keepalived/check_drbd.sh#!/bin/bash # Author:hukey systemctl is-active --quiet drbd if [ $? -ne 0 ]; then systemctl start drbd if [ $? -ne 0 ]; then umount /mnt drbdadm secondary drbd0 systemctl stop keepalived fi fi/etc/keepalived/notify_stop.sh#!/bin/bash # Author:hukey time=`date "+%F %H:%M:%S"` echo -e "$time ------notify_stop------\n" >> /etc/keepalived/logs/notify_stop.log umount /mnt drbdadm secondary drbd0 /usr/sbin/fuser -k /mnt &>> /etc/keepalived/logs/notify_stop.log /bin/umount /mnt &>> /etc/keepalived/logs/notify_stop.log echo -e "\n" >> /etc/keepalived/logs/notify_stop.log/etc/keepalived/notify_master.sh#!/bin/bash # Author:hukey time=`date "+%F %H:%M:%S"` echo -e "$time ------notify_master------\n" >> /etc/keepalived/logs/notify_master.log systemctl start drbd &>> /etc/keepalived/logs/notify_master.log for (( i=1; i <=3; i++)) do drbdadm status | egrep -q 'Primary' if [ $? -ne 0 ]; then mount /mnt drbdadm primary drbd0 /bin/mount /dev/drbd0 /mnt &>> /etc/keepalived/logs/notify_master.log break else sleep 3 fi done echo -e "\n" >> /etc/keepalived/logs/notify_master.脚本所需目录和命令yum install -y psmisc mkdir -pv /etc/keepalived/logs------------------------alpha节点操作完毕------------------------bravo节点操作/etc/keepalived/keepalived.conf! Configuration File for keepalived global_defs { notification_email { root@localhost } notification_email_from keepalived@localhost smtp_server 127.0.0.1 smtp_connect_timeout 30 router_id DRBD_HA_BACKUP } vrrp_instance VI_1 { state BACKUP interface ens160 virtual_router_id 51 priority 100 advert_int 1 authentication { auth_type PASS auth_pass 1111 } notify_master /etc/keepalived/notify_master.sh notify_backup /etc/keepalived/notify_backup.sh virtual_ipaddress { 192.168.110.88 } }/etc/keepalived/notify_master.sh#!/bin/bash # Author:hukey time=`date "+%F %H:%M:%S"` echo -e "$time ------notify_master------\n" >> /etc/keepalived/logs/notify_master.log systemctl start drbd &>> /etc/keepalived/logs/notify_master.log for (( i=1; i <=3; i++)) do drbdadm status | egrep -q 'Primary' if [ $? -ne 0 ]; then drbdadm primary drbd0 mount /dev/drbd0 /mnt /bin/mount /dev/drbd0 /mnt &>> /etc/keepalived/logs/notify_master.log break else sleep 3 fi done echo -e "\n" >> /etc/keepalived/logs/notify_master.log/etc/keepalived/notify_backup.sh#!/bin/bash # Author:hukey time=`date "+%F %H:%M:%S"` echo -e "$time ------notify_backup------\n" &>> /etc/keepalived/logs/notify_backup.log umount /mnt drbdadm secondary drbd0 /usr/sbin/fuser -k /data &>> /etc/keepalived/logs/notify_backup.log /bin/umount /dev/drbd0 &>> /etc/keepalived/logs/notify_backup.log echo $time echo -e "\n" &>> /etc/keepalived/logs/notify_backup.log启动服务 两台主机操作#开机自启 systemctl enable drbd keepalived systemctl restart drbd keepalived启动后查看 alpha 节点 VIP 和 目录是否挂载：注意：​ 如果发现 vip 或者 drbd 没有挂载，检查一下配置文件是否有问题。测试1.关闭 alpha 节点 keepalived 服务systemctl stop keepalived2.查看 bravo 节点是否升级为主节点3.alpha开启节点 keepalived 服务 bravo 节点关闭keepalived 服务如果vip会自动漂移和会自动挂载就说明成功了还有一个可以测试的，主服务器systemctl stop drbd,五秒后使用systemctl status drbd查看，如果已经启动说明脚本基本没问题。最后drbd+keepalived就完成了，还可以加上nfs。nfs比较简单就不写了。。。
提前准备好1.主机名提前改好 hostnamectl set-hostname #改主机名字 2.硬盘提前准备好可以用lvs以后方便扩容 fdisk /dev/ #你的硬盘 3.时间要更新好，防火墙关闭 setenforce 0 systemctl stop firewalld iptables -t filter -F开始安装sudo dnf update sudo dnf install https://www.elrepo.org/elrepo-release-9.el9.elrepo.noarch.rpm -y sudo dnf install drbd drbd-utils -y sudo dnf install epel-release -y sudo dnf install kmod-drbd9x -y dnf search drbd #查询drbd安装包加载内核[root@alpha ~]# sudo modprobe drbd [root@alpha ~]# lsmod | grep drbd drbd 983040 0 libcrc32c 16384 5 nf_conntrack,nf_nat,nf_tables,xfs,drbd配置 DRBD 资源文件，确保配置中的节点 IP 地址和磁盘设备路径正确。[root@alpha ~]# cat /etc/drbd.d/drbd0.res resource drbd0 { disk /dev/nvme0n2p1; #要根据自己配置改 device /dev/drbd0; meta-disk internal; on alpha { address 192.168.110.204:7789; #要根据自己配置改 } on bravo { address 192.168.110.201:7789; #要根据自己配置改 } }[root@alpha ~]# cat /etc/drbd.d/global_common.conf global { usage-count no; } common { net { protocol C; } }创建 DRBD 资源元数据在两个节点上执行以下命令来创建 DRBD 资源元数据：sudo drbdadm create-md r0启动系统服务systemctl start drbd#开始 systemctl enable drbd#开机自启在alpha上操作，启动设备并使其成为主节点drbdadm up drbd0 drbdadm primary drbd0如果启用主节点命令失败需要使用以下命令drbdadm primary drbd0 --force在bravo上操作，启动设备drbdadm up drbd0查看同步进程drbdadm status sudo drbdadm -- --overwrite-data-of-peer primary drbd0 #重要这步如果状态不正常就开始同步 drbdadm primary --force drbd0 #强制同步 drbdadm primary drbd0 #切换成主服务器 期间可以看到同步的状态格式化DRBD设备并挂载 【alpha 操作】mkfs.ext4 /dev/drbd0 mkdir -pv /mnt mount /dev/drbd0 /mnt touch /mnt/test/file{1..10}卸载文件系统并切换为备节点umount /mnt drbdadm secondary drbd0在bravo上执行以下命令确认文件drbdadm primary drbd0 mount /dev/drbd0 /mnt ls -l /mnt total 0 -rw-r--r--. 1 root root 0 Jun 10 09:43 file1 -rw-r--r--. 1 root root 0 Jun 10 09:43 file2 -rw-r--r--. 1 root root 0 Jun 10 09:43 file3