ansible https://docs.ansible.com/
Ansible优化 https://ansible.leops.cn/advanced/optimization/
Ansible Best Practices
inventories/
production/
hosts # inventory file for production servers
group_vars/
group1.yml # here we assign variables to particular groups
group2.yml
host_vars/
hostname1.yml # here we assign variables to particular systems
hostname2.yml
staging/
hosts # inventory file for staging environment
group_vars/
group1.yml # here we assign variables to particular groups
group2.yml
host_vars/
stagehost1.yml # here we assign variables to particular systems
stagehost2.yml
library/ # if any custom modules, put them here (optional)
module_utils/ # if any custom module_utils to support modules, put them here (optional)
filter_plugins/ # if any custom filter plugins, put them here (optional)
site.yml # master playbook
webservers.yml # playbook for webserver tier
dbservers.yml # playbook for dbserver tier
files/ # here we assign files for simple plays
plays/ # here we assign plays as the entrance
tasks/ # here we assign tasks for plays to call
roles/
common/ # this hierarchy represents a "role"
tasks/ #
main.yml # <-- tasks file can include smaller files if warranted
handlers/ #
main.yml # <-- handlers file
templates/ # <-- files for use with the template resource
ntp.conf.j2 # <------- templates end in .j2
files/ #
bar.txt # <-- files for use with the copy resource
foo.sh # <-- script files for use with the script resource
vars/ #
main.yml # <-- variables associated with this role
defaults/ #
main.yml # <-- default lower priority variables for this role
meta/ #
main.yml # <-- role dependencies
library/ # roles can also include custom modules
module_utils/ # roles can also include custom module_utils
lookup_plugins/ # or other types of plugins, like lookup in this case
webtier/ # same kind of structure as "common" was above, done for the webtier role
monitoring/ # ""
fooapp/ # ""
提升 Ansible 执行效率的插件
众所周知,Ansible 是基于 ssh(当然还有 telnet,winrm 等连接插件)的自动化配置管理工具,其简单易用,无 agent 式的工作方式在很多场景中都有不少优势,不过也是由于这种工作方式导致了它没有其他 c/s 类的工具执行效率高,饱受其他 C/S 类工具使用者的讥讽,对此,Ansible 官方也对 Ansible 的速度效率做了不少优化手段。
参数名 / 优化类别 | 说明 | ||
---|---|---|---|
fact cache | 将 facts 信息第一次收集后缓存到 memory 或者 redis 或者文件中。 |
||
gather_subset | 可选择性的收集 network ,hardware 等信息,而不是全部 |
||
control_path | 开启 ssh socket 持久化,复用 ssh 连接 |
||
pipelinling | 开启 ssh pipelining , 客户端从管道中读取执行渲染后的脚本,而不是在客户端创建临时文件 |
||
fork | 提高并行执行主机的数量 | ||
serial | 将 play_hosts``① 中主机再分批执行 |
||
strategy | 默认 linear , 每个主机的单个 task 执行完成会等待其他都完成后再执行下个任务,设置 free 可不等待其他主机,继续往下执行(看起来会比较乱),还有一个选项 host_pinned ,我也不知道干嘛的 |
无意发现了一个 Mitogen 的 Ansible plugin(strategy plugin),当前已迭代到 0.29 版本(目前只支持2.9版本),看介绍说能提升 1.2x ~ 7x 以上的执行效率,着实惊人!
大体就是执行过程中主机使用一个连接(默认每执行一个 task 或者 loop 循环都会重新打开一次连接的);渲染的执行代码暂存于内存中;减少多路复用 ssh 隧道的时间消耗;减少临时文件传输的带宽;代码重用,避免代码的重新编译成本等
①. play_hosts
为内置参数,指当前正在执行的 playbook 中的主机列表
②. 尽可能快的
指到运行模块前的阶段h
Download and extract mitogen-0.2.9.tar.gz
Modify ansible.cfg
[defaults]
strategy_plugins = /path/to/mitogen-0.2.9/ansible_mitogen/plugins/strategy
strategy = mitogen_linear
https://networkgenomics.com/ansible/ https://mitogen.networkgenomics.com/ansible_detailed.html
ansible.cfg 配置解析
| ansible.cfg 不影响执行结果但合理的配置会有效提升效率
# 配置文件路径(优先级)
./ansible.cfg
/etc/ansible/ansible.cfg
# 配置文件内容
[defaults]
#inventory = /etc/ansible/hosts
#log_path = /var/log/ansible.log
forks = 100 # 设置并发数
host_key_checking = False # 不检查 SSH 主机登录的密钥
display_skipped_hosts = False # 不显示已跳过的主机
retry_files_enabled = False # 不创建任务失败后的重试文件
# 按照 1d 设置 setup 缓存,优化执行效率
gathering = smart
fact_caching_timeout = 86400
fact_caching = jsonfile
fact_caching_connection = /tmp
roles_path = ./roles
[ssh_connection]
pipelining = True
ssh_args = -o ConnectionAttempts=100 -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ForwardAgent=yes
[inventory]
enable_plugins = yaml, ini
核心用法
Linux
# 检测 ansible 是否可以正常访问主机
ansible-playbook -i hosts playbooks/ping.yml -v
# 配置好 inventory,执行以下命令创建用户并建立信任关系
ansible-playbook -i hosts playbooks/user/default.yml -v
# 配置时间同步 / 进程服务 / 基线文件
ansible-playbook -i hosts playbooks/baseline/cfgset.yml -v
ansible-playbook -i hosts playbooks/baseline/cfgset.yml -v --tags="repo"
ansible-playbook -i hosts playbooks/baseline/cfgset.yml -v --skip-tags="ntp,repo"
# 更新系统软件包和补丁包
ansible-playbook -i hosts playbooks/baseline/pakset.yml -v
# 修改用户密码
ansible-playbook -i hosts_changepw playbooks/user/changepw.yml -v -e "@userpass.json"
# 备份配置,支持自定义日期命名,默认为 "%Y%m%d"
ansible-playbook -i hosts backup/backup.yml -v
# 恢复配置,支持按日期目录全局或者局部主机恢复
ansible-playbook -i hosts backup/restore.yml -v -e "var_backup_date=20180305"
Windows
# 检测 ansible 是否可以正常访问主机
ansible-playbook -i hosts win_playbooks/ping.yml -v
# 配置好 inventory,执行以下命令创建用户并建立信任关系
ansible-playbook -i hosts win_playbooks/user/default.yml -v
# 配置时间同步 / 进程服务 / 基线文件
ansible-playbook -i hosts win_playbooks/baseline/cfgset.yml -v
ansible-playbook -i hosts win_playbooks/baseline/cfgset.yml -v --tags="wsus"
ansible-playbook -i hosts win_playbooks/baseline/cfgset.yml -v --skip-tags="ntp,wsus"
# 更新系统软件包和补丁包
ansible-playbook -i hosts win_playbooks/baseline/pakset.yml -v
# 修改用户密码
ansible-playbook -i win_hosts_changepw win_playbooks/user/changepw.yml -v -e "@userpass.json"
# 备份配置,支持自定义日期命名,默认为 "%Y%m%d"
ansible-playbook -i win_hosts win_backup/backup.yml -v
# 恢复配置,支持按日期目录全局或者局部主机恢复
ansible-playbook -i win_hosts win_backup/restore.yml -v -e "var_backup_date=20180305"
Where X=ansible
https://learnxinyminutes.com/docs/ansible/
---
"{{ Ansible }}" is an orchestration tool written in Python.
...
---
- hosts: apache
vars:
apache2_log_level: "warn"
handlers:
- name: restart apache
service:
name: apache2
state: restarted
enabled: True
notify:
- Wait for instances to listen on port 80
become: True
- name: reload apache
service:
name: apache2
state: reloaded
notify:
- Wait for instances to listen on port 80
become: True
- name: Wait for instances to listen on port 80
wait_for:
state: started
host: localhost
port: 80
timeout: 15
delay: 5
tasks:
- name: Update cache
apt:
update_cache: yes
cache_valid_time: 7200
become: True
- name: Install packages
apt:
name={{ item }}
with_items:
- apache2
- logrotate
notify:
- restart apache
become: True
- name: Configure apache2 log level
lineinfile:
dest: /etc/apache2/apache2.conf
line: "LogLevel {{ apache2_log_level }}"
regexp: "^LogLevel"
notify:
- reload apache
become: True
...
# Universal way
$ pip install ansible
# Debian, Ubuntu
$ apt-get install ansible
# Command pings localhost (defined in default inventory: /etc/ansible/hosts)
$ ansible -m ping localhost
# You should see this output
localhost | SUCCESS => {
"changed": false,
"ping": "pong"
}
$ ansible -m ping all
$ ansible -m shell -a 'date; whoami' localhost #hostname_or_a_group_name
$ ansible -m command -a 'date; whoami' # FAILURE
$ ansible -m command -a 'date' all
$ ansible -m command -a 'whoami' all
- hosts: all
tasks:
- name: "ping all"
ping:
- name: "execute a shell command"
shell: "date; whoami; df -h;"
$ ansible-playbook path/name_of_the_playbook.yml
localhost
[some_group]
hostA.mydomain.com
hostB.localdomain
1.2.3.4
[a_group_of_a_groups:children]
some_group
some_other_group
- hosts: all
tasks:
- name: "ping all"
ping:
- name: "execute a shell command"
shell: "date; whoami; df -h;"
roles:
- some_role
- { role: another_role, some_variable: 'learnxiny', tags: ['my_tag'] }
pre_tasks:
- name: some pre-task
shell: echo 'this task is the last, but would be executed before roles, and before tasks'
$ # The following example contains a shell-prompt to indicate the venv and relative path
$ git clone git@github.com:sirkubax/ansible-for-learnXinYminutes.git
user@host:~/$ cd ansible-for-learnXinYminutes
user@host:~/ansible-for-learnXinYminutes$ source environment.sh
$
$ # First lets execute the simple_playbook.yml
(venv) user@host:~/ansible-for-learnXinYminutes$ ansible-playbook playbooks/simple_playbook.yml
$ source environment.sh
$ # Now we would run the above playbook with roles
(venv) user@host:~/ansible-for-learnXinYminutes$ ansible-playbook playbooks/simple_role.yml
roles/
some_role/
defaults/ # contains default variables
files/ # for static files
templates/ # for jinja templates
tasks/ # tasks
handlers/ # handlers
vars/ # more variables (higher priority)
meta/ # meta - package (role) info
playbooks/roles/simple_apache_role/
├── tasks
│ └── main.yml
└── templates
└── main.yml
# read playbooks/lookup.yml
# then run
(venv) user@host:~/ansible-for-learnXinYminutes$ ansible-playbook playbooks/lookup.yml
ansible -m shell -a 'echo "{{ my_variable }}"' -e 'my_variable="{{ lookup("pipe", "date") }}"' localhost
ansible -m shell -a 'echo "{{ my_variable }}"' -e 'my_variable="{{ lookup("pipe", "hostname") }}"' all
# Or use in playbook
(venv) user@host:~/ansible-for-learnXinYminutes$ ansible-playbook playbooks/lookup.yml
(venv) user@host:~/ansible-for-learnXinYminutes$ ansible-playbook playbooks/register_and_when.yml
---
- hosts: localhost
tasks:
- name: check the system capacity
shell: df -h /
register: root_size
- name: debug root_size
debug:
msg: "{{ root_size }}"
- name: debug root_size return code
debug:
msg: "{{ root_size.rc }}"
# when: example
- name: Print this message when return code of 'check the system capacity' was ok
debug:
msg: "{{ root_size.rc }}"
when: root_size.rc == 0
...
---
- hosts: localhost
tasks:
- name: check the system capacity
shell: df -h /
when: some_variable in 'a string'
roles:
- { role: mid_nagios_probe, when: allow_nagios_probes }
...
ansible-playbook playbooks/simple_playbook.yml --tags=tagA,tag_other
ansible-playbook playbooks/simple_playbook.yml -t tagA,tag_other
There are special tags:
always
--skip-tags can be used to exclude a block of code
--list-tags to list available tags
ansible-playbook playbooks/simple_playbook.yml --limit localhost
--limit my_hostname
--limit groupname
--limit some_prefix*
--limit hostname:group #JM
Some static content
{{ a_variable }}
{% for item in loop_items %}
this line item is {{ item }}
{% endfor %}
$ source environment.sh
$ # Now we would run the above playbook with roles
(venv) user@host:~/ansible-for-learnXinYminutes$ ansible-playbook playbooks/simple_role.yml --tags apache2
ansible -m shell -a 'echo {{ my_variable }}' -e 'my_variable=something, playbook_parameter=twentytwo' localhost
# check part of this playbook: playbooks/roles/sys_debug/tasks/debug_time.yml
- local_action: shell date +'%F %T'
register: ts
become: False
changed_when: False
- name: Timestamp
debug: msg="{{ ts.stdout }}"
when: ts is defined and ts.stdout is defined
become: False
# get first item of the list
{{ some_list | first() }}
# if variable is undefined - use default value
{{ some_variable | default('default_value') }}
# Try (this would fail)
$ ansible-playbook playbooks/vault_example.yml
$ echo some_very_very_long_secret > ~/.ssh/secure_located_file
# in ansible.cfg set the path to your secret file
$ vi ansible.cfg
ansible_vault_password_file = ~/.ssh/secure_located_file
#or use env
$ export ANSIBLE_VAULT_PASSWORD_FILE=~/.ssh/secure_located_file
$ ansible-playbook playbooks/vault_example.yml
# encrypt the file
$ ansible-vault encrypt path/somefile
# view the file
$ ansible-vault view path/somefile
# check the file content:
$ cat path/somefile
# decrypt the file
$ ansible-vault decrypt path/somefile
$ etc/inv/ec2.py --refresh
$ ansible -m ping all -i etc/inv/ec2.py
vi ansible.cfg
# set this to:
callback_whitelist = profile_tasks
vi ansible.cfg
# if set to a persistent type (not 'memory', for example 'redis') fact values
# from previous runs in Ansible will be stored. This may be useful when
# wanting to use, for example, IP information from one group of servers
# without having to talk to them in the same playbook run to get their
# current IP information.
fact_caching = jsonfile
fact_caching_connection = ~/facts_cache
fact_caching_timeout = 86400
# recreate ansible 2.x venv
$ rm -rf venv2
$ source environment2.sh
# execute playbook
(venv2)$ ansible-playbook playbooks/ansible1.9_playbook.yml # would fail - deprecated syntax
# now lets install ansible 1.9.x next to ansible 2.x
(venv2)$ deactivate
$ source environment.1.9.sh
# execute playbook
(venv1.9)$ ansible-playbook playbooks/ansible1.9_playbook.yml # works!
# please note that you have both venv1.9 and venv2 present - you need to (de)activate one - that is all
- name: Ensure the httpd service is running
service:
name: httpd
state: started
become: true
ansible -m ping web*
ansible -m ping web*:!backend:monitoring:&allow_change
参考
Ansible 学习路径
Ansible Documentation
ansible-workshops
Ansible入门
Ansible Wiki