How to copy files between two nodes using ansible
AnsibleAnsible Problem Overview
I need to copy file form machine A to machine B whereas my control machine from where i run all my ansible tasks is machine C(local machine)
I have tried the following:
Use scp command in shell module of ansible
hosts: machine2
user: user2
tasks:
- name: Copy file from machine1 to machine2
shell: scp user1@machine1:/path-of-file/file1 /home/user2/file1
This approach just goes on and on never ends.
use fetch & copy modules
hosts: machine1
user: user1
tasks:
- name: copy file from machine1 to local
fetch: src=/path-of-file/file1 dest=/path-of-file/file1
hosts: machine2
user: user2
tasks:
- name: copy file from local to machine2
copy: src=/path-of-file/file1 dest=/path-of-file/file1
This approach throws me an error as follows:
error while accessing the file /Users/<myusername>/.ansible/cp/ansible-ssh-machine2-22-<myusername>, error was: [Errno 102] Operation not supported on socket: u'/Users/<myusername>/.ansible/cp/ansible-ssh-machine2-22-<myusername>'
Any suggestions would be helpful.
Ansible Solutions
Solution 1 - Ansible
As ant31 already pointed out you can use the synchronize
module to this. By default, the module transfers files between the control machine and the current remote host (inventory_host
), however that can be changed using the task's delegate_to
parameter (it's important to note that this is a parameter of the task, not of the module).
You can place the task on either ServerA
or ServerB
, but you have to adjust the direction of the transfer accordingly (using the mode
parameter of synchronize
).
Placing the task on ServerB
- hosts: ServerB
tasks:
- name: Transfer file from ServerA to ServerB
synchronize:
src: /path/on/server_a
dest: /path/on/server_b
delegate_to: ServerA
This uses the default mode: push
, so the file gets transferred from the delegate (ServerA
) to the current remote (ServerB
).
This might sound like strange, since the task has been placed on ServerB
(via hosts: ServerB
). However, one has to keep in mind that the task is actually executed on the delegated host, which in this case is ServerA
. So pushing (from ServerA
to ServerB
) is indeed the correct direction. Also remember that we cannot simply choose not to delegate at all, since that would mean that the transfer happens between the control machine and ServerB
.
Placing the task on ServerA
- hosts: ServerA
tasks:
- name: Transfer file from ServerA to ServerB
synchronize:
src: /path/on/server_a
dest: /path/on/server_b
mode: pull
delegate_to: ServerB
This uses mode: pull
to invert the transfer direction. Again, keep in mind that the task is actually executed on ServerB
, so pulling is the right choice.
Solution 2 - Ansible
To copy remote-to-remote files you can use the [synchronize][1] module with 'delegate_to: source-server
' keyword:
- hosts: serverB
tasks:
- name: Copy Remote-To-Remote (from serverA to serverB)
synchronize: src=/copy/from_serverA dest=/copy/to_serverB
delegate_to: serverA
This playbook can run from your machineC. [1]: http://docs.ansible.com/synchronize_module.html
Solution 3 - Ansible
If you need to sync files between two remote nodes via ansible you can use this:
- name: synchronize between nodes
environment:
RSYNC_PASSWORD: "{{ input_user_password_if_needed }}"
synchronize:
src: rsync://user@remote_server:/module/
dest: /destination/directory/
// if needed
rsync_opts:
- "--include=what_needed"
- "--exclude=**/**"
mode: pull
delegate_to: "{{ inventory_hostname }}"
when on remote_server
you need to startup rsync with daemon mode. Simple example:
pid file = /var/run/rsyncd.pid
lock file = /var/run/rsync.lock
log file = /var/log/rsync.log
port = port
[module]
path = /path/to/needed/directory/
uid = nobody
gid = nobody
read only = yes
list = yes
auth users = user
secrets file = /path/to/secret/file
Solution 4 - Ansible
I was able to solve this using local_action to scp to file from machineA to machineC and then copying the file to machineB.
Solution 5 - Ansible
If you want to do rsync and use custom user and custom ssh key, you need to write this key in rsync options.
---
- name: rsync
hosts: serverA,serverB,serverC,serverD,serverE,serverF
gather_facts: no
vars:
ansible_user: oracle
ansible_ssh_private_key_file: ./mykey
src_file: "/path/to/file.txt"
tasks:
- name: Copy Remote-To-Remote from serverA to server{B..F}
synchronize:
src: "{{ src_file }}"
dest: "{{ src_file }}"
rsync_opts:
- "-e ssh -i /remote/path/to/mykey"
delegate_to: serverA
Solution 6 - Ansible
You can use deletgate
with scp
too:
- name: Copy file to another server
become: true
shell: "scp -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null admin@{{ inventory_hostname }}:/tmp/file.yml /tmp/file.yml"
delegate_to: other.example.com
Because of delegate
the command is run on the other server and it scp
's the file to itself.
Solution 7 - Ansible
A simple way to used copy module to transfer the file from one server to another
Here is playbook
---
- hosts: machine1 {from here file will be transferred to another remote machine}
tasks:
- name: transfer data from machine1 to machine2
copy:
src=/path/of/machine1
dest=/path/of/machine2
delegate_to: machine2 {file/data receiver machine}
Solution 8 - Ansible
In 2021 you should install wrapper:
ansible-galaxy collection install ansible.posix
And use
- name: Synchronize two directories on one remote host.
ansible.posix.synchronize:
src: /first/absolute/path
dest: /second/absolute/path
delegate_to: "{{ inventory_hostname }}"
Read more:
> https://docs.ansible.com/ansible/latest/collections/ansible/posix/synchronize_module.html
Checked on:
ansible --version
ansible 2.10.5
config file = /etc/ansible/ansible.cfg
configured module search path = ['/home/daniel/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
ansible python module location = /usr/lib/python3.9/site-packages/ansible
executable location = /sbin/ansible
python version = 3.9.1 (default, Dec 13 2020, 11:55:53) [GCC 10.2.0]