Secure copy files ( SCP ) between two EC2 instances in AWS

We are extensively using the Amazon EC2 instances for hosting our applications and require setup of the communication between two of more instances for file copy or service discovery. In this post, I am going to show how to setup two instances to communicate each other and do a SCP ( Secure copy from one instance to another ).

Security groups

First and foremost we need to make sure  that both instances are under the same security group in AWS. Also we need to enable all the inbound traffic between the servers when they are originating from the same security group.

For that go to the Security group and click on Inbound tab
Click on Add Rule and  add the rule as below

Select the Port Range as ‘Custom IP’ and then start typing the name of the security group in the text box. This will show suggestions and select the one corresponding to your security requirement.

Preparing source server ( System where scp is initiated)

These are the steps to be completed on the server where you want to run the SCP command. So this will be the source system. If you want all servers to be source, you will need to perform the following steps on each of them.

1.  We need to generate the keypair on server 1 on which you plan to run scp
2.  Login to server through SSH and run the following command :
    ssh-keygen -t rsa
3.  It usually shows location where files will be generated
    /root/.ssh/id_rsa or /home/ec2-user/.ssh/id_rsa
4.  If this prompts you to select a file, you can either suggest a new name or use the     default one ( id_rsa)
5. Do not enter any passphrase ( leave as empty and hit [Enter] )

Preparing destination server

These are the steps to be performed on the server(s) where you would like to have a file copied to ( destincation for scp command ).

Edit the sshd_config file

1. Check the “sshd_config” on the server
2. Typically it’s present in :
   /etc/ssh/sshd_config
3. Please uncomment following two lines in sshd_config
   RSAAuthentication yes
   PubkeyAuthentication yes
4. Note => If you are logged in as EC2 User. Please do sudo su because only then it will  allow editing of file /etc/ssh/sshd_config

Adding to the authorized_keys

1. Now find authorized_keys file of server2
2. Location of authorized_keys file of server 2 as
   /home/ec2-user/.ssh/authorized_keys OR /root/.ssh/authorized_keys
3. Next step is to append contents of id_rsa.pub file of source server to authorized_keys file of destination server. If you specified a different name for the rsa file, it will be <yourname>.pub
4. Location of id_rsa.pub file of server 1 as
   /home/ec2-user/.ssh/id_rsa.pub OR /root/.ssh/id_rsa.pub
5. Append the content to the authorized_keys file

Perform SCP from source to destination

Once the above steps are completed in source and destination, we can perform SCP using the following command:

> scp /source/file/path username@privateip:/destination/file/path

Eg: To copy a file test1.txt in /opt/source in source server  to /opt/dest in destination server

> scp /opt/source/test1.txt root@privateip:/opt/dest

Hope this helps someone looking for a way to copy files between EC2 instances. Let me know if you have any queries on this.

regards
S