Secure copy files ( SCP ) between two EC2 instances in AWS

We are extensively using the Amazon EC2 instances for hosting our applications and require setup of the communication between two of more instances for file copy or service discovery. In this post, I am going to show how to setup two instances to communicate each other and do a SCP ( Secure copy from one instance to another ).

Security groups

First and foremost we need to make sure  that both instances are under the same security group in AWS. Also we need to enable all the inbound traffic between the servers when they are originating from the same security group.

For that go to the Security group and click on Inbound tab
Click on Add Rule and  add the rule as below

Selection_015

Select the Port Range as ‘Custom IP’ and then start typing the name of the security group in the text box. This will show suggestions and select the one corresponding to your security requirement.

Preparing source server ( System where scp is initiated)

These are the steps to be completed on the server where you want to run the SCP command. So this will be the source system. If you want all servers to be source, you will need to perform the following steps on each of them.

  1.  We need to generate the keypair on server 1 on which you plan to run scp
  2.  Login to server through SSH and run the following command :
     ssh-keygen -t rsa
  3.  It usually shows location where files will be generated
     /root/.ssh/id_rsa or /home/ec2-user/.ssh/id_rsa
  4.  If this prompts you to select a file, you can either suggest a new name or use the     default one ( id_rsa)
  5. Do not enter any passphrase ( leave as empty and hit [Enter] )

MICROIDEATION APP: Programming and tech topics explained as quick learning cards ( ideations ) .
We have launched our new mobile app for learning programming and other tech-based topics in under 30 seconds. The content is crafted by subject-matter-experts on the field and each topic is explained in 500 or fewer characters with examples and sample code. You can swipe for the next content or freeze and follow a particular topic. Each content is self-contained and complete with links to related ideations. You can get it free ( no registration required ) in the play store now.

Visit : https://portal.microideation.com/about

Preparing destination server

These are the steps to be performed on the server(s) where you would like to have a file copied to ( destincation for scp command ).

Edit the sshd_config file

  1. Check the “sshd_config” on the server
  2. Typically it’s present in :
    /etc/ssh/sshd_config
  3. Please uncomment following two lines in sshd_config
    RSAAuthentication yes
    PubkeyAuthentication yes
  4. Note => If you are logged in as EC2 User. Please do sudo su because only then it will  allow editing of file /etc/ssh/sshd_config

Adding to the authorized_keys

  1. Now find authorized_keys file of server2
  2. Location of authorized_keys file of server 2 as
    /home/ec2-user/.ssh/authorized_keys OR /root/.ssh/authorized_keys
  3. Next step is to append contents of id_rsa.pub file of source server to authorized_keys file of destination server. If you specified a different name for the rsa file, it will be <yourname>.pub
  4. Location of id_rsa.pub file of server 1 as
    /home/ec2-user/.ssh/id_rsa.pub OR /root/.ssh/id_rsa.pub
  5. Append the content to the authorized_keys file

Perform SCP from source to destination

Once the above steps are completed in source and destination, we can perform SCP using the following command:

> scp /source/file/path username@privateip:/destination/file/path

Eg: To copy a file test1.txt in /opt/source in source server  to /opt/dest in destination server

> scp /opt/source/test1.txt root@privateip:/opt/dest

Hope this helps someone looking for a way to copy files between EC2 instances. Let me know if you have any queries on this.

regards
S

You may also like...

2 Responses

  1. Rocio says:

    Can scp be scheduled?
    Is there another way to transfer information from one db to another if I have different webs running on cloud but I still need to share someinformation between them

  1. May 30, 2016

    […] SCP (Secure copy ) of files from application servers to the ELK server path. Please see my post on how to setup SCP between two servers in AWS. I have created a script that would copy the file with a date suffix to the […]

Leave a Reply

Your email address will not be published. Required fields are marked *