Persistent SCP connections (with a script)

Depending on the internet connection, scp connections can stall when transferring files from one computer to another. By default, scp will either wait forever, stalled, or eventually timeout and move on to the next file to be copied. Neither of these are particular useful.

Below is an example of a BASH script which operates more intelligently, checking more rapidly whether the connection is stalled and if so, retrying with the same file until it completes. This script requires you to have set up Passwordless SSH from the local machine to the remote one.

The script, called SCPAlive, takes arguments just as scp:

 user:~> scpalive myfile1 myfile2 ... user@remote.machine:~/myfolder1/myfolder2/...

Here is the script:

#!/bin/bash -f

#
# SCPAlive: persistent SCP connections script
#

# Populate an array of the passed arguments
i=0;
for arg in $@; do
    i=`echo $i+1|bc`;    
    files[$i]="$arg";
done

# get number of arguments
nfiles=$#;

# the target location is the last argument
target=${files[${nfiles}]};
nfiles=`echo $#-1|bc`;

echo "target is $target";
echo "nfiles is $nfiles";

# break up the target into the host name and the directory name
colpos=`expr index $target :`
host=${target:0:colpos-1}
let colpos=$colpos+1;
remotedir=${target:colpos-1}


i=1;
while [ "$i" -le "$nfiles" ]; do
    file=${files[${i}]};

    # NOTE: this script is designed to go into subfolders and recursively SCP everything in them
    #       If this behavior is not desired, comment out the commands in this part of the IF 
    #       statement below

    # if the file is a directory, go into it and call scpalive
    if [ -d $file ]; then
        # make new directory on server (in case one does not exist; scp will fail otherwise)
	ssh $host "mkdir "$remotedir/$file
        cd $file;
        scpalive  `ls` $target/$file
        cd ..
	i=`echo $i+1|bc`;

    else
        # copy this file. The two options specify the interval to check if the connection
        # is alive, and how many times to check (by default, it checks 3 times), before
        # throwing an error. The values are in seconds

	scp -o ServerAliveInterval=10 -o ServerAliveCountMax=1 -B $file $target;

        # if the command returns an error status, retry. Otherwise, continue
	if [ $? == 0 ]; then
	    i=`echo $i+1|bc`;
#       # delete the file
#	rm -v $file;
	fi 
    fi
done