Log in

28 May 2008 @ 12:08 pm
sudo-rsync: Run rsync with root privs while still using regular user's SSH keys  
This is dated 2008-05-24, and is currently the newest script in my collection.   sudo-rsync lets you perform a data & metadata copy as the root user (so perms & ownership are preserved) without forcing you to log in to the remote system as root (good for Ubuntu systems and anyone else appropriately paranoid about allowing root network logins.)


sudo-rsync /source/dir/ remotehost:/remote/dir/


Sometimes you want to copy a set of files and directories  (including block and character devices, named pipes, etc) while preserving their access permissions, timestamps, ownership, and group membership.  In other words, when you want to exactly replicate not just the content of some data, but the metadata as well.  You might do this, for example, when migrating data to a new disk or a new server.

There are plenty of tools that will do that, but all of them need to be run as root in order to preserve the metadata onto the target disk.  (They may also need root access in order to read the file contents, if the source files are owned by various users.)  My tools of choice for this purpose are tar and rsync.  I use tar when I'm just copying a few megabytes of files once or for the first time; I use rsync if the files have already been copied once but need to be resynchronized after changes were made, or just to resume a large copy after, say, the source machine crashed.

If the goal is to transfer the data & metadata to another computer, you also need a tool to be your network transport.  SSH is my tool of choice for this sort of thing.  (Although I've also used netcat and socat in certain situations.)  In general, the data & metadata copy phase of the process needs to be run as root, but the network transport phase of the process does not.

tar'ing between two machines is easy.  Assume I'm logged in as a non-root user which is allowed to run sudo, and which has SSH key access to a remote account which is also allowed to run sudo.  Then do this:

    ( cd /source/dir && sudo tar cf - * ) | ssh remotehost ' ( cd /remote/dir && sudo tar xf - --preserve )'

Pretty easy.  The ampersands are nice because if the cd fails, nothing gets copied--not true if you used semicolons.  Notice that both tar commands are run as root, but nothing else is. 

But if the above command gets interrupted, you have to start over from the beginning.  The rsync command is one that can resume where it left off.   Assume I'm logged in as root, and that root has SSH key access to the root account on the remote system. 

    rsync -av /source/dir/ remotehost:/remote/dir/

(Be sure you know what you're doing before you use rsync.  The presense of the final "/" on "/source/dir/" is important.)

HOWEVER.  This past winter I switched all my home computers from Fedora to Ubuntu, after feeling out Ubuntu on my laptop for a couple of years.  I like Ubuntu for a lot of reasons, and one is how it treats the root account.  Unless you take steps to make it otherwise, the root account of an Ubuntu system doesn't have a password.  Perhaps more accurately, the ability to log in (remotely or locally) as the root user is disabled.  No amount of brute forcing will guess the password.  Similarly, OpenSSH server is configured to disable root access.  The way to do any administrative task is to log in (remotely or locally) as a regular user, then use sudo (http://www.sudo.ws/sudo) to perform the task.  By discouraging root logins, Ubuntu is encouraging the good habit of only running with administrative privileges when needed (rather than, say, logging in to the Desktop as root and doing everything with privileges).  Of course, I still do "sudo su -" on a regular basis...

ANYWAYS.  One drawback to the above-described configuration is that you can't run rsync as the root user, because as long as the remote system is running unmodified Ubuntu, an rsync process started by root can't log in to the remote host to complete the copy.   The rsync process needs to be root in order to preserve all metatadata and read all data, but I don't want to modify the Ubuntu system to undo the restrictions on root logins.

So, I finally figured out how to run the data & metadata copy part of the rsync as root while running the network transport as a regular user.  BEHOLD.

BEGIN sudo-rsync


if [ -z "$USER" ]; then
if [ -z "$USER" ]; then

if [ -z "$HOME" ]; then
        HOME=`awk "BEGIN { FS=\":\" }/$USER/ { print \$6 }"`

sudo rsync -avP -e "ssh -l $USER -i $HOME/.ssh/id_rsa " --rsync-path="sudo rsync" $*

END sudo-rsync

The stuff at the beginning with HOME and USER is some paranoia that probably isn't necessary on modern Linux distributions.  But I never trust environmental variables to be set correctly, and I don't know why there exists a LOGNAME, USERNAME, and USER variable all set to my logged in user's name.  Better safe than sorry.  The awk thing on HOME is just me showing off. 

The real important part is just the last line.  The entire thing gets run locally as root thanks to the initial "sudo".  "-avP" are my standard rsync args for getting all the metadata copied correctly and providing status of the copy.  "-e ssh..." tells rsync not to run regular ssh client (which would try to use root's SSH key material and try to connect to the remote host as root) but instead connect as the unprivileged $USER using that user's key material as found in $HOME/.ssh.  This makes Ubuntu happy.  The "--rsync-path-..." says that, once it logs in to the remote side, instead of running rsync, it runs rsync via sudo, effectively as root.  This makes the metadata happy. 

The "$*" adds all the command line arguments that you gave the script to the rsync command.  So you run sudo-rsync just like you run rsync.  (I suppose I should remove the "-avP" from the script and let the user decide if & when they want those features turned on.)

There.  The data/metadata copy is run as root on both ends while the network transport runs as the regular user.  OK, strictly speaking the ssh process on the local side is run as root, but it's connecting to the remote machine as the regular user, using the regular user's key material.