I am trying to use RSync to speed up a nightly get of around 3 gigs of data from the east coast to the west over a VPN. I have around 10Mbits of bandwidth between the two locations at a latency of around 110ms to the server. The problem I have is that I can't get the transfer rate to exceed 70kByte/s. If I do a raw Windows copy, I can get 220kB/s, so this seems really low. I have tried turning on compression, running a TCP/IP optimizer on the server (the client is Vista, which claims not to need this), switching to DeltaCopy, and removing the network share (it was copying from a network drive at the server location, now it's direct local drive). None of these had any impact on transfer speed.
I am seeing reduced overall bandwidth usage (rsync cuts 50% of the data transfer), but it's 3x slower, so it's not a net gain. Both machines are very fast (quad core, 10k drives). Both cpus are idling at <1% used by rsync. It seems like it has to be a network issue, but I can't find anything in Rsync to tweak that.
Thoughts? Did I miss something in trying to get this transfer to go faster?
How does your rsync command look like ?
rsync --compress-level=9 --recursive --times --archive --human-readable --verbose --progress --delete rsync://SERVERNAME/FOLDER/ /cygdrive/D/LocalFolder/LocalFolder
Another interesting tidbit... the rate of transfer isn't really affected by other transfers running on the same connection. Specifically, if I run rsync by itself, then I try it again while doing a file copy from another server, it gets the same transfer rate (~70kB/s). Weird.
Just on a hunch, I tried syncing 2 separate files at the same time from the 1 rsync server. They both get ~70kB/s. wtfbbq. Either rsync has a built in transfer cap of some kind, or I am getting traffic shaped somewhere.
AFAIK, there is no built-in performance cap in rsync. I have now tested it between two windows machines (client to daemon) via a 2 Mbps link and I can easily achieve 150 KB or more.
It may be cygwin and its interaction with TCP/IP... I can't reproduce this over a lan, just my highish latency VPN link, so I can't say much for certain, but the "80kBps" cap in these articles is suspiciously close to what I see if you add in some protocol overhead:
http://www.mail-archive.com/cygwin-patches@cygwin.com/msg03644.html
http://support.microsoft.com/kb/329781
I'm gonna try doing the sync to a public rsync server to see if it's the VPN or just the link speed.