Cygwin Subversion Redux

One fun part about running your own website is that you get to see, among other things, what search queries lead users to your site.  We were expecting most of what we saw in the logs, searches like "Power of Two Games", "Power of 2 Games", and the like.  What did surprise us, however, was that some queries had to do with getting Subversion up and running!  After posting our article on housing source code, I also got a few emails asking for help setting up Subversion under Cygwin using SSH tunneling, and they came with the suggestion that I detail our experiences in a separate article.  So, here we go.  This article is about the step-by-step process we underwent to get our Subversion server up and running on Windows XP with SSH tunneling via Cygwin.

Disclaimer:  This set of instructions is coming largely from memory.  I am not running through a full setup and documenting as I go.  Since we set this machine up, we've moved our Subversion repository to a P3 1ghz running Debian GNU/Linux.  We didn't do this in any sort of reaction to problems with our Cygwin/Svn/SSH setup, it was simply that having our automated builder running on the same computer as our file server and svn repo slowed everythig down too much.  Also, thanks for the "new" machine, Joel!

Step 1: Set up Cygwin and the OpenSSH daemon on the server

We installed cygwin on our new server, a WinXP SP2 Dell Athlon Duo named "lechimp".  We installed to the default directory of "C:\cygwin".  Make sure that "OpenSSH" and "cygrunsrv" are selected from the package list during installation!

Edit the sshd config file at

c:\cygwin\etc\sshd_config

and add the line

UseDNS no

This isn't so important for working from the local subnet, but if you use dyndns to hit your repository from the internet cloud, not setting this option can add an extra 15-20 seconds of hang-time to every ssh connection you make.  By default, the OpenSSH daemon is configured to do a reverse name lookup on whoever is connecting.  This... can take a while.  Set up sshd to run as a service by running

cygrunsrv -S sshd

from the cygwin shell. Sshd on cygwin actually respects the WinXP user list, so we made a new user called 'svn'.  We only used this user for logging into lechimp via ssh.  The 'svn' user got his own home directory inside the cygwin folder (i think C:\cygwin\home\svn) the first time we ran the cygwin shell when logged in as that user.

At this point, we confirmed that we could actually ssh into lechimp as the 'svn' user.  There wasn't much to do there, just an empty '/home/svn' directory, but we knew sshd was running as a service and that we could get in.

Step 2:  Set up Subversion on the server

We installed the win32 binary version of subversion 1.4.3 on lechimp.

We had our svn repo somewhere else at the time, but we dumped it to a file using svnadmin dump.
We copied it over (pow2svn.dump) to lechimp and dropped it in c:\cygwin\home\svn.
We then opened up cygwin and (as the svn user) created the repo with svnadmin load.

Alternatively, you'd create your repo using svnadmin create if you were making a brand new repository.

Make sure that you can get to your new repository locally with "svn status file://path/to/your/repo".  Note that you don't need a drive letter, it will default to whatever the current drive is.  We like using the command-line tools here, they're simpler than TortoiseSVN for testing purposes.

Step 3:  Glue them together

Now the repo exists at c:\cygwin\home\svn\pow2 and we confirmed it by hitting it from lechimp itself via the WinXP command shell (not bothering with cygwin yet) with

svn status file://cygwin/home/svn/pow2

So the repo exists and works locally, it's time to hook it up to ssh. 
The sshd daemon is already running, and we can ssh into lechimp and login as the 'svn' user.  We can cd around the repository manually.

We made the /home/svn/.ssh/authorized_keys file. It's got 2 entries: one for myself and one for Noel. 
These are the magic lines, straight from the svnserve manual - the file looks like this:

command="svnserve -t --tunnel-user=chas", \
    no-port-forwarding, no-agent-forwarding,no-X11-forwarding,no-pty \
    rsa CHAS-PUBLIC-KEY charles@powerof2games.com

command="svnserve -t --tunnel-user=noel", \
    no-port-forwarding, no-agent-forwarding,no-X11-forwarding,no-pty \
    rsa NOEL-PUBLIC-KEY noel@powerof2games.com

You can use puttygen to make ssh key pairs, but we brought our pre-existing key pairs over from the original linux shared-host build server.  When we made them there, we used ssh-keygen to spit out two key files, rsakey and rsakey.private:

ssh-keygen -b 1024 -t rsa -f rsakey

The public key (rsakey) ends up in authorized_keys (the two-line file above) on lechimp at

c:\cygwin\home\svn\.ssh\authorized_keys

The private key (rsakey.private) end up on our WinXP workstations as .ppk files for putty and plink.  These .ppk files live in our keyrings in Pageant, which we have in our startup folders.  We converted these rsakey.private files from the ssh-keygen format to the putty .ppk format using puttygen.  When you set this up on your client, don't forget to tell subversion to use plink by editing your subversion configuration file at

C:\Documents and Settings\YOUR-USER-NAME\Application Data\Subversion\config

and change your tunnels section to use plink for ssh:

[tunnels]
ssh = c:/putty/plink.exe

(I don't remember if slash direction is important or not in this case, but it sure will be later!)  This will tell subversion to use plink when you access a repository using the svn+ssh protocol.

This one-user-per-key techninque works well for us because it means we don't need to add a new WinXP user for every new developer (not that we have any new developers, but we have a build server 'user' and our offsite backup 'user').  We also don't need to maintain any username/password files.  svnserve does no authentication when run in tunnel mode, so it'll take any user name you give it.  Adding new svn users is as easy as adding a new line to authorized_keys.

Now when we connect to lechimp via ssh, we authenticate using our keys.  sshd (on lechimp) does a lookup in the authorized_keys file, matches my key to the first line, and follows the 'command' it's given.  In this case it spawns svnserve for the duration of my connection with my name ("chas") as the subversion user.  If I ssh in using my key through puTTY, i see the following:

Authenticating with public key "chas@pow2" from agent 
Server refused to allocate pty 
( success ( 1 2 ( ANONYMOUS EXTERNAL ) ( edit-pipeline svndiff1 absent-entries ) ) )

Success!  The presence of our private keys in our keyring when we ssh into lechimp is now redirecting us straight to svnserve.  That's the last piece of the puzzle.  We can now issue svn commands to our repo, and everything Just Works.  It's all happening over a connection that's secure enough for internet usage, to boot!

It's important to note that in this setup, svnserve isn't running persistently as a daemon or a service.  The svnserve process is only alive for as long as my ssh session, which itself is only alive for as long as it takes for TortoiseSVN or the command-line tools to do whatever i happen to be doing at the moment.

Step 4: Finding your path

So now we have everything up and running on lechimp.  Ssh encrypts the connection and spawns svnserve, the users are controlled by key files we distribute, and things are good.  The only thing that's still pretty ugly is that our repository name is terribly long, absolute, and not fun to type:

svn status svn+ssh://svn@lechimp/cygwin/home/svn/pow2

Not so great.  We hunted around and eventually found the "-r" switch in svnserve, which lets you specify a "virtual root" that will prepend all repository paths.  We changed our authorized_keys file to look instead like this:

command="svnserve -r c:/cygwin/home -t --tunnel-user=chas" ...

Then the client-side svn line will shorten down to

svn+ssh://svn@lechimp/pow2/

which is much nicer.  Note that it's a windows style DRIVE:PATH format but linux-style slashes (/ instead of \).  IT DOES NOT WORK ANY OTHER WAY.  We tried.  A lot.  Go figure.

One last note:  You may be tempted to use -r to describe the full path to your repository so that your repository can be accessed with simply

svn+ssh://svn@lechimp/

This is very nice, but you may have a hard time issuing svn commands on that.  For whatever reason, subversion doesn't understand that syntax, you have to issue a revision as well, like this:

svn co svn+ssh://svn@lechimp@HEAD

I don't know if it's a bug or what, but it's a bit frustrating that svn doesn't understand a nice terse repository name, it seems that you need to have at least one directory after the repository host's name.  It happens with and without the trailing '/'.  Oh well.  I guess you can't have everything. 

Postscript:

As I mentioned in the beginning of the article, we're not using WinXP/cygwin/sshd/subversion to host our code anymore.  We're still using svn+ssh and key-based authentication, but now our repository is housed on a Debian GNU/Linux pc named stan.  Poor lechimp was simply doing too much (serving files, running an svn repo, and running CruiseControl.NET which involves constantly building all of our game, engine, and tools).  Anyway, the authorized_keys file copied right over to stan and instantly we could ssh in just like on lechimp.  We used svnsync to copy the repository from lechimp to stan so the history was preserved (making sure to delete our revprop script after the sync was done!), and then we uninstalled cygwin and deleted the 'svn' user from lechimp.  We had to blow away our checked-out copies from our workstations and re-check-out from stan, but that was very easy.