A couple of days ago I posted on how to set up a subversion server with ssh access. In that post, in the section about creating your private ssh key I said that if you already had an id_rsa or id_dsa file in your ~/.ssh directory not to go any further because the generation of a new private key file could over write it. This post explains how to cope with multiple private keys.

ssh uses ~/.ssh/identity as the default filename for the private key when using version 1 of the ssh protocol and it uses ~/.ssh/id_rsa and ~/.ssh/id_dsa as the default filenames for the RSA and DSA private keys when using version 2 of the ssh protocol. However there are several ways of changing this behaviour.

Command line

ssh allows you to provide a different identity/id file using the -i option on the ssh command line as follows:

$ ssh -i /path/to/id_rsa user@some.domain.org

Where id_rsa can be substituted with id_dsa or identity.

This is the simplest method. And when used in conjunction with a subversion client you can create a [tunnel] in the subversion configuration file like so:

[general]
ssh = ssh -i /path/to/identity_file

If you want to support different identity/id files create a new [tunnel] ‘alias’ for each identity file. You can also use the -p option if you want to specify a different port at the same time.

Config file – first method

Another method is to use the ssh config file. The global file is found in /etc/ssh/config however you can also provide config files on a per user basic in ~/.ssh. In the config file you can restrict declarations using the Host declaration. Per the man page:

Host
   Restricts the following declarations (up to the next Host
   keyword) to be only for those hosts that match one of the
   patterns given after the keyword.  If more than one pattern
   is provided, they should be separated by whitespace.  A
   single `*' as a pattern can be used to provide global defaults
   for all hosts.  The host is the hostname argument given on
   the command line (i.e. the name is not converted to a
   canonicalized host name before matching).

The declaration IdentityFile allows you to specify the private key file that ssh should use. By using Host and IdentityFile you can let ssh pick the identity/id file itself based upon the host name. The following example will pick ~/.ssh/id_dsa for any host other than my.specific.host.org and my.other.host.com. In those cases the specified private key files are used.

# General settings
IdentityFile ~/.ssh/id_rsa
IdentifyFile ~/.ssh/id_dsa
IdentityFile ~/.ssh/identity

Host my.specific.host.org
IdentityFile ~/.ssh/my_specific_key_file

Host my.other.host.com
IdentityFile ~/.ssh/my_other_key_file

Note how the config file allows you to specify more than one file at once. When you do ssh will try each of the files in turn or as appropriate.

Config file – second method

The problem with the above method is that you have to keep adding new entries to the config file for new hosts. IdentityFile is actually very powerful and allows you to have wild cards in the path specification. The full man page definition says:

IdentityFile
   Specifies a file from which the user's RSA or DSA
   authentication identity is read.  The default is ~/.ssh/identity
   for protocol version 1, and ~/.ssh/id_rsa and ~/.ssh/id_dsa
   for protocol version 2. Additionally, any identities
   represented by the authentication agent will be used for
   authentication.

   The file name may use the tilde syntax to refer to a user's
   home directory or one of the following escape characters:
   `%d' (local user's home directory), `%u' (local user name),
    `%l' (local host name), `%h' (remote host name) or `%r'
   (remote user name).

   It is possible to have multiple identity files specified in
   configuration files; all these identities will be tried in
   sequence.

So I have set up my config file as below. It basically tries ~/.ssh/idents/hostname/username/id_* first, then ~/.ssh/idents/hostname/id_*, then the defaults before giving up.

# SSH config file

IdentityFile ~/.ssh/idents/%h/%u/id_rsa
IdentityFile ~/.ssh/idents/%h/%u/id_dsa
IdentityFile ~/.ssh/idents/%h/id_rsa
IdentityFile ~/.ssh/idents/%h/id_dsa
IdentityFile ~/.ssh/id_rsa
IdentityFile ~/.ssh/id_dsa

This allows me to support a new server/key file combination just by putting the key file in the appropriate place in the ~/.ssh without having to edit config files.

As someone who is a strong advocate of source control at work I’m actually quite bad at following what I preach. So this lunch time I decided to finally get around to setting up a Subversion server on my Linux server.

Installation

I am currently running Ubuntu 9.10 so installed subversion as follows:

$ sudo apt-get install subversion

If you don’t have ssh installed (why not?) install it similarly.

Users and groups

Once you have subversion and ssh installed you can set up the SVN server. Firstly you need to create a suitable users group, an admin user and a repository.

Create the group like so:

$ sudo groupadd -g 127 svnusers

The group id (127 above) needs to be unique so unless you know a free group id value check to see the last id used in the groups file as follows:

$ tail /etc/group

Then create an SVN admin user using useradd as below:

$ sudo useradd -c "SVN Admin" -u 2000 -g 127 -d /home/svnadmin -m -s /bin/bash svnadmin

Again the user id (2000 above) needs to be unique. This time look at /etc/passwd to find out the last id used. The group id (127 above) should be the same value as the group you created. The rest of the options tell useradd to create a home directory and to set the default shell to be bash.

Set the password for the SVN admin user like so:

$ sudo passwd svnadmin

Finally add any other users to the svnusers group by editing /etc/group and adding the additional users in a comma separated list on the end. For example:

svnusers:x:127:svnadmin,melanie

Server setup

Now its time to create a repository. Firstly create the directory where all the repositories will be stored. I put mine in /var since my /var is on separate partition (to protect from disk full issues). However you can put them where you want. So create the directory and set the permissions as below:

$ mkdir -p /path/to/subversion/repositories
$ chown -R root.svnusers /path/to/subversion/repositories
$ chmod -R 770 /path/to/subversion/repositories

When using ssh, svnserve is used as the SVN server. svnserve requires the full path to repository to be provided by the client to access the repository. You may not want to expose your file system that much so you can write a small script to modify the behaviour. Use which to find out where svnserve is and rename it.

$ which svnserve
/usr/bin/svnserve
$ sudo mv /usr/bin/svnserve /usr/bin/svnserve.bin

Then using your favourite editor create the script below and save it under the old svnserve name.

#!/bin/bash
exec /usr/bin/svnserve.bin -r /path/to/subversion/repositories "$@"

Finally set the execute permissions.

$ sudo chmod 755 /usr/bin/svnserve

Creating a project

Now its time to create a project. Change to the svnadmin user and enter the password.

$ su - svnadmin
password:

Now create a project and turn off other access so that only members of the svnusers group and the admin user can access it.

$ svnadmin create /path/to/subversion/repositories/my_first_project
$ chmod -R o-rwx /path/to/subversion/repositories/my_first_project

Finally you need to configure the project’s access permissions by editing

/path/to/subversion/repositories/my_first_project/conf/svnserve.conf

In the [general] section uncomment or add:

anon-access = none
auth-access = write

Now save the file.

Testing

Now you should be able to access the svn repository from anywhere provided the user is in the svnusers group. Test as follows from a client.

$ svn list svn+ssh://user@some.domain.org/my_first_project

You will be asked for the ssh password for the machine running subversion. And that’s it. You should be able to use your project. You can add new projects using the svnadmin user.

Further tweaks

There a couple of additional things you may want to do. Firstly my subversion server is behind a firewall. The machine it is on is accessed using ssh on a different port to the default. Secondly keep being asked for the ssh password can become quite a pain. Using public key authentication does make things somewhat easier.

Using an alternative ssh port

Annoyingly SVN does not understand the ‘username@domain:port’ syntax. It doesn’t know what to do with the port. However you can use svn’s configuration files to create a custom ‘protocol’. This allows you to add support for an alternative port through the configuration file.

You client user should have a file

~/.subversion/config

If not, create it. In it there should be a [tunnels] section. To that you can add a new ‘protocol’. I created sshhome as follows:

[general]
sshhome = ssh -q -p 1234

The -q option prevents the “Killed by signal 15″ message that you see with some versions of subversion. You can then use sshhome instead of ssh as the protocol and it will use the specified port. E.g.

$ svn list svn+sshhome://user@some.domain.org/my_first_project

Public key authentication

Finally to save having to enter your password all the time you can create a public key and install it on the server. On the client change to the ~/.ssh directory. Now if there’s already a file in it called id_dsa or id_rsa STOP. I’m not sure how to support multiple keys yet and create a new key will over write these files. If all is well create a set of keys as below. Do not use an empty passphrase. It’s also good security not to use your password as the passphrase.

$ ssh-keygen -t dsa -C "username@some.domain.org"
Generating public/private dsa key pair.
Enter file in which to save the key (/Users/username/.ssh/id_dsa):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /Users/username/.ssh/id_dsa.
Your public key has been saved in /Users/username/.ssh/id_dsa.pub.
The key fingerprint is:

Now you need to put the public key on to the server. Copy the file in to your home directory on the SVN server. You can use scp for that.

$ scp id_dsa.pub username@some.domain.org:/path/to/home/directory/.ssh

Then log in to the SVN server and change to the ~/.ssh directory. In that directory add the public key to the file of authorized keys and set the permissions. Then delete the public key file. See below:

$ cat id_dsa.pub  >> ~/.ssh/authorized_keys
$ chmod 600 ~/.ssh/authorized_keys
$ rm id_dsa.pub

And that should be it. Try listing your project directory again. This time for the first time it will ask you for your password but then it will only ask for the password infrequently.

Once upon a time there was the Bourne command shell. sh as it is known is core at the heart of any UNIXy type operating system and others too. However for some it didn’t do enough so the Bourne Again shell, bash, was written and nearly every system ships with that. These shells are not the same. The have similar syntax but do not behave similarly.

There are other shells which have a completely different syntax and other scripting languages such as Perl and PHP so when writing a shell script you can indicate the correct interpreter using the ‘hash bang bin bash’ comment to indicate you are using bash. I.e.

#!/bin/bash

Some lazy programmers don’t indicate the exact shell they require and use

#!/bin/sh

to mean bash when it means sh.

Ubuntu, in their wisdom, have decided to go for dash as their shell. It is an evolution of the Almquist shell and is smaller and lighter weight than bash however they install bash anyway. dash and bash have the common subset of sh commands but are different. So if you find you have a problem building or similar on Ubuntu it may be that the wrong shell is being used. Ubuntu has a symlink as follows:

lrwxrwxrwx 1 root root       4 Mar 29  2009 sh -> dash

You may need to change this to point to dash as follows:

$ cd /bin
$ sudo rm sh
$ sudo ln -s bash sh

While we’re handling Ubuntu oddities building against NCurses, the terminal output library, can also be problematical. A little while ago NCurses was split in to two libraries, libncurses.so and libtinfo.so. Some badly written software will try and link explicitly with libtinfo.so directly rather than trying libncurses.so first. Ubuntu does not split these two libraries but provides just the one libncurses.so library so if you have an issue with a missing libtinfo.so you will need to add symlinks in to your library directory as follows:

$ cd /usr/lib
$ ln -s libncurses.so.5 libtinfo.so.5
$ ln -s libtinfo.so.5 libtinfo.so

I have just taken over maintenance of the PECL Direct IO extension as it has been unmaintained for a while. Naturally the first thing I did before writing any new code was to check out the extension from SVN and try and build it. Building the source was fine but when I tried make test it failed producing output like the following:

PHP Warning:  PHP Startup: Unable to load dynamic library 'modules/gd.so' -
modules/gd.so: cannot open shared object file: No such file or
directory in Unknown on line 0

Warning: PHP Startup: Unable to load dynamic library 'modules/gd.so' -
modules/gd.so: cannot open shared object file: No such file or
directory in Unknown on line 0

The default PHP package on Ubuntu 9.10 is fairly minimalist. Most extensions are provided as separate packages which include an INI file and a dynamic library. The dynamic library is stored in:

/usr/lib/php5/20060613+lfs

And the INI file for each extension is in:

/etc/php5/conf.d.

The make tests rule in the generated Makefile in the extension you are compiling modifies the path that PHP searches for extensions by to point to the modules directory within the extension source. So PHP, which is used to run the tests, cannot find the extensions.

I’ve found the easiest way (if slightly hacky) to fix the make test failure is just to copy the .so files from the PHP5 extension directory in to the modules directory within the extension source.

I can do almost everything I need to do in OS X but just occasionally I need Linux. So I recently installed a VirtualBox VM running Ubuntu 9.10. I use Eclipse as my favoured IDE since I can run it on all three of the major OSes and it has good support for C, C++ and Java which is what I use mostly for work. So the next thing for me was to install Eclipse. This is where things became interesting.
(more…)