bohnsack.com

It's kind of a running joke that practical fusion power is always 20-30 years away, but this recent advancement with "Linear Transformer Driver technology" on Sandia's Z Machine causes some to make the 20 year claim once again:

Sandia Press Release
Slashdot Story

Charlie's blogging over at pullmonkey.com.

Elisa has a new weblog at http://hur-hur.com/

I've been using Subversion (wikipedia page, reference) for much of my SCM needs at work and at home for quite a while now, but I was still using a few CVS repositories hosted at bohnsack.com that needed to be accessed from many locations (e.g., a repository for my grad school files).

This morning I decided to migrate these last few personal CVS repositories to SVN and while I was at it, create a new SVN repository on bohnsack.com for a collaborative effort I'm working on (more than one person will need to access it).

Here are some notes from the experience, split up into four major parts:

1. Installing SVN software on the server
2. Migrating existing personal CVS repositories to SVN
3. Creating a new SVN repository that's meant to be shared by more than one person
4. Setting up email commit notifications
5. Backing up the SVN repositories using svnsync

Step #1: Install the latest SVN software

First, I installed all the most recent SVN tools on bohnsack.com, which currently runs RHEL3. Checking out available packages from here and more specifically here, the installation process went something like this:

# get the files
downloadsite="http://summersoft.fay.ar.us/pub/subversion/latest/rhel-3/i386"
wget ${downloadsite}/subversion-1.4.3-1.i386.rpm
wget ${downloadsite}/cvs2svn-1.2.1-1.noarch.rpm
wget ${downloadsite}/httpd-2.0.46-61.1.ent.i386.rpm
wget ${downloadsite}/httpd-devel-2.0.46-61.1.ent.i386.rpm
wget ${downloadsite}/mod_dav_svn-1.4.3-1.i386.rpm
wget ${downloadsite}/mod_ssl-2.0.46-61.1.ent.i386.rpm
wget ${downloadsite}/neon-0.24.7-1.i386.rpm
wget ${downloadsite}/subversion-devel-1.4.3-1.i386.rpm
wget ${downloadsite}/subversion-perl-1.4.3-1.i386.rpm
wget ${downloadsite}/subversion-python-1.4.3-1.i386.rpm
wget ${downloadsite}/subversion-tools-1.4.3-1.i386.rpm
wget ${downloadsite}/neon-devel-0.24.7-1.i386.rpm
	
# install them
rpm -ivh httpd-2.0.46-61.1.ent.i386.rpm httpd-devel-2.0.46-61.1.ent.i386.rpm \
mod_ssl-2.0.46-61.1.ent.i386.rpm mod_dav_svn-1.4.3-1.i386.rpm \
subversion-1.4.3-1.i386.rpm subversion-devel-1.4.3-1.i386.rpm  \
neon-devel-0.24.7-1.i386.rpm neon-0.24.7-1.i386.rpm \
subversion-perl-1.4.3-1.i386.rpm subversion-python-1.4.3-1.i386.rpm \
cvs2svn-1.2.1-1.noarch.rpm

Although I needed the libraries provided by the installed httpd software, I didn't want it to actually run, so the newly installed webserver was shutdown and prevented from starting at boot time:

chkconfig httpd off
/etc/init.d/httpd stop

Step #2 CVS to SVN Migration

I had two personal CVS repositories to migrate. A step-by-step process for one of them is shown below:

Migrate the repository

# Create a directory for the SVN repository to live in
mkdir -p /home/user1/svn/repos/
	
# Specify local CVS/SVN repositories
cvsrepo=/home/user1/cvsroot/UNM-files/
svnrepo=/home/user1/svn/repos/UNM-files
	
# See if the migration tool works
cvs2svn --dry-run -s ${svnrepo} ${cvsrepo}
	
# Actually do the migration
cvs2svn -s ${svnrepo} ${cvsrepo}
.
.
Starting Subversion commit 1049 / 1050
Starting Subversion commit 1050 / 1050
Done.
	
cvs2svn Statistics:
------------------
Total CVS Files:              2415
Total CVS Revisions:          4129
Total Unique Tags:               1
Total Unique Branches:           1
CVS Repos Size in KB:       604725
Total SVN Commits:            1050
First Revision Date:    Tue Jan 25 21:17:41 2005
Last Revision Date:     Sat Apr  7 11:00:03 2007
------------------
Timings:
------------------
pass 1:    24 seconds
pass 2:     0 seconds
pass 3:     0 seconds
pass 4:     0 seconds
pass 5:     1 second
pass 6:     0 seconds
pass 7:     0 seconds
pass 8:   500 seconds
total:    528 seconds

Checkout working copy of new repository

cd /home/user1/rcs/
	
# if checking out files on a different host than the one that hosts the repo
svn co svn+ssh://user1@bohnsack.com:/home/user1/svn/repos/UNM-files
	
# if checking out files on the same host that hosts the repo
svn co file:///home/user1/svn/repos/UNM-files

For the remote checkout option to work, ssh to the remote host must be possible. For it to be convenient, you'll probably want to setup an ssh public key pair.

Take care of .cvsignore migration

With this reference:

# cd into checked SVN working copy directory
cd /home/user1/rcs/UNM-files
	
# translate .cvsignore to svn propset
find -name .cvsignore | while read file; do
    svn propset svn:ignore "`cat "$file"`" "`echo "$file" | sed 's,/[^/]*$,,'`"
done
	
# remove .cvsignore files
svn rm `find -name .cvsignore |xargs`
	
# commit
svn commit -m "migrate .cvsignore information to SVN"

Step #3: Setup Shared Repository

It's possible to setup a really slick shared SVN repository with Apache and WebDAV, but I choose to keep things really simple with some simple Linux system administration and the svn+ssh access method. The SVN book covers lots of different ways to do this kind of shared repository, but this is how I did it:

Create users, groups, and repository

projectname="foobar"
newuser="newuser"
	
# create new group and user
groupadd ${newuser}
useradd -g ${newuser} ${newuser}
	
# create project group and add me a new user to this group
groupadd ${projectname}
	
# use vi on /etc/group to add user1 (me) and ${newuser}
# to the ${projectname} group
	
# create repo directory with suitable name, ownership, and setgid bit
mkdir -p cd /usr/local/svn/repos/
chmod 777 /usr/local/svn/
chmod 777 /usr/local/svn/repos
svnadmin create --fs-type fsfs /usr/local/svn/repos/${projectname}
chown -R bohnsack:${projectname} /usr/local/${projectname}
cd /usr/local/svn/repos/
find ${projectname} -type f | xargs chmod g+rw
find ${projectname} -type d| xargs chmod g+rwx
find ${projectname} -type d| xargs chmod g+s
find ${projectname} | xargs chmod o-rwx

Set umask

I want newly created files to be readable and writable by the user and group but not anyone else. An easy was to accomplish this was adding the following to each user's ~/.bashrc:

umask 007

A much better solution would be for svnserve to set this SVN-specific umask at invocation time and leave the users' general umasks alone. This is undoubtedly possible.

Create some files to import

From a remote host:

projectname="foobar"
cd /home/user1/rcs
mkdir ${projectname}
cd ${projectname}
mkdir trunk
mkdir branches
mkdir tags
svn import -m "initial files" \
svn+ssh://user1@bohnsack.com/usr/local/svn/repos/${projectname}
cd ..
rm -rf ${projectname}
svn co svn+ssh://user1@bohnsack.com/usr/local/svn/repos/${projectname}

Step #4: Commit Notifications

For shared repositories, it's nice to know what the other people iwth write access are up to. One really good way to accomplish this is to install a post-commit hook to send out email with diffs. I'm using SVN::Notify (Author's home page, example output)

perl -MCPAN  -e 'install SVN::Notify'

Then, modify hooks/post-commit to be something like this for each repository you want email notifications for:

REPOS="$1"
REV="$2"
	
/usr/bin/svnnotify --repos-path "$REPOS" --revision "$REV" \
  --to             "user1@bohnsack.com, user2@otherplace.com" \
  --from           "user1@bohnsack.com"                 \
  --reply-to       "user1@bohnsack.com"                 \
  --subject-prefix "[SVN UNM-Files]"                       \
  --subject-cx                                             \
  --svnlook        /usr/bin/svnlook                        \
  --sendmail       /usr/sbin/sendmail                      \
  --handler        Alternative                             \
  --alternative    HTML::ColorDiff                         \
  --with-diff

Step #5: Backups

Wanting to take advantage of my at-home RAIDed ReadyNAS as one backup of these important files, I wrote the following scripts to sync these repositories from bohnsack.com to the NAS:

Initial Setup

#!/bin/bash -x
# init-remote-svn-sync-repos.sh
	
localdir=/readynas/data_files/svn/repos
personalrepos="UNM-files otherrepo"
sharedrepos="foobar"
	
for repo in $personalrepos $sharedrepos
do
  svnadmin create --fs-type fsfs ${localdir}/${repo}
  echo "#!/bin/sh" > ${localdir}/${repo}/hooks/pre-revprop-change
  chmod +x ${localdir}/${repo}/hooks/pre-revprop-change
done
	
for repo in $personalrepos
do
  svnsync init \
  file://${localdir}/${repo} \
  svn+ssh://user1@bohnsack.com/home/user1/svn/repos/${repo}
done
	
for repo in $sharedrepos
do
  svnsync init \
  file://${localdir}/${repo} \
  svn+ssh://user1@bohnsack.com/usr/local/svn/repos/${repo}
done

Subsequent Syncing

#!/bin/bash -x
# svn-sync.sh
	
localdir=/readynas/data_files/svn/repos
personalrepos="UNM-files other-repos"
sharedrepos="foobar"
	
for repo in $personalrepos $sharedrepos
do
  svnsync sync file://${localdir}/${repo}
done