Migrating to Bullseye

Feature Discussion
1

Well I wanted to move my Buster Sympl installation to Bullseye, because it’s there, and also because if you don’t keep moving on, you eventually lose contact with updated important libraries. Bullseye isn’t really a dramatic change from Buster, PHP moved to 7.4, Python moves to 3.9, neither of these need massive source code change.There’s not a lot of stuff on my machine, some email accounts and some websites, most using PHP. Then, of course, the sites are supported by DNS. I’ve moved all my domains onto Mythic Beasts because then they are all in one place.

So step one was to commission a new virtual server, set up Sympl Bullseye, then install my usual account and my working environment. The web form for a new VPS asks for an SSH key - and that should be the public rsa key that you will use when you ssh to the new system (I had to ask about this). Then it’s was a good plan ensure that it’s possible to use ssh and its derivatives in and out of the new machine as me, the sympl user and actually also as root. Copying as root is allowed by the default setup as long as you are coming into the machine using a key and not a password. So being old school, I set up keys and authorized_keys files and suchlike.

You get to the point of moving stuff and here, my old friend rsync does most of the work. It’s completely possible to use rsync to move the entire sub-tree in /srv and have it working on the new machine. I (sort of) didn’t expect this, but I thought I would try. I’d do something like: get into /srv on the new machine, create the example.com directory then back to the old machine and have a script like:

#!/bin/sh
# called with a domain name argument
cd /srv
if [ $(whoami) != 'root' ]; then
   echo Run as root
   exit 0
fi
TEST=""
SNAME=$1
rsync -alv$TEST --delete --exclude public/htdocs/stats \
-exclude public/logs/ --exclude config/stats --exclude config/webalizer.conf \
/srv/$SNAME/ target_machine.domain:/srv/$SNAME/

The real script doesn’t have backslashes. I decided not to move web stats and logs. If you set the TEST to ‘n’ - then you can dry-run the copy. If you’ve not used rsync before, beware that the / at the ends of the source and destination directory names are critical, they tell rsync to treat the string as a directory name. Care: it’s very easy to splat files where you really don’t want them.

So this as well as moving all the files, notice that it’s 1) copying all the current LetsEncrypt files so the certs go across intact and 2) copying any Dovecot mail files. I’ve been copying Dovecot maildir trees about for years so I knew that would be OK. I’d also moved LetEncrypt certificates to a different machine before, so I was reasonably certain that this would work too. Then on the new machine, you may want to update some things, perhaps setting config/dkim to use a new location qualifier. If you have mail coming in on your old machine, then you can use rsync again just to move the mail trees. The --delete flag will delete anything on the destination that doesn’t exist at the source.

To make the new site active, you can wait for cron to do the sympl magic for you. I am unsure about all the sympl things that need to be run but:

# as root
sympl-web-configure -v

installs things into /etc/apache2 so the websites run - and actually that’s all I did to get sites operational. I am unsure what inserts email passwords into the right place so dovecot can find them - but I had no problem with checking that email was working once I’d moved the DNS to point to the new machine.

The next step is to change the DNS for the site. As I said, my domains are all on the Mythic DNS but using a web interface is a pain if you want to make a lot of changes. However, Mythic Beasts have a nice little API for their DNS. You need to get onto your admin pages to set up an API key for their v2 API which gives you a KEY_ID and a SECRET. Then you can write a shell script to get a access token, like:

KEY_ID='YOUR_KEY'
SECRET='YOUR SECREY'
TOKEN=`curl -s -d 'grant_type=client_credentials' -u "$KEY_ID:$SECRET" https://auth.mythic-beasts.com/login | jq -r .access_token`
echo $TOKEN

I called this script access.

To get your current zone file, you create getzone:

#!/bin/sh
# Get a Zone file ignoring records we cannot change
# Arguments are list of domain names
TOKEN=$(access)
for domain in "$@"; do
    curl -s -n -H "Authorization: Bearer $TOKEN" -H "Accept: text/dns" "https://api.mythic-beasts.com/dns/v2/zones/$domain/records?exclude-template&exclude-generated" > $domain
done

which creates a file named for the domain with the bits of your zone file that you can change. Some of the full zone file is boiler plate for the Mythic DNS and not allowing to you change this makes sense.

Then you need putzone to upload changes:

#!/bin/sh
# Put the Zone file for the domain
# Arguments are list of domain names
TOKEN=$(access)
for domain in "$@"; do
    if [ ! -f $domain ]; then
	echo "File for $domain not found"
	continue
    fi
    cat $domain | curl -n -X PUT --data-binary @- -H "Authorization: Bearer $TOKEN" -H "Content-Type: text/dns" "https://api.mythic-beasts.com/dns/v2/zones/$domain/records?exclude-template&exclude-generated"
done

The putzone script changes the DNS atomically, and only takes a couple of minutes before things are live. In the help pages for the API you can find a script that tells you whether the new values have been fully installed, I made a script and rarely used it.

With these two tools and your favourite editor (sed is good here), changing masses of IP addresses takes seconds. I am a bit unsure about formats used for long DKIM strings, I managed to get an extra space in one, not sure how. But I reverted to the web interface to upload the whole string and that sorted it.

So, I did my move in about 3 hours one morning when I woke up early and was really surprised how easy it was. If you’ve managed to get here, thanks and good luck with your move. :grinning:

1 Like