Backup2l not enough space

Support
1

Problem Description

My Sympl backup2l cron job is now emailing me to say that there isn’t enough space for a backup:

=> Checking there is enough free space for a backup...

*  Free space on /var/backups/localhost:     15.6 GB
*  Estimated backup size:                    15.6 GB

!! There is not enough space to take a backup.  Aborting!

However, using backup2l --estimate /etc/sympl/backup.d/backup2l.conf gives:

Estimating differential level-2 backup <all.244> based on <all.243>...
  6655 / 200548 file(s), 382 / 21896 dir(s), 333.6MB / 15.6GB (uncompressed)
  skipping: 177 file(s), 3 dir(s), 3.8MB (uncompressed)

That looks to me as though it should only be backing up 333.6 MB, not 15.6 GB.

The regex /\/ (?<int>[0-9]*\.?[0-9]+)(?<suff>[ KMGT]B) \(uncompressed\)$/m in /etc/sympl/backup.d/pre-backup.d//05-check-availspace is selecting the second figure from the output of the estimation command; is that correct?

Environment

  • Sympl Version: 11
  • Debian Version: 11
  • Hardware Type: VM
  • Hosted With: Digital Ocean
2

I’ve found I need significantly more available disk space for the backup than is actually needed in terms of the diff backup as though it’s overestimating the amount required.

That wasn’t too bad in the Bytemark BigV days when the backup storage was on a separate cheaper volume so the size could be bigger without much additional cost.

3

I raised this question with MB Support a while back:

I have a Synology NAS backing up to archive grade storage discs at
Bytemark which I need to migrate. The data’s about 500Gb before
weeding. I guess the best option would be to setup a low spec Mythic vm
with plenty of backup space and thereafter mount the backup space. I’d
value your advice?

Response from MB Support:

Another option would be to order a low spec HDD-based virtual server, and just backup straight to that. The cost per GB is cheaper than backup space (2p/GBmo vs 8p/GBmo) but this reflects the fact that the backup space is mirrored to two different sites, and we take ZFS snapshots of the data.

4

I replaced the regex on line 51 of /etc/sympl/backup.d/pre-backup.d/05-check-availspace with:

, (?<int>[0-9]*\.[0-9]+)(?<suff>[ KMGT]B) \/ [0-9]*\.?[0-9]+[ KMGT]B \(uncompressed\)$

… which selects the first number not the second.

The backup now runs fine, and the compressed size is only 79 MB, so I think this might be a bug in the check script. @Kelduum does that make sense?

5

Yes, I’ll take a look at this see if I can replicate the issue, then push a fix.

Thanks for reporting it!

1 Like
6

The backup2l estimate could be considerably less than the space required to perform the backup (at least, last time I looked at compression under windows). Either way, although there’s virtually no risk with the stated numbers, this change takes us closer to the edge … and, at some point, perhaps over.

Arguably, the core problem is running out of disk space, not being unable to take backups, so the current ‘bug’ is a useful early warning. I’d like to see a persistent service-monitoring style alert, if it doesn’t already exist, before taking backups to the edge.

Btw, while we’re in Space, back in the day, systemd logs went to several GB on my machine…

journalctl --disk-usage

I’m not sure it’s that useful on a production system, so set SystemMaxUse=500M in /etc/systemd/journald.conf. Looking now, I lost that change two dist upgrades ago :wink: … it’s now set for 1G.

Also, almost on-topic, editing the backup2l SKIPCOND might help:

/etc/sympl/backup.d/conf.d/10-directories.conf

I’m currently running with;

SKIPCOND=(-name "*.o" -o
          -path "*.nobackup*" -o
           '(' -path "/srv/*/php_sessions" -o
               -path "/srv/*/php_tmp" -o
               -path "/srv/*/private/cmsms/*/tmp" -o
               -path "/srv/*/private/cmsms/tmp" -o
               -path "/srv/*/public/htdocs/tmp" -o
               -path "/srv/*/public/htdocs/uploads/_CGSmartImage" -o
               -path "/srv/*/public/htdocs/uploads/_SmartImage" -o
               -path "/var/lib/mysql" -o
               -path "/var/lib/php"
            ')' -prune
         )

(-prune excludes the full directory tree).

I bet there’s more that could be usefully excluded.