Announcing pgBackRest for Azure -  Fast, Reliable Postgres Backups

Backups are a key staple of running any database. Way back in the day, a good friend and colleague wrote one of the most used Postgres backup tools called wal-e. Wal-e was initially written in just a few days, and rolled out to the fleet of databases we managed in the early days at Heroku. We got pretty lucky with rolling that out, because shortly after we had there was the great AWS Apocalypse of 2011. This was a full day outage of AWS with lingering effects for nearly a week... Reddit was down, Netflix was down, so you couldn't even kill time waiting for things to come back up. At the time, AWS came back to us saying they couldn't recover a number of disks. Had it not been for wal-e and our disaster recovery setup customers would have lost data. Luckily no bytes of data were lost, and customers were back up and running much faster than had they been on RDS.

Fast forward nearly 10 years and now there are numerous backup options for Postgres. And while wal-e was a great tool for it's time and place, it hasn't materially changed in the last 10 years. Enter pgBackRest. pgBackRest aims to be a fast, reliable, easy-to-use backup and restore solution with the ability to seamlessly scale to the largest databases. A key tenant of pgBackRest is that it stores a backup manifest – allowing it to ensure whether a backup is good or not. If you’ve ever tried to restore from a bad backup you can appreciate this.

pgBackRest has been around for a number of years with support for AWS. Over the past year or so it's received a subtle update under the covers being re-written in C – thus allowing for even better performance which is a big win. And just recently received one more exciting update: support for Azure.

If you're brand new to pgBackRest let’s go ahead and set it up:

sudo yum -y install pgbackrest

Now we need to tell Postgres to leverage pgBackRest as our backup mechanism. We can do that by editing our postgresql.conf:

archive_command = 'pgbackrest --stanza=demo archive-push %p'

After setting this you'll want to reload your Postgres config.

Now we need to configure pgBackRest. You'll want to edit /etc/pgbackrest.conf. First we're going to give the path for the database that we're backing up. Then we're going to specify the details relevant for Azure.

[demo]
pg1-path=/var/lib/postgresql/12/demo

[global]
process-max=4
repo1-azure-account=your_azure_account
repo1-azure-container=your_azure_container
repo1-azure-key=your_azure_key
repo1-cipher-pass=your_pass
repo1-cipher-type=aes-256-cbc
repo1-path=/demo-repo
repo1-retention-diff=2
repo1-retention-full=2
repo1-type=azure
start-fast=y

Now we can fire off our first full backup:

sudo -u postgres pgbackrest --stanza=demo --type=full backup

And you can see info about the completed backup with:

sudo -u postgres pgbackrest info

And with just a few commands you have fast, reliable, and an easy to use backup and restore solution for running Postgres on Azure. If you give it a try or have any questions about it we’d love to hear from you @crunchydata