Again, quick, Flynn is a self hosted platform as a service which is easy to setup and operate.

It stores the files of applications to deploy (along all releases for rollbacks) within a Flynn internal application called blobstore. It is a data store for large files and the underlying implementation uses the Large Object support of PostgreSQL.

If you have an application of the size like 100MB, which is quite common with dependencies, this release history can grow quickly and your available hard drive space on your cluster servers can shrink in the same speed.

And your backups grows with the same speed as well. It takes longer to backup and longer to restore which increases your downtime in case.

With the release of the stable version v20160624.1, Flynn gained the ability to use Amazon S3 as storage system for the blobs solving the mentioned problems.

This posting shows you how to migrate a Flynn cluster to S3.

There are four steps for this migration.

Create a Bucket at S3

The first step is to setup an Amazon S3 bucket where all the blobs will land. Let’s call it “flynnblobstore” and place it in eu-central-1. It should get an IAM user with access key and secret access key having a policy like this:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "s3:DeleteObject",
                "s3:GetObject",
                "s3:ListBucket",
                "s3:PutObject",
                "s3:ListMultipartUploadParts",
                "s3:AbortMultipartUpload",
                "s3:ListBucketMultipartUploads"
            ],
            "Resource": [
                "arn:aws:s3:::flynnblobstore",
                "arn:aws:s3:::flynnblobstore/*"
            ]
        }
    ]
}

Setup two Environment Variables

Now, two environment variables have to be set for the Flynn application blobstore:

flynn -a blobstore env set BACKEND_S3MAIN="backend=s3 region=eu-central-1\
bucket=flynnblobstore access_key_id=<yourAccessKeyId>\
secret_access_key=<yourSecretAccessKey>"
flynn -a blobstore env set DEFAULT_BACKEND=s3main

Replace “<yourAccessKeyId>” and “<yourSecretAccessKey>” with your values of the just created IAM account.

And take care of whitespace at the environment variable “BACKEND_S3MAIN”, one space too much will break the next step.

Execute the Migration

Now it’s finally time to execute the migration. Go back to your command line and execute this:

flynn -a blobstore run -e /bin/flynn-blobstore-migrate -- -delete

You will get some output indicating the current moving file and the total amount. Depending on your amount of releases, this can take some time.

Vacuum the Database

Finally we will do some cleanup in the database and actually free up the disk space.

Connect with the PostgreSQL of the blobstore:

flynn -a blobstore pg psql

And execute the vacuum:

VACUUM FULL;

Again, this will take some time.

Results

In our case, the blobstore grew and grew and a backup .tar of the cluster was at 7.6 GB. Now, it is just a tiny 602.5 KB (Yes, KB!). The step “restoring Postgres” on restoring a backup took about 45 minutes before. Now it should be a matter of seconds.