Why You Should be Continuously Delivering Drupal Updates - All the Time

This is a special blog post written for Townsend Security by the Drupal Drop Guard team.

While developing a system to automate Drupal updates and using that technology to fulfill our Drupal support contracts, we ran into many issues and questions about the workflows that integrate the update process into our overall development and deployment cycles. In this blog post, we’ll outline the best practices for handling different update types with different deployment processes – as well as the results thereof.

The general deployment workflow
Most professional Drupal developers work in a dev-stage-live environment. Using feature branches has become a valuable best-practice for deploying new features and hotfixes separately from the other features developed in the dev branch. Feature branches foster continuous delivery, although it does require additional infrastructure to test feature branches in separate instances. Let us sum up the development activity of the different branches.

Dev
This is where the development of new features happens and where the development team commits their code (or in a derived feature branch). When using feature branches, the dev branch is considered stable; features can be deployed forward separately. Nevertheless, the dev branch is there to test the integration of your locally developed changes with the code contributions of other developers, even if the current code of the dev branch hasn’t passed quality assurance. Before going live, the dev branch will be merged into the stage branch to be ready for quality assurance.

Stage
The stage branch is where code that’s about to be released (merged to the master branch and deployed to the live site) is thoroughly tested; it’s where the quality assurance happens. If the stage branch is bug-free, it will be merged into the master branch, which is the code base for the live site. The stage branch is the branch where customer acceptance happens.

Master
The master branch contains the code base that serves the live site. No active changes happen here except hotfixes.

Hotfix branches
Hotfixes are changes applied to different environments without passing through the whole dev-stage-live development cycle. Hotfixes are handled in the same way as feature branches but with one difference: whereas feature branches start from the HEAD of the dev branch, a hotfix branch starts from the branch of the environment that requires the hotfix. In terms of security, a highly critical security update simply comes too late if it needs to go through the complete development cycle from dev to live. The same applies if there’s a bug on the live server that needs to be fixed immediately. Hotfix branches need to be merged back to the branches from which they were derived and all previous branches (e.g. if the hotfix branch was created from the master branch, it needs to be merged back to the master to bring all commits to the live site, and then it needs to be merged back to the stage and dev branch as well, so that all code changes are available for the development team)

Where to commit Drupal updates in the development workflow?
To answer this question we need to consider different types of updates. Security updates (including their criticality) and non-security updates (bug fixes and new features).

If we group them by priority we can derive the branches to which they need to be committed and also the duration of a deployment cycle. If you work in an continuous delivery environment, where you ship code continuously,the best way is to use feature branches derived from the dev branch.

Low (<=1 month):
- Bug fix updates - Feature updates

These updates should be committed by the development team and analysed for side effects. It’s still important to process these low-prio updates, as high-prio updates assume all previous code changes from earlier updates. You might miss some important quality assurance during high-prio updates to a module that hasn’t been updated for a long time.

Medium (<5 days):
- Security updates that are not critical and not highly critical

These updates should be applied in due time, as they’re related to the site's security. Since they’re not highly critical, we might decide to commit them on the stage branch and send a notification to the project lead, the quality assurance team or directly to you customer (depending on your SLA). Then, as soon as they’ve confirmed that the site works correctly, these updates will be merged to the master branch and back to stage and dev.

High (<4 hours):
- Critical and highly critical security updates

For critical and highly critical security updates we follow a "security first" strategy, ensuring that all critical security updates are applied immediately and as quickly as possible to keep the site secure. If there are bugs, we’ll fix them later! This strategy instructs us to apply updates directly to the master branch. Once the live site has been updated with the code from the master branch, we merge the updates back to the stage and dev branch. This is how we protected all our sites from Drupalgeddon in less than two hours!

Updates automation options
There are only a few ways to ensure the updates will be applied just in time and when it’s really needed, depending on the type of update. Any of those have positive and negative sides, and it’s only up to you to choose what suites you the best:

1. Monitoring for updates manually or via one of available services or custom scripts, and once the security update is detected, process it according to the workflow defined in your organization. This approach works in most cases, but it requires someone to be ready to take action 24/7;
2. Building a completely custom solution, which will not only detect updates, but also take care of applying them when it’s time. The only obvious drawback of this is that you have to spend a lot of time building and maintaining your custom tool.
3. Using the updates automation service, such as Drop Guard, which will integrate seamlessly in your workflow and process updates in exactly the way you want. You don’t have to worry about being alerted all the time, or spending too much time on building your own solution, but be prepared to spend a few dollars on the 3rd party solution.

Requirements for automation
If you want to automate your Drupal security updates with the Drop Guard service, all you need is the following:

• Code deployment with GIT
• Trigger the update of an instance by URL using e.g. Travis.ci, Jenkins CI, DeployHQ or other services to manage your deployment or alternatively execute SSH commands from the Drop Guard server.

Also to keep in mind:

• Know what patches you’ve applied and don't forget to re-apply them during the update process (Drop Guard helps with its automated patch detection feature)
• Automated tests reduce the time you spend on quality assurance

Conclusion
Where to commit an update depends on its priority and on the speed with which it needs to be deployed to the live site. Update continuously to ensure the ongoing quality and security of your project and to keep it future-proof. Feature and bug fix updates are less critical but also important to apply in due time.

There are many ways of ensuring the continuous security for your website, and it’s up to you whether to go with a completely manual process, try to automate some things, or opt-in for a fully automated solution such as Drop Guard.