Best of - Update marathon 2018
I've collected a bunch of articles for you, where Drupal agencies describe their processes, workflows and experience with the Drupal update release PSA-2018-001.
Hard facts: The update was announced one week earlier and released on March 28th between 18:00 and 19:30 UTC. Due to the flood of site views and very motivated F5 finger exercises, Drupal.org was down for around an hour. Fortunately, the Drupal Community worldwide was prepared with snacks, pizza, and more pizza, remote hangman, and a lot of memes.
For detailed information, the Drupal Security Team provided this FAQ about SA-CORE-2018-002.
But sometimes speed trumps process - ComputerMinds
First, I really enjoyed the article by ComputerMinds, their nice post image caught my eyes. Their set the goal to secure and patch all their sites within 30 minutes after the release was available. A clear plan, where sites are assigned to developers and a checklist for everyone (Mike Dixon, who wrote the article, indicates himself as old school, as his list is printed; honestly, I’d be that kind of developer).
The plan is to assess the severity of the issue and make a quick decision on the approach. If we are dealing with a DEFCON 1 issue then the plan will be to 'hack' the patch direct onto the webroot of the live sites, and then sort out the proper build process once the sites are secure. We normally have a clear pull request based build workflow (requiring approval) to prevent code going live that shouldn't
- but sometimes speed trumps process.
I’d be highly interested, if the ComputerMinds team could follow their plans and how much time they needed. Click here for Mike’s post.
Wunder -ful helping checklist
Peeter Pratka spent time to explain this update, why it’s special, what it means for thew Drupal users out there, and how he’d suggest to get prepared for its release.
A shortcut of the reasons why the PSA-2018-001 were expected as a special big deal:
It was the first major security update since 2014’s Drupalgeddon
One week’s notice for a highly critical security vulnerability fix
“...exploits might be developed within hours or days...”
The unusual support for older Drupal versions also highlighted the seriousness
His checklist to get prepared includes the following notes (click here to read Peeter’s full checklist with all the details):
Speak with your internal development team and prepare them (...)
If you’re working with a Drupal agency, ask them how they’re going to handle the update and ensure that they’re also prepared.
Keep your eye on the Drupal core updates page (...)
Backup your current website so that you have a clean copy of your website’s code and database available prior to the update release. (...)
If you struggle to allocate resources who can carry out the update within the recommended time frame, you might have to consider putting your site into maintenance mode (...). Unfortunately, this will mean that your web services will be unavailable to your users during this time.
Forewarned is Forearmed
If you want to know details about the background of Highly Critical Core Updates in Drupal, and another evaluation of the PSA-2018-001, you will definitely enjoy James Oakley’s post Drupal Security: Forewarned is Forearmed.
Beating the Black Hats in time - Amazee.io
Michael Schmidt wrote about their steps to mitigate the SA-CORE-2018-001 severity on an infrastructure level. At the end, he emphasizes: We believe in Open Source and the higher security this brings. We will continue to open source as much as we can! - no addition needed.
Here’s what Amazee.io did in short:
We adapted the request sanitizer from the Drupal code base so it can run fully independent of Drupal Code and created a separated PHP file from it (...)
We created a bash script (...) that checks each project inside OpenShift to see if it has an nginx pod with a PHP container, if it finds one:
It creates a new ConfigMap based on the request sanitizer PHP file in each project.
It adds this ConfigMap to each PHP container as a volume.
It tells PHP to auto_prepend this request sanitizer PHP file file via an environment variable
The bash script also checks if the request sanitizer file is correctly included and informs if not.
We then run the script against all projects in OpenShift which automatically get updated. A rolling deployment within OpenShift makes sure that there is no downtime of any sites. Plus as the PHP fix code is injected on an infrastructure level it persists even through future deployments.
All requests that trigger the mitigation code are logged by the internal ELK Stack, allowing us to monitor and report any exploitation attempts. This can all be done in real-time which allow us to immediately react to any potential issues that might develop after the patch has been reverse engineered.
Thank you guys for sharing this with the community! Read the detailed version here.
But still, please update - Platform.sh
Besides general information about the SA-CORE-2018-002, Platform.sh took two steps on response. Ori Pekelman wrote:
We've added a new rule to our Web Application Firewall (WAF) on all regions and on all Enterprise clusters that detects and blocks requests trying to exploit this latest attack vector, even if your site hasn't been updated. (But still, please update.)
We are adding a check to our protective block to prevent deployment of affected Drupal versions. If you try to push an insecure Drupal version our system will flag it for you and warn you that you are pushing known-insecure code. Please update your code base as soon as possible.
You can follow up the whole post here.
UEBERBIT referred that their team of seven patched approx 30 websites within 1,25 hours - nice work guys! Read the German announcement here.
In the last 20 years, Hagen Graf has seen many websites which were in a pitiable condition. Not maintained, not patched, not updated. And it affects ALL websites in ALL systems.
I want to conclude this overview article to mention the post of Hagen - it was appealing to read and made me laugh. Even if it is wrote in German, you will enjoy some nice Tweet & Meme quotes. It summarizes technical background information and the social media activities, enjoy best with a glass of wine. Thank you for your contribution!
Thank you for reading! I’m happy to hear your thoughts and please let me know who else should be referred in this article above.
Next, I will give you a detailed insight into Drop Guard’s performance, so stay tuned - but for now I wish you a well-deserved Easter weekend! ;-)