The recent global cyber blackout, which occurred this Friday morning (19), affected the functioning of banks and airports around the world. According to investigations carried out so far, the problem was caused by a failure to update CrowdStrike's Falcon software. The episode brought to light a dilemma faced by many companies and users: how to deal with the need to keep systems up to date and, at the same time, avoid problems resulting from unsuccessful updates?

Cybersecurity experts agree that keeping software up to date is extremely important to fix vulnerabilities and prevent attacks. However, the CrowdStrike case shows that even companies specializing in security can face problems with update failures.

No software is immune to failures. Therefore, it is necessary to have a contingency and reversion plan in case something goes wrong. Check out the main solutions suggested by RNP's Cybersecurity Intelligence experts below:

Exhaustive testing of updates: According to RNP’s Security Engineering Manager, Humberto Forsan, one of the first steps to minimize errors is to perform exhaustive testing of updates. “A recommended approach for technical teams is to test updates in a controlled environment before applying them to production and to segregate software updates and security vaccine updates on separate dates. Creating a mirrored test environment allows you to verify the stability and compatibility of updates before implementing them on a large scale, minimizing the risk of negative impacts”, he says.

Disaster recovery plan and backups: Another important step is to have a disaster recovery plan and regular backups of your systems, because if an update causes problems, it is essential to be able to quickly return to a previous stable state. In addition, it is important to ensure that backups are stored in secure locations and are readily available when needed.

System redundancy: Having system redundancy can help mitigate the impact of occasional failures, avoiding prolonged service interruptions. By having redundant systems operating in parallel, it is possible to quickly switch to a secondary system if the primary system fails. Performing periodic failover tests, simulating failures in critical components, for example, helps identify weak points and improve the recovery capacity of organizations.

“Best practices regarding the secure development scenario lead us to version update tracks based on stable versions and test versions, where test builds can perhaps be triggered for customers with the appropriate alert layers. This provides for levels of validation and approval before sending to the customer both in stable and test versions. Regarding the receipt of updates, it is mandatory to segregate the homologation, test and production environments, thus limiting unavailability in the environments”, points out RNP's Security Operations Manager, Ivan Benevides.

Therefore, finding the balance between the need for updates and system stability requires a careful and planned approach. Thoroughly testing updates, having robust contingency plans, and adopting resilient architectures are all necessary measures to reap the benefits of security updates without compromising system availability and reliability in the process.