Zero-touch change management: striking the right balance between speed and security

Being able to make changes to security policies quickly, easily and efficiently is essential for an agile organisation.  Automation plays an important part in this.  But it is important to understand how to strike the right balance between speed and security — to avoid getting bogged down with cumbersome manual processes, while ensuring that human intervention takes place when needed.

To achieve this balance, let’s begin by considering what a typical firewall change workflow looks like:

•  A change workflow usually starts with a request asking for traffic to be allowed from a source to a destination, using a particular protocol. For example, ‘we want to traffic to be allowed from 1.1.1.1 to 2.2.2.2 using HTTPS’.
•  Next, the change request has to be planned. That is, the firewalls that will need to be modified to allow this new traffic path must be identified, and the proposed modifications must be checked to ensure that they will not violate any security or compliance guidelines.
• The change request is then translated into a technical work order. Typically this is a table, with all fields populated in the language of the specific device or devices used to carry out the change request.
• Finally, the work order has to be implemented across the relevant firewalls, and then verified to ensure that it’s been implemented correctly and that everything is working as expected.

Automation vs. control

There are solutions available on the market that semi-automate this workflow:  once the request is made, the planning stage can be automated using routing analysis to find the relevant firewalls.  The risks and compliance checks and the creation of the work order can also be semi-automated. The implementation stage can be either manual, requiring the administrator to point-and-click their way through the firewall vendor’s user interface, or semi-automated using the vendor’s API calls triggered by the operator clicking on a button.

The advantage of the semi-automated process is that the information security and network architecture teams retain a crucial level of control.  If something goes wrong during one of the workflow stages – perhaps a high-severity risk is identified, or some information is incorrectly translated from one stage to the next – then the human operator can halt the process, investigate, and either rectify the problem or manually process an exception.

An alternative is ‘zero-touch’ orchestration solutions which can accelerate change requests, going from making request to implementation in a matter of minutes. However, such orchestration usually bypasses the change process: the input information is assumed to be perfectly valid, and it is injected as-is into the security devices. While this speed is extremely beneficial for organizations that need to be as agile as possible, or who use DevOps methodologies, it also comes at a price.

The price is that there is little control over the process.  If honoring the request as-is will trigger any risks or compliance violations, if there is missing information, or if the firewalls are not identified accurately – then zero-touch orchestration could potentially introduce a misconfiguration into the security devices. This could create a significant security risk, or break critical traffic flows.  Either way, it’s a potentially huge operational problem.

Striking the right balance

How, then, do we strike a balance between these two approaches? Can we have our cake and eat it?  The best approach is taking the speed and efficiency of zero-touch orchestration and embedding conditional logic in it, to catch any problems before they have a major impact on security compliance and application availability.

Essentially, this approach involves taking each step in the change workflow – each of which is automated in a zero-touch manner – and then wrapping them individually with conditionals.  For example, once a change request is defined and the solution identifies all the relevant firewalls that need to be changed, an automated processes checks that those are reasonable firewalls for the change request.  If the firewalls have been correctly identified, then the automated process continues onto the next step with no need for intervention. If however, a problem is discovered during the check – perhaps an unusual number of firewalls has been identified, or none at all — then the process is stopped and the system raises a flag for human intervention. Likewise, the automated process activates a risk check – but moves on to the next step only if risks are not detected; if risks are detected, the automated process is stopped.

As a result, normal change process requests can zip through from request to implementation in minutes, with no demands placed on IT personnel to double-check change request steps that are already confirmed to be correct.   With effective security policy management, this should happen in the vast majority of cases.   However, if for some reason a problem is flagged, then manual intervention can take place at the right point in the change process. And in either case, the process is supported by a complete audit trail and history of each stage, whether automatic or manually activated.

Crucially, this automation also ensures that no steps in the process are missed out.  Manual overrides need only take place when they are actually needed to correct a problem.  The net result is a more efficient IT team and a more efficient and secure firewall change processes – giving organizations the agility and speed of zero-touch, combined with zero loss of control.

See my new whiteboard video to hear me explain this issue in person: