In today’s IT environment, the only constant is change. Not only is change rampant, but it often occurs at breakneck speed. Rapid business growth from mergers and acquisitions, development of new and de-commissioning of old applications, new users, micro-segmentation, cloud migrations and more make for a dynamic environment that poses new security challenges all the time.
In today’s IT environment, the only constant is change. Not only is change rampant, but it often occurs at breakneck speed. For a variety of reasons – rapid business growth from mergers and acquisitions, development of new applications, de-commissioning of old applications, new users, evolving networks and evolving cyberthreats – business needs change and, as they do, so must security policies.
But change comes with challenges, often leading to major headaches for IT operations and security teams. The headaches sometimes develop into huge business problems:
Some organizations have grown so wary of change control and its potential negative impact that they resort to network freezes during peak business times rather than attempt to implement an urgent change in their network security policies.
AlgoSec has another point of view. We want to help you embrace change through process improvement, identifying areas where automation and actionable intelligence can simultaneously enhance security and business agility – without the headaches.
Herein, you will learn the secrets of how to elevate your firewall change management from manual labor-intensive work to a fully automated change management process.
Placing a sticky note on your firewall administrator’s desk and expecting the change request to be performed pronto does not constitute a formal policy. Yet, shockingly, this is common practice. A formal change request process is in order. Such a process dictates clearly defined and documented steps for how a change request is to be handled, by whom, how it is addressed within a specified SLA, and more.
Popular IT ticketing systems, like ServiceNow and Remedy, are a good place to manage your firewall change requests. However, these system are built for tracking general requests and were never designed for handling complex requests such as opening the network flow from server A to server B or revising user groups.
Having a policy state “this is what we must do” is a start, but without a formal set of steps for carrying out and enforcing that policy, you still have a long way to go in terms of smoothing out your change processes. In fact, the majority of challenges for managing network security devices include:
Firewall change management requires detailed and concise steps that everyone understands and follows. Exceptions must be approved and documented, continuously improving the process over time.
Network security and operations staff work in separate silos. Their goals, and even their languages, are different. Working in silos is a clear recipe for trouble. It is a major contributor to out-of-band (unexpected) changes which are notorious for resulting in “out-of-service.” In many large companies, routine IT operational and administrative tasks may be handled by a team other than the one that handles security and risk-related tasks. Although both teams work toward the same goal – smooth operation of the digital side of the business – decisions and actions made by one team may lead to problems for the other. Sometimes, these situations are alleviated in a rush with the good intention of dealing with security issues “later.” But this crucial “later” never arrives and the network remains open to breaches.
In fact, according to a large-scale survey of our own customers, out-of-process firewall changes resulted in system outages for a majority of them. In addition, our customers pointed out that out-of-process changes have caused them exposure to data breaches and costly audit failures.
It’s imperative to know what the business is up against from the perspective of threats and vulnerabilities. What’s often overlooked, however, is the no-less-devastating impact of poorly managed firewall changes. Without carefully analyzing how even the most minor firewall changes are going to impact the network environment, businesses can suffer dramatic problems. Without thoughtful analysis, they might not know:
A lot of money and effort is put into keeping the bad guys out, while forgetting that “we have seen the enemy and he is us.”
Renowned security expert, Bruce Schneier, has stated, “Complexity is the worst enemy of security.” The sheer complexity of any given network can lead to a lot of mistakes, especially when it comes to multiple firewalls with complex rule sets. Simplifying the firewall environment and management processes is necessary for good management.
Up to 30 percent of implemented rule changes in large firewall infrastructures are unnecessary because the firewalls are already allowing the requested traffic!
Under time pressure, firewall administrators often create more rules which turn out to be redundant given already-existing rules. This wastes valuable time and makes the firewalls even harder to manage.
The introduction of new things opens up security gaps. New hires, software patches, upgrades and network updates all increase risk exposure. The situation is further complicated in larger organizations which may have a mixed security estate comprising traditional, next-generation and virtualized firewalls from multiple vendors across clouds and on-premise data centers, all with hundreds of policies and thousands of rules.
What about unexpected, quick-fixes that enable access to certain resources or capabilities? In many cases, a fix is made in a rush (after all, who wants a C-level exec breathing down their neck because he wants to access the network from his new tablet RIGHT NOW?) without sufficient consideration of whether that change is allowable under current security policies, or if it introduces new exposures.
Sure, you can’t predict when users will make change requests, but you can certainly prepare the process for handling these requests whenever they arise. Bringing both IT operations and security teams together to prepare game plans for these situations – and for other ‘knowns’ such as network upgrades, change freezes, and audits – helps to minimize the risk of security gaps.
What’s more, there are solutions that automate day-to-day firewall management tasks and link these changes and procedures so that they are recorded as part of the change management plan. In fact, automated technologies can help bridge the gap between change management processes and what’s really taking place. They enhance accuracy, by removing people from the equation to a very large degree. For example, a sophisticated firewall and topology-aware workflow system that is able to identify redundant and unneeded change requests can increase the productivity of the IT staff.
IT operations and security groups are ultimately responsible for making sure that systems are functioning properly so that business goals are continuously met. However, these teams approach business continuity from different perspectives. The security department’s number one goal is to protect the business and its data whereas the IT operations team is focused on keeping systems up and running. It is natural for these two teams to clash. However, oftentimes, IT operations and security teams align their perspectives because both have a crucial ownership stake. The business has to keep running AND it has to be secure.
But this kind of alignment of interests is easier said than done.
To achieve the alignment, organizations must re- examine current IT and security processes. Let’s have a look at some examples of what happens when alignment is not performed.
A classic lack of communication between the IT operations and security groups put XYZ Corporation at risk. An IT department administrator, who was trying to be helpful, took the initiative to set up (on his own, with no security involvement or documentation) an FTP share for a user who needed to upload files in a hurry.
By making this off-the-cuff change, the IT admin quickly addressed the client’s request and the files were uploaded. However, the FTP account lingered unsecured well beyond its effective “use by” date. By the next day, the security team noticed larger spikes of inbound traffic to the server from this very FTP account. Hackers abound. The FTP site had been compromised and was being exploited to host pirated movies.
A core provider of e-commerce services to businesses in the U.S. suffered a horrible fate due to a simple, but poorly managed, firewall change. One day, all e-commerce transactions in and out of its network ceased and the entire business was taken offline for several hours. The costs were astronomical.
An out-of-band (and untested) change to a core firewall broke the communication between the e-commerce application and the internet. Business activity ground to a halt. Executive management got involved and the responsible IT staff members were reprimanded. Hundreds of thousands of dollars later, the root cause of the outage was uncovered: IT staff, oblivious to the consequences, chose not to test their firewall changes, bypassing their “burdensome” ITIL-based change management procedures.
Taken from The Big Collection of Firewall Management Tips
Document, document, document … And when in doubt, document some more!
“It is especially critical for people to document the rules they add or change so that other administrators know the purpose of each rule and whom to contact about it. Good documentation can make troubleshooting easy. It reduces the risk of service disruptions that inadvertently occur when an administrator deletes or changes a rule they do not understand.”
– Todd, InfoSec Architect, United States
“Keep a historical change log of your firewall policy so you can return to safe harbor in case something goes wrong. A proper change log should include the reason for the change, the requester and approval records.”
– Pedro Cunha, Engineer, Oni, Portugal
Automation is the key. It helps staff disengage from firefighting and bouncing reactively between incidents. It helps them gain control.
The right automation solution can help teams track down potential traffic or connectivity issues and highlight areas of risk. Administrators can get a handle on the current status of policy compliance across mixed estates of traditional, next-generation and virtualized firewalls as well as hybrid on-prem and cloud estates. The solution can also automatically pinpoint the devices that may require changes and show how to create and implement those changes in the most secure way. Automation not only makes firewall change management easier and more predictable across large estates and multiple teams, but also frees staff to handle more strategic security and compliance tasks. Let the solution handle the heavy lifting and free up the staff for other things.
To ensure a proper balance between business continuity and security, look for a firewall policy management solution that:
Taken from The Big Collection of Firewall Management Tips
“Perform reconciliation between change requests and actual performed changes. Looking at the unaccounted changes will always surprise you. Ensuring every change is accounted for will greatly simplify your next audit and help in day-to-day troubleshooting.”
– Ron, Manager, Australia
“Have a workflow process for implementing a security rule from the user requesting change, through the approval process and implementation.”
– Gordy, Senior Network Engineer, United States
Here is the secret to getting network security policy change management right.
Once a request is made, a change-request process should include the following steps:
In some cases (e.g., data breach) a change to a firewall rule set must be made immediately, where, even with all the automation in the world, there is no time to go through the 10 steps. To address this type of situation, an emergency process should be defined and documented.
The best solutions allow for:
Manual firewall change management is a time-consuming and error-prone process. Consider a typical change order that requires a total of four hours of work by several team members during the change lifecycle, including communication, validation, risk assessment, planning and design, execution, verification, documentation, auditing and measurement.
Based on these assumptions, AlgoSec customers have reported significant cost savings (as much as 60%) achieved through:
While change management is complex stuff, the decision for your business is actually simple. You can continue to slowly chug along with manual change management processes that drain your IT resources and impede agility. Or you can accelerate your processes with an automated network change- management workflow solution that aligns the different stakeholders involved in the process (network operations, network security, compliance, business owners, etc.) and helps the business run more smoothly.
Think of your change process as a key component of the engine of an expensive car (in this case, your organization). Would you drive your car at high speed if you didn’t have tested, dependable brakes or a steering wheel? Hopefully, the answer is no! The brakes and steering wheel are analogous to change controls and processes. Rather than slowing you down, they actually make you go faster, securely! Power steering and power brakes (in this case, firewall-aware integration and automation) help you zoom to success.