The Dangers of the Holiday Freeze

During the Christmas holiday season many companies utilize network freezes to protect themselves from errant changes which could cause outages during valuable high traffic days. Many companies put these freezes into effect to prevent business disruption – and potentially lost business – and are generally good ideas. But by protecting the organization could we really be putting ourselves at risk? To freeze for the sake of freezing can cause a company to stay frozen in a vulnerable, unprotected state.

Before we go on let me say I’m all for the network and system freeze ideology… if it’s done right. It’s a responsible thing to do when you have a large business, especially one that makes a large deal of revenue from the holiday season. Every time you make a change to a production system or network you risk the possibility of having something go wrong. That something could be a downed system, corrupted database, faulty route, misplaced firewall rule, etc. all culminating to the end result: system outage. It’s bad enough that a system outage could take place during normal non-holiday traffic, but if there’s an outage during your peak period, heads will roll.

So I understand why we need network freezes, it’s the responsible thing to do, but as a company we shouldn’t hide behind them to protect ourselves from the true threats. In my past experience I’ve had to wait months for a simple firewall change to be pushed out due to freezes. It’s not only the freeze itself, but the amount of work that builds up after a freeze that can put projects and security behind. As I said, I’m all for freezes, but when simple non-production or security changes are overlooked because of them, it becomes counter intuitive. In these cases the business should have the confidence to move forward with these changes.

It starts with the information security department and IT being on the same page when it comes to network freezes. During the freeze IT should be aware that anything InfoSec brings to their attention is a matter of extreme importance. The security department is here to protect the company, not make work for other departments. On the other hand, the IT department is in the business of keeping systems up and working. InfoSec needs to understand that if there’s a low-risk vulnerability and it’s not easily exploitable, they might have to wait for the freeze to thaw before remediating. Both teams need to work in tandem to achieve that proper balance of security while keeping the business profitable. Using simple risk management and common sense can go a long way here.

However, many times the ideologies between these two departments clash, and we see it the most during the time of network freezes. If there’s a security remediation or change to an internal system with no impact to production systems, there really isn’t any reason why this shouldn’t be completed. It’s just easier at times to say “No” to everything during a freeze, which can stall projects and put your environment at unneeded risk.

It’s also imperative to continue your standard operating procedures in information security during freezes; at no time should you deviate from your SOP… EVER. As I mentioned before, this is not a time to put security on hold. This is a time to be diligent and proactive to guard against attacks during this profitable time, all the while using common sense. We in security need to be mindful of the business; it’s because of them that we have jobs in the first place. If we stop them from making money we won’t have anything to protect. There’s a fine line we need to walk during freeze periods, but we should never get complacent.

Since vulnerabilities never sleep and neither do the hackers, we shouldn’t let our guard down during a holiday freeze. Just because it’s a freeze doesn’t mean that we should stop performing security. I would argue that this is the time that security should be taken most seriously. Having a known vulnerability on your external network during a freeze could potentially cost you more in the long run if it remains. It’s here that you need to make a calculated decision to remediate a system during a holiday freeze.

So in short, during this holiday freeze don’t stand behind the frozen shield with your hands up, do something!! Security is here to protect your systems and network, even during freezes. Just because there’s a freeze on the network doesn’t mean people should stop working. The business needs to make money, but they shouldn’t pull the plug on all system changes, especially if they’re not in production. We shouldn’t hide behind the freeze or use it as an excuse to disallow work our update our security posture. Don’t wait until after a freeze to start thinking like a security professional again.