Spooky Network Security and Operations Incidents

Since it’s Halloween, I thought it would be fun to revisit some chilling tales of network operations and security incidents. Ghosts and goblins aren’t the culprits here… just network and security processes that are lacking or not being followed. The below spooky network security stories are shared by Matthew Pascucci, a security practitioner (and AlgoSec guest blogger), security consultant and author of Hacking for Dummies, Kevin Beaver, and AlgoSec’s own security architect, Kyle Wickert…

High Availability Becomes Unavailable… as told to by Matthew Pascucci

“Many years ago I worked with an organization that had routers in a high-availability cluster for redundancy. During one of our busy times of the year a consultant made a change to the cluster regarding a project he was working on which brought the company down during a very busy time of the year. There were no change control tickets or review process put in and the change cost the company a great deal of revenue. If there were a ticket put in for this incident it would have been caught, but even if that wasn’t there having the ability to monitor changes in your network is a way to catch rogue changes like these.”

Misconfigured Firewalls Expose the Network to Attack… as told to by Matthew Pascucci

“There was a particular network I worked in once that was constantly being breached. We started looking at ways the attackers were gaining access and noticed that there were improperly configured firewall rules that allowed full NetBios access to all systems in the DMZ. These webservers were also running all applications as administrator with an old version of Microsoft IIS. (Here are tips on how to effectively architect the DMZ) After cleaning up the firewall access rules, removing unneeded services and updating vulnerable software we were able to help the network owners for the time being. There should be a constant audit of your environment as well as vulnerability scans both internally and externally that would find this low hanging fruit. Using tools that point out vulnerabilities and areas that you’re not compliant are extremely beneficial to your security posture.”

HTTP Open on the Router!?… as told to by Matthew Pascucci

“While reviewing security for a company from the perimeter I discovered that HTTP was enabled on their core Cisco routers. They were both running very old versions of the IOS and were using the default credentials to log into the device. After getting into the router I was able to escalate to “enable” mode and could in theory have changed routes or wiped the NVRAM.”

Out-of-Band Firewall Changes Cause Out-of-Service… as told to by Kevin Beaver

“One day all of the transactions in and out of an e-commerce provider’s network ceased and the entire business was taken offline for a number of hours. It ended up being a few members of the firewall team had made some out-of-band (and untested) changes to a core firewall that broke the communication between the e-commerce application and the rest of the Internet. Hundreds of thousands of dollars later, the root cause of the outage was revealed: IT staff chose not to test their firewall changes—bypassing their “burdensome” ITIL-based change management procedures—and ignored the consequences.”

Who’s Controlling Your Cloud? … as told to by Kyle Wickert

“Once your information is sent into the cloud, there is no pulling it back in. Cloud service providers use varying architectures, processes and procedures that may place your data in many precarious places. In one past experience, a cloud email record keeping service were themselves leveraging several different cloud providers to provide a service to its customers. Data was being processed and stored through these different clouds, finally ending up in a server rack, which resided in a bedroom apartment. The troublesome piece – this email record keeping service was being provided to multi-nation banks across the globe, storing the emails of traders.”

Hopefully these spooky security stories of firewall and router mishaps, security change management breakdowns and moving data and applications to the cloud can scare more organizations into better security and operations. Have a happy and safe Halloween!