Earlier this year, I blogged about the need to link security with DevOps, and how this can help the development, application delivery and security teams to collaborate more effectively, improve the security of business applications, and cut the costs and time of fixing any problems that may arise.
To illustrate how this approach can work in practice, I’m going to look a typical DevOps scenario. In this case, the development team has added new functionality to an existing business application, and is rolling the updated application out. However, while the new functionality worked as planned in both the test and pre-production environments, the application fails when moved to the live production environment. So, what went wrong, and how can we fix it?
A common reason for this type of application outage is that the new functionality required new connectivity flows, but these flows were blocked in the live production environment. This can happen because security policies are often more relaxed in the development, test and pre-production environments: the network filters or gateways which would typically be deployed in production may not be in place in the test or pre-production environment, which results in more open connectivity than in production. As a result, everything works fine in the development and test environments, but when moved to production, the additional security measures block some of the connectivity to resources that the application needs to function.
Here are four tips to help you avoid connectivity related outages, when applications are moved from test and pre-production to the live production environment:
It’s important to document the connectivity flows that each application requires. The development team should to maintain an up-to-date repository of all relevant flows including the source and destination IP addresses of each flow, and the services and network functions in use between the application and the various external resources that it relies on.
Make sure to update these application records whenever new functionality is added that requires new connectivity. A solution that automatically handles these updates can also ensure that any changes to these connectivity flows will trigger a review by the security team as early as possible – preferably during the development cycle.
It’s good practice to tighten security in the test and pre-production environments, so that both environments mimic the live production environment as closely as possible in terms of their functionality, network topology and the filtering policies.
This way, even if the development team forgets to document new connectivity flows, or does not realise that there is a new flow needed, any problems will be caught early in test or pre-production, and not in the live environment. It’s much better, and less time-consuming, to fix problems before the application moves to full production.
When you structure your test and pre-production environments so that they’re as similar as possible to the production environment, there’s an important point you need to bear in mind: There should be a separate record in the application connectivity repository, for each application, in each of the separate environments. This is because in each environment, the application will have the same logical structure and services, but the servers, databases and other resources will have different IP addresses from the servers and databases in the other environments.
Then as you migrate between stages in the application lifecycle, from development to production, you need to ensure that the security policies for the next stage allow the necessary traffic. It’s not just a simple process of copying: you need replace all the IP addresses in the rules and policies of the previous stage with the new addresses for the new stage. A security policy automation that supports DevOps methodologies can automatically manage this process for you across each new environment in the DevOps lifecycle, as I describe in this whiteboard video.
By building security into the DevOps process early, organizations can maximize the speed and agility of application development that the process promises, while minimizing the risks of problems and outages when they go live.
Receive notifications of new posts by email.