NACL best practices: How to combine security groups with network ACLs effectively

Like all modern cloud providers, Amazon adopts the shared responsibility model for cloud security. Amazon guarantees secure infrastructure for Amazon Web Services, while AWS users are responsible for maintaining secure configurations.

That requires using multiple AWS services and tools to manage traffic. You’ll need to develop a set of inbound rules for incoming connections between your Amazon Virtual Private Cloud (VPC) and all of its Elastic Compute (EC2) instances and the rest of the Internet. You’ll also need to manage outbound traffic with a series of outbound rules.

Your Amazon VPC provides you with several tools to do this. The two most important ones are security groups and Network Access Control Lists (NACLs).

• Security groups are stateful firewalls that secure inbound traffic for individual EC2 instances.

• Network ACLs are stateless firewalls that secure inbound and outbound traffic for VPC subnets.

Managing AWS VPC security requires configuring both of these tools appropriately for your unique security risk profile. This means planning your security architecture carefully to align it the rest of your security framework.

For example, your firewall rules impact the way Amazon Identity Access Management (IAM) handles user permissions. Some (but not all) IAM features can be implemented at the network firewall layer of security.

Before you can manage AWS network security effectively, you must familiarize yourself with how AWS security tools work and what sets them apart.

Everything you need to know about security groups vs NACLs

AWS security groups explained:

Every AWS account has a single default security group assigned to the default VPC in every Region. It is configured to allow inbound traffic from network interfaces assigned to the same group, using any protocol and any port. It also allows all outbound traffic using any protocol and any port. Your default security group will also allow all outbound IPv6 traffic once your VPC is associated with an IPv6 CIDR block.

You can’t delete the default security group, but you can create new security groups and assign them to AWS EC2 instances. Each security group can only contain up to 60 rules, but you can set up to 2500 security groups per Region.

You can associate many different security groups to a single instance, potentially combining hundreds of rules. These are all allow rules that allow traffic to flow according the ports and protocols specified.

For example, you might set up a rule that authorizes inbound traffic over IPv6 for linux SSH commands and sends it to a specific destination. This could be different from the destination you set for other TCP traffic.

Security groups are stateful, which means that requests sent from your instance will be allowed to flow regardless of inbound traffic rules. Similarly, VPC security groups automatically responses to inbound traffic to flow out regardless of outbound rules.

However, since security groups do not support deny rules, you can’t use them to block a specific IP address from connecting with your EC2 instance. Be aware that Amazon EC2 automatically blocks email traffic on port 25 by default – but this is not included as a specific rule in your default security group.

AWS NACLs explained:

Your VPC comes with a default NACL configured to automatically allow all inbound and outbound network traffic. Unlike security groups, NACLs filter traffic at the subnet level. That means that Network ACL rules apply to every EC2 instance in the subnet, allowing users to manage AWS resources more efficiently.

Every subnet in your VPC must be associated with a Network ACL. Any single Network ACL can be associated with multiple subnets, but each subnet can only be assigned to one Network ACL at a time. Every rule has its own rule number, and Amazon evaluates rules in ascending order.

The most important characteristic of NACL rules is that they can deny traffic. Amazon evaluates these rules when traffic enters or leaves the subnet – not while it moves within the subnet. You can access more granular data on data flows using VPC flow logs.

Since Amazon evaluates NACL rules in ascending order, make sure that you place deny rules earlier in the table than rules that allow traffic to multiple ports. You will also have to create specific rules for IPv4 and IPv6 traffic – AWS treats these as two distinct types of traffic, so rules that apply to one do not automatically apply to the other.

Once you start customizing NACLs, you will have to take into account the way they interact with other AWS services. For example, Elastic Load Balancing won’t work if your NACL contains a deny rule excluding traffic from 0.0.0.0/0 or the subnet’s CIDR. You should create specific inclusions for services like Elastic Load Balancing, AWS Lambda, and AWS CloudWatch. You may need to set up specific inclusions for third-party APIs, as well.

You can create these inclusions by specifying ephemeral port ranges that correspond to the services you want to allow. For example, NAT gateways use ports 1024 to 65535. This is the same range covered by AWS Lambda functions, but it’s different than the range used by Windows operating systems.

When creating these rules, remember that unlike security groups, NACLs are stateless. That means that when responses to allowed traffic are generated, those responses are subject to NACL rules. Misconfigured NACLs deny traffic responses that should be allowed, leading to errors, reduced visibility, and potential security vulnerabilities.

How to configure and map NACL associations

A major part of optimizing NACL architecture involves mapping the associations between security groups and NACLs. Ideally, you want to enforce a specific set of rules at the subnet level using NACLs, and a different set of instance-specific rules at the security group level. Keeping these rulesets separate will prevent you from setting inconsistent rules and accidentally causing unpredictable performance problems.

The first step in mapping NACL associations is using the Amazon VPC console to find out which NACL is associated with a particular subnet. Since NACLs can be associated with multiple subnets, you will want to create a comprehensive list of every association and the rules they contain.

To find out which NACL is associated with a subnet:

1. Open the Amazon VPC console.

2. Select Subnets in the navigation pane. Select the subnet you want to inspect.

3. The Network ACL tab will display the ID of the ACL associated with that network, and the rules it contains.

To find out which subnets are associated with a NACL:

1. Open the Amazon VPC console.

2. Select Network ACLS in the navigation pane. Click over to the column entitled Associated With.

3. Select a Network ACL from the list.

4. Look for Subnet associations on the details pane and click on it. The pane will show you all subnets associated with the selected Network ACL.

Now that you know how the difference between security groups and NACLs and you can map the associations between your subnets and NACLs, you’re ready to implement some security best practices that will help you strengthen and simplify your network architecture.

5 best practices for AWS NACL management

1. Pay close attention to default NACLs, especially at the beginning
   

Since every VPC comes with a default NACL, many AWS users jump straight into configuring their VPC and creating subnets, leaving NACL configuration for later. The problem here is that every subnet associated with your VPC will inherit the default NACL. This allows all traffic to flow into and out of the network.

Going back and building a working security policy framework will be difficult and complicated – especially if adjustments are still being made to your subnet-level architecture. Taking time to create custom NACLs and assign them to the appropriate subnets as you go will make it much easier to keep track of changes to your security posture as you modify your VPC moving forward.

1. Implement a two-tiered system where NACLs and security groups complement one another

Security groups and NACLs are designed to complement one another, yet not every AWS VPC user configures their security policies accordingly. Mapping out your assets can help you identify exactly what kind of rules need to be put in place, and may help you determine which tool is the best one for each particular case.

For example, imagine you have a two-tiered web application with web servers in one security group and a database in another. You could establish inbound NACL rules that allow external connections to your web servers from anywhere in the world (enabling port 443 connections) while strictly limiting access to your database (by only allowing port 3306 connections for MySQL).

1. Look out for ineffective, redundant, and misconfigured deny rules

Amazon recommends placing deny rules first in the sequential list of rules that your NACL enforces. Since you’re likely to enforce multiple deny rules per NACL (and multiple NACLs throughout your VPC), you’ll want to pay close attention to the order of those rules, looking for conflicts and misconfigurations that will impact your security posture.

Similarly, you should pay close attention to the way security group rules interact with your NACLs. Even misconfigurations that are harmless from a security perspective may end up impacting the performance of your instance, or causing other problems. Regularly reviewing your rules is a good way to prevent these mistakes from occurring.

1. Limit outbound traffic to the required ports or port ranges

When creating a new NACL, you have the ability to apply inbound or outbound restrictions. There may be cases where you want to set outbound rules that allow traffic from all ports. Be careful, though. This may introduce vulnerabilities into your security posture.

It’s better to limit access to the required ports, or to specify the corresponding port range for outbound rules. This establishes the principle of least privilege to outbound traffic and limits the risk of unauthorized access that may occur at the subnet level.

1. Test your security posture frequently and verify the results

How do you know if your particular combination of security groups and NACLs is optimal? Testing your architecture is a vital step towards making sure you haven’t left out any glaring vulnerabilities. It also gives you a good opportunity to address misconfiguration risks.

This doesn’t always mean actively running penetration tests with experienced red team consultants, although that’s a valuable way to ensure best-in-class security. It also means taking time to validate your rules by running small tests with an external device. Consider using AWS flow logs to trace the way your rules direct traffic and using that data to improve your work.

How to diagnose security group rules and NACL rules with flow logs

Flow logs allow you to verify whether your firewall rules follow security best practices effectively. You can follow data ingress and egress and observe how data interacts with your AWS security rule architecture at each step along the way. This gives you clear visibility into how efficient your route tables are, and may help you configure your internet gateways for optimal performance.

Before you can use the Flow Log CLI, you will need to create an IAM role that includes a policy granting users the permission to create, configure, and delete flow logs.

Flow logs are available at three distinct levels, each accessible through its own console:

• Network interfaces

• VPCs

• Subnets

You can use the ping command from an external device to test the way your instance’s security group and NACLs interact. Your security group rules (which are stateful) will allow the response ping from your instance to go through. Your NACL rules (which are stateless) will not allow the outbound ping response to travel back to your device.

You can look for this activity through a flow log query. Here is a quick tutorial on how to create a flow log query to check your AWS security policies.

First you’ll need to create a flow log in the AWS CLI. This is an example of a flow log query that captures all rejected traffic for a specified network interface. It delivers the flow logs to a CloudWatch log group with permissions specified in the IAM role:

Assuming your test pings represent the only traffic flowing between your external device and EC2 instance, you’ll get two records that look like this: 

To parse this data, you’ll need to familiarize yourself with flow log syntax. Default flow log records contain 14 arguments, although you can also expand custom queries to return more than double that number:

• Version tells you the version currently in use. Default flow logs requests use Version 2. Expanded custom requests may use Version 3 or 4.

• Account-id tells you the account ID of the owner of the network interface that traffic is traveling through. The record may display as unknown if the network interface is part of an AWS service like a Network Load Balancer.

• Interface-id shows the unique ID of the network interface for the traffic currently under inspection.

• Srcaddr shows the source of incoming traffic, or the address of the network interface for outgoing traffic. In the case of IPv4 addresses for network interfaces, it is always its private IPv4 address.

• Dstaddr shows the destination of outgoing traffic, or the address of the network interface for incoming traffic. In the case of IPv4 addresses for network interfaces, it is always its private IPv4 address.

• Srcport is the source port for the traffic under inspection.

• Dstport is the destination port for the traffic under inspection.

• Protocol refers to the corresponding IANA traffic protocol number.

• Packets describes the number of packets transferred.

• Bytes describes the number of bytes transferred.

• Start shows the start time when the first data packet was received. This could be up to one minute after the network interface transmitted or received the packet.

• End shows the time when the last data packet was received. This can be up to one minutes after the network interface transmitted or received the data packet.

• Action describes what happened to the traffic under inspection:

  • ACCEPT means that traffic was allowed to pass.

  • REJECT means the traffic was blocked, typically by security groups or NACLs.
    

• Log-status confirms the status of the flow log:

  • OK means data is logging normally.

  • NODATA means no network traffic to or from the network interface was detected during the specified interval.

  • SKIPDATA means some flow log records are missing, usually due to internal capacity restraints or other errors.

Going back to the example above, the flow log output shows that a user sent a command from a device with the IP address 203.0.113.12 to the network interface’s private IP address, which is 172.31.16.139. The security group’s inbound rules allowed the ICMP traffic to travel through, producing an ACCEPT record.

However, the NACL did not let the ping response go through, because it is stateless. This generated the REJECT record that followed immediately after. If you configure your NACL to permit output ICMP traffic and run this test again, the second flow log record will change to ACCEPT.

azon Web Services (AWS) is one of the most popular options for organizations looking to migrate their business applications to the cloud. It’s easy to see why:  AWS offers high capacity, scalable and cost-effective storage, and a flexible, shared responsibility approach to security. Essentially, AWS secures the infrastructure, and you secure whatever you run on that infrastructure.

However, this model does throw up some challenges. What exactly do you have control over?  How can you customize your AWS infrastructure so that it isn’t just secure today, but will continue delivering robust, easily managed security in the future?

The basics:  security groups

AWS offers virtual firewalls to organizations, for filtering traffic that crosses their cloud network segments.  The AWS firewalls are managed using a concept called Security Groups.  These are the policies, or lists of security rules, applied to an instance – a virtualized computer in the AWS estate.  AWS Security Groups are not identical to traditional firewalls, and they have some unique characteristics and functionality that you should be aware of, and we’ve discussed them in detail in video lesson 1:  the fundamentals of AWS Security Groups, but the crucial points to be aware of are as follows.

First, security groups do not deny traffic – that is, all the rules in security groups are positive, and allow traffic.  Second, while security group rules can be set to specify a traffic source, or a destination, they cannot specify both on the same rule.  This is because AWS always sets the unspecified side (source or destination) as the instance to which the group is applied.

Finally, single security groups can be applied to multiple instances, or multiple security groups can be applied to a single instance:  AWS is very flexible.  This flexibility is one of the unique benefits of AWS, allowing organizations to build bespoke security policies across different functions and even operating systems, mixing and matching them to suit their needs.

Adding Network ACLs into the mix

To further enhance and enrich its security filtering capabilities AWS also offers a feature called Network Access Control Lists (NACLs).  Like security groups, each NACL is a list of rules, but there are two important differences between NACLs and security groups.

The first difference is that NACLs are not directly tied to instances, but are tied with the subnet within your AWS virtual private cloud that contains the relevant instance.  This means that the rules in a NACL apply to all of the instances within the subnet, in addition to all the rules from the security groups.  So a specific instance inherits all the rules from the security groups associated with it, plus the rules associated with a NACL which is optionally associated with a subnet containing that instance.  As a result NACLs have a broader reach, and affect more instances than a security group does.

The second difference is that NACLs can be written to include an explicit action, so you can write ‘deny’ rules – for example to block traffic from a particular set of IP addresses which are known to be compromised.  The ability to write ‘deny’ actions is a crucial part of NACL functionality.

It’s all about the order

As a consequence, when you have the ability to write both ‘allow’ rules and ‘deny’ rules, the order of the rules now becomes important.  If you switch the order of the rules between a ‘deny’ and ‘allow’ rule, then you’re potentially changing your filtering policy quite dramatically.

To manage this, AWS uses the concept of a ‘rule number’ within each NACL.  By specifying the rule number, you can identify the correct order of the rules for your needs. You can choose which traffic you deny at the outset, and which you then actively allow.

As such, with NACLs you can manage security tasks in a way that you cannot do with security groups alone.  However, we did point out earlier that an instance inherits security rules from both the security groups, and from the NACLs – so how do these interact?

The order by which rules are evaluated is this; For inbound traffic, AWS’s infrastructure first assesses the NACL rules.  If traffic gets through the NACL, then all the security groups that are associated with that specific instance are evaluated, and the order in which this happens within and among the security groups is unimportant because they are all ‘allow’ rules.

For outbound traffic, this order is reversed:  the traffic is first evaluated against the security groups, and then finally against the NACL that is associated with the relevant subnet.

You can see me explain this topic in person in my new whiteboard video: