Securing AWS Security Groups: Restricting Egress Rules

Good afternoon!

Today’s article demonstrates a surprisingly easy way to tighten the network-layer permissions in an AWS VPC. (If you’re in AWS but you’re not in a VPC:Ā šŸ˜”)

Security Groups have ingress and egress rules (also called inbound and outbound rules). In most SGs, the egress rules allow all traffic to everywhere. You’ve probably seen this:


That’s a problem because, someday, you will get hacked. Breaches are inevitable, perfect security doesn’t exist. Someone or some bot will get access to what that SG was protecting (EC2 instance, Fargate ENI, whatever). When that happens, they can send out anything they want. Instead, you want them to have the most limited capabilities possible. They should find walls everywhere they turn. It’s not just about what’s coming in, it’s also about what’s going out.

I like AWS’s directive from the third bullet of theĀ Security Pillar in their Well-Architected Framework: “Apply security at all layers”. Incoming and outgoing.

Fortunately, SG outbound rules are easy to tighten!

Imagine an Autoscaling Group of Linux instances. They handle background jobs and sometimes engineers SSH into them to run diagnostics (for this example we’re imagining you haven’t set up SSM Session Manager). At boot time they yum install some packages.

As usual, you need an incoming rule to allow SSH:


Now, here’s how to determine the outgoing rules you need: if the resource protected by the SG starts the connection, you need an outgoing rule. If it only replies to connections started by someone else, you don’t need an outgoing rule. Details farther down.

For anyone who’s forgotten, yum uses HTTP/S (ports 80/443) and FTP (ports 20/21).

The instance receives and then replies to SSH requests. That means it didn’t start those connections, so we don’t need an outgoing rule. But, it sends HTTP/S and FTP requests because it has to request to download packages from the yum servers. That means it starts those connections, so we need outgoing rules:


That’s it. If you attach this SG to the instances, engineers will still be able to SSH into it and it will still be able to yum install packages. But, if the instance tries to do something else, like maybe connect to a MySQL database (port 3306), the SG will block that traffic and the connection will time out. When something Evil breaks into this instance, itĀ won’t be able to access that database.

These three rules are enough because Security Groups are stateful. To dramatically simplify statefulness, it means that SGs know whether traffic passing through them is part of a connection the instance has already agreed to. If it is, they pass the traffic whether or not a rule is present.

I’m skipping a ton of details. This is meant to help you quickly do one round of tightening on your network. There are advantages and disadvantages to stateful filtering, and the details can take youĀ deep into the weeds, but most of the time it’s enough to know what rules you need. If you want to go farther with this on your own, check out this project for a demonstration environment where you can experiment with variations.

Two details:

  • We have to allow yum traffic to because we don’t know the IP addresses of the upstream yum servers.
  • VPC subnet ACLs areĀ not stateful, so you need different rules for those. I’ll cover that in another article.

In this example we weren’t able to stop whatever Evil Thing had broken into your instance from sending your Super Secret Stuff to some Evil Webserver somewhere, but we did take away some of their other tools. We’ve put up one more wall they might run in to, and every wall they hit might stop them. Walls at every turn.

Stay safe!


Need more than just this article? We’re available to consult.

You might also want to check out these related articles:

Beating AWS Security Groups


Today I’ll show you how to pass traffic through an AWS Security Group that’s configured not to allow that traffic.

This isn’t esoteric hacking, it’s a detail in the difference between config and state that’s easy to miss when you’re operating an infrastructure.

Like I showed in a previous post, AWS Security Groups are stateful. They know the difference between the first packet of a new connection and packets that are part of connections that are already established.

This statefulness is why you can let host A SSH to host B just by allowing outgoing SSH on A’s SG and incoming SSH on B’s SG. B doesn’t need to allow outgoing SSH because it knows the return traffic is part of a connection that was already allowed. Similarly for A and incoming SSH.

Here’s the detail of today’s post: if the Security Group sees traffic as part of an established connection, it’ll allow it even if its rules say not to. Ok now let’s break a Security Group.

The Lab

Two hosts, testa and testb. One SG for each, both allowing all outgoing traffic. Testb’s SG allows incoming TCP on port 4321 (a random ephemeral port I’m using for this test):


To test traffic flow, I’m going to useĀ nc. It’s a common Linux utility that sends and receives TCP traffic:

  • Listen: nc -l [port]
  • Send: nc [host] [port]

Test Steps:

(screenshots of shell output below)

  1. Listen on port 4321 on testb.
  2. Start a connection from testa to port 4321 on testb.
  3. Send a message. It’s delivered, as expected.
  4. Remove testb’s SG rule allowing port 4321:TrafficDenied
  5. Send another message through the connection. It will get through! There’s no rule to allow it, but it still gets through.


To show nothing else was going on, let’s redo the test with the security group as it is now (no rule allowing 4321).

  1. Quit ncĀ on testa to close the connection. You’ll see it also close on testb.
  2. Listen on port 4321 on testb.
  3. Start a connection from tests a to port 4321 on testb.
  4. Send a message. Not delivered. This time there was no established connection so the traffic was compared to the SGs rules. There was no rule to allow it, so it was denied.

Testb Output

(where we listened)


Only two messages got through.

Testa Output

(where we sent)


We sent three messages. The last two were sent while the SG had the same rules, but the first message was allowed and the second was denied.


The rules in Security Groups don’t apply to open (established) TCP connections. If you need to ensure traffic isn’t flowing between two instances you can’t just remove rules from your SGs. You have to close all open connections.

Happy securing,


Need more than just this article? We’re available to consult.

You might also want to check out these related articles:

AWS Security Groups: Stateful Statelessness



Recently, I rediscovered a fiddly networking detail: although ICMP’s ping is stateless, AWS security groups will pass returnĀ ping traffic even when only one direction is defined in their rules. I wanted to see this in action, so I built a lab.

If you just asked, “Watā“”, keep reading. Skip to the next section if you just want the code.


Network hosts using stateful protocols (like TCP) distinguish between packets that are part of an established connection and packets that are new. For example, when I SSH (which runs on TCP) from A to B:

  1. A asks B to start a new connection.
  2. B agrees.
  3. A and B exchange bunches of packets that are part of the connection they agreed to.
  4. A and B agree to close the connection.

There’s a difference between a new packet and a packet that’s part of an ongoing connection. That means the connection, and its packets, haveĀ state (e.g. new vs established). Stateful firewalls (vs stateless) are aware of this:

  1. A ask B to start a new connection.
  2. Firewalls in between allow these packets if there is an explicit rule allowing traffic from A to B.
  3. A and B exchange bunches of packets.
  4. Firewalls in between allow the packets from A to B because of the explicit rule above. However, they allow the return traffic from B to AĀ even if there is no explicit rule to allow it. Since B agreed to the connection the firewall assumes that packets in that connection should be allowed.

This is why you only need an outgoing rule on A’s Security Group (SG) and an incoming rule on B’s Security Group to SSH from A to B. AWS SGs are stateful, and allow the return traffic implicitly.

Ok, here’s the gnarly bit. ICMP (the protocol behind ping) is stateless. Hosts don’t have a negotiation phase where the agree to establish a connection. They just send packets and hope. So, doesn’t that mean I need to write explicit firewall rules in the SGs to allow the return traffic? If the firewall can’t see the state of the connection, it won’t be able to implicitly figure out to allow that traffic, right?

Nope, they infer state based on timeouts and packet types. ICMP pings are ECHO requests answered by ECHO replies. If the SG has seen a request within the timeout, it makes the educated guess that replies are essentially part of “established” connections and allows them. This is what I wanted to see in action.

The Lab

I setup a VPC with two hosts, A ( and B ( They’re in different subnets but the ACLs allow all traffic so they don’t influence the test. Here are the SG rules for A ( covers the entire VPC):


And the rules for B:


A allows outgoing ICMP to B, and B allows incoming ICMP from A. The return traffic is not allowed by any rules.

The Test Script

I didn’t find a way to send just replies without requests in Linux, so I bodged together a Python script:

This is a stripped-down version of ping that allows you to send a reply without responding a request. This was needed
to test the details of how security groups handle state with ICMP traffic. You shouldn't use this for normal

The ping implementation was based on Samuel Stauffer's python-ping: (which only
works with Python 2).

This must be run as root.
You must tell the Linux kernel to ignore ICMP before you run this or it'll eat some of the traffic:
    echo 1 > /proc/sys/net/ipv4/icmp_echo_ignore_all

import argparse, socket, struct, time

def get_arguments():
    parser = argparse.ArgumentParser(formatter_class=argparse.ArgumentDefaultsHelpFormatter)
    parser.add_argument('--send-request', metavar='IP_ADDRESS', type=str, help='IP address to send ECHO request.')
    parser.add_argument('--receive', action='store_true', help='Wait for a reply.')
    parser.add_argument('--send-reply', metavar='IP_ADDRESS', type=str, help='IP address to send ECHO reply.')
    return parser.parse_args()

def receive(my_socket):
    while True:
        recPacket, addr = my_socket.recvfrom(1024)
        icmpHeader = recPacket[20:28]
        icmp_type, code, checksum, packetID, sequence = struct.unpack("bbHHh", icmpHeader)
        print('Received type {}.'.format(icmp_type))

def ping(my_socket, dest_addr, icmp_type):
    dest_addr = socket.gethostbyname(dest_addr)
    bytesInDouble = struct.calcsize("d")
    data = (192 - bytesInDouble) * "Q"
    data = struct.pack("d", time.time()) + data
    dummy_checksum = 1 & 0xffff
    dummy_id = 1 & 0xFFFF
    # Header is type (8), code (8), checksum (16), id (16), sequence (16)
    header = struct.pack("bbHHh", icmp_type, 0, socket.htons(dummy_checksum), dummy_id, 1)
    packet = header + data
    my_socket.sendto(packet, (dest_addr, 1))

if __name__ == '__main__':
    args = get_arguments()
    icmp = socket.getprotobyname("icmp")
    my_socket = socket.socket(socket.AF_INET, socket.SOCK_RAW, icmp)
    if args.send_request:
        ping(my_socket, args.send_request, icmp_type=8)  # Type 8 is ECHO request.
    if args.receive:
    if args.send_reply:
        ping(my_socket, args.send_reply, icmp_type=0)  # Type 0 is ECHO reply.

You can skip reading the code, the important thing is that we can individually choose to listen for packets, send ECHO requests, or send ECHO replies:

python --help
usage: [-h] [--send-request IP_ADDRESS] [--receive]
               [--send-reply IP_ADDRESS]

optional arguments:
  -h, --help            show this help message and exit
  --send-request IP_ADDRESS
                        IP address to send ECHO request. (default: None)
  --receive             Wait for a reply. (default: False)
  --send-reply IP_ADDRESS
                        IP address to send ECHO reply. (default: None)<span 				data-mce-type="bookmark" 				id="mce_SELREST_start" 				data-mce-style="overflow:hidden;line-height:0" 				style="overflow:hidden;line-height:0" 			></span>

The Experiment

SSH to each host and tell Linux to ignore ICMP traffic so I can use the script to capture it (see docstring in the script above):

sudo su -
echo 1 > /proc/sys/net/ipv4/icmp_echo_ignore_all

Normal Ping

I send a request from A to B and expect the reply from B to A to be allowed. Here’s what happened:

RememberĀ A is and B is


Ok, nothing surprising. I sent a request from A to B and started listening on A, a little later I send a reply from B to A and it was allowed. You can do this test with the normal Linux ping command (but not until you tell the kernel to stop ignoring ICMP traffic). This test just validates that my bodged Python actually works.

Reply Only

First we wait a bit. The previous test sent a request from A to B, which started a timer in the SG. Until that timer expires, reply traffic will be allowed. We need to wait for that expiration before this next test is valid.


Boom! I start listening on A, without sending a request. On B I send a reply to A but it never arrives. The Security Group didn’t allow it. This demonstrates that Security Groups are inferring the state of ICMP pings by reading their type.

Other Tests

I also tried a couple other things that I’ll leave to you to reproduce in your own lab if you want to see them.

  • Start out like the normal test. Send a request from A to B and start listening on A. Then send several replies from B to A. They’re all allowed. This shows that the SG isn’t counting to ensure it only allows one reply for each request; if it has seen just one request within the timeout it allows replies even if there are multiple.
  • Edit the script above to set the hardcoded ID to be different on A than it is on B. Then nothing works at all. I’m not actually sure what causes this. Could be that the SG is looking at more than just the type, but it could also be in the kernel or the network drivers or somewhere else I haven’t thought of. If you figure it out, message me!


I had free time over the holidays this year! Realistically, understanding this demo isn’t a priority for doing good work on AWS. I just enjoy unwrapping black boxes to see how the parts move.

Be well!


Need more than just this article? We’re available to consult.

You might also want to check out these related articles: