Enhancing security through simplicity: the dangers of complex firewall rulesets

I've worked with a number of different firewall products ranging from Cisco FWSM, PIX, and ASA devices to Netscreen 5 series SOHO firewalls to Secure Computing's Sidewinder G2. All of them have their pros and cons, but one thing that has been constant is that in a live production environment as ruleset complexity increases, configuration errors, security holes, and mistakes increase.

Very few organizations can afford the time and resources it requires to re-factor a ruleset on a major datacenter firewall device, and change management and rule requests are rarely a closed loop process that ensures that every device and rule is removed when it should be. Even ruleset builder tools can lead to their own issues, either by creating difficult to understand rulesets, or by concealing the complexity that they have created.

Where does this leave the staff members maintaining the ruleset?

They're usually aware that their ruleset is larger than it needs to be, that it is more complex than it needs to be, and that there are orphaned rules and holes that don't need to exist.

But there is often little they can do about it without either starting clean, or having organization wide commitment to a clean and simple policy. If you do have the opportunity, here are a few pointers on cleaning up your rulesets:

1. Get complete architecture diagrams, and pseudocode your rules.This provides a good overview of what your systems are expected to do, and how they communicate. Verify ports, protocols, and hosts.
2. Map hosts into functional groups. Look for common access methods, common rule requirements, and common functions. You will have some hard decisions based on functional groups, and may have to change them over time, but a reasonable starting map can make rulesets are more manageable. This is a good time to agree on naming conventions with the system administrators and firewall rule change requesters. Having pre-agreed upon names can simplify rule requests as you won't have to match groups to hosts - requests can be made in the same shorthand that you yourself will use to create the rules.
3. Write a sample pseudocode ruleset, and review with administrators. Translate requests into firewall rules, then use that and your mappings to write extensible, reasonable firewall rules. Make sure that you leave yourself enough flexibility to add and remove hosts by creating generic rules where possible. Consider trade-offs between a lax ruleset and maintenance concerns. Often, a slightly relaxed ruleset is not an unreasonable compromise given a multi-layered approach for many assets.
4. Migrate into a clean environment using your draft document, and then migrate machines. If you can migrate into a clean environment - and this is a great place to use virtual firewalls like those the FWSM and higher end Netscreen devices support - you can use only your new ruleset.
5. Require testing. Require administrators and functional users to test the environment, and monitor during that test. Verify traffic via each requested port and protocol to each requested host, or document why it will not show up. Note unused rules, and review them with the requesters.
6. Provide documentation to the areas you support. Providing a written copy of the groups and aliases that you have created will help system administrators and others understand how the firewall treats their systems and applications. Often, the best return on investment is seen here, as they can request correct firewall rule changes, and you can communicate using the same terms.
   

I've used this process in a number of instances, from small-scale group firewall policy rewrites to a clean pre-production re-write of a major ERP system which was heading into a new environment. The advantages are significant - the ERP ruleset went from over 200 rules created during pre-production and test to less than 80, and remained far more manageable over time.