Web application firewalls (WAF) are powerful tools that improve the security of web assets. But is a web application firewall enough? Should WAFs be treated as the primary security measure?
In this white paper, we take a look at how web application firewalls work and how they should be used to get the most out of their capabilities without hindering your web security.
The web application firewall is your first line of defense.
When someone connects to your website or web application, it is the web application firewall that actually receives that request. It analyzes that request for possible malicious content and, if deemed safe, forwards it to your website or web application to process.
If a web application firewall is to be effective, this process must involve every single request with no exceptions. If even one request is able to bypass the WAF, it would create a potential opening for a malicious hacker to exploit.
The way that web application firewalls work introduces certain problems. Some of them are inherent and cannot be resolved but many of them can be minimalized or even neutralized if the web application firewall is used as part of the security infrastructure and not treated as the primary solution for web application security.
To check if a request is safe or not, the web application firewall must check that request against a set of rules. Such rules are usually very complex patterns and often involve regular expressions. If the request matches a pattern that is defined as malicious, the request is rejected. In most cases, the WAF is configured to send the client a response informing that the request was rejected due to the rule but it is also possible to configure a WAF so that such requests are simply deleted.
WAF rules are usually added from four sources:
Note that if your business hosts web assets on large clouds or uses third-party services such as Cloudflare traffic optimization, the web application firewall may be part of such clouds or services and your organization may not even be in control of the rules at all. In such a case, you may have no choice but to trust a third-party company to manage the WAF efficiently.
No matter the source of WAF rules, there is always a chance that these rules are wrong. Consequences may manifest in one of two types:
All in all, for a WAF that is used as the primary source of web application defense, both false negatives and false positives are very unwelcome. Unfortunately, they always happen due to human error.
In the worst-case scenario, a wrong rule may even temporarily bring down the entire web application. For example, in July 2019 Cloudflare experienced a worldwide outage of their systems, which was caused by WAF misconfiguration. A wrongly formulated regular expression made it to production systems and, as a consequence, caused a massive internal DoS.
Another issue associated with the way that WAFs work is their efficiency. A single request may need to be compared to hundreds or thousands of rules, one by one. Since this applies to every request, it means that every small piece of information that the client browser fetches from your website, for example, a script, CSS, or an image file, must go through hundreds or thousands of checks before it is forwarded to the web server.
In the case of very popular or very intensely used websites and web applications, this means that the web application firewall may actually require more processing resources than the web application itself. Therefore, if the WAF is not a managed service, it may require major investment and upkeep, especially when traffic starts growing.
The term security through obscurity was coined to describe situations where developers or administrators attempt to secure the resource by making it hard to find. For example, some developers may think that putting the admin interface in an obscure URL such as http://example.com/g7nebg7jmf/noej9bvke/bg7wvf98q.php will make it unnecessary to protect the interface with a password. This is a very wrong approach that causes a lot of problems.
Unfortunately, there are some similarities between this approach and the approach that the WAF takes. The WAF does not address the problem at all. The WAF does not even know whether your application is vulnerable or not. The WAF only examines the data sent for something that it considers malicious and then blocks such attempts. Obviously, if for any reason the WAF fails, the application is then wide open – just like an obscure URL once it is found by the malicious hacker.
Another problem with WAFs related to the one described above is the fact that if you only use a WAF to handle your web application security, you might actually not know your web application security state at all. The WAF does not give you any information about whether your application is safe or not, it does not show you where its weak spots are, and it does not show whether it can be attacked at all. You might have a perfectly safe application that does not require a WAF at all or you might have an application riddled with holes, which would fall in seconds if not for the WAF – and you don’t know which is the case.
This approach also promotes bad security habits. For example, not updating third-party applications, libraries, or server software due to the reliance on the WAF. Some managers may think that since the server is protected by a WAF, there is no need to invest a lot of time and resources to keep it updated.
If you perceive a WAF as the key aspect of web application security, you most likely are not going to invest any time and effort in vulnerability prevention. After all, it makes no sense with such an approach. The WAF is there to eliminate attacks so there is no need to eliminate vulnerabilities.
The consequence of such an approach is that developers are usually not trained in secure coding and may introduce vulnerabilities into your web applications. And as we mentioned earlier, no WAF is perfect and malicious hackers may find a way around some of the rules. If they find a way around a WAF and there is no vulnerability, there is no problem – the attack attempt is futile. But if they find a way around a WAF and there is an underlying vulnerability, the application is left wide open.
The culture of securing web applications by preventing attacks is unhealthy. Web attacks should be prevented at the source, by eliminating vulnerabilities that make such attacks possible. The WAF culture comes from the network security world, where the owner of vulnerable network software or devices would have few choices because network software and devices are not developed in-house. This culture must not be applied to web applications.
To make matters worse, WAFs can never be used as the only means of preventing web attacks simply because in some cases, they are completely ineffective.
As an example, we can take DOM-based cross-site scripting (DOM XSS). DOM XSS payloads often never reach the server, and attacks happen completely client-side. Since a WAF works only on requests that are sent from the browser to the server, the payload will be completely missed. There is no way whatsoever to protect your web application against DOM XSS using a WAF.
If you’re relying on a WAF as the primary means to ensure the safety of your web applications, you can never let go of that WAF. You are stuck with a WAF permanently.
This is simply because if you were ever to turn off your WAF (or if it would stop working for any reason), your application would be completely exposed to attacks that you know nothing about because you did not evaluate your attack surface and you do not know any of your vulnerabilities.
Web application firewalls are applications, just like the applications that they are protecting. It has been discovered in the past that WAFs can have the same vulnerabilities that they have been made to protect against.
If the vulnerability is in your own application, you can always fix it and eliminate the danger. However, if the vulnerability is in the web application firewall, which is provided by a third party, the only thing that you can do is wait for an official update or change the third party that supplies your firewall, which is often a lengthy and costly process. And you have no guarantee that your newly chosen WAF will be 100% secure, either.
WAFs can also be bypassed even if they have no vulnerabilities. This is because their default rules often do not filter out potentially malicious requests – instead, they try to disable the malicious payload by replacing some characters in that payload. If a malicious hacker knows that this is the case, they can trick the WAF into enabling the payload. You can even find cheat sheets that describe how to detect a WAF (based on the way they behave when they encounter a malicious request) and how to bypass them.
Most of the WAF problems mentioned above are caused by the wrong approach to using a WAF. The concept of WAFs and the products available on the market are excellent. The important thing is to know how to use them properly.
In the next part of this white paper, we would like to propose to you a setup that we believe is the best way to use the potential of the WAF, greatly reducing or eliminating all of the problems mentioned above.
If a WAF is treated as part of a defense-in-depth web application security suite and used as the final line of defense, not the main line of defense, most of the disadvantages of using a WAF are gone.
We believe that the best practice workflow for web application security is as follows:
With the above approach, the WAF is used primarily to prevent known vulnerabilities on a temporary basis.
Steps one and two of the above workflow may be performed using a web vulnerability scanner or a web vulnerability scanner paired with additional penetration testing. Performing these steps by manual penetration testing would simply require too much time and resources.
The above best practice process would, in many cases, require a dedicated security resource who would analyze selected vulnerabilities and develop relevant WAF rules to detect and eliminate potential payloads. Unfortunately, that might also make the process inefficient because usually, the number of vulnerabilities that can be fixed immediately is much lower than the number of vulnerabilities that must temporarily be left in the backlog.
To make the process efficient, the vulnerability management software must be able to convert a discovered vulnerability to a WAF rule. This is only possible if the vulnerability management software is integrated with a vulnerability scanner – the WAF rule must be based on the attack payloads.
This is where one of the many advantages of Acunetix comes in.
Acunetix can automatically export the payload used to detect a vulnerability as a WAF rule. Currently, it supports the following WAFs:
Acunetix is also continuously working on supporting more WAF export formats.
One client uses the Acunetix WAF export features together with FortiWeb and the API interfaces of both products to completely automate the process.
The client’s setup includes a dedicated staging/testing environment that contains the production environment versions of their web applications. This environment was created to avoid production resource utilization.
The client has a nightly cron job configured that runs a Python script. This script performs the following functions:
There is also a weekend cron job that performs the same actions but runs a complete scan of all applications instead. While all this could be done manually using the Acunetix interface (including the daily/weekly schedules – using the continuous scanning feature), the client needed the actions fully automated and wanted to use the Acunetix API to its full extent.
Thanks to this setup, for every newly discovered vulnerability a WAF rule is created automatically after the scan with no human intervention at all, along with an issue automatically generated in the internal issue management system. This way, the customer is protected immediately, and the issue is tracked to be fixed at the source.
Most of the problems mentioned in the first part of this white paper are either greatly reduced in scope or completely eliminated if WAF rules are based on web vulnerability scanner exports:
When a WAF is no longer the center of your web application security environment, and that center spot is taken by web vulnerability management software instead, you can relax and adjust your WAF to improve your security even more while not worrying about the consequences.
For example, if you know that you have all the discovered vulnerabilities covered, you may decide to use certain generic WAF rule lists to potentially cover any false negatives – vulnerabilities that might not have been detected by the vulnerability scanner. While it is very unlikely that such vulnerabilities will exist and be exploited, it provides an extra layer of protection.
You can also extend this web security setup with even more tools. You can, for example, include vulnerability scanning in your SDLC and complement it with solutions such as SAST or SCA. As such, you are able to build a complete web application security environment that will let you be sure that all of your web applications are completely safe.
"Acunetix is our vulnerability scanning tool of choice for situations where information security is a real concern and confidence in safety is key."Read case study
"VITA’s Web Application Vulnerability Scanning Program, implemented in 2016, uses Acunetix to check more than 1,600 public-facing web applications and another couple thousand internal ones at more than 67 state agencies every quarter."Read case study
"We tried several web vulnerability scanning solutions and only Acunetix met our expectations. With it, we were able to identify security vulnerabilities in our flagship product."Read case study