Test and evaluate your WAF before hackers
2024-3-11 21:57:52 Author: lab.wallarm.com(查看原文) 阅读量:17 收藏

Since 1991, Web Application Firewall, commonly referred to as WAF, has become one of the most common application security technologies available on the market. Since the last century, WAFs have evolved by incorporating the cloud and using Machine Learning instead of RegExp.

Currently, few technologies, such as NG-WAF, RASP, WAAP, and a few others, have internal WAF capabilities, which prevent web applications and API threats.

Majority of the fintech, health tech, and e-commerce companies have had WAFs installed for years to protect their APIs, but also due to PCI DSS, SOC2, and HIPAA compliance requirements, bot mitigation, and OWASP Top-10 attack prevention needs.

How good is my WAF?

WAF-like technologies have already been in place for a while, but how good are they? Given the notorious issue of false positives that WAFs have always been known for, people often focus on evaluating false positive rates while ignoring the testing of false negative rates. In addition, it is not easy to test and check the actual level of WAF or RASP protection. 

So how can we really test how good your current WAF is? What attacks can it really stop and where can application and API attacks still hit even with a WAF in place? Where are the blind spots? There isn’t a simple tool that any developer, QA, or security engineer can run to get a PDF report on your WAF coverage.

Meet GoTestWAF

To address the issue, we are open-sourcing a project called GoTestWAF

GoTestWAF generates requests with predefined, basic payloads as well as attacks specific to different APIs (REST, SOAP, XMLRPC, gRPC). Afterwards, it sends them to the application and analyzes the responses to generate a detailed report in the console output or as a PDF.

The usage is pretty simple. It only requires that you run this tool against your WAF protected application.

docker run --rm -it -v "$(pwd)/reports:/app/reports" wallarm/gotestwaf --url=http://the-waf-you-wanna-test/

It gives clear results and indicates which of the attacks are detected with your existing appsec solution and what ways attackers can still hit your apps:

INFO[0000] GoTestWAF started                             version=v0.4.14
INFO[0000] Test cases loading started                   
INFO[0000] Test cases loading finished                  
INFO[0000] Test cases fingerprint                        fp=23c3ae919db5e6edcb62815de1a09fdf
INFO[0000] Try to identify WAF solution                 
INFO[0000] WAF was not identified                       
INFO[0000] WAF pre-check                                 url="http://host.docker.internal:8080"
INFO[0000] WAF pre-check                                 blocked=true code=403 status=done
INFO[0000] WebSocket pre-check                           status=started url="ws://host.docker.internal:8080"
INFO[0000] WebSocket pre-check                           connection="not available" error="websocket: bad handshake" status=done
INFO[0000] gRPC pre-check                                status=started
INFO[0000] gRPC pre-check                                connection="not available" status=done
INFO[0000] Scanning started                              url="http://host.docker.internal:8080"
INFO[0012] Scanning finished                             duration=12.856448447s                                                                                                                          
True-Positive Tests:
+-----------------------+-------------------------+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+
|       TEST SET        |        TEST CASE        |     PERCENTAGE, %     |        BLOCKED        |       BYPASSED        |      UNRESOLVED       |         SENT          |        FAILED         |
+-----------------------+-------------------------+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+
| community             | community-128kb-rce     |                  0.00 |                     0 |                     0 |                     1 |                     1 |                     0 |
| community             | community-128kb-sqli    |                  0.00 |                     0 |                     0 |                     1 |                     1 |                     0 |
| community             | community-128kb-xss     |                  0.00 |                     0 |                     0 |                     1 |                     1 |                     0 |
| community             | community-16kb-rce      |                100.00 |                     1 |                     0 |                     0 |                     1 |                     0 |
| community             | community-16kb-sqli     |                100.00 |                     1 |                     0 |                     0 |                     1 |                     0 |
| community             | community-16kb-xss      |                100.00 |                     1 |                     0 |                     0 |                     1 |                     0 |
| community             | community-32kb-rce      |                100.00 |                     1 |                     0 |                     0 |                     1 |                     0 |
| community             | community-32kb-sqli     |                100.00 |                     1 |                     0 |                     0 |                     1 |                     0 |
| community             | community-32kb-xss      |                100.00 |                     1 |                     0 |                     0 |                     1 |                     0 |
| community             | community-64kb-rce      |                100.00 |                     1 |                     0 |                     0 |                     1 |                     0 |
| community             | community-64kb-sqli     |                100.00 |                     1 |                     0 |                     0 |                     1 |                     0 |
| community             | community-64kb-xss      |                100.00 |                     1 |                     0 |                     0 |                     1 |                     0 |
| community             | community-8kb-rce       |                100.00 |                     1 |                     0 |                     0 |                     1 |                     0 |
| community             | community-8kb-sqli      |                100.00 |                     1 |                     0 |                     0 |                     1 |                     0 |
| community             | community-8kb-xss       |                100.00 |                     1 |                     0 |                     0 |                     1 |                     0 |
| community             | community-lfi           |                100.00 |                     8 |                     0 |                     0 |                     8 |                     0 |
| community             | community-lfi-multipart |                  0.00 |                     0 |                     0 |                     9 |                     9 |                     0 |
| community             | community-rce           |                 83.33 |                    10 |                     2 |                     0 |                    12 |                     0 |
| community             | community-sqli          |                100.00 |                    32 |                     0 |                     0 |                    32 |                     0 |
| community             | community-user-agent    |                 70.00 |                     7 |                     3 |                     0 |                    10 |                     0 |
| community             | community-xss           |                 95.80 |                   502 |                    22 |                     0 |                   524 |                     0 |
| community             | community-xxe           |                  0.00 |                     0 |                     2 |                     0 |                     2 |                     0 |
| owasp                 | crlf                    |                 77.78 |                     7 |                     2 |                     0 |                     9 |                     0 |
| owasp                 | ldap-injection          |                  3.13 |                     2 |                    62 |                     0 |                    64 |                     0 |
| owasp                 | mail-injection          |                 12.50 |                     3 |                    21 |                     0 |                    24 |                     0 |
| owasp                 | nosql-injection         |                  0.00 |                     0 |                    70 |                     0 |                    70 |                     0 |
| owasp                 | path-traversal          |                 24.77 |                    27 |                    82 |                     1 |                   110 |                     0 |
| owasp                 | rce                     |                 33.33 |                    22 |                    44 |                     0 |                    66 |                     0 |
| owasp                 | rce-urlparam            |                 33.33 |                     3 |                     6 |                     0 |                     9 |                     0 |
| owasp                 | shell-injection         |                 27.08 |                    13 |                    35 |                     0 |                    48 |                     0 |
| owasp                 | sql-injection           |                 24.36 |                    38 |                   118 |                     0 |                   156 |                     0 |
| owasp                 | ss-include              |                 37.50 |                    15 |                    25 |                     0 |                    40 |                     0 |
| owasp                 | sst-injection           |                 18.75 |                    12 |                    52 |                     0 |                    64 |                     0 |
| owasp                 | xml-injection           |                  0.00 |                     0 |                    12 |                     1 |                    13 |                     0 |
| owasp                 | xss-scripting           |                 33.20 |                   167 |                   336 |                     1 |                   504 |                     0 |
| owasp-api             | graphql                 |                  0.00 |                     0 |                     6 |                     0 |                     6 |                     0 |
| owasp-api             | graphql-post            |                 50.00 |                     2 |                     2 |                     0 |                     4 |                     0 |
| owasp-api             | grpc                    |                  0.00 |                     0 |                     0 |                     0 |                     0 |                     0 |
| owasp-api             | non-crud                |                100.00 |                     2 |                     0 |                     0 |                     2 |                     0 |
| owasp-api             | rest                    |                 23.08 |                     3 |                    10 |                     0 |                    13 |                     0 |
| owasp-api             | soap                    |                 23.08 |                     3 |                    10 |                     0 |                    13 |                     0 |
+-----------------------+-------------------------+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+
|         DATE:         |      PROJECT NAME:      | TRUE-POSITIVE SCORE:  |  BLOCKED (RESOLVED):  | BYPASSED (RESOLVED):  |  UNRESOLVED (SENT):   |      TOTAL SENT:      |    FAILED (TOTAL):    |
|      2024-03-04       |         GENERIC         |        49.12%         |   890/1812 (49.12%)   |   922/1812 (50.88%)   |    15/1827 (0.82%)    |         1827          |    0/1827 (0.00%)     |
+-----------------------+-------------------------+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+
True-Negative Tests:
+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+
|       TEST SET        |       TEST CASE       |     PERCENTAGE, %     |        BLOCKED        |       BYPASSED        |      UNRESOLVED       |         SENT          |        FAILED         |
+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+
| false-pos             | texts                 |                 85.65 |                    31 |                   185 |                     0 |                   216 |                     0 |
+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+
|         DATE:         |     PROJECT NAME:     | TRUE-NEGATIVE SCORE:  |  BLOCKED (RESOLVED):  | BYPASSED (RESOLVED):  |  UNRESOLVED (SENT):   |      TOTAL SENT:      |    FAILED (TOTAL):    |
|      2024-03-04       |        GENERIC        |        85.65%         |    31/216 (14.35%)    |   185/216 (85.65%)    |     0/216 (0.00%)     |          216          |     0/216 (0.00%)     |
+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+
Summary:
+-----------------------------+-----------------------------+-----------------------------+-----------------------------+
|            TYPE             | TRUE-POSITIVE TESTS BLOCKED | TRUE-NEGATIVE TESTS PASSED  |           AVERAGE           |
+-----------------------------+-----------------------------+-----------------------------+-----------------------------+
| API Security                | 26.32%                      | n/a                         | 26.32%                      |
| Application Security        | 49.61%                      | 85.65%                      | 67.63%                      |
+-----------------------------+-----------------------------+-----------------------------+-----------------------------+
|                                                                        SCORE            |           46.97%            |
+-----------------------------+-----------------------------+-----------------------------+-----------------------------+
Do you want to include payload details to the report? ([y/N]): y
INFO[0022] Export full report                            filename=reports/waf-evaluation-report-2024-March-04-15-56-05.pdf
Email to send the report (ENTER to skip): 
INFO[0024] Skip report sending to email
code gtw.txt
Displaying code gtw.txt.

In conclusion, the GoTestWAF is a tool to test WAFs, RASPs, and WAAP for application and API attacks, not just CGI payloads from 90th. 

How it works?

The main idea is to encode and place attack payloads in different parts of an HTTP request: its body, headers, URL parameters, etc. To simplify things, we decided to implement the following logic:

Payload -> Encoder -> Placeholder

It means that every payload sample (malicious attack sample such as an XSS string like “<script>alert(1)</script>”) will be first encoded in some way then placed into an HTTP request. There is also an option to use a plain encoder that keeps the string as-is.

To make tests readable, we introduced a YAML DSL with an equal structure (payload->encoder->placeholder), where all the fields are arrays and make a test by permutations between them. 

As a result of the permutation of each 4 payloads, 2 encoders, and 4 placeholders, this test will send 4*2*4=32 requests.

Note: be careful with the YAML. If you need to send a few binary payloads, then it is best to use !!binary attribute on the payload fields.

How to run

There are 3 options to run GoTestWAF:
1) using the official docker image - https://hub.docker.com/r/wallarm/gotestwaf
2) independently according to the instructions:

There is an Alpine-based Docker image available. It is ready to start out-of-the-box with a single command-line right after “git clone” and “docker build”:

If you need the report, you are required to map it from your host machine via -v docker attribute. Otherwise, all the stats will be available in your console only.

If you need the opportunity to answer a couple of questions from GTW (do we want to insert payloads into the report, do we want to send the report by email), use the -it attribute

$ git clone https://github.com/wallarm/gotestwaf
$ cd gotestwaf
$ make gotestwaf
$ docker run --rm -it -v "$(pwd)/reports:/app/reports" gotestwaf --url=http://the-waf-you-wanna-test/

If you need to configure some more parameters, here are the command-line arguments and config.yaml: 

Options:
      --addDebugHeader          Add header with a hash of the test information in each request
      --addHeader string        An HTTP header to add to requests
      --blockConnReset          If true, connection resets will be considered as block
      --blockRegex string       Regex to detect a blocking page with the same HTTP response status code as a not blocked request
      --blockStatusCodes ints   HTTP status code that WAF uses while blocking requests (default [403])
      --configPath string       Path to the config file (default "config.yaml")
      --email string            E-mail to which the report will be sent
      --followCookies           If true, use cookies sent by the server. May work only with --maxIdleConns=1
      --grpcPort uint16         gRPC port to check
      --idleConnTimeout int     The maximum amount of time a keep-alive connection will live (default 2)
      --ignoreUnresolved        If true, unresolved test cases will be considered as bypassed (affect score and results)
      --includePayloads         If true, payloads will be included in HTML/PDF report
      --logFormat string        Set logging format: text, json (default "text")
      --logLevel string         Logging level: panic, fatal, error, warn, info, debug, trace (default "info")
      --maxIdleConns int        The maximum number of keep-alive connections (default 2)
      --maxRedirects int        The maximum number of handling redirects (default 50)
      --noEmailReport           Save report locally
      --nonBlockedAsPassed      If true, count requests that weren't blocked as passed. If false, requests that don't satisfy to PassStatusCodes/PassRegExp as blocked
      --openapiFile string      Path to openAPI file
      --passRegex string        Regex to a detect normal (not blocked) web page with the same HTTP status code as a blocked request
      --passStatusCodes ints    HTTP response status code that WAF uses while passing requests (default [200,404])
      --proxy string            Proxy URL to use
      --quiet                   If true, disable verbose logging
      --randomDelay int         Random delay in ms in addition to the delay between requests (default 400)
      --renewSession            Renew cookies before each test. Should be used with --followCookies flag
      --reportFormat string     Export report to one of the following formats: none, pdf, html, json (default "pdf")
      --reportName string       Report file name. Supports `time' package template format (default "waf-evaluation-report-2006-January-02-15-04-05")
      --reportPath string       A directory to store reports (default "reports")
      --sendDelay int           Delay in ms between requests (default 400)
      --skipWAFBlockCheck       If true, WAF detection tests will be skipped
      --skipWAFIdentification   Skip WAF identification
      --testCase string         If set then only this test case will be run
      --testCasesPath string    Path to a folder with test cases (default "testcases")
      --testSet string          If set then only this test set's cases will be run
      --tlsVerify               If true, the received TLS certificate will be verified
      --url string              URL to check
      --version                 Show GoTestWAF version and exit
      --wafName string          Name of the WAF product (default "generic")
      --workers int             The number of workers to scan (default 5)
      --wsURL string            WebSocket URL to check

Command-line flags have priority over configuration files. As you can see, the most important thing here is how to detect a blocking page. Currently, you can do it with a response status or RegExp that will be applied to the request body. By default, the 403 response status code will be used for this.

 You can also choose test cases between two embedded: OWASP Top-10, OWASP-API, or your own. 

3) Using Free Online WAF Tester Tool - https://www.wallarm.com/gotestwaf/overview

The report sample

Again, the main goal of this tool is to easily generate readable reports to check the current state of protection according to OWASP guidelines. 

To see what it looks like, we can install mod_security WAF and run it on the 8080 port:

Next, we can launch the GoTestWAF utility:

docker run -v /tmp:/tmp/report gotestwaf --url=http://the-waf-you-wanna-test/

The results will be printed in your console as such:

owasp   path-traversal   6/18  (0.33)
owasp   sql-injection    4/16  (0.25)
owasp   ss-include       2/8   (0.25)
owasp   xml-injection    6/8   (0.75)
owasp   xss-scripting    4/12  (0.33)
owasp   ldap-injection   0/8   (0.00)
owasp   mail-injection   3/12  (0.25)
owasp   nosql-injection  0/18  (0.00)
owasp   shell-injection  3/8   (0.38)
owasp   sst-injection    5/20  (0.25)

The PDF version will also contain bupasses with details, screenshot below:

We updated the 2023-2024 GoTestWAF tool with the following features:

  1. Email Report Sending: GoTestWAF interactively prompts for an email address to send the report. If the desired behavior is not selected through options, it will ask for an email. The email can be set using the options --email or --noEmailReport.
  2. OpenAPI Specification Testing: Testing can be performed using the OpenAPI specification (specified through the --openapiFile option, detailed in the readme).
  3. Debug Header Addition: Ability to add a debug header (activated through the --addDebugHeader option). GoTestWAF will transmit payload information hash in the X-GoTestWAF-Test header, enabling the unique identification of the payload-encoder-placeholder combination during traffic analysis.
  4. gRPC Support: Added support for gRPC (via the --grpcPort flag). GoTestWAF sends vectors to this port where gRPC is specified as a placeholder in the test case.
  5. New Placeholders: Numerous new placeholders have been added, listed in the readme's "How It Works" section. The RawRequest placeholder is described, allowing precise request creation.
  6. Multiple Status Codes: Multiple status codes can be specified (options --blockStatusCodes, --passStatusCodes), enabling the inclusion of an array of status codes instead of just one. This is useful when applications react differently to different status codes
  7. Regular Expressions for WAF Response Analysis: Regular expressions (--blockRegex and --passRegex) can be used to analyze WAF responses.
  8. Automatic Vendor Identification: With session handling improvements, GoTestWAF can automatically identify multiple (2 or 3) vendors and include them in the report. This feature works by sending a test vector and observing the response. It operates by default, but can be disabled with --skipWAFIdentification
  9. Report Changes: The report format has changed, and now there's an option to receive a full report with all bypassed vectors (--includePayloads option). Alternatively, a short version with graphs and statistical data is available by default.
  10. JSON Report and Output: JSON reports and output are introduced to integrate GoTestWAF into pipelines. Log output format during execution is controlled by the --logFormat option (text, json), and the report format is set through --reportFormat (default is pdf, with other options available like none, html, json).
  11. Specific Test Set and Test Case: The ability to specify a particular test set (--testSet) or test case (--testCase) to narrow down the vectors during testing.
  12. Cookie Handling: GoTestWAF can consider cookies during scanning (--followCookies) and update the session before each request (--renewSession). This allows scanning hosts that require specific WAF-specific cookies, as otherwise, requests are blocked.
  13. OWASP Core Rule Set Testing: A script is added to generate test sets from the OWASP Core Rule Set regression testing suite. These vectors are not available by default and require additional steps as outlined in the readme.

The new report looks like this:

report screenshot

Graphs from the new report:

For the technical details about payload, encoders, and responses, you can check the CSV file that will be generated in the same folder as PDF report.

The new report has a comparison with other solutions:

Future improvements

We invite everyone to join our mission and commit to the project, so that we can collectively improve the task at hand. The current roadmap includes support of WebSockets and .NET specific attack improvements, as well as some bots-emulations and other new ideas recommended by current users. Any pull requests and improvement suggestions are greatly appreciated.

Frequently Asked Questions

How do you test WAF rules?
To test WAF rules, you can use penetration testing tools such as Burp Suite, OWASP ZAP, or Nmap to simulate attacks on your website. These tools can help identify vulnerabilities and ensure that WAF rules are functioning correctly.

How do I check my WAF application?
One way to check your WAF application is to perform regular security audits and vulnerability assessments. This can help identify any potential WAF issues that need to be addressed.

How do you audit a WAF?
Auditing a WAF involves evaluating the configuration, log files, and network traffic to ensure that the system is functioning correctly. Additionally, it is important to review attack detection and mitigation policies to determine their effectiveness.

What is WAF testing?
WAF testing is a process of evaluating the effectiveness of a web application firewall (WAF) against potential web-based attacks. This involves testing the WAF configuration, monitoring system logs, and assessing the response to known vulnerabilities.

Explore more:

What is WAAP?

Wallarm Cloud-Native WAAP - Product


文章来源: https://lab.wallarm.com/test-your-waf-before-hackers/
如有侵权请联系:admin#unsafe.sh