In a previous post, we talked a bit about what pen testing is and how to use the organizations that provide them to your benefit. But, what about when one of them hands a client a failing grade?

Consider this, you’re an MSP and you get a letter or email from one of your customers that reads:

“Dear ACME MSP,

We regret to inform you that you’ve had a Penetration Test Failure produced by: “FreindlyHacker-Pentesting Inc” and we’d like to discuss the details further to determine if you have what it takes to continue to handle our security needs.

Regards,

Largest MSP Customer.”

A customer may not pass along this exact wording, but the implications are clear. The results can be embarrassing or at worst devastating. When a customer reaches out after failing penetration testing, it can put an MSP on its heels and create unnecessary angst. Should the MSP have been more involved in the testing? Did my tools cause the failure Has the MSP soured its relationship with its client? Will the business be lost?

So, how should an MSP respond when a customer fails a pen test?

Some MSPs turn to self-doubt and start wondering if the layers of protection they’ve put in place are worth the costs. Others will immediately start pointing fingers at the tools that were identified in the pen test report. When a report comes through with a failure, it’s usually unexpected and can take time away from more important activities.

To save time and effort if this should happen to you, here are a few key elements of a good response to a pen test failure.

Immediately start asking questions.

• What kind of penetration testing was involved?
• Who performed the testing and what are their credentials?
• How was the penetration testing organization positioned to start taking action?
• Where the testers acting as “Red Team” or “Blue Team” actors?
• When did the testing take place?
• May I examine the data and reporting?

Review your tools configurations.

Rather than immediately assume bad tech, it’s best to step back and evaluate each tool identified in the pen test report and the associated configurations, policies and control points. Often, a security tool is designed to identify, evaluate and/or stop bad actors along the threat chain. If it failed, it could be that a setting was disabled or miss-configured. Review all tools’ “best practice” guides, documents and suggestions before making assumptions.

Ask for partnership with the customer during their next review.

If the customer did not provide a heads up or pretesting communication, request that you be more involved during their next review. If pen testing is important enough for them to do once, it’s probably that they’ll do it bi-annually or annually, depending on the industry and regulatory concerns. It’s always good to be involved in advanced than after the fact.

Blue Teams vs. Red Teams: Which type of test was conducted?

The difference between a Blue Team and Red Team is how much previous access they have to a target’s networks and devices. This can make a huge difference in how the results of a pen test are interpreted. When a Blue Team—with some previous knowledge of an organization and its IT systems—is able to breach a business, it may not be representative of real-world circumstance. It could be an internal IT admin who was able to find a vulnerability after poking around in a system she previously had access to.

When a Red Team compromises a client, on the other hand, it’s time to examine the reporting closely. Starting with zero knowledge of an organization’s systems, this type of breach could point to serious flaws in the defenses an MSP has set up for a client. Likely there are real holes here which need to be patched.

Evaluate the pen testing organizations

While there are many levels of testing capability, keep in mind that pen testers come from many IT walks of life. Former sysadmins, hackers and network administrators make the most common tester. They come with their own experiences, specialties and biases.

One question to always ask is, what are the testing organizations credentials? What is their background and how did they come to the business? How long have they been testing?

The goal is to guage whether the individuals who’ve conducted the test are knowledgeable enough to make judgments about your organization’s defenses? Did they actually breach the defenses or are they simply reporting on a “potential” for a breach?

Not all testers are alike, not all testing organizations are alike.  Each has to successfully make the case of its own expertise in coming to the conclusion that it has.

As I say, trust but verify. And be prepared to ask LOTS of questions if a client ever fails a pen test.