A penetration test answers the question written in its scope, and nothing else. If the scope says "test our web application", you will get a report on whatever the testers found most interesting in the time available, which may not be what you needed to know.
The fix is preparation. Decide the question, the rules, the access, the data handling and the definition of done before you ask for quotes.
Start with the question, not the target
A list of IP addresses and URLs is a target list, not an objective. The objective is the decision the test should support. NIST SP 800-115 says technical security tests serve several purposes, such as finding vulnerabilities in a system or network and verifying compliance with a policy or other requirements. Those are different jobs and they need different tests.
Write the objective as a sentence a non-technical sponsor can read. Some examples:
- Can a logged-in customer read another customer's invoices in the new portal?
- Can an attacker with no account reach the admin interface of the booking system?
- Does the application meet the access control requirements our customer's security questionnaire quotes?
Each of these implies a different scope, different accounts and a different report. If you cannot write one sentence, you are not ready to buy yet. A short scoping call with the testers or an advisor, such as our consulting team, is worth having before you ask for quotes.
Pick a reference for "tested"
"Fully tested" means nothing until you name what it was tested against. For web applications, two OWASP projects give you a shared vocabulary.
The OWASP Application Security Verification Standard (ASVS) is a basis for testing web application technical security controls, and it gives developers a list of requirements for secure development. The project page lists 5.0.0 as the latest stable version. Each requirement has a numbered identifier, and the project recommends citing it with the version, for example v5.0.0-1.2.5. The version matters because identifiers can change between releases.
The OWASP Web Security Testing Guide describes how to test. Its page lists v4.2 as the current release, with 5.0 still in development. Name the version you expect in the contract, otherwise each side assumes a different one.
Then state which parts of the ASVS are in scope, by chapter and section as they appear in the version you chose. The identifier v5.0.0-1.2.5, for instance, points to the 5th requirement in the "Injection Prevention" section of the "Encoding and Sanitization" chapter. Pick the sections that match your objective, and ask for findings mapped to requirement identifiers. That makes retesting and developer follow-up much simpler.
Rules of engagement
The rules of engagement are the written limits on what the testers may do. These are our recommendations rather than anything the sources prescribe, but every scope document should settle them:
| Item | Decide |
|---|---|
| Environment | Production, staging, or both. Staging is safer but only useful if it matches production configuration. |
| Window | Dates and hours, and whether testing may happen outside business hours. |
| Source addresses | The IPs the testers will use, so your monitoring team can tell them from a real attacker. |
| Excluded techniques | Denial of service, social engineering and physical access are common exclusions. Write them down either way. |
| Stop conditions | What makes the testers pause: an outage, access to unexpected personal data, a critical finding. |
| Contacts | Two named people on each side, with phone numbers that work at night. |
Decide beforehand whether your SOC will be told about the test. Telling them tests the process, while keeping it quiet tests detection. Both are valid, but you should choose deliberately.
Test accounts and data handling
Provide accounts, because a tester without a login cannot examine what sits behind it. Give one for each role the application has, plus two accounts in the same role belonging to different customers or tenants. Without the pair, nobody can check whether one user can reach another's data.
Create the accounts in advance, test that they log in, and say how multi-factor authentication will be handled.
Agree on data handling in the contract:
- Use synthetic data in staging wherever possible. If production is in scope, say which real records the testers may view and which they must not.
- State where evidence such as screenshots, request logs and extracted data will be stored, who can access it, and how long the testers keep it.
- Require the report itself to travel encrypted. It is a list of your weaknesses.
- Set the rule for sensitive finds: if testers hit personal data, they stop, record the location without copying the content, and call your contact.
Success criteria and reporting
"No critical findings" is not a success criterion. It rewards a shallow test. Define done as coverage plus quality of output: the agreed requirements were examined, and every finding comes with enough detail to reproduce and fix it.
NIST describes its guide as helping with analysing findings and developing mitigation strategies, as well as running the tests. Ask for the same. Each finding should include reproduction steps, the affected component, a severity rating with a stated method so you can check it yourself, and a concrete remediation. Add a retest of fixed findings to the quote, and agree who accepts the report and when.
If you buy from a provider like our penetration testing service, put these points in the statement of work, not only in the kick-off call.
Where to start
- Write the objective as one sentence and get the business owner to agree to it.
- List the applications, environments and roles in scope, and name what is explicitly out of scope.
- Choose the reference: the ASVS chapters and sections in scope and a Web Security Testing Guide version, all written with version numbers.
- Fill in the rules-of-engagement table, including contacts, window and source addresses.
- Create and verify test accounts, including two accounts in the same role for different tenants, and set the data handling rules.
- Write the success criteria into the contract: finding format, severity method, mapping to requirement identifiers and a retest.
