Vendor Security Assessment: Evidence, Questionnaire and Red Flags

A vendor security assessment checks a supplier’s security before you trust it with your data or systems. For software and SaaS, it combines independent evidence, such as a SOC 2 report or ISO/IEC 27001 certificate, with a security questionnaire and the contract’s security terms.

What a vendor security assessment is, and when to do one

It answers one question: can this vendor protect the data and access you’re about to give it? It’s one step in vendor risk management, the program that covers a vendor from onboarding to exit. The vendor selection guide covers the rest of due diligence.

Assess a vendor:

• Before the contract, while findings can still change the deal.
• At renewal, to confirm the evidence is current and your use hasn’t grown.
• After a significant change at the vendor, such as an acquisition, a new hosting provider or sub-processor, or a new AI feature that touches your data.
• After an incident at the vendor or one of its suppliers.

Tier vendors by risk first

Match the effort to the risk. Three questions set the tier: what data the vendor will store, process or see; what access it gets to your systems, such as admin rights or API access; and how critical the service is, including how hard it is to replace.

The NIST Cybersecurity Framework (CSF) 2.0 lists “suppliers are known and prioritized by criticality” among its supply chain outcomes. As an example only:

Write the rules down and apply them to every vendor, including SaaS bought on a credit card.

The evidence to ask for

Ask for independent evidence first, use the questionnaire for the gaps, and read what you get.

SOC 2 Type II report

A SOC 2 report is an attestation report from an independent CPA firm, under AICPA standards, on a service organization’s controls relevant to one or more trust services categories: security, availability, processing integrity, confidentiality and privacy. A Type I report covers the design of controls at a point in time. A Type II report also covers whether they operated effectively throughout a period, with the auditor’s tests and results, so ask for Type II. Then check:

• Scope. The system description should cover the service you’re buying. Note which categories are included.
• Period. If it ended long ago, ask when the next report is due.
• Exceptions. Read the test results, not only the opinion, for controls that didn’t work as described, and ask what the vendor did about each.
• Subservice organizations. If the vendor relies on one, such as a cloud hosting provider, the report either includes that provider’s controls (the inclusive method) or excludes them (the carve-out method). For a carved-out provider, get evidence about it too.
• Complementary user entity controls. Controls the vendor’s design assumes you will run, such as removing access when your staff leave. Give each one an owner.

A SOC 3 report is a general-use version without the system description or the auditor’s tests and results, so it can’t answer these checks. The SOC 2 report’s use is restricted, so the vendor may ask for a nondisclosure agreement first.

ISO/IEC 27001 certificate

ISO/IEC 27001 sets requirements for an information security management system (ISMS). ISO doesn’t certify organizations. An independent certification body audits the ISMS and issues a certificate with a defined scope. Check:

• Scope. It should include the service you’re buying and the teams and locations that run it.
• Validity. Check the dates, and confirm with the certification body that the certificate is current.
• Issuer. Note which certification body issued it, and whether a national accreditation body has accredited that certification body, which demonstrates its competence.
• Statement of applicability. The standard requires one. It lists the controls the organization decided it needs, why, whether each is implemented, and why it excluded any of the reference controls in Annex A. Ask for it: excluded or unimplemented controls are ones you can’t count on.

Other documents

• Penetration test summary or letter from an independent tester: the date, what was tested, and whether serious findings were fixed and retested.
• Policies for security, incident response, business continuity and secure development, each with an owner and a recent review date.
• A completed security questionnaire: yours, or one of the standardized security questionnaires published by industry groups.
• Data processing agreement and sub-processor list. The agreement sets how the vendor processes personal data for you. The list should name the companies that process your data for the vendor, what each does and where. Check how you’ll hear about a new one.

Vendor security questionnaire: sections and sample questions

A vendor security questionnaire, also called a third-party risk assessment questionnaire, covers what the documents don’t. Say which answers need evidence.

Governance and policies

• Who is accountable for information security?
• When were your security policies last reviewed and approved?
• Do staff with access to customer data get security training and, where the law allows, background checks?

Identity and access

• Can we enforce single sign-on (SAML or OIDC) and provision users through SCIM?
• Do all your staff who can reach production systems use multi-factor authentication?
• How is privileged access granted, reviewed and removed?

Data protection and encryption

• Where are our data and backups stored, and can we choose the region?
• How is our data encrypted in transit and at rest, and who manages the keys?
• How do we export our data, and how will you confirm its deletion at exit?

Application security and vulnerability management

• How do you check your code and open-source components for vulnerabilities before release?
• What are your target times to fix vulnerabilities, by severity?
• Do you publish a vulnerability disclosure policy?

CISA’s Software Acquisition Guide, written for government buyers, has more supplier questions on how software is built and maintained.

Logging and incident response

• Which security events do you log, for how long, and can we get the logs for our account?
• How soon after confirming an incident affecting our data will you tell us, and what will you include?
• When did you last test your incident response plan?

Business continuity

• When did you last test restoring the service from backup?
• What are your recovery time and recovery point objectives, and will you put them in the contract?
• What happens if your hosting provider’s region fails?

Sub-processors and the vendor’s own suppliers

• Which sub-processors handle our data, for what, and in which countries?
• How do you assess your own critical suppliers’ security?
• How much notice will you give before adding a sub-processor?

AI features and data use, where relevant

• Which features use AI, and can our administrators switch them off?
• Are our data, prompts or outputs used to train models that serve other customers, and can we opt out?
• Which outside model providers process our data, and how long are prompts and outputs kept?

Putting security requirements in the RFP

Put a short list of security Must-haves in the RFP and send the full questionnaire to your finalists. The cybersecurity RFP guide explains how the two divide the work.

The free RFP template already has them in its security and compliance requirements, such as a current SOC 2 Type II report or ISO/IEC 27001 certificate covering the proposed service. Say which evidence you’ll need, and when. Check the evidence Must-haves pass or fail, and score the rest with your weighted criteria, as the RFP evaluation criteria guide describes. For a security product, start from one of the cybersecurity RFP templates.

Findings and follow-up

Record each gap as a finding, rate it by the harm it could do (high, medium or low, for example), and decide what happens:

• Fix before signing, for gaps that matter at the vendor’s tier.
• Fix by a date written into the contract.
• Compensating control, such as keeping one type of data out of the service.
• Accept the risk, signed by the business owner, with a review date.
• Reject the vendor when a gap can’t be fixed or offset.

Then write the vendor’s promises into the contract: incident notification within a defined time, notice of sub-processor changes, copies of new reports as they’re issued, and data return and deletion at exit.

Set how often each tier is reassessed in your policy, plus triggers: renewal, a significant change at the vendor, an incident. NIST CSF 2.0 describes supplier risks as assessed and monitored over the course of the relationship.

Red flags

Ask about each, and record the answer.

• No independent report or certificate, and no plan to get one.
• A report or certificate whose scope leaves out the service you’re buying.
• Unexplained SOC 2 exceptions, or an old report period with no newer report due.
• Refusal to name sub-processors.
• Vague incident notification, such as “promptly”, with no defined time.
• Questionnaire answers that contradict the report, or “yes” to everything with no evidence.
• No clear answer on whether your data trains models for other customers.

Start from the free RFP template or a cybersecurity RFP template, and see the vendor risk management guide for what comes after you sign.

Frequently asked questions

What is a vendor security assessment?

It’s a review of a supplier’s security before you buy, at renewal, and after a significant change or incident at the vendor. It combines independent evidence, such as a SOC 2 report, with a security questionnaire and the contract’s security terms. Its depth should match the vendor’s risk.

What should a vendor security questionnaire include?

Governance, identity and access, data protection, application security, logging and incident response, business continuity, and the vendor’s sub-processors and suppliers. Add AI features and data use where the product has them. Keep it short for low-risk vendors.

Is a SOC 2 report enough?

Not on its own. Check that its scope covers your service, that the period is recent, and what the exceptions and the controls left to you mean. CISA’s Secure by Demand guide notes that many compliance standards used in procurement focus on a vendor’s enterprise security and few on product security, so also ask how the product is built.

How often should vendors be reassessed?

Set an interval for each risk tier in your policy, with high-risk vendors reviewed most often. Also reassess at renewal, after a significant change at the vendor and after an incident.

Who performs a vendor security assessment?

The security team, or a third-party risk team if you have one, should lead it, working with procurement and the business owner who will use the service. A small team can use a short checklist for low-risk vendors and get outside help for high-risk ones.