A large workforce needs more than a bigger recipient list. Different business units, identity systems, languages, and support teams change how a phishing simulation program must operate.
Use this guide to define your requirements, test a representative pilot, and compare platforms with evidence your security and procurement teams can review.
Audience governance • Representative pilots • Useful reporting
NIST SP 800-50 Rev. 1 describes a learning program that evolves through evaluation and supports different audiences. For an enterprise, a practical starting point is to assign a central program owner and a named contact in each participating business unit.
Decide who approves scenarios, who handles reported messages, and who can see individual results. Map employees, contractors, shared mailboxes, recent acquisitions, and people on leave before importing an audience. Record exclusions and an owner for updating them; stale identity data can undermine a well-designed campaign.
Source: NIST SP 800-50 Rev. 1: Building a Cybersecurity and Privacy Learning Program
Use these as procurement checks, not assumed features of any product. Mark each requirement as demonstrated, documented, or unresolved.
| Requirement | Pilot test | Evidence to retain |
|---|---|---|
| Audience changes | Add, move, and remove a test employee across business units. | Update behavior, exclusions, and time required. |
| Delegated access | Have a regional administrator inspect only their approved scope. | Role permissions and cross-group visibility checks. |
| Email delivery | Send to approved test accounts in each actual mail environment. | Delivery outcomes and required configuration changes. |
| Languages and access | Review a scenario and lesson in each priority language and on assistive technology. | Content quality and accessibility findings. |
| Data and reporting | Export one campaign and reconcile its audience and events. | Field definitions, retention settings, and missing events. |
NIST recommends checking urgent requests through known contact details. The following original scenario ideas turn that principle into role-specific practice; use fictional content and approved exercise destinations.
A fictional supplier requests a new payment destination. The learning objective is to verify the change through the established vendor contact and approval process, without initiating a payment or handling real account details.
A simulated collaboration notice asks for access to an unfamiliar workspace. Ask employees to use the known application entry point and report a request that does not match their work.
A mock process update references a department-specific task. Review whether staff can verify the owner and use the reporting route when the message reaches a shared mailbox.
Source: NIST: Recognizing, verifying, and reporting phishing
Document required populations, supported events, report access, support coverage, and configuration limits. Include a stop condition and the person authorized to pause a campaign.
Include a central office, a regional team, and a business function with a distinct workflow. A small but varied pilot can reveal issues that a larger single-department test misses.
Send approved test messages, submit a report, and verify the handoff to the response team. Review the employee explanation and the next learning activity before broad delivery.
Approve each additional group after resolving delivery, access, and support findings. Keep a record of changes so later campaigns have a comparable operating baseline.
Retain an approved audience count, named administrators, a record of delivery exceptions, one reconciled export, and a tested report-to-response handoff. Add the time your team spent configuring, reviewing, and supporting the pilot to your cost comparison.
Compare results within similar populations and scenario difficulty. A lower click rate in a heavily filtered office cannot by itself demonstrate better awareness than another office. Explain denominators, automated link inspection, excluded accounts, and changes in scenario design before presenting a trend to leadership.
Capacity is only one requirement. Ask for evidence of audience controls, administrator permissions, delivery across your environments, data handling, and support. A representative trial is more useful than an enterprise label.
A shared introductory exercise may be useful, but finance, IT, regional offices, and contractors often need different scenarios. Keep learning objectives clear and account for those differences when interpreting results.
Bring your requirements for identity provisioning, single sign-on, delegated administration, localization, data residency, retention, and integrations to a demo. Confirm current support and contractual scope directly; this evaluation guide does not establish that every capability is available.
Summarize audience coverage, relevant employee actions, reporting and response findings, completed reinforcement, and unresolved operational issues. Include the next action and its owner rather than ranking departments by clicks alone.
Walk through your audience, administration, and reporting needs with Phishmake, then identify which requirements need further validation before a pilot.
