Security Assessment Types: A Complete Guide for Tech Leaders
Learn the 5 key security assessment methods—from threat modeling to red teaming. Discover when to use each type in your organization's road to security for maximum ROI.
Security assessment types confuse many tech leaders. In my daily work with clients, I frequently encounter organizations struggling to choose between threat modeling, penetration testing, vulnerability assessments, and other security evaluation methods for their IT systems.
The goal of this series is to improve this state of affairs. Today, we’ll take a high-level overview of the taxonomy for security assessments, and in subsequent parts, we’ll examine each one of them in more detail.
Continue reading if you’re interested in topics like Product Security, Secure SDLC, DevSecOps, or security automation. Subscribe if you want to receive updates straight into your mailbox.
This article series includes:
Part 1: Security Assessment Types Overview (you are here)
Part 2: Threat Modeling in Practice (coming soon)
Part 3: Security Audits Explained (coming soon)
Part 4: Vulnerability Assessment Guide (coming soon)
Part 5: Penetration Testing Deep Dive (coming soon)
Part 6: Red Teaming for Organizations (coming soon)
TL;DR: Security assessments fall into 5 types: Threat Modeling (design phase), Security Audit (compliance), Vulnerability Assessment (finding issues), Penetration Testing (exploiting issues), and Red Teaming (full attack simulation). Each serves different goals on your organization’s road to security.
The Business Case for Security Assessment Taxonomy
Before diving into the details, I will take a little detour and first answer the following question: How can we differentiate between various types of security assessments?
We can do that by zooming in on certain features and using them to highlight differences. This will enable us to develop a coherent model of security assessments that can be used to integrate them into the organization.
I will return to these features, but for now, an even more important question arises: Why does it matter to have a coherent model of security assessments? It’s important because without it, implementing or maintaining effective cybersecurity program becomes harder than it should be (unless your focus is solely on compliance rather than security engineering). This is due to two main challenges.
Matching Security Assessments to Development Stages
Different types of assessments provide different insights, making them suitable for achieving different goals. So, the first challenge is determining the type of security assessment one wants to use, along with choosing when and where it should be used. This is hard when you don’t have a coherent taxonomy.
Let’s take a quick look at vulnerability assessment versus penetration testing – both can be used within the broader Software Development Lifecycle. However, the highest ROI comes from continuous vulnerability assessment executed in pre-release phase and point-in-time penetration testing performed in post-release phase.
This is closely linked to the teams that carry out the work at the ground level. While basic security testing, such as vulnerability assessment, can be conducted by non-professionals (e.g., QA/SDET), penetration testing requires subject-matter experts.
And the human aspect is crucial not only because of labor costs but also because of the long-term morale of the team. In my experience, penetration testers excel when faced with challenges and tend to dislike reporting simple vulnerabilities repeatedly (which is typical for vulnerability assessments). Meanwhile, engineers resent having to demonstrate the business impact (which is crucial for penetration testing).
One example of this fundamental difference in character between security and engineering is this classic response by the creator of Linux:
Measuring Security Progress Over Time
Imagine you order a penetration test for your system from an external service provider, but instead of receiving a penetration testing report, you receive something more similar to a vulnerability assessment.
This happens very often and is a problem in itself. However, what’s even more problematic is the following: If you later decide to order another penetration test for the same system but from a different provider, and this time you receive the results of service acquired, you won’t be able to compare the current state with the previous one. It would be like comparing apples to oranges – the goals and effects of a penetration test are different from those of vulnerability assessment (regardless of the quality of the service itself).
That’s why you need to fully understand what you want to achieve and ensure that your service provider’s understanding aligns with yours. You need to do that in order to solve the second challenge, which is progress over time. This may not seem like a big deal, but repeatedly fixing the same issues is both inefficient and can eventually lead to burnout.
Also, with regard to progress, there is one additional factor to consider: automation. Any process that can be automated should be automated. But without a clear understanding of the process itself (where does it fit? what does it produce? et cetera), automation becomes challenging.
5 Types of Security Assessments Explained
Now that we understand why this matters, let’s examine how you can actually assess the security of an IT system. To accomplish this task, you can employ five distinct assessment types:
Threat Modeling
Security Audit
Vulnerability Assessment
Penetration Test
Red Teaming
This list can be translated into following diagram:
The order is intentional and should be viewed as a gradient without clear boundaries in-between. Furthermore, each subsequent type can be seen as a progression from the previous one as you –generally– wouldn’t want to perform red teaming exercise before penetration testing, nor would you do penetration testing before doing vulnerability assessment.
Also, it’s worth pointing out that the complexity of the entire exercise increases from left to right. Along with it, the security level of the component, system, or the entire organization might increase as well.
The taxonomy outlined above is based on the features I previously mentioned, and although there can be more such features, I will focus on the five I consider to be the most important. These are:
Goal. What is our objective? For example, although vulnerability assessment and penetration testing are similar at the tactical level, they differ significantly from a strategic perspective.
Scope. What will be considered during the assessment? In threat modeling, for instance, we analyze the system from a different perspective compared to a vulnerability assessment. Subsequently, with a vulnerability assessment, we might look at the system differently than with penetration testing.
Effort. How many people do we need to engage to achieve a satisfactory return on investment? Vulnerability assessment or security audit requires less effort in terms of people than penetration testing or red teaming exercise.
Time. How much time can we allocate? Typically, one would allocate different amounts of time for a vulnerability assessment or a penetration test compared to an audit or threat modeling.
Automation. To what extent can we automate the work? While we can automate vulnerability assessment or an audit to a considerable extent, threat modeling or penetration testing are harder to automate.
Understanding Security Testing: A Simple Analogy for Stakeholders
As I mentioned in the introduction, I quite often encounter a lack of solid understanding of the various ways one can use to assess the security of an IT system. In fact, I encounter it so often that I managed to come up with a useful analogy for non-technical people.
Imagine you bought a plot of land with the goal of building a house. How would you approach securing such a project? What would you do? When would you do it? Let’s walk through it.
You’ll start with threat modeling during the design phase for your new home and plot of land. At this point, your focus is on identifying and understanding the various threats that could cause harm to you, your family, or your property. This process aims to help you prevent potential problems and vulnerabilities, both in terms of architecture and implementation. Moreover, you can utilize the resulting threat model in other assessments as a guiding tool to verify the assumptions made during this stage.
Once the house is built, you can conduct a security audit to ensure that your home and property comply with applicable standards. If you’re required you can then present the resulting audit report as proof of compliance with the specified standard.
After building the house, you’ll also conduct a vulnerability assessment, focusing on identifying as many security issues as possible in the house and overall property (e.g., its fence). However, this assessment will not delve into the specifics of whether or how these vulnerabilities can be exploited. The primary goal of the vulnerability assessment is to identify and manage issues that are moderately easy to discover.
By the way, you might consider conducting periodic threat modeling, auditing, and vulnerability assessments during the construction phase of your house. This is because all of them can be performed without the entire system being fully operational; you can focus on specific components and evaluate their threats, compliance issues, or vulnerabilities.
Finally, you’ll commission both penetration testing and red teaming from an external provider that specializes in these activities. This will be done after conducting several iterations of vulnerability assessments. In both cases, the task will be to emulate a real attacker to verify the effectiveness of your current defensive capabilities (e.g., alarm system) and understand attack chains. Both of these approaches aim to show you how different vulnerabilities can be combined to achieve a specific outcome, such as car theft, rather than just identifying individual problems. By undertaking these actions, you aspire not only to become secure or compliant but also resilient (or even antifragile).
Summary: Building Your Security Assessment Strategy
Effective cybersecurity programs require a clear understanding of different security assessment types and how they complement each other. The five key methods—Threat Modeling, Security Audit, Vulnerability Assessment, Penetration Testing, and Red Teaming—each serve a distinct purpose.
By understanding their differences in terms of goals, scope, effort, time, and automation potential, we will be able to:
Deploy the right assessment at the right time within SDLC
Track meaningful progress by comparing like-to-like results over time
Optimize resources by matching assessment complexity to organizational needs
Build toward resilience through a progressive, layered approach to security
Without a coherent model, there’s a risk of inefficient security spending, team burnout from dealing with repetitive issues, and difficulty in demonstrating improvement. The proposed taxonomy offers a foundation for effectively integrating security assessments into your organization’s processes.
In future articles, I will explore each assessment type in detail, examining practical implementation strategies and common pitfalls to avoid.
Addendum
I have three additional points I’d like to make that are more peripheral than central to the main issue; hence, I include them only at the end.
First, I’m not the first to highlight the significant linguistic issues in cybersecurity, where certain keywords are used liberally, and their meaning heavily depends on the context in which they are used. For example, you might notice that I have intentionally avoided using risks in this article, even though it’s often used interchangeably with threats. This deliberate choice illustrates that, while these terms might be treated as synonyms in casual conversations among experts who quickly grasp their intended meaning from context, they aren’t truly interchangeable.
Secondly, I’m well aware that there are numerous definitions for all the assessment types I listed in my article. For example, one might use the definitions provided by NIST. So, why do I bother writing about this matter? Well, even though one could use the definitions provided by NIST, this approach is problematic because truly understanding them requires extensive experience at the ground level—experience that many lack and will never gain.
Third and final point is the answer to the following question: But surely you’re not the first one writing about this topic? Of course not. As is often the case, I’m standing on the shoulders of giants—one of them is Daniel Miessler, whom I’ve been following for more than a decade. He wrote about this topic years ago, but the problem with his take is that it only scratched the surface. So, even though I’m sure Daniel has a coherent model of security assessments, I still found it necessary to extend some points in his public writing for myself. My aim is to save the reader this effort.