As you can see Kasper shared a provocative post about the never-ending debate in tech: should you build software solutions in-house or purchase them from the market? His post, discussing the common engineer mindset of “just scripting it in [insert any technology you choose],” struck a nerve with me because it’s a mindset I have observed repeatedly—and rarely with successful outcomes.

For starters, let me share a personal story.

When “Free” Becomes the Most Expensive Option

A while back, a software engineer reached out to pitch his project to me, which was a custom Application Security Posture Management (ASPM) solution. Of course, there was more to this story (no one builds an ASPM just for fun), and the full picture looked like this: his company needed to secure their software, looked at commercial options, and decided they were “too expensive.” So he decided to build one himself after hours, then his company deployed it for free, and he framed his arrangement as being a “design partner” (although to me, that seemed like a generous way to describe working for free).

Now, in order to secure ongoing support and maintenance for his recently developed ASPM, he wanted to take it to market and was looking for a partner or investor. One of the first questions I asked was this: “If your company wouldn’t pay for an existing ASPM solution because it was too expensive, then why would another similar company pay for yours?” This question might seem harsh, but it came from a place of sincerity.

Anyhow, as you may guess, he didn’t have a clear answer, as the economics of his own story contradicted the business plan. And I’ve noticed a deeper issue here.

Why Companies Buy Software (Hint: It’s Not Because They Can’t Build It)

At the beginning of working on any solution to the problem in tech, it’s easy to assume that “build it yourself” is free because you already know how to do that. However, the true cost of building isn’t in the actual building itself, but in the ongoing maintenance of the solution. Most of the time, this cost far exceeds the apparent price tag of vendor products.

So let me state frankly what many engineers fail to understand: you don’t buy software because you can’t build it. You buy it because maintaining, securing, and evolving it isn’t free, and it doesn’t make you money. In other words, it’s not your business.

But what is your business? Let’s say that we’re talking about a SaaS company building accounting software for small consultancies. Its core competency should be understanding accounting workflows, tax regulations, what makes bookkeepers’ lives easier, et cetera. Building own security solutions? That’s about as relevant to this business as baking own bread would be.

Which brings me to an important concept in economics that somehow gets lost in the “we can build anything” mindset: the division of labor.

The Economics of the Buy or Build Decision

Speaking of bread, could I bake my own? Absolutely! Would it be fresh, warm, and taste better than store-bought? Maybe. But here’s what I’d lose: the time and focus I could spend working on what I’m actually good at—helping companies secure their products.

So instead of baking bread, I buy bread. In this way, the baker is happy because they do what they like and know how to do well. I’m happy because I can also focus on things I like and know how to do well, while still having fresh bread. In short, we’re both better off because we’ve specialized.

Which makes so much sense that it is in fact the foundation of how the economy works. Yet in tech, I constantly see this principle ignored. Engineers look at a problem and think, “I could knock that out in a weekend.” But what they don’t calculate is:

• The opportunity cost of not working on core product features that differentiate the business

• The ongoing maintenance cost as dependencies update, security vulnerabilities emerge, and requirements change

• The knowledge transfer cost when that engineer inevitably leaves and nobody knows how “that weekend project” works

• The scale and reliability cost of running it in production at enterprise scale

When to Build In-House: A lesson from Boeing

Now, to be clear, I’m not advocating for outsourcing everything to the bone. The pendulum can swing too far in the other direction, and it has for various companies many times.

A prime example is Boeing, whose own experience demonstrates that the buy vs. build decision isn’t binary. When companies outsource core competencies (what makes them unique), they lose competitive advantage. Conversely, when they insist on building everything in-house, they waste resources on solved problems.

There is a balance to be found here, which requires understanding what is core to the business and what isn’t. And as it often is for such matters—it sounds easy, but it is hard.

For an elaborate critique of Boeing, read the excellent paper “Outsourced Profits – The Cornerstone of Successful Subcontracting”.

Why Engineering Should Advise, Not Decide

Engineers will almost always believe they can “just build it.” I don’t think it’s malice or arrogance—I think it’s professional optimism. You know what I mean—give me six engineers and a pizza budget, and we could build a basic version of most SaaS products over a few weekends. (Especially now with AI, am I right?)

But “can we build it?” is the wrong question to ask. The right questions to ask are:

• Should we build it?

• What’s the total cost over 3-5 years?

• What core business value are we deferring to build this?

• Who will own it when the original builder leaves?

• What happens when it needs to scale 10x?

These are business questions that require business thinking, which in the long-term is a competitive advantage over those who think strictly in engineering terms.

BTW, I also don’t believe that this is an instance of NIH Syndrome, which in my mind is a separate, deeper issue.

Build What Makes You Unique, Buy Everything Else

The build vs. buy decision ultimately comes down to understanding your core business. Build the solutions that give you a competitive advantage (security is not it unless you’re a security startup). Buy everything else, even if you could build it. Focus your engineering talent on innovation, not on reinventing solved problems.

Getting this right isn’t just about saving money—it’s about strategic focus. Those who master it move faster, innovate more, and compete better.

Subscribe now

P.S. If you’re interested in learning more about Kasper’s work, check out Aporio, where he’s building an enterprise identity intelligence platform.

P.P.S. Recent article by also touched upon this subject with an excellent passage: “People who believe enterprises buy software only because they can’t build it in-house fundamentally misunderstand how large organizations operate.” You can read the full article here.

An audit is a specific type of assessment with a specific purpose, and that’s what I’ll be unpacking today.

Throughout this series, I examine security assessment methods through five key characteristics:

• Goal. What are we trying to achieve?

• Scope. What falls within our assessment boundaries?

• Time. What’s our timeline?

• Effort. How much resource investment delivers adequate ROI?

• Automation. What portions of this work can we automate?

These five dimensions naturally reveal the contrasts between assessment types. They help you understand not just what makes an audit different from a pentest or threat modeling, but more importantly, when to deploy each method within your organization.

Because here’s the thing: calling everything a “pentest” or labeling every assessment an “audit” creates dysfunction. You need different tools for different stages. Sometimes you need an audit. Other times you need vulnerability assessment, threat modeling, or penetration testing. Knowing which stage calls for which method requires understanding the fundamental differences between them.

And yes, “all models are wrong, but some are useful”—this one included.

This article series includes:

• Part 1: Security Assessment Types Overview

• Part 2: Threat Modeling

• Part 3: Security Audits (you are here)

• Part 4: Vulnerability Assessments (coming soon)

• Part 5: Penetration Testing (coming soon)

• Part 6: Red Teaming (coming soon)

TL;DR: Security audits evaluate compliance with standards rather than directly measuring security effectiveness. They provide essential benefits: establishing your current security position, ensuring alignment with industry standards, and enabling progress tracking through repeatable assessments. While compliant doesn’t always mean secure, audits remain crucial for understanding where you stand and planning improvements.

What is a Security Audit? Definition and Purpose

The word “audit” originates from the Latin term “auditor,” which means listener. This word was historically associated with the “hearing of accounts” in ancient Rome, where one official would listen to another reading documents to compare them and identify any discrepancies.

To this day audits remain a cornerstone of the financial world and while I do enjoy discussing financial and investment topics, I’ll set that aside for now. Today, we’ll focus on the realm of security audits.

So, what is a security audit then? In its broadest definition, a security audit is an independent evaluation of a system against a standard in order to verify and/or ensure compliance. Not security, compliance.

The system mentioned in the definition could be a product, an IT system, or even an entire organization. In turn, the standard might encompass an industry standard, a norm, a legal regulation, or even an internal policy.

I’d like to reiterate a hidden point from above—the goal of an audit is not to confirm the effectiveness of security measures or prove that a control is actively fulfilling its intended purpose. The goal of an audit, its strategic objective, is compliance.

Consider the following example: During an audit, if a control specifies that my web applications must be protected by a Web Application Firewall (WAF), the auditor usually will not examine whether my WAF configuration is correct or incorrect. As long as a WAF is present, the assumption is that applications are in fact “protected by it” (i.e., the control is considered satisfied). So, even if the WAF is improperly configured and thus ineffective, it does not impact the audit results.

And this approach is appropriate because assessing the effectiveness of the WAF should be part of a penetration test, which itself can be part of a larger audit. In fact, performing one security assessment type on behalf of another is a standard way of doing things (e.g., threat modeling done at the beginning of a pentest; pentesting performed as part of a larger audit effort; et cetera). I call it the Matryoshka Effect of Security Assessments™.

Anyway, the point I tried to make by consistently emphasizing compliance over security is this: results of an audit can be manipulated (and they often are, but that’s a topic for another discussion).

The need for compliance, and therefore an audit, is usually external. It might be driven by changes in the regulatory landscape (e.g., regulatory requirements like DORA, CRA, or NIS2), or it could be due to business needs, such as when a European SaaS company wants to gain clients in the US and is required to comply with SOC 2. Ultimately, being compliant with a specific standard makes it easy to conduct apples-to-apples comparisons, which is one of the primary goals of any regulation or standard.

For example, in my backyard (Poland), companies typically start considering ISO 27001 when they begin sales to international clients. For these clients, compliance with this standard serves as proof that the company meets the minimum baseline for information security necessary to become their business partner.

BTW, I was once told by an experienced auditor that “not every organization compliant with ISO 27001 is secure, but every secure organization is compliant with ISO 27001”, which perfectly captures the point I wanted to make earlier: an audit can only confirm compliance, and it’s possible to be compliant while not being secure. (Warm regards, Jakub! 😊)

And while we’re still on the topic of standards—it’s worth noting that some of them are stricter than others (e.g., FedRAMP vs. SOC 2), which of course has an impact on scope…

How Long Do Security Audits Take? Scope, Timeline, and Resource Requirements

An audit can be broad, encompassing various aspects such as people, processes, and IT. But it can also be narrow. The scope is determined by the standard against which we are auditing.

However, regardless of the scope, for an auditor to perform a reliable and trustworthy audit, they must have access to all the necessary information to carry out their task effectively. In fact, unless we are doing shady things, it’s in our own best interest to help the auditor. (The Enron case could serve here as a classic counterexample.)

An audit can focus exclusively on policies or on technical aspects. However, most audits are a combination of both—an audit against industry-accepted standards and norms involves a review of policies alongside a technical evaluation. For example, ISO 27001 involves policy validation, vulnerability assessments, and penetration testing.

Internally, audits are managed by the Governance, Risk, and Compliance (GRC) unit, if one exists, or by a special task group formed specifically for the audit if the GRC unit is not available.

The time needed to conduct an audit depends on the standards being audited against and the size of the organization undergoing the audit. An audit could take a couple of weeks, such as auditing a small or medium-sized business (SMB) against ISO 27001. However, it might take several months when auditing the same SMB against SOC 2 Type 2 due to the need for extensive proof of controls.

This is related to the demand for resources—the larger the audit, the more people are needed. In theory, even the largest audit could be performed by just one person; however, in practice, this would be unfeasible. In practice, audits are conducted by small teams of auditors and—similar to Red Teaming—domain experts can be utilized “on the fly,” so they don’t need to be full-time team members.

Conversely, from the perspective of the organization being audited, engaging a broader range of individuals in various roles can help prevent the problem of tunnel vision.

A great example of this is found in the book The Phoenix Project, where John, the security expert at Parts Unlimited, falls victim to tunnel vision during the “SOX404 audit”. To John’s great surprise, the this audit was passed successfully despite his conviction of a lack of security controls in the IT systems under his watch. The twist was that John had only focused on his own area of expertise and failed to consider the entire business. As a result, he was unaware that certain controls he deemed critical were actually unnecessary because other business processes already covered them. Thus, while the goals of the audit were being met, they were not being fulfilled in the exact area John had anticipated, which is a textbook example (he, he) of tunnel vision during an audit.

Automation Capabilities in Security Audits

When it comes to audits, the feasibility of automation depends on the subject of the audit and the standards being evaluated against.

For example, automating an audit of an entire organization can be challenging. On the other hand, automating an audit of a Kubernetes platform against CIS Benchmarks can be done quite easily (although it might cause some pain in the beginning).

Both of these are audits, but their scope is entirely different. So, since I am a consultant, with regard to automation, I’d simply say that it depends.

Yes, I’m aware of companies such as Vanta or Drata; they do help with automation of certain parts of the process for certain standards, but I wouldn’t call it automation in the engineering sense (set it and forget it). At best, it’s semi-automation.

Summary: Why Security Audits Matter

Security audits operate at a broader and higher altitude than vulnerability assessments or penetration tests. Yet, they play an essential role in security. And I’m using “security” deliberately here, not limiting it to cybersecurity alone, but encompassing the wider domains of security and safety.

Why do audits matter? Because they answer a fundamental question: Where are we right now?

Think of it as navigation. To reach any destination, you need three things in order: (1) know where you’re going, (2) understand where you currently are, and (3) plot the route between them. Skip step two and you’re navigating blind. You can’t chart a meaningful course without knowing your starting coordinates.

But audits deliver two additional strategic benefits beyond establishing your baseline.

First, they enable you to leverage collective intelligence. Aligning with industry standards means you’re tapping into accumulated market insights and battle-tested practices rather than reinventing the wheel.

Second, audits are inherently repeatable. This replicability is what transforms a single assessment into a measurement system. You can conduct the same audit quarterly, annually, or on whatever cadence makes sense—and each iteration reveals whether you’re actually moving forward or just running in place.

This last point about tracking progress over time is particularly important, and I’ll be returning to it in future articles. Subscribe to receive updates directly in your inbox.

Subscribe now

Addendum: How to Challenge Auditor Requirements Effectively

I have one more thing to say about audits: Auditors should not be viewed as the ultimate authority. There may be instances where the auditor does not completely understand the purpose of a particular requirement. This issue is related to tunnel vision, but this time it affects the auditor and not the party being audited.

This is because, often, a given control requires domain knowledge to grasp its underlying purpose. In such cases, the auditor might interpret the control too literally, missing this hidden objective—specifically, why the control exists and what it’s intended to achieve.

Consider this scenario: An auditor requires you to conduct periodic vulnerability scans of your infrastructure using a market-recognized tool, let’s say Nessus. This is a control that needs to be satisfied, but the problem is that your infrastructure consists of ephemeral Docker containers that are part of a cloud-based cluster. So, do you really need to periodically run Nessus on this cluster? Certainly not, that would be absurd!

The objective of the control is to ensure that everything within your infrastructure is up-to-date and free from known vulnerabilities. This is an actual purpose of these scans. So, in the environment I described, achieving this goal isn’t about scanning live infrastructure and performing security patching. Instead, it is about ensuring that your images are up-to-date and free from known vulnerabilities before they enter the cluster. Which is important because this goal is accomplished by a different team, using a different tool, and at a different point in the software development lifecycle.

Now, let’s return to the topic of manipulating audit results because sometimes—even when you’re correct—strict compliance is still an obligation. And on this topic, I remember the epic way Cloudflare bypassed the audit requirement very similar to our example.

In a nutshell: Cloudflare is one of those edge cases that doesn’t fit into the cookie-cutter approach required by a typical audit. Consequently, they weren’t fully satisfied with the offerings from market leaders in vulnerability scanning and the capabilities of their tools. So, with the help of an intern, they developed their own vulnerability scanner, FLAN Scan. This scanner uses Nmap as the network scanning engine and incorporates the Vulners script to provide vulnerability references. They enhanced the tool by adding detailed PDF reporting and released it as an open-source project. Cloudflare saved a significant amount of money with this approach while satisfying the audit requirement for vulnerability scans.

A similar scenario might occur when the auditor relies on a set of specific answers, requiring additional justification for any answers outside that framework.

On several occasions, I have had to challenge the auditor’s assessment, which ultimately benefited my clients. One such instance involved contesting the implementation of a static analysis. The client was using SonarQube, but the auditor deemed it an unacceptable substitute for solutions like Checkmarx or Veracode. I stepped in and argued that, in my SME opinion—based on my knowledge of how static analysis works, my experience with various tools, and my understanding of its objectives—SonarQube is indeed a suitable and more cost-effective alternative. The argument I made won, but it was only because I had all the necessary background to understand it and then make it.

What makes threat modeling special? Unlike other security assessments that find problems after they exist, threat modeling prevents them from occurring in the first place. It’s the only truly proactive security practice in the SDLC, allowing teams to identify and address potential vulnerabilities before writing a single line of code.

In this article, I’ll demystify threat modeling by covering:

• What threat modeling actually is and what it produces

• The strategic objectives it helps achieve

• How scope affects implementation across different organizational levels

• Practical considerations for time, effort, and automation

• Why it’s foundational to modern secure development practices

Whether you’re implementing threat modeling for compliance, cost reduction, or genuine security improvement, understanding these fundamentals will help you navigate the practice effectively.

This article series includes:

• Part 1: Security Assessment Types Overview

• Part 2: Threat Modeling (you are here)

• Part 3: Security Audits

• Part 4: Vulnerability Assessment Guide (coming soon)

• Part 5: Penetration Testing Deep Dive (coming soon)

• Part 6: Red Teaming for Organizations (coming soon)

TL;DR: Threat modeling is the only proactive security assessment—it prevents vulnerabilities rather than finding them. By answering “What are we working on?”, “What can go wrong?”, and “What are we going to do about it?”, teams identify and address security issues before implementation. It serves two strategic goals: reducing defect costs and demonstrating compliance. Scope varies from entire organizations to individual features, with time requirements scaling accordingly. Best conducted with 3-6 diverse participants in focused sessions. While tools can help, the real value lies in embedding security thinking into teams through repeated practice. Unlike other assessments that verify security after the fact, threat modeling builds it in from the start.

What is Threat Modeling? Definition and Core Concepts

For starters, let’s define what threat modeling actually is. The best definition I know comes from the Threat Modeling Manifesto and goes like this: “Threat modeling is analyzing representations of a system to highlight concerns about security and privacy characteristics.”

This analysis occurs during a threat modeling session, where we address three fundamental questions:

• What are we working on?

• What can go wrong?

• What are we going to do about it?

To these three fundamental questions, we can add a bonus one: Did we do a good enough job?

Our analysis ends with an artifact in the form of a threat model. Such a threat model can look like this:

Or like this:

Or even like this:

The form of the resulting threat model isn’t important, as there are many ways it can be represented. However, what is important is that by conducting threat modeling, you deliberately create a threat model that can later be used to prevent issues, demonstrate compliance, or serve as a baseline for further verification (i.e., answering the bonus question—did we do a good enough job)… but I got ahead of myself here, so let’s step back.

Why Use Threat Modeling: Strategic Benefits and ROI

Now that we know what threat modeling is and what are its products, we can dive deeper and answer questions like when, where, and why we would use it. Among these, answering the why question seems like a good first step.

So, why would we bother with threat modeling in the first place? Or, to put it more professionally: What are the strategic objectives we might want to achieve with threat modeling?

While working on implementing this process, I identified only two such objectives:

1. Reducing the cost of software defects and security verification

2. Demonstrating compliance

As I already mentioned, we can use threat modeling to prevent issues from escalating into vulnerabilities. We can also use the resulting threat models to serve as a baseline for further verification through other assessments (e.g., penetration testing). Both of these reduce the overall cost of the SDLC:

As for compliance with regulations or industry standards, performing threat modeling and storing the resulting threat models is direct proof of the practice. Furthermore, if compliance is the sole reason for the practice of threat modeling, then we could perform it retroactively (i.e., after the thing we are building is built). Of course, working retroactively decreases ROI, but it often meets regulatory obligations and might be a good first step in initiating threat modeling as a practice.

Threat Modeling Scope: From Features to Enterprise

The scope of threat modeling can vary significantly, which is why it is often challenging to understand for newcomers. To clarify this matter, let’s consider a typical tech-savvy organization as an example to illustrate how we could approach threat modeling.

To begin, we could perform threat modeling for the entire organization to identify potential threats, such as advanced persistent threats (APTs) that could lead to serious incidents. This process would consider all aspects, including people, processes, and IT. Subsequently, we could intensify our focus by conducting threat modeling exclusively on the IT infrastructure. We could then analyze a specific network segment, such as a Kubernetes cluster, or an individual system within that segment, like a group of microservices. From there, we could further narrow our focus to a specific application within the system, like an individual microservice. Finally, we could zero in on specific features within this application, such as a feature developed during the current sprint.

In all these cases, we would perform threat modeling, but since our POV is changing, we would require different people and different tools for each perspective.

How Long Does Threat Modeling Take? Time Requirements by Scope

Time depends on the scope, meaning the breadth and depth of the perspective we take—if we’re talking about threat modeling for a specific feature, it will take less time than threat modeling for the entire IT infrastructure. There is a simple relation here:

more elements to analyze = more time required.

In one of the gazillion podcasts I listen to, I learned that threat modeling for the Xbox ecosystem took several months, which is entirely feasible considering its scope. This doesn’t mean all work stopped while everyone focused solely on threat modeling. Rather, I assume it was part of a larger architectural design process for the solution considering Microsoft SDL.

One thing to keep in mind is that scope and time required significantly affect the frequency of threat modeling. So, how often should it occur? It depends on what is being modeled.

For example, if we’re talking about threat modeling at the feature level and we’re using Agile methodology, then each sprint could include a dedicated threat modeling session to focus on the security aspects of what we intend to build. However, this doesn’t mean that each time we need to create a comprehensive threat model for the entire system. Instead, we could allocate 45-60 minutes to address the questions “what can go wrong” and “what are we going to do about it” with regard to the scope defined by the sprint itself (“what are we working on”). These insights can then be documented and added to the relevant tasks within the backlog.

On the other hand, if we were to perform threat modeling for an entire organization, then it would take more time and could be more infrequent (e.g., annually).

Who Should Participate in Threat Modeling Sessions?

When considering the size of the working group conducting threat modeling, it is recommended to keep it small.

Avoiding overly large groups is important because each additional member increases communication issues non-linearly. Based on my experience, the ideal group size for a single threat modeling session is 3 to 6 people, including the facilitator who leads the discussion.

I’d also like to emphasize the crucial role of having a diverse array of roles involved in a threat modeling session. For example, when conducting threat modeling for an application’s features, the working group might include developers, architects, QA engineers, and even product owners. On the other hand, modeling threats for infrastructure requires the involvement of security professionals and operations staff, rather than software engineers.

Can You Automate Threat Modeling? Tools vs. Human Analysis

Threat modeling can be enhanced with the use of specialized tools. A popular choice is the Microsoft Threat Modeling Tool or its OWASP counterpart, Threat Dragon. Both of these tools are available for free, alongside various commercial alternatives. These tools primarily aim to automate certain aspects of the threat modeling process. However, it is important to understand that this semi-automation cannot yet replace the analysis performed by those who build and maintain the system. It remains fundamentally limited since it often generates only generic threats and frequently results in false positives.

The advancements in LLMs offer a promising future, but based on my experience and that of others (warm regards, Filip 😊), they work best with an operator who is already proficient in threat modeling and knows what to ask for.

So, while certain aspects of threat modeling can be automated, I believe its primary value lies in raising the baseline for security awareness. When a team actively engages in these sessions, security thinking gradually becomes embedded in their daily work through repeated iterations. As a result, during feature implementation, team members instinctively start to consider potential issues and what might go wrong.

A similar effect on the improvement of security awareness can be observed with a well-executed SAST process, which not only establishes guardrails but at the same time gradually elevates the team’s baseline security awareness.

Bottom Line: Threat Modeling as Your Security Foundation

In the home building analogy I introduced in the first post of this series, threat modeling takes place at the very beginning; we conduct it before we start constructing anything in order to understand the threats we need to consider. Here, we answer, “What are we working on?”—which is our home. We address “What can go wrong?” in the context of our property, location, and fence, and finally we determine “What are we going to do about it?”

This process helps us avoid potential vulnerabilities, both architectural and implementation-related. Furthermore, we can later utilize this threat model to verify our assumptions during other types of assessments. Additionally, the threat model can serve compliance purposes during a security audit.

In my opinion, threat modeling is one of the fundamental practices in the SDLC because it enables you to avoid mistakes, setting it apart from other security assessment types as the only proactive action.

That’s all for today. Next, I’ll concentrate on security audits. Subscribe to receive updates directly in your inbox.

Subscribe now

The goal of this series is to improve this state of affairs. Today, we’ll take a high-level overview of the taxonomy for security assessments, and in subsequent parts, we’ll examine each one of them in more detail.

Continue reading if you’re interested in topics like Product Security, Secure SDLC, DevSecOps, or security automation. Subscribe if you want to receive updates straight into your mailbox.

This article series includes:

• Part 1: Security Assessment Types Overview (you are here)

• Part 2: Threat Modeling

• Part 3: Security Audits

• Part 4: Vulnerability Assessment (coming soon)

• Part 5: Penetration Testing (coming soon)

• Part 6: Red Teaming (coming soon)

TL;DR: Security assessments fall into 5 types: Threat Modeling (design phase), Security Audit (compliance), Vulnerability Assessment (finding issues), Penetration Testing (exploiting issues), and Red Teaming (full attack simulation). Each serves different goals on your organization’s road to security.

The Business Case for Security Assessment Taxonomy

Before diving into the details, I will take a little detour and first answer the following question: How can we differentiate between various types of security assessments?

We can do that by zooming in on certain features and using them to highlight differences. This will enable us to develop a coherent model of security assessments that can be used to integrate them into the organization.

I will return to these features, but for now, an even more important question arises: Why does it matter to have a coherent model of security assessments? It’s important because without it, implementing or maintaining effective cybersecurity program becomes harder than it should be (unless your focus is solely on compliance rather than security engineering). This is due to two main challenges.

Matching Security Assessments to Development Stages

Different types of assessments provide different insights, making them suitable for achieving different goals. So, the first challenge is determining the type of security assessment one wants to use, along with choosing when and where it should be used. This is hard when you don’t have a coherent taxonomy.

Let’s take a quick look at vulnerability assessment versus penetration testing – both can be used within the broader Software Development Lifecycle. However, the highest ROI comes from continuous vulnerability assessment executed in pre-release phase and point-in-time penetration testing performed in post-release phase.

This is closely linked to the teams that carry out the work at the ground level. While basic security testing, such as vulnerability assessment, can be conducted by non-professionals (e.g., QA/SDET), penetration testing requires subject-matter experts.

And the human aspect is crucial not only because of labor costs but also because of the long-term morale of the team. In my experience, penetration testers excel when faced with challenges and tend to dislike reporting simple vulnerabilities repeatedly (which is typical for vulnerability assessments). Meanwhile, engineers resent having to demonstrate the business impact (which is crucial for penetration testing).

One example of this fundamental difference in character between security and engineering is this classic response by the creator of Linux:

Measuring Security Progress Over Time

Imagine you order a penetration test for your system from an external service provider, but instead of receiving a penetration testing report, you receive something more similar to a vulnerability assessment.

This happens very often and is a problem in itself. However, what’s even more problematic is the following: If you later decide to order another penetration test for the same system but from a different provider, and this time you receive the results of service acquired, you won’t be able to compare the current state with the previous one. It would be like comparing apples to oranges – the goals and effects of a penetration test are different from those of vulnerability assessment (regardless of the quality of the service itself).

That’s why you need to fully understand what you want to achieve and ensure that your service provider’s understanding aligns with yours. You need to do that in order to solve the second challenge, which is progress over time. This may not seem like a big deal, but repeatedly fixing the same issues is both inefficient and can eventually lead to burnout.

Also, with regard to progress, there is one additional factor to consider: automation. Any process that can be automated should be automated. But without a clear understanding of the process itself (where does it fit? what does it produce? et cetera), automation becomes challenging.

5 Types of Security Assessments Explained

Now that we understand why this matters, let’s examine how you can actually assess the security of an IT system. To accomplish this task, you can employ five distinct assessment types:

• Threat Modeling

• Security Audit

• Vulnerability Assessment

• Penetration Test

• Red Teaming

This list can be translated into following diagram:

The order is intentional and should be viewed as a gradient without clear boundaries in-between. Furthermore, each subsequent type can be seen as a progression from the previous one as you –generally– wouldn’t want to perform red teaming exercise before penetration testing, nor would you do penetration testing before doing vulnerability assessment.

Also, it’s worth pointing out that the complexity of the entire exercise increases from left to right. Along with it, the security level of the component, system, or the entire organization might increase as well.

The taxonomy outlined above is based on the features I previously mentioned, and although there can be more such features, I will focus on the five I consider to be the most important. These are:

• Goal. What is our objective? For example, although vulnerability assessment and penetration testing are similar at the tactical level, they differ significantly from a strategic perspective.

• Scope. What will be considered during the assessment? In threat modeling, for instance, we analyze the system from a different perspective compared to a vulnerability assessment. Subsequently, with a vulnerability assessment, we might look at the system differently than with penetration testing.

• Effort. How many people do we need to engage to achieve a satisfactory return on investment? Vulnerability assessment or security audit requires less effort in terms of people than penetration testing or red teaming exercise.

• Time. How much time can we allocate? Typically, one would allocate different amounts of time for a vulnerability assessment or a penetration test compared to an audit or threat modeling.

• Automation. To what extent can we automate the work? While we can automate vulnerability assessment or an audit to a considerable extent, threat modeling or penetration testing are harder to automate.

Understanding Security Testing: A Simple Analogy for Stakeholders

As I mentioned in the introduction, I quite often encounter a lack of solid understanding of the various ways one can use to assess the security of an IT system. In fact, I encounter it so often that I managed to come up with a useful analogy for non-technical people.

Imagine you bought a plot of land with the goal of building a house. How would you approach securing such a project? What would you do? When would you do it? Let’s walk through it.

You’ll start with threat modeling during the design phase for your new home and plot of land. At this point, your focus is on identifying and understanding the various threats that could cause harm to you, your family, or your property. This process aims to help you prevent potential problems and vulnerabilities, both in terms of architecture and implementation. Moreover, you can utilize the resulting threat model in other assessments as a guiding tool to verify the assumptions made during this stage.

Once the house is built, you can conduct a security audit to ensure that your home and property comply with applicable standards. If you’re required you can then present the resulting audit report as proof of compliance with the specified standard.

After building the house, you’ll also conduct a vulnerability assessment, focusing on identifying as many security issues as possible in the house and overall property (e.g., its fence). However, this assessment will not delve into the specifics of whether or how these vulnerabilities can be exploited. The primary goal of the vulnerability assessment is to identify and manage issues that are moderately easy to discover.

By the way, you might consider conducting periodic threat modeling, auditing, and vulnerability assessments during the construction phase of your house. This is because all of them can be performed without the entire system being fully operational; you can focus on specific components and evaluate their threats, compliance issues, or vulnerabilities.

Finally, you’ll commission both penetration testing and red teaming from an external provider that specializes in these activities. This will be done after conducting several iterations of vulnerability assessments. In both cases, the task will be to emulate a real attacker to verify the effectiveness of your current defensive capabilities (e.g., alarm system) and understand attack chains. Both of these approaches aim to show you how different vulnerabilities can be combined to achieve a specific outcome, such as car theft, rather than just identifying individual problems. By undertaking these actions, you aspire not only to become secure or compliant but also resilient (or even antifragile).

Summary: Building Your Security Assessment Strategy

Effective cybersecurity programs require a clear understanding of different security assessment types and how they complement each other. The five key methods—Threat Modeling, Security Audit, Vulnerability Assessment, Penetration Testing, and Red Teaming—each serve a distinct purpose.

By understanding their differences in terms of goals, scope, effort, time, and automation potential, we will be able to:

• Deploy the right assessment at the right time within SDLC

• Track meaningful progress by comparing like-to-like results over time

• Optimize resources by matching assessment complexity to organizational needs

• Build toward resilience through a progressive, layered approach to security

Without a coherent model, there’s a risk of inefficient security spending, team burnout from dealing with repetitive issues, and difficulty in demonstrating improvement. The proposed taxonomy offers a foundation for effectively integrating security assessments into your organization’s processes.

In future articles, I will explore each assessment type in detail, examining practical implementation strategies and common pitfalls to avoid.

Addendum

I have three additional points I’d like to make that are more peripheral than central to the main issue; hence, I include them only at the end.

First, I’m not the first to highlight the significant linguistic issues in cybersecurity, where certain keywords are used liberally, and their meaning heavily depends on the context in which they are used. For example, you might notice that I have intentionally avoided using risks in this article, even though it’s often used interchangeably with threats. This deliberate choice illustrates that, while these terms might be treated as synonyms in casual conversations among experts who quickly grasp their intended meaning from context, they aren’t truly interchangeable.

Secondly, I’m well aware that there are numerous definitions for all the assessment types I listed in my article. For example, one might use the definitions provided by NIST. So, why do I bother writing about this matter? Well, even though one could use the definitions provided by NIST, this approach is problematic because truly understanding them requires extensive experience at the ground level—experience that many lack and will never gain.

Third and final point is the answer to the following question: But surely you’re not the first one writing about this topic? Of course not. As is often the case, I’m standing on the shoulders of giants—one of them is Daniel Miessler, whom I’ve been following for more than a decade. He wrote about this topic years ago, but the problem with his take is that it only scratched the surface. So, even though I’m sure Daniel has a coherent model of security assessments, I still found it necessary to extend some points in his public writing for myself. My aim is to save the reader this effort.

• How do you design a system that's secure by default?

• How do you ensure your code isn't vulnerable to injection attacks?

• How do you verify that authentication controls work as intended?

• And —maybe most importantly— how do we know if our security measures are appropriate for our specific context?

ASVS answers all these questions with a systematic approach that adapts to your security needs, whether you're building a simple CRUD application or a critical financial system.

Subscribe now

In this deep dive, we'll explore how ASVS can help you approach application security in a systematic way. We'll break down its structure, understand its levels, and learn how to apply it practically in your projects. Let's begin!

Understanding ASVS

The Application Security Verification Standard (ASVS) is a comprehensive standard that transforms application security into a structured, verifiable process. Through its three-level approach, it provides specific, actionable requirements—both functional and non-functional—that scale with your security needs.

ASVS has evolved significantly since its first release in 2009. The current version (4.0), released in 2019, reflects more than a decade of real-world application security experience and community feedback.

The 14 Security Domains

At its core, ASVS is organised into 14 security domains:

• Architecture, Design and Threat Modeling

• Authentication

• Session Management

• Access Control

• Validation, Sanitisation and Encoding

• Stored Cryptography

• Error Handling and Logging

• Data Protection

• Communications

• Malicious Code

• Business Logic

• File and Resources

• API and Web Service

• Configuration

Each domain provides an overview, specific security controls, and detailed references. What makes ASVS particularly powerful is its modular structure—domains are further divided into focused sub-areas that can be applied selectively based on your application's architecture.

Take the API and Web Services domain, for instance. It includes sub-areas for:

• Generic Web Service Security

• RESTful Web Service

• SOAP Web Service

• GraphQL

This modular approach makes ASVS highly practical. For example, if you're building a modern application using REST and GraphQL (but not SOAP), you can focus specifically on:

1. Core API security requirements that apply universally

2. REST-specific security controls

3. GraphQL-specific security measures

This targeted approach ensures you implement precisely the security controls relevant to your technology stack, without getting bogged down in inapplicable requirements.

ASVS Security Levels

In the introduction, we mentioned security levels, and that wasn't a casual reference—each ASVS control belongs to one of three levels, where each subsequent level raises the security bar higher.

Let's look at a practical example. Control 13.2.2 states that "when using JSON, schema validation must occur before accepting input data" and is required from level one upward:

However, control 13.2.5, which requires "REST services must verify the expected Content-Type," only applies at levels 2 and 3. This means applications targeting only level 1 compliance can skip this requirement:

You might be wondering, "What determines the appropriate level for my application?" The consultant in me wants to say "It depends!"—and it does—but let me give you some concrete guidance.

Level 1: The Security Baseline

Level 1 (L1) requirements target projects with basic security needs. Think of it as your essential security baseline—one that can be largely (though not entirely) verified through automated testing. Importantly, L1 requirements cover most of the OWASP Top 10 vulnerabilities. (This means implementing ASVS L1 as part of your secure development process effectively addresses nearly all OWASP Top 10 risks.)

💡 NOTE: See our previous article if you aren’t familiar with OWASP Top 10.

A crucial point about L1: it's the only level that can be fully tested using a black-box approach—testing without access to documentation, source code, or the development team. However, the ASVS authors (with whom we agree 100%) strongly caution against relying solely on black-box testing. Here's why:

• In the real world, attackers have unlimited time while security testers face strict deadlines

• Effective testing requires maximum insight into the application

Consider this: Would you trust a financial audit where the auditor couldn't access accounting records or talk to the finance team? Yet this scenario plays out regularly in application security testing. That's why ASVS promotes comprehensive security testing that combines traditional testing with code reviews, architectural analysis, and direct collaboration with development teams.

Level 2: Business-Critical Security

Level 2 (L2) targets applications handling important business operations—like processing personal data under GDPR. In practice, L2 is recommended for most applications with user data. Meeting L2 requirements means your application has effective controls against common security issues (e.g. you shouldn’t find basic vulnerabilities like XSS in a search bar).

Level 3: High-Security Applications

Level 3 (L3), the highest tier, is designed for critical applications—think financial systems or medical data processors. These are applications that demand the highest level of trust. L3-compliant applications must excel not just in implementation security but also in architectural design, incorporating secure architecture principles like Defense in Depth or Least Privilege.

Choosing the Right Level

Ultimately, choosing the right ASVS level depends on two contexts: your system's requirements and your organisation's environment. It's simple logic—you expect different security levels from a banking application compared to a meme-sharing website.

Related Standards and Alignment

While ASVS is unique in its domain, as there are no other similar industry standards, OWASP itself maintains several complementary verification standards:

• OWASP Mobile Application Security Verification Standard (MASVS), which adapts ASVS principles for mobile applications

• OWASP Internet of Things Verification Standard (ISVS), which extends ASVS concepts to IoT and embedded systems

• OWASP Software Component Verification Standard (SCVS) addresses supply chain security; however, for practical supply chain security, consider the SLSA framework instead

Also, it’s worth pointing out that ASVS doesn't exist in isolation—it's carefully aligned with established industry standards like PCI DSS and NIST frameworks. Most notably, ASVS controls map to the Common Weakness Enumeration (CWE), providing a bridge between ASVS and the vulnerability classifications discussed in the OWASP Top 10.

💡 NOTE: See our previous article if you aren’t familiar with OWASP Top 10 or CWE.

Practical Applications of ASVS

Role-Based Implementation

ASVS serves as a versatile tool that can be utilised throughout the entire software development lifecycle. For example:

• Architects can leverage ASVS during the design phase to establish secure architecture foundations and define comprehensive security requirements.

• Developers can integrate ASVS into the implementation phase through Secure Code Checklists, making security verification a natural part of code reviews.

• QA Engineers can utilise ASVS Level 1 requirements to validate fundamental security properties, incorporating security testing into standard quality assurance.

• Last, but not least, Security Teams can conduct thorough security assessments using ASVS Levels 2 and 3, ensuring applications meet advanced security requirements.

But ASVS can go beyond the development teams, as Project Managers can incorporate ASVS into third-party software contracts, using specified ASVS levels as concrete acceptance criteria for the software delivered.

Measuring Security Progress

What makes ASVS particularly valuable is its ability to track security progress over time. Unlike broader frameworks such as OWASP Top 10, ASVS provides detailed, measurable criteria that enable organisations to:

• Monitor security improvements across development cycles

• Maintain consistent security standards across projects

• Demonstrate concrete security progress to stakeholders

Aaaand that’s all for today. Best regards and see you next week!

Subscribe now

BTW. If you found this article useful, please pay the fee by sharing it with someone you know will also benefit from it. Thank you very much! 🙏

But what exactly is it? Is there just one definitive Top 10 list? How does it relate to other security standards? And perhaps most importantly, how can you actually use it to make your applications more secure?

In this deep dive, we'll cut through the confusion and explore the world of web application security through the lens of the OWASP Top 10. We'll unpack not just what it is, but why it matters and how it shapes the security landscape of modern web development.

No jargon, no complexity—just clear, practical insights into one of the most important security frameworks for building secure software products. Let's begin!

First things first: What is OWASP?

Before we discuss the Top 10, let's clarify what OWASP actually is. It's a detail that many people get wrong, even in the tech industry.

OWASP (Open Web Application Security Project) is a global organization that unites individuals who are committed to making software safer. Think of it as a worldwide community where security experts and developers collaborate to help others build secure software.

Here's what makes OWASP special: it's a non-profit organization that shares all its knowledge freely. They create guides, tools, and resources that anyone can use to make their software more secure. The Top 10 is just one of their many projects.

There's one common mistake I often hear in meetings and conferences. People refer to the "OWASP methodology"—but that's not quite right. It's like saying "Toyota methodology" when you mean Lean or Kanban. OWASP isn't a methodology; it's an organization that creates security guidelines and tools.

Now that we're clear on what OWASP is, let's look at their most famous project: the OWASP Top 10.

So What is Top 10?

The OWASP Top 10 is exactly what it sounds like: a list of the ten biggest security problems that affect web applications today. But there's more to it than just a simple countdown.

You can think of it as a ranked list, where the position of each security problem shows how severe it is for a typical organisation.

For example, when you see "A1: Broken Access Control" at the top of the list, it means this issue typically causes more trouble than "A10: Server-Side Request Forgery" which sits at the bottom.

Here's where things get interesting: while the Top 10 talks about risks, it's not really a risk list. Why? Because risk always depends on context.

Let me give you a real-world example: imagine finding the same vulnerability, like Cross-Site Scripting, in two different places:

1. In an internal tool that only employees can access.

2. On your public website that thousands of customers use every day.

Same vulnerability, same severity, but very different risk levels. (The public website version is much more dangerous because it's exposed to more potential attackers and affects more users.)

Top 10 Evolves Over Time

The security landscape is always changing, and the Top 10 changes with it. Since its first release in 2003, the list has been regularly updated to reflect new security problems and changing security patterns.

Let me show you how this works with a real example:

• In the 2017 version, Cross-Site Scripting (XSS) was listed as a separate problem.

• But in the current 2021 version, XSS became part of a larger category called "A3: Injections."

Right now, we're using the 2021 version, which makes it 4 years old in 2025. Based on the past, we're likely to see a new update this year (work is being done as you read this).

How is the Top 10 Created?

The process of updating the Top 10 is quite interesting. Here's how it works:

• First, the security community provides real-world data about vulnerabilities they find in their day-to-day work.

• Then, OWASP selects the 8 most common problems from this data.

• Finally, the community votes through an open survey to add 2 more issues.

You might wonder: Why not just pick the top 10 most common issues? Well, even with data from over 200,000 vulnerabilities (that's what they had for the 2021 version), some important problems might not show up often enough in the statistics.

So think of the community survey as a safety net. It helps catch new or emerging security threats that might not yet show up in the data but could be major problems in the future.

There is no one Top 10 list

The Top 10 tries to reflect reality in most cases, but different types of applications face different security challenges. For instance:

• A PHP application might face different security issues than a Ruby on Rails application.

• Different development teams tend to make different types of security mistakes.

This is why the Top 10 is skewed towards enterprise-level technology stacks (that’s where the data is coming from), and it’s also why you can create custom Top 10 lists.

Indeed, OWASP itself has created additional Top 10 lists for different needs. Examples include:

• OWASP Top 10 for APIs

• OWASP Top 10 for Serverless Applications

• OWASP Top 10 for Large Language Models (like ChatGPT)

Think of these specialised lists as focused security guidelines for specific types of applications. While the main Top 10 works as a good general benchmark for a typical web application, these specialised lists can help you focus on security issues most relevant to your specific domain.

From Awareness Tool to Industry Standard?

The OWASP Top 10 has an interesting story. It started as a simple way to help developers and managers understand security vulnerabilities affecting web applications. But over time, it became something bigger—many organisations now treat it as an industry standard.

Here's the catch, though: the Top 10 isn't meant to be a complete security standard. Even its creators are clear about this. When organisations need a proper security standard, OWASP itself recommends using the Application Security Verification Standard (ASVS) instead.

The creators also suggest that organisations should:

• Use ASVS for detailed security requirements.

• Build a complete security program using the Software Assurance Maturity Model (SAMM).

By the way, we will cover both of them in the future. Subscribe below & stay tuned! ☺️

Subscribe now

Top 10 2021: The List Itself

We've talked a lot about what the Top 10 is, but let's get to the heart of it: what are these top security problems? Here's the current list from 2021, with plain English explanations of each:

1. Broken Access Control involves managing who can access what in your application. A good example is when users can view other users' private data simply by changing numbers in the URL. This is known as Insecure Direct Object Reference and is one of the most dangerous and common vulnerabilities in web applications.

2. Cryptographic Failures cover problems with how applications protect sensitive data. A classic example is when applications use outdated methods like MD5 or SHA-1 to store user passwords. Yes, there are much better ways to do that!

3. Injections happen when applications don't properly check what users type in. The well-known Cross-Site Scripting (XSS) falls into this category, allowing attackers to inject and run malicious JavaScript code on your website. It's worth noting that this particular problem has been haunting web applications for over 20 years and still persists.

4. Insecure Design refers to security flaws that exist in the basic design of applications. These problems run deeper than simple coding mistakes—think of a banking app that doesn't properly verify fund transfers. This is about fundamental design flaws rather than bugs at the implementation layer.

5. Security Misconfiguration occurs when security settings aren't set up correctly. Often, these are just default settings that nobody changed. These problems affect both applications and their infrastructure, such as when security headers are misconfigured or missing entirely.

6. Vulnerable and Outdated Components—a fascinating category that doesn't directly involve application code but rather the libraries that applications use. This problem can affect everything: front-end, back-end, and even the infrastructure serving them.

7. Identification and Authentication Failures is a broad category that involves problems with how users prove who they are. In extreme cases, vulnerabilities in this category can have truly catastrophic consequences, such as allowing attackers to bypass login completely.

8. Software and Data Integrity Failures involve problems with keeping code and data safe from tampering. A common example is when applications load external scripts from CDNs without verifying their integrity, potentially allowing attackers to inject malicious code.

9. Security Logging and Monitoring Failures refer to the issue of not maintaining proper records of activities within your application. A typical example is when applications fail to detect and block automated attacks, leaving them vulnerable to sustained malicious activity.

10. Server-Side Request Forgery (SSRF) is the only stand-alone vulnerability on the list, rather than a broad category. It occurs when attackers can make your server send requests it shouldn't, potentially exposing internal systems to attacks from outside world.

Here's something crucial to understand: while earlier versions of the Top 10 focused on specific vulnerabilities, the 2021 version mostly deals with broader categories of problems (except for SSRF at number 10).

This makes the list more comprehensive but also makes it impossible to definitively say an application is "free from OWASP Top 10 problems" because each category contains many specific vulnerabilities. Some categories, like Insecure Design, are so broad that they're almost philosophical in nature. Authors seem to agree:

Internal Structure

When we look inside, we'll see that each position from the Top 10 list has a broader description that contains the following sections:

1. A brief summary of the problem.

2. A broader, but not too broad, description of the given problem (for example, a short list of situations that can lead to the application being vulnerable).

3. A concise description of how to prevent the given problem.

4. Several examples of typical attacks (i.e., exploitation of vulnerabilities) in the form of scenarios.

5. References, most often to other OWASP projects such as the previously mentioned ASVS, but also to Cheat Sheets Series or Proactive Controls.

6. And finally, a list of CWEs that are part of the given problem class.

And you might wonder: What is CWE? CWE (Common Weakness Enumeration) is a separate project from the MITRE organisation, whose authors set themselves the goal of enumerating all specific, individual weaknesses that can occur in software.

Are There Any Alternatives?

We’ve covered the most important aspects of the OWASP Top 10, and perhaps now you're wondering, "Are there any alternatives?" Of course! There are definitely similar non-OWASP lists.

A good example is the HackerOne Top 10, which is a Top 10 created by the world's largest Bug Bounty platform, through which almost 200,000 reports of actual vulnerabilities pass annually (along with probably the same amount of false positives…).

Another well-known market list of common security problems affecting all kinds of software is the CWE Top 25. As I mentioned earlier, a significant advantage of CWE is its attention to the level of abstraction. Each weakness listed in the CWE Top 25 is a specific issue, unlike the OWASP Top 10, which represents a class of problems. On the other hand, a downside is that it groups all weaknesses together regardless of application type.

For example, in the latest CWE Top 25 (which also changes over time), you’ll find Use-After-Free. This is fine, but this weakness is typical for applications written in low-level languages such as C++ (e.g., browsers). So, if you’re working in web development—and that's where most work is done—then Use-After-Free doesn't really concern you because it doesn't affect you.

Next Steps

Now that you know what the Top 10 is, you probably have a question forming in your head: How can it be used? I'll give you two directions:

1. The Top 10 works great in building basic awareness about web application security problems among development teams. Here I talk the talk and walk the walk—OWASP Top 10 is one of the foundations of my own training programmes with which I have had great success in Poland.

2. The Top 10 can be used in early development phases, e.g., as an attack library in Threat Modeling sessions and/or as a checklist baseline during the implementation phase in the Secure Code Review process. There are better tools for both these areas, but the Top 10 can definitely be a starting point there.

That's all for today folks. Best regards, and see you next week!

P.S. If you found this article useful, please pay the fee by sharing it with someone you know will also benefit from it. Thank you very much! 🙏

My first deep dive lands next week. Looking forward to sharing actionable insights on creating resilient, secure-by-default systems with you!