AI moved appsec into the design layer. Scale-ups are hiring for the wrong one.
The security problems AI brings to a product live in the architecture, not the programme — which means the fractional hire most scale-ups reach for is the wrong shape.
Every few years a platform shift lands that the security function has to absorb faster than it would like. Cloud was one. Mobile was one. Putting language models and agents into the path of real product decisions is the current one, and it is arriving faster than either of those did, because the barrier to shipping an AI feature is so much lower. A team that would never have stood up its own datacentre can wire an agent into production over a long weekend.
When a scale-up notices it is now shipping AI into its product and decides it should get serious about securing it, the reflex is to hire senior security help. The title that comes out of the HR system is “CISO” — usually fractional, because a Series B company isn’t going to put a full-time C-level security exec on the books. That reflex is understandable and, I think, slightly wrong. It’s worth being precise about why, because the gap isn’t about the quality of the people. It’s about what kind of problem AI security actually is.
What traditional appsec is, and what a fractional CISO is for
Application security, as a discipline, is largely a programme. You manage your dependencies and patch known vulnerability classes. You run SAST and DAST and pen tests against the OWASP Top 10. You get through SOC 2, you write the policies, you stand up incident response, you do vendor risk reviews, you report posture to the board. This is real, skilled, valuable work, and the fractional CISO market exists to do exactly it for companies that aren’t yet big enough to hire it full-time. At the right stage, for the right company, a fractional CISO is precisely the right call.
None of what I’m about to say is a knock on that work. It’s a claim that AI security is a different kind of problem, sitting at a different layer, and that the programme-building skillset doesn’t reach it — not because the people aren’t good, but because they’re answering a different question.
The three things AI changes, and they all land in the design
The trust boundary moves inside the system. In a classic web application the trust boundary sits at the request edge: you authenticate the caller, authorise the action, validate and sanitise the input, and everything inside is code you wrote and control. An LLM agent breaks that cleanly. The model takes natural-language instructions, and it cannot reliably tell your instructions from instructions that arrive inside the data it processes — a tool’s response, a retrieved document, a file a user uploaded. Prompt injection isn’t a bug you patch and close; it’s a structural property of putting an instruction-follower inside the trust boundary and then feeding it untrusted text. There is no equivalent of parameterised queries that makes the whole class disappear. You design around it, in layers, expecting each layer to fail in some cases.
The dangerous flaws are architectural, and architecture isn’t scannable. The mature appsec toolchain is built to catch known patterns in code and dependencies, and it still matters for the application around the model. But the failures that actually hurt in an agentic system live in the design. An agent handed broad service credentials that calls tools “as itself” will cheerfully read or write things the human driving it could never touch — it runs with the agent’s permissions and has no notion of the user’s identity. A gateway in front of a tool server that enforces rate limits but never checks consent. An orchestrator that treats model output as control flow without constraining the set of actions it can take. None of these show up in a scan. You find them by reading the design and asking where the trust assumptions quietly break.
Non-determinism breaks the assurance model. Security assurance has always leaned on repeatability: the same input gives the same output, and a test passes or fails. LLM-based systems are non-deterministic, and the failure mode that matters most isn’t a crash or a blocked request — it’s the model being perfectly agreeable and then confidently doing the wrong thing. You can’t pin that down with a fixed assertion. Assurance shifts toward eval-harness design: synthetic tasks, behaviour scored over a distribution rather than a single run, tests that inspect the tool-call trace and not just the final answer. That is an engineering discipline, and a fairly young one. “We test that it refuses the obvious bad prompts” is necessary and nowhere near sufficient.
And it all has to happen at speed
The reason all of this is urgent is that AI features ship fast and often arrive before anyone has labelled them security-relevant, because to the team building them it looks like an API call. The orchestration logic, the tool wiring, the identity decisions — the places where the design goes wrong — get made in ordinary engineering work, under ordinary deadline pressure, frequently without security in the room. The cheap place to catch a design flaw is in the engineering review, before it ships. The expensive place is a governance forum reviewing it a quarter later, after it’s load-bearing.
That timing is itself an argument for a different kind of hire. Catching design flaws before they ship means sitting in engineering reviews and being technically credible to the engineers in the room — not receiving a system description after the fact and mapping it to a control framework.
Why the programme skillset doesn’t transfer
So you have a class of problem that is architectural, unscannable, non-deterministic, and best caught at design time inside the engineering org. Ask what skillset that calls for and it isn’t the one the fractional CISO market is optimised to supply. Running a security programme and reviewing an agent’s architecture are genuinely different jobs. One is about governance, process, and posture across an organisation; the other is about reading a specific system and knowing, from having built things like it and watched them go wrong, where it will break. Asking a programme-builder to do design review is a category mismatch, not a competence failure — and the inverse is just as true. Most principal engineers would make poor heads of a security programme.
Regulatory literacy sits on top of this, and in regulated or regulated-adjacent domains it genuinely matters — knowing what an examiner will ask, what evidence a model-risk function needs, where the EU AI Act or SR 11-7 actually bite. But it sits on top of the engineering judgement, not instead of it. A regulatory answer to “is this agent design safe to ship?” that never engages with the design is theatre, and an expensive kind, because it produces paperwork that makes everyone feel safer without anyone being safer.
What the role actually is
The hire a scale-up shipping AI actually needs looks more like a principal security engineer who works fractionally than a fractional CISO. The deliverables are engineering deliverables: architecture reviews that name specific design changes an engineer can implement this week; threat models for the AI surfaces of the product that map agent capabilities to where things go wrong; controls that look more like middleware specifications than policy documents; eval-harness designs that test the failure modes that matter; and the minimum set of policy-shaped artefacts the board, the auditors and the regulators need — produced downstream of the engineering rather than instead of it.
Concretely, it’s the person who can sit in a review and say: split the orchestrator into a planner and an executor, give the executor a constrained tool allowlist, scope the planner’s access to read-only, pass the user’s identity through to the tool calls so the tool layer can enforce the user’s permissions rather than the agent’s — and be right about why each of those matters. That conversation is engineering from start to finish. The regulatory framing comes after, when someone needs to explain to a third party why the system is sound.
The market doesn’t have a clean name for this yet
The reason the title that gets posted is “CISO” is that it’s the senior security title that exists, and the HR system has a field for it. The job, increasingly, is something else — and the people answering the CISO ad are, mostly and reasonably, answering the older question it appears to be asking.
If you’re a scale-up putting AI into the path of decisions where being wrong costs money, trust, or a regulator, the useful move is to be honest with yourself about which problem you have. If it’s “we need a security programme,” hire for that. If it’s “we’re shipping AI features and we need someone to tell us what’s wrong with the design before we ship,” that’s a principal security engineer, whatever the job spec ends up calling it. The two overlap less than the shared word “security” suggests.