The Hidden Security Cost of Self-Hosting Kimi, DeepSeek, and Qwen

Kimi, DeepSeek, and Qwen changed the economics of enterprise AI. Open Chinese models now run 60 to 90 percent cheaper than the leading US APIs, and the shift has already happened: Chinese open-weight models account for 46.4 percent of routed token usage on OpenRouter, ahead of US-origin models at 35.7 percent, up from just 4.5 percent a year earlier.

Teams can run reasoning, coding, and agentic workloads on their own laptops, GPU clusters, and private clouds. The appeal is obvious. Lower inference cost, deployment flexibility, and control over where data is processed.

Self-hosting an open-weight model does not remove risk. It transfers ownership of that risk to you. With a managed AI service, the model infrastructure sits behind the provider's security boundary. Once the model runs locally, the enterprise owns the model artifact, the inference runtime, authentication, network exposure, patching, guardrails, tools, credentials, and every agent connected to it.

The core risk is not that these models are Chinese. The risk is that powerful AI is being downloaded and operated as software infrastructure, usually outside the visibility of the security team. Kodem works on this layer directly: runtime intelligence built on user-space memory analysis, which reports what an application actually loaded and executed rather than what a manifest or a config file claims.

Local AI becomes shadow infrastructure within minutes

A developer can install Ollama, pull a model, and connect it to internal code or documents in minutes. A team can deploy vLLM on a cloud GPU instance without waiting for a central AI platform. A desktop agent can talk to a local MCP server without a single request crossing the corporate AI gateway.

This is not hypothetical. Cisco Talos scanned Shodan for ten minutes and found 1,139 publicly exposed Ollama endpoints. Of those, 214 answered model queries with no credentials at all, and 36.6 percent were located in the United States. Those are the deployments visible from the public internet. The ones inside the perimeter are harder to count and more numerous.

The resulting AI environment is frequently missing from inventory, outside approved network routes, weakly authenticated or unauthenticated, unpatched, connected to sensitive data, and running on a developer endpoint.

Finding the process is not enough. Security needs to know which model is loaded, which agent is calling it, what tools are connected, what data it can reach, and which controls are actually present.

The model supply chain is an execution path, not a file download

Self-hosted models are not always passive weight files. A deployment can load configuration, tokenizers, processors, custom Python modules, adapters, and secondary weights. Loading a model can mean loading executable software from an external repository.

CVE-2026-41523: a malicious model can execute code through vLLM

A 2026 vLLM advisory described an assert statement used as the only security check for activation-function loading. Under Python optimized mode (python -O or PYTHONOPTIMIZE=1), assertions are stripped and the check disappears. A malicious Hugging Face model could then trigger arbitrary code execution during initialization, with the privileges of the vLLM process. Fixed in 0.22.0. Details in CVE-2026-41523.

Model provenance is not only a governance question. It can become host compromise.

CVE-2026-47155: a pinned Kimi deployment can still load unpinned components

A second vLLM advisory showed that revision pins were not consistently applied to every artifact a model loads. The clearest example involved Kimi-Audio, where the primary model could be pinned to an audited revision while the Whisper config and weights in the same repository were loaded with no revision argument, always resolving from the default branch.

The advisory did not claim a confirmed compromise. It exposed the gap between intended and actual state. A deployment can look reviewed and pinned while its runtime behavior still depends on mutable artifacts. See CVE-2026-47155.

Configuration tells you what should be running. Security needs independent evidence of what actually loaded.

The inference runtime is a production attack surface

Local AI depends on fast-moving infrastructure: Ollama, vLLM, llama.cpp, model repositories, container images, plugins, and orchestration frameworks. These components now sit beside valuable data and powerful compute.

CVE-2024-37032: Ollama path traversal led to remote code execution

Wiz Research disclosed CVE-2024-37032, known as Probllama, in Ollama. A path traversal flaw in the model-pull flow allowed arbitrary file writes and could be escalated to remote code execution. Impact was most severe in exposed Docker installations running with elevated privileges.

This is a familiar vulnerability class inside a rapidly deployed AI stack, and it is not isolated. The vLLM advisory history includes a separate critical RCE in multimodal inference, which we covered in CVE-2026-22778, and llama.cpp carries its own advisory in CVE-2026-34159. Three inference runtimes, the same pattern.

Inference servers are no longer experimental utilities. They are production services and should be secured as production services. Attackers have noticed, as we documented in how attackers are gaining access to LLM inference.

The real risk appears when a local model becomes an agent

A local model on its own is one component. The risk compounds when it is connected to agents, tools, credentials, data sources, and other agents.

Consider a local coding agent that can read source code, access cloud credentials, search internal documentation, invoke MCP tools, delegate work to a sub-agent, and run shell commands.

Each capability is defensible on its own. The exposure lives in the combination.

A model can receive untrusted content from logs, documents, repositories, or web results, then carry that context through a delegation path into a privileged action. The final shell command is local. The delegation happens in process. No gateway request ever exposes the full chain. This is the same reachability problem that runs through the whole AI application stack, and it is why local AI security cannot stop at model inventory or network inspection.

Gateways, DLP, and EDR each stop short of the local inference path

AI gateways, DLP, CASB, EDR, SIEM, and distributed tracing all produce useful evidence. Each also has a structural boundary, and a self-hosted model tends to fall outside it.

The complete risk path fragments wherever registration, instrumentation, routing, or propagated context ends. A self-hosted model is precisely the case where all four end early.

Runtime evidence shows what a self-hosted model actually loaded

Every gap above has the same shape. The declared state and the loaded state diverge, and the tooling reports the declared one. Kodem's runtime intelligence reads the user-space memory of running processes, so it reports what a workload actually loaded and executed. For self-hosted AI, that is the difference between the deployment you reviewed and the deployment that is running.

Vulnerable inference runtimes, prioritized by what actually executes

What this delivers today on covered workloads and endpoints:

• Vulnerable inference runtimes, prioritized by execution. Runtime-powered SCA identifies affected Ollama, vLLM, and llama.cpp components, then separates the instances where the vulnerable code is actually loaded and reachable from the ones where it is not. On CVE-2026-41523, that is the difference between every vLLM install and the vLLM installs running in optimized mode.
• Exploitation signal at runtime. Path traversal and RCE against an inference server is behavioral, which is Application Detection and Response territory: anomalous subprocess activity, unexpected file writes, outbound connections a model server should never make.
• Remediation verification. After a runtime is patched or replaced, Kodem re-inspects the running state to confirm the vulnerable component is no longer loaded. Not a ticket closed, evidence that the specific condition is gone.
• Triage that reflects reality. The same evidence feeds vulnerability management, so a critical advisory in a runtime you do not actually load does not outrank one you do.

"When you self-host a model, the interesting target stops being the model and becomes the process that loaded it. Attackers do not care which lab trained the weights. They care that an inference server is running with elevated privileges, resolving artifacts from a mutable branch, and holding credentials for everything the agent can reach. None of that is visible in a config file."

Pavel Furman, Co-founder and CTO at Kodem

A guardrail you cannot see in the process is a claim, not a control

Self-hosting an open-weight model quietly removes a safety layer most teams never accounted for. A hosted API arrives with vendor-side filtering already in the request path. Kimi, DeepSeek, or Qwen on your own vLLM node arrives with exactly the guardrails your team wired, and frequently that is none.

The two places teams look for an answer both miss it. A framework config or a provider console records the guardrail that was intended. An AI gateway can only enforce policy on traffic that reaches it, and local inference never sends any. For a self-hosted model, the guardrail question has to be answered inside the process.

This is the layer Kodem's Runtime AI Security works on. Reading the running process surfaces guardrails as they exist at that moment: which agents have one, its category and level, the action it takes, and the direction it covers. Direction is the detail that decides whether a control is real. An input filter tuned for prompt injection does nothing for an egress path that carries internal data to an external destination, and no config file will tell you which edge you actually protected.

The useful findings are the negative ones:

• Agents running with no guardrail at all. Including sub-agents created dynamically at runtime, which no code review ever saw and no inventory recorded. Unused or ungoverned AI capability is itself a finding.
• A guardrail on the wrong edge. An agent can be filtered on input and completely open on output, which reads as protected in a console and is not.
• A missing guardrail at the boundary where the risk should break. Untrusted content reaching a privileged tool, or sensitive data reaching an external destination through a delegation chain. The map is drawn from how the agent is wired, so it shows that the path exists and is reachable. It is not recorded traffic, and that is the point: the exposure is visible before it fires.
• A model bound off the approved list. Compare the models actually loaded in memory against the list your governance process signed off on. For an environment where developers can pull new open weights in minutes, this is the simplest control that holds.

None of these are Kodem-invented categories. They map to the OWASP Top 10 for LLM applications: untrusted content reaching the model is LLM01, sensitive data reaching it is LLM02, and unbounded egress through an over-permissioned agent is LLM06.

To be precise about status: this agent-aware discovery is an emerging capability built on Kodem's runtime intelligence. It is advancing through active development with design partners and is not yet generally available. The current focus is discovery, evidence, and prioritization.

Where this goes next: agent-aware control

Discovery is the first half. The second is acting on it, and the reason that follows naturally from runtime evidence is identity. Most controls see one workload with one identity, even when several decision makers live inside it: an orchestrator, its sub-agents, deterministic code, and occasionally an agent nobody declared. The same action carries different risk depending on which one originated it. Reconstructing the agent hierarchy from memory is what makes it possible to attribute an action to its agent of origin and to the delegation chain behind it, which is the difference between a blunt control and a targeted one.

That points toward agent-aware policy, enforcement per agent and per delegation, and remediation verified by re-inspecting runtime state rather than waiting for the event to recur. This is the product direction and a later phase, not something shipping today.

The reason it belongs on runtime evidence is structural. A gateway can only reason about requests it receives. A code scanner can only reason about what was written. Agents are wired dynamically, tool surfaces expand at runtime through MCP and other agent frameworks, and a sub-agent that is loaded but has never fired is invisible to anything watching activity. Reading the running process is the only place that information exists.

Self-hosted AI is an infrastructure decision, so govern it like one

Kimi, DeepSeek, Qwen, and the open-weight models after them will keep spreading across enterprise infrastructure. The economics are too strong and the barrier to deployment is too low. Policy will not change that: open weights, once downloaded, run offline.

So the question is not whether enterprises will run local AI. The question is whether security can answer what is loaded, what it can reach, and whether the last fix held, before someone else asks.

Self-hosting is a reasonable decision. Self-hosting without evidence of what loaded is not. Cutting inference cost by 60 to 90 percent is worth doing. It is worth doing with an inventory that comes from the running process rather than a survey.

With Kodem, enterprises can adopt self-hosted AI without accepting unmanaged risk. One platform, built on runtime intelligence: it discovers the models and inference runtimes actually loaded, identifies vulnerable infrastructure and untrusted artifacts, prioritizes by what really executes, guides precise remediation, and re-inspects the running state to confirm the specific condition is gone. As Runtime AI Security matures, the same evidence extends that loop to the agents, tools, and loaded guardrails around a local model, so a guardrail is something you can verify rather than something you declared.

Frequently Asked Questions

References

1. Cisco Talos: Detecting Exposed LLM Servers, a Shodan Case Study on Ollama
2. Wiz Research: Probllama, Ollama Remote Code Execution Vulnerability
3. vLLM Advisory: CVE-2026-41523
4. vLLM Advisory: CVE-2026-47155
5. llama.cpp Advisory: CVE-2026-34159