The New Black Box Is Sitting in Your Server Room
InfoWorld's recent piece on controlling AI token costs nails the industry's mood: we spent a decade building entire FinOps departments just to decipher the cloud bill, and just as we finally learned to stop leaving idle compute instances running, generative AI introduced "an infinitely more opaque, faster-moving layer of spend." The deeper problem, the article argues, isn't the price per token — it's attribution. When API calls are wrapped in layers of automated agents and prompt templates, your application becomes a black box that consumes capital to spawn tokens.
If you run servers for a living, re-read that paragraph and swap one word: the black box isn't only in someone else's cloud. It's on your racks.
The RAG pipeline behind the new "AI summarize" button in your helpdesk? It's a Node service on RAG-APP-01 calling an LLM API through three chained agents. The vector store? A Postgres instance with pgvector on VECDB-01 that nobody has opened since it was stood up in an afternoon. The internal chatbot? Ollama on the spare GPU box that used to be accounting's workstation — "just a trial," so it never made it into monitoring.
When that stack misbehaves, it burns two currencies at once: tokens on someone's invoice and resources on your servers. FinOps can eventually tell you what you spent. Your job is to answer the question they can't: what exactly spent it, on which server, and is it about to take something else down with it?
That's an infrastructure monitoring problem. It always was.
The Problem in Depth: Nobody Registered the AI Stack as Production
Here's how this plays out in real environments — four scenarios every sysadmin and MSP tech will recognize.
Scenario 1: The retry storm. Your helpdesk vendor ships an AI summarization feature. At 02:00, the upstream LLM API starts rate-limiting. The vendor's retry logic has no effective backoff, so the agent fires the API 40,000+ times before morning. Server-side, that means the app's worker process pinned at 100% CPU for six hours, a log directory that grows 30 GB overnight, a volume that hits 92%, and the reporting service sharing that host quietly timing out. Cost-side, token spend triples for the month. Who notices first? Nobody on your team. The uptime checker shows the server as "up" — ping green, agent reporting. The first signal is either the invoice or a ticket that says "reports are slow."
Scenario 2: The inference box that swapped itself to death. Someone loads a second, larger model into Ollama next to the first. RAM is exhausted, the host starts thrashing swap, and every chatbot response goes from 3 seconds to 45. The service eventually dies. Users report it at 09:40: "the chatbot is broken." Nobody monitored that box because it was a trial — but three internal apps were already quietly depending on it.
Scenario 3: The vector database that ate the disk. The nightly embedding job appends to a collection with no retention policy — 2 GB a week, every week. Six months later the volume is full: WAL can't checkpoint, embeddings fail silently, and the backup job on that host starts failing. The backup failure alerts in your backup console while the disk pressure sits invisible in your uptime tool, because the two systems don't talk. Two alerts, two tools, zero correlation.
Scenario 4: The 3 AM scheduled task. "Embedding sync" runs at 03:00, spikes CPU above 90% for two hours, and collides with the backup window. Backups fail. The task's Last Task Result reads 0x1 — which means nothing to anyone until a tech opens Task Scheduler manually at 14:00 on a Tuesday.
Why the gaps exist. AI features ship through app teams, not server provisioning. Nobody adds them to monitoring scope, documents the service names, or defines thresholds. Legacy tooling answers "is the server up?" — not "what is this server doing, for which application, at what cost?" And the stack is sprawled: uptime in one tool, process and service detail in an RMM, tickets in a separate helpdesk, logs nowhere in particular. Attribution requires correlating process-level telemetry with schedules, dependencies, and user complaints. Disconnected point tools turn that into a manual archaeology project.
What it costs you. Detection time measured in end-user complaints (40+ minutes, minimum). Attribution time measured in hours (which VM even hosts this thing?). SLA misses on perfectly healthy services that shared a host with an AI workload. Techs burning half a day tracing a loop. MSPs juggling 12 tabs across 5 tools per client. And when the meeting happens — "why did token spend triple?" — there's no data, because nothing was measured at the layer where the work actually ran.
How AlertMonitor Turns the Black Box Into a Dashboard
AlertMonitor's whole model is one platform, one alert stream, one pane of glass for servers, services, applications, Windows workstations, and scheduled tasks. That architecture is precisely what the attribution problem demands.
Register the AI stack as first-class citizens. The Ollama service, the RAG app pool, the pgvector Postgres instance, the embedding scheduled task — they get real-time monitors in the same stream as your domain controllers and file servers. No more "trial" boxes sitting outside the watch perimeter.
Alert on the resources AI actually consumes. Disk 80% warning and 90% critical on VECDB-01. Sustained CPU above 85% for 20 minutes on a specific process on RAG-APP-01. Service-stopped alerts on the inference host. Scheduled task failures and runs that exceed baseline duration. When the retry storm starts at 02:00, the right person is paged at 02:04 — with the process name attached — not at 09:40 by an end user.
Correlate alerts with tickets automatically. Because AlertMonitor includes an integrated helpdesk, the CPU alert and the three "chatbot is slow" tickets land in the same console, stacked on top of each other. Attribution stops being archaeology and becomes a glance.
See the blast radius before you act. Network topology mapping shows that RAG-APP-01 depends on VECDB-01 and the inference API. You know what else breaks before you restart anything.
Close the loop with RMM and patch management. That GPU box on accounting's old desk? It's enrolled, patched, monitored, and on the compliance report like every other endpoint — because it lives in the same platform, not a fourth tab.
The old workflow: user ticket → remote into an unknown server → hunt for the process → dig through logs → guess → meanwhile the disk filled and backups failed.
The AlertMonitor workflow: alert at 02:04 (process CPU anomaly plus abnormal log growth) → open the alert, see related tickets and the topology map → kill the loop or restart the service → done in 15 minutes, invoice impact contained, no 09:40 ticket flood.
That's detection moving from 40+ minutes to seconds, and attribution from hours to minutes. For an MSP, it means catching Client C's runaway AI integration before their invoice arrives — and before their outage becomes your midnight.
Practical Steps: Get Attribution This Week
1. Inventory your AI workloads like any other service. Ask app owners, check firewall logs for calls to LLM API endpoints, and map each AI feature to a server, a service, a run account, and a schedule. If it's not in your monitoring scope, you can't attribute it — or control it.
2. Attribute CPU and memory to a process, not just a server. When a host runs hot, this tells you who's burning it:
Get-Process |
Sort-Object CPU -Descending |
Select-Object -First 10 Name, Id, CPU,
@{N='WorkingSetMB';E={[math]::Round($_.WorkingSet64/1MB,1)}},
Path | Format-Table -AutoSize
3. Verify the inference service is running and set to auto-start:
Get-Service -Name 'Ollama' | Select-Object Name, Status, StartType
4. Find what's eating the disk. Runaway log growth is the fingerprint of an agent loop gone wrong:
Get-ChildItem 'D:\Apps\Summarizer\logs' -Recurse -File |
Sort-Object Length -Descending |
Select-Object -First 10 FullName,
@{N='SizeMB';E={[math]::Round($_.Length/1MB,1)}}, LastWriteTime
5. Sweep disk usage across every AI host — MSP techs, this works per client with one line changed:
$servers = 'RAG-APP-01','VECDB-01','INFERENCE-01'
Invoke-Command -ComputerName $servers -ScriptBlock {
Get-PSDrive -PSProvider FileSystem |
Select-Object @{N='Server';E={$env:COMPUTERNAME}}, Name,
@{N='UsedGB';E={[math]::Round($_.Used/1GB,1)}},
@{N='FreeGB';E={[math]::Round($_.Free/1GB,1)}}
}
6. On a Linux inference host, check GPU pressure and what the service actually logged overnight:
nvidia-smi --query-gpu=utilization.gpu,memory.used,memory.total --format=csv
systemctl status ollama --no-pager
journalctl -u ollama --since "2 hours ago" --no-pager | tail -n 50
journalctl --disk-usage
7. Check the scheduled tasks that move your AI data:
Get-ScheduledTask -TaskName 'Embedding Sync' | Get-ScheduledTaskInfo |
Select-Object TaskName, LastRunTime, LastTaskResult, NumberOfMissedRuns
8. Then put all of it under real monitoring in AlertMonitor. Add the hosts, create service and scheduled-task monitors, set the thresholds above, route alerts to on-call with automatic ticket creation, and map the dependencies so the next anomaly arrives with context instead of a ping time.
The InfoWorld article's five keys are largely FinOps discipline: tag everything, allocate the spend, measure the value. But none of those keys turn if you can't see the machine doing the work. Attribution starts at the infrastructure layer — with monitoring that knows which service, on which server, burned the CPU, filled the disk, and spawned the tokens. Get that right, and the AI bill becomes just another line item you can explain. Get it wrong, and you'll keep learning about your most expensive workloads from your users and your accountant.
Related Resources
AlertMonitor Infrastructure & Server Monitoring AlertMonitor Platform Overview Book a Demo Infrastructure & Server Monitoring Resources
Is your security operations ready?
Get a free SOC assessment or see how AlertMonitor cuts through alert noise with automated triage.