Ofcom Just Lived Out Your Worst On-Call Week
The Register reported this week that Ofcom is learning an uncomfortable lesson about the Online Safety Act: issuing fines is easy, collecting them is not. Platforms have worked out that they can comply just enough to avoid being blocked — the one punishment the regulator can actually enforce — and then quietly sit on the invoice. The enforcement machinery produces documents; the outcomes never arrive.
Every IT professional who has carried an on-call phone should read that story as a mirror. Your monitoring stack has been running the same playbook on your team for years. It issues alerts — thousands of them. Your technicians, drowning in noise, comply just enough to make the noise stop: bulk-acknowledge the queue at shift start, close notifications without fixing anything, mute the sensors that cry wolf. And the handful of alerts that actually mattered get their real response forty minutes later, when an end user calls the helpdesk because the file server is down.
Issuing the fine was never the hard part. Getting a human to act on it is. Same goes for alerts.
The Problem in Depth: You Built an Alert Generator, Not a Response System
What the current tooling actually does
Walk through a typical week at a 20-tech MSP or a mid-size internal IT shop running a stitched-together stack — PRTG or Zabbix for network monitoring, NinjaOne or ConnectWise Automate as the RMM, a separate helpdesk like Autotask or ConnectWise Manage, and email acting as the glue between all of them:
- Threshold alerting with no context. A PRTG sensor flips red and fires "Disk usage on FS-PROD-01 above 90%." That is all the technician gets. What grew — log files, a shadow copy explosion, a user dumping video projects? What does healthy look like; was this disk sitting at 88% for six months? Reconstructing the story means opening four tools, so mostly the tech doesn't. They acknowledge and move on.
- Cascade storms. A core switch flaps at 01:37. By 01:39 the monitor has generated 300+ downstream "device unreachable" alerts. The one alert that mattered is buried at position 214. Every default RMM monitor set produces these.
- Acknowledge-without-resolve culture. This is the comply-just-enough move. Acknowledging stops the page; nothing stops the underlying problem. Your metrics say the alert was handled in four minutes. Reality says the disk filled three days later and took file services down with it.
- Escalation that dies in email. Most escalation policies are a distribution list. After hours, nobody reads DL email. The escalation path exists on paper and nowhere else.
- No maintenance windows. Sunday night patching reboots 40 servers. Monday's on-call tech inherits 60 pages and bulk-acknowledges all of them — including the one genuine post-reboot boot failure.
Why the gaps exist
These are not implementation failures; they are architecture failures. Standalone monitoring tools were built as data collectors, not response engines. The alert is a row in a database plus an SMTP send — the tool's job ends at issuance. Getting an alert to a human, confirming a human saw it, and connecting that human to the fix requires the monitoring tool, the RMM, and the helpdesk to share a brain. When they are three products from three vendors, that connection is an email connector or a brittle API integration that strips context on the way through. The ticket that arrives reads "check FS-PROD-01 disk" with no device history, no baseline, no link. Response quality degrades to match.
What it costs you
- Mean time to acknowledge creeps from six minutes to forty-plus — not because techs got lazy, but because they rationally learned that most pages are noise.
- SLA reports nobody trusts. The SLA clock starts when the alert fires, but a human saw it forty-five minutes later. Helpdesk and monitoring each tell a different story, and producing one accurate report means a spreadsheet marathon.
- Burnout and attrition. An on-call tech woken five times a night for two false positives and three reboots stops treating pages as urgent — including the one that is on fire. Ask any MSP operations lead why they cannot keep level-one techs; 2am pages for scheduled tasks are a big part of the answer.
- Margin erosion. Senior engineers doing manual triage of noise is unbilled work. At scale, that is the difference between a profitable NOC and a cost center.
How AlertMonitor Closes the Loop
AlertMonitor was designed around a specific insight: alert fatigue is not a volume problem, it is a signal quality problem. You do not fix it by paging less or paging more — you fix it by making every alert worth answering.
Context on every alert. Each alert carries the device, the client, what changed, and what healthy looks like for that metric. The 01:40 disk alert on FS-PROD-01 shows the growth curve, the baseline, and the likely culprit before the tech opens anything. When every page answers "is this real, and how bad?", responding becomes faster than ignoring.
Smart deduplication and correlation. A flapping switch produces one alert that updates as downstream devices drop — not 300 rows. The on-call tech sees the root cause first, at 01:37 instead of 02:15.
Multi-level on-call routing that actually escalates. Escalation policies are configurable per client, per severity, per time of day: push notification at T+0, SMS at T+5 if unacknowledged, phone call at T+15, then the next person in the chain. Acknowledgment is a real gate, not a checkbox — an unacknowledged alert keeps climbing until a human owns it.
Maintenance window suppression. Define windows per client, device group, or policy. Patch reboots inside the window generate records, not pages. The Monday-morning bulk-acknowledge ritual disappears.
One platform, one loop. Because monitoring, RMM, helpdesk, and patch management live in one product, an alert becomes a ticket, launches a remote session, and can push a remediation — and when it resolves, the alert and ticket close together. SLA reporting is finally accurate because the data was never split across systems in the first place.
The same incident, two ways
The old fragmented way: 01:40 — PRTG emails "disk 90%". The email lands in a distribution list nobody monitors at night. 02:20 — a user cannot save to the file share and tickets it. 02:35 — the on-call tech's phone rings; he wakes up, opens the monitor, the RMM, and the helpdesk, reconstructs context, remotes in, and finds SQL backup temp files ate the volume. 03:10 — resolved. Total human response latency: ninety minutes, triggered by a user — not by the tool that knew first.
The AlertMonitor way: 01:40 — the deduplicated alert routes to the on-call tech's phone with full context attached. 01:43 — acknowledged; a ticket auto-creates with the alert timeline on it. 01:47 — a remote session launches from the alert; temp backups cleared, cleanup script scheduled. 01:55 — alert and ticket closed with resolution notes. The 2am page happened — because it was a real problem. It lasted fifteen minutes instead of ninety, and nobody in the office ever noticed.
Practical Steps You Can Take This Week
1. Audit your alert volume. Pull the last 30 days of alerts and group by rule. If your top 10 rules account for 70–80% of volume, you have found your noise sources. Most teams discover the monitor has been screaming about the same five things for two years.
2. Kill or tune the top offenders. Fixed thresholds on grow-and-stay-grown metrics like disk produce repeat pages forever. Replace them with scheduled checks that surface only exceptions. This PowerShell check prints nothing unless a server breaches the threshold — schedule it hourly and alert on its output instead of paging on every 1% increment:
# Disk usage check across servers - prints ONLY problem drives
$servers = Get-Content "C:\Monitoring\servers.txt"
$threshold = 85
$results = foreach ($server in $servers) {
Get-CimInstance -ComputerName $server -ClassName Win32_LogicalDisk -Filter "DriveType=3" -ErrorAction SilentlyContinue |
Select-Object @{n='Server';e={$server}},
@{n='Drive';e={$_.DeviceID}},
@{n='FreeGB';e={[math]::Round($_.FreeSpace/1GB,1)}},
@{n='UsedPct';e={[math]::Round(($_.Size-$_.FreeSpace)/$_.Size*100,1)}}
}
$problems = $results | Where-Object { $_.UsedPct -ge $threshold }
if ($problems) { $problems | Format-Table -AutoSize | Out-String }
3. Test your escalation path on purpose. Fire a fake critical alert at 02:00 on a quiet night and time how long it takes for a human to acknowledge. If the answer is "it went to email and was found at 08:30," your escalation policy is Ofcom's invoice — issued, never collected.
4. Set maintenance windows before your next patch cycle. Map every scheduled reboot and WSUS maintenance window to a suppression rule in your alerting first. Then verify that nobody got paged. If pages still came through, the windows are not wired to the alerts that matter.
5. Turn chronic service-restart pages into self-healing checks. If a monitor has paged you about the same service five times and the fix is always identical, automate the fix and page only on failure to recover:
# Self-healing service check: restart if stopped, surface only failures
$watched = "Spooler","wuauserv","MSSQLSERVER"
$servers = "SRV-APP01","SRV-SQL01","RDS-FARM-01"
foreach ($s in $servers) {
foreach ($name in $watched) {
$svc = Get-Service -ComputerName $s -Name $name -ErrorAction SilentlyContinue
if ($svc -and $svc.StartType -ne 'Disabled' -and $svc.Status -ne 'Running') {
try {
Invoke-Command -ComputerName $s -ScriptBlock { param($n) Start-Service -Name $n -ErrorAction Stop } -ArgumentList $name
Write-Output "$s / $name : was $($svc.Status), restarted OK"
} catch {
Write-Output "$s / $name : RESTART FAILED - escalate to on-call"
}
}
}
}
The same idea on the Linux side — one health script that stays silent unless something genuinely needs a human:
#!/bin/bash
# Health check: prints findings and exits 1 only when action is needed
THRESHOLD=85
findings=""
while read -r fs pct mnt; do
pct="${pct//[!0-9]/}"
[ "$pct" -ge "$THRESHOLD" ] && findings+="DISK: $mnt at ${pct}%\\n"
done < <(df -h -x tmpfs -x devtmpfs | awk 'NR>1 {print $1, $5, $6}')
for svc in nginx sshd; do
systemctl is-active --quiet "$svc" || findings+="SERVICE: $svc is not running\\n"
done
if [ -n "$findings" ]; then
echo -e "$findings"
exit 1
fi
6. Then move the whole loop into one system. Hygiene and scripts get you partway there. The remaining gap — alert-to-ticket context, real escalation gating, accurate SLA data, remote remediation launched from the alert itself — only closes when monitoring, helpdesk, RMM, and patching share one platform. That is the gap AlertMonitor exists to close, and it is the difference between a NOC that issues alerts and a NOC that collects on them.
The Takeaway
Ofcom's problem is not a lack of power to fine. It is that its one credible punishment — blocking — is so much costlier to wield than fining that platforms can safely ignore the fine itself. Your monitoring stack has the same structural flaw: generating the signal is cheap, so it does it constantly, while the actual mechanism of consequence — a rested, informed human who responds — was never engineered into the system.
Fix the loop, not the volume. Give every alert context, make escalation mean something, suppress the noise you created on purpose, and connect each alert to the tools that resolve it. Your on-call techs will notice within a week — because the phone stays quiet, and when it rings, it matters.
Related Resources
AlertMonitor Alert Management & On-Call Operations AlertMonitor Platform Overview Book a Demo Alert Management & On-Call Operations Resources
Is your security operations ready?
Get a free SOC assessment or see how AlertMonitor cuts through alert noise with automated triage.