A Strange Morning at BT — and a Very Familiar Morning for Your NOC
Recently, BT email customers reported something surreal: hundreds of unsolicited password reset PINs arriving in rapid bursts, sometimes dozens back to back. BT's official position was that accounts are secure — but for anyone manning a service desk, the root cause on BT's side was almost beside the point. The event generated exactly the kind of load that breaks IT teams: a flood of identical, alarming, user-facing noise.
If you run a help desk — internal or MSP — you already know what that morning looks like on your side:
- 9:12 AM: First ticket lands. "User is getting BT reset codes they didn't request."
- 9:25 AM: Twelve more, across five different clients.
- 9:40 AM: A technician is copy-pasting the same reassurance into ticket #27 while the queue keeps growing.
- 10:05 AM: An actual alert — a file server's D: volume at 91% — is buried under the noise and gets noticed an hour late.
This is the ticket storm problem, and it's one of the most quietly expensive operational failures in IT support. It never makes outage headlines because nothing "goes down." But it eats hours, blows SLAs, buries real incidents, and burns out the exact people you can least afford to lose.
Here's the uncomfortable part: most tool stacks don't just fail to prevent this — they actively amplify it.
The Problem in Depth: Why a Third-Party Blip Becomes a 270-Ticket Morning
Let's model the BT scenario against a realistic mid-size MSP: 45 managed clients, mail hosted by a mix of Microsoft 365, Google Workspace, and regional providers like BT. A provider-side event like this touches users across 20+ clients. Say 270 users contact you. Now watch what your tooling does with that.
1. Your RMM treats every event as the first event
RMM platforms — NinjaOne, ConnectWise RMM, Datto RMM, N-able — evaluate conditions per device, per client. There's no cross-tenant, cross-signal correlation layer saying "this is the same incident happening 270 times." So if mail-related monitors fire at all, you get one alert per device or per client. Twenty clients equals twenty "separate" incidents that are actually one. And if they don't fire — far more likely, because BT's infrastructure isn't in your monitoring estate — the first signal you receive is a human being on the phone.
Neither path gives you the one thing you need in the first five minutes: the blast radius.
2. Your monitoring doesn't see the services users actually feel
Your monitors watch CPU, disk, services, ping, backup jobs — the stuff inside your estate. But the user's experience lives upstream: their mail provider, their identity provider, their connectivity. An entire client workforce can be effectively disrupted in their inboxes while every server you monitor shows a fat, green "healthy."
Traditional infrastructure monitoring answers "is my server up?" It does not answer "can my users do their jobs?" — and during a BT-style event, that second question is the only one anyone is asking.
3. Monitoring and helpdesk live in different universes
This is the structural gap. Your alerts live in the RMM. Your tickets live in the PSA — ConnectWise Manage, Autotask, HaloPSA, whatever it is. The two systems don't share state:
- Nothing in the ticket queue says "these 270 tickets are one incident."
- Nothing on the monitoring side knows that 40 tickets already exist about this symptom.
- Correlation happens the old-fashioned way: a senior tech reads ticket titles for 20 minutes and figures it out manually.
4. The math of a flood
Run the numbers on 270 identical tickets:
- Triage time: 3–4 minutes each just to read, tag, reassure, and route — that's 13–18 tech-hours of pure duplication before you even talk about resolution.
- SLA clocks: every ticket starts its first-response timer. Auto-acknowledgements might save the metric, but they won't save the client experience — and if your PSA starts clocks at creation, you're now fighting a second fire on the SLA report.
- Opportunity cost: every tech pulled into duplicate triage is a tech not doing patching, projects, or noticing the disk-space alert that just got buried.
- Morale: nothing demoralizes a technician like spending a Tuesday closing 250 copies of the same ticket. This is a top driver of help desk churn, and churn is expensive.
5. "Are we affected?" has no fast answer
When the provider says "accounts are secure," your clients don't take the provider's word for it — they ask you. Can your techs answer in five minutes with evidence ("zero reset events on your domain controllers — your local identity is quiet")? On most stacks, the honest answer is "give me an hour to check." That's an hour of client anxiety, and it's avoidable.
None of this happens because your team is bad. It happens because the tools were built as point solutions — one for devices, one for tickets, one for networks — with per-seat licensing that discourages integration and alert engines designed for single events, not storms.
How AlertMonitor Collapses a 270-Ticket Morning Into a 20-Minute Incident
AlertMonitor was built multi-tenant from day one, and it shows exactly where this scenario hurts. Here's the same BT-style event, run through the platform.
Step 1: The storm is detected and grouped automatically
AlertMonitor's intelligent alerting correlates events by signature across devices and clients. When matching events spike past a rate threshold — say, 20 related events across all tenants within 10 minutes — the platform groups them into a single storm alert with a live count, instead of firing 270 individual notifications.
One alert. One page. The on-call tech sees "Storm detected: 34 events / 12 clients / signature: auth-reset-flood" and knows in ten seconds that this is external and widespread — not a targeted attack on one client.
Step 2: One NOC view shows the pattern instantly
Because every client lives on one unified NOC dashboard, the tech immediately sees the same symptom across a dozen clients with nothing in common except their mail provider. Per-client isolation means they can drop into any single client's view without losing the big picture. Twelve client reports become evidence of one incident — not twelve investigations.
Step 3: The helpdesk is already part of the picture
The correlated alert auto-creates a parent ticket. Individual user reports — however they arrive — are linked as children automatically or with one click during triage. First-response SLA is satisfied once, centrally, with a status update pushed to all affected clients. When the provider confirms resolution, the tech closes the parent and the children close with it. One written client communication per client, from a pre-built template — not 270 individual conversations.
Step 4: Synthetic checks catch the upstream problem first
External service checks against the endpoints your clients depend on — SMTP, webmail login, DNS — mean AlertMonitor often flags a provider-side disruption before the first human ticket lands. Your mean time to acknowledge on the real problem drops from "whenever the phone rings enough" to roughly 90 seconds.
Old way vs. AlertMonitor
| Fragmented stack | AlertMonitor | |
|---|---|---|
| Time to understand blast radius | 30–60 min of manual reading | Under 2 min, live count on one screen |
| Tickets generated | 270 individually triaged | 1 parent + auto-linked children |
| Pages to on-call tech | Per-client alert spam | 1 grouped storm alert |
| SLA handling | 270 clocks, manual mass-update | Handled once at incident level |
| Real-incident visibility | Buried | Intact — noise is grouped, not dropped |
| Client comms | Ad hoc, repetitive | One template, one send per client |
The headline number: a storm that would consume 13–18 tech-hours of duplication gets handled in roughly 20–30 minutes of coordinated response — and your alert-to-resolution time on genuine incidents stays fast, because the noise never reaches the queue as noise.
Practical Steps You Can Take Today
1. Build a five-minute blast-radius check
When a provider-side reset storm hits, your first job is proving whether your own environment is implicated. This counts password reset events on your domain controllers over the last hour:
# Count password reset events across all domain controllers in the last 60 minutes
$Since = (Get-Date).AddHours(-1)
$DCs = (Get-ADDomainController -Filter *).HostName
$report = foreach ($dc in $DCs) {
$count = (Get-WinEvent -FilterHashtable @{
LogName = 'Security'
Id = 4724, 4723 # password reset events
StartTime = $Since
} -ComputerName $dc -ErrorAction SilentlyContinue).Count
[PSCustomObject]@{
DomainController = $dc
ResetEvents = $count
}
}
$report | Sort-Object ResetEvents -Descending
Zero events on every DC? You can tell every affected client, with evidence, that their local identity environment is untouched and the event is upstream. That one answer defuses dozens of anxious conversations.
2. Verify your own mail flow isn't the problem
If you run any on-prem Exchange or SMTP relay in the path, confirm transport is healthy and the queue isn't ballooning:
# Check the Exchange transport service and surface any deep mail queues
Get-Service -Name "MSExchangeTransport" -ErrorAction SilentlyContinue |
Select-Object Name, Status
# Run from the Exchange Management Shell (or a remote session to the server)
Get-Queue |
Where-Object { $_.MessageCount -gt 50 } |
Select-Object Identity, DeliveryType, MessageCount, LastError
A running transport service with clean queues is another line in your "we're not the problem" evidence pack.
3. Check upstream reachability in one line
Quick sanity check that an upstream mail host is even accepting SMTP connections:
# Verify an upstream mail provider is accepting connections on SMTP
timeout 10 bash -c 'cat < /dev/null > /dev/tcp/mail.provider.com/25' \
&& echo "SMTP endpoint reachable" || echo "SMTP endpoint UNREACHABLE"
4. Put your storm rules in writing — and in the platform
Define, before the next storm, what constitutes one. In AlertMonitor, that's a correlation rule: matching signature, N events, M minutes, evaluated across all clients, grouped into a parent ticket with child alerts suppressed. The concept, spelled out:
# Storm-detection rule: mass password-reset event
rule_name: auth-reset-storm
match:
signature: "auth.reset.request"
conditions:
threshold: 20
window_minutes: 10
scope: all_clients # evaluate across tenants, not per client
actions:
group_alerts: true
create_parent_ticket: true
route_to: noc-incident-channel
suppress_child_alerts: true
suppression_minutes: 60
The specific numbers are yours to tune. The principle isn't negotiable: a storm should page once, not N times.
5. Pre-write the mass-incident communication
Today, while nothing is on fire, draft the template: "We're aware of a widespread issue affecting [provider] mail users. Your local systems show no impact. We're actively monitoring and will update you by [time]." Attach it to your storm runbook ticket template in the helpdesk so the first responder sends it in one action. During a real event, that single paragraph buys more client goodwill than anything else you'll do all morning.
The BT PIN flood will not be the last mass third-party event your clients' users experience — providers have incidents, bulk processes misfire, and upstream change management fails. The MSPs that come out of those mornings looking calm and in control aren't the ones with the best crystal ball. They're the ones whose platform already understood that 270 identical events are one incident — and whose technicians spent the morning communicating instead of copy-pasting.
Related Resources
AlertMonitor MSP Operations & Team Efficiency
Is your security operations ready?
Get a free SOC assessment or see how AlertMonitor cuts through alert noise with automated triage.