When One Oracle 0-Day Becomes Three: Keeping the Alert Storm Off Your On-Call Rotation

Oracle closed Cybersecurity Awareness Month the way it usually does: with trouble. As The Register reported, attackers are actively probing a bug in Oracle E-Business Suite, and the PeopleSoft 0-day that ShinyHunters claimed turns out to be not one vulnerability but three. If you run EBS or PeopleSoft — and thousands of IT departments and MSPs do, because that's where payroll, GL, and HR live — you already know what the next 72 hours look like.

Not because the news breaks anything. Because the news detonates your alerting. Your RMM starts firing patch-status alerts per device. A security-news webhook dumps every headline into Slack. A vulnerability scanner re-scans and re-flags. And somewhere in that pile is the one signal that matters — the EBS concurrent manager that didn't come back after the emergency patch restart, or the /u01 mount filling with opatch logs.

If your on-call rotation is about to learn about Oracle's vulnerabilities one duplicate page at a time, this post is for you.

The Problem: Your Tools Treat Every Alert Like It's the Only Alert

Walk through what actually happens when a vendor story like this breaks, in a mid-size IT shop or an MSP:

• The RMM (Ninja, ConnectWise Automate, N-able) has a patch policy. Each server missing the patch generates its own alert, per scan. Thirty-eight EBS servers, three overnight scans — that's 114 alerts, none of them related to each other as far as the tool is concerned.
• Standalone monitoring (PRTG, Nagios, Zabbix, SolarWinds — whatever predates the RMM) fires its own service and threshold alerts into a completely separate console.
• The security feed webhook pushes fifteen headlines into #security. Three of them describe the same Oracle situation. The channel doesn't know or care.
• The helpdesk — ConnectWise Manage, Freshservice, HaloPSA — sees none of it. No ticket. No SLA clock. If a business-critical ERP goes down overnight, the first written record of it is tomorrow morning's user tickets.

Then the emergency patch window opens at 10pm. Services restart. Monitoring, which has no concept of a maintenance window, fires "service down" for every single restart. Thirty-eight more alerts. On-call receives roughly 130 pages between 10pm and 2am describing what is really only three facts: patch missing on 38 servers, patch applied during the window, and one server where a service never came back.

That last one is the incident. It's alert #87 in the queue. The on-call tech, who stopped trusting their phone two hours ago because every ping looked identical, batches "review alerts" for the morning shift. MTTA on the real problem: seven hours. The ERP's users discover the outage before your team does, and the morning becomes triage theater — sorting 130 notifications, opening tickets by hand, reconstructing a timeline from four tools' logs.

Why does this keep happening? Because the tools were never designed to share a single event model. The RMM was built to manage endpoints. The monitor was built to check thresholds. The helpdesk was built to track tickets. Each treats its own output as its own problem. Deduplication, correlation, suppression, and escalation exist as afterthoughts — email filters and Slack channels. The architecture is siloed, so your technician becomes the integration layer, using memory and a spreadsheet to decide which of 130 pings is a fire.

The business cost is measurable and ugly:

• MTTA inflates from minutes to hours, because techs rationally ignore low-trust alerts.
• SLA reports are fiction, because the ticket was created after the outage, not at detection.
• Burnout becomes a retention problem. The senior sysadmin who quit "for a quieter role" usually means "for a rotation that didn't page them 40 times a night."
• And the failure mode everyone fears: a real, revenue-impacting alert buried under vendor-news noise, acknowledged at 9:15am by someone holding a coffee and a grudge.

How AlertMonitor Solves This

AlertMonitor was designed around a specific insight: alert fatigue isn't a volume problem — it's a signal quality problem. You don't fix the Oracle-news scenario by turning alerts off. You fix it by making every alert carry context and routing it by what it actually means.

• Context in every alert. Each alert carries the device, the client (for MSPs), what changed, and what healthy looks like. "Oracle EBS Apps service down on ebs-app01, client AcmeCorp — was running 11 minutes ago, zero downtime in 30 days" is an alert a human can act on. "CHECK_NRPE: critical" is a coin flip.
• Smart deduplication. Thirty-eight servers missing the same patch don't produce thirty-eight alerts. They produce one incident with 38 devices attached, and the incident updates as devices become compliant. Your overnight page count for this scenario drops from ~130 to something a person can actually read.
• Maintenance window suppression. Set a window for patch night (say 22:00–02:00). Planned service restarts during the window are suppressed — recorded in the timeline, not lost. If a service fails to come back up after its restart, that alert fires. It's the only one that does. Buried alert #87 becomes the single page of the night.
• Multi-level on-call routing. Not every signal deserves a phone call. Informational items (patch missing, scan findings) go to the dashboard and a daily digest. Warnings go to the assigned tech's email and Slack. Critical — an ERP service down outside a maintenance window — pages on-call with a 15-minute acknowledgment timeout, then escalates to tier 2, then to the IT manager. You configure the ladder once; the tool climbs it, not your staff.
• Alerts become tickets, automatically. A critical alert opens a helpdesk ticket with full context attached. The SLA clock starts at detection, not at a human noticing. Remote sessions, script runs, and response notes log back into the same record. Your Monday SLA report is finally true.
• One platform instead of five tabs. Acknowledge the alert, remote into the box, run a validation script, verify the patch, close the ticket — one interface, one timeline. No correlation by memory, no copy-pasting device names between tools.

In the scenario above, on-call gets one page instead of 130 — the service that failed to restart. Acknowledgment takes about 90 seconds because the alert already names the server, service, and client, and links to the incident. The morning shift reviews a digest, not a queue. The helpdesk already has the ticket. Nobody rebuilds the timeline at 9am from four tools' logs.

Practical Steps You Can Take Today

1. Measure your noise this week. Count pages and notifications for one rotation and sort them: duplicates, informational-dressed-as-critical, maintenance-window noise, real. Most teams find 80%+ of pages fall into the first three categories. That number is your business case.

2. Stop paging for information. Patch-missing, scan findings, and capacity trends belong in a digest, not a 2am SMS. Route by severity before you tune anything else.

3. Pre-stage post-patch validation so 2am-you doesn't rely on memory. Before the Oracle patch window, drop a validation script on your jump host:

# Post-patch service validation for Oracle EBS / PeopleSoft app servers
$servers  = @("ebs-app01","ebs-web01","ebs-conc01","psft-app01","psft-prcs01")
$services = @("Apache2.4","OracleEBSApps","OracleProcessScheduler","PSAdmin")  # adjust to your stack

foreach ($srv in $servers) {
    foreach ($svc in $services) {
        $state = (Get-Service -ComputerName $srv -Name $svc -ErrorAction SilentlyContinue).Status
        if ($state -ne "Running") {
            Write-Warning "$srv  $svc  $state"
        } else {
            Write-Host "$srv  $svc  OK"
        }
    }
}

And confirm the database listeners are accepting connections after the restart:

$dbHosts = @("ebs-db01","psft-db01")
foreach ($db in $dbHosts) {
    $up = Test-NetConnection -ComputerName $db -Port 1521 -WarningAction SilentlyContinue -InformationLevel Quiet
    if ($up) { Write-Host "$db listener on 1521: UP" }
    else     { Write-Warning "$db listener on 1521: NOT RESPONDING" }
}

4. Watch the disk. Oracle patching is a disk event — opatch logs, EBS patch logs, and staged patches have filled more /u01 mounts than any growth forecast. Add this to the pre-patch checklist:

for host in ebs-app01 ebs-db01 psft-app01; do
  echo "=== $host ==="
  ssh "$host" "df -h /u01 | tail -1"
done

5. Verify the patch actually landed — one line per host, not one scan queue:

export ORACLE_HOME=/u01/app/oracle/product/19.0.0/dbhome_1
$ORACLE_HOME/OPatch/opatch lsinventory | grep -q "35648110" && echo "Patch 35648110: APPLIED" || echo "Patch 35648110: MISSING"

6. Encode the escalation ladder once. In AlertMonitor, a policy for ERP-critical systems looks like this:

escalation_policy: oracle-erp-critical
levels:

• severity: informational # patch missing, scan findings action: dashboard + daily digest email
• severity: warning # degraded performance, single failed check action: notify assigned tech (email + Slack)
• severity: critical # ERP service down outside maintenance window action: push + SMS to on-call ack_timeout: 15m escalate_to: [tier2-oncall, it-manager] deduplicate_by: [client, device, alert_type] suppress_during_maintenance_windows: true

**7. Set maintenance windows for every planned change.** Not just patching — reboots, certificate rotations, failover tests. Suppression with a timeline record beats "I forgot to disable monitoring" every time.

**8. Test the ladder with a synthetic critical alert.** Page yourself on purpose. If tier 2 doesn't receive the escalation at minute 15, you want to learn that on a quiet Tuesday — not during the next Oracle event.

Cybersecurity Awareness Month ends. The next advisory, the next 0-day, the next "actually, it's three vulnerabilities" is already on the calendar. The difference between a rotation that dreads those headlines and one that handles them isn't headcount — it's whether your alerting tells the truth: one incident, full context, one page, one ticket.

## Related Resources

[AlertMonitor Alert Management & On-Call Operations](https://alertmonitor.ai/platform/alert-management)
[AlertMonitor Platform Overview](https://alertmonitor.ai/platform)
[Book a Demo](https://alertmonitor.ai/contact)
[Alert Management & On-Call Operations Resources](https://alertmonitor.ai/blog/alert-management)