It's Always DNS: Why Green Dashboards Miss the Most Common Failure on Your Network

It's 9:12 AM and the tickets are flooding in: the intranet is down, Teams won't connect, the ERP portal times out. You ping the domain controller — replies. You ping 8.8.8.8 — replies. Your monitoring dashboard shows every device green. Forty-five minutes later, someone finally runs nslookup against the internal resolver and discovers the DNS Server service on DC02 has been hung since 6:47 AM. The server never went down. The service just stopped answering queries — and your entire monitoring stack was built to catch servers going down.

The Register recently ran a column titled "No shame in the Domain Name game: DNS is doomed, literally," and the summary underneath it is the most honest incident report in IT: it's always DNS. It's a running joke because it keeps being true. And it keeps being true because most monitoring tools structurally cannot see a DNS failure. They can ping the server. They cannot tell you it stopped resolving names for half the office.

Why "Up" Doesn't Mean Working: The Problem in Depth

ICMP tells you the host is alive. It tells you nothing about port 53.

DNS fails in ways that keep the host perfectly pingable. The Windows DNS Server service, BIND, or dnsmasq can hang with the process running and port 53 open while queries time out or return SERVFAIL. Device-up monitoring reports healthy; users report catastrophe. Or a firewall firmware update silently drops the outbound forwarding rule for port 53: internal names resolve, external names don't. Half-up, half-down is the worst scenario to triage, and a ping-based tool calls it "green."

Stale diagrams and undocumented resolvers

DC01 was decommissioned in March, but half your DHCP scopes still hand it out as the first resolver in option 6. Every client eats a three-second timeout before failing over, and you've been collecting "the network feels slow" tickets for months without connecting them. The branch office resolver is a VM on a host nobody remembers. The network map is a Visio diagram from 2019 and a quarterly scan spreadsheet. When DNS breaks, you're not troubleshooting against reality — you're troubleshooting against folklore.

Tool sprawl means nobody connects the dots

Your RMM pings the server. Your standalone network monitor watches switch ports. Your helpdesk collects tickets. The pattern — 23 tickets in 8 minutes, all "can't reach internal apps," all from VLAN 20 — is only visible if a human jumps between three consoles and mentally joins the data. MSP technicians live this at scale: 12 tabs across 5 tools to support one client, with per-client resolver configs that exist only as tribal knowledge.

What it actually costs

• 30–60 minutes of wrong-layer triage before anyone suspects DNS, because the dashboard said the network was fine
• One root cause generating 30+ tickets, each with its own SLA clock running while techs chase the wrong layer
• 2 AM pages for "outages" that turn out to be resolver timeouts — which trains your team to ignore alerts
• MTTR that doubles because identification time dominates, and end users who conclude "IT is down again" every single time

How AlertMonitor Makes "It's Always DNS" a 90-Second Problem

Real queries, not pings

AlertMonitor tests DNS with actual queries against every internal resolver, plus a public baseline. A hung-but-pingable resolver triggers an alert in under a minute — while a ping-only monitoring tool still reports everything green. Latency degradation (the "everything feels slow" mode caused by a dead first resolver in the DHCP list) shows up as a trend you can catch before users do.

A live map that shows the blast radius

AlertMonitor continuously discovers and maps every device on the network — switches, firewalls, access points, printers, IP cameras, and unmanaged endpoints — using SNMP, ARP, and active scanning. The map is never stale, because it isn't a document; it's the live state of the network. When the DNS service on DC02 stops answering, the alert doesn't just say "host down." It reports a DNS query failure on 192.168.10.11 with full network context: the 14 monitored endpoints downstream of SW-CORE-01 that stopped resolving within two minutes, and the segments affected. And when a new device appears squatting on the old resolver's IP, an alert fires instantly — not at the next quarterly scan.

One console instead of five

The alert links to the affected devices on the topology map and to an incident your team can work immediately. RMM actions run from the same screen: restart the DNS service, flush client caches, correct DHCP option 6. The helpdesk tickets already flooding in get linked to the same network event, so 30 duplicates collapse into one incident with one timeline. That's also how your SLA reporting finally stops lying: identification, remediation, and ticket resolution live in one system instead of three.

For MSPs: the map is the documentation

Per-client topology and resolver inventory maintain themselves. When the on-call tech gets the 2 AM page, they open the client's live map, see exactly which resolver failed and which sites are affected, and act — no digging through a spreadsheet last updated by someone who left the company.

The old way: ticket → ping tests → green dashboard → manual nslookup → DHCP audit → discover hung service. Forty-five to sixty minutes, minimum.

The AlertMonitor way: synthetic query fails → contextual alert names the resolver and affected segments → confirm on the live map → one-click service restart → linked tickets updated. Roughly 90 seconds to identify, minutes to resolve.

Practical Steps: Audit Your DNS Weak Spots Today

1. Find stale resolvers in your DHCP scopes. If a decommissioned DC shows up here, fix option 6 today — this is the "everything is slow" ticket generator:

Get-DhcpServerv4Scope -ComputerName DHCP01 | ForEach-Object {
    $scopeName = $_.Name
    Get-DhcpServerv4OptionValue -ComputerName DHCP01 -ScopeId $_.ScopeId -OptionId 6 -ErrorAction SilentlyContinue |
        Select-Object @{n='Scope';e={$scopeName}},
                      @{n='DNSServers';e={$_.Value -join ', '}}
}

2. Test every internal resolver with a real query, not a ping:

$resolvers  = @('192.168.10.10','192.168.10.11')
$testRecord = 'fileserver.corp.local'

foreach ($r in $resolvers) {
    try {
        $answer = Resolve-DnsName -Name $testRecord -Server $r -Type A -DnsOnly -ErrorAction Stop
        "[$r] OK -> $($answer.IPAddress -join ', ')"
    }
    catch {
        Write-Warning "[$r] FAILED: $($_.Exception.Message)"
    }
}

3. Restart a hung DNS service remotely instead of walking to the server room:

Invoke-Command -ComputerName DC02 -ScriptBlock {
    Restart-Service -Name DNS -Force
    Get-Service -Name DNS | Select-Object Name, Status
}

4. On Linux resolvers and forwarders, dig with tight timeouts — if the internal query times out while the public one answers, your forwarder path or the internal service is the problem:

dig @192.168.10.10 fileserver.corp.local A +time=2 +tries=1 +noall +answer
dig @8.8.8.8 google.com A +short

5. Check the local resolver state on Linux endpoints during triage:

resolvectl status | head -n 20
systemctl status systemd-resolved --no-pager

6. Flush client caches once the server is fixed, so stale negative answers don't keep tickets alive after the root cause is gone:

Clear-DnsClientCache

7. Then make sure you never troubleshoot this blind again. In AlertMonitor, add a synthetic DNS check for every internal resolver and one public baseline, scope the alert to the topology segments each resolver serves, and link it to your standard incident template. Next time port 53 hangs, the alert fires with the affected VLANs named — before the first user ticket lands.

DNS being "doomed" is a joke. Being blind to it doesn't have to be your reality.

Related Resources

AlertMonitor Network Monitoring & Visibility AlertMonitor Platform Overview Book a Demo Network Monitoring & Visibility Resources