NXDOMAIN is a negative response by a resolver telling that a domain does not exist. A NXDOMAIN response can be also triggered by a query to a subzone of a zone for which a DNS server is authoritative.
In the former case, a flood of queries relating non-existent domains is called a random NXDOMAIN attack, and when an authoritative is attacked with a flood of queries related to non-existant subdomains, we name it a phantom subdomain NXDOMAIN attack.
The flood is normally generated by a whole Botnet where originating IP addresses are more difficult to identify (e.g. hundreds of thousands of hacked ADSL modems).
In any case, the effect is that of a Distributed Denial of Service, that exhausts DNS server resources like CPU cores or it fills up the cache causing either swapping or excessive I/O, or in the best case slowing down the responsiveness of the DNS server.
Another variant of NXDOMAIN is the NXRRSET attack for both recursive and authoritative, that queries e.g. for an IPv6 address (RR of type AAAA) that doesn't exist. So, the non-existance of a Resource Record queried in excess is also a form of Denial of Service.
Yet another variant is the NXNSAttack that originates in fake domains delegated to fake nameservers with random names and no IP addresses (no glue records. This attack is meant to exhaust the resources of resolvers as well as parent domain nameservers.
Remember that the whole Internet DNS System is nothing but a hierarchy of delegations starting at the ROOT Nameservers.
NXDOMAIN attack mitigation by Planisys DNS Firewall
There are a number of techniques that are put in place by Planisys DNS Firewall to mitigate these kinds of attacks.
One of the most effective ways to evade these attacks is by fostering the use of DNSSEC, as it contains an inherent caching method of delegation information through NSEC records that makes negative answers much more immediate and almost immune to floods.
This is also known as Aggressive Use of DNSSEC-Validated Cache (RFC 8198)
Another technique involves a Machine Learning model trained on benign and Bot-like datasets called Random Forest, that permits the recognition of botnet patterns during a Flood attack, and thus being able to block them for certain periods of time until the attack becomes useless.
Yet another simple measure is the real-time sampling of server logs to identify an excessive number of negative responses and timeouts, and trigger reconfiguration actions such as increasing negative cache timeouts (Cache Pollution prevention) and decreasing query timeouts to avoid too many threads hanging and waiting for answers (Dynamic Recursive Timeouts).
Planisys makes also extensive use of controlled rate-limit thereby taking excluding internal networks that are known to be safe.
DDoS attack variants
DDoS stands for Distributed Denial of Serivce. It is used to describe attacks that can slow down the nameserver or even fill entire fiber links when the DNS is not correctly configured.
The DNS Flood Attacks can be like the formerly described NXDOMAIN attacks, but also attacks using legitimate existing domain names.
On the other hand, DDoS Reflection Attacks are any DDoS attacks that use a spoofed origin IP address that becomes flooded with responses.
There could be also DDoS Amplification Attacks which is a variant of the Reflection Attack where the attacker obtains a lot of data with a simple query, e.g. asking for ANY or for DNSSEC responses (which are larger by a factor of 10x than normal DNS responses), in general seeking responses that could make up a huge bandwidth volume in responses.
In any case, a Botnet would attempt to flood recursive or authoritative nameservers by sending UDP packets and getting responses that could be 50 times bigger than the query packet.
DDoS attack mitigation by Planisys DNS Firewall
As previously mentioned, Planisys makes extensive use of controlled rate-limiting rules thereby excluding internal networks that are known to be safe.
When the firewall detects a pattern of "too many" queries it also forces the presumably attacking client to switch from UDP to TCP thereby slowing down naturally the response flow, this is also known as slip and consists of setting the TC bit (Truncate) in a response.
Attack pattern detection is being done by analyzing the server behaviour in realtime, producing a dataset that could possibly match a trained dataset through ML (Machine Learning) techniques and thereby triggering an alert to reconfigure as it recognizes that is under attack.
Planisys uses nameservers on different networks routed through several high-capacity fiber links in the middle of the Internet Backbone to avoid being flooded, as is the case of offices or providers with only one or two links.
For DNS tunneling to happen, there has to be a domain with malicious information, and an infected computer.
The infected computer will eventually retrieve Command and Control information, e.g. pieces of a malicious script to be later assembled, by querying specific Resource Records of the Malware Domain.
There is also the possibility that an attacker is executing an Out of Band or Exfiltration attack, where through e.g. SQL Injection is stealing and sending out valuable internal information.
DNS Tunneling attack mitigation
The first instance for DNS tunneling mitigation is the RPZ Database of hundreds of thousands of possibly dormant or active domains that are being filtered and redirected to a Blackhole Server, thus blocking the dialog between the Malware and its C&C server.
The second instance is the pattern recognition made of very large domain names and many TXT records, that could also be used for Data Exfiltration in the opposite direction. The pattern would match an Planisys DNS ML dataset that has been trained with a tool such as Iodine to tunnelize information in the DNS packets, an alarm will be triggered and the traffic stopped.