Around Jan 18th,
Several folks are reporting odd queries hitting their DNS servers at a steady rate of about two per second. The queries invariably ask for the name server of the domain “.” (NS query for a single dot). Since “.” is a query for the root name servers, it has a very short query packet but a pretty long answer. Our current theory therefore is that this is a denial of service (DoS) attack in progress, where the DNS servers are used as “amplifiers” and unwittingly flood the (spoofed) source by providing a long answer to a system which never asked.
http://isc.sans.org/diary.html?storyid=5713
I noticed one of the servers I am responsible was getting hit with this query about four times a second. The server was already configured to reject the query, but with each one it logged a warning. named (bind DNS server) and syslogd were now the second and third largest resource users on the server.
I’m sure there are other system administrators dealing with the same problem, so I’m going to outline the process I went through to eventually drop them with iptables (the server is not protected by a hardware firewall, so I had to use the linux firewall iptables). Note, the system is Ubuntu Server 7.10, so the paths used here should be accurate for related systems.
First of all, to survey the damage:
thayward@sea1:~$ tail /var/log/syslog Jan 24 11:52:28 sea1 named[23890]: client 206.71.158.30#43845: query (cache) './NS/IN' denied Jan 24 11:52:29 sea1 named[23890]: client 206.71.158.30#43332: query (cache) './NS/IN' denied Jan 24 11:52:29 sea1 named[23890]: client 206.71.158.30#23849: query (cache) './NS/IN' denied Jan 24 11:52:29 sea1 named[23890]: client 206.71.158.30#65188: query (cache) './NS/IN' denied Jan 24 11:52:29 sea1 named[23890]: client 206.71.158.30#45150: query (cache) './NS/IN' denied Jan 24 11:52:30 sea1 named[23890]: client 206.71.158.30#1362: query (cache) './NS/IN' denied Jan 24 11:52:31 sea1 named[23890]: client 206.71.158.30#39496: query (cache) './NS/IN' denied Jan 24 11:52:31 sea1 named[23890]: client 206.71.158.30#27892: query (cache) './NS/IN' denied Jan 24 11:52:31 sea1 named[23890]: client 206.71.158.30#52019: query (cache) './NS/IN' denied Jan 24 11:52:32 sea1 named[23890]: client 206.71.158.30#36766: query (cache) './NS/IN' denied
You can see named goes to a lot of effort to get these denials logged. While I figure out how to drop the packets, let’s set bind’s log level to something that won’t log this attack. The system’s bind configuration file is located at /etc/bind/named.conf. I added these lines:
logging{
channel default_syslog {
syslog daemon;
severity notice;
};
};
This changes the default syslog behavior of bind from info to notice. This stops logging of the denial messages.
Okay, now that we’ve avoided the problem, let’s try to actually block it. The simple solution is to block the source IP address. In this case, that is spoofed as it is the IP address of the victim. Blocking this will keep me out of the attack for now, but when they choose a new victim I could begin sending more refusals.
To block the source IP, I issued this command for each of the offending IPs:
sudo iptables -I INPUT -s 206.71.158.30 -j DROP
This calmed things down a bit, but still wouldn’t protect me against future attacks. For this I would need to do some investigation and learn how to block the packets more specifically.
I started a packet capture on the server with tcpdump.
thayward@sea1:~$ sudo tcpdump -i eth0 -s 1500 -w dump4
eth0 specifies the network adapter to capture from; size of 1500 tells tcpdump to cature the full packet (by default, tcpdump only captures the first 68 bytes); and dump4 is the file I saved the capture to.
I downloaded dump4 and loaded it into Wireshark. A filter expression of “dns” filters out what I need.
I knew these packets would be port 53 because they are DNS queries. I also knew they would be UDP because it is a short DNS query. What I wanted to know was the length of the packet (the tiny packet length is what makes this attack work). I experimented with some of the values I found in Wireshark, I found one that matches iptables’ length rule:
Under Internet Protocol, I found Total Length to be 45, and used this in the iptables rule:
sudo iptables -I INPUT -p udp --dport 53 -m length --length 45 -j DROP
This will drop all root DNS queries.
To test it out:
dig . NS @yourserver.com
If dig just hangs without returning a result, the request was dropped and it’s working.