CALL US: (323) 804-0098 or (805) 205-4533

DNS problems slow down manufacturing

A manufacturing company with multiple worldwide sites had recently consolidated all their servers at the headquarters datacenter. Rather than use dedicated hardware or software load balancers, they decided to use “round robin” DNS load balancing. Users at a remote manufacturing site were complaining that the network was slow because it took a significant amount of time to access and download bar code label files which were required during the manufacturing process.

The server group at the data center insisted that there were no problems with the application or database servers and the network team had already checked all the network infrastructure and WAN links for over utilization and errors. The network engineer decided to investigate the traffic to and from the application server cluster and in doing so noticed an unusually high level of UDP traffic which was predominantly attributed to the Domain

(or DNS) protocol. Knowing that the server cluster used DNS load balancing both for application server as well as database server load balancing, she decided to check for correct operation of the DNS services. Using the portable analyzer, she ensures that the primary DNS server is correctly resolving some of the server names, but noticed that there are multiple DNS servers being queried, and that the time taken to resolve the names is relatively slow.

She then captured all packets containing the DNS protocol and quickly determines that in some cases, the primary DNS server provides a negative response to the application server request which subsequently, after a long timeout, makes additional requests to other DNS servers until the name is resolved thereby adding a significant amount of traffic and time which resulted in an apparent application slowdown. Reconfiguration of the primary DNS server solved the problem.

This scenario illustrates how to use the portable analyzer to identify what was perceived as an application performance problem but was caused by incorrectly configured network services.