December 12, 2010

There are currently no logon servers available to service the logon request - The bane of rouge firewall rules



I always dreaded seeing this msg because in my experience, there can be different reasons that cause this msg and the solution might not be so outright. Usually this also means you will need to use the server's local administrator account to login for troubleshooting.

My main experience is if I can still ping the DNS server and gateway from the affected server, an unjoin followed by rejoin of the server to the domain (the computer account might need to be deleted) will resolve the issue.

There was however another time when I saw this error msg after trying to rdp into a server. I also could not ping the DNS server or any other servers outside of the same segment from the local server. However other servers within the same segment, including the gateway, could be pinged.

This was a weird case. I restarted the server a few times, but the situation persists. 

As this server lies in the DMZ zone, my next guess was something wrong with the firewall, maybe a firewall rule. I checked with the network guys whether any incoming/outgoing traffic to the server is blocked. No blocks.

Thinking that the HP teamed network adapter might be corrupted, I unteamed the 2 physical adapters and manually assigned IP address to each one, to no avail. 

It did not seem likely, but I went down to the datacenter with one of the network guys to try changing the physical ports connected to the server. No use.

We then plugged a cable directly from our laptop to one of the server connected ports and tried to ping again the DNS server and other servers outside of the DMZ segment. No change.

All this while, the network guy was also monitoring traffic from his switches. He saw all our ping and reply traffic to the server, but ping traffic from the server to other servers never saw any reply.

Our troubleshooting was getting nowhere except getting us more and more confused. In a last ditch attempt, I asked the network guy to give me another free DMZ IP within the same segment for testing. This time we were able to ping the DNS server!

Turns out that there were firewall issues at work, I am not a network guy but will try to explain this. 

The network team had implemented some rules where specific network segments should belong to which particular switches, so traffic from network A is assigned to Switch A, and will be denied at Switch B.

One of the guys had wrongly assigned the IP of this server to a switch that only accepts some other network segments. This caused the traffic from this server to be routed to that switch, only for the switch to deny it to go through.

After a switch of the firewall rules, the server was once again able to communicate to the DNS server and everything went back to normal.



No comments:

Post a Comment