Interesting news at Network World today: apparently, the San Francisco network administrator that held the city’s network hostage and is still in jail; in an interesting turn of events he apparently installed a “rogue device” on the network:

The device, referred to as a “terminal server” in court documents, appears to be a router that was installed to provide remote access to the city’s Fiber WAN network, which connects municipal computer and telecommunication systems throughout the city. City officials haven’t been able to log in to the device, however, because they do not have the username and password. In fact, the city’s Department of Telecommunications and Information Services (DTIS) isn’t even certain where the device is located, court filings state.

OK, this can happen; someone installs a device on your network, and you can’t find it. But in this case I’m a bit surprised, and I can understand how the network could be “held hostage” by a single network administrator. Just read this:

When investigators attempted to log in to the device, they were greeted with what appears to be a router login prompt and a warning message saying “This system is the personal property of Terry S. Childs

OK, read that again. What does this tell you? That they do know where the device is! Seriously, here are some tips for the San Francisco network administrators. 

 

  • First, try documenting your network. Seriously: it is useful for more than one person to know the structure of your network, and to know all relevant passwords. And if you don’t know the passwords, good documentation and backup procedures will help you recover. 
  • Typical Cisco hardware makes it really easy to reset passwords while retaining your configuration. Since they’ve already spent $182.000 on Cisco consultants, I assume that is their major hardware supplier. That must make this page worth millions! 
  • And now onto the rogue device. They managed to get a login prompt, so they must have an IP address for it.
  • They must know to what location that address is routed. If they don’t, a simple traceroute will show this when no MPLS network is in use.
  • When the right network segment is located, you can do an ARP lookup from any machine in that network to find the MAC address for the rogue device
  • Then log on to a random ethernet switch in that location and find out on which port that MAC address is seen. Follow the cable in that port. 
  • If that leads to another switch, keep doing this until you find the device.
  • If it’s on a wireless segment, get an EtherScope to find it. 

To sum this up, finding rogue devices is not difficult. The problem is most organizations don’s know there is one. If you know there is one, find it’s IP address using tools such as nmap, and once you have the IP locating it is a piece of cake.