There is an issue I have at work, something I’ve written about before in my logs that I’ve found a solution for that I feel I can blog about. I can’t really talk about the why behind all of this, but I can share a technical explanation of how I am addressing this problem. It’s a half-thing, bear with me.
At Western, I’m very interested in the number of open TCP connections that a workstation has open at any one time. I don’t care what state the connection is in, ACK_WAIT to any of the others, if there is a line, I want to know about it. Specifically I want to know how many lines there are. Mac OSX is based on Darwin, and Darwin is based on BSD – so you get a shell to work with when you start Terminal.app. There is a lot of power in the command line interface and once you get the hang of it, it’s really quite useful.
So remote stations, at least two of them I have turned on “Remote Login” in their Sharing applet in System Preferences which enables the machines SSH servers to answer incoming connections. I can use SSH to call up a command line window to those remote stations, feeding them commands. I have done this for a long while for our servers in the office but this is the first time I’ve seriously done this for workstations. So, with this connection established I want to collect the number of TCP connections that machine has established. On the command line there are lots of pieces to get this to work:
First, you need a loop structure so that the command happens regularly:
while true; do [command]; sleep 60; done
This will run a command every 60 seconds and it will never end unless I send a Control-C character which represents “Break” to the shell.
At first I just needed to count how many connections. You get this number, or at least an approximation of it this way:
netstat -p tcp|wc -l
That calls netstat to list out all the TCP connections, which then I pipe, using the pipe character ‘|’ to another command called wc, which calculates word-counts. I make wc ignore words and just count lines by using the -l switch. I don’t really care what other stations my targets are communicating with, just a count of how many. And yes, technically the SSH connection inflates this by at least one connection, it’s not intended to be forensic.
But something was missing. I need a date stamp. In BSD, there is a command called date, and you can give it a format so you can make date write out the Hours, Minutes, and Seconds the way you want to see them, but date has an annoyance to it. The command date always inserts a ‘newline’ character at the end, so what you’d get is a date, a new line, and your count. It’s okay, but it’s annoying. It would be far better to get rid of that newline character altogether. Enter in the ‘tr’ command, which translates characters. In this case, we tell tr to just delete the newline character, so ask date for the right sort of date, have tr nail off that newline at the end because it’s annoying and…
while true; do date '+%H:%M:%S '|tr -d '\n'; netstat -p tcp|wc -l; sleep 60; done
This outputs a very nicely formatted report on a remote workstation. So now I have datestamps, connection count levels, and when the count gets to a certain number and things happen, I can be faux-psychic.
UPDATE: Apparently I just can’t leave well enough alone. Seeing a slow parade of numbers trot by is rather dull when all I really want to know is when these numbers say, get over 70. So…
while true; do test "$(netstat -p tcp|wc -l)" -gt 70 && (date '+%H:%M:%S '|tr -d '\n'; netstat -p tcp|wc -l;); sleep 60; done