15 May 2008
Perl-Fu: Automating ILOM SSH Sessions
Posted by aastaneh under: Uncategorized .
We at USF Research Computing take pride in automating tasks as much as possible. We write script after script, tool after tool, to make our lives easier so we have more time to conquer the real challenges. One of the tasks we would like to automate with the new cluster on the way is talking to the administrative interfaces on the nodes. ILOM, or “Integrated Lights-Out Manager”, allows us to tell the machine to do real neat things remotely, like reboot and prepare to PXE. ILOMs have their own network interface and are accessible via SSH.
Brian and I thought- cool. Let’s script these operations through SSH then, and make our lives easier. If we can automate these tasks, it would be a lot easier to manage maintenance of 120 new machines.
More, after the jump.
Usually, when you wish to remotely execute a command with SSH, you just tack whatever you want on the end of the ssh command, and it will execute the command on the remote host and return the results to you. However, not all systems support this. Notably, ILOM interfaces that you find on switches and some Sun hardware, do not support this kind of thing:
ssh root@ilom1 ls
Password:
Invalid operation
Invalid operation? But why, you ask? You can ssh into the host and execute commands there, but you can’t push a command out to be executed? What gives?
Simple. SSH daemons can operate in two modes- interactive and non-interactive. Interactive mode is when you log into the host and type commands there. Non-interactive is when you pass commands like the failed example above. The problem is, some ssh daemons, mostly on switch hardware and other network appliances, do not support the non-interactive mode. Which means, that if you want to admin these devices, you have to login to each of them and type in the commands, one by one.
Wrong.
Brian had a really cool idea. What if we wrote a program that initiated an interactive ssh session and we passed commands to it for it to execute? In short, the program would connect like a human would with his ssh client.
Enter Net::SSH::Perl, which is a module that allows you to initiate ssh sessions, and has all the features your standard ssh client has. We could just start an interactive session with it, and execute a whole bunch of commands through the script. One problem though- interactive sessions expect the user to type in the password into the TTY. That’s lame. How can we avoid that? It isn’t like we have sshkeys built into the host, and even if they could, they won’t when we first use them. I wouldn’t want to set that up for 120 machines by hand.
Basic UNIX knowledge tells us that every program has STDIN(program input), STDOUT (program output), and STDERR(out-of-band diagnostic messages/errors). So, in our case, what we need is to start a separate process and push our password and our commands through STDIN, and read our output through STDOUT/STDERR.
Perl saves the day again, with a module called IPC::Open3. It allows us to create those handles and attach them to a separate process for us to manipulate. In our case, we just want to fork. The parent process will be responsible for sending commands and aggregating output, and the child will be responsible for actually initiating the connection and invoking those commands and reading the output. We will also use IO::Select to manage reading from STDOUT/STDERR.
Final outcome: we can execute commands and process it’s output. We wrote a script just to print the mac address of the ILOM:
7:0 devel # ./ssh_test
MAC Address: 00:14:4F:20:D6:90
Here’s some code for you to play with: ssh_test Edit to your heart’s content.
A WARNING: This piece of perl does stuff that the designers of SSH did not intend. This script involves storing a password, and if you do not give this file proper permissions, you are guaranteeing a way to get rooted. You’ve been warned. (Might I suggest chmod 700 with chown root:root?)
With that said, Happy Hacking! 
3 Comments so far...
aastaneh Says:
24 May 2008 at 6:26 pm.
Apparently you can also accomplish the same goal using Net::Telnet; but instead of using a perl sub as the child, the ssh binary is called instead. Our solution’s advantage is that it’s all perl-based. To each his own, I suppose.
biffsocko Says:
31 October 2008 at 10:39 pm.
most networks don’t allow telnet
brs Says:
3 November 2008 at 12:10 am.
You wouldn’t be using Net::Telnet to connect (or nc/telnet as the backend), you’d use SSH as the backend and Net::Telnet to handle console I/O. Its perfectly doable.
Leave a Reply
You must be logged in to post a comment.