Useful Isilon Commands for Troubleshooting

Here are some some useful Isilon commands to assist you in troubleshooting Isilon storage array issues.

 

Grep the log for stalled drives on the isilon cluster

     cat /var/log/messages |grep -o 'stalled: [0-9,*:]*'|sort |uniq -c

(Stalled drives are bad, and can cause cluster problems. you could also run this command on the individual nodes /var/log/restripe.log )

Grep the log for stalled drives on the isilon cluster for month of Sept

grep 'Nov ' /var/log/messages |grep -o 'stalled: [0-9,*:]*'|sort |uniq -c

Use this on the restripe.log

  grep 'Nov ' /var/log/restripe.log |grep -o 'Stalled drives are \[[0-9,*:]*\]'|sort |uniq -c 

When reviewing the results of the stalled drives it is important to note that the drive numbers listed is the logical drive number and not the bay number.  You need to run the command “isi devices” on the node with the suspect drive to determine what bay the drive is actually in.

Display the SMART error log of all the drives on a given isilon node:

isi_radish -a|less

Display the current isilon Flexprotect Policy

isi get /ifs

Display the current isilon node hardware status:

isi_hw_status

Display the status of the isilon node network config

isi config

then while in the config utility

 status 

Display this list of alerts in wide format

 isi alerts -w

Start/Stop/Resume/Pause Restriper jobs

 isi restripe pause 
isi restripe start 
isi restripe stop 
 isi restripe resume -i

Display the drive status of a given isilon node

     #for node 3
     isi devices -d 3  

Display the SAS drives Physical Monitoring stats for errors

     less /var/log/isi_sasphymon.acc

Test Active Directory connections from all isilon nodes

     isi_for_array wbinfo -t

To find an open file on Isilon Windows share

     isi_for_array -q -s smbstatus | grep 
  then find the PID from the results and  then run this to get the user
     isi_for_array -q -s smbstatus -u| grep    to get the user

Note: The isi_for_array command runs the command on all of the nodes. This command will ask for the user’s password so that it can login to the other nodes and complete the command. When passing the results of a “isi_for_array” command to another command such as grep (like the example above) will require the user password so that it can be passed to the other nodes. There is no prompt for the password so you must enter it on the next line and press enter to get the results of the command.