NASLite Network Attached Storage

www.serverelements.com
Task-specific simplicity with low hardware requirements.
It is currently Mon May 05, 2025 4:58 pm

All times are UTC - 5 hours [ DST ]




Post new topic Reply to topic  [ 7 posts ] 
Author Message
 Post subject: Disk Status Script
PostPosted: Sat Sep 20, 2008 8:33 pm 
Offline

Joined: Sat Nov 19, 2005 6:39 pm
Posts: 633
Location: California
Hello all:

I wanted to create a log of temperatures on my NASLite's disk drives. Since I have another Linux system in the network, I figured I'd write a shell script for this task. NL updates "html" files which are the basis of the "Status Information Pages" (see page 21 of the Manual). They look nice when displayed in a browser, but I need a TEXT version to do what I want.

I am providing the below shell script for anyone's enjoyment ... hope you find it useful. I will expand it some more for my purposes, but as shown it provides a text version of the "Server Storage" page ... have fun with it. ( I am using "NASLite-2 USB v2.06 08-2007" ... have not tested other versions. )

I haven't automated the logging yet (at the NL update interval) ... working on that as time allows. And naturally many other uses for the "html" pages can be imagined ... an automated email perhaps if "errors" are recorded in the SysLog file ???

:) Georg

Code:
#!/bin/sh
# ------------------------------------------------------------------------------
# Simple script to report Disk-# status on NASLite server.
# Extracts the useful info from NASLite's "storage.htm" file.
# Not too intelligent ... does not check for errors or #disks, etc.
# Tested with "NASLite-2 USB v2.06 08-2007"  ... No implied warranties.  AS-IS ...
# ------------------------------------------------------------------------------
#
NL_IP="192.168.1.xxx"      # Use *YOUR* NASLite server IP address here.
NL_Data_Folder="/usr/tmp"   # Use *YOUR* prefered location for temp files.
#
#
# ------------------------------------------------------------------------------
# Get the HTM file into local folder. The "wget" may require ftp services ??
# ------------------------------------------------------------------------------
cd $NL_Data_Folder
rm -f storage.htm                2> /dev/null
wget http://$NL_IP/Status/htm/storage.htm      2> /dev/null

# ------------------------------------------------------------------------------
# Figure out how many disks are listed ... and set NumDisks.
# Assumption: There's one "Disk-" line for each disk.
# ------------------------------------------------------------------------------
NumDisks=`cat storage.htm | grep -i "Disk-" | wc -l`

# ------------------------------------------------------------------------------
# Create a table header, extract "Last update" info, extract NumDisks info.
# Assumption: There's one "TEMP" line for each disk.
# ------------------------------------------------------------------------------
echo -n " Disk#   Size    Used    Free   "       >  disks1.txt
cat storage.htm | grep -i update | cut -d. -f3       >> disks1.txt
cat storage.htm | grep -i temp   | tail -$NumDisks    >> disks1.txt


# ------------------------------------------------------------------------------
# Get rid of most of the html code; then clean up some left-over special
# html formatting.  **NOTE** very specific to the html code implemented
# in whatever current NL version.
# ------------------------------------------------------------------------------
sed -e 's/<[^>]*>//g'             disks1.txt > disks2.txt
sed -e 's/\&nbsp;/ /g'             disks2.txt > disks3.txt
sed -e 's/CUSED/C  Used/g'          disks3.txt > disks4.txt
sed -e 's/READ ONLY /\t/g'          disks4.txt > disks5.txt
sed -e 's/READ\/WRITE /\t/g'          disks5.txt > disks6.txt
sed -e 's/\&#176;/ /g'             disks6.txt > disks7.txt

# ------------------------------------------------------------------------------
# Convert UNIX EOL char. to DOS equivalent, and save somewhere as extra copy.
# ------------------------------------------------------------------------------
sed 's/$/\r/' disks7.txt       > /home/shared/NL_DisksStatus.txt
cat disks7.txt             > NL_DisksStatus.txt

# ------------------------------------------------------------------------------
# Display results.
# ------------------------------------------------------------------------------
more NL_DisksStatus.txt



Top
 Profile  
 
 Post subject: Re: Disk Status Script
PostPosted: Mon Sep 22, 2008 10:09 am 
Offline

Joined: Sat Nov 19, 2005 6:39 pm
Posts: 633
Location: California
Wow 8) "Sticky" status ! Thanks, SiteAdmins.

Update 22Sep2008 Version 1.1: Ok ... here's a more sophisticated version (able to send an email if TEMP on Disk hits or exceeds a threshhold). Runs continously until user aborts it. ***READ DOCUMENTATION*** at beginning (about 30 lines, including where you customize to your setup), and at the end (mostly hints about "sendmail"). The way it is written "SENDMAIL" is DISABLED. You can enable by removing the hash-mark (Section L) after you're sure it will work in your environment.

Update 29Sep2008 Version 1.2: Fixed some minor issues. Added CatNap to reduce reporting time since last available update. (This version should soon appear on Eden's site.)

( I wish I could figure out how to paste this code with the tab-stops properly aligning the code ... oh well :cry: )

ENJOY !
:) Georg


Code:
#!/bin/sh
# ------------------------------------------------------------------------------
# NASLite Disk Monitor: A script to report Disk-# status on NASLite server.
# Extracts the useful info from NASLite's "storage.htm" file.
# Semi-intelligent ... but not all possible conditions fully tested (for
# example: S.M.A.R.T. turned off or unavailable, or Update turned off, etc).
# Also, if temporary files exist, and this script is run with different
# users permission problems may prevent a clean "rm" of old files.
# Can send WARNING EMAIL if temperature threshhold is reached/exceeded.
# But ***NOTE*** that "sendmail" is **TURNED OFF** by default.
# Remaining known issue: if local & NL time are not sync'ed.
# Refer to more documentation at end of this script.
# ------------------------------------------------------------------------------
# By Georg for NASLite Forum Community; tested on "NASLite-2 USB v2.06 08-2007"
# ------------------------------------------------------------------------------
# Version Info/Change History:
# 0.1 never released.
# 1.0 released 20Sep2008 (still did not contain this version info block)
#   Added automatic determination of NumDisks.
# 1.1 released 22Sep2008 Added Section lettering.  Added Section E: "Figure
#   out interval" and continous While/Do Loop (Section F+);  The
#   reports in the loop are placed in a cumulative "LOG".
#   Added "sendmail" capability.
# 1.2 released 29Sep2008 Added "$CatNap" (more real-time) Section E.1.
#   Fixed bug in Section F & adjust Section K (case of TEMP is DISABLED).
#   Fixed problem if NL server is not running (!) (Section C.1).
# ------------------------------------------------------------------------------
Version="1.2"
# ------------------------------------------------------------------------------
# SECTION A: User MUST ADJUST VARIABLES in this section, especially first 4.
# ------------------------------------------------------------------------------
NL_IP="192.168.1.XXX"      # Use *YOUR* NASLite server IP address here.
TEMP_WARN=38         # Your DISK TEMP THRESHOLD WARNING LEVEL.
Recipient="xxx@yyy.zzz"    # Use *YOUR* email address.
NL_LOG="/home/shared/NL_"   # Use *YOUR* prefered LOG FILE name/location.

NL_Data_Folder="/usr/tmp"   # Use *YOUR* prefered location for temp files.


# ------------------------------------------------------------------------------
# SECTION B: Below are additional variables/parameters.
# ------------------------------------------------------------------------------
NL_LOG=$NL_LOG`date +%Y%m%d%H%M`".log"   # Append time-stamp & LOG extension.
HDINFO="storage.htm"
Report="NL_DisksStatus.txt"
WarnMail="MailWarning.txt"
WARN_MSG1="******* WARNING *******"
WARN_MSG2=" AT or EXCEEDING THRESHOLD of $TEMP_WARN"
Title="----- NASLite Disk Monitor version $Version. ------ IP=$NL_IP"
ERR_MSG1="Unable to retrieve HTML from NL Server."
ERR_MSG2="$0 terminated ... "

# ------------------------------------------------------------------------------
# SECTION C: Get HTM file into local folder. "wget" may require ftp services ??
# Version 1.2: Added file cleanup -- permission problems if run from different
# user accounts.
# ------------------------------------------------------------------------------
rm -f disks?.txt $Report $WarnMail         2> /dev/null
cd $NL_Data_Folder
rm -f $HDINFO                   2> /dev/null
wget http://$NL_IP/Status/htm/$HDINFO         2> /dev/null

# ------------------------------------------------------------------------------
# SECTION C.1: Make sure wget is successful.
# ------------------------------------------------------------------------------
echo $Title | sed 's/$/\r/'             >  $NL_LOG
if [ ! -f $HDINFO ] ; then
   echo $ERR_MSG1                >> $NL_LOG
   echo $ERR_MSG2                >> $NL_LOG
   echo $ERR_MSG1
   echo $ERR_MSG2
   exit 1 ;
fi

# ------------------------------------------------------------------------------
# SECTION D: Figure out how many disks are listed ... and set NumDisks.
# Assumption: There's one "Disk-" line for each disk.
# ------------------------------------------------------------------------------
NumDisks=`cat $HDINFO | grep -i "Disk-" | wc -l`

# ------------------------------------------------------------------------------
# SECTION E: Figure out interval.
# Possibilities: 01, 05, 10, 15, 30, 45, 1h, 4h, 8h, --
# So far only tested 15 & 5 minute & 1h intervals ...
# ------------------------------------------------------------------------------
#
UpdateMSG=`cat $HDINFO | grep -i "Content updated every"`
Interval=`echo $UpdateMSG | sed -e 's/<[^>]*>//g' | cut "-d " -f7`
Units=`echo $UpdateMSG | sed -e 's/<[^>]*>//g' | cut "-d " -f8 | cut -d. -f1`
echo -n "Update Interval is" $Interval $Units
if [ $Units == "min" ] ; then
   Mult=60 ;
else
   Mult=3600 ;
fi
Rest=`/usr/bin/expr  $Mult \* $Interval `
echo "  == Sleep Time of" $Rest "seconds."

# ------------------------------------------------------------------------------
# SECTION E.1: For first $Rest SLEEP, reduce to $CatNap.
# Reason: If Update $Interval is very long, and the program was started close
# to the NEXT update, then a large amount of SLEEP is wasted until the
# actual delayed reporting for that soon-to-occur update with its second set
# of data.  Calculate the shortened time to next update+2mins as $CatNap.
# Problem = if this system & NASLite system are not synchronized !!!
# ------------------------------------------------------------------------------
#
LastUpdat=`cat $HDINFO | grep -i "Last update" | cut -d. -f3`
TLastStr=`echo $LastUpdat | sed -e 's/<[^>]*>//g' | cut -c12-`

TLast=`date -u --date="$TLastStr" +%s`   # Secs since 1970 (%s option).
TNow=`date -u +%s`         # Secs since 1970 (%s option).
Tdiff=`expr $TNow \- $TLast`      # Secs since last update.
T2Next=`expr $Rest \- $Tdiff`      # Secs to NEXT update.
CatNap=`expr $T2Next \+ 120`      # Will never be ZERO ... use it once!
echo "First time take shorter CatNap of $CatNap (instead of $Rest) seconds."


# ------------------------------------------------------------------------------
# ******************************************************************************
# MAIN LOOP: Run continously ... report each $Interval.  Email if Alarm.
# ******************************************************************************
# ------------------------------------------------------------------------------
while [ "1" == "1" ] ; do

# ------------------------------------------------------------------------------
# SECTION F: Create a table header, extract "Last update" and NumDisks info.
# Assumption: There's one "TEMP" line for each disk.
# Bug or Problem: if S.M.A.R.T. is UNAVAILABLE or DISABLED.
# ------------------------------------------------------------------------------
NumTemps=`cat $HDINFO | grep -i temp | wc -l`
echo -n " Disk#   Size    Used    Free   "       >  disks1.txt
cat $HDINFO | grep -i update | cut -d. -f3       >> disks1.txt
#BUG: ##cat storage.htm | grep -i temp   | tail -$NumDisks    >> disks1.txt
cat $HDINFO | grep -i "Disk-"  | tail -$NumDisks    >> disks1.txt

# ------------------------------------------------------------------------------
# SECTION G: Remove bulk of the html code; then clean up some left-over special
# html formatting.  **NOTE** very specific to the html code implemented
# in whatever current NL version.  If more commands needed, continue incementing
# '#' in "disks#.txt", then in next section copy final txt file to $Report.
# ------------------------------------------------------------------------------
sed -e 's/<[^>]*>//g'             disks1.txt > disks2.txt
sed -e 's/\&nbsp;/ /g'             disks2.txt > disks3.txt
sed -e 's/CUSED/C  Used/g'          disks3.txt > disks4.txt
sed -e 's/READ ONLY /\t/g'          disks4.txt > disks5.txt
sed -e 's/READ\/WRITE /\t/g'          disks5.txt > disks6.txt
sed -e 's/\&#176;/ /g'             disks6.txt > disks7.txt

# ------------------------------------------------------------------------------
# SECTION H: Copy final txt file to $Report (for later ease of use).
# Convert UNIX EOL char. to DOS equivalent.
# Append current status report to cumulative NL_LOG file.
# ------------------------------------------------------------------------------
cat disks7.txt             >  $Report
sed 's/$/\r/' $Report          >> $NL_LOG

# ------------------------------------------------------------------------------
# SECTION I: Display $Report.
# ------------------------------------------------------------------------------
echo "Local time is" `date`
cat $Report

# ------------------------------------------------------------------------------
# Now lets put some REAL INTELLIGENCE here ... TEMPs are known, could, for
# example, send email in case a THRESH-HOLD TEMP is EXCEEDED ...
# ------------------------------------------------------------------------------
# SECTION J: First, extract TEMPS into a single word list.
# ------------------------------------------------------------------------------
Result=`sed 's/ [ ]*/ /g' $Report | cut "-d " -f 2,7 | tail -$NumDisks`

# ------------------------------------------------------------------------------
# SECTION K: Now scan the list of results (word-pairs DiskName/TEMP).
# ------------------------------------------------------------------------------
Alarm=0               # RESET TEMPerature Alarm to OFF.
toggle=0
for WORD in $Result ; do
   toggle=`expr 1 - $toggle`   # Toggle flag so that words in
   if [ $toggle -eq 1 ] ; then    # $Result can be handled in pairs.
      DISK=$WORD      # Get DiskName (=1st) field in $Result.
      continue ;      # Skip to second field.
   else
      TEMP=$WORD      # Get TEMP (=2nd) field in $Result.

      # Ensure that $TEMP is numeric (NOT Alpha as in "DISABLED")
      # before testing its numeric value against threshhold.
      Alpha=`echo $TEMP | sed 's/[0-9]//g'`
      if [ -z $Alpha ] ; then
      if [ $TEMP -ge $TEMP_WARN ] ; then
         echo "$WARN_MSG1" $DISK at $TEMP "$WARN_MSG2"
         echo -n "$WARN_MSG1" $DISK at $TEMP    >> $NL_LOG
         echo "$WARN_MSG2" | sed 's/$/\r/'    >> $NL_LOG
         Alarm=1      # Trigger the Alarm.
         ListEnd=1 ;
      else
         ###echo $DISK is at $TEMP degrees C
         ListEnd=1 ;
      fi
      ListEnd=1 ;
      fi
   ListEnd=1 ;
   fi
   ListEnd=1 ;
done

# ------------------------------------------------------------------------------
# SECTION L: Send WARNING EMAIL ...
# (Obviously requires a "sendmail" mail server running on local system.)
# ------------------------------------------------------------------------------
if [ $Alarm -eq 1 ] ; then
   domain=`echo $Recipient | cut -d@ -f2`
   echo "Date: `date`"              >> MailMsg
   echo "From: no-reply@$domain"          >  MailMsg
   echo "Subject: NL TEMP WARNING"       >> MailMsg
   echo "To: $Recipient"             >> MailMsg
   echo ''                >> MailMsg
   echo -n "$WARN_MSG1"            >> MailMsg
   echo At least one disk "$WARN_MSG2"      >> MailMsg
   echo ''                >> MailMsg
   echo "----------- Full Report is ------------"   >> MailMsg
   cat $Report               >> MailMsg

   sed 's/$/\r/' MailMsg             >  $WarnMail

   # ----------------------------------------------------------------------
   # NOTE: Actual emailing is currently ***DISABLED***
   # EN-ABLE 'sendmail' by REMOVING the '#' in front of it.
   # Using "-f" so "Return-Path" won't be "root" or other unusual user.
   # ----------------------------------------------------------------------
   rm -f MailMsg 2> /dev/null
   ###/usr/sbin/sendmail -fDoNotReply -t       <  $WarnMail
   ListEnd=1 ;
fi

# ------------------------------------------------------------------------------
# SECTION M: SLEEP until next Interval; then grab updated HTM, and loop around.
# Updated after Section E.1 was added with $CatNap.
# ------------------------------------------------------------------------------
rm -f $HDINFO                2> /dev/null
if [ $CatNap -gt 0 ] ; then
   sleep $CatNap
   CatNap=0          # Reset so it's used only once.
   ListEnd=1 ;
else
   sleep $Rest
   ListEnd=1 ;
fi
wget http://$NL_IP/Status/htm/$HDINFO      2> /dev/null

ListEnd=1 ;
# ------------------------------------------------------------------------------
done      # END of continuous WHILE LOOP
# ------------------------------------------------------------------------------

exit 0   # Clean non-error exit if this point is reached.


# ------------------------------------------------------------------------------
# Regarding "SENDMAIL":
# This script uses "sendmail" at bottom of Section L.  Obviously the CPU this
# script runs on must contain a properly set up mechanism for sending mail.
# For most users this may have already been arranged, but if not, is not a
# trivial excercise.  In particular Section L assumes that (for example) the
# user running this script (may be 'root' because the script is part of the
# startup sequence) is not necessarily properly authenticated, and since
# an ISP may require that mail headers match a certain domain the "Return-Path"
# is set to the domain of the recipient ... etc etc.  The file "main.cf" in
# "/etc/postfix" may also require adjustments.
#
# I have been testing this script using the "root" login on a server running
# Linux.  The server does not run an email program ... email is done at
# the WinXP clients, and therefore user accounts on the server do NOT correlate
# to ISP account (and their email addresses).  For testing I temporarily set
# up SMTP on the server.
#
# So ... the general comments is **** USER BEWARE ***.  For "sendmail" to
# properly work the user may have to do his/her own investigation. 
# ------------------------------------------------------------------------------
# Regarding "ListEnd=1 ;"
# I don't like how "/bin/sh" uses the ";" to terminate the "list" of commands
# to execute in an "if" or "while" statement.  I use a (dummy) separate line
# to designate a more obvious last statement for the section in question.
# Non-standard -- I know -- apologies to hot-shot programmers (which I am not).
# ------------------------------------------------------------------------------
# Can add a line to "/etc/rc.local" to start this script on boot.  However,
# a couple of things to watch: 1] make sure to start it with a "/bin/sh" shell,
# and 2] route 1> and 2> to some specific desired location , and 3] use "&". 
# Here's an example: (where xxx is name you give this script, "NL_HD_Stat.sh")
# /bin/sh ~/bin/xxx 1> /usr/temp/xxx_out.txt 2> /usr/temp/xxx_err.txt &
# ------------------------------------------------------------------------------
# Other "to-do":
# Consider allowing command line options to designate the $Recipient,
# the loop count, the $TEMP_WARN threshhold, etc.
# Fix problem of de-synced clocks of NL & this system running script (de-sync
# messes up the SLEEP/CatNap intervals), which can waste a lot of time between
# current TEMPs and last status if the Update interval is long.
# Fix "permission denied" errors if temp files exist and can not be removed
# by the user currently running the script.
# Add general clean ERROR control (ON ERROR perform some action).
# ------------------------------------------------------------------------------


Last edited by georg on Mon Sep 29, 2008 1:19 pm, edited 2 times in total.

Top
 Profile  
 
 Post subject: Re: Disk Status Script
PostPosted: Tue Sep 23, 2008 9:33 am 
Offline

Joined: Sun Jul 09, 2006 10:26 am
Posts: 428
Location: UK
Hi Georg,

If its ok with you i'd like to put the script on my site?
After 4-5 months of not being able to alter the site due to a permission 550 error server side, yes its taken this long to fix.
So I am going to go in and tidy up a little bit.

Eden


Top
 Profile  
 
 Post subject: Re: Disk Status Script
PostPosted: Tue Sep 23, 2008 10:17 am 
Offline

Joined: Sat Nov 19, 2005 6:39 pm
Posts: 633
Location: California
Hello again, Eden !

Great Idea ... but hold off a few days. Something odd happened and I haven't figured out the problem yet. It needs a bug fix. Seems the script usually works fine with root-login, but hiccups sometimes if logged in as regular user, and last night would not run correctly if started from "rc.local" ... oh the joys of programming ! (Plus I am adding one more improvement (which I call CatNap) if status updates are on an infrequent cycle (like 1h+).)

I'll update post # 2 as needed and will let you know when it's ready to post on your site (those darn 550's !!)

:) Georg


Top
 Profile  
 
 Post subject: Re: Disk Status Script
PostPosted: Tue Sep 23, 2008 10:40 am 
Offline

Joined: Sun Jul 09, 2006 10:26 am
Posts: 428
Location: UK
Ok sure Georg,

take you time, I must add though.. it was my ISP's fault regarding the 550 errors, may people suffered the same fate, oh and still do.
if it was me and I hadnt fixed it in a 4-5 month time frame i'd have taken a hammer to the server and be done with it. :D
anyway I hope you get the script sorted and if there is anything like images and or other stuff you want adding to the section just send them over.

eden


Top
 Profile  
 
 Post subject: Re: Disk Status Script
PostPosted: Mon Sep 29, 2008 1:48 pm 
Offline

Joined: Sat Nov 19, 2005 6:39 pm
Posts: 633
Location: California
Here's some sample output of version 1.2 if run interactively:

Code:
$ NL
Update Interval is 15 min  == Sleep Time of 900 seconds.
First time take shorter CatNap of 503 (instead of 900) seconds.
Local time is Mon Sep 29 10:38:38 PDT 2008
 Disk#   Size    Used    Free   Last update Mon Sep 29 17:30:01 UTC 2008
 Disk-0  183.4G  123.9G  59.5G          TEMP 35 C  Used 68% (95%)
 Disk-1  275.1G  246.0G  29.1G          TEMP 34 C  Used 89% (95%)
 Disk-2  458.4G  419.8G  38.6G          TEMP 31 C  Used 92% (95%)

And if the TEMP threshold is triggered (threshold temporarily set arbitrarily lower):

Code:
$ NL
Update Interval is 15 min  == Sleep Time of 900 seconds.
First time take shorter CatNap of 211 (instead of 900) seconds.
Local time is Mon Sep 29 10:43:30 PDT 2008
 Disk#   Size    Used    Free   Last update Mon Sep 29 17:30:01 UTC 2008
 Disk-0  183.4G  123.9G  59.5G          TEMP 35 C  Used 68% (95%)
 Disk-1  275.1G  246.0G  29.1G          TEMP 34 C  Used 89% (95%)
 Disk-2  458.4G  419.8G  38.6G          TEMP 31 C  Used 92% (95%)
******* WARNING ******* Disk-0 at 35  AT or EXCEEDING THRESHOLD of 35


Top
 Profile  
 
 Post subject: Re: Disk Status Script
PostPosted: Mon Sep 29, 2008 2:46 pm 
Offline

Joined: Sun Jul 09, 2006 10:26 am
Posts: 428
Location: UK
You can download Version 1.2 of the script in zip form from http://www.nasbox.me.uk scripts section.


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 7 posts ] 

All times are UTC - 5 hours [ DST ]


Who is online

Users browsing this forum: No registered users and 5 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB® Forum Software © phpBB Group