saneproc - quick at a glance process summary

This simple script will show you the number of processes running under each unique user. Simple as that :-). The beauty of this script is that you can pass any stream of data with multiple values and the script will automatically process the data without any changes:

#--- START ---
#!/bin/sh

clear

ps -ef | cut -c 1-8 | sort | awk ' { print $1 } ' | sort |

awk '

BEGIN {
       print "Cranies Process Checker..."
       print " "
       printf ( "%8s%6s%9s\n", "User:", "  |   ", "No. Processes" )
       print "-------------------------"
       usertotal = 1
}

# MAIN

{

if ( $1 == stored ) {
    usertotal += 1
} else {
    if ( stored != "" ) {
        printf ( "%8s%6s%9d\n", stored, "  |   ", usertotal )
    }
    totalused += usertotal
    usertotal = 1
}

stored = $1

}

END {
       printf ( "%8s%6s%9d\n", stored, "  |   ", usertotal )
       print "-------------------------"
       print " "
       print "Total Processes: " totalused
       print "-------------------------"
       print " "
}

 '
#--- END ---

http://pastie.org/616117

Holiday

I am going to be away from a computer from tomorrow for the next week or so. I will try and post some more snippets. Going to aim for a Useful function once a week. So will get that in tomorrow and then continue after with more random scripts. If I manage to update any of my existing ones I will re-post them.

As I am using pastie.org I will update the source and just make a new post once a week with the various updates unless there are any big changes.

Twitter alerter for computer(s)

Recently I was new to the world of Twitter. Once I started I found it very addictive both for the information you could find on there and the possibilities of which it could be used.

The following script should be run via cron, if you run it on anything else other than a Mac you will need to replace the Mac specific HardwareMonitor application with one for your operating system. The code started to add functionality to allow tweeting DM's to the computer and it would reply with certain information like IP address. However, since I found TweetMyMac I have abandoned this functionality. I could be persuaded to re-visit this script an the DM aspect if there was a reason / idea.

The crontab code I use for this on my Mac is:

  15 *  *  *  *  /Users/cranie/mytwitter/twitter

basically (if you are unfamiliar with cron) run crontab -e and enter the above (replacing the /Users... part with the location to your script). This will then run every 15minutes and tweet the appropriate messages if required.

Updates to come:

• Random messages (i.e. damn its 55deg in here or Oh my I am on fire.... etc - ideas welcome)
• DM if there is any useful application for implementing this
• ???

The script:

#--- START ---
#!/bin/sh

# Limit thresholds before script alerts to twitter
CPU_LIM=55
PROC_LIM=100
DISK_LIM=95


USERNAME="your twitter username"
PASSWORD="your twitter password"
URL=http://twitter.com/statuses/update.json

CPU=$(/Applications/HardwareMonitor.app/Contents/MacOS/hwmonitor 2> /dev/null | grep "SMC CPU A DIODE" | sed -e 's/.*: //g' -e 's/ C//g')

if [ ${CPU} -ge ${CPU_LIM} ]
then
    MESSAGE="${MESSAGE}Its roasting in here: CPU Temperature is ${CPU} deg C. "
fi

DISK=$( df -m | grep "/dev/disk0s2" | awk ' { print $5 } ' | sed 's/%//g' )

if [ ${DISK} -ge ${DISK_LIM} ]
then
    MESSAGE="${MESSAGE} I'm getting full! Root disk is ${DISK}% full. "
fi

PROC=$( ps -ef | wc -l | awk ' { print $1 } ')

if [ ${PROC} -ge ${PROC_LIM} ]
then
    MESSAGE="${MESSAGE}${PROC} processes running. "
fi

UPTIME=$( uptime | awk ' { print $3 } ' | sed 's/:.*//g' )
TIMER=$( uptime | awk ' { print $5 } ' | sed -e 's/,//g' -e 's/:.*//g' )
UPTIME_UNIT=$( uptime | awk ' { print $4 } ' | sed 's/,//g' )

if [ "$(( ${UPTIME} % 8 ))" -eq 0 -a "${UPTIME_UNIT}" == "days" -a "${TIMER}" -eq 1 ]
then
    MESSAGE="${MESSAGE}Up ${UPTIME} ${UPTIME_UNIT}. "
fi

if [ "${MESSAGE}" != "" ]
then
     #echo "Posting: ${MESSAGE}"
     curl --basic --user "${USERNAME}:${PASSWORD}" --data-ascii "status=`echo ${MESSAGE}|tr ' ' '+'`" "${URL}" -o /dev/null
else
     echo "Nothing to post"
fi


# --- Checks for ask for info below here:
URL2=https://twitter.com/statuses/mentions.xml

#TIME_CHK=`date | awk ' { split($4,A,":") ; if ( A[2] <= 14 ) { printf("%02d:%02d:%02d\n", A[1] - 1, A[2] -15, A[3]) } else { printf("%02d:%02d:%02d\n", A[1], A[2] - 15, A[3]) } } '`
# MAC
TIME_CHK=`date | awk ' { split($5,A,":") ; if ( A[2] <= 14 ) { printf("%02d:%02d:%02d\n", A[1] - 1, A[2] -15, A[3]) } else { printf("%02d:%02d:%02d\n", A[1], A[2] - 15, A[3]) } } '`
DATE_CHK="`date +'%a %h %d'`"

RUN=`curl --basic --user "${USERNAME}:${PASSWORD}" -k ${URL2} 2> /dev/null | egrep "|" |
awk ' {
    if ( match($1,"created_at") != 0 ) {
        printf $0
    } else {
        print $0
    }
} ' |
grep "" |
sed -e 's/.*//' -e 's/<\/created_at>//' -e 's///' -e 's/<\/text>//' |
awk -v q=\' -v TIME_CHK=${TIME_CHK} -v DATE_CHK="${DATE_CHK}" ' {
    MSG_DATE = $1 " " $2 " " $3
    if ( DATE_CHK == MSG_DATE ) {
         split(TIME_CHK,T,":")
         split($4,M,":")
# +1 hour due to twitter being none BST
HOUR = M[1] + 1
         if ( HOUR >= T[1] && M[2] >= T[2] ) {
             if ( $8 == "show" ) {
                 if ( $9 == "time" ) {
                      EXTRA = EXTRA "My clock tells me its @DATE. "
                 }
                 if ( $9 == "temp" ) {
                      EXTRA = EXTRA "My temp is currently @TEMP. "
                 }
                 if ( $9 == "ip" ) {
                     EXTRA = EXTRA "Sent direct message of my IP address @IP. "
                     DM = DM "My IP address is @IP. "
                 }
             }
        }
    }
} ' `

DATE="`date`"
RUNNER="`echo ${RUN} | sed "s/@DATE/${DATE}/g"`"
RUNNER="`echo ${RUN} | sed "s/@TEMP/${CPU}/g"`"
IP=`curl http://checkip.dyndns.org/ | awk ' { print $6 } ' | sed 's/<.*//g'`
DMER="`echo ${DM} | sed "s/@IP/${IP}/g"`"
if [ "${RUNNER}" != "" ]
then
     curl --basic --user "${USERNAME}:${PASSWORD}" --data-ascii "status=`echo ${RUNNER}|tr ' ' '+'`" "${URL}" -o /dev/null
else
     echo "Nothing to post"
fi

USRL3=http://twitter.com/direct_messages/new.xml
if [ "${DMER}" != "" ]
then
     curl --basic --user "${USERNAME}:${PASSWORD}" -d "text=${DMER}&user=cranies" ${URL3} -o /dev/null
else
     echo "Nothing to post"
fi
#--- END ---

http://pastie.org/615819

Requests?

Whilst on my journey through pasting things in here which are useful to me, if there are any requests (preferably simple ones like I've posted :-) ) then let me know in the comments and I'll do my best to help out where I can.

Alternatively mail me your request and we can see if I can help out on a bigger scale.

Formatting - code in this blog

The code in this blog uses pastie.org - and it is not formatting correctly. I will fix this when I get the chance. For now all the code and descriptions are available. So try to enjoy :-)

Useful functions part1

This is the first in useful functions for shell scripts. This is useful when deploying to several servers at different versions / distributions. It allows you to check all the main programs are available to your script before it runs:

#--- START ---
cmd_check() { 
    x=0 
    for cmd in $@ 
    do 
        which ${cmd} 2>&1 > /dev/null 
        if [ $? -ne 0 ] 
        then 
            echo "${cmd} Not Found. Either install command or add path location to the PATH variable" 
            x=$(( ${x} + 1 )) 
        fi 
    done 
    if [ ${x} -ne 0 ] 
    then 
        exit ${x} 
    fi 
} 

cmd_check pax mkfifo ssh rm echo cat 
#--- END ---

http://pastie.org/614974

When running call the functions cmd_check with all the program names as arguments. If they are not available / in the PATH then the script will echo out an error and exit.

scp large directories / lots of files easily

Recently I came across an issue where I needed to scp hundreds of files and directories totalling several hundred GB. The easiest way was to scp the folders one by one, but some files would be too large for SCP (AIX) to deal with. Also there is no additional compression (aside from what scp provides) for this. The solution was to pax the folders on the fly to a fifo file, (gzip not yet implemented but will be added soon), scp the fifo file (not scp but ssh due to scp limitations) and then on the remote side to extract the files. This is easy to do manually, but very laborious, logging on to each server and running the make fifo's, running the commands in the correct order.

To simplify this I have made a script to automate each step. The only requirement is to have SSH keys set up from the source server to the target server. This is a god method to transfer large amounts of data or lots of files, it is overkill for simple operations.

#--- START ---
#!/usr/bin/ksh 

echo " 
Unrestricted SCP. 
" 

cmd_check() { 
    x=0 
    for cmd in $@ 
    do 
        which ${cmd} 2>&1 > /dev/null 
        if [ $? -ne 0 ] 
        then 
            echo "${cmd} Not Found. Either install command or add path location to the PATH variable" 
            x=$(( ${x} + 1 )) 
        fi 
    done 
    if [ ${x} -ne 0 ] 
    then 
        exit ${x} 
    fi 
} 

cmd_check pax mkfifo ssh rm echo cat 

if [ $# != 4 ] 
then 
    echo "--------------------------------------------------------------------------------" 
    echo "- This script will copy a directory and all the folders and files.             -" 
    echo "- Unlike SCP / tar and gzip there are no file restrictions.                    -" 
    echo "- All transfers / and extracts are done on the go, so there is no space        -" 
    echo "- requirements to land the data                                                -" 
    echo "--------------------------------------------------------------------------------" 
    echo " " 
    echo "Usage: firo_scp.ksh    " 
    echo " " 
    exit 1 
fi 

DIR=${1} 
DIRNAME=`echo ${DIR} | sed 's/.*\///g'` 
USERNAME=${2} 
SERVER=${3} 
RPATH=${4} 
FIFO=./FSCP.$$ 

echo "Making a fifo file" 
mkfifo -m 644 ${FIFO} 

echo "Initiating pax to ${FIFO} fifo for ${DIR}" 
pax -wf ${FIFO} -x pax ${DIR} & 

echo "Making remote fifo file: Authentication Successful should be displayed" 
ssh ${USERNAME}@${SERVER} "mkfifo ${RPATH}/${DIRNAME}.ar" 

echo "catting the local ${FIFO} fifo into remote fifo file: Authentication Successful should be displayed" 
cat ${FIFO} | ssh ${USERNAME}@${SERVER} "cat > ${RPATH}/${DIRNAME}.ar" & 

echo "Initiating remote extract on fifo file: Authentication Successful should be displayed" 
ssh ${USERNAME}@${SERVER} "cat ${RPATH}/${DIRNAME}.ar | pax -rxpax" 

echo "Removing remote fifo file: Authentication Successful should be displayed" 
ssh ${USERNAME}@${SERVER} "rm ${RPATH}/${DIRNAME}.ar" 

echo "Removing local ${FIFO} fifo" 
rm ${FIFO}
#--- END ----

http://pastie.org/614963