Configure Infiniband and bonding at once

Just after the deploying of several nodes, running this on each of them gets them configured to be able to reboot and start running.

The condition here is that each node will get its IP addresses from its own name,

If hostname is cn10 , the ip addresses will be 10.18.1.10 and 10.30.0.10 .
Of course we didn’t have any cn0. And there were about 100 nodes.

#!/bin/bash

HOST=$(hostname)
HOSTNUM=${HOST/cn/}

echo ${HOSTNUM}

if [[ "${HOSTNUM}" -gt 108 ]]; then
	echo "This is not a compute node, bailing out..."
	exit 1
fi

for interface in eth2 eth3
do
	cat > /etc/sysconfig/network-scripts/ifcfg-${interface} <<EOF
DEVICE="${interface}"
BOOTPROTO=none
ONBOOT=yes
MASTER=bond0
SLAVE=yes
EOF

done

cat > /etc/sysconfig/network-scripts/ifcfg-bond0 <<EOF
DEVICE=bond0
BOOTPROTO=none
ONBOOT=yes
IPADDR=172.18.1.${HOSTNUM}
NETMASK=255.255.0.0
NETWORK=172.18.0.0
EOF

ifdown eth2
ifdown eth3
ifdown bond0
ifup bond0

cat > /etc/sysconfig/network-scripts/ifcfg-ib0 <<EOF
DEVICE=ib0
BOOTPROTO=none
ONBOOT=yes
IPADDR=10.30.0.${HOSTNUM}
NETMASK=255.255.0.0
NETWORK=10.30.0.0
EOF

ifdown ib0
ifup ib0

Desktop login and SMB/CIFS mountpoints made easy

Having lots of users with kind of “magic hands” got us to think what they did not need to think about, so they could do more of their research.

Since we got everyone on AD using Quest (see previous post) we where able to launch the following script each time they log on their Desktops, making a nice direct access to all the resource from their groups and from their research.

Also, we used the library.

#!/bin/bash -x
#===============================================================================
#
#          FILE:  fbm_logon_script_workstation.sh
# 
#         USAGE:  ./fbm_logon_script_workstation.sh 
# 
#   DESCRIPTION:  
# 
#       OPTIONS:  ---
#  REQUIREMENTS:  ---
#          BUGS:  ---
#         NOTES:  ---
#        AUTHOR: Joan Marc Riera Duocastella (), 
#       COMPANY: Fundació Barcelona Media
#       CREATED: 12/22/2010 11:07:00 AM CET
#      REVISION:  ---
#===============================================================================

set -o nounset                              # Treat unset variables as an error

logger -i -s "$0 - launched"
#clean temp files
LOGFILE=/$0.log
cd /tmp
exec 1>>$LOGFILE
exec 2>>$LOGFILE
SERVERIP="xxyy.tt.rrr"
SERVERPATH="scripts"
MASTERURL=http://$SERVERIP/$SERVERPATH
## Don't edit above this line
SCRIPTNAME=$0
DATETIME=`date +%Y%m%d%H%M%S`
HNAME=`hostname`
# load marcs library
tempscript="library.sh"
if [ ! -f /tmp/$tempscript ]
then
         wget $MASTERURL/lib/$tempscript -O /tmp/$tempscript
         . /tmp/$tempscript && rm /tmp/$tempscript
fi
# define environment
shootProfile
# check if USER env variable is set, or otherwise set it
[ -n "${USER:-x}" ] && USER=`whoami` 
#check if .desktop exists else do it
if [[ -f /home/$USER/.config/user-dirs.dirs ]]
then
	desktop_dir=/home/$USER/`grep XDG_DESKTOP_DIR /home/$USER/.config/user-dirs.dirs  | cut -f2 -d\" | cut -f2 -d\/`
fi
if [[ -n "${desktop_dir:+x}" ]] && [[ ! -f "$desktop_dir/montar unitat de xarxa.desktop" ]]
then
	
	case $OS in
		 linux )	
			  tempscript="fs_fbm.tar"
			  ;;
		 mac )
			  tempscript="fs_fbm_macosx.tar"
			  ;;
		 * ) #at least try if there is luck
			  tempscript="fs_fbm.tar"
			  ;;
	esac
	
	#download fs_fbm.tar
	if [ ! -f /tmp/$tempscript ]
	then
		 wget $MASTERURL/src/$tempscript -O /tmp/$tempscript
	fi
	cd /tmp
	tar -xvf /tmp/$tempscript 
	rm  /tmp/$tempscript
	cd ${tempscript%%.???}
	datasrc_dir="`pwd`/data"
	#if [[ $CANGOSUDO -eq $TRUE ]]
	#then
	#we are not installing anything on a logonscript 
	#	sudo ./install.sh 
	#else
	cp $datasrc_dir/*.desktop $desktop_dir
	chmod +x $desktop_dir/montar*.desktop
	chmod +x $desktop_dir/desmontar*.desktop
	#fi
#	rm -r /tmp/${tempscript%%.???}
fi
#############################################fbm_mount end
##########################check sudoers
 # :TODO:12/22/2010 11:09:09 AM CET:: if uptime>15 dies mail with reboot and upgrade request
uptime
exit 0

Expect script to mount the remote CIFS

#!/usr/bin/expect -f

#mount cm01 software repository

# Se definen la svariables USERNAME y PASS con el valor de los parametros que recibe el script.
set USERNAME [lindex $argv 0]
set PASS [lindex $argv 1]

# Montar //fbmcm01/software en /home/sysop/fs con los credenciales del usuario pasado como parametro
spawn mount.cifs //84.88.79.25/software /home/sysop/fs -o username=$USERNAME
#spawn mount -t cifs //84.88.79.25/software /home/sysop/fs -o username=$USERNAME
# Con la orden expect, indicamos la cadena de texto que esperamso que aparezca y con send, 
# se envia el parametro que queremso introducir de forma automatica
expect "Password:^"
sleep 5
send "$PASS\n"

expect eof

Link Unix machines to AD using Quest Autentication Services. Install it using expect.

This next script uses the library introduced on a previous post =>

This script was launched at the end of the installation of all the workstations on the company.
It configures the linux, or MAC , to be part of an Active Directory, which allows the administrator to manage the root passwords, what runs at boot, and even the desktop background.

http://www.quest.com/authentication-services/

However, the installation and configuration was slow and wanted to be interactive, so in this case we used expect to automate the human interaction.

The first script is the bash that was launched after the pre-seed job, then comes the expect (which is launched from inside the bash).

#!/bin/bash -x

# Script de instalacion de QAS en un desktop
# El script llama a fbm_qas_install.expect, donde se ejecutan instrucciones expect. 

# Las salidas por pantalla, se redirigen al fichero output.expect.o
# Los errores se redirigen al fichero output.expect.e # :TODO:01/14/2011 10:32:18 AM CET:: unificar output
exec 1>./output.expect.o
exec 2>./output.expect.e

# Definicion de variables.
USERNAME="username_only_allowed_to_add_machines_to_AD"
PASS="pass for that user"
SERVERIP="xx.yy.tt.rr ip of our repository with all the scripts , also the library. "
SERVERPATH="scripts"
MASTERURL="http://$SERVERIP/$SERVERPATH"

# Funcion para instalar paquete 'smbfs'.
installsmbfs(){
	 echo "Installing smbfs"
	 apt-get update
	 apt-get -y upgrade
	 apt-get -y autoremove
	 apt-get -y install smbfs 
}

 # :TODO:01/14/2011 10:32:34 AM CET:: smbfs is not necessary 
# Comprueba si el paquete 'smbfs' esta instalado en el sistema. Si no lo esta, ejecuta la funcion 
# llamada 'installsmbfs'.
dpkg-query -l 'smbfs'
[ "$?" -eq  "0" ] && echo "smfs is already installded. Keep going" || installsmbfs

 # :TODO:01/14/2011 10:32:34 AM CET:: smbfs is not necessary 
# Comprueba si el paquete 'smbfs' esta instalado en el sistema. Si lo esta, no hace nada.
dpkg-query -l 'smbfs'
[ "$?" -eq  "0" ] && echo "smfs has been installed. Keep going" || die " smfs can not be installed " 

# Modifica un parametro del fichero /etc/ssh/sshd_config y reinicia el servicio sshd
sed -i 's/ChallengeResponseAuthentication no/ChallengeResponseAuthentication yes/g' /etc/ssh/sshd_config
/etc/init.d/ssh reload
 # :TODO:01/14/2011 10:32:58 AM CET:: check if directory already existes , also check if files already exist
# Descarga ficheros vas.conf y vgp.conf en sus rutas correspondientes
mkdir -p /etc/opt/quest/vas
mkdir -p /etc/opt/quest/vgp
wget $MASTERURL/etc/opt/quest/vas/vas.conf  -O /etc/opt/quest/vas/vas.conf
wget $MASTERURL/etc/opt/quest/vgp/vgp.conf  -O /etc/opt/quest/vgp/vgp.conf

# Modifica el template vas.conf. Escribe el nombre del PC
sed  -i "s/%%HOSTNAME%%/`hostname`/g" /etc/opt/quest/vas/vas.conf

[ -d /home/sysop/fs ] && echo "fs exists" || echo "fs does not exist"
[ -d /home/sysop/fs/QAS_4_0_1_22 ] && echo "QAS dir  exists" || echo "QAS dir  does not exist"

cd /home/sysop
 # :TODO:01/14/2011 10:33:33 AM CET:: this should be done on /tmp
wget $MASTERURL/src/QAS_4.tgz
tar xvzf QAS_4.tgz

cd /home/sysop/QAS_4_0_1_22
./install.sh -q vasclnt
./install.sh -q vasgp
/opt/quest/bin/vastool configure pam common-password

apt-get -y install expect
############################# install qas
tempscript="fbm_qas_install.expect"
if [ ! -f ./$tempscript ]
then
        wget $MASTERURL/bin/$tempscript
fi
chmod +x $tempscript
./$tempscript $USERNAME $PASS

exit 0
#!/usr/bin/expect -f

# Script de expect. Script pide pasar parametros en la instalacion de l aaplicacion QAS
# Con expect, esos parametros se pasan de forma automatica.
#echo "dentro expect"


# Se definen la svariables USERNAME y PASS con el valor de los parametros que recibe el script.
set USERNAME [lindex $argv 0]
set PASS [lindex $argv 1]

# ejecutar el fichero ./install.sh del instalador de QAS
spawn /opt/quest/bin/vastool -u $USERNAME join corp.barcelonamedia.org
# contestar las preguntas de la instalacion de QAS de forma desatendida
sleep 1
expect "CORP.ACTIVEDIRECTORYDOMAIN.ORG:*" 
sleep 1
send "$PASS\r"
sleep 60

The scripts that become functions, and the functions that belong to a library .

Long ago, when I had the chance to build by myself a set of scripts that performed certain operations I ended up with a nice set of small things.

I would like to share, but it needs some explanation.

This library, group of tools, they do nothing by themselves, their only serve a single purpose which is “To standardize” .
At that time I was a “one man army”, I built stuff by myself, for myself and to myself, but then I realized that after me would come someone else, with different styles, different habits and so on,..

Since I needed to change as less code possible this library can be sourced and its functions called many times, they become read only , or they are atomic, so nothing has be worried about using its functions several times in a row.

#!/bin/bash
# Marcs library 
#TODO
# there are two lower case converters
# funcion to download all the scripts at once
# funcion to clean up all the scripts

# set variables 
declare -r TRUE=0
declare -r FALSE=1
declare -r PASSWD_FILE=/etc/passwd
declare -r RED='\033[0;41;30m'
declare -r STD='\033[0;0;39m'
declare -r CYAN='\e[1;37;44m'
declare -r TMPDIR='/tmp/'
##################################################################
# Purpose: Converts a string to lower case
# Arguments:
#   $1 -> String to convert to lower case
##################################################################
function to_lower() 
{
    local str="$@"
    local output     
    output=$(tr '[A-Z]' '[a-z]'<<<"${str}")
    echo $output
}
##################################################################
# Purpose: Display an error message and die
# Arguments:
#   $1 -> Message
#   $2 -> Exit status (optional)
##################################################################
function die() 
{
    local m="$1"	# message
    local e=${2-1}	# default exit status 1
    echo "$m" 
    exit $e
}
##################################################################
# Purpose: Return true if script is executed by the root user
# Arguments: none
# Return: True or False
##################################################################
function is_root() 
{
   [ $(id -u) -eq 0 ] && return $TRUE || return $FALSE
}
##################################################################
# Purpose: Return true $user exits in /etc/passwd
# Arguments: $1 (username) -> Username to check in /etc/passwd
# Return: True or False
##################################################################
function is_user_exits() 
{
    local u="$1"
    grep -q "^${u}" $PASSWD_FILE && return $TRUE || return $FALSE
}



##################################################################
# Purpose: Inform the user that this functions is not implemented
# Arguments: No arguments 
# Return: No returns
##################################################################
function notImplemented()
{
	 echo "This functions is not implemented yet."
	 echo "Press any Key to continue or press Ctrl-C."
	 read x
}
##################################################################
# Purpose: Converts a string to lower case
# Arguments:
#   $1 -> String to convert to lower case
##################################################################
lowercase(){
	echo "$1" | sed "y/ABCDEFGHIJKLMNOPQRSTUVWXYZ/abcdefghijklmnopqrstuvwxyz/"
}
##################################################################
# Purpose: Recover important information 
# Arguments: If $1 == "debug" then output for each var will be printed
# Return: TRUE is first time execution. FALSE if already executed
# Read Only variables set:
#	OS
#	DIST
#	DistroBasedOn
#	PSUEDONAME
#	REV
#	KERNEL
#	MACH
##################################################################
shootProfile(){
	 # This funcion creates readonly variables, if one of them has been set before it will crash
	 # check that all variables are unset.
	 [ -n "${OS+x}" ] && [ -n "${DIST+x}" ] && [ -n "${DistroBasedOn+x}" ] && [ -n "${PSUEDONAME+x}" ] && [ -n "${REV+x}" ] && [ -n "${KERNEL+x}" ] && [ -n "${MACH+x}" ] && echo " !!! shootProfile was called before. The variables are already available" &&  return $FALSE

	 OS=`lowercase \`uname\``
	 KERNEL=`uname -r`
	 MACH=`uname -m`

	 if [ "{$OS}" == "windowsnt" ]; then
		  OS=windows
	 elif [ "{$OS}" == "darwin" ]; then
		  OS=mac
	 else
		  OS=`uname`
		  if [ "${OS}" = "SunOS" ] ; then
				OS=Solaris
				ARCH=`uname -p`
				OSSTR="${OS} ${REV}(${ARCH} `uname -v`)"
			elif [ "${OS}" = "AIX" ] ; then
				OSSTR="${OS} `oslevel` (`oslevel -r`)"
			elif [ "${OS}" = "Linux" ] ; then
				if [ -f /etc/redhat-release ] ; then
					 DistroBasedOn='RedHat'
					 DIST=`cat /etc/redhat-release |sed s/\ release.*//`
					 PSUEDONAME=`cat /etc/redhat-release | sed s/.*\(// | sed s/\)//`
					 REV=`cat /etc/redhat-release | sed s/.*release\ // | sed s/\ .*//`
				elif [ -f /etc/SuSE-release ] ; then
					 DistroBasedOn='SuSe'
					 PSUEDONAME=`cat /etc/SuSE-release | tr "\n" ' '| sed s/VERSION.*//`
					 REV=`cat /etc/SuSE-release | tr "\n" ' ' | sed s/.*=\ //`
				elif [ -f /etc/mandrake-release ] ; then
					 DistroBasedOn='Mandrake'
					 PSUEDONAME=`cat /etc/mandrake-release | sed s/.*\(// | sed s/\)//`
					 REV=`cat /etc/mandrake-release | sed s/.*release\ // | sed s/\ .*//`
				 elif [ -f /etc/debian_version ] ; then
					 DistroBasedOn='Debian'
					 DIST=`cat /etc/lsb-release | grep '^DISTRIB_ID' | awk -F= '{ print $2 }'`
					 PSUEDONAME=`cat /etc/lsb-release | grep '^DISTRIB_CODENAME' | awk -F= '{ print $2 }'`
					 REV=`cat /etc/lsb-release | grep '^DISTRIB_RELEASE' | awk -F= '{ print $2 }'`
				 fi
				 if [ -f /etc/UnitedLinux-release ] ; then
					 DIST="${DIST}[`cat /etc/UnitedLinux-release | tr "\n" ' ' | sed s/VERSION.*//`]"
				  fi
				OS=`lowercase $OS`
				DistroBasedOn=`lowercase $DistroBasedOn`
				readonly OS
				readonly DIST
				readonly DistroBasedOn
				readonly PSUEDONAME
				readonly REV
				readonly KERNEL
				readonly MACH
		  fi

	 fi
	local debug=${1:-nodebug}
	 if [[ $debug = "debug" ]]; then
				echo $OS
				echo $DIST
				echo $DistroBasedOn
				echo $PSUEDONAME
				echo $REV
				echo $KERNEL
				echo $MACH
	 fi
	 return $TRUE
}
##################################################################
# Purpose: Make sure package on $1 is installed 
# Arguments:
#   $@ -> Packages wich should be installed
##################################################################
installPkg()
{
	 [ -n "${DistroBasedOn-x}" ] && shootProfile
	for i in $@
	do
		echo -n "Installing $i on $DistroBasedOn."
		case $DistroBasedOn in
			Debian | debian )
				 apt-get -y install $i
				 ;;
			SuSe | suse )
				 zypper install $i
				 ;;
			Mandrake | mandrake )
				 echo "do not know how to install $i package on mandrake "
				 ;;
			RedHat | redhat )
				 yum -y install $i
				 ;;
			*)
				 echo "\$DistroBasedOn variable has not a valid value. Value : ||$DistroBasedOn||"
				 ;;
		esac
	done
return 0
}
##################################################################
# Purpose: Compare hostname
# Arguments: 
#	$1 => string 
# Return : TRUE if $1 = `hostname` false otherwhise.
##################################################################
hostnameGet(){
	 [[ "`hostname`" = "$1" ]] && return $TRUE || return $FALSE
}
##################################################################
# Purpose: Return TRUE if $1 is confirmed, or keypresed
# Arguments: 
#	$1 => string 
# Return : TRUE if $1 user presses the key.
##################################################################
confirm(){
	 read -sn 1 -p "$1 [y/N]? "
	 [[ $REPLY = [Yy] ]] && return $TRUE|| return $FALSE
}

#===  FUNCTION  ================================================================
#          NAME:  downloadAndExec
#   DESCRIPTION:  downloads and launches a script
#    PARAMETERS:  $1 = script name
#       RETURNS:  TRUE if OK
#===============================================================================

downloadAndExec ()
{
	scriptname=$1
	workdir="/tmp"
	workscript=$workdir/$scriptname
	[ -f $workscript ] && rm $workscript
	wget $MASTERURL/bin/$scriptname -O $workscript
	chmod +x $workscript
	$workscript
	[[ $? -eq 0 ]] && rm $workscript && return $TRUE || return $FALSE

}	# ----------  end of function downloadAndExec  ----------

return 0

New monitoring for standard environments

From some time now we have started using a “monitoring distribution”.
It uses passive monitoring where it can be used, and has lots of plugins for standard services.

It gives you a nice output, check it out:

Selection_045

Selection_044

Credit has to be given to http://mathias-kettner.de/
All you need to know is here => http://omdistro.org/

To install the FULL pack you just need to do the following:


# where to start => http://omdistro.org/

wget http://ftp.cica.es/epel/6/x86_64/epel-release-6-8.noarch.rpm
rpm -ivh epel-release-6-8.noarch.rpm
rpm -Uvh "https://labs.consol.de/repo/stable/rhel6/x86_64/labs-consol-stable.rhel6.noarch.rpm"
yum update
yum install omd-1.10
omd create monit
yum install check-mk-agent
omd create lab

After this, you can go to your server http://server/monit/ with user omdadmin and password omd , and you will find yourself adding machines to the monitoring.

The agents can be found on the server itself, at the /opt/omd/sites/monit/share/agents/ directory (I may misspell but is arround that )

Visual Basic script to pack, zip and send a full report.

Each first Monday of each month I needed to prepare and send a report to some customers.

Everything was on windows, so Visual Basic was the one to be used.
Nowadays I would ask to reinstall the machine to be able to use Powershellv3 or so, but at that point this is what I got.

'compress folder and send a zip file
'TODO : 


Option Explicit

Dim outputFileZip, outputErrFile, mailTextFile, reportFolder, emailFrom, emailTo, dateStamp, emailSubject
outputFileZip = "D:\Omv_Reports\OmniVisionGQR-ReportLastMonth.zip"
mailTextFile = "D:\Omv_Reports\report_scripts\file1.txt"
outputErrFile = "D:\Omv_Reports\report_scripts\file2.txt"
reportFolder = "D:\Omv_Reports\OmniVisionGQR-LastMonth"
emailFrom = "edited@edited.es"
emailTo = "edited@edited.es;edited@edited.es;edited@edited.es"
emailSubject = "Summary report for the A3S "
dateStamp = Date()

' -=-=-=-=-=-=-=-=-=-
''''''''''''''' Prepare log file
dim loggit_logfilename, loggit_fso, loggit_silent, tempFile

loggit_logfilename = outputErrFile
loggit_silent = true   ' log file only or with MsgBox/Echo
set loggit_fso = CreateObject("Scripting.FileSystemObject")
set tempFile = loggit_fso.OpenTextFile(loggit_logfilename, 2, True)
tempFile.Write ""
tempFile.Close 

' -=-=-=-=-=-=-=-=-=-
sub loggit (msg)
	Dim stream
    set stream = loggit_fso.OpenTextFile(loggit_logfilename, 8, True)
    stream.writeline date & " " & time & ": " & msg
    stream.close
    if not loggit_silent then
       WScript.echo msg 
    end if   
end sub
' -=-=-=-=-=-=-=-=-=-

'''''''''''''''''''' Log file ready. :)
loggit "Started..."

' -=-=-=-=-=-=-=-=-=-
' Funtion used on date string
' -=-=-=-=-=-=-=-=-=-

Dim objFSO,strDate

Function padDate(intNumber)
	if intNumber <= 9 Then
		padDate = "0" & CStr(intNumber)
	Else
		padDate = CStr(intNumber)
	End If
End Function

' -=-=-=-=-=-=-=-=-=-


'''''''''''''''beggin
Dim arrResult
'''ZipFolder funcion is at the end of this file. 
loggit "Launch ziping function."
arrResult = ZipFolder( reportFolder, outputFileZip )
loggit "Finish ziping function. "

If arrResult(0) = 0 Then
    If arrResult(1) = 1 Then
       'WScript.Echo "Done; 1 empty subfolder was skipped."
	   loggit "Done; 1 empty subfolder was skipped."
    Else
       'WScript.Echo "Done; " & arrResult(1) & " empty subfolders were skipped."
	   loggit "Done; " & arrResult(1) & " empty subfolders were skipped."
    End If
Else
    'WScript.Echo "ERROR ziping the lastMonth folder. Call Marc Riera ASAP" & Join( arrResult, vbCrLf )
	loggit "ERROR ziping the lastMonth folder. We are going to look at this and report back to you as soon as possible. :: " & Join( arrResult, vbCrLf )
End If

'''' Writing Done before the file gets into the may body text.
loggit "Done."
''''
'prepare mail content
Dim objEmail
Set objEmail = CreateObject("CDO.Message")
Const ForReading = 1, ForWriting = 2, ForAppending = 8
Dim fso, f, f2
Set fso = CreateObject("Scripting.FileSystemObject")
'Open the file for reading
Set f = fso.OpenTextFile(mailTextFile, ForReading)
Set f2 = fso.OpenTextFile(outputErrFile, ForReading)
'The ReadAll method reads the entire file into the variable BodyText
Dim BodyText
BodyText = f.ReadAll&f2.ReadAll
'Close the file
f.Close
f2.Close
Set f = Nothing
Set f2 = Nothing


'send mail
objEmail.From = emailFrom
objEmail.To = emailTo
objEmail.Subject = emailSubject & " --- " & dateStamp
objEmail.TextBody = BodyText
objEmail.AddAttachment outputFileZip
objEmail.Configuration.Fields.Item _
    ("http://schemas.microsoft.com/cdo/configuration/sendusing") = 2
objEmail.Configuration.Fields.Item _
    ("http://schemas.microsoft.com/cdo/configuration/smtpserver") = _
        "172.16.23.135"
objEmail.Configuration.Fields.Item _
    ("http://schemas.microsoft.com/cdo/configuration/smtpserverport") = 25
objEmail.Configuration.Fields.Update
objEmail.Send

loggit "File has been send"

Set objFSO = CreateObject("Scripting.FileSystemObject")
'MsgBox	Year(Date) & "-" & padDate(Month(Date)) & "-" & padDate(Day(Date))
strDate = Year(Date) & "-" & padDate(Month(Date)) & "-" & padDate(Day(Date))

loggit "Moving zip file to zip file with date"
If (objFSO.FileExists(outputFileZip)) Then
	objFSO.MoveFile outputFileZip , outputFileZip & "-" & strdate & ".zip"
End If
loggit "Moved."
loggit "Done. :) "

'''
Function ZipFolder( myFolder, myZipFile )
' This function recursively ZIPs an entire folder into a single ZIP file,
' using only Windows' built-in ("native") objects and methods.
'
' Last Modified:
' March 8, 2012
'
' Arguments:
' myFolder   [string]  the fully qualified path of the folder to be ZIPped
' myZipFile  [string]  the fully qualified path of the target ZIP file
'
' Return Code:
' An array with the error number at index 0, the source at index 1, and
' the description at index 2. If the error number equals 0, all went well
' and at index 1 the number of skipped empty subfolders can be found.
'
' Notes:
' [1] If the specified ZIP file exists, it will be overwritten
'     (NOT APPENDED) without notice!
' [2] Empty subfolders in the specified source folder will be skipped
'     without notice; lower level subfolders WILL be added, wether
'     empty or not.
'
' Based on a VBA script (http://www.rondebruin.nl/windowsxpzip.htm)
' by Ron de Bruin, http://www.rondebruin.nl
'
' (Re)written by Rob van der Woude
' http://www.robvanderwoude.com
' (Re) Joan Marc Riera - Bull - added outfile for errors/log

    ' Standard housekeeping
    Dim intSkipped, intSrcItems
    Dim objApp, objFolder, objFSO, objItem, objTxt, objErrFSO, objErrTxt
    Dim strSkipped

    Const ForWriting = 2

    intSkipped = 0
	
    ' Make sure the path ends with a backslash
    If Right( myFolder, 1 ) <> "\" Then
        myFolder = myFolder & "\"
	Else
		loggit "	- The path does  not end with backslash " & "\" 
    End If

    ' Use custom error handling
    On Error Resume Next

    ' Create an empty ZIP file
    Set objFSO = CreateObject( "Scripting.FileSystemObject" )
    Set objTxt = objFSO.OpenTextFile( myZipFile, ForWriting, True )
    objTxt.Write "PK" & Chr(5) & Chr(6) & String( 18, Chr(0) )
    objTxt.Close
    Set objTxt = Nothing

    ' Abort on errors
    If Err Then
        ZipFolder = Array( Err.Number, Err.Source, Err.Description )
		loggit "	- ERROR !!!! - " & Err.Number & " " & Err.Source & " " & Err.Description & "------" 
        Err.Clear
        On Error Goto 0
        Exit Function
    End If
    
    ' Create a Shell object
    Set objApp = CreateObject( "Shell.Application" )
	
	loggit "	- copy files to compressed folder . started. "
    ' Copy the files to the compressed folder
    For Each objItem in objApp.NameSpace( myFolder ).Items
        If objItem.IsFolder Then
            ' Check if the subfolder is empty, and if
            ' so, skip it to prevent an error message
            Set objFolder = objFSO.GetFolder( objItem.Path )
            If objFolder.Files.Count + objFolder.SubFolders.Count = 0 Then
                intSkipped = intSkipped + 1
            Else
                objApp.NameSpace( myZipFile ).CopyHere objItem
            End If
        Else
            objApp.NameSpace( myZipFile ).CopyHere objItem
        End If
    Next
	loggit "	- copy files to compressed folder . finished ."
	
    Set objFolder = Nothing
    Set objFSO    = Nothing

    ' Abort on errors
    If Err Then
        ZipFolder = Array( Err.Number, Err.Source, Err.Description )
		loggit "	- ERROR !!!! - " & Err.Number & " " & Err.Source & " " & Err.Description & "------" 
        Set objApp = Nothing
        Err.Clear
        On Error Goto 0
        Exit Function
    End If
	loggit "	- Compression started.  "
    ' Keep script waiting until compression is done
    intSrcItems = objApp.NameSpace( myFolder  ).Items.Count
    Do Until objApp.NameSpace( myZipFile ).Items.Count + intSkipped = intSrcItems
        WScript.Sleep 2000
    Loop
    Set objApp = Nothing
	loggit "	- Compression finished.  "
	
    ' Abort on errors
    If Err Then
        ZipFolder = Array( Err.Number, Err.Source, Err.Description )
		loggit "	- ERROR - " & Err.Number & " " & Err.Source & " " & Err.Description & "------" 
        Err.Clear
        On Error Goto 0
        Exit Function
    End If

    ' Restore default error handling
    On Error Goto 0

    ' Return message if empty subfolders were skipped
    If intSkipped = 0 Then
        strSkipped = "No items skipped."
    Else
        strSkipped = "skipped empty subfolders"
    End If

    ' Return code 0 (no error occurred)
    ZipFolder = Array( 0, intSkipped, strSkipped )
End Function

' close error file, here is where loggit sends the strings
Set fso = Nothing

WScript.Quit(0)

SH ( not bash) menu driven for AIX (for example)

Useful menu structure.
Nicely used when bash is not around, and you need some interactive menu.

#!/bin/sh
# sample.mnu
# A simple script menu under Unix
# Main logic starts at MAIN LOGIC
# The logo will be displayed at the top of the screen
LOGO="Oracle info report  Menu - Marc Riera (Bull 2012)"
#------------------------------------------------------
# MENU PROMPTS
#------------------------------------------------------
# A list of menu prompts to be displayed for the user.
# The list can be modified.
# In this first list, enter the menu prompt as it should appear
# on the screen for each of the letters A - L. In this example
# menu pick variables emenu through lmenu are blank as there
# are no menu selections for keys E through L.

amenu="a.  whoami.sql			:	rem Purpose:    Reports information about your current database context";
bmenu="b.  ver.sql			:	rem Purpose:    Show database version with options intalled"; 
cmenu="c.  usedsp.sql			:	rem Purpose:    Used space in Meg by segment type"; 
dmenu="d.  datos_generals.sql		:	rem Purpose:    General info about the connected database"; 
emenu="e.  version_bbdd.sql		:	rem Purpose:    Display v\$version database ";
fmenu="f.  tablespaces_size_free_percent2.sql	: rem Purpose:	Tablespaces size free percent 2  ";
gmenu="g.  redologs_status.sql 		:	rem Purpose:	Redologs status";
hmenu="h.  datafile_size_free_percent.sql	: rem Purpose:	Datafile size free percent ";
imenu="i.  control_file.sql 		:	rem Purpose:	Control files";
jmenu="j.  nls_parameters.sql		: 	rem Purpose:	Nls parameters ";
kmenu="k.  db_options.sql		:	rem Purpose:	Database options ";
lmenu=" "                                 ;
mmenu="m   Payroll Menu "                 ;

#------------------------------------------------------
# MENU FUNCTION DEFINITIONS
#------------------------------------------------------
 
# Define a function for invalid menu picks
# The function loads an error message into a variable
badchoice () { MSG="Invalid Selection ... Please Try Again" ; } 
 
# For each prompt displayed above, there is a list of 
# commands to execute in response to the user picking the
# associated letter.
# They are defined as functions
# apick () through lpick () where
# apick () corresponds to the menu
# prompt amenu which is selected
# selected by pressing a or A.
# bpick () corresponds to the menu
# prompt bmenu which is selected by
# pressing b or B and so on.
# Any menu item that is not
# assigned a set of commands, is
# assigned
# the function badchoice () as a default for that pick.
# If the user
# selects a menu key that is assigned
# to badchoice (). This function
# causes an error message to be
# displayed on the screen.
# To add items to this second
# list, replace badchoice ()
# with the commands to run when
# that letter is pressed.
# The following steps simply define
# the functions, but do not cause
# any shell program steps to be executed.

# Command example : sqlplus -S sys/gonna@bi as sysdba @ask_version.sql
apick () { sqlplus -S sys/<YOURPASSHERE>@<INSTANCE> as sysdba @whoami.sql ; echo Press Enter ; read DUMMY ; }
bpick () { sqlplus -S sys/<YOURPASSHERE>@<INSTANCE> as sysdba @ver.sql; echo Press Enter ; read DUMMY ; }
cpick () { sqlplus -S sys/<YOURPASSHERE>@<INSTANCE> as sysdba @usedsp.sql; echo Press Enter ; read DUMMY ; }
dpick () { sqlplus -S sys/<YOURPASSHERE>@<INSTANCE> as sysdba @datos_generals.sql; echo Press Enter ; read DUMMY ; }
epick () { sqlplus -S sys/<YOURPASSHERE>@<INSTANCE> as sysdba @version_bbdd.sql; echo Press Enter ; read DUMMY ; }
fpick () { sqlplus -S sys/<YOURPASSHERE>@<INSTANCE> as sysdba @tablespaces_size_free_percent2.sql; echo Press Enter ; read DUMMY ; }
gpick () { sqlplus -S sys/<YOURPASSHERE>@<INSTANCE> as sysdba @redologs_status.sql; echo Press Enter ; read DUMMY ; }
hpick () { sqlplus -S sys/<YOURPASSHERE>@<INSTANCE> as sysdba @datafile_size_free_percent.sql; echo Press Enter ; read DUMMY ; }
ipick () { sqlplus -S sys/<YOURPASSHERE>@<INSTANCE> as sysdba @control_file.sql; echo Press Enter ; read DUMMY ; }
jpick () { sqlplus -S sys/<YOURPASSHERE>@<INSTANCE> as sysdba @nls_parameters.sql ; echo Press Enter ; read DUMMY ; }
kpick () { sqlplus -S sys/<YOURPASSHERE>@<INSTANCE> as sysdba @db_options.sql ; echo Press Enter ; read DUMMY ; }
lpick () { badchoice ; }
mpick () { payroll.mnu ; }

#------------------------------------------------------
# DISPLAY FUNCTION DEFINITION
#------------------------------------------------------
# This function displays the menu.
# The routine clears the screen, echoes
# the logo and menu prompts
# and some additional messages.
# Note that this definition does
# not cause the function to
# be executed yet, it just defines
# it ready to be executed.

themenu () {
	# clear the screen
	clear
	echo `date`
	echo
	echo "\t\t\t" $LOGO
	echo
	echo "\t\tPlease Select:"
	echo
	echo "\t\t\t" $amenu
	echo "\t\t\t" $bmenu
	echo "\t\t\t" $cmenu
	echo "\t\t\t" $dmenu
	echo "\t\t\t" $emenu
	echo "\t\t\t" $fmenu
	echo "\t\t\t" $gmenu
	echo "\t\t\t" $hmenu
	echo "\t\t\t" $imenu
	echo "\t\t\t" $jmenu
	echo "\t\t\t" $kmenu
	echo "\t\t\t" $lmenu
	echo "\t\t\tx. Exit"
	echo
	echo $MSG
	echo
	echo Select by pressing the letter and then ENTER ;
}
 
#------------------------------------------------------
# MAIN LOGIC
#------------------------------------------------------
# Every thing up to this point has been to define
# variables or functions.
# The program actually starts running here.

# Clear out the error message variable
MSG=

# Repeat the menu over and over
# Steps are:
# 1. Display the menu
# 2. 'read' a line of input from the key board
# 3. Clear the error message
# 4. Check the answer for a or A or b or B etc. and dispatch
#    to the appropriate program or function or exit
# 5. If the entry was invalid call the badchoice () function
#    to initialize MSG to an error message
# 6. This error message is used when setting up the menu
#    for a menu pick that is valid but has no command
#    associated with it.

while  true
do
	# 1. display the menu
	themenu

	# 2. read a line of input from the keyboard
	read answer

	# 3. Clear any error message
	MSG=

	# 4. Execute one of the defined functions based on the
	#    letter entered by the user.

	# 5. If the choice was E through L, the pre-defined
	#    function for that pick will execute badchoice ()
	#    which loads an error message into MSG  
		case $answer in
		        a|A) apick;;
		        b|B) bpick;;
		        c|C) cpick;;
		        d|D) dpick;;
		        e|E) epick;;
		        f|F) fpick;;
		        g|G) gpick;;
		        h|H) hpick;;
		        i|I) ipick;;
		        j|J) jpick;;
		        k|K) kpick;;
		        l|L) lpick;;

	#      If the user selects =91x=92 to exit then break out
	#      of this loop
			x|X) break;;

	# 6. If the entry was invalid call the badchoice function
	#    to initialize MSG to an error message
			*) badchoice;;
																 
		esac

	#     Do it again until the user enters =91x=92.
	done

exit 0

Dataguard alike using bash and cron

Oracle EE is quite more expensive than SE, however sometimes you need a standby replica “just in case”.

This is how you get a standby replica on a different server, without breaking any Oracle licensing rules:

1) the following script has to be on both servers. It calls itself on remote.
2) set up a cronjob to launch the script on the primary server.
3) Check the log.

I must say that this script needs some big cleaning and ordering, but it works.

#!/bin/bash +x
#joanmarcrieraATgmailDOCcom

#TODO:: fix the output processing when DEBUG=1
#TODO:: too many paths that must become variables


STB_SERVER=oranode2                                     #the Standby database server # needs passwordless authentication to user oracle
LOG_FILE=/home/oracle/scripts/db_status.log             #

#load the environment variables
. ~/.bash_profile

# When debug=1 the scripts must be executed by hand , and most functions will not work as expected. Processing the functions output must be fixed
DEBUG=0

# is the local database OK ?
function check_status ()
{
[[ $DEBUG -eq 1 ]] && echo "DEBUG=check_Status"
#expected status "ok=1 ko=0"
INSTANCE_STATUS=$(echo "select STATUS from v\$instance; " | sqlplus -S / as sysdba|grep -E "MOUNTED|OPEN"|wc -l )
INSTANCE_ARCHIVER=$(echo "select ARCHIVER from v\$instance; " | sqlplus -S / as sysdba|grep "STARTED"| wc -l)
INSTANCE_DB_STATUS=$(echo "select DATABASE_STATUS from v\$instance; " | sqlplus -S / as sysdba|grep "ACTIVE" |wc -l)
INSTANCE_ACTIVE_STATE=$(echo "select ACTIVE_STATE from v\$instance; " | sqlplus -S / as sysdba|grep "NORMAL"|wc -l)

[[ $INSTANCE_STATUS -eq 1 ]] && [[ $INSTANCE_ARCHIVER -eq 1 ]] && [[ $INSTANCE_DB_STATUS -eq 1 ]] && [[ $INSTANCE_ACTIVE_STATE -eq 1 ]] && echo "STATUS = OK" || echo "STATUS = KO"

}

#check tns ping .
function tnsping_status ()
{
[[ $DEBUG -eq 1 ]] && echo "DEBUG=tnsping_status"
TNSPING=$(tnsping eypesa|tail -n1 |cut -d' ' -f1 |grep "OK"|wc -l)
[[ $TNSPING -eq 1 ]] && echo "STATUS = OK" || echo "STATUS = KO"
}

# get value for logcount on local database (open mode)
function archive_log_count ()
{
[[ $DEBUG -eq 1 ]] && echo "DEBUG=archive_log_count"
#don't return current, because is not finished. so return the last already switched.
#select max(SEQUENCE#) from V$ARCHIVED_LOG;
ARCHIVE_LOG=$(echo "select max(SEQUENCE#) from V\$ARCHIVED_LOG;" |sqlplus -S / as sysdba|tail -n2|head -n1|tr -d ' \t\r\f')
echo "LAST = $ARCHIVE_LOG"
}

#ask database to check for its local logcount  (standby mode)
function archive_log_count_stb ()
{
#select max(recid) from v$log_history;"
ARCHIVE_LOG_STB=$(echo "select max(recid) from v\$log_history;"|sqlplus -S / as sysdba |tail -n2 |head -n1|tr -d ' \t\r\f')
echo "LAST_STB = $ARCHIVE_LOG_STB"
}

#ask for remote logcount (standby mode)
function archive_log_count_stb_remote ()
{
ssh $STB_SERVER ~/scripts/db_status.sh archive_log_stb
}

#send new archive files to STB_SERVER
function send_files ()
{
#seq 452 $(expr 454 - 1)
FROM_FILE=$(ssh $STB_SERVER ~/scripts/db_status.sh archive_log_stb |cut -d'=' -f2)
TO_FILE=$(~/scripts/db_status.sh archive_log |cut -d'=' -f2)

[[ $FROM_FILE -eq $TO_FILE ]] && echo "SAME STATUS ON BOTH SIDES. " && exit 0

#files to send FROM TO -1
for FILE in $(seq $FROM_FILE $TO_FILE)
do
rsync -v /u01/eypesa/ORADATA/DB_ARCHIVE/1_${FILE}_* $STB_SERVER:/u01/eypesa/ORADATA/DB_ARCHIVE
done
}

#set remote database to recover automatically with archive logs already sent
function recover_standby_remote ()
{
#start recovery on standby
ssh $STB_SERVER ~/scripts/db_status.sh recover_local_standby
}

#recover with local archive logs
function recover_standby_local ()
{
sqlplus -S / as sysdba @/home/oracle/scripts/recover_standby_database.sql
}

#switch logfile  (debug purposes)
function switch_local ()
{
echo "alter system switch logfile;" |sqlplus -S / as sysdba
}

#delete old archive log files, this is used on STB_DATABASE because it does not have rman to clean it.
function clean_old_files_remote ()
{
ssh $STB_SERVER "find /u01/eypesa/ORADATA/DB_ARCHIVE -mtime +7 -exec rm -rf {} \;"
}

#procedure when launched by cron.
function cron_mode ()
{
exec 1>> $LOG_FILE
exec 2>> $LOG_FILE
echo "CRON EXECUTION : $(date +%Y%m%d%H%M)"
#check status
check_status
tnsping_status
archive_log_count
archive_log_count_stb_remote
#send files
send_files
#recover
recover_standby_remote
clean_old_files_remote

echo "###### END CRON EXECUTION : $(date +%Y%m%d%H%M)"

}

case $1 in
        status)
                check_status
                ;;
        tnsping)
                tnsping_status
                ;;
        archive_log)
                archive_log_count
                ;;
        archive_log_stb)
                archive_log_count_stb
                ;;
        send_files)
                send_files
                ;;
        recover_remote_standby)
                recover_standby_remote
                ;;
        recover_local_standby)
                recover_standby_local
                ;;
        switch_logfile_local)
                switch_local
                ;;
        cron_mode)
                cron_mode
                ;;
        check_all)
                check_status
                tnsping_status
                archive_log_count
                archive_log_stb
                ;;
        *)
                echo "Usage: $0 {cron_mode|status|tnsping|archive_log|archive_log_stb|send_files|recover_remote_standby|recover_local_standby|switch_logfile_local|check_all}"
                exit 1
esac

Use python to handle email, translate , and publish to wordpress

This weekend I tried to refresh my python with one single idea in mind.

But now I don’t have time to explain it, so here comes the code:

#!/usr/bin/env python
#
#joanmarcrieraATgmail(dot)com

from GmailClass import GmailClass
import goslate
import datetime, xmlrpclib
import time

from wordpress_xmlrpc import Client, WordPressPost
from wordpress_xmlrpc.methods.posts import NewPost
from datetime import datetime


gs=goslate.Goslate()

correo=GmailClass("jmraccount@gmail.com","Password","Foldera")
correo.connect()
correo.select_mailbox()
no_leidos=correo.get_unreaded_list()

wp_url = 'http://www.joanmarcriera.es/blog/xmlrpc.php'
wp_username = 'username'
wp_password = 'password'




for un_correo_no_leido in no_leidos[0].split():
      
    print "\nProcessing one unread..."
    msg=correo.get_a_mail(un_correo_no_leido)
    titulo_original=correo.get_the_subject(msg)
    print "\t " + titulo_original
    body_original=correo.get_the_body(msg)
    titulo_es=gs.translate(titulo_original,'es')
    body_es=gs.translate(body_original,'es')
      
    title=titulo_es
    content=body_es
    
    post = WordPressPost()
    post.title = title
    post.content = content
    post.date = datetime(2014, 8, 10, 12, 34, 2, 860000)
    post.post_status = 'draft'
    try:
        wp = Client(wp_url,wp_username,wp_password) 
        wp.call(NewPost(post))
    except  ServerConnectionError as e:
        #do something with e
        #or by now pass
        print "serverconnection error"
        pass
    except ProtocolError as e:
        #do something or pass
        print "protocol error"
        pass
    else:
        del post
        del wp
    
    print "\t Done."
    print "Now wait 10 seconds\n"
    time.sleep(10)
    
#!/usr/bin/env python
#
#joanmarcrieraATgmail(dot)com

import sys
import imaplib
#not needed #import getpass
import email
import email.header
import datetime

#class to manage gmail.
class GmailClass(object):
   
    #Basic initialize.
    #Here is were we need to declare which account and folder we would like to process
    #example a=GmailClass('jmr@gmail.com','superpass','kate')
    def __init__(self, account, pwsd, folder):
        self.folder = folder
        self.email_account = account
        self.email_password = pwsd

    #automatic for test 
    def prepare(self):
        self.connect()
        self.select_mailbox()
        
    
    #Connect to the mailbox.
    def connect(self):
        self.mailbox = imaplib.IMAP4_SSL('imap.gmail.com')
        try:
            rv, data = self.mailbox.login(self.email_account, self.email_password)
        except imaplib.IMAP4.error:
            print "LOGIN FAILED!!! "
            sys.exit(1)
        print rv, data
        print "You can now select a mailbox"
    
    #just if you want to know how many folders do you have    
    def list_mailboxes(self):
        #needs pretty print
        rv, mailboxes = self.mailbox.list()
        if rv == 'OK':
            print "Mailboxes:"
            print mailboxes
            print "Select one of them to be able to process it."
    
    #go into the folder
    def select_mailbox(self):
        rv, data = self.mailbox.select(self.folder)
        if rv == 'OK':
            print "You can now start processing mailbox...\n"
        else:
            print "ERROR: Unable to open mailbox ", rv    
            
    #which mails are still unread
    def get_unreaded_list(self):
        rv, data = self.mailbox.search(None, "UnSeen")
        if rv != 'OK':
            print "No messages found!"
            return
        else:
            print "Messages found "
            return data
    
    #return a mail object
    #the list from get_unreaded_list needs to be looped data[i]
    def get_a_mail(self,num):
        rv, data = self.mailbox.fetch(num, '(RFC822)')
        if rv != 'OK':
            print "ERROR getting message", num
            return
        else:
            msg = email.message_from_string(data[0][1])
            return msg

     
    #from a mail object get the subject   
    def get_the_subject(self,msg):
        decode = email.header.decode_header(msg['Subject'])[0]
        subject = unicode(decode[0])
        #print 'Message Subject: %s' % ( subject)
        return subject
    
    #from a mail object the date
    def get_the_date(self,msg):
        #print 'Raw Date:', msg['Date']
        return msg['Date']

        
    #from a mail object get the body    
    def get_the_body(self,msg):
        maintype = msg.get_content_maintype()
        if maintype == 'multipart':
            for part in msg.get_payload():
                if part.get_content_maintype() == 'text':
                    return part.get_payload()
        elif maintype == 'text':
            return msg.get_payload()


    #method to process mailbox    
    def process_mailbox(self):
        """
        Do something with emails messages in the folder.  
        For the sake of this example, print some headers.
        """
        rv, data = self.mailbox.search(None, "ALL")
        if rv != 'OK':
            print "No messages found!"
            return
    
        for num in data[0].split():
            rv, data = self.mailbox.fetch(num, '(RFC822)')
            if rv != 'OK':
                print "ERROR getting message", num
                return
    
            msg = email.message_from_string(data[0][1])
            decode = email.header.decode_header(msg['Subject'])[0]
            subject = unicode(decode[0])
            print 'Message %s: %s' % (num, subject)
            print 'Raw Date:', msg['Date']
            # Now convert to local date-time
            date_tuple = email.utils.parsedate_tz(msg['Date'])
            if date_tuple:
                local_date = datetime.datetime.fromtimestamp(
                    email.utils.mktime_tz(date_tuple))
                print "Local Date:", \
                    local_date.strftime("%a, %d %b %Y %H:%M:%S")
                
    #Close the sessions at the end
    def close(self):
        self.mailbox.close()
        self.mailbox.logout()