UNIX Shell Scripting
The basic concept of a shell script is a list of commands, which are listed in the order of execution. A good shell script will have comments, preceded by a pound sign, #, describing the steps. There are conditional tests, such as value A is greater than value B, loops allowing us to go through massive amounts of data, files to read and store data, and variables to read and store data, and the script may include functions.
Shell scripts and functions are both interpreted. This means they are not compiled. Both shell scripts and functions are ASCII text that is read by the Korn shell command interpreter. When we execute a shell script, or function, a command interpreter goes through the ASCII text line by line, loop by loop, test by test and executes each statement, as each line is reached from the top to the bottom.
1. A shell is an environment in which we can run our commands, programs, and shell scripts. There are different flavors of shells, just as there are different flavors of operating systems. Each flavor of shell has its own set of recognized commands and functions.
Korn Shell /bin/ksh OR /usr/bin/ksh
The standard Shell on Linux is the Bourne Again shell (bash) shell, and some others use Bourne shell (sh) as the default. To find your Default shell:
Echo $SHELL
2. If the special characters are used to in a way that their special meaning is not needed then they must be escaped. To escape or remove its special function, the character must be preceded with a backslash \ or enclosed within ““forward tic mark.
\ ( ; # $ ? & * ( ) [ ] ` ‘ “ +
3. A shell script can be executed in the following ways:
Ksh shell_script_name
a. Will create a Korn shell and execute the shell_script_name in the newly created Korn shell environment.
shell_script_name
The script will execute in the shell that is declared on the first line of the shell script. If no shell is declared on the first line of the shell script, it will execute in the default shell, which is the user’s system-defined shell. Executing in an unintended shell may result in a failure and give unpredictable results.
Declare the Shell in the Shell Script:
We MUST declare the shell in the very first line of the script. If no shell is declared, the script will execute in the default shell, defined by the system for the user executing the shell script.
#! /usr/bin/ksh OR #! /bin/ksh.
#! /usr/bin/sh OR #! /bin/sh declares a Bourne shell
#! /usr/bin/ksh OR #! /bin/ksh declares a Korn shell
#! /usr/bin/csh OR #! /bin/csh declares a C shell
#! /usr/bin/bash OR #! /bin/bash declares a Bourne-Again shell
Comments:
We have to write code that is readable and has an easy flow. This involves writing a script that is easy to read and easily maintained, which means that it must have plenty of comments describing the steps.
#!/usr/bin/ksh
#
# SCRIPT : NAME_of_SCRIPT
# AUTHOR : AUTHORS_NAME
# DATE : DATE_of_CREATION
# REV : 1.1.A (Valid are A, B, D, T and P)
(For Alpha, Beta, Dev, Test and Production)
## PLATFORM : (SPECIFY: AIX, HP-UX, Linux, Solaris or Not platform dependent)
#
# PURPOSE : Give a clear, and if necessary, long, description of the purpose
Of the shell script.
#
# REV LIST:
# DATE : DATE_of_REVISION
# BY : AUTHOR_of_MODIFICATION
# MODIFICATION : Describe what was modified, new feature, etc.
#
# set -n # Uncomment to check your syntax, without execution.
# # NOTE: Do not forget to put the comment back in or
# # the shell script will not execute!
# set -x # Uncomment to debug this shell script (Korn shell only)
##########################################################
########### DEFINE FILES AND VARIABLES HERE ##############
##########################################################
############### DEFINE FUNCTIONS HERE ####################
##########################################################
##########################################################
################ BEGINNING OF MAIN #######################
##########################################################
# End of script
Functions:
we can write a piece of code, which is used over and over, just once and use it without having to rewrite the code every time. We just call the function instead.
Form of the function
Function function_name
{
Commands to execute
}
Or
function_name ()
{
Commands to execute
}
Control Structures:
If ... then Statement
If [test_command]
Then
Commands
Fi
If ... then ... else Statement
If [test_command]
Then
Commands
Else
Commands
Fi
If ... then ... elif ... (else) Statement
If [test_command]
Then
Commands
Elif [test_command]
Then
Commands
Else (Optional)
Commands
Fi
For ... in Statement
For loop_variable in argument list
Do
Commands
Done
While Statement
While test_command_is_true
do
Commands
Done
Until Statement
until test_command_is_true
do
Commands
Done
Case Statement
case $variable in
match_1)
commands_to_execute_for_1
;;
match_2)
commands_to_execute_for_2
;;
match_3)
commands_to_execute_for_3
;;
.
.
.
*) (Optional - any other value)
commands_to_execute_for_no_match
;;
esac
Break Continue, exit and return:
It is sometimes necessary to break out of a for or while loop, continue in the next block of code, exit completely out of the script, or return a function’s result back to the script that called the function.
Break: Is used to terminate the execution of the entire loop, after completing the execution of all of the lines of code up to the break statement. It then steps down to the code following the end of the loop.
Continue: Is used to transfer control to the next set of code, but it continues execution of the loop.
Exit: Will do just what one would expect: It exits the entire script. An integer may be added to an exit command (for example, exit 0), which will be sent as the return code.
Return: Is used in a function to send data back, or return a result, to the calling script.
Here Document:
A here document is used to redirect input into an interactive shell script or program. The input that is supplied must be the exact data that the program is expecting, and many programs will fail if spaces are added to the input.
Syntax for a Here Document
program_name <<LABEL
Program_Input_1
Program_Input_2
Program_Input_3
Program_Input_#
LABEL
EXAMPLE:
/usr/local/bin/My_program << EOF
Randy
Robin
Rusty
Jim
EOF
Most useful UNIX commands in Shell Scripting
Passwd : Change user password
|
Pwd : print working Directory
|
Cd : Change directory
|
Ls : list of files in a directory
|
Wildcards : * matches any number of
characters, ‘?’ matches a single
character
|
File : print the type of file
|
Cat : Displays the content of the file
|
Pr : Displays the contents of a file
|
pg or page : Display the contents of a file one
page at a time
|
More : Display the contents of a file one
page at a time
|
Clear : Clears the screen
|
Cp or
Copy : copy a file
|
Chown : Change the owner of a file
|
chgrp : Change the group of a file
|
chmod : Change file modes, permissions
|
rm : Remove a file from the system
|
mv : Rename a file
|
mkdir : Create a directory
|
rmdir : Remove a directory
|
grep : Pattern matching
|
Egrep : grep command for extended
regular expressions
|
find : Used to locate files and directories
|
>> : Append to the end of a file
|
> : Redirect, create, or overwrite a
file
|
| : Pipe, used to string commands
together
|
|| : Logical OR—command1 ||command2
—execute command2 if command1
fails
|
& : Execute in background
|
&& : Logical AND—command1 &&
command2—execute command2 if
command1 succeeds
|
date : Display the system date and time
|
echo : Write strings to standard output
|
sleep : Execution halts for the specified
number of seconds
|
wc : Count the number of words, lines,
and characters in a file
|
head : View the top of a file
|
tail : View the end of a file
|
diff : Compare two files
|
sdiff : Compare two files side by side
(requires 132-character display)
|
spell : Spell checker
|
lp, lpr,
enq, qprt : Print a file
|
lpstat :Status of system print queues
|
enable : Enable, or start, a print queue
|
disable : Disable, or stop, a print queue
|
rwall : Display a message to all users on a
remote host
|
rsh or remsh
: Execute a command, or log in, on a
remote host
|
df : File systems statistics
|
ps : Information on currently running
processes
|
netstat : Show network status
|
vmstat :Show virtual memory status
|
cal : Display a calendar
|
who : Display information about users on
the system
|
w : Extended who command
|
whoami : Display $LOGNAME or $USER
environment parameter
|
who am I : Display login name, terminal, login
date/time, and where logged in
|
f, finger : Information about logged-in users
including the users .plan and .project
|
talk : Two users have a split screen
conversation
|
write : Display a message on a user’s screen
|
wall : Display a message on all logged-in
users’ screens
|
iostat : Show input/output status
|
uname : Name of the current operating
system, as well as machine
information
|
sar : System activity report
|
basename : Base filename of a string
parameter
|
man : Display the on-line reference manual
|
su : Switch to another user, also known as
super-user
|
cut : Write out selected characters
|
awk : Programming language to parse
characters
|
sed : Programming language for character
substitution
|
vi : Start the vi editor
|
emacs : Start the emacs editor
|
`command` : Command substitution
|
( ) : Run the enclosed command in a sub-
shell
|
(( )) : Evaluate and assign value to variable
and do math in a shell
|
$(( )) : Evaluate the enclosed expression
|
[ ] : Same as the test command
|
[[ ]] : Used for string comparison
|
$( ) : Command substitution
|
Variables:
A variable is a character string to which we assign a value. The value assigned could be a number, text, filename, device, or any other type of data. A variable is nothing more than a pointer to the actual data.
To assign a variable to point to data:
$VARIABLE_NAME=”value_to_assign”
To view the data assigned to the variable
Echo $UPPERCASE or print $UPPERCASE for variables
Cat $UPPERCASE if the variable pointing to a file, as a command structure
Command-Line Arguments:
The command-line arguments $1, $2, $3,...$9 are positional parameters, with $0 pointing to the actual command, program, shell script, or function and $1, $2, $3, ...$9 as the arguments to the command.
Note: The positional parameters, $0, $2, etc., in a function, are for the function’s use and may not be in the environment of the shell script that is calling the function. Where a variable is known in a function or shell script is called the scope of the variable.
Shift Command
The shift command is used to move positional parameters to the left; for example, shift causes $2 to become $1. We can also add a number to the shift command to move the positions more than one position; for example, shift 3 causes $4 to move to the $1 position.
Special Parameters $* and $@
The $* special parameter specifies all command-line arguments.
The $@ special parameter also specifies all command-line arguments.
The “$*” special parameter takes the entire list as one argument with spaces between.
The “$@” special parameter takes the entire list and separates it into separate arguments.
Double Quotes “, Forward Tics ’, and Back Tics `
We use “, double quotes, in a statement where we want to allow character or command substitution.
We use ‘, forward tics, in a statement where we do not want character or command substitution.
We use `, back tics, in a statement where we want to execute a command, or script, and have its output substituted instead; this is command substitution.
Math in a Shell Script
The Korn shell let command and the ((expr)) command expressions are the most commonly used methods to evaluate an integer expression.
Operators
++ —
|
Auto-increment and auto-decrement, both prefix and postfix
|
+
|
Unary plus
|
-
|
Unary minus
|
!~
|
Logical negation; binary inversion (one’s complement)
|
* / %
|
Multiplication; division; modulus (remainder)
|
+ -
|
Addition; subtraction
|
<< >>
|
Bitwise left shift; bitwise right shift
|
<= >=
|
Less than or equal to; greater than or equal to
|
< >
|
Less than; greater than
|
== !=
|
Equality; inequality (both evaluated left to right)
|
&
|
Bitwise AND
|
^
|
Bitwise exclusive OR
|
|
|
Bitwise OR
|
&&
|
Logical AND
|
||
|
Logical OR
|
Chmod Permissions
4000
|
Sets user ID on execution
|
2000
|
Sets group ID on execution
|
1000
|
Sets the link permission to directories or sets the save-text attribute for files.
|
0400
|
Permits read by owner
|
0200
|
Permits write by owner
|
0100
|
Permits execute or search by owner
|
0040
|
Permits read by group
|
0020
|
Permits write by group
|
0010
|
Permits execute or search by group
|
0004
|
Permits read by others
|
0002
|
Permits write by others
|
0001
|
Permits execute or search by others
|
Setting Traps
When a program is terminated before it would normally end, we can catch an exit signal. This is called a trap.
Exit Signals
0
|
Normal termination, end of script
|
1
|
Hang up, line disconnected
|
2
|
Terminal interrupt, usually CONTROL-C
|
3
|
Quit key, child processes to die before terminating
|
9
|
kill -9 command, cannot trap this type of exit status
|
15
|
kill command’s default action
|
24
|
Stop, usually CONTROL-z
|
Uppercase or Lowercase Text
VARIABLE VALUES
Expected input: TRUE
Real input: TRUE
Possible input: true TRUE True True, etc...
UPCASING
UPCASEVAR=$(echo $VARIABLE | tr ‘[a-z]’ ‘[A-Z]’)
DOWNCASING
DOWNCASEVAR=$(echo $VARIABLE | tr ‘[A-Z]’ ‘[a-z]’)
‘[a-z]’ ‘[A-Z]’ Used for lower to uppercase
‘[A-Z]’ ‘[a-z]’ Used for upper to lowercase
Typeset -u VARIABLE: The -u switch to the typeset command is used for uppercase.
Example: EXAMPLE:
Typeset -u VARIABLE
VARIABLE=”True”
echo $VARIABLE
TRUE
Typeset -l VARIABLE: always translate characters to lowercase
Example:
Typeset -l VARIABLE
VARIABLE=”True”
echo $VARIABLE
true
Cron Tables
15 3 8 1 * /usr/local/bin/somescript.sh
|
Minute (0 through 29)
Hour (0 through 23)
Day of the Month (1 through 31)
Month (1 through 12)
Weekday (0 - 6 for Sunday to Saturday)
at Command
Like a cron table, the at command executes commands based on time. Using the at command we can schedule a job to run once, at a specific time.
Silent Running
To execute a script in silent mode (without any output to the screen), we can use the following syntax:
/PATH/script_name 2>&1 > /dev/null
Getopts
The getopts command is built in to the Korn shell. It retrieves valid command-line options specified by a single character preceded by a - (minus sign) or + (plus sign). To specify that a command switch requires an argument to the switch, it is followed by a : (colon). If the switch does not require any argument then the : should be omitted. All of the options put together are called the OptionString, and this is followed by some variable name. The argument for each switch is stored in a variable called $OPTARG. If the entire OptionString is preceded by a : (colon), then any unmatched switch option causes a ? to be loaded into the VARIABLE.
Example:
SECS=0 # Initialize all to zero
MINUTES=0
HOURS=0
DAYS=0
PROCESS= # Initialize to null
while getopts :s:m:h:d:p: TIMED 2>/dev/null
do
case $TIMED in
s) SECS=$OPTARG
;;
m) (( MINUTES = $OPTARG * 60 ))
;;
h) (( HOURS = $OPTARG * 3600 ))
;;
d) (( DAYS = $OPTARG * 86400 ))
;;
p) PROCESS=$OPTARG
;;
\?) usage
exit 1
;;
esac
done
(( TOTAL_SECONDS = SECONDS + MINUTES + HOURS + DAYS ))
