String Math allows manipulation
of strings: search/replace, +, -
Constants,
passed on the
command line or set as environment variables, which override variables
in the script. Powerful feature allows any variable to be passed on the
command line
Debugger
improvements show environment, improved paste support, and
expect timeout operations
Introduction
Automatic
Login
expect-lite was born into a multi-host environment, and therefore
there was the early requirement to remote login to a host (usually a
Linux machine) allocated by a sharing facility such as the commercial
package, LSF. Therefore, a remote host must be specified on the command
line where the script will wake up and begin executing.
Three methods of remote login are supported:
telnet
ssh with password
ssh with keys (no password)
ssh with keys is the preferred method, since no password is required,
however this requires some environment setup before hand.
Even
when no remote host is required, it is best to log into the localhost,
since the underlying Expect has problems with synchronization (between
send and expect strings) when a remote host is not specified. The
provided shell script 'setup_local_ssh.sh' will setup the localhost
with ssh keys. It is only necessary to run it once:
./setup_local_ssh.sh
Based on the command line arguments passed, expect-lite will do the
following:
Automatically log into the remote host via telnet or
ssh
(if ssh
keys have been
previously setup)
Change directory to the passed directory (if included)
expect-lite will log into remote-host-018, change directory to /etc/
and
begin to send commands from basic_ping_test.elt to the remote-host. The
parameters of remote_host and cmd_file are required, as expect-lite
must know what host to log into, and what script (aka command file) to
run.
Environment
Variables
The
following environment variables permit the customization of expect-lite
without having to edit the expect-lite script itself. It is recommended
that commonly used environment variables be placed in the .bashrc or
.cshrc file where they will be automatically set upon login.
Env
Var
Description
EL_REMOTE_HOST
Name or IP of remote
host
EL_CMD_FILE
Name of expect-lite script to run
EL_USER_DIR
Change to this directory upon login before
executing script
EL_CONNECT_METHOD
The method expect-lite uses to connect to remote
host. Valid options are: none|telnet|ssh|ssh_key
Default is none
EL_CONNECT_USER
User name to use for login on remote host
(telnet|ssh)
EL_CONNECT_PASS
Password to use for login on remote host
(telnet|ssh)
EL_LOG_EXT
The log file name
extension string, default=".log" (see *LOG)
EL_INFINITE_LOOP=N
Infinite Loop
protection value, set N to user's default value (default=5000) (see *INFINITE_LOOP)
EL_DELAY_WAIT_FOR_HOST
Delay
(in ms) to wait for host in Not Expect, and Dynamic Var Capture. 100 ms
is a good value for a local LAN, 200 ms if running across high speed
internet
EL_SHELL
Start this shell
(default=bash) when using EL_CONNECT_METHOD=none
EL_DYN_VAR_PROMPT
Set the Dynamic
Variable capture method to require a prompt (default=1). Disable by
setting to 0.
EL_*
Any other shell environment variables starting with
EL_ will become constants
Special
Character Sequences
Although basic expect-lite scripts can be created
by simply cutting and pasting text from a terminal window into a
script, and adding '>' '<' characters, such as:
>pwd </home/user
Much more functionality is available. expect-lite interprets
punctuation characters at the
beginning of each line in the script:
label -
used
for jumping to
labels (see conditionals)
~filename var=value
includes
a
expect-lite
automation file, useful
for creation of common variable files, or 'subprograms/subroutines'
(see include files)
In addition to the above special characters, blank lines or lines
starting with any non-special character are allowed to make the script
file more readable.
Sometimes control characters must be sent. Currently all
control characters are supported (from ^@ to ^\):
^C or break, when it is desired to stop a running
program
e.g.
>^C or >>^C
^] or escape from telnet, e.g. >^] This will
bring
the
script
to the telnet prompt, then use: >quit
^<any char> will send a control
character to
the
remote-host,
such as ^D to logout
Control characters must be the first two characters on the line
(e.g.[^][A-Z]). The sequence of ^C anywhere else in the line will not
be interpreted as a control-C, but rather the two characters '^' 'C'.
Lastly, any line beginning with an
asterisk '*' will be interpreted as
an expect-lite directive. Directives change the behaviour of
expect-lite.
Expect-lite
Directives
*~filename
Include
a fail
script, which expect-lite runs only if the main script fails (see fail script)
Script will run to
completion, and returns 1, if
failure occurred during execution, or 0, if pass
*NOINTERACT
Will
ignore breakpoints (*INTERACT), good for regression testing (see directives)
**SHELL=<shell>
Configuration
directive which sets the shell immediately after automatic login to
remote host (see setting the
shell)
*INTERACT
Sets a
breakpoint and places
user in
interact
mode, which pauses the script and turns control of the keyboard over to
the user (see Tips
&
Techniques: Interact)
*FORK
<session>
Multiple
session support.
Directs expect-lite to open a new session and spawns a newshell (see multiple sessions)
*SHOW
VARS
Debug
information, displays
all expect-lite variables. Can be used in interact mode.
*EOLS LF
*EOLS CRLF
Controls end of line sequence
sent to remote
host, either line feed, or carriage return & line feed.
*DVPROMPT
*NODVPROMPT
Enable/disable the
Dynamic Variable capture method to require a prompt
(default=*DVPROMPT). (see Dynamic
Variable capture method)
*INFINITE_LOOP
<N>
Sets/resets
the
infinite loop protection value, which is a total count of all loops in
script. Default is 5000
*NOINCLUDE
Ignore include files
(was default behavour in library mode)
Logging
Directives
*LOG
*NOLOG
Enable/disable
logging to a file. Will create file <script_name>.log (same as
$arg0.log) if no path/file_name is supplied.(see managing
logging)
*LOG
<file_name>
*LOGAPPEND <file_name>
Enable
logging to a user specified path/file_name. When invoked on the CLI,
the specified file_name must end in ".log" to avoid ambiguity
*INFO
*NOINFO
Enable/disable
informational messages
*WARN
*NOWARN
Enable/disable
warning messages
*DEBUG
*NODEBUG
Enable/disable
debug
messages
*NOCOLOR
Disables color output, best
when logging
output
*TIMESTAMP
<ISO|YMD|DMY|MDY>
Prints Date and Timestamp for
each command sent,
ISO is default
*NOTIMESTAMP
Disables timestamp printing
IPv6 Support
expect-lite is an application, and will use communications
protocols such as telnet and ssh to connect to a remote host. If telnet
or ssh is IPv6 enabled, then expect-lite can/will
use it.
However, it may be important to include IPv6 addresses in expect-lite
scripts. expect-lite now fully supports IPv6 addresses including short
hand notation with double colons, e.g. 2001:db8::dead
Care has been taken to have expect-lite do the right thing with IPv6
addresses, however, it may be necessary to be more explicit when using
conditionals (if/then/else), see conditionals
and IPv6.
General Tips for
writing scripts
Here are some simple tips when script writing:
Use reasonable timeouts, if 30 seconds is needed to
get a
response, set the timeout at 45 or 60 seconds, not 600.
There is no cost to changing the timeout, timeout
values
can
also be variables
Beware of expect-lite using regex, when creating
lines
like:
<0.005 secs (5 micro secs)
The parentheses is used by the regex engine,
instead
escape
these characters: <0.005 secs \(5 micro secs\) (see Use of Regex in expect-lite)
or use '<<' which does not use regex,
and
does not require escaping: <<0.005 secs (5 micro secs)
(see using non-regex evaluation)
Use the expect character '<' or
'<<'
often. Check for valid
results
when possible. A script which expects nothing will never fail! (see Test Failure)
Use
printable comments ';' and ';;' often. Think of it as
writing
a note to oneself, it will make reading log files much easier. As of
version 3.7.0 printable comments will be coloured blue (this is
user configurable).
Variable assignments use no spaces e.g. $var=value. Note no
spaces
around the equals sign, this permits leading spaces in the value such
as $var= myvalue
Debugging
Expect-lite Scripts
Expect-lite was written to permit quick and easy automation of
repetitive tasks. However, sometimes a little additional debugging is
required.
Instant-Interact
(as
of version 3.5.0) or instant breakpoint,
provides instant debugging assistance. Pressing
^\ (control+backslash) will force expect-lite to drop into interact
mode, a mode which turns control of the keyboard over to the user, so
that one may type directly to the process on the remote host. To return
to the script, type '+++' (three pluses). During interactive mode,
timeouts do not apply, *NOFAIL is automatically enabled, and the script
will wait indefinitely for the
user to exit this debugging mode. Switching between multiple sessions
(see *FORK) is
possible via typing the
*FORK command in interact mode (see *FORK
debugging).
Additionally, the interact mode (or Integrated Debugger Environment)
performs three primary functions: 1) connecting the user to the remote
host or device under test, 2) monitoring special commands prefaced with
the escape key for stepping, and other functions, and 3) the debugger
will allow expect-lite script lines to be executed by either typing
directly or pasting them into the IDE.
The periods are preceded by a backslash to indicate to regex that a
period must be returned rather than the regex dot '.' which indicates
any character. The \d indicates any single digit.
Because regex is always enabled for expected results, some 'escaping'
of characters must be done when using '<'. The following
characters
must be escaped with a backslash to convey their literal meaning:
Example
Escaped
Character
(abc)
\(abc\)
parenthesis
[abc]
\[abc\]
square
brackets
\
\\
backslash
.
\.
period
$
\$
dollar
sign
+
\+
plus
*
\*
asterisk (or star)
|
\|
pipe
Regex is a very powerful syntax unto itself, and one can create quite
complex regex strings. There is an excellent regex
cheat sheet
created by David Child, which is very helpful. That said, one does not
need to be a regex expert to use expect-lite, it is possible to use
regex in a very basic way, see examples for more info.
Using
<< for non-regex evaluation
There
are times when the use of regex is not desired, or 'escaping'
characters is burdensome. As of version 3.6.0, expect lines can begin
with '<<' which will not use regex, but rather literal
evaluation
of expected results.
Handling
HTML/XML Tags in
expect lines
The << may cause a backward compatibility problem, if the
script was expecting an HTML tag, such as
>grep body my_html_file.html <<body.+>
In
the above example, the script is expecting a body tag with 1 or more
characters between body and the closing greater-than of the tag.
However the new feature will not interpret this line as regex, but
rather interpret the beginning of the line as <<, and
evaluate it
as a literal. The method to force expect-lite to evaluate the line as
regex, is to change the line is slightly:
>grep body my_html_file.html <[<]body.+>
By
making the first expected character as a regex character class, it
signals to expect-lite that this line should be evaluated as regex,
rather than a literal.
Using
Variables
Expect-lite supports two types of variables:
Static, which are bound at invocation
Dynamic, which are bound during execution of the
script
Lines starting with a '$' indicate variables which are assigned and set
at script invocation, and therefore are static. In expect-lite variable
names will always be preceded with a '$'. Variable names are limited to
the following set of characters
[A-Za-z0-9_]. When making variable assignments, there must NOT be a
space between the variable name and the equals sign:
$myvar=myvalue
There are times when it is necessary to unassign or unset a variable,
because,
it is desired to have the variable be dereferenced by the underlying
shell (see bash shell), or that pseudo-
arrays are being used (see pseudo-arrays).
To unassign a variable (v4.8.0), merely assign it an empty value.
$myvar=
Dynamic Variables
Sometimes it is desirable to bind a variable during the execution of
the script. Dynamic variables fill this need by utilizing Expect's
built-in regex capture mechanism. Only the portion of the match inside
the parenthesis will be bound to the variable. The format of the line
is as follows:
+$somevar=text that is not put into the var (text which is put into the var)
Dynamic Variables are always bound to Expect output, meaning text which
is returned from the remote host. Therefore something must be sent to
the remote host, before text can be returned. A typical usage would be:
> command +$myvar=command output (capture value) more command output
If the value of the var is successfully captured then expect-lite will
print:
Assigned Var:somevar=sometext which is put into the var
If, however the dynamic var is not successfully captured, expect-lite
will print:
Assigned Var:somevar=__NO_STRING_CAPTURED__
Avoid using an overly general capture as it will tend to capture too
much, or the wrong info. e.g:
> command +$myvar=\n(.*)
Instead use a specific capture when possible. When capturing
the current directory, the current directory will always start with
'/', and the path will
be made up of a known character set:
> echo $PWD +$mypwd=\n(/[a-zA-Z0-9/\-_]+)
Expect-lite will print to stdout the value of the assigned variable,
assisting the script writer in understanding the value that is bound to
the variable during runtime. The value of "__NO_STRING_CAPTURED__"
indicates that the
regex pattern used did not match the return data. It is best to start
using this feature with short scripts targeted at capturing the desired
information. Examples
can be found in
the example section of this document.
Math
Functions with Variables
There are times when writing a script it may be necessary to perform
arithmetic or bitwise operations, also known as math functions.
Expect-lite inherits expect/tcl math functions and natively supports
math/bitwise operations on a variable, using the following slightly odd
notation:
$myvar=1 =$myvar + 2 * 10
In the above example, 2 will be multiplied by 10 and added to value of
$myvar (1). The result of 21 will be placed in $myvar, overwriting the
previous value. Since multiplication has a higher precedence than
addition, (2 * 10) will be performed before the addition to $myvar.
Parenthesis may be used to enforce user defined precedence. In the
following example, the result will be 23:
$myvar=1 =$myvar + 2 * (10 + 1)
Both bitwise and arithmetic operators are supported with the following
precedence (left to right):
bitwise
<<
>> & ^ |
shift left,
shift
right, bit
and, bit exclusive or, or
arithmetic
* /
% + -
multiply,
divide,
modulo, add,
subtract
If $myvar does not exist before the math function, it will be
initialized to blank and then math functions will be performed. In the
following example the result, $myvar, will be 20:
=$myvar + 2 * 10
However, realistically, only the '+' operator works as expected in the
above example. Using other operators will more than likely yield a
syntax error:
Because of the expect/tcl inheritance of math functions, all operations
will be performed as integers (or whole numbers) unless a decimal (or
real) number is used explicitly. In the following example the result is
2.5:
$five=5 =$myvar + $five / 2.0
However, the following (because of the use of integers) example will
have the result of 3:
$five=5 =$myvar + $five / 2
Math functions can be used with static and dynamic variables.
String Math
Functions with Variables
With the introduction of version 4.6.0, String Math functions have been
added. The functions which enable simple manipulation of strings are:
Concatenation using +
Removal using -
Search and Replace using / or //
In order to use string math functions, the variable must already be
defined as a string (think: non-numeric). The following is an example
of concatenation:
The above example searches for a string of digits, dot, digits, dot,
digits, dot, then lastly captures the last set of digits. It then
replaces the entire IP address with captured value (using the \1),
resulting in $myip equaling 50.
Pseudo
Arrays
Expect-lite supports pseudo arrays, which are different from a real
array (e.g. item[1], item[2],...). Pseudo arrays are made possible by
the fact that variables on the left side of the equals can be
dereferenced at assignment time. For example, below, the variable
$count will be dereferenced (or resolved) prior to assignment:
$myarray$count=some $value
When placed in a loop with a looping variable of $count, a pseudo array
of $myarray1, $myarray2, $myarray3,... will be created. Individual
values can be retrieved from the array by using the index (or $count)
such as $myarray23.
The pseudo array name (variable + index) must be a valid variable name,
in the set of characters (A to Z, a to z, 0 to 9, and underscore) and
no spaces.
Constants
Expect-lite Constants are represented as variables which are passed
into expect-lite at runtime or retrieved from environment variables
starting with EL_. Constants will override any script
variable already defined inside the script, are immutable, and
cannot be changed. Constants can be used to change the behaviour of the
script.
For example, one might invoke the following test as:
Inside the script basic_ping_test.txt any reference to $local_eth would
be set to eth2, thus allowing the script actions to be changed based on
the constant passed at invocation. Constants will override any script
variable already defined inside the script.
Shell
Variables
Expect-lite allows the use of shell variables, which can be more
powerful than the built-in variable mechanism. However shell variables
will only be resolved by the shell. For example, assigning current
working directory to a shell var:
> PWD=`pwd` > echo $PWD
However, the following test would fail, since $PWD is a shell variable,
not an expect-lite variable:
> PWD=`pwd` >pwd < $PWD
Shell variables (or environment variables) must be dereferenced by the
shell. Similarly, expect-lite variables must be dereferenced by
expect-lite.
It is possible to read a shell variable into an expect-lite variable by
using the dynamic variable method:
> echo $PWD +$mypwd=\n(/[a-zA-Z0-9/\-_]+)
Undefined
Variables
Since expect-lite allows the use of shell variables, it is possible
that a variable will not be known to expect-lite (but known to the
shell). Expect-lite will attempt to dereference all variables, the ones
which are not defined remain in variable form (perhaps it is a shell
variable). However, it can be useful to test if a variable is an
expect-lite variable. As of version 4.1.2, in a conditional (if
statement) undefined variables will be automatically defined as blank (
or ""). See conditional for more
information.
Environment
Variables starting with EL_
When
expect-lite starts up it will automatically search the environment
variables (aka shell variables), and import any found starting with EL_
as constants. This enables setting environment variables as passwords,
or remote hosts.
For example, the environment variable EL_sudopass might be set to
"mysecret". In the script, the variable would be used as:
Conditional (if/then/else) statements are natively supported in
expect-lite. The conditional uses a c-style syntax with a question mark
(?) at the beginning of the line, and double colon to indicate the else
statement, using the format ?cond?action::else_action
Although spaces are not required around the conditional characters (?
and ::), it is recommended for ease of reading. The comparison
operators are: '==', '!=' ,'>=',
'<=', '>'
and '<'. If the compared values can be evaluated as a numbers,
then
larger and less than will yield expected results. A simple conditional
example:
In the above example if $age is larger than or equal to 55 then the
action 'echo "freedom at 55!". If $age is less than 55 then the action
'echo "keep working!"' will be sent.
Each action or else_action is as if it began on a separate line. Since
in the above example '>' is used (e.g. >echo) the echo
line will
be sent to the remote host. Any expect-lite action (see Special Characters) can
be placed after
the second question mark. For example, an include
file may be executed based on a conditional:
The message prints: what was evaluated (expr { "ppc" == "i386"
}), 'then', and 'else' actions, as well as a result (true or
false)
IPv6
Addresses
Because IPv6 addresses are quite long, it is a common short hand to
represent a long string of zeros with a double colon '::', such as
2001:db8::f00d. The conditional also uses a double colon to signal the else portion of the conditional, so
what happens when:
Fortunately, as of version 4.2.2, it will be as expected, $my_addr will
be set to 2001:db8::feed
If in doubt, be sure to place spaces around the else double colon, as spaces are
never allowed in an IPv6 address.
Undefined variables
Undefined variables can be tested in a conditional (as of version
4.1.2). In and only in a conditional, variables unknown to expect-lite
are dereferenced as blank. This allows testing if a variable has been
defined such as:
$blank= ?if $TEST ==$blank? ; === echo this :: ; === echo that ?if $TEST != ? ; === echo this :: ; === echo that
The first example, is setting a variable to blank and then
testing for equality. The second example demonstrates testing
inequality with blank. Either conditional will work, and it is a matter
of style which one is preferred.
Labels
Conditionals are limited to a single line. Sometimes this is too
limiting, as it would be nice to have several commands be executed
based on the success of a conditional. To support this, the concept of labels has been
introduced. A label
is defined as having the first character a '%'. Although the label line
itself does nothing, it provides a location to the conditional to Jump To Label. The
following is a
simple example of using a conditional in conjunction with a label:
When a label is the action of a conditional, there is an implied Jump to Label.
Lines between the
conditional and the label will be skipped. The Jump to Label
action is no longer
limited to only
jumping forward. (see Looping with
Conditionals)
Multiple labels with the same name are permitted. For example, a
conditional action may be Jump
to
Label %SKIP. However this is only recommended for looping (as of
version 4.1.1 expect-lite searches backwards
for labels) Of course it is easier for the script reader if
clearer label names are used. Labels may contain spaces, as in this
example:
; === Test conditional jump to label ? $jump == true ?%move along, nothing to see >echo "1" >echo "2" >echo "3" %move along, nothing to see
To assist reading the logs, the action Jump to Label will
generate a
message to standard out (captured in the log). From the above example,
the following would be printed:
Jumping to label:%move along, nothing to see
Looping with
Conditionals & Labels
Simple looping is supported by allowing jump to label
backwards in the script. The Repeat
Loop is the easiest loop to implement:
Because of jump to label
default behaviour has been changed in version 4.1.1 to jump backwards,
it is no longer important to assign unique
looping labels. Non-looping labels, as
illustrated in the previous section, must (as of version 4.1.1) be
unique.
Also included in the above example is incrementing an expect-lite
variable: +$count
This will add 1 to the value of $count. If $count is not an integer,
the value of $count will remain unchanged (can't add 1 to a string).
As part of the
looping
enhancement, there
is infinite loop
protection.
The maximum amount of looping is defined with the directive
*INFINITE_LOOP N. This value is decremented with
each iteration of all loops for the entire script. Typically this would
be in the range of 1000 to 10000 to be safe. For example, if a complex
expect-lite script had 4 loops each with 100 iterations, the
directive *INFINITE_LOOP should be set larger than 400. As of version
4.5.0, the environment variable EL_INFINITE_LOOP is used to override
the default value.
Using
Code Blocks with Conditionals, While Loops and Foreach Loops
expect-lite is a line oriented scripting language, but there are times
when it is convenient to group lines together. As of version 4.2.0, the
grouping of lines can be achieved through code blocks. A quick example
using a code block and a conditional:
A code block begins with a [ (left
square bracket) and ends with a ] (right
square bracket). When used with an if statement, it allows more than
one "line" to be executed based on the evaluation of the if statement.
Conditionals with code blocks also support the else construct, such as:
However, using the square brackets on the same line is still not supported, since it appears
ambiguous:
; === NOT SUPPORTED Conditional using code block then and else statement ? $case == true ? [ :: [ >echo " 1" ]
Nested code blocks can also be used. In the following example each if
statement is using a code block for the then statement, and a non-code
block for the else statement:
Code blocks can also be used for quick looping using an implied while
loop construct. In the following example, the expression at the
beginning of the code block will be evaluated, if true, the lines
inside the code block will be executed. The expression (e.g. $i < 3)
will be evaluated each time the code block has completed execution.
Once the
expression evaluates false (in the example $i is equal to 3), the
script will continue on the line after the bottom of the code block.
The <test> operators are:
Test
Action
==
equal
!=
non equal
<=
less than or equal
>=
greater than or equal
<
less than
>
greater than
The following example tests if $i is less than 3, and continues to loop
until $i is equal or greater than 3:
;; == Nested while loops, should repeat "hello ken" 3 times ;; == with 2 "bye now's inside each outer loop > $i=0 [ $i < 3 >echo "hello ken" <ken $j=0 [ $j < 2 >echo "bye now" > +$j ] +$i ]
Foreach
Loops
Code blocks also support foreach loops, where a loop will iterate of a
known number of items. Each iteration, the variable will be set to the
next item in the list. Items in a list must be separated by spaces. As
of version 4.6.1, the list variable will be normalized (extra spaces
removed) before executing the loop. Use
string math functions to create a list of items separated by spaces.
In the following example, a list of network connected devices are named
after planets. The variable $planet will be set to mercury, then ping
it. Next time through the loop, $planet will be set to venus, then ping
it, and so on. After the last item, neptune, the loop will complete and
the script will continue after the bottom of the code block.
Of course, foreach loops can be nested with other foreach loops, or
while loops.
Infinite loop protection applies both the "jump-to-label" method as
well as to the "while-loop" and "foreach-loop" methods, each iteration
of the loop
decrements the infinite loop counter (default is 5000). It may
be a good idea to set the infinite loop value to something low, when
first writing code block loops, just in case:
*INFINITE_LOOP 20
Slowing Loops with sleep
For years, I have resisted putting a sleep function in expect-lite.
Sleep is often abused by scripters who blindly put in a 60 second sleep
(or longer). I have seen 5 minute (600 seconds) sleeps put into scripts
because the scripter didn't want to poll for a change in device status.
Long sleeps are just bad programming, and should be avoided.
Version 4.9.0 adds native support for sleep. There are very valid
reasons why in the polling loop, the scripter may want to slow things
down, and only check device status, say every 5 seconds. This can
easily be done with a while loop:
; === while loop with sleep $eth_status=DOWN [ $eth_status == DOWN >ip link show dev eth0 +$eth_status=(DOWN|UP) :5 ]
The script will check eth_status of eth0 every 5 seconds, and continue
(exit the while loop) after $eth_status is no longer DOWN.
While the script is sleeping, if *INFO is enabled, expect-lite will
print out a ticker of "dots" to indicate that the script is sleeping:
$ ip link show dev eth0 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
Assigned Var:eth_status=UP
Sleeping: 5 ....+
Sleep "dots" include dots, plus (every 5 seconds) and a number (every
10 seconds), to show the progress of the sleep.
Sleeping: ....+....10....+....20....
Mili-second sleep is also supported by using a decimal value for sleep,
e.g. 0.005 equals 5 mili-seconds
#sleep 1/10th of a second :.1
Of course, sleep value can also be a variable:
#sleep using a variable $sleeper=5.5 :$sleeper
Please use sleep wisely.
Script Help with --help -h
Once the script running, it might be handy to add help to make it more
user-friendly or refresh the mind when running the script at a later
date. As of version 4.3.0, t is possible to use the build-in script
help function which when --help or -h are typed on the command line, a
help section and assigned
variables will be printed.
The help section is defined in the script as a block of text which
starts with a triple-semi-colon, and ends with a triple-semi-colon,
text inside this block will not
be executed by the script. For example:
Then when running the script, with the --help or -h cli argument, the
following will be printed
$ ./test_bash_here_doc.txt -h Help for: ./test_bash_here_doc.txt
Test: Test bash here doc
Assumptions: bash Platforms: anywhere bash runs
Displaying assigned variables, which can be overridden with a Constant on command line e.g var=value tempfile=junk
The second portion of the help is a list of variables inside the script
which can be overridden as constants (see Constants)
on the command line. The assigned variables section of the help will
always be printed regardless if a triple-semi-colon help block has been
defined in the script. This is useful for older scripts which do not
have a triple-semi-colon help block.
Constants are a very powerful feature of expect-lite which allow the
behaviour of the script to change without having to edit the script. In
the example above, if the script were run with the following arguments:
When the script runs, rather than creating a file called 'junk', it
will instead create a file in the /tmp directory called mydata.txt
Expect-lite
& Bash
Although not limited to working with bash, bash is invoked upon logging
into the remote host, and therefore will be discussed more here.
Since the bash shell is well documented, and supported, and therefore
can be
leveraged to assist in expect-lite's limitations such as looping and
branching (see limitations below). A simple bash loop inside
expect-lite can be created for example:
$set=1 2 3 4 5 >for i in $set #expanded by expect-lite >{ >echo $i #expanded by bash >}
In the above example, $set is an expect-lite variable, not a bash
variable. An expect-lite variable is always declared before its use
(e.g. $set= 1 2 3 4 5). If a variable cannot be dereference by
expect-lite it is passed to the shell. The loop will execute after the
final "}" line is sent to the
remote-host. Because of this "execute after" effect, an expect-lite
'<' line can not be used within the bash loop.
Conditionals via bash are also supported, as in this example:
$max=4 >if [ $max -ne 5 ]; then >echo "max is not equal to 5" >else >echo "we have a winner" >fi
More complex bash assists can be constructed. For example, one may want
to set a looping variable based on the processor type of machine at the
time (some hosts may be faster than others). The answer is echoed, and
then captured using a dynamic variable. Note that that 'proc' is
prepended to the answer, as it enables the capture to be much more
specific.
Note that 'proc' is outside the parenthesis, and therefore will not be
captured into the expect-lite dynamic variable $loop_counter.
Using
Bash to create executable expect-lite scripts
It is possible to make expect-lite scripts executable. The old way was
to use an embedded bash script, but version 4.0.1 makes this task more
standard. Insert the following at the top of
the expect-lite script:
#!/usr/bin/env expect-lite
The final step to making the script executable is to use a chmod
command:
chmod 775 my_expect-lite_script.elt
Now the expect-lite script can be run my typing the following:
The old bash scriptlet method can be found in
expect-lite.proj/examples/test_masquerade.txt
Include
Files
Include files are a quick way to develop script snippets which can be
included into larger scripts or to include a common variable file. When
an include file is executed, it is as if the file were just pasted into
the script file, and therefore has access to the variable space of the
main
script, and can modify that variable space as well. In this example, a
common variable file is "sourced":
# Source common variable file ~asic_vars.inc
Common
functions, such as telnet'ing to the DUT, are a good use of
include files. With version 4.3.0, it is possible to pass var=value
parameters to the include file. The var=value operate just like
constants overriding variables inside the include file (with a scope of
just the include file). In the following example it is possible to
override the variable 'user' and assign the value of 'root':
; === Connect to DUT ~dut_connect.inc user=root
Include filenames can also be assigned in a variable, such that the
file names can be declared at the top of the script but used later
within
the script. For example:
# Source Var file to be used $asic_include=asic_vars.inc ... ~$asic_include
Include
files can also be used for a simple regression, including each
sub-script (see regression example)
. If the *NOFAIL directive is used, and something inside an
include file should fail, a failure message will be reported at the
completion of the include file:
##Include Result: FAILED: dns_connect.inc
Limitations on include file
names
With the version of 4.7.0, a line beginning wih tilde character has a
dual purpose, marking a fuzzy expect line, and also include files. This
means that an include file name must NOT
start with a greater than '<' or an equals '='. All other characters
should be allowed in an include file name.
Embedded
Expect
There are situations when expect-lite cannot provide a solution. Rather
than force the user to abandon the simplicity of expect-lite, an
embedded expect is supported. This functionality is provided to assist
expect-lite rather than replace it. Lines beginning with '!' will be
interpreted as native expect (see limitations below).
Embedded Expect requires the script writer to know native
expect.
It runs with some variable protection, and provides access to
expect-lite variables and constants, as well as the expect variable
timeout. This permits the script writer to query the timeout, store it
away, and then reset the timeout later.
# Preserve old timeout using embedded expect !set user_namespace(TIMEOUT) $timeout # set timeout to 15 seconds - max time to wait for login prompts @15 >telnet $dut_ip <login >>root <assword >>my_secret > # Set the timeout back to old timeout @$TIMEOUT
Expect-lite variable/values are stored in the tcl array
user_namespace()
with the variable name as the index name. Constants are stored
similarly in the tcl array cli_namespace().
Expect code is collected in islands
and executed after an island
shore
is reached. The island
shore
is represented by lines that start with any special character,
except '#' and blank
lines. However, it is recommended that the '>' be used, as this
implies a "wait for prompt."
Branching and looping are supported in embedded expect e.g.:
Embedded expect can fail the entire expect-lite script by calling the
built-in
_el_fail_test function:
!if { $arch == "ppc" } { ! puts "\narch IS $arch\n" ! } else { ! puts "\narch is NOT ppc, but $arch\n" ! _el_fail_test ! } >
TCL files can be sourced, and functions can be declared and called from
within embedded expect.
Limitations
of Embedded
Expect:
Limited support of expect global variables. To
declare a
variable
at the top level as global use "set ::<var>
<value>" The
double
colon '::' makes var global (this is a TCL standard)
tcl/expect switch
command not supported
expect statements must be on one line (as in the
example
above)
Please remember, this functionality is to provide assistance to
expect-lite
rather than replace it. (see What's
Missing)
Expect-lite Directives '*'
Expect-lite directives are indicated by lines which start with the
asterisk, '*'. The directives change the behaviour of expect-lite in a
user defined manner. These directives include:
fail script - run when the script fails
*~include_fail_script
user defined prompt - for those non-unix/linux remote
hosts
*/regex/
stopping the script - *TERM & * FAIL
setting the default shell - **SHELL
multiple sessions - for hard to script environments
*FORK
control end of line character(s) - *EOLS LF, *EOLS
CRLF
control dynamic variable capture method to require a prompt
or not *DVPROMPT, *NODVPROMPT
allow script to run to completion regardless of failure
status *NOFAIL
ignore breakpoints (used for regression) *NOINTERACT
Directives can also be placed on the command line at run time. This is
useful for controlling logging, such as *NOINFO or *NOCOLOUR.
Additionally, *NOFAIL can be useful when debugging a script, since the
script will not stop prematurely.
Fail
Script *~include_fail_script
The purpose of the fail script is to clean up after the failed script,
reseting times, deleting temporary files and such. It is a special
include script which is declared near the top of the script:
*~clean_up.inc
Should the script fail, the fail script will be "sourced".
The scope of the fail script is always local to the running script.
This allows normal include scripts to declare a separate fail script
from the main script. Thus enabling a different clean up mechanism for
the include script.
The fail script mechanism should no longer be needed for script
development. The technique was useful before debugging tools like
instant-interact and the powerful IDE built into version 4.0.1.
User
Defined Prompt */prompt/
With each '>' command an implied "wait for prompt" occurs (see implementation details).
The predefined
prompts are based on standard shell prompts (>%$#). However, it
is
quite possible that expect-lite will not be interacting with a shell,
but another application (such as gdb) or device which does not issue a
shell-like prompt.
As of version 3.1.5, the user may define a custom prompt. User defined
prompts are defined from the newline (beginning of line) to the end of
the prompt. For example:
*/my_prompt /
The succeeding '>' lines will interpret a prompt as
the
standard shell prompts and 'my_prompt '. The user defined prompt
definition between the slashes is interpreted as regex, and therefore
regex rules such as escaping
applies. For
example, to set a user defined prompt for gdb the following would be
used:
*/\(gdb\) /
Only one (1) user defined prompt can be active at a time,
however
multiple prompts can be represented using the regex OR '|', for
example, creating a gdb prompt and my_prompt:
*/\(gdb\) |my_prompt /
Another application of the user defined prompt is to make telnet logins
easier:
*/login: |.+ssword: / >telnet remote-host-018 <Connected to >$username >$password >
Clearing a user defined prompt. There may be times where a previously
user defined prompt is causing problems by output which falsely
triggers the user defined prompt. It is possible to clear the user
defined prompt by:
*//
Debug the user defined prompt by using the -vv or --verbose cli
parameter. In the example below, the user defined prompt has been
incorrectly */\(gbb\) / while running gdb (a linux debugger). The debug
output shows the user defined prompt and the string expect-lite is
searching for the prompt.
(gdb) Warning:Prompt Timed Out! DEBUG: User Defined Prompt:<<\(gbb\) >> Not Found in<< (gdb) >>
The scope of the user defined prompt is global, that is, it extends to
the main script as well as all include files referenced.
Setting
the
default shell with
**SHELL=<shell>
expect-lite
has been tested with bash, however it should operate with other shells.
Based on the user's preference, a shell can be selected with the
**SHELL directive. Because the shell is invoked immediately after login
to the remote host, this is a configuration directive with is read
before the script executes. Therefore, it can be placed anywhere in the
script, and is not limited to the first line.
Stopping
the script
early with *TERM, *PASS & *FAIL
There
are times when it is desirable to terminate the execution of a script
early (earlier than the end of the script). For example, while
debugging a script, it is convenient to stop execution mid way in the
script while focusing a troublesome section. Or to perform a test, such
as this is Monday, and terminate the script.
*PASS will stop
execution and return 0, or Pass. This is useful for debugging. *TERM
<N> (as of version 4.8.0) will terminate the script early, and
return N (0..255). User defined return can be useful for signalling to
the calling program that the script returned abnormally (e.g. lab setup
was not correct).
*FAIL
will stop execution and return 1, or Fail. If a fail script has been
declared earlier, it will be executed before terminating execution. If
*NOFAIL has been used prior to *FAIL, then the script will run to
completion, but will declare fail at that time, and return 1.
No stopping the script with
*NOFAIL
There
are times, especially when debugging a script, where it is desirable to
not stop the script due to a failure, but instead allow the script to
run until completion. Setting the directive *NOFAIL will prevent the
script from terminating prematurely. Once invoked, there is no
disabling *NOFAIL.
Upon completion, if there are failures during the script, expect-lite
will return a 1, and indicate that there was a failure.
More no stopping with
*NOINTERACT
When running regression scripts, it is not desirable to have each
include script stop at breakpoints (*INTERACT). Rather than force the
re-editing of each include file, it is possible to ignore the
breakpoints with *NOINTERACT.
This directive is sticky, and can not be turned off once it is invoked.
Debugging
by examining expect-lite variables with *SHOW VARS
It
can be useful when debugging a script to see what the value of
variables are at some point in the script. Additionally, overriding
constants will also be printed. This can be used with *INTERACT, or
pausing the script, however using the debugger command <esc>v is
less typing.
Multiple
Sessions
*FORK
Until version release 3.5.0, expect-lite has been intentionally limited
to a single session keeping it simple. However, there are certain
environments where it might be advantageous for a single script to
control/monitor both a client (e.g wget) and a server (httpd log).
Without multiple session support this type of environment would be
difficult to automate with expect-lite.
What is a new session? A new session starts with a new shell on the remote host. All
commands '>'
and received text '<' are constrained to that session. It is
possible to use multiple sessions on the localhost using r=none,
however r=none
support is limited due to timing issues (the timing issues have
been removed in 4.7). If multiple sessions
on the
localhost (the same host where expect-lite is running), it is better to
ssh to the localhost by specifying r=localhost. This loop back method
ensures that the
timing between commands and received text stay in sync.
With the version of 3.5.0, expect-lite supports nearly limitless
multiple sessions. However, it is a good guideline to use as few
sessions as needed. For backward compatibility, the first session is
the
"default" session. Additional sessions are assigned a name at
invocation with the *FORK <session_name> directive:
*FORK Server INFO: FORK session is: Server
expect-lite will print an "INFO" line stating the current session name
Session names may not contain spaces, and must be unique for each
session. To switch back to a previously started session, re-use the
session name. The session name "default" (without quotes) is reserved
for the first session.
*FORK default INFO: FORK session is: default
Print the current session name by using the *FORK with no session name:
*FORK INFO: FORK session is: Server
*FORK and variables. It is possible to assign a session name to a
variable, and then create a new session with that name. For example:
$my_session=Client *FORK $my_session INFO: FORK session is: Client
All sessions will be automatically closed when the script terminates.
Debugging
in
a multi-session
environment: When in INTERACT, either via the *INTERACT command in the
script or via instant-interact (by pressing ^\), the *FORK command is
still available, permitting the switching of sessions while in INTERACT
mode. For example:
expect-lite directive: *INTERACT
Press '+++' to end interact session & return to script
% *FORK CONSOLE
INFO: FORK session is: CONSOLE, Active sessions are: default CONSOLE #
With
the addition of
multiple sessions, decoding the output will become
more difficult. Remember, although multiple sessions are supported,
expect-lite is still single threaded, meaning commands will always be
executed the order as they appear in the script.
Expect-lite
provides additional information (logging) which makes it
easier to debug failures. As of version 4.4.0 expect-lite supports
native saving output to a file (logging). Using *LOG, *LOGAPPEND, and
*NOLOG, logging can be controlled within a script. On the CLI,
the *LOG commands will override those inside the script.
The following logging commands are supported:
*LOG: with no file name, will automatically create a log
file with the script name + ".log" in the same directory as the script.
If a path/file name is supplied, expect-lite will write standard out
and additional info (INFO, WARN, DEBUG) to the path/file name file.
Because parameters can be entered in any order on the CLI, the
path/file_name must end in ".log" to prevent ambiguity.
*NOLOG: closes any open logging files, and disables any
additional output from be saved to a file. When used on the CLI, all
logging to a file will be disabled.
*LOGAPPEND: Will cause output to be appended to an existing
file, or create a new file if existing file doesn't exist. Output will
be appended to path/file_name, when using *LOGAPPEND
<path/file_name>.
The output is coloured
based on three levels of information desired: info, warnings,
and debug. By default, info and
warnings are
turned on and exp_info is
turned off.
However, using the log information directives: *INFO, *NOINFO, *WARN,
*NOWARN,
*DEBUG, *NODEBUG, these messages can be managed. Additionally,
*NOCOLOR, will turn off color output.
Example of the different information levels are:
INFO: Entering FORK sessions, Constant substitution,
Conditional
Statement Results, Jumping to labels, Including files, Dynamic Variable
Assignments, User Defined Prompt assignments, Sleep progress dots and
##Overal Result.
EXP_INFO: prints "expected" text (lines with < or
<<). Also enabled with -V flag on the command line.
WARN: Instant-Interact feature disabled,
Wait-for-prompt
timeouts, Using localhost (r=none), Unable to parse Conditional,
Dynamic Variable Capture timeouts, Unable to parse math variable
statement, Unable to parse FORK session name, Unable to parse
Conditional Statement
DEBUG: Additional debugging information for: User
Defined
Prompt,
FORK session instantiation/switching,
'<' searching, Dynamic Variable capture
In addition to the Logging Directives, invoking expect-lite with a -v
or -vv will force INFO & WARN on or WARN & DEBUG on,
respectively. It should be possible to direct the amount of logging
desired by using a combination of the command line arguments
(-v
or -vv) and the Logging Directives.
In addition to the three logging levels above, there is another more
serious level of ERROR,
which can not be suppressed, and will terminate
expect-lite with a non-zero return code. Examples of ERRORs are:
unable
to connect or lost connection with remote host, include file not found,
or infinite loop detection.
It may be tempting to turn off logging once a script is debugged.
However, if the script is being run in an automated environment (like
crontab), it is recommended that INFO and WARN be turned on, as the
additional information will make debugging failures much easier.
Controlling End
of Line Sequence with *EOLS
There
are some remote devices which require carriage return + line feed at
the end of each line sent. In past versions of expect-lite, only line
feed was appended to each '>' or '>>' line. This
limitation
made it difficult to automate tasks with this class of remote devices.
With
version 3.7.0, the end of line sequence can be controlled with the
following expect-lite directives: *EOLS LF & *EOLS CRLF. The
default is *EOLS LF.
These directives can be placed anywhere in
the script and the setting remains in effect until another *EOLS
directive is encountered. For example, enabling a script to control a
linux machine (using *EOLS LF), and later in the script controlling a
remote power-switch device (using *EOLS CRLF).
Controlling Dynamic Variable
capture method with *DVPROMPT/*NODVPROMPT
The default Dynamic Variable Capture method requires a trailing prompt
to signal when to look for the text of interest. This is based on the
assumption that a command would be issued, and somewhere in the output
is the text of interest. For example:
$ date Sat Jul 19 18:19:58 EDT 2014 $
A Dynamic Variable capture script might be looking for the month.
>date +$month=\n\w+ (\w+)
With the default capture method, *DVPROMPT, expect-lite will wait for
the trailing prompt (after the date output) before searching the output
for $month. If the text of interest (e.g. month) is not found, then a
warning will be issued, and the script will move on quickly.
There are times when there is no prompt (e.g. it is just the running
output of /var/log/messages), and the default dynamic capture method
will not find the desired text.
*NODVPROMPT turns off the requirement of the trailing prompt for these
instances. The Dynamic Variable capture method will search the output
until the timeout value (set with @, e.g. @5) expires. Caution: this can slow down the script
considerably, if the expect timeout is set to a large value, say 600
(10 minutes), and the text of interest doesn't appear. The script will
just sit and wait for 600 seconds, before issuing a warming and finally
moving on.
Like all directives, *DVPROMPT/*NODVPROMPT can be used at anytime in
the script, on the command line when invoking the script, or even
in the Debugger.
Examples of Expect-lite
Below are some examples to better illustrate what can be accomplished
with expect-lite:
Setting an IP address
Setting the IP address on a secondary interface of the remote host (be
sure to escape the dot's in the IP address)
$local_eth=eth1 $ip_addr=192\.168\.10\.2 # change timeout value to 10 sec @10 >ifconfig $local_eth $ip_addr #check to ensure that we set the right IP address >ifconfig <$ip_addr
Telnet to another host
Starting a telnet to another host
@2 >telnet remote-host-018 <Connected to <login >>root <assword >>secret_password # issue a command once logged in >pwd >^] >quit
ssh to another host
Starting a ssh session to another host
>ssh root@host-021 <assword: >>secret_password # issue a command once logged in >ls >exit
Use
Screen to keep session alive
Use of 'screen'
command
to keep a command
alive after test completes
; ==== Use Screen command for to keep application running ==== >screen > $pmm # Check that HW is alive >ping <Successfully pinged the PM H/W ; ==== Keep pmm application active, and detach from this screen >>^A >>^D <detached >
Assigning
a dynamic variable
Assigning a
dynamic
variable $host using
regex to capture the hostname
from 'uname -a' command
>uname -a +$host=Linux ([a-z\-0-9]+) Assigned Var:host=remote-host-008 >
Assigning multiple dynamic variables using regex to capture environment
variables $HOME and $SHELL, while expecting $TERM=xterm
Assigning
a dynamic variable to a list of items (or multiple lines). The value of
the dynamic variable will be a space delimited list (replacing end of
lines with spaces):
#prompt is $ and space after $ $prompt=\$ >env | sort +$env_list=((.|\n|\r)*)$prompt
Allowing multiple
responses with RegEx
Using regex in expect-lite to expect users OR wheel
>id # allow either groups users or wheel <(users|wheel)
Use
of bash-here-doc to create temp file
Create a
temporary
file with unique
name including script name and date. The predefined variable
$arg0 is contains
the expect-lite script name.
# date stamp of script run time >echo `date +_%Y_%m_%d_%H-%M-%S` +$DATE=\n(_[0-9_-]+)
Force arguments with help
Creating script help. First, require a
parameter be passed via the CLI, such as sw_build:
./upgrade_script.elt sw_build=0015
Inside the script itself, default $sw_build to 0, the CLI constant will
override.
$sw_build=0000 ; === Check passed parameters > *NOINFO ?if $sw_build != 0000 ? %CONTINUE ;; ############################## ;; $arg0 Requires parameters: sw_build, IP ;; e.g. $arg0 sw_build=0015 IP=10.3.5.4 ;; ############################## # quit the script without continuing *FAIL %CONTINUE *INFO
Now when just the script is executed, help will be displayed, and the
script will terminate.
./upgrade_script.elt
=== Check passed parameters
expect-lite directive: *NOINFO ############################## upgrade_script.elt Requires parameters: sw_build, IP e.g. upgrade_script.elt sw_build=0015 IP=10.3.5.4 ##############################
expect-lite directive: *FAIL
User
Defined Help
As
of version 4.3.0, User defined help can make scripts more
user-friendly. Text between triple-semi-colons ';;;' will be displayed
when the -h or --help option is used on the command line
Then running the script, the help text as well as variables in
the script which can be overridden on the command line (as constants).
$ ./test_bash_here_doc.txt -h Help for: ./test_bash_here_doc.txt
Test: Test bash here doc
Assumptions: bash Platforms: anywhere bash runs
Displaying assigned variables, which can be overridden with a Constant on command line e.g var=value tempfile=junk
Sample
Regression Script
It
is possible to use expect-lite to run simple regressions by including
multiple include files. It is also possible to pass var=value
parameters to the include files changing the nature of the test. In the
following example, pseudo-arrays and a while loop is used to create a
quick regression.
# ignore breakpoints in include files *NOINTERACT # do not stop on failures *NOFAIL
######### List of tests to run in regression $blank= $tc0=ipv6_telnet.elt $tc1=ipv6_ndp.elt $tc2=ipv6_self_syslog.elt mode=ipv4 $tc3=ipv6_self_syslog.elt mode=ipv6 $tc4=
Run the regression script and log it using the unix/linux 'tee'
command. By defining 'start' as a constant on the command line, it is
possible to start the regression with the second test.
./regression.elt start=1 *LOG
Test
Failure?!?
When will expect-lite fail a test? It will only fail a test when an
expected result (after issuing a command) does not appear in the
specified timeout period. For example in the following command file:
The first line will be sent (blindly), since there is no expected
return. On line 3, the ifconfig command is sent again, and the script
is looking for a desired result of 192.168.10.99 (using the
var
$ip_addr). If for some reason the ip address is different than ip
address was returned, the script would fail.
Not
Expect
A test can also fail should text appear that is unexpected (such as an
error). This does not clear the expect
input
buffer,
and should be used before a valid expect. How long should the script
wait for the un-expected? In order to reduce delays in script running
time, the Not Expect feature only waits for 100ms. This is usually
enough time to
detect quick error responses such as "file not found".
For example: Fail device doesn't exist
>ls -l /dev/linux -<No such file
In another example: We don't like Mondays
; === fail if today is 'Monday' >date +%A -<Monday
Since the expect input buffer is not consumed, valid expects can still
be performed. in the following example, ttyS2 will still be found.
>ls -1 /dev/ttyS* -<ttyX <ttyS2
It is recommended that Not
Expect linesbe
used
before valid expect lines. There is an exception to this rule, when
looking at output which may be slow (longer than 100ms). For example
there may be a linux machine with several interfaces (eth0 thru eth4).
You may want to ensure that eth4 is not down.
>ip link show <eth4 -<state DOWN
In the example above, the 100ms counter will start after finding 'eth4'
Fuzzy
Expect
Fuzzy expect is expecting an approximate number (either decimal or
hex). There are situations where an approximate value, or a range of
numbers is acceptable. An example, one might be looking at cpu load (it
is OK if it is between 0 and 2, but the test should fail if it over 2).
Fuzzy expect operates in two parts. The first part is to set a range or
tolerance
(of fuzziness). This defines a range, for example entering 7, will
match + or - 7. Set the range by:
; set the fuzzy expect range to 7 ~=7
This value will be in effect until it is changed.
The second part is to use fuzzy expect. In the following example, it is
important to the script to run in the first 2 weeks of the month. Using
the date command to print out the date/day of the month.
; Check the date/day of the month >date "+the day is: %d" ~<the day is: (7)
The value in parens will be checked against the range earlier defined
(plus or minus). In the above example, if the date command returns the
string "the day is : 13", it will be pass, since 13 < 7+7
Of course this could have been done in a more programmatic way, of
capturing the value returned by date (using Dynamic Variables), then
using a conditional (if statement) to do the comparison. But fuzzy
expect is much simpler to use.
Additional important details are: it consumes the expect buffer,
hexidecimal numbers must start with 0x (e.g. 0xbeef), parens in
the string which are not matching the fuzzy value must be escaped with
a backslash (e.g. \(15\) ), regex
evaluation is active (like '<'), and the default range is 10.
How can it help?
Expect-lite can be used to quickly create automated tests. It can
collapse complicated tests into a single command line. It can also be
used in nightly regression testing, performing simple functional
tests providing confidence that core functionality has not been broken
by the previous day's submissions.
Limitations
Expect-lite is limited to what a person could do in one (1) terminal
window (or xterm). It cannot start a program in one window and run a
different program in a separate window. However it does
support multiple sessions via *FORK
which should meet the needs of most.
Standard programming constructs such as looping and branching are
supported as of version 3.0.0. Real Expect can do these things and
more. That said, looping
is limited to a repeat loop and while loop and branching can be
accomplished with
Labels.
Expect-lite has been purposely limited to keep scripts simple, easy
to use and maintain. Although more complex scripts can be created,
basic expect-lite scripts can be created by simply cutting and pasting
text from a terminal window into a script and adding '>'
'<'
characters.
Implementation Details
Variables
Variable names must be restricted to the following set of characters
[A-Za-z0-9_]. Variable values may include spaces, and quoting is not
required nor permitted. For example:
$bell=For whom the bell tolls >echo $bell For whom the bell tolls
Implied
"wait
for prompt"
Using the '>' character implies waiting for the prompt before
sending the command. The definition of a shell prompt a: $, #, % or
>. It is often useful to follow a '<' with a
'>' for force the script to wait for the prompt:
> $some_long_command <critical value # wait for prompt >
Prompt detection can be problematic, and by no means is expect-lite's
method perfect. Expect-lite looks for a prompt character
($, #, %
or
>) followed
by a space
at the end of
line.
This should work for most standard prompts, however coloured or fancy
prompts may not be detected. As of
version 3.1.5, user
defined prompt
is supported.
It may be useful to create files on the fly which will be used in the
test. Since expect-lite does not read/write files directly, this
function can be done with the linux command 'cat' and the bash here doc
method. Creating a
regex
file for example:
; ==== Create Test File on the fly > cat > $regex_test_file << '+++' >T00001/a(?$XL.*)/tag=0x08000001 >T00002/b(?$XR.*)/tag=0x08000002 >T00003/c(?$XO[0-7]*)/tag=0x08000003 >T00004/d(?$XH[0-9A-F]*)/tag=0x08000004 >T00005/e(?$XD[0-9]*)/tag=0x08000005 >+++ >
In the above example the script initiates a cat to a file (referenced
by $regex_test_file ). Normally the '>' would be used to send
lines
to the remote host. However, there is an inherent "wait for prompt",
which will politely wait for the next line. The last '>' is
there to
wait for the prompt to return
after the cat command.
Predefined
variables
Expect-lite has minimized the use of predefined variables to allow the
user the widest latitude in selecting variable names. However a few
predefined variables are useful and listed below:
Predefined
variable
Usage
$arg0
expect-lite
script name.
Set to the c= or cmd_file= value. Useful for creating temporary files
unique to the test script.
$timeout_multiplier
Typically
used as a
constant (passed in from the command line) which the integer value is
multiplied by each of the timeouts in the script. i.e. @10 x
$timeout_multiplier = real timeout. The value default is 1.
Timeouts
Setting the expect-lite timeout to 0 isn't actually zero as this will
cause the keyboard input buffer on the remote machine to overflow (and
lose characters). Expect-lite adds a 50ms delay between lines when the
@0 is used to prevent the overflow.
Expect
Input
Buffer
Expect by default places strings that are sent back (from the remote
host) into an expect buffer ($expect_out(buffer)). Expect-lite
leverages this built-in function, and uses it to perform multiple
searches through the buffer when using the '<' , '+<' or
'-<'
actions. The actions '>' and '>>' will clear
expect buffer to
prevent spurious matches.
The default size of the expect_out(buffer) is 10000 bytes. This has
been selected as a compromise between performance (larger buffer slows
down searches), and large enough for most users. If searches (using
'<' , '+<' or
'-<') or dynamic variable captures are being truncated, it may be
required to increase the size of the expect_out(buffer) size. Caution: This should only be a last
resort (as it will impact performance). Place the following near the
top of the script.
; === set expect buffer big !match_max -d 20000
Timing
is everything:
delay_wait_for_host
When communicating with a remote host, enough time must be provided for
the remote host to respond. There is an internal global value of
$delay_wait_for_host (in mili-seconds) which can be tuned to your
environment using the environment variable EL_DELAY_WAIT_FOR_HOST (see
sample bashrc). This value is also used by the Expect Not feature to
determine how long Expect-Not should wait for the unexpected.
Unsynchronized
messages
disclaimer
Effort has been put into making the output as synchronous as possible.
However because what is seen on stdout or screen is data
returned/echoed from the remote host and printable comments are
directly printed to stdout, it is possible to have a printable comment
appear in the stdout before the action it references is returned from
the remote host. This lack of synchronization can sometimes be
minimized by adding an additional '>' after the printable
comment as
shown in the following:
; === starting long command > >start_long_command
Instant-Interact:
More than just
a pretty prompt
When the user presses ^\ (control+backslash) during the running of a
script, expect-lite will drop into interactive mode aka the integrated
debugger. In version 4.0.1, the debugger has been greatly enhanced and
performs three primary functions: 1) connecting the user to the remote
host or device under test, 2) monitoring special commands prefaced with
the escape key for stepping, and other functions, and 3) the debugger
will allow expect-lite script lines to be executed by either typing
directly or pasting them into the debugger. The TCL package TclX is no
longer required.
Control will be returned to the script when '+++' is typed. In addition
to returning to the script, the current command in the script will be
"Passed", preventing it from failing the script. This is useful when
debugging a script, the user notices that the response is not as
expected, and presses ^\ to move to interact mode. Returning from
interact mode, the line which would have failed the script (because of
the response not being as expected) will be "Passed" and the script
will continue. This Passing action is line oriented, and other
'<'
lines in the script can still fail the script.
In the IDE (debugger), expect-lite
script commands can be typed, even assigning variables or displaying
them inside the
paused script, for example:
$MYVAR=today
*SHOW VARS Var:0 Value:0 Var:MYVAR Value:today Var:TEST Value:/proc/cpuinfo
>>pwd pwd /home/user
Additionally, expect-lite lines (code fragments) can be copied from
your favourite editor and pasted directly into the debugger. All
variables will be automatically dereferenced (assuming they were set in
the first place). This is very useful for testing Dynamic Variable
capture lines, which can be tricky to define.
When
lines are pasted into the debugger, or the Step or sKip function are
used, the debugger clumps lines which begin with '>' and any
following lines with start with '<' or '+$' This enables expected
results and dynamic variables to be interpretted correctly. For example:
If
the above lines are copied and pasted into the debugger, in fact they
will be executed as 2 clumps. Usually this is of no concern, however
when stepping/skipping through an expect-lite script, one will notice
that clumps are executed rather than indivdual lines.
A new *FORK session actually creates an additional login(s) to the
remote host from the localhost using the same connect method as the
first (or default) session. It then sets a sane prompt, and waits for
the first script command in that session. This provides multiple
sessions on the remote host, which then could be used to drive separate
programs (see test_fork_web_client_server.txt for example). Graphically
it would appear as:
In order to maintain an easy-to-learn, simple-to-use expect like
language, looping has been intentionally kept simple and intuitive.
Most expect-lite scripts will be to be top down, single pass scripts,
making the scripts easy to create and read. These limitations preserve
the KISS principle.
If more complexity is required, consider using embedded expect inside
expect-lite, or writing a native expect script.
Why Expect-lite
Expect-lite was written so you don't have to learn expect. It provides
a quick way to write
simple expect scripts using just the > and < characters.