Core pexpect components¶

Pexpect is a Python module for spawning child applications and controlling them automatically. Pexpect can be used for automating interactive applications such as ssh, ftp, passwd, telnet, etc. It can be used to a automate setup scripts for duplicating software package installations on different servers. It can be used for automated software testing. Pexpect is in the spirit of Don Libes’ Expect, but Pexpect is pure Python. Other Expect-like modules for Python require TCL and Expect or require C extensions to be compiled. Pexpect does not use C, Expect, or TCL extensions. It should work on any platform that supports the standard Python pty module. The Pexpect interface focuses on ease of use so that simple tasks are easy.

There are two main interfaces to the Pexpect system; these are the function, run() and the class, spawn. The spawn class is more powerful. The run() function is simpler than spawn, and is good for quickly calling program. When you call the run() function it executes a given program and then returns the output. This is a handy replacement for os.system().

For example:

pexpect.run('ls -la')

The spawn class is the more powerful interface to the Pexpect system. You can use this to spawn a child program then interact with it by sending input and expecting responses (waiting for patterns in the child’s output).

For example:

child = pexpect.spawn('scp foo user@example.com:.')
child.expect('Password:')
child.sendline(mypassword)

This works even for commands that ask for passwords or other input outside of the normal stdio streams. For example, ssh reads input directly from the TTY device which bypasses stdin.

Credits: Noah Spurrier, Richard Holden, Marco Molteni, Kimberley Burchett, Robert Stone, Hartmut Goebel, Chad Schroeder, Erick Tryzelaar, Dave Kirby, Ids vander Molen, George Todd, Noel Taylor, Nicolas D. Cesar, Alexander Gattin, Jacques-Etienne Baudoux, Geoffrey Marshall, Francisco Lourenco, Glen Mabey, Karthik Gurusamy, Fernando Perez, Corey Minyard, Jon Cohen, Guillaume Chazarain, Andrew Ryan, Nick Craig-Wood, Andrew Stone, Jorgen Grahn, John Spiegel, Jan Grant, and Shane Kerr. Let me know if I forgot anyone.

Pexpect is free, open source, and all that good stuff. http://pexpect.sourceforge.net/

PEXPECT LICENSE

This license is approved by the OSI and FSF as GPL-compatible.

http://opensource.org/licenses/isc-license.txt

Copyright (c) 2012, Noah Spurrier <noah@noah.org> PERMISSION TO USE, COPY, MODIFY, AND/OR DISTRIBUTE THIS SOFTWARE FOR ANY PURPOSE WITH OR WITHOUT FEE IS HEREBY GRANTED, PROVIDED THAT THE ABOVE COPYRIGHT NOTICE AND THIS PERMISSION NOTICE APPEAR IN ALL COPIES. THE SOFTWARE IS PROVIDED “AS IS” AND THE AUTHOR DISCLAIMS ALL WARRANTIES WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.

spawn class¶

class pexpect.spawn(command, args=, []timeout=30, maxread=2000, searchwindowsize=None, logfile=None, cwd=None, env=None, ignore_sighup=True, echo=True)[source]¶

This is the main class interface for Pexpect. Use this class to start and control child applications.

__init__(command, args=, []timeout=30, maxread=2000, searchwindowsize=None, logfile=None, cwd=None, env=None, ignore_sighup=True, echo=True)[source]¶

This is the constructor. The command parameter may be a string that includes a command and any arguments to the command. For example:

child = pexpect.spawn('/usr/bin/ftp')
child = pexpect.spawn('/usr/bin/ssh user@example.com')
child = pexpect.spawn('ls -latr /tmp')

You may also construct it with a list of arguments like so:

child = pexpect.spawn('/usr/bin/ftp', [])
child = pexpect.spawn('/usr/bin/ssh', ['user@example.com'])
child = pexpect.spawn('ls', ['-latr', '/tmp'])

After this the child application will be created and will be ready to talk to. For normal use, see expect() and send() and sendline().

Remember that Pexpect does NOT interpret shell meta characters such as redirect, pipe, or wild cards (>, |, or *). This is a common mistake. If you want to run a command and pipe it through another command then you must also start a shell. For example:

child = pexpect.spawn('/bin/bash -c "ls -l | grep LOG > logs.txt"')
child.expect(pexpect.EOF)

The second form of spawn (where you pass a list of arguments) is useful in situations where you wish to spawn a command and pass it its own argument list. This can make syntax more clear. For example, the following is equivalent to the previous example:

shell_cmd = 'ls -l | grep LOG > logs.txt'
child = pexpect.spawn('/bin/bash', ['-c', shell_cmd])
child.expect(pexpect.EOF)

The maxread attribute sets the read buffer size. This is maximum number of bytes that Pexpect will try to read from a TTY at one time. Setting the maxread size to 1 will turn off buffering. Setting the maxread value higher may help performance in cases where large amounts of output are read back from the child. This feature is useful in conjunction with searchwindowsize.

The searchwindowsize attribute sets the how far back in the incoming seach buffer Pexpect will search for pattern matches. Every time Pexpect reads some data from the child it will append the data to the incoming buffer. The default is to search from the beginning of the incoming buffer each time new data is read from the child. But this is very inefficient if you are running a command that generates a large amount of data where you want to match. The searchwindowsize does not affect the size of the incoming data buffer. You will still have access to the full buffer after expect() returns.

The logfile member turns on or off logging. All input and output will be copied to the given file object. Set logfile to None to stop logging. This is the default. Set logfile to sys.stdout to echo everything to standard output. The logfile is flushed after each write.

Example log input and output to a file:

child = pexpect.spawn('some_command')
fout = file('mylog.txt','w')
child.logfile = fout

Example log to stdout:

child = pexpect.spawn('some_command')
child.logfile = sys.stdout

The logfile_read and logfile_send members can be used to separately log the input from the child and output sent to the child. Sometimes you don’t want to see everything you write to the child. You only want to log what the child sends back. For example:

child = pexpect.spawn('some_command')
child.logfile_read = sys.stdout

To separately log output sent to the child use logfile_send:

self.logfile_send = fout

If ignore_sighup is True, the child process will ignore SIGHUP signals. For now, the default is True, to preserve the behaviour of earlier versions of Pexpect, but you should pass this explicitly if you want to rely on it.

The delaybeforesend helps overcome a weird behavior that many users were experiencing. The typical problem was that a user would expect() a “Password:” prompt and then immediately call sendline() to send the password. The user would then see that their password was echoed back to them. Passwords don’t normally echo. The problem is caused by the fact that most applications print out the “Password” prompt and then turn off stdin echo, but if you send your password before the application turned off echo, then you get your password echoed. Normally this wouldn’t be a problem when interacting with a human at a real keyboard. If you introduce a slight delay just before writing then this seems to clear up the problem. This was such a common problem for many users that I decided that the default pexpect behavior should be to sleep just before writing to the child application. 1/20th of a second (50 ms) seems to be enough to clear up the problem. You can set delaybeforesend to 0 to return to the old behavior. Most Linux machines don’t like this to be below 0.03. I don’t know why.

Note that spawn is clever about finding commands on your path. It uses the same logic that “which” uses to find executables.

If you wish to get the exit status of the child you must call the close() method. The exit or signal status of the child will be stored in self.exitstatus or self.signalstatus. If the child exited normally then exitstatus will store the exit return code and signalstatus will be None. If the child was terminated abnormally with a signal then signalstatus will store the signal value and exitstatus will be None. If you need more detail you can also read the self.status member which stores the status returned by os.waitpid. You can interpret this using os.WIFEXITED/os.WEXITSTATUS or os.WIFSIGNALED/os.TERMSIG.

The echo attribute may be set to False to disable echoing of input. As a pseudo-terminal, all input echoed by the “keyboard” (send() or sendline()) will be repeated to output. For many cases, it is not desirable to have echo enabled, and it may be later disabled using setecho(False) followed by waitnoecho(). However, for some platforms such as Solaris, this is not possible, and should be disabled immediately on spawn.

expect(pattern, timeout=-1, searchwindowsize=-1)[source]¶

This seeks through the stream until a pattern is matched. The pattern is overloaded and may take several types. The pattern can be a StringType, EOF, a compiled re, or a list of any of those types. Strings will be compiled to re types. This returns the index into the pattern list. If the pattern was not a list this returns index 0 on a successful match. This may raise exceptions for EOF or TIMEOUT. To avoid the EOF or TIMEOUT exceptions add EOF or TIMEOUT to the pattern list. That will cause expect to match an EOF or TIMEOUT condition instead of raising an exception.

If you pass a list of patterns and more than one matches, the first match in the stream is chosen. If more than one pattern matches at that point, the leftmost in the pattern list is chosen. For example:

# the input is 'foobar'
index = p.expect(['bar', 'foo', 'foobar'])
# returns 1('foo') even though 'foobar' is a "better" match

Please note, however, that buffering can affect this behavior, since input arrives in unpredictable chunks. For example:

# the input is 'foobar'
index = p.expect(['foobar', 'foo'])
# returns 0('foobar') if all input is available at once,
# but returs 1('foo') if parts of the final 'bar' arrive late

After a match is found the instance attributes ‘before’, ‘after’ and ‘match’ will be set. You can see all the data read before the match in ‘before’. You can see the data that was matched in ‘after’. The re.MatchObject used in the re match will be in ‘match’. If an error occurred then ‘before’ will be set to all the data read so far and ‘after’ and ‘match’ will be None.

If timeout is -1 then timeout will be set to the self.timeout value.

A list entry may be EOF or TIMEOUT instead of a string. This will catch these exceptions and return the index of the list entry instead of raising the exception. The attribute ‘after’ will be set to the exception type. The attribute ‘match’ will be None. This allows you to write code like this:

index = p.expect(['good', 'bad', pexpect.EOF, pexpect.TIMEOUT])
if index == 0:
    do_something()
elif index == 1:
    do_something_else()
elif index == 2:
    do_some_other_thing()
elif index == 3:
    do_something_completely_different()

instead of code like this:

try:
    index = p.expect(['good', 'bad'])
    if index == 0:
        do_something()
    elif index == 1:
        do_something_else()
except EOF:
    do_some_other_thing()
except TIMEOUT:
    do_something_completely_different()

These two forms are equivalent. It all depends on what you want. You can also just expect the EOF if you are waiting for all output of a child to finish. For example:

p = pexpect.spawn('/bin/ls')
p.expect(pexpect.EOF)
print p.before

If you are trying to optimize for speed then see expect_list().

expect_exact(pattern_list, timeout=-1, searchwindowsize=-1)[source]¶

This is similar to expect(), but uses plain string matching instead of compiled regular expressions in ‘pattern_list’. The ‘pattern_list’ may be a string; a list or other sequence of strings; or TIMEOUT and EOF.

This call might be faster than expect() for two reasons: string searching is faster than RE matching and it is possible to limit the search to just the end of the input buffer.

This method is also useful when you don’t want to have to worry about escaping regular expression characters that you want to match.

expect_list(pattern_list, timeout=-1, searchwindowsize=-1)[source]¶: This takes a list of compiled regular expressions and returns the index into the pattern_list that matched the child output. The list may also contain EOF or TIMEOUT(which are not compiled regular expressions). This method is similar to the expect() method except that expect_list() does not recompile the pattern list on every call. This may help if you are trying to optimize for speed, otherwise just use the expect() method. This is called by expect(). If timeout==-1 then the self.timeout value is used. If searchwindowsize==-1 then the self.searchwindowsize value is used.

compile_pattern_list(patterns)[source]¶

This compiles a pattern-string or a list of pattern-strings. Patterns must be a StringType, EOF, TIMEOUT, SRE_Pattern, or a list of those. Patterns may also be None which results in an empty list (you might do this if waiting for an EOF or TIMEOUT condition without expecting any pattern).

This is used by expect() when calling expect_list(). Thus expect() is nothing more than:

cpl = self.compile_pattern_list(pl)
return self.expect_list(cpl, timeout)

If you are using expect() within a loop it may be more efficient to compile the patterns first and then call expect_list(). This avoid calls in a loop to compile_pattern_list():

cpl = self.compile_pattern_list(my_pattern)
while some_condition:
   ...
   i = self.expect_list(clp, timeout)
   ...

send(s)[source]¶: Sends string s to the child process, returning the number of bytes written. If a logfile is specified, a copy is written to that log.

sendline(s='')[source]¶: Wraps send(), sending string s to child process, with os.linesep automatically appended. Returns number of bytes written.

write(s)[source]¶: This is similar to send() except that there is no return value.

writelines(sequence)[source]¶: This calls write() for each element in the sequence. The sequence can be any iterable object producing strings, typically a list of strings. This does not add line separators. There is no return value.

sendcontrol(char)[source]¶

Helper method that wraps send() with mnemonic access for sending control character to the child (such as Ctrl-C or Ctrl-D). For example, to send Ctrl-G (ASCII 7, bell, ‘’):

child.sendcontrol('g')

Controlling the child process¶

class pexpect.spawn[source]

kill(sig)[source]¶: This sends the given signal to the child application. In keeping with UNIX tradition it has a misleading name. It does not necessarily kill the child unless you send the right signal.

terminate(force=False)[source]¶: This forces a child process to terminate. It starts nicely with SIGHUP and SIGINT. If “force” is True then moves onto SIGKILL. This returns True if the child was terminated. This returns False if the child could not be terminated.

isalive()[source]¶: This tests if the child process is running or not. This is non-blocking. If the child was terminated then this will read the exitstatus or signalstatus of the child. This returns True if the child process appears to be running or False if not. It can take literally SECONDS for Solaris to return the right status.

wait()[source]¶: This waits until the child exits. This is a blocking call. This will not read any data from the child, so this will block forever if the child has unread output and has terminated. In other words, the child may have printed output then called exit(), but, the child is technically still alive until its output is read by the parent.

close(force=True)[source]¶: This closes the connection with the child application. Note that calling close() more than once is valid. This emulates standard Python behavior with files. Set force to True if you want to make sure that the child is terminated (SIGKILL is sent if the child ignores SIGHUP and SIGINT).

getwinsize()[source]¶: This returns the terminal window size of the child tty. The return value is a tuple of (rows, cols).

setwinsize(rows, cols)[source]¶: This sets the terminal window size of the child tty. This will cause a SIGWINCH signal to be sent to the child. This does not change the physical window size. It changes the size reported to TTY-aware applications like vi or curses – applications that respond to the SIGWINCH signal.

getecho()[source]¶

This returns the terminal echo mode. This returns True if echo is on or False if echo is off. Child applications that are expecting you to enter a password often set ECHO False. See waitnoecho().

Not supported on platforms where isatty() returns False.

setecho(state)[source]¶

This sets the terminal echo mode on or off. Note that anything the child sent before the echo will be lost, so you should be sure that your input buffer is empty before you call setecho(). For example, the following will work as expected:

p = pexpect.spawn('cat') # Echo is on by default.
p.sendline('1234') # We expect see this twice from the child...
p.expect(['1234']) # ... once from the tty echo...
p.expect(['1234']) # ... and again from cat itself.
p.setecho(False) # Turn off tty echo
p.sendline('abcd') # We will set this only once (echoed by cat).
p.sendline('wxyz') # We will set this only once (echoed by cat)
p.expect(['abcd'])
p.expect(['wxyz'])

The following WILL NOT WORK because the lines sent before the setecho will be lost:

p = pexpect.spawn('cat')
p.sendline('1234')
p.setecho(False) # Turn off tty echo
p.sendline('abcd') # We will set this only once (echoed by cat).
p.sendline('wxyz') # We will set this only once (echoed by cat)
p.expect(['1234'])
p.expect(['1234'])
p.expect(['abcd'])
p.expect(['wxyz'])

Not supported on platforms where isatty() returns False.

waitnoecho(timeout=-1)[source]¶

This waits until the terminal ECHO flag is set False. This returns True if the echo mode is off. This returns False if the ECHO flag was not set False before the timeout. This can be used to detect when the child is waiting for a password. Usually a child application will turn off echo mode when it is waiting for the user to enter a password. For example, instead of expecting the “password:” prompt you can wait for the child to set ECHO off:

p = pexpect.spawn('ssh user@example.com')
p.waitnoecho()
p.sendline(mypassword)

If timeout==-1 then this method will use the value in self.timeout. If timeout==None then this method to block until ECHO flag is False.

pid¶: The process ID of the child process.

child_fd¶: The file descriptor used to communicate with the child process.

Handling unicode¶

For backwards compatibility, spawn can handle some Unicode: its send methods will encode arbitrary unicode as UTF-8 before sending it to the child process, and its expect methods can accept ascii-only unicode strings. However, for a proper unicode API to a subprocess, use this subclass:

class pexpect.spawnu(*args, **kwargs)[source]¶

Bases: pexpect.spawn

Works like spawn, but accepts and returns unicode strings.

Extra parameters:

Parameters:	encoding – The encoding to use for communications (default: ‘utf-8’) errors – How to handle encoding/decoding errors; one of ‘strict’ (the default), ‘ignore’, or ‘replace’, as described for `decode()` and `encode()`.

There is also a runu() function, the unicode counterpart to run().

Note

Unicode handling with pexpect works the same way on Python 2 and 3, despite the difference in names. I.e.:

spawn works with str on Python 2, and bytes on Python 3,
spawnu works with unicode on Python 2, and str on Python 3.

run function¶

pexpect.run(command, timeout=-1, withexitstatus=False, events=None, extra_args=None, logfile=None, cwd=None, env=None)[source]¶

This function runs the given command; waits for it to finish; then returns all output as a string. STDERR is included in output. If the full path to the command is not given then the path is searched.

Note that lines are terminated by CR/LF (rn) combination even on UNIX-like systems because this is the standard for pseudottys. If you set ‘withexitstatus’ to true, then run will return a tuple of (command_output, exitstatus). If ‘withexitstatus’ is false then this returns just command_output.

The run() function can often be used instead of creating a spawn instance. For example, the following code uses spawn:

from pexpect import *
child = spawn('scp foo user@example.com:.')
child.expect('(?i)password')
child.sendline(mypassword)

The previous code can be replace with the following:

from pexpect import *
run('scp foo user@example.com:.', events={'(?i)password': mypassword})

Examples

Start the apache daemon on the local machine:

from pexpect import *
run("/usr/local/apache/bin/apachectl start")

Check in a file using SVN:

from pexpect import *
run("svn ci -m 'automatic commit' my_file.py")

Run a command and capture exit status:

from pexpect import *
(command_output, exitstatus) = run('ls -l /bin', withexitstatus=1)

The following will run SSH and execute ‘ls -l’ on the remote machine. The password ‘secret’ will be sent if the ‘(?i)password’ pattern is ever seen:

run("ssh username@machine.example.com 'ls -l'",
    events={'(?i)password':'secret\n'})

This will start mencoder to rip a video from DVD. This will also display progress ticks every 5 seconds as it runs. For example:

from pexpect import *
def print_ticks(d):
    print d['event_count'],
run("mencoder dvd://1 -o video.avi -oac copy -ovc copy",
    events={TIMEOUT:print_ticks}, timeout=5)

The ‘events’ argument should be a dictionary of patterns and responses. Whenever one of the patterns is seen in the command out run() will send the associated response string. Note that you should put newlines in your string if Enter is necessary. The responses may also contain callback functions. Any callback is function that takes a dictionary as an argument. The dictionary contains all the locals from the run() function, so you can access the child spawn object or any other variable defined in run() (event_count, child, and extra_args are the most useful). A callback may return True to stop the current run process otherwise run() continues until the next event. A callback may also return a string which will be sent to the child. ‘extra_args’ is not used by directly run(). It provides a way to pass data to a callback function through run() through the locals dictionary passed to a callback.

pexpect.runu(command, timeout=-1, withexitstatus=False, events=None, extra_args=None, logfile=None, cwd=None, env=None, **kwargs)[source]¶

This offers the same interface as run(), but using unicode.

Like spawnu, you can pass encoding and errors parameters, which will be used for both input and output.

Exceptions¶

class pexpect.EOF(value)[source]¶: Raised when EOF is read from a child. This usually means the child has exited.

class pexpect.TIMEOUT(value)[source]¶: Raised when a read time exceeds the timeout.

class pexpect.ExceptionPexpect(value)[source]¶: Base class for all exceptions raised by this module.

Utility functions¶

pexpect.which(filename)[source]¶: This takes a given filename; tries to find it in the environment path; then checks if it is executable. This returns the full path to the filename if found and executable. Otherwise this returns None.

pexpect.split_command_line(command_line)[source]¶: This splits a command line into a list of arguments. It splits arguments on spaces, but handles embedded quotes, doublequotes, and escaped characters. It’s impossible to do this with a regular expression, so I wrote a little state machine to parse the command line.