POE::Wheel::Run - portably run blocking code and programs in subprocesses
#!/usr/bin/perl
use warnings;
use strict;
use POE qw( Wheel::Run );
POE::Session->create(
inline_states => {
_start => \&on_start,
got_child_stdout => \&on_child_stdout,
got_child_stderr => \&on_child_stderr,
got_child_close => \&on_child_close,
got_child_signal => \&on_child_signal,
}
);
POE::Kernel->run();
exit 0;
sub on_start {
my $child = POE::Wheel::Run->new(
Program => [ "/bin/ls", "-1", "/" ],
StdoutEvent => "got_child_stdout",
StderrEvent => "got_child_stderr",
CloseEvent => "got_child_close",
);
$_[KERNEL]->sig_child($child->PID, "got_child_signal");
# Wheel events include the wheel's ID.
$_[HEAP]{children_by_wid}{$child->ID} = $child;
# Signal events include the process ID.
$_[HEAP]{children_by_pid}{$child->PID} = $child;
print(
"Child pid ", $child->PID,
" started as wheel ", $child->ID, ".\n"
);
}
# Wheel event, including the wheel's ID.
sub on_child_stdout {
my ($stdout_line, $wheel_id) = @_[ARG0, ARG1];
my $child = $_[HEAP]{children_by_wid}{$wheel_id};
print "pid ", $child->PID, " STDOUT: $stdout_line\n";
}
# Wheel event, including the wheel's ID.
sub on_child_stderr {
my ($stderr_line, $wheel_id) = @_[ARG0, ARG1];
my $child = $_[HEAP]{children_by_wid}{$wheel_id};
print "pid ", $child->PID, " STDERR: $stderr_line\n";
}
# Wheel event, including the wheel's ID.
sub on_child_close {
my $wheel_id = $_[ARG0];
my $child = delete $_[HEAP]{children_by_wid}{$wheel_id};
# May have been reaped by on_child_signal().
unless (defined $child) {
print "wid $wheel_id closed all pipes.\n";
return;
}
print "pid ", $child->PID, " closed all pipes.\n";
delete $_[HEAP]{children_by_pid}{$child->PID};
}
sub on_child_signal {
print "pid $_[ARG1] exited with status $_[ARG2].\n";
my $child = delete $_[HEAP]{children_by_pid}{$_[ARG1]};
# May have been reaped by on_child_close().
return unless defined $child;
delete $_[HEAP]{children_by_wid}{$child->ID};
}
POE::Wheel::Run executes a program or block of code in a subprocess,
created the usual way: using fork(). The parent process may exchange
information with the child over the child's STDIN, STDOUT and STDERR
filehandles.
In the parent process, the POE::Wheel::Run object represents the child
process. It has methods such as PID() and kill() to query and manage
the child process.
POE::Wheel::Run's put() method sends data to the child's STDIN. Child
output on STDOUT and STDERR may be dispatched as events within the
parent, if requested.
POE::Wheel::Run can also notify the parent when the child has closed
its output filehandles. Some programs remain active, but they close
their output filehandles to indicate they are done writing.
A more reliable way to detect child exit is to use POE::Kernel's
sig_child() method to wait for the wheel's process to be reaped. It
is in fact vital to use sig_child() in all circumstances since without
it, POE will not try to reap child processes.
Failing to use sig_child() has in the past led to wedged machines.
Long-running programs have leaked processes, eventually consuming all
available slots in the process table and requiring reboots.
Because process leaks are so severe, POE::Kernel will check for this
condition on exit and display a notice if it finds that processes are
leaking. Developers should heed these warnings.
POE::Wheel::Run communicates with the child process in a line-based
fashion by default. Programs may override this by specifying some
other POE::Filter object in StdinFilter, StdoutFilter,
StdioFilter and/or StderrFilter.
POE::Wheel subclasses tend to perform a lot of setup so that they run
lighter and faster. POE::Wheel::Run's constructor is no exception.
new() creates and returns a new POE::Wheel::Run object. If it's
successful, the object will represent a child process with certain
specified qualities. It also provides an OO- and event-based
interface for asynchronously interacting with the process.
Conduit specifies the inter-process communications mechanism that will
be used to pass data between the parent and child process. Conduit
may be one of ``pipe'', ``socketpair'', ``inet'', ``pty'', or ``pty-pipe''.
POE::Wheel::Run will use the most appropriate Conduit for the run-time
(not the compile-time) operating system, but this varies from one OS
to the next.
Internally, POE::Wheel::Run passes the Conduit type to
the POE::Pipe::OneWay manpage and the POE::Pipe::TwoWay manpage. These helper classes
were created to make IPC portable and reusable. They do not require
the rest of POE.
Three Conduit types use pipes or pipelike inter-process communication:
``pipe'', ``socketpair'' and ``inet''. They determine whether the internal
IPC uses pipe(), socketpair() or Internet sockets. These Conduit
values are passed through to the POE::Pipe::OneWay manpage or
the POE::Pipe::TwoWay manpage internally.
The ``pty'' conduit type runs the child process under a pseudo-tty,
which is created by the IO::Pty manpage. Pseudo-ttys (ptys) convince child
processes that they are interacting with terminals rather than pipes.
This may be used to trick programs like ssh into believing it's secure
to prompt for a password, although passphraseless identities might be
better for that.
The ``pty'' conduit cannot separate STDERR from STDOUT, but the
``pty-pipe'' mode can.
The ``pty-pipe'' conduit uses a pty for STDIN and STDOUT and a one-way
pipe for STDERR. The additional pipe keeps STDERR output separate
from STDOUT.
The the IO::Pty manpage module is only loaded if ``pty'' or ``pty-pipe'' is used.
It's not a dependency until it's actually needed.
Winsize sets the child process' terminal size. Its value should be an
arrayref with four elements. The first two elements must be the
number of lines and columns for the child's terminal window,
respectively. The second pair of elements describe the terminal's X
and Y dimensions in pixels. If the last pair is missing, they will be calculated
from the lines and columns using a 9x16 cell size.
$_[HEAP]{child} = POE::Wheel::Run->new(
# ... among other things ...
Winsize => [ 25, 80, 720, 400 ],
);
Winsize is only valid for conduits that use pseudo-ttys: ``pty'' and
``pty-pipe''. Other conduits don't simulate terminals, so they don't
have window sizes.
Winsize defaults to the parent process' window size, assuming the
parent process has a terminal to query.
CloseOnCall, when true, turns on close-on-exec emulation for
subprocesses that don't actually call exec(). These would be
instances when the child is running a block of code rather than
executing an external program. For example:
$_[HEAP]{child} = POE::Wheel::Run->new(
# ... among other things ...
CloseOnCall => 1,
Program => \&some_function,
);
CloseOnCall is off (0) by default.
CloseOnCall works by closing all file descriptors greater than $^F in
the child process before calling the application's code. For more
details, please the discussion of $^F in perlvar.
StdioDriver specifies a single the POE::Driver manpage object to be used for
both STDIN and STDOUT. It's equivalent to setting StdinDriver and
StdoutDriver to the same the POE::Driver manpage object.
POE::Wheel::Run will create and use a the POE::Driver::SysRW manpage driver of
one isn't specified. This is by far the most common use case, so it's
the default.
StdinDriver sets the the POE::Driver manpage used to write to the child
process' STDIN IPC conduit. It is almost never needed. Omitting it
will allow POE::Wheel::Run to use an internally created
the POE::Driver::SysRW manpage object.
StdoutDriver sets the the POE::Driver manpage object that will be used to
read from the child process' STDOUT conduit. It's almost never
needed. If omitted, POE::Wheel::Run will internally create and use
a the POE::Driver::SysRW manpage object.
StderrDriver sets the driver that will be used to read from the
child process' STDERR conduit. As with StdoutDriver, it's almost
always preferable to let POE::Wheel::Run instantiate its own driver.
CloseEvent contains the name of an event that the wheel will emit when
the child process closes its last open output handle. This is a
consistent notification that the child is done sending output. Please
note that it does not signal when the child process has exited.
Programs should use sig_child() to detect that.
While it is impossible for ErrorEvent or StdoutEvent to happen after
CloseEvent, there is no such guarantee for CHLD, which may happen before
or after CloseEvent.
In addition to the usual POE parameters, each CloseEvent comes with
one of its own:
ARG0 contains the wheel's unique ID. This can be used to keep
several child processes separate when they're managed by the same
session.
A sample close event handler:
sub close_state {
my ($heap, $wheel_id) = @_[HEAP, ARG0];
my $child = delete $heap->{child}->{$wheel_id};
print "Child ", $child->PID, " has finished.\n";
}
ErrorEvent contains the name of an event to emit if something fails.
It is optional; if omitted, the wheel will not notify its session if
any errors occur. However, POE::Wheel::Run->new() will still throw an
exception if it fails.
ARG0 contains the name of the operation that failed. It may be
'read', 'write', 'fork', 'exec' or the name of some other function or
task. The actual values aren't yet defined. They will probably not
correspond so neatly to Perl builtin function names.
ARG1 and ARG2 hold numeric and string values for $!,
respectively. "$!" will eq "" for read error 0 (child process
closed the file handle).
ARG3 contains the wheel's unique ID.
ARG4 contains the name of the child filehandle that has the error.
It may be ``STDIN'', ``STDOUT'', or ``STDERR''. The sense of ARG0 will
be the opposite of what you might normally expect for these handles.
For example, POE::Wheel::Run will report a ``read'' error on ``STDOUT''
because it tried to read data from the child's STDOUT handle.
A sample error event handler:
sub error_state {
my ($operation, $errnum, $errstr, $wheel_id) = @_[ARG0..ARG3];
$errstr = "remote end closed" if $operation eq "read" and !$errnum;
warn "Wheel $wheel_id generated $operation error $errnum: $errstr\n";
}
Note that unless you deactivate the signal pipe, you might also see EIO
(5) error during read operations.
StdinEvent contains the name of an event that Wheel::Run emits
whenever everything queued by its put() method has been flushed to the
child's STDIN handle. It is the equivalent to POE::Wheel::ReadWrite's
FlushedEvent.
StdinEvent comes with only one additional parameter: ARG0 contains
the unique ID for the wheel that sent the event.
StdoutEvent contains the name of an event that Wheel::Run emits
whenever the child process writes something to its STDOUT filehandle.
In other words, whatever the child prints to STDOUT, the parent
receives a StdoutEvent---provided that the child prints something
compatible with the parent's StdoutFilter.
StdoutEvent comes with two parameters. ARG0 contains the
information that the child wrote to STDOUT. ARG1 holds the unique
ID of the wheel that read the output.
sub stdout_state {
my ($heap, $input, $wheel_id) = @_[HEAP, ARG0, ARG1];
print "Child process in wheel $wheel_id wrote to STDOUT: $input\n";
}
StderrEvent behaves exactly as StdoutEvent, except for data the child
process writes to its STDERR filehandle.
StderrEvent comes with two parameters. ARG0 contains the
information that the child wrote to STDERR. ARG1 holds the unique
ID of the wheel that read the output.
sub stderr_state {
my ($heap, $input, $wheel_id) = @_[HEAP, ARG0, ARG1];
print "Child process in wheel $wheel_id wrote to STDERR: $input\n";
}
This is a filehandle or filename to which standard output will be redirected.
It is an error to use this option together with StdoutEvent. This is useful
in case your program needs to have standard I/O, but do not actually care for
its contents to be visible to the parent.
Just like RedirectStdout, but with standard error. It is an error to use this
together with StderrEvent
This is a filehandle or filename which the child process will use as its
standard input. It is an error to use this option with StdinEvent
This will redirect stderr and stdout to the same filehandle. This is equivalent
to do doing something like
$ something > /path/to/output 2>&1
in bourne shell.
While output filehandles will be closed if there are no events to be received on
them, stdin is open by default - because lack of an event handler does not
necessarily mean there is no desired input stream. This option explicitly
disables the creation of an IPC stdin conduit.
StdioFilter, if used, must contain an instance of a POE::Filter
subclass. This filter describes how the parent will format put() data
for the child's STDIN, and how the parent will parse the child's
STDOUT.
If STDERR will also be parsed, then a separate StderrFilter will also
be needed.
StdioFilter defaults to a POE::Filter::Line instance, but only if both
StdinFilter and StdoutFilter are not specified. If either StdinFilter
or StdoutFilter is used, then StdioFilter is illegal.
StdinFilter may be used to specify a particular STDIN serializer that
is different from the STDOUT parser. If specified, it conflicts with
StdioFilter. StdinFilter's value, if specified, must be an instance
of a POE::Filter subclass.
Without a StdinEvent, StdinFilter is illegal.
StdoutFilter may be used to specify a particular STDOUT parser that is
different from the STDIN serializer. If specified, it conflicts with
StdioFilter. StdoutFilter's value, if specified, must be an instance
of a POE::Filter subclass.
Without a StdoutEvent, StdoutFilter is illegal.
StderrFilter may be used to specify a filter for a child process'
STDERR output. If omitted, POE::Wheel::Run will create and use its
own POE::Filter::Line instance, but only if a StderrEvent is
specified.
Without a StderrEvent, StderrFilter is illegal.
Group contains a numeric group ID that the child process should run
within. By default, the child process will run in the same group as
the parent.
Group is not fully portable. It may not work on systems that have no
concept of user groups. Also, the parent process may need to run with
elevated privileges for the child to be able to change groups.
User contains a numeric user ID that should own the child process. By
default, the child process will run as the same user as the parent.
User is not fully portable. It may not work on systems that have no
concept of users. Also, the parent process may need to run with
elevated privileges for the child to be able to change users.
When true, NoSetSid disables setsid() in the child process. By
default, the child process calls setsid() is called so that it may
execute in a separate UNIX session.
When true, NoSetPgrp disables setprgp() in the child process. By
default, the child process calls setpgrp() to change its process
group, if the OS supports that.
setsid() is used instead of setpgrp() if Conduit is pty or pty-pipe.
See NoSetSid.
Priority adjusts the child process' niceness or priority level,
depending on which (if any) the underlying OS supports. Priority
contains a numeric offset which will be added to the parent's priority
to determine the child's.
The priority offset may be negative, which in UNIX represents a higher
priority. However UNIX requires elevated privileges to increase a
process' priority.
Program specifies the program to exec() or the block of code to run in
the child process. Program's type is significant.
If Program holds a scalar, its value will be executed as
exec($program). Shell metacharacters are significant, per
exec(SCALAR) semantics.
If Program holds an array reference, it will executed as
exec(@$program). As per exec(ARRAY), shell metacharacters will not be
significant.
If Program holds a code reference, that code will be called in the
child process. This mode allows POE::Wheel::Run to execute
long-running internal code asynchronously, while the usual modes
execute external programs. The child process will exit after that
code is finished, in such a way as to avoid DESTROY and END block
execution. See Coderef Execution Side Effects for more details.
perlfunc has more information about exec() and the different ways
to call it.
Please avoid calling exit() explicitly when executing a subroutine.
The child process inherits all objects from the parent, including ones
that may perform side effects. POE::Wheel::Run takes special care to
avoid object destructors and END blocks in the child process, but
calling exit() will trigger them.
If specified, ProgramArgs should refer to a list of parameters for the
program being run.
my @parameters = qw(foo bar baz); # will be passed to Program
ProgramArgs => \@parameters;
event() allows programs to change the events that Wheel::Run emits
when certain activities occurs. EVENT_TYPE may be one of the event
parameters described in POE::Wheel::Run's constructor.
This example changes the events that $wheel emits for STDIN flushing
and STDOUT activity:
$wheel->event(
StdinEvent => 'new-stdin-event',
StdoutEvent => 'new-stdout-event',
);
Undefined EVENT_NAMEs disable events.
put() queues up a list of RECORDS that will be sent to the child
process' STDIN filehandle. These records will first be serialized
according to the wheel's StdinFilter. The serialized RECORDS will be
flushed asynchronously once the current event handler returns.
get_stind_filter() returns the POE::Filter object currently being used
to serialize put() records for the child's STDIN filehandle. The
return object may be used according to its own interface.
get_stdout_filter() returns the POE::Filter object currently being
used to parse what the child process writes to STDOUT.
get_stderr_filter() returns the POE::Filter object currently being
used to parse what the child process writes to STDERR.
Set StdinFilter and StdoutFilter to the same new FILTER_OBJECT.
Unparsed STDOUT data will be parsed later by the new FILTER_OBJECT.
However, data already put() will remain serialized by the old filter.
Set StdinFilter to a new FILTER_OBJECT. Data already put() will
remain serialized by the old filter.
Set StdoutFilter to a new FILTER_OBJECT. Unparsed STDOUT data will be
parsed later by the new FILTER_OBJECT.
Set StderrFilter to a new FILTER_OBJECT. Unparsed STDERR data will be
parsed later by the new FILTER_OBJECT.
Pause reading of STDOUT from the child. The child process may block
if the STDOUT IPC conduit fills up. Reading may be resumed with
resume_stdout().
Pause reading of STDERR from the child. The child process may block
if the STDERR IPC conduit fills up. Reading may be resumed with
resume_stderr().
Resume reading from the child's STDOUT filehandle. This is only
meaningful if pause_stdout() has been called and remains in effect.
Resume reading from the child's STDERR filehandle. This is only
meaningful if pause_stderr() has been called and remains in effect.
shutdown_stdin() closes the child process' STDIN and stops the wheel
from reporting StdinEvent. It is extremely useful for running
utilities that expect to receive EOF on STDIN before they respond.
ID() returns the wheel's unique ID. Every event generated by a
POE::Wheel::Run object includes a wheel ID so that it can be matched
to the wheel that emitted it. This lets a single session manage
several wheels without becoming confused about which one generated
what event.
ID() is not the same as PID().
PID() returns the process ID for the child represented by the
POE::Wheel::Run object. It's often used as a parameter to
sig_child().
PID() is not the same as ID().
POE::Wheel::Run's kill() method sends a SIGNAL to the child process
the object represents. kill() is often used to force a reluctant
program to terminate. SIGNAL is one of the operating signal names
present in %SIG.
kill() returns the number of processes successfully signaled: 1 on
success, or 0 on failure, since the POE::Wheel::Run object only
affects at most a single process.
kill() sends SIGTERM if SIGNAL is undef or omitted.
get_driver_out_messages() returns the number of put() records
remaining in whole or in part in POE::Wheel::Run's POE::Driver output
queue. It is often used to tell whether the wheel has more input for
the child process.
In most cases, StdinEvent may be used to trigger activity when all
data has been sent to the child process.
get_driver_out_octets() returns the number of serialized octets
remaining in POE::Wheel::Run's POE::Driver output queue. It is often
used to tell whether the wheel has more input for the child process.
In the past POE::Wheel::Run did not support MSWin32 and users had to
use custom work-arounds. Then Chris Williams ( BINGOS ) arrived and
saved the day with his the POE::Wheel::Run::Win32 manpage module. After some
testing, it was decided to merge the win32 code into POE::Wheel::Run.
Everyone was happy!
However, after some investigation Apocalypse ( APOCAL ) found out that
in some situations it still didn't behave properly. The root cause was
that the win32 code path in POE::Wheel::Run didn't exit cleanly. This
means DESTROY and END blocks got executed! After talking with more
people, the solution was not pretty.
The problem is that there is no equivalent of POSIX::_exit() for MSWin32.
Hopefully, in a future version of Perl this can be fixed! In the meantime,
POE::Wheel::Run will use CORE::kill() to terminate the child. However,
this comes with a caveat: you will leak around 1KB per exec. The code
has been improved so the chance of this happening has been reduced.
As of now the most reliable way to trigger this is to exec an invalid
binary. The definition of ``invalid binary'' depends on different things,
but what it means is that Win32::Job->spawn() failed to run. This will
force POE::Wheel::Run to use the workaround to exit the child. If this
happens, a very big warning will be printed to the STDERR of the child
and the parent process will receive it.
If you are a Perl MSWin32 hacker, PLEASE help us with this situation! Go
read rt.cpan.org bug #56417 and talk with us/p5p to see where you can
contribute.
Thanks again for your patience as we continue to improve POE::Wheel::Run
on MSWin32!
Windows will often fail to report EOF on pipes when subprocesses are
killed. The work-around is to catch the signal in the subprocess, and
exit normally:
my $child = POE::Wheel::Run->new(
Program => sub {
$SIG{INT} = sub { exit };
...;
},
...,
);
Be sure to kill() the subprocess using the same signal that it catches
and exits upon. Remember, not all signals can be caught by user code.
$child->kill("INT");
It's common to scrub a child process' environment, so that only
required, secure values exist. This amounts to clearing the contents
of %ENV and repopulating it.
Environment scrubbing is easy when the child process is running a
subroutine, but it's not so easy---or at least not as intuitive---when
executing external programs.
The way we do it is to run a small subroutine in the child process
that performs the exec() call for us.
Program => \&exec_with_scrubbed_env,
sub exec_with_scrubbed_env {
delete @ENV{keys @ENV};
$ENV{PATH} = "/bin";
exec(@program_and_args);
}
That deletes everything from the environment and sets a simple, secure
PATH before executing a program.
The child process is created by fork(), which duplicates the parent
process including a copy of POE::Kernel, all running Session
instances, events in the queue, watchers, open filehandles, and so on.
When executing an external program, the UNIX exec() call immediately
replaces the copy of the parent with a completely new program.
When executing internal coderefs, however, we must preserve the code
and any memory it might reference. This leads to some potential side
effects.
Objects that were created in the parent process are copied into the
child. When the child exits normally, any DESTROY and END blocks are
executed there. Later, when the parent exits, they may run again.
POE::Wheel::Run takes steps to avoid running DESTROY and END blocks in
the child process. It uses POSIX::_exit() to bypass them. If that
fails, it may even kill() itself.
If an application needs to exit explicitly, for example to return an
error code to the parent process, then please use POSIX::_exit()
rather than Perl's core exit().
This warning is displayed from POE::Kernel's DESTROY method. It's a
side effect of calling exit() in a child process that was started
before POE::Kernel->run() could be called. The child process
receives a copy of POE::Kernel where run() wasn't called, even if it
was called later in the parent process.
The most direct solution is to call POSIX::_exit() rather than exit().
This will bypass POE::Kernel's DESTROY, and the message it emits.
Calling POE::Kernel->run() in the child process effectively
resumes the copy of the parent process. This is rarely (if ever)
desired.
More commonly, an application wants to run an entirely new POE::Kernel
instance in the child process. This is supported by first stop()ping
the copied instance, starting one or more new sessions, and calling
run() again. For example:
Program => sub {
# Wipe the existing POE::Kernel clean.
$poe_kernel->stop();
# Start a new session, or more.
POE::Session->create(
...
);
# Run the new sessions.
POE::Kernel->run();
}
Strange things are bound to happen if the program does not call
stop in the POE::Kernel manpage before run in the POE::Kernel manpage. However this is
vaguely supported in case it's the right thing to do at the time.
the POE::Wheel manpage describes wheels in general.
The SEE ALSO section in POE contains a table of contents covering
the entire POE distribution.
POE::Wheel::Run's constructor should emit proper events when it fails.
Instead, it just dies, carps or croaks. This isn't necessarily bad; a
program can trap the death in new() and move on.
Priority is a delta, not an absolute niceness value.
It might be nice to specify User by name rather than just UID.
It might be nice to specify Group by name rather than just GID.
POE::Pipe::OneWay and Two::Way don't require the rest of POE. They
should be spun off into a separate distribution for everyone to enjoy.
If StdinFilter and StdoutFilter seem backwards, remember that it's the
filters for the child process. StdinFilter is the one that dictates
what the child receives on STDIN. StdoutFilter tells the parent how
to parse the child's STDOUT.
Please see POE for more information about authors and contributors.
|