Recipe 16.10 Communicating Between Related Processes

16.10.1 Problem

You have two related processes that need to communicate, and you need better control than you can get from open, system, and backticks.

16.10.2 Solution

Use pipe and then fork:

my ($reader, $writer);
pipe $reader, $writer;
if (fork) {
    # run parent code, either reading or writing, not both
} else {
    # run child code, either reading or writing, not both
}

Or use a special forking form of open:

if ($pid = (open $child, "|-")) {
        # run parent code, writing to child
} else {
    die "cannot fork: $!" unless defined $pid;
    # otherwise run child code here, reading from parent
}

Or, going the other way:

if ($pid = open ($child, "-|")) {
    # run parent code, reading from child
} else {
    die "cannot fork: $!" unless defined $pid;
    # otherwise run child code here, writing to parent
}

16.10.3 Discussion

Pipes are simply two connected filehandles, where data written to one filehandle can be read by the other. The pipe function creates two filehandles linked in this way, one writable and one readable. Even though you can't take two already existing filehandles and link them, pipe can be used for communication between processes. One process creates a pair of filehandles with the pipe functions, then forks off a child, resulting in two distinct processes both running in the same program, each with a copy of the connected filehandles. As with open, if pipe is passed undefined scalars instead of filehandles, it creates filehandles in those scalars.

It doesn't matter which process is the reader and which is the writer, so long as one of them takes one role and its peer process takes the other. You can only have one-way communication. (But read on.)

We'll pull in the IO::Handle module so we can call its autoflush method. (You could instead play the select games described in Chapter 7, if you prefer a lightweight solution.) If we didn't, our single line of output would get lodged in the pipe and not make it through to the other side until we closed that handle.

The version of the parent writing to the child is shown in Example 16-3.

Example 16-3. pipe1

  #!/usr/bin/perl -w
  # pipe1 - use pipe and fork so parent can send to child
  
  use IO::Handle;
  my ($reader, $writer);
  pipe $reader, $writer;
  $writer->autoflush(1);
  
  if ($pid = fork) {
      close $reader;
      print $writer "Parent Pid $$ is sending this\n";
      close $writer;
      waitpid($pid,0);
  } else {
      die "cannot fork: $!" unless defined $pid;
      close $writer;
      chomp($line = <$reader>);
      print "Child Pid $$ just read this: `$line'\n";
      close $reader;  # this will happen anyway
      exit;
  }

In the examples in this recipe, most error checking has been left as an exercise for the reader. This is so you can more clearly see how the functions interact. In real life, test the return values of all syscalls.

The version of the child writing to the parent is shown in Example 16-4.

Example 16-4. pipe2

  #!/usr/bin/perl -w
  # pipe2 - use pipe and fork so child can send to parent
  
  use IO::Handle;
  my ($reader, $writer);
  pipe($reader, $writer);
  $writer->autoflush(1);
  
  if ($pid = fork) {
      close $writer;
      chomp($line = <$reader>);
      print "Parent Pid $$ just read this: `$line'\n";
      close $reader;
      waitpid($pid,0);
  } else {
      die "cannot fork: $!" unless defined $pid;
      close $reader;
      print $writer "Child Pid $$ is sending this\n";
      close $writer;  # this will happen anyway
      exit;
  }

In most code, both halves would go into loops, with the reader continuing to read until end-of-file. This happens when the writer closes or exits.

Because piped filehandles are not bidirectional, each process uses just one of the pair and closes the filehandle it doesn't use. The reason is subtle; picture the situation where the reader does not close the writable filehandle. If the writer then exits while the reader is trying to read something, the reader will hang forever. This is because the system won't tell the reader that there's no more data to be read until all copies of the writable filehandle are closed.

The open function, when passed as its second argument either "-|" or "|-", will implicitly pipe and fork. This makes the piping code shown earlier slightly easier. The child talks to the parent over STDIN or STDOUT, depending on whether "-|" or "|-" was used.

Using open this way, if the parent wants to write to the child, it does something like what's shown in Example 16-5.

Example 16-5. pipe3

  #!/usr/bin/perl -w
  # pipe3 - use forking open so parent can send to child
  
  use IO::Handle;
  if ($pid = open ($child, "|-")) {
      $child->autoflush(1);
      print $child "Parent Pid $$ is sending this\n";
      close $child;
  } else {
      die "cannot fork: $!" unless defined $pid;
      chomp($line = <STDIN>);
      print "Child Pid $$ just read this: `$line'\n";
      exit;
  }

Since the child already has STDIN set to the parent, the child could exec some other program that expects to read from standard input, such as lpr. In fact, this is useful and commonly done.

If the child wants to write to the parent, it does something like what's shown in Example 16-6.

Example 16-6. pipe4

  #!/usr/bin/perl -w
  # pipe4 - use forking open so child can send to parent
  
  use IO::Handle;
  if ($pid = open $child, "-|") {
      chomp($line = <$child>);
      print "Parent Pid $$ just read this: `$line'\n";
      close $child;
  } else {
      die "cannot fork: $!" unless defined $pid;
      STDOUT->autoflush(1);
      print STDOUT "Child Pid $$ is sending this\n";
      exit;
  }

Again, since the child already has its STDOUT connected to the parent, this child could exec some other program to produce something interesting on its standard output. That output would be available to the parent as input from <CHILD>.

When using open this way, we don't have to manually call waitpid since we didn't do a manual fork. We do have to call close, though. In both cases, the $? variable will have the child's wait status in it (see Recipe 16.19 to see how to interpret this status value).

The preceding examples were unidirectional. What if you want both processes talking to each other? Just make two calls to pipe before forking. You must be careful about who tells whom what and when, though, or you're apt to deadlock. (See Example 16-7.)

Example 16-7. pipe5

  #!/usr/bin/perl -w
  # pipe5 - bidirectional communication using two pipe pairs
  #         designed for the socketpair-challenged
  use IO::Handle;
  my ($parent_rdr, $child_wtr, $child_rdr, $parent_wtr);
  pipe $parent_rdr, $child_wtr;
  pipe $child_rdr,  $parent_wtr;
  $child_wtr->autoflush(1);
  $parent_wtr->autoflush(1);
  
  if ($pid = fork) {
      close $parent_rdr; close $parent_wtr;
      print $child_wtr "Parent Pid $$ is sending this\n";
      chomp($line = <$child_rdr>);
      print "Parent Pid $$ just read this: `$line'\n";
      close $child_rdr; close $child_wtr;
      waitpid($pid,0);
  } else {
      die "cannot fork: $!" unless defined $pid;
      close $child_rdr; close $child_wtr;
      chomp($line = <$parent_rdr>);
      print "Child Pid $$ just read this: `$line'\n";
      print $parent_wtr "Child Pid $$ is sending this\n";
      close $parent_rdr; close $parent_wtr;
      exit;
  }

That's getting complicated. It just so happens that there's a special syscall, shown in Example 16-8, that makes the last example simpler. It's called socketpair, and it works like pipe, except that both handles can be used for reading and for writing.

Example 16-8. pipe6

  #!/usr/bin/perl -w
  # pipe6 - bidirectional communication using socketpair
  #   "the best ones always go both ways"
  
  use Socket;
  use IO::Handle;
  # We say AF_UNIX because although *_LOCAL is the
  # POSIX 1003.1g form of the constant, many machines
  # still don't have it.
  socketpair($child, $parent, AF_UNIX, SOCK_STREAM, PF_UNSPEC)
      or  die "socketpair: $!";
  
  $child->autoflush(1);
  $parent->autoflush(1);
  
  if ($pid = fork) {
      close $parent;
      print $child "Parent Pid $$ is sending this\n";
      chomp($line = <$child>);
      print "Parent Pid $$ just read this: `$line'\n";
      close $child;
      waitpid($pid,0);
  } else {
      die "cannot fork: $!" unless defined $pid;
      close $child;
      chomp($line = <$parent>);
      print "Child Pid $$ just read this: `$line'\n";
      print $parent "Child Pid $$ is sending this\n";
      close $parent;
      exit;
  }

In fact, some systems have historically implemented pipes as two half-closed ends of a socketpair. They essentially define pipe($reader, $writer) this way:

socketpair($reader, $writer, AF_UNIX, SOCK_STREAM, PF_UNSPEC);
shutdown($reader, 1);        # no more writing for reader
shutdown($writer, 0);        # no more reading for writer

16.10.4 See Also

Chapter 29 of Programming Perl or perlfunc(1) for all functions used here; the documentation for the standard IPC::Open2 module; Advanced Programming in the UNIX Environment; Recipe 16.8; Recipe 19.5