Chapter 20. Web Automation

The web, then, or the pattern, a web at once sensuous and logical, an elegant and pregnant texture: that is style, that is the foundation of the art of literature.

—Robert Louis Stevenson, On some Technical Elements of Style in Literature (1885)

			Introduction
			Recipe 20.1.  Fetching a URL from a Perl Script
			Recipe 20.2.  Automating Form Submission
			Recipe 20.3.  Extracting URLs
			Recipe 20.4.  Converting ASCII to HTML
			Recipe 20.5.  Converting HTML to ASCII
			Recipe 20.6.  Extracting or Removing HTML Tags
			Recipe 20.7.  Finding Stale Links
			Recipe 20.8.  Finding Fresh Links
			Recipe 20.9.  Using Templates to Generate HTML
			Recipe 20.10.  Mirroring Web Pages
			Recipe 20.11.  Creating a Robot
			Recipe 20.12.  Parsing a Web Server Log File
			Recipe 20.13.  Processing Server Logs
			Recipe 20.14.  Using Cookies
			Recipe 20.15.  Fetching Password-Protected Pages
			Recipe 20.16.  Fetching https:// Web Pages
			Recipe 20.17.  Resuming an HTTP GET
			Recipe 20.18.  Parsing HTML
			Recipe 20.19.  Extracting Table Data
			Recipe 20.20.  Program: htmlsub
			Recipe 20.21.  Program: hrefsub