Creating the Script to Initialize the Scraping Session

We're now going to create a small script to initialize our scraping session. It's a common practice to run a script at the very beginning of a scraping session that can initialize variables and such. That's what we'll be doing here.

Recording Search Results

As in the first tutorial, we'll be recording a browser session using the proxy server. Remember that a proxy session holds all of the HTTP requests and responses from your browser for the period of time you run it.

Screen-Scraping Overview Review

As you'll remember from the previous tutorials, extracting information from web sites using screen-scraper typically involves four main steps: