Tutorial Details

Our "Shopping Site" example is pretty limited in that it can only handle one search term. What if we want to extract products for multiple search terms? For example, we may want to scrape various DVD titles that would fit with the other titles in our collection. We could search for the new DVD's using a series of keywords.

We're going to alter the existing "Shopping Site" scraping session so that it reads in a file containing search terms, and performs a search for each one. Just as before, as it performs a search it will follow the "details" links and extract out information for each product. Once the information is extracted it will write it out to a file.