Tutorial 4: Page 3: Invoking screen-scraper from Cold Fusion

Before we dig into the code you'll probably want to take a minute to review our Invoking screen-scraper from ColdFusion documentation page. Remember that you need to add the "screen-scraper.jar" file for you classpath in order to be able to interact with screen-scraper.

In order to invoke screen-scraper from ColdFusion, screen-scraper needs to be running in server mode. If you'd like a refresher on how to start up screen-scraper in server mode go ahead and follow that link, then come back here.

Okay, let's try it out before we go over the code. Download the shopping.cfm.txt file here, then save it in a directory that will be accessible from your web server. Rename the file from "shopping.cfm.txt" to "shopping.cfm". After that start up screen-scraper in server mode.

Open up your web browser and go to the URL corresponding to the "shopping.cfm" file (e.g, "http://localhost/screen-scraper/shopping.cfm"). You'll see a simple search form. Type in a product keyword, such as "bug", then hit the "Go" button. If all goes well the page will take a little while to load (it's waiting as screen-scraper extracts the data), then it will output the corresponding products.

If that didn't go quite as you expected here are some things to check:

• Make sure screen-scraper is running as a server, and that nothing is blocking its ports (such as a firewall running on your machine).
• If you're running screen-scraper on a different machine than the one your ColdFusion file resides on, make sure that screen-scraper is allowing connections from the ColdFusion machine. In the screen-scraper workbench click on the (wrench) icon, then on the "Servers" button, and check the "Hosts to allow to connect" includes the IP address (or perhaps just the first part of the IP address) of the ColdFusion machine. You might also try blanking that property out entirely, which will allow connections from any host. When developing, this is the usually the easiest approach on Windows Vista and Mac OS X, for example.
• Ensure that the permissions on the "shopping.cfm" file are such that your web server can execute it.
• Check screen-scraper's "log" folder for a "Shopping Site" log file. If you find one it means that screen-scraper is at least receiving the request. Open the log file in a text editor to see if you find any error messages.
• If you still can't seem to get it to work feel free to post to our forum.

Assuming that test worked, fire up your favorite ColdFusion editor and open the "shopping.cfm" file in it. The file is pretty heavily commented, so hopefully it makes sense what's going on. If not, try reviewing ColdFusion documentation or posting to our forum.

When you invoke screen-scraper as a server it creates log files corresponding to your scraping session in its "log" folder. Take a look in that folder for your "Shopping Site" log file and take a look through it. It should look similar to what you see when you run scraping sessions in the workbench.