File corruption bug
I've not had the time to reproduce or try to reproduce this yet but:
1) Scrape a page, save session variables
2) Write session variables to a file
3) session.pause( 5000 );
4) Stop the scrape before the 5 second pause is up
5) The session does not stop immediately and may continue onto the next records if in a loop.
Re-run the scrape session without restarting SS yields corrupt data.
File corruption bug
Ryan,
We really appreciate your sharing the issues you've been having with screen-scraper. It's our goal to continuously improve screen-scraper.
I was unable to replicate the issue you're having. I created a simple scraping session attempting to cover each of the characteristics you described. Is it true that you are reading in a large amount of data from a file? If so, when you stop screen-scraper do you have "Stopping the scraping session" written to the log a number of times but does it finally say, "Scraping session finished"?
We're aware of screen-scraper not stopping immediately when asked when reading in large amounts of data from a file and I'm wondering if your issues isn't some how related.
In order to replicate your issue would you mind sending me your scraping session?
You can do so by emailing me at my username scottw and use our Website as the domain.
Thank you,
Scott