Weird problem with errors
Hello, I hope someone might be able to help with this problem...
I'm experiencing a problem where my session will run perfectly 5 or 6 times in a row, but then for some reason it will begin to produce strange errors, like an extractor pattern only getting the first record of the set instead of 40, and an error like this:
An error occurred while processing the script: Go Browse Pg1
The application script threw an exception: java.lang.StringIndexOutOfBoundsException: String index out of range: -2 BSF info: null at line: 0 column: columnNo
The script of the error only has this:
session.scrapeFile( "Browse Page 1" );
Is there a reason anyone can see for this kind of behavior?
Weird problem with errors
I should be able to reproduce the error in the script that produced these errors and send them to you. Thanks for your suppport on it!
Weird problem with errors
Thanks for the tip. Is there a chance we could take a closer look at your scraping session that produces this issue? We'd like to get it cleared up. If that's a possibility, please drop me an email. My address is my first name at screen-scraper.com.
Kind regards,
Todd
Weird problem with errors
I saw today with another script writing data to a file after an extractor pattern that if I had a mistake in the script, the error it would return would be from the previous scrapable file.
It looks like when there's a loop going (a parameter being read in from a file from the previous page and looping to another page and back for the next iteration) that the error may read that it occurred on the file where the loop begins, instead of in the script where it is occurring. I don't know if this is the behavior that's intended or not, but it could be confusing.
Weird problem with errors
It looks like the error was due to a script that was writing data to 2 sepate files in the same script. I took the second one out and put it into a separate script running after it in sequence, and that solved the problem.
Unfortunately, I had changed the script befor you suggested sending it to you.
In the end, however, I decided to post it directly to the database without writing it to a file first, so the point became moot. Although the script ran much faster writing it to a file (I'm using basic version on my computer, and then posting it to a MySQL db on the web), I found I wasn't successful in being able to upload the csv files consistently for whatever reason.
Thanks for answering and offering to help on it. If I encounter the problem again, I'll save it and send it to you.
Weird problem with errors
Hi,
What version of screen-scraper are you currently running? Is there a chance you could send me your scraping session so that we can try to reproduce the issue? My email address is my first name at screen-scraper.com.
Kind regards,
Todd Wilson
Weird problem with errors
OK, I've analyzed it further, and this is very odd - maybe one of my scripts got corrupted?
On really analyzing the log file, I can see that the error is being called on a page and script that is not even in use at the time! The reason the extractor pattern is only pulling one result is that an error is happening when the script is called to write the data, but the log is showing the script of the error is not even in the scrapable file of the script where we're supposed to be in the process.
LOG FILE:
Storing this value in a session variable. // This is good...
Browse Page 1: Processing scripts after a pattern application. // As usual...
Processing script: "Browse Pg 1 Write to File" // Good - writing the data to a file...
Navigate to Browse Page: An error occurred while processing the script: Browse-Go Pg 1 // This script got us to the Browse Page 1 scrapable file, and is not called again, even for subsequent pages!? It shouldn't be here!
Navigate to Browse Page: The error message was: The application script threw an exception: java.lang.StringIndexOutOfBoundsException: String index out of range: -2 BSF info: null at line: 0 column: columnNo // Once again, this scrapable file should not be in use at this moment. :-(
0
Any ideas? I've tried it both on my mac and pc, and the same thing happens.