Saving Issues
I am using the following to save files after scraping. The scraping pattern is not nearly as perfect as I thought earlier either! It seems that our sample data is near perfect while the real data has numerous flaws in it that causes the patterns to misfire or nor fire.
Anyway the following code is saving the data from the first page over and over again. Watching the logs, the correct information was pulled from the scrape, so I am at a loss to why it is saving only the first set of data. Do I need to manually clear the session variables after save?
Thanks,
Stacy
{
session.log( "Writing data to file." );
// Open up the file to be appended to.
out = new FileWriter( "mdj_data.txt", true );
// Write out the data to the file.
out.write( dataRecord.get( "NAME" ) + "\t" );
out.write( dataRecord.get( "ADDRESS1" ) + "\t" );
out.write( dataRecord.get( "CITY" ) + "\t" );
out.write( dataRecord.get( "STATE" ) + "\t" );
out.write( dataRecord.get( "ZIP" ) + "\t" );
out.write( dataRecord.get( "LOANDATEMONTH" ) + ", " + dataRecord.get( "LOANDATEYEAR" ) + "\t" );
out.write( dataRecord.get( "PRINCIPLE" ) + "\t" );
out.write( dataRecord.get( "LEGAL" ) + "\t" );
out.write( "\n" );
out.write( "\n" );
// Close up the file.
out.close();
}
catch( Exception e )
{
session.log( "An error occurred while writing the data to a file: " + e.getMessage() );
};
Saving Issues
Todd on vacation, will send it soonest! Have to work out some other issues first with sub-extractors
Saving Issues
Hi Stacy,
You're script looks good. Most likely it's an issue of how/when it's being invoked. The easiest way for us to help would be for you to email your scraping session to us at [email protected] . If you'd prefer not to do that would you mind replying with the portion of your log where the script is getting invoked?
Thanks,
Todd Wilson