Write to CSV

The following code is useful for writing session variables out to a comma-seperated value (CSV) document. CSV files are very useful for viewing in spreadsheets or inserting values into a database.

Also, you'll notice that the session variables are cleared out at the end of the script. This would be done when you don't want a session variable to persist into the next dataRecord. For more about scope and dataRecords please go here.

// Set name of file to write to. the session variable "CSV_NAME" should be declared in another script such as an init script.
outputFile = session.getVariable ("CSV_NAME");

// another convenient way to set up your output file is to name the output after the scraping session.
// outputFile = session.getName() + ".csv";

// Error catching.
try
{
  //the following code is necessary to set up the file to be written to.
  File file = new File( outputFile );
  fileExists = file.exists();

  // Open up the file to be appended to.
  out = new FileWriter( outputFile, true );
  session.log( "Writing data to a file." );

  //this piece of code is responsible to write out the headers only 1 time.
  if (!fileExists)
  {
    // Write out the headers.
    out.write("\"" + "VIN" + "\"" + ",");
    out.write("\"" + "Year" + "\"" + ",");
    out.write("\"" + "Make" + "\"" + ",");
    out.write("\"" + "Model" + "\"" + ",");
    out.write("\"" + "Body type" + "\"" + ",");
    out.write("\"" + "Price" + "\"" + ",");
    out.write("\"" + "Mileage" + "\"" + ",");
    out.write("\"" + "Phone" + "\"" + ",");
    out.write("\"" + "Description" + "\"" + ",");
    out.write("\"" + "Source URL" + "\"");
    out.write( "\n" );
  }

  // Write columns.
  // the important part of this code is where the variable comes from (dataRecord or session variable)
  // if the variable was not saved as a session variable but instead this script was invoked after a dataRecord match you would use this code
  // out.write( prepareStringForOutput(dataRecord.get("VIN")) + "," );
  out.write( session.getVariable( "VIN" )+ "," );
  out.write( session.getVariable( "YEAR" )+ "," );
  out.write( session.getVariable( "MAKE" ) + "," );
  out.write( session.getVariable( "MODEL" )+ "," );
  out.write( session.getVariable( "BODY_TYPE") + "," );
  out.write( session.getVariable( "PRICE" )+ "," );
  out.write( session.getVariable( "MILEAGE" )+ "," );
  out.write( session.getVariable( "PHONE" )+ "," );
  out.write( session.getVariable( "DESCRIPTION" )+ "," );

  //if you would like to include the URL as a field of your csv you'd use this command
  out.write( scrapeableFile.getCurrentURL()); //note that the last out.write doesn't have a comma
  out.write( "\n" );

  // Close up the file.
  out.close();

  // Clear variables. you only need to clear session variables because dataRecord variables don't persist
  session.setVariable("VIN","");
  session.setVariable("YEAR","");
  session.setVariable("MAKE","");
  session.setVariable("MODEL","");
  session.setVariable("BODY_STYLE","");
  session.setVariable("PRICE","");
  session.setVariable("MILEAGE","");
  session.setVariable("PHONE","");
  session.setVariable("DESCRIPTION","");
  session.setVariable ("SOURCE_URL", "");
}
catch( Exception e )
{
  session.log( "An error occurred while writing the data to a file: " + e.getMessage() );
}