scrape file with todays date
My scrap-able URL has the current date in it. Everything else is static. How can I grab that portion of the URL (maybe make a GET parameter?) and change it to todays date so that it brings in that HTML? I've seen bits of ideas for this in my search but I am not sure. Thanks.
Here is a sample URL to show what I am trying to do. The date of 052814 is always the current date.
http://www.thesite.com/static/entry/xyz052814test.html
That's pretty easy. On the
That's pretty easy. On the scrapableFile you'd set the URL to use a session variable named "URL", and sometime before you call the scrapeableFile have a script to:
url += sutil.getCurrentDate("MMddyy");
url += "test.html";
session.log("==>Setting URL to " + url);
session.setv("URL", url);
Awesome!
Thanks that rocks! Only 1 question. When I start the entire process and set my LAN settings to use a proxy server and then record my link - do I record the link with the current date? Then in the session do I leave that alone? I am assuming I am only changing the URL in the scrapable file. Thanks again.
Well, it depends on what your
Well, it depends on what your scrape needs, but what you describe should work fine.