Creating header records
I am scraping this url but, although I can see the data in chrome tools and I can collect using curl I can't seem to get the headers to work in SS.
https://webapp.artisio.co/website/lots/?page=1&limit=100&sort=end_date&sort-by=asc&lot_status=published&auction_uuid=ad5917f5-c056-4037-92f0-7e57d8ecaa2d
These are the header records needed:
^-H "authority: webapp.artisio.co" ^
-H "accept: application/json, text/plain, */*" ^
-H "accept-language: en-GB,en;q=0.9" ^
-H "artisio-client-id: 45781238" ^
-H "artisio-language: en" ^
-H "cache-control: no-cache" ^
-H "origin: https://gpsv.co.uk" ^
-H "pragma: no-cache" ^
-H "referer: https://gpsv.co.uk/" ^
-H "sec-ch-ua-mobile: ?0" ^
-H "sec-ch-ua-platform: ^\^"Windows^\^"" ^
I am sure they need to go in a script somewhere and get called at the appropriate time, but I have tried loads of options and I can't get to work.
Any ideas?
Thanks
Jason
The method you need is
The method you need is scrapeableFile.addHTTPHeader().
Here's a sample of a similar one, and your run the script on the scrapeable file before the file is scraped.
scrapeableFile.addHTTPHeader("Connection", "keep-alive");
scrapeableFile.setContentType("application/x-www-form-urlencoded");
scrapeableFile.addHTTPHeader("DNT", "1");
scrapeableFile.addHTTPHeader("Sec-Fetch-Dest", "document");
scrapeableFile.addHTTPHeader("Sec-Fetch-Mode", "navigate");
scrapeableFile.addHTTPHeader("Sec-Fetch-Site", "same-origin");
scrapeableFile.addHTTPHeader("Sec-Fetch-User", "?1");
scrapeableFile.addHTTPHeader("Upgrade-Insecure-Requests", "1");
Worked Perfectly
Thank you very much for your help!