Strange data being appended to sesion variable
Hi all
getting a bizarre problem, can anyone help ?
In my script file for a scraping session I have :
objHttp.Send sData
sResponse = objHttp.ResponseText
session.setVariable "data",sResponse
session.log sResponse
looking in the log, I find that sResponse was correct (it's a 116kb chunk of XML).
However, in my ASP I find that retrieving the session variable :
dataFile=dumpToFile(objSS.GetVariable("data"))
results in the characters "___CR______NL___" being appended.
using the VID debugger, it appears the extra characters are being appended somewhere during the call to getVariable ...
Strange data being appended to sesion variable
Hi Jason,
That could work as well. If you strip out those characters in the scraping session itself, you shouldn't see the funny tokens come through on the other side.
Kind regards,
Todd
Strange data being appended to sesion variable
Thanks for that Todd ! I have contacted the suppliers of the application I am working with, as this is an intermittent problem, (it does't happen with all records). If I recall correctly non-printing characters are not XML-compliant, and should be sent as entities, &cr; &lf etc ....
In the meantime, as long as I strip out the carriage returns/line feeds before I assign the data to a session variable, I should be OK ?
thanks again
Strange data being appended to sesion variable
Hi,
Looks like you've found a flaw in our COM driver that we'll need to fix. When strings contain carriage returns and line feeds (as your chunk of XML likely does), when screen-scraper passes the text back to the application that called it, it replaces those carriage returns and line feeds with special markers. Those markers apparently aren't being swapped out correctly in the COM driver. The quick solution is for you to simply find and replace them once you get the string in your ASP code. I'll write up a bug, though, so that we're sure to fix this in the next version of screen-scraper.
Also, it's our policy to offer a discount on a Professional Edition license of screen-scraper to anyone who finds a previously-unknown bug in our code. If you'd like to take advantage of this, feel free to email me personally. My email address is my first name at screen-scraper.com.
Kind regards,
Todd Wilson
Strange data being appended to sesion variable
I have added some more debug. In the scraping script :
[b]session.setVariable "dataLength",len(sResponse)[/b]
in the ASP :
[b]dataLength=objSS.GetVariable("dataLength")
if clng(dataLength)<> len(objSS.GetVariable("data")) then
response.write "returned dataLength=" & dataLength & ". But length of data=" & len(objSS.GetVariable("data"))
Response.End
else
dataFile=dumpToFile(left (objSS.GetVariable("data"),objSS.GetVariable("dataLength")-2))
end if[/b]
The output :
[b]returned dataLength=118835. But length of data=118849[/b]
Anyone any ideas ?