NavigationUser loginscreen-scraper.com welcomes...
Currently online
There are currently 1 user and 5 guests online.
Online users
|
I'd like to scrape information from a web site that requires me to log in first. Can screen-scraper handle this?Yes. This is a common situation, and generally just requires that you create a scrapeable file to handle logging in. This scrapeable file should be run first in the scraping session, allowing the web site to set cookies, which screen-scraper will then track for you. For example, if you wanted to scrape a list of all auctions you're watching from the ebay web site, you would create a scrapeable file that would first log you in (issue a POST request with your username and password), then you would create subsequent scrapeable files that would scrape the information you're interested in. There is also a special type of authentication known as "BASIC" or "WWW-Authenticate". You'll know a web site is using this when, upon attempting to access a particular URL, you are presented with a small dialog box requesting a username and password. When setting up screen-scraper to scrape a page using this type of authentication you simply need to enter in the username and password in the "Properties" tab under "BASIC Authentication Parameters" for the scrapeable file you set up to scrape the page. Note that you generally only need to enter the username and password once for a given site on a single scrapeable file, as screen-scraper will retain the username and password for you. We give an example of configuring screen-scraper to log in to a site in our second tutorial
|
SearchNew Video!Tags Throughout this Site |
Recent comments
1 hour 9 min ago
1 hour 16 min ago
3 hours 22 min ago
22 hours 48 min ago
23 hours 5 min ago
23 hours 53 min ago
1 day 1 min ago
1 day 11 min ago
1 day 21 min ago
2 days 21 hours ago