selenium - python requests sometimes returns empty list -
so i've been trying scrape "2005 - 2013" "drink between 2005 2013" , @ first code worked me empty lists returned, yet requests still have 200 status code
import requests, lxml.html, csv headers = {'user-agent':'mozilla/5.0 (windows nt 10.0; win64; x64) applewebkit/537.36 (khtml, gecko) chrome/59.0.3071.115 safari/537.36'} page = requests.get('http://www.cellartracker.com/wine.asp? iwine=91411',headers=headers) print(page.status_code) html = lxml.html.fromstring(page.content) content_divs = html.xpath('//a[@title="source: community"]/text()') print(content_divs)
not sure if should start using selenium scraping because it's js site, , if not sure how either basic useful! thanks!!
Comments
Post a Comment