python - Extracting all the contents from webpage having similar xpath(contents that can be extracted in a list) -


i have scrape page, http://www.imdb.com/search/title?count=100&release_date=2016,2016&title_type=feature. while using xpath scrape movie name, 'the lost city of z'. here code:

  driver_t.get('http://www.imdb.com/search/title?count=100&release_date=2016,2016&title_type=feature')   x= driver_t.find_element_by_xpath('//*[@id="main"]/div/div/div[3]/div[1]/div[3]/h3/a')   print x.text 

in order scrape movies, removed [1] xpath

  driver_t.get('http://www.imdb.com/search/title?count=100&release_date=2016,2016&title_type=feature')   x= driver_t.find_element_by_xpath('//*[@id="main"]/div/div/div[3]/div/div[3]/h3/a')   print x.text 

but, output first movie's name('the lost city of z')

this worked in 'r' not working in python(selenium webdriver). tell me going wrong?

here answer question:

in order scrape movies page url http://www.imdb.com/search/title?count=100&release_date=2016,2016&title_type=feature instead of find_element_by_xpath using driver.find_elements_by_xpath return list. next iterate through list , retrieve text , print them 1 one. here code block reference:

from selenium import webdriver selenium.webdriver.chrome.options import options  options = options() options.add_argument("start-maximized") options.add_argument("disable-infobars") options.add_argument("--disable-extensions") driver = webdriver.chrome(chrome_options=options, executable_path="c:\\utility\\browserdrivers\\chromedriver.exe") driver.get("http://www.imdb.com/search/title?count=100&release_date=2016,2016&title_type=feature") titles = driver.find_elements_by_xpath("//h3[@class='lister-item-header']/a") title in titles:     movie_name = title.get_attribute("innerhtml")     print(movie_name) 

let me know if answers question.


Comments

Popular posts from this blog

php - Vagrant up error - Uncaught Reflection Exception: Class DOMDocument does not exist -

vue.js - Create hooks for automated testing -

Add new key value to json node in java -