None value obtained from soup.find() when using Beautifulsoup and Selenium

The first solution provided in the output section works perfectly. If you don’t have ‘lxml’, you can try using ‘html.parser’ as noted in the output. However, some websites may simply return a blank when you request a value. To solve this issue, you can download chromedriver from the given link, unzip it, and put chromedriver.exe in C:Python27Scripts. Then, try the given code which should print the desired result. If the webpage has JavaScript code, the request may not return a complete result as the JS code is necessary to complete the page.


Question:

After spending a considerable amount of time on the site, I have been unable to locate a solution to my problem. As someone who is new to web scraping, my goal is to extract specific values from a web page using selenium and the hashtag
beautiful soup
.

from bs4 import BeautifulSoup
from selenium import webdriver
import requests
import time
PATH = "C:Program Files (x86)chromedriver.exe"
url = "https://meteofrance.com/previsions-meteo-france/nanterre/92000"
driver = webdriver.Chrome(PATH)
driver.get(url)
time.sleep(3)
page = driver.page_source
driver.quit()
soup = BeautifulSoup(page, 'html.parser')
temp = soup.find('strong', class_="temp")
info = soup.find('p', class_="svg_container")
wind = soup.find('strong', class_="wind-speed")
print(temp.text)
print(info)
print(wind)

Essentially, my aim is to reach a particular tag on the website in a simple manner.

Although it works for temperature, it fails to provide any information or wind data. I am experiencing a similar issue with another website, and I suspect that the problem is due to incomplete HTML. Unfortunately, I have been unable to find a solution, and even using Selenium does not resolve the issue.

I hope someone can help me !

Thanks Zway

from bs4 import BeautifulSoup
from selenium import webdriver
import requests
import time
PATH = "C:Program Files (x86)chromedriver.exe"
url = "https://meteofrance.com/previsions-meteo-france/nanterre/92000"
driver = webdriver.Chrome(PATH)
driver.get(url)
time.sleep(3)
page = driver.page_source
driver.quit()
soup = BeautifulSoup(page, 'html.parser')
temp = soup.find('strong', class_="temp")
info = soup.find('p', class_="svg_container")
wind = soup.find('strong', class_="wind-speed")
print(temp.text)
print(info)
print(wind)

This is what I get when I execute

17°
None
None



Solution:

You’re almost there! Simply refine your element choices to obtain the information you need.

...
temp = soup.find('strong', class_="temp")
info = soup.find('div', class_="svg_container").img['title']
wind = soup.find('p', class_="wind-speed").text.strip()
print(temp.text)
print(info)
print(wind)
...

Output:

18°
Très nuageux
10

Frequently Asked Questions