How to get specific data using BeautifulSoup


How to get specific data using BeautifulSoup



I'm not sure how to get a specific result from this:


<div class="videoPlayer">
<div class="border-radius-player">
<div id="allplayers" style="position:relative;width:100%;height:100%;overflow: hidden;">
<div id="box">
<div id="player_content" class="todo" style="text-align: center; display: block;">
<div id="player" class="jwplayer jew-reset jew-skin-seven jw-state-paused jw-flag-user-inactive" tabindex="0">
<div class="jw-media jw-reset">
<video class="jw-video jw-reset" x-webkit-playsinline="" src="https:EXAMPLE-URL-HERE" preload="metadata"></video>
</div">



How would I get the src in <video class="jw-video jw-reset" x-webkit-playsinline="" src="https:EXAMPLE-URL-HERE" preload="metadata"></video>


src


<video class="jw-video jw-reset" x-webkit-playsinline="" src="https:EXAMPLE-URL-HERE" preload="metadata"></video>



This is what I've tried so far:


import urllib.request
from bs4 import BeautifulSoup

url = "https://someurlhere"

a = urllib.request.Request(url, headers={'User-Agent' : "Cliqz"})
b = urllib.request.urlopen(a) # prevent "Permission denies"

soup = BeautifulSoup(b, 'html.parser')

for video_class in soup.select("div.videoPlayer"):
print(video_class.text)



Which returns parts of it but not down to video class


video class





Requests only download a static webpage and are unable to deal with javascript code. Can you do a simple string search in b to make sure that the elements you need exists in the html code?
– Simas Joneliunas
Jul 2 at 0:40





It doesn't, it goes down to box but I thought that BeautifulSoup would be able to handle that.
– HelloThereToad
Jul 2 at 0:47


box





Try for video_class in soup.select("div.videoPlayer video.jw-video.jw-reset"): print(video_class.attrs['src'])
– The fourth bird
Jul 2 at 7:34


for video_class in soup.select("div.videoPlayer video.jw-video.jw-reset"): print(video_class.attrs['src'])




1 Answer
1



Requests is a simple html client, it cannot execute javascripts.



You have three more options to try here though!






By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

-

Popular posts from this blog

How to add background colour in existing image using Swift?

How to add background colour in existing image using Swift? I am trying to add background colour to existing image but it is not working. Here below is my code:- extension UIImage { func imageWithColor(tintColor: UIColor) -> UIImage { let rect = CGRect(x: 0, y: 0, width: 1, height: 1) UIGraphicsBeginImageContextWithOptions(rect.size, false, 0) tintColor.setFill() UIRectFill(rect) let image: UIImage = UIGraphicsGetImageFromCurrentImageContext()! UIGraphicsEndImageContext() //self.type(of: init)(ciImage: CIImage(image: image)!) return image } } extension UIImage { /** Returns an UIImage with a specified background color. - parameter color: The color of the background */ convenience init(withBackground color: UIColor) { let rect: CGRect = CGRect(x: 0, y: 0, width: 1, height: 1) UIGraphicsBeginImageContext(rect.size); let context:CGContext = UIGraphicsGetCurrentContext()!; context.setFillColor(color.cgColor); context.fill(rec...
Read more

Moria Casán

Image
Moria Casán Casán in 2016 Born Ana María Casanova ( 1946-08-16 ) August 16, 1946 (age 71) Buenos Aires, Argentina Residence Parque Leloir, Ituzaingó, Buenos Aires Province, Argentina [1] Occupation Actress, presenter, theatre producer, TV personality, talent manager Years active 1970–present Spouse(s) Mario Castiglione ( m.  1986; div.  1990) Children Sofía Gala Castiglione Signature Ana María Casanova (born August 16, 1946), known by her stage name Moria Casán , is an Argentine actress and TV personality. Casán made her theatre debut in 1970, and quickly became one of the country's leading vedettes during Argentina's Golden Age of revue. Her sex symbol status was further cemented during the late 1970s and 1980s, as she starred in various sexual comedy films alongside Alberto Olmedo, Jorge Porcel and Susana Giménez. Since the mid-1980s, Casán has also established herself as an important TV personality, hostin...
Read more

How to make file upload 'Required' in Contact Form 7?

How to make file upload 'Required' in Contact Form 7? I have a Contact Form 7 form on my Wordpress website and I have specified that the file upload is mandatory by putting a * , however the users are able to submit the form without file upload. * Relevant Contact Form 7 form is given below: <div class="filelink">[file* AddanImage filetypes:gif|png|jpg|jpeg]</div> How do I write the Contact Form 7 code or add a validation perhaps so that the users are not able to submit the form without uploading an image? Any hints/help/guidance is highly appreciated. Have you check doc - contactform7.com/file-uploading-and-attachment – Mukesh Panchal Jun 30 at 8:38 @MukeshPanchal I have and the code complies with the documentation – Hamza Ahmad Jul 1 at 10:44 ...
Read more