Gtjkyu

XPATH syntax validator in Python I develop a crawler with many actions. Many xpaths are involving and for this reason I use a json file for storing. Then crawler start running I would like to make a basic syntax check (before xpath usage) on xpaths and raise error for invalid xpaths. for example: xpath1 = '//*[@id="react-root"]/section' xpath2 = '//*[[@id="react-root"]/section' xpath3 = '//*[@id="react-root"]section' from these xpaths only xpath1 is valid is there any module or regex which does this kind of validation? 1 Answer 1 You can compile the xpath strings with lxml.etree.XPath which will raise an exception if the syntax is incorrect: lxml.etree.XPath >>> import lxml.etree >>> lxml.etree.XPath('//*[@id="react-root"]/section') //*[@id="react-root"]/section >>> lxml.etree.XPath('//*...

Webcrawler extract link elements I want to extract elements from a webpage. $html = file_get_contents($link); That function returns the complete html file, and I only want the internal and external links to save them in the database. $sql = "INSERT INTO prueba (link, title, description) VALUES (?, ?, ?)"; //preparando los datos $query = $pdo->prepare($sql); //orden de ejecucion $result = $query->execute([ $link, $title_out, $description ]); Here, I am already managing to extract the description and the title, and I manage to place them in the database, but I want to extract all the external and internal links. The internal links in one column and the external links in another. I already have both columns in the database created. 2 Answers 2 I suggest using a DOM-Parser library like: Parse the HTML and just "query" for all anchors ( a tags). a Much less error-prone t...