<?xml version="1.0" encoding="UTF-8"?>
<response><data><post><id>11</id><slug>xpath-for-nested-structures-11</slug><title>XPath for nested structures</title><excerpt>When the data you want is several levels deep and the markup has few useful attributes, XPath's ability to navigate up, down and sideways through the tree becomes very handy.</excerpt><seriesName>XPath for nested structures</seriesName><seriesPart>1</seriesPart><sections><item><heading>Introduction</heading><paragraphs><item>When the data you want is several levels deep and the markup has few useful attributes, XPath's ability to navigate up, down and sideways through the tree becomes very handy.</item></paragraphs></item><item><heading>What to watch for</heading><paragraphs><item>Before writing a single line of code, look at how the page actually loads its data. Open the network panel, reload, and note which requests return the content you care about. Very often the data you want arrives as JSON from an endpoint you can call directly.</item><item>Selectors are the most fragile part of any scraper. Prefer stable attributes such as data-testid or semantic structure over generated class names, which can change on every deploy.</item></paragraphs></item><item><heading>Step by step</heading><paragraphs><item>Read the site's terms of service before you start. Many sites welcome automated access within limits and say so explicitly; others ask you to use an API instead.</item><item>Log every request with its URL, status code and duration. When something goes wrong at two in the morning, those logs are the difference between a quick fix and a long investigation.</item></paragraphs></item><item><heading>Key takeaway</heading><paragraphs><item>Anchor on a stable nearby element, then use relative paths and axes such as following-sibling and ancestor rather than absolute paths from the root.</item></paragraphs></item></sections><authorId>11</authorId><authorName>Oliver Kim</authorName><authorSlug>oliver-kim</authorSlug><category>Scraping Basics</category><tags><item>python</item></tags><publishedAt>2024-04-29T00:00:00.000Z</publishedAt><wordCount>189</wordCount><readingTimeMinutes>2</readingTimeMinutes></post></data></response>
