<?xml version="1.0" encoding="UTF-8"?>
<response><data><post><id>111</id><slug>xpath-for-nested-structures-part-6-111</slug><title>XPath for nested structures, part 6</title><excerpt>In part 6 we take the ideas from earlier instalments and apply them to a larger, messier site. When the data you want is several levels deep and the markup has few useful attributes, XPath's ability…</excerpt><seriesName>XPath for nested structures</seriesName><seriesPart>6</seriesPart><sections><item><heading>Introduction</heading><paragraphs><item>In part 6 we take the ideas from earlier instalments and apply them to a larger, messier site. When the data you want is several levels deep and the markup has few useful attributes, XPath's ability to navigate up, down and sideways through the tree becomes very handy.</item></paragraphs></item><item><heading>Common pitfalls</heading><paragraphs><item>Read the site's terms of service before you start. Many sites welcome automated access within limits and say so explicitly; others ask you to use an API instead.</item><item>Before writing a single line of code, look at how the page actually loads its data. Open the network panel, reload, and note which requests return the content you care about. Very often the data you want arrives as JSON from an endpoint you can call directly.</item><item>Time zones matter more than they seem. Store timestamps in UTC and convert only when displaying them.</item></paragraphs></item><item><heading>What to watch for</heading><paragraphs><item>Store raw responses alongside parsed output while you are developing. When a parser breaks, you can replay the saved HTML instead of hitting the site again.</item><item>Validate the data you extract. Prices without currencies, dates without time zones and empty fields are common sources of silent errors downstream.</item></paragraphs></item><item><heading>Key takeaway</heading><paragraphs><item>Anchor on a stable nearby element, then use relative paths and axes such as following-sibling and ancestor rather than absolute paths from the root.</item></paragraphs></item></sections><authorId>3</authorId><authorName>Priya Nakamura</authorName><authorSlug>priya-nakamura</authorSlug><category>Scraping Basics</category><tags><item>http</item><item>testing</item></tags><publishedAt>2025-03-08T00:00:00.000Z</publishedAt><wordCount>212</wordCount><readingTimeMinutes>2</readingTimeMinutes></post></data></response>
