XPath for nested structures, part 3
Part 3 looks at what changes when the same technique has to run every day in production. When the data you want is several levels deep and the markup has few useful attributes, XPath's ability to…
Staff Writer
Sofia writes about practical data extraction and test automation.
6 articles
Part 3 looks at what changes when the same technique has to run every day in production. When the data you want is several levels deep and the markup has few useful attributes, XPath's ability to…
This is part 5 of our series, and this time we focus on edge cases readers asked about. robots.txt is a short text file with a long history. It tells automated clients which parts of a site the…
For part 10, we revisit the basics with a fresh example and a few lessons learned the hard way. Offset pagination asks for 'items 40 to 60'. Cursor pagination asks for 'the next page after this…
This is part 7 of our series, and this time we focus on edge cases readers asked about. CSV is the lingua franca of spreadsheets and the most common way scraped data leaves a project. It is also…
For part 7, we revisit the basics with a fresh example and a few lessons learned the hard way. Logging in once and staying logged in is a matter of keeping the cookies the server gives you and…
This is part 7 of our series, and this time we focus on edge cases readers asked about. Before you crawl a site link by link, check whether it already publishes a list of every page it wants found.…