This is part 5 of our series, and this time we focus on edge cases readers asked about. When the data you want is several levels deep and the markup has few useful attributes, XPath's ability to navigate up, down and sideways through the tree becomes very handy.
A practical approach
Time zones matter more than they seem. Store timestamps in UTC and convert only when displaying them.
Deduplicate as you go. The same item often appears on several listing pages, and a stable identifier such as a SKU or ID makes duplicates easy to spot.
Step by step
Store raw responses alongside parsed output while you are developing. When a parser breaks, you can replay the saved HTML instead of hitting the site again.
Pagination comes in several shapes: numbered pages, next links, offsets and opaque cursors. Each has trade-offs, and a robust scraper should detect the end of the result set rather than assuming a fixed number of pages.
Key takeaway
Anchor on a stable nearby element, then use relative paths and axes such as following-sibling and ancestor rather than absolute paths from the root.