For part 9, we revisit the basics with a fresh example and a few lessons learned the hard way. A selector that worked yesterday and fails today is usually a selector that depended on something nobody promised to keep the same, such as a generated class name or the position of a div.
A practical approach
Selectors are the most fragile part of any scraper. Prefer stable attributes such as data-testid or semantic structure over generated class names, which can change on every deploy.
Keep configuration such as base URLs, delays and selectors out of your code. Changing a delay should not require a redeploy.
Dynamic pages render content after the initial HTML arrives. A browser automation tool can wait for a specific element to appear, which is far more reliable than sleeping for a fixed amount of time.
Step by step
Validate the data you extract. Prices without currencies, dates without time zones and empty fields are common sources of silent errors downstream.
Write small, focused tests for your parsers using saved HTML fixtures. They run in milliseconds and catch layout changes long before a full crawl would.
Key takeaway
Prefer data-testid attributes, ARIA roles and visible labels. Avoid long chains of nth-child and anything that looks like a hash.