Part 10 looks at what changes when the same technique has to run every day in production. Lazy loading defers images and sections until they are about to scroll into view. A script that reads the page immediately sees placeholders instead of content.
Putting it into practice
Time zones matter more than they seem. Store timestamps in UTC and convert only when displaying them.
Keep configuration such as base URLs, delays and selectors out of your code. Changing a delay should not require a redeploy.
Write small, focused tests for your parsers using saved HTML fixtures. They run in milliseconds and catch layout changes long before a full crawl would.
Step by step
Store raw responses alongside parsed output while you are developing. When a parser breaks, you can replay the saved HTML instead of hitting the site again.
Selectors are the most fragile part of any scraper. Prefer stable attributes such as data-testid or semantic structure over generated class names, which can change on every deploy.
Key takeaway
Scroll the element into view and wait for its real attributes to appear, or read the data-src style attributes that hold the eventual values.