For part 9, we revisit the basics with a fresh example and a few lessons learned the hard way. Running a browser without a window is faster and works on servers, but some problems are much easier to understand when you can watch the browser do its work.
Going further
Treat HTTP 429 responses as a signal, not an obstacle. Read the Retry-After header, back off, and resume at a slower pace.
Character encodings still cause trouble. Decode responses using the charset the server declares, and normalise text before comparing or storing it.
Step by step
Validate the data you extract. Prices without currencies, dates without time zones and empty fields are common sources of silent errors downstream.
Before writing a single line of code, look at how the page actually loads its data. Open the network panel, reload, and note which requests return the content you care about. Very often the data you want arrives as JSON from an endpoint you can call directly.
Key takeaway
Develop headed with slow motion turned on, run headless in CI, and capture screenshots or traces on failure so headless runs are still debuggable.