For part 5, we revisit the basics with a fresh example and a few lessons learned the hard way. A login form is a small piece of UI with a surprising number of paths through it: success, wrong password, locked account, slow responses and expired forms.
What to watch for
Keep configuration such as base URLs, delays and selectors out of your code. Changing a delay should not require a redeploy.
Deduplicate as you go. The same item often appears on several listing pages, and a stable identifier such as a SKU or ID makes duplicates easy to spot.
Validate the data you extract. Prices without currencies, dates without time zones and empty fields are common sources of silent errors downstream.
Common pitfalls
Selectors are the most fragile part of any scraper. Prefer stable attributes such as data-testid or semantic structure over generated class names, which can change on every deploy.
Time zones matter more than they seem. Store timestamps in UTC and convert only when displaying them.
Dynamic pages render content after the initial HTML arrives. A browser automation tool can wait for a specific element to appear, which is far more reliable than sleeping for a fixed amount of time.
Key takeaway
Cover each account state with its own test, assert on visible messages and on the resulting URL, and give slow paths an explicit, generous timeout.