{"data":{"id":134,"slug":"testing-login-flows-end-to-end-part-7-134","title":"Testing login flows end to end, part 7","excerpt":"In part 7 we take the ideas from earlier instalments and apply them to a larger, messier site. A login form is a small piece of UI with a surprising number of paths through it: success, wrong…","seriesName":"Testing login flows end to end","seriesPart":7,"sections":[{"heading":"Introduction","paragraphs":["In part 7 we take the ideas from earlier instalments and apply them to a larger, messier site. A login form is a small piece of UI with a surprising number of paths through it: success, wrong password, locked account, slow responses and expired forms."]},{"heading":"What to watch for","paragraphs":["Always identify your client honestly with a descriptive User-Agent, and keep your request rate modest. A crawler that behaves like a considerate visitor is far less likely to cause problems for the site operator.","Be careful with relative URLs. Resolve every link against the page it came from, not against the site root, or you will request pages that do not exist.","Read the site's terms of service before you start. Many sites welcome automated access within limits and say so explicitly; others ask you to use an API instead."]},{"heading":"Putting it into practice","paragraphs":["Selectors are the most fragile part of any scraper. Prefer stable attributes such as data-testid or semantic structure over generated class names, which can change on every deploy.","Pagination comes in several shapes: numbered pages, next links, offsets and opaque cursors. Each has trade-offs, and a robust scraper should detect the end of the result set rather than assuming a fixed number of pages.","Validate the data you extract. Prices without currencies, dates without time zones and empty fields are common sources of silent errors downstream."]},{"heading":"Key takeaway","paragraphs":["Cover each account state with its own test, assert on visible messages and on the resulting URL, and give slow paths an explicit, generous timeout."]}],"authorId":8,"authorName":"Hannah Novak","authorSlug":"hannah-novak","category":"Testing","tags":["http","scrapy","json","ethics"],"publishedAt":"2026-08-25T00:00:00.000Z","wordCount":246,"readingTimeMinutes":2}}