Testing login flows end to end, part 10
This is part 10 of our series, and this time we focus on edge cases readers asked about. A login form is a small piece of UI with a surprising number of paths through it: success, wrong password,…
QA Engineer
Oliver writes about practical data extraction and test automation.
17 articles
This is part 10 of our series, and this time we focus on edge cases readers asked about. A login form is a small piece of UI with a surprising number of paths through it: success, wrong password,…
This is part 3 of our series, and this time we focus on edge cases readers asked about. Offset pagination asks for 'items 40 to 60'. Cursor pagination asks for 'the next page after this marker'. The…
For part 3, we revisit the basics with a fresh example and a few lessons learned the hard way. Logging in once and staying logged in is a matter of keeping the cookies the server gives you and…
This is part 9 of our series, and this time we focus on edge cases readers asked about. Networks fail, servers restart and rate limits kick in. A scraper that gives up at the first error is fragile;…
Part 9 looks at what changes when the same technique has to run every day in production. Offset pagination asks for 'items 40 to 60'. Cursor pagination asks for 'the next page after this marker'. The…
For part 4, we revisit the basics with a fresh example and a few lessons learned the hard way. When the data you want is several levels deep and the markup has few useful attributes, XPath's ability…
In part 3 we take the ideas from earlier instalments and apply them to a larger, messier site. Running a browser without a window is faster and works on servers, but some problems are much easier to…
Part 9 looks at what changes when the same technique has to run every day in production. Before you crawl a site link by link, check whether it already publishes a list of every page it wants found.…
This is part 8 of our series, and this time we focus on edge cases readers asked about. If a site offers an API for the data you need, scraping its HTML is usually the harder, more fragile and less…
Part 9 looks at what changes when the same technique has to run every day in production. CSV is the lingua franca of spreadsheets and the most common way scraped data leaves a project. It is also…
If a site offers an API for the data you need, scraping its HTML is usually the harder, more fragile and less polite option.
For part 10, we revisit the basics with a fresh example and a few lessons learned the hard way. A well-behaved API tells you how many requests you have left before it starts saying no. Reading those…
This is part 10 of our series, and this time we focus on edge cases readers asked about. Running a browser without a window is faster and works on servers, but some problems are much easier to…
When the data you want is several levels deep and the markup has few useful attributes, XPath's ability to navigate up, down and sideways through the tree becomes very handy.
In part 3 we take the ideas from earlier instalments and apply them to a larger, messier site. Infinite scroll replaces the familiar Next button with a page that keeps growing as you move down it,…
Part 5 looks at what changes when the same technique has to run every day in production. CSV is the lingua franca of spreadsheets and the most common way scraped data leaves a project. It is also…
In part 4 we take the ideas from earlier instalments and apply them to a larger, messier site. Before you crawl a site link by link, check whether it already publishes a list of every page it wants…