Working with CSRF tokens, part 9
In part 9 we take the ideas from earlier instalments and apply them to a larger, messier site. Cross-site request forgery tokens are one of the first things that trip up a script trying to log in to…
Data Engineer
Aisha writes about practical data extraction and test automation.
13 articles
In part 9 we take the ideas from earlier instalments and apply them to a larger, messier site. Cross-site request forgery tokens are one of the first things that trip up a script trying to log in to…
For part 10, we revisit the basics with a fresh example and a few lessons learned the hard way. Tables look like the easiest thing on a page to scrape, until you meet merged cells, header rows that…
Part 4 looks at what changes when the same technique has to run every day in production. A selector that worked yesterday and fails today is usually a selector that depended on something nobody…
For part 3, we revisit the basics with a fresh example and a few lessons learned the hard way. Lazy loading defers images and sections until they are about to scroll into view. A script that reads…
Part 4 looks at what changes when the same technique has to run every day in production. Running a browser without a window is faster and works on servers, but some problems are much easier to…
Part 7 looks at what changes when the same technique has to run every day in production. Many product pages already contain a clean, structured description of themselves in a script tag meant for…
Part 2 looks at what changes when the same technique has to run every day in production. If a site offers an API for the data you need, scraping its HTML is usually the harder, more fragile and less…
In part 2 we take the ideas from earlier instalments and apply them to a larger, messier site. Many product pages already contain a clean, structured description of themselves in a script tag meant…
In part 4 we take the ideas from earlier instalments and apply them to a larger, messier site. A well-behaved API tells you how many requests you have left before it starts saying no. Reading those…
In part 8 we take the ideas from earlier instalments and apply them to a larger, messier site. A well-behaved API tells you how many requests you have left before it starts saying no. Reading those…
In part 10 we take the ideas from earlier instalments and apply them to a larger, messier site. When the data you want is several levels deep and the markup has few useful attributes, XPath's ability…
For part 4, we revisit the basics with a fresh example and a few lessons learned the hard way. Networks fail, servers restart and rate limits kick in. A scraper that gives up at the first error is…
For part 2, we revisit the basics with a fresh example and a few lessons learned the hard way. A well-behaved API tells you how many requests you have left before it starts saying no. Reading those…