Working with CSRF tokens, part 10
Part 10 looks at what changes when the same technique has to run every day in production. Cross-site request forgery tokens are one of the first things that trip up a script trying to log in to a…
Showing 37–48 of 59 articles
Part 10 looks at what changes when the same technique has to run every day in production. Cross-site request forgery tokens are one of the first things that trip up a script trying to log in to a…
Most flaky Selenium tests are timing problems in disguise. The element exists, just not yet, and a fixed sleep is either too short on a slow day or wastes time on a fast one.
In part 8 we take the ideas from earlier instalments and apply them to a larger, messier site. Tables look like the easiest thing on a page to scrape, until you meet merged cells, header rows that…
Part 9 looks at what changes when the same technique has to run every day in production. CSV is the lingua franca of spreadsheets and the most common way scraped data leaves a project. It is also…
Part 10 looks at what changes when the same technique has to run every day in production. If a site offers an API for the data you need, scraping its HTML is usually the harder, more fragile and less…
Part 8 looks at what changes when the same technique has to run every day in production. A selector that worked yesterday and fails today is usually a selector that depended on something nobody…
Part 6 looks at what changes when the same technique has to run every day in production. If a site offers an API for the data you need, scraping its HTML is usually the harder, more fragile and less…
This is part 8 of our series, and this time we focus on edge cases readers asked about. Logging in once and staying logged in is a matter of keeping the cookies the server gives you and sending them…
For part 10, we revisit the basics with a fresh example and a few lessons learned the hard way. A well-behaved API tells you how many requests you have left before it starts saying no. Reading those…
This is part 8 of our series, and this time we focus on edge cases readers asked about. Lazy loading defers images and sections until they are about to scroll into view. A script that reads the page…
This is part 10 of our series, and this time we focus on edge cases readers asked about. Infinite scroll replaces the familiar Next button with a page that keeps growing as you move down it, which…
For part 6, we revisit the basics with a fresh example and a few lessons learned the hard way. Before you crawl a site link by link, check whether it already publishes a list of every page it wants…