{"data":[{"id":30,"slug":"choosing-css-selectors-that-survive-redesigns-part-2-30","title":"Choosing CSS selectors that survive redesigns, part 2","excerpt":"Before writing a single line of code, look at how the page actually loads its data. Open the network panel, reload, and note which requests return the content…","authorId":8,"authorName":"Hannah Novak","category":"Ethics & Law","tags":["ethics","testing","selenium"],"publishedAt":"2025-12-27T00:00:00.000Z","readingTimeMinutes":2},{"id":55,"slug":"retry-and-backoff-patterns-part-3-55","title":"Retry and backoff patterns, part 3","excerpt":"Before writing a single line of code, look at how the page actually loads its data. Open the network panel, reload, and note which requests return the content…","authorId":1,"authorName":"Amelia Okafor","category":"APIs","tags":["selenium","http","testing"],"publishedAt":"2025-12-23T00:00:00.000Z","readingTimeMinutes":2},{"id":32,"slug":"session-cookies-for-scrapers-part-2-32","title":"Session cookies for scrapers, part 2","excerpt":"Dynamic pages render content after the initial HTML arrives. A browser automation tool can wait for a specific element to appear, which is far more reliable…","authorId":7,"authorName":"Kwame Mensah","category":"Testing","tags":["testing","scrapy","selenium","javascript"],"publishedAt":"2025-12-20T00:00:00.000Z","readingTimeMinutes":2},{"id":192,"slug":"session-cookies-for-scrapers-part-10-192","title":"Session cookies for scrapers, part 10","excerpt":"Pagination comes in several shapes: numbered pages, next links, offsets and opaque cursors. Each has trade-offs, and a robust scraper should detect the end of…","authorId":11,"authorName":"Oliver Kim","category":"Testing","tags":["json","playwright","testing","scrapy"],"publishedAt":"2025-12-11T00:00:00.000Z","readingTimeMinutes":2},{"id":144,"slug":"building-a-polite-scrapy-spider-part-8-144","title":"Building a polite Scrapy spider, part 8","excerpt":"Pagination comes in several shapes: numbered pages, next links, offsets and opaque cursors. Each has trade-offs, and a robust scraper should detect the end of…","authorId":5,"authorName":"Chen Patel","category":"Data Engineering","tags":["ethics","http","json","selenium"],"publishedAt":"2025-12-06T00:00:00.000Z","readingTimeMinutes":2},{"id":164,"slug":"building-a-polite-scrapy-spider-part-9-164","title":"Building a polite Scrapy spider, part 9","excerpt":"Always identify your client honestly with a descriptive User-Agent, and keep your request rate modest. A crawler that behaves like a considerate visitor is far…","authorId":12,"authorName":"Yuki Andersson","category":"Ethics & Law","tags":["scrapy","playwright","python","selenium"],"publishedAt":"2025-11-14T00:00:00.000Z","readingTimeMinutes":2},{"id":149,"slug":"rate-limit-headers-explained-part-8-149","title":"Rate limit headers explained, part 8","excerpt":"Treat HTTP 429 responses as a signal, not an obstacle. Read the Retry-After header, back off, and resume at a slower…","authorId":3,"authorName":"Priya Nakamura","category":"Data Engineering","tags":["scrapy","selenium","python"],"publishedAt":"2025-11-06T00:00:00.000Z","readingTimeMinutes":2},{"id":41,"slug":"parsing-html-tables-with-beautifulsoup-part-3-41","title":"Parsing HTML tables with BeautifulSoup, part 3","excerpt":"Store raw responses alongside parsed output while you are developing. When a parser breaks, you can replay the saved HTML instead of hitting the site…","authorId":9,"authorName":"Mateo Hughes","category":"Scraping Basics","tags":["testing","http"],"publishedAt":"2025-11-05T00:00:00.000Z","readingTimeMinutes":2},{"id":190,"slug":"choosing-css-selectors-that-survive-redesigns-part-10-190","title":"Choosing CSS selectors that survive redesigns, part 10","excerpt":"Validate the data you extract. Prices without currencies, dates without time zones and empty fields are common sources of silent errors…","authorId":9,"authorName":"Mateo Hughes","category":"Ethics & Law","tags":["ethics","selenium"],"publishedAt":"2025-10-30T00:00:00.000Z","readingTimeMinutes":2},{"id":170,"slug":"choosing-css-selectors-that-survive-redesigns-part-9-170","title":"Choosing CSS selectors that survive redesigns, part 9","excerpt":"Validate the data you extract. Prices without currencies, dates without time zones and empty fields are common sources of silent errors…","authorId":5,"authorName":"Chen Patel","category":"Ethics & Law","tags":["scrapy"],"publishedAt":"2025-10-21T00:00:00.000Z","readingTimeMinutes":2},{"id":24,"slug":"building-a-polite-scrapy-spider-part-2-24","title":"Building a polite Scrapy spider, part 2","excerpt":"Selectors are the most fragile part of any scraper. Prefer stable attributes such as data-testid or semantic structure over generated class names, which can…","authorId":5,"authorName":"Chen Patel","category":"Testing","tags":["http","python","javascript"],"publishedAt":"2025-10-07T00:00:00.000Z","readingTimeMinutes":2},{"id":43,"slug":"respecting-robots-txt-in-practice-part-3-43","title":"Respecting robots.txt in practice, part 3","excerpt":"Treat HTTP 429 responses as a signal, not an obstacle. Read the Retry-After header, back off, and resume at a slower…","authorId":6,"authorName":"Sofia Fischer","category":"Scraping Basics","tags":["ethics","javascript"],"publishedAt":"2025-10-04T00:00:00.000Z","readingTimeMinutes":2},{"id":125,"slug":"cursor-vs-offset-pagination-part-7-125","title":"Cursor vs offset pagination, part 7","excerpt":"Pagination comes in several shapes: numbered pages, next links, offsets and opaque cursors. Each has trade-offs, and a robust scraper should detect the end of…","authorId":11,"authorName":"Oliver Kim","category":"Testing","tags":["http","selenium"],"publishedAt":"2025-10-01T00:00:00.000Z","readingTimeMinutes":2},{"id":96,"slug":"detecting-lazy-loaded-content-part-5-96","title":"Detecting lazy-loaded content, part 5","excerpt":"Treat HTTP 429 responses as a signal, not an obstacle. Read the Retry-After header, back off, and resume at a slower…","authorId":11,"authorName":"Oliver Kim","category":"Data Engineering","tags":["javascript","json"],"publishedAt":"2025-09-28T00:00:00.000Z","readingTimeMinutes":2},{"id":3,"slug":"respecting-robots-txt-in-practice-3","title":"Respecting robots.txt in practice","excerpt":"Selectors are the most fragile part of any scraper. Prefer stable attributes such as data-testid or semantic structure over generated class names, which can…","authorId":11,"authorName":"Oliver Kim","category":"Ethics & Law","tags":["scrapy"],"publishedAt":"2025-09-27T00:00:00.000Z","readingTimeMinutes":2},{"id":112,"slug":"session-cookies-for-scrapers-part-6-112","title":"Session cookies for scrapers, part 6","excerpt":"Pagination comes in several shapes: numbered pages, next links, offsets and opaque cursors. Each has trade-offs, and a robust scraper should detect the end of…","authorId":7,"authorName":"Kwame Mensah","category":"Data Engineering","tags":["scrapy","testing","python"],"publishedAt":"2025-09-24T00:00:00.000Z","readingTimeMinutes":2},{"id":106,"slug":"waiting-strategies-in-selenium-part-6-106","title":"Waiting strategies in Selenium, part 6","excerpt":"Pagination comes in several shapes: numbered pages, next links, offsets and opaque cursors. Each has trade-offs, and a robust scraper should detect the end of…","authorId":1,"authorName":"Amelia Okafor","category":"Scraping Basics","tags":["python","javascript"],"publishedAt":"2025-09-22T00:00:00.000Z","readingTimeMinutes":2},{"id":118,"slug":"headless-vs-headed-browsers-part-6-118","title":"Headless vs headed browsers, part 6","excerpt":"Validate the data you extract. Prices without currencies, dates without time zones and empty fields are common sources of silent errors…","authorId":9,"authorName":"Mateo Hughes","category":"Ethics & Law","tags":["testing"],"publishedAt":"2025-09-16T00:00:00.000Z","readingTimeMinutes":2},{"id":188,"slug":"working-with-csrf-tokens-part-10-188","title":"Working with CSRF tokens, part 10","excerpt":"Store raw responses alongside parsed output while you are developing. When a parser breaks, you can replay the saved HTML instead of hitting the site…","authorId":10,"authorName":"Aisha Rossi","category":"Scraping Basics","tags":["javascript","ethics","scrapy","python"],"publishedAt":"2025-09-12T00:00:00.000Z","readingTimeMinutes":2},{"id":2,"slug":"handling-infinite-scroll-in-playwright-2","title":"Handling infinite scroll in Playwright","excerpt":"Selectors are the most fragile part of any scraper. Prefer stable attributes such as data-testid or semantic structure over generated class names, which can…","authorId":12,"authorName":"Yuki Andersson","category":"Testing","tags":["selenium"],"publishedAt":"2025-09-10T00:00:00.000Z","readingTimeMinutes":2}],"meta":{"total":200,"limit":20,"offset":0,"pagination":"page","page":1,"totalPages":10},"links":{"next":"/api/blog?page=2","prev":null}}