<?xml version="1.0" encoding="UTF-8"?>
<response><data><post><id>113</id><slug>exporting-scraped-data-to-csv-part-6-113</slug><title>Exporting scraped data to CSV, part 6</title><excerpt>For part 6, we revisit the basics with a fresh example and a few lessons learned the hard way. CSV is the lingua franca of spreadsheets and the most common way scraped data leaves a project. It is…</excerpt><seriesName>Exporting scraped data to CSV</seriesName><seriesPart>6</seriesPart><sections><item><heading>Introduction</heading><paragraphs><item>For part 6, we revisit the basics with a fresh example and a few lessons learned the hard way. CSV is the lingua franca of spreadsheets and the most common way scraped data leaves a project. It is also surprisingly easy to produce a CSV that opens incorrectly.</item></paragraphs></item><item><heading>Putting it into practice</heading><paragraphs><item>Treat HTTP 429 responses as a signal, not an obstacle. Read the Retry-After header, back off, and resume at a slower pace.</item><item>Dynamic pages render content after the initial HTML arrives. A browser automation tool can wait for a specific element to appear, which is far more reliable than sleeping for a fixed amount of time.</item><item>Before writing a single line of code, look at how the page actually loads its data. Open the network panel, reload, and note which requests return the content you care about. Very often the data you want arrives as JSON from an endpoint you can call directly.</item></paragraphs></item><item><heading>Going further</heading><paragraphs><item>Selectors are the most fragile part of any scraper. Prefer stable attributes such as data-testid or semantic structure over generated class names, which can change on every deploy.</item><item>Keep configuration such as base URLs, delays and selectors out of your code. Changing a delay should not require a redeploy.</item><item>Deduplicate as you go. The same item often appears on several listing pages, and a stable identifier such as a SKU or ID makes duplicates easy to spot.</item></paragraphs></item><item><heading>Key takeaway</heading><paragraphs><item>Use a proper CSV writer rather than joining strings, quote fields that contain commas or newlines, write UTF-8, and keep dates in ISO 8601.</item></paragraphs></item></sections><authorId>4</authorId><authorName>Lucas Silva</authorName><authorSlug>lucas-silva</authorSlug><category>Data Engineering</category><tags><item>http</item><item>testing</item></tags><publishedAt>2025-07-11T00:00:00.000Z</publishedAt><wordCount>252</wordCount><readingTimeMinutes>2</readingTimeMinutes></post></data></response>
