{"data":{"id":193,"slug":"exporting-scraped-data-to-csv-part-10-193","title":"Exporting scraped data to CSV, part 10","excerpt":"For part 10, we revisit the basics with a fresh example and a few lessons learned the hard way. CSV is the lingua franca of spreadsheets and the most common way scraped data leaves a project. It is…","seriesName":"Exporting scraped data to CSV","seriesPart":10,"sections":[{"heading":"Introduction","paragraphs":["For part 10, we revisit the basics with a fresh example and a few lessons learned the hard way. CSV is the lingua franca of spreadsheets and the most common way scraped data leaves a project. It is also surprisingly easy to produce a CSV that opens incorrectly."]},{"heading":"What to watch for","paragraphs":["Keep configuration such as base URLs, delays and selectors out of your code. Changing a delay should not require a redeploy.","Log every request with its URL, status code and duration. When something goes wrong at two in the morning, those logs are the difference between a quick fix and a long investigation."]},{"heading":"Going further","paragraphs":["Time zones matter more than they seem. Store timestamps in UTC and convert only when displaying them.","Store raw responses alongside parsed output while you are developing. When a parser breaks, you can replay the saved HTML instead of hitting the site again."]},{"heading":"Key takeaway","paragraphs":["Use a proper CSV writer rather than joining strings, quote fields that contain commas or newlines, write UTF-8, and keep dates in ISO 8601."]}],"authorId":9,"authorName":"Mateo Hughes","authorSlug":"mateo-hughes","category":"Data Engineering","tags":["http"],"publishedAt":"2024-11-10T00:00:00.000Z","wordCount":168,"readingTimeMinutes":2}}