{"data":{"id":41,"slug":"parsing-html-tables-with-beautifulsoup-part-3-41","title":"Parsing HTML tables with BeautifulSoup, part 3","excerpt":"This is part 3 of our series, and this time we focus on edge cases readers asked about. Tables look like the easiest thing on a page to scrape, until you meet merged cells, header rows that repeat…","seriesName":"Parsing HTML tables with BeautifulSoup","seriesPart":3,"sections":[{"heading":"Introduction","paragraphs":["This is part 3 of our series, and this time we focus on edge cases readers asked about. Tables look like the easiest thing on a page to scrape, until you meet merged cells, header rows that repeat every twenty lines and numbers formatted for humans rather than machines."]},{"heading":"Going further","paragraphs":["Keep configuration such as base URLs, delays and selectors out of your code. Changing a delay should not require a redeploy.","Pagination comes in several shapes: numbered pages, next links, offsets and opaque cursors. Each has trade-offs, and a robust scraper should detect the end of the result set rather than assuming a fixed number of pages."]},{"heading":"Putting it into practice","paragraphs":["Validate the data you extract. Prices without currencies, dates without time zones and empty fields are common sources of silent errors downstream.","Selectors are the most fragile part of any scraper. Prefer stable attributes such as data-testid or semantic structure over generated class names, which can change on every deploy."]},{"heading":"Key takeaway","paragraphs":["Locate the table by a stable attribute, read the header row once, and convert every cell to a typed value before you store it."]}],"authorId":4,"authorName":"Lucas Silva","authorSlug":"lucas-silva","category":"Scraping Basics","tags":["javascript","python","json"],"publishedAt":"2024-11-25T00:00:00.000Z","wordCount":180,"readingTimeMinutes":2}}