{"data":{"id":187,"slug":"extracting-json-ld-from-product-pages-part-10-187","title":"Extracting JSON-LD from product pages, part 10","excerpt":"In part 10 we take the ideas from earlier instalments and apply them to a larger, messier site. Many product pages already contain a clean, structured description of themselves in a script tag meant…","seriesName":"Extracting JSON-LD from product pages","seriesPart":10,"sections":[{"heading":"Introduction","paragraphs":["In part 10 we take the ideas from earlier instalments and apply them to a larger, messier site. Many product pages already contain a clean, structured description of themselves in a script tag meant for search engines. Reading it is often more reliable than parsing the visible HTML."]},{"heading":"Common pitfalls","paragraphs":["Pagination comes in several shapes: numbered pages, next links, offsets and opaque cursors. Each has trade-offs, and a robust scraper should detect the end of the result set rather than assuming a fixed number of pages.","Be careful with relative URLs. Resolve every link against the page it came from, not against the site root, or you will request pages that do not exist."]},{"heading":"Putting it into practice","paragraphs":["Write small, focused tests for your parsers using saved HTML fixtures. They run in milliseconds and catch layout changes long before a full crawl would.","Deduplicate as you go. The same item often appears on several listing pages, and a stable identifier such as a SKU or ID makes duplicates easy to spot."]},{"heading":"Key takeaway","paragraphs":["Select script[type='application/ld+json'], parse every block, and pick the one whose @type matches what you need. Fall back to HTML only for missing fields."]}],"authorId":5,"authorName":"Chen Patel","authorSlug":"chen-patel","category":"Data Engineering","tags":["ethics","http","javascript","python"],"publishedAt":"2025-10-29T00:00:00.000Z","wordCount":188,"readingTimeMinutes":2}}