{"data":{"id":127,"slug":"extracting-json-ld-from-product-pages-part-7-127","title":"Extracting JSON-LD from product pages, part 7","excerpt":"Part 7 looks at what changes when the same technique has to run every day in production. Many product pages already contain a clean, structured description of themselves in a script tag meant for…","seriesName":"Extracting JSON-LD from product pages","seriesPart":7,"sections":[{"heading":"Introduction","paragraphs":["Part 7 looks at what changes when the same technique has to run every day in production. Many product pages already contain a clean, structured description of themselves in a script tag meant for search engines. Reading it is often more reliable than parsing the visible HTML."]},{"heading":"Going further","paragraphs":["Read the site's terms of service before you start. Many sites welcome automated access within limits and say so explicitly; others ask you to use an API instead.","Dynamic pages render content after the initial HTML arrives. A browser automation tool can wait for a specific element to appear, which is far more reliable than sleeping for a fixed amount of time.","Before writing a single line of code, look at how the page actually loads its data. Open the network panel, reload, and note which requests return the content you care about. Very often the data you want arrives as JSON from an endpoint you can call directly."]},{"heading":"Step by step","paragraphs":["Be careful with relative URLs. Resolve every link against the page it came from, not against the site root, or you will request pages that do not exist.","Store raw responses alongside parsed output while you are developing. When a parser breaks, you can replay the saved HTML instead of hitting the site again.","Deduplicate as you go. The same item often appears on several listing pages, and a stable identifier such as a SKU or ID makes duplicates easy to spot."]},{"heading":"Key takeaway","paragraphs":["Select script[type='application/ld+json'], parse every block, and pick the one whose @type matches what you need. Fall back to HTML only for missing fields."]}],"authorId":10,"authorName":"Aisha Rossi","authorSlug":"aisha-rossi","category":"Data Engineering","tags":["testing","python","ethics","json"],"publishedAt":"2025-09-06T00:00:00.000Z","wordCount":261,"readingTimeMinutes":2}}