{"data":{"id":27,"slug":"extracting-json-ld-from-product-pages-part-2-27","title":"Extracting JSON-LD from product pages, part 2","excerpt":"In part 2 we take the ideas from earlier instalments and apply them to a larger, messier site. Many product pages already contain a clean, structured description of themselves in a script tag meant…","seriesName":"Extracting JSON-LD from product pages","seriesPart":2,"sections":[{"heading":"Introduction","paragraphs":["In part 2 we take the ideas from earlier instalments and apply them to a larger, messier site. Many product pages already contain a clean, structured description of themselves in a script tag meant for search engines. Reading it is often more reliable than parsing the visible HTML."]},{"heading":"What to watch for","paragraphs":["Before writing a single line of code, look at how the page actually loads its data. Open the network panel, reload, and note which requests return the content you care about. Very often the data you want arrives as JSON from an endpoint you can call directly.","Read the site's terms of service before you start. Many sites welcome automated access within limits and say so explicitly; others ask you to use an API instead.","Selectors are the most fragile part of any scraper. Prefer stable attributes such as data-testid or semantic structure over generated class names, which can change on every deploy."]},{"heading":"Step by step","paragraphs":["Always identify your client honestly with a descriptive User-Agent, and keep your request rate modest. A crawler that behaves like a considerate visitor is far less likely to cause problems for the site operator.","Character encodings still cause trouble. Decode responses using the charset the server declares, and normalise text before comparing or storing it.","Validate the data you extract. Prices without currencies, dates without time zones and empty fields are common sources of silent errors downstream."]},{"heading":"Key takeaway","paragraphs":["Select script[type='application/ld+json'], parse every block, and pick the one whose @type matches what you need. Fall back to HTML only for missing fields."]}],"authorId":10,"authorName":"Aisha Rossi","authorSlug":"aisha-rossi","category":"Data Engineering","tags":["playwright"],"publishedAt":"2025-07-08T00:00:00.000Z","wordCount":251,"readingTimeMinutes":2}}