{"data":{"id":76,"slug":"detecting-lazy-loaded-content-part-4-76","title":"Detecting lazy-loaded content, part 4","excerpt":"This is part 4 of our series, and this time we focus on edge cases readers asked about. Lazy loading defers images and sections until they are about to scroll into view. A script that reads the page…","seriesName":"Detecting lazy-loaded content","seriesPart":4,"sections":[{"heading":"Introduction","paragraphs":["This is part 4 of our series, and this time we focus on edge cases readers asked about. Lazy loading defers images and sections until they are about to scroll into view. A script that reads the page immediately sees placeholders instead of content."]},{"heading":"Common pitfalls","paragraphs":["Always identify your client honestly with a descriptive User-Agent, and keep your request rate modest. A crawler that behaves like a considerate visitor is far less likely to cause problems for the site operator.","Dynamic pages render content after the initial HTML arrives. A browser automation tool can wait for a specific element to appear, which is far more reliable than sleeping for a fixed amount of time.","Character encodings still cause trouble. Decode responses using the charset the server declares, and normalise text before comparing or storing it."]},{"heading":"Going further","paragraphs":["Treat HTTP 429 responses as a signal, not an obstacle. Read the Retry-After header, back off, and resume at a slower pace.","Keep configuration such as base URLs, delays and selectors out of your code. Changing a delay should not require a redeploy."]},{"heading":"Key takeaway","paragraphs":["Scroll the element into view and wait for its real attributes to appear, or read the data-src style attributes that hold the eventual values."]}],"authorId":7,"authorName":"Kwame Mensah","authorSlug":"kwame-mensah","category":"Browser Automation","tags":["playwright","javascript"],"publishedAt":"2025-10-08T00:00:00.000Z","wordCount":200,"readingTimeMinutes":2}}