SocialFeed-Sandbox
A local sandbox that simulates a modern social media feed for testing and benchmarking web scrapers — SSR + hydration, cursor-paginated infinite scroll, header/session validation, rate limiting, and DOM obfuscation.
Scraper challenges
- Default headless / CLI User-Agents get a 403 (toggle in /admin).
- The feed is gated: log in at
/loginto get a signed auth token. - The login form carries a signed CSRF token you must extract and submit.
- /api/feed requires the session cookie issued by /feed.
- Wrapper class names are obfuscated — use data attributes, microdata, XPath, or text.
- Initial state is mirrored in a
#__FEED_HYDRATION_DATA__JSON script tag. - Rate limits can trigger HTTP 429.