User-agent: * Allow: / # Block account/utility pages and the API to save crawler budget Disallow: /dashboard/ Disallow: /admin/ Disallow: /api/ Disallow: /login/ Disallow: /register/ Disallow: /accounts/ Disallow: /password-reset Disallow: /toggle-save/ Disallow: /subscribe/ Disallow: /jobs/suggest/ Sitemap: https://techjobsdata.com/sitemap.xml