# robots.txt — MKB-ICT.nl User-agent: * Allow: / # Disallow API and internal/legal/thank-you pages (not useful for search) Disallow: /api/ Disallow: /contact-bedankt/ Disallow: /offerte-bedankt/ Disallow: /disclaimer/ Disallow: /privacybeleid/ Disallow: /algemene-voorwaarden/ # Sitemap Sitemap: https://mkb-ict.nl/sitemap.xml # AI crawlers — allow for llms.txt visibility User-agent: GPTBot Allow: / User-agent: ChatGPT-User Allow: / User-agent: CCBot Allow: / User-agent: Google-Extended Allow: / User-agent: PerplexityBot Allow: / User-agent: ClaudeBot Allow: /