User-Agent: AhrefsBot User-Agent: SemrushBot User-Agent: MJ12bot User-Agent: DotBot User-Agent: BLEXBot User-Agent: DataForSeoBot User-Agent: PetalBot User-Agent: Bytespider User-Agent: serpstatbot User-Agent: ZoominfoBot Disallow: / User-Agent: * Allow: / Disallow: /api/ Disallow: /app/ Disallow: /signup/welcome Host: https://marketcensus.io Sitemap: https://marketcensus.io/sitemap.xml