Start a fight
Home/Tools/Robots.txt generator

Robots.txt
generator

Build a correct robots.txt to control how search engines and AI crawlers read your site.

Technical SEO

Broken link checker

Scan any page for broken links, redirects and errors that cost you rankings.

Use tool → Technical SEO

Robots.txt generator

Build a correct robots.txt in seconds, with one-tap control over AI crawlers like GPTBot and ClaudeBot.

Use tool → Structured data

Schema markup generator

Make JSON-LD for 9 page types so Google can show rich results for you.

Use tool → AI-ready

llms.txt generator

Tell ChatGPT, Claude and Perplexity what your site is about, in the file they look for.

Use tool →
Checks up to 150 links on the page: broken (4xx/5xx), redirects (3xx) and timeouts.
Search engines
One per line. These stay out of Google.
AI crawlers

robots.txt

Save this as robots.txt at the root of your site, like yoursite.com/robots.txt.

  


  
  

    

      

      

    

    
JSON-LD

      
Paste it inside the <head> of the page. Then check it with Google's Rich Results Test.

  


  
  

    

      

      

      

      
One per line: Title | URL | short note

    

    
llms.txt
Save as llms.txt at your site's root. AI assistants read it to understand and quote you.

  


  
  

    

      

        

        

      

      
Choosing a country other than the US is a Pro feature.

      

        

      

      
    

    
  


  
  

    

      Prospect Hunter
      
How it works

      
    
        
  1. Pick an industry and a city. Roofers in Vancouver, dentists in Austin, anything.
    2. 
        
  2. We find the businesses losing. Who ranks on page 1 for buyer searches, and who is stuck behind them.
    3. 
        
  3. You get the proof. A fight page per prospect: their site vs the competitor beating them, round by round.
    4. 
        
  4. You send the pitch. A ready-to-send email for each one: "Your rival beats you 91–62. Here's why."
    5. 
      

      
200 fights / month20 Keyword Battles30 hunts / month$99 / month

    

    

      Prospect Hunter
      
Find clients losing to rivals

      
We search Google, fight up to 6 local businesses against the one ranking #1, and write a pitch for each. Takes about a minute.

      

      

      

      

      

    

  

  
  


  
What robots.txt does
Robots.txt tells crawlers which parts of your site they may read. Blocking a path keeps it out of search. Blocking AI crawlers like GPTBot means ChatGPT can't learn about or recommend your site, so only block them on purpose.