You asked ChatGPT for the best contractor, dentist or accountant in your part of the Triangle. It named three of your competitors. It did not name you.
The easy conclusion is that AI search has it in for you. Sometimes the real answer is much duller. Your website is shutting the door.
This is the first thing to rule out, and it takes about ten minutes.
Why does a business sometimes never appear in ChatGPT at all?
There are a lot of reasons an assistant skips a business. Thin reviews. A name that appears nowhere else on the web. A site that never says plainly what you do or where you do it.
Those are real, and they are slower to fix.
Then there is a shorter list of mechanical causes:
- A setting that tells crawlers to stay out
- A platform default you never chose
- Pages that only load properly for a person in a browser
- Addresses that lead nowhere
Mechanical problems are worth checking first for one reason. They are cheap to find, and nothing else you do matters while they exist.
How do AI assistants actually reach the pages on your website?
Think of an assistant as a visitor that cannot click around and cannot be charmed by your photos. It asks a server for a page, and it reads whatever text comes back.
Several different visitors can do this. Some gather pages to build a search index. Some fetch a page live because a user asked a question. Some collect text for training. OpenAI, for example, uses separate names for these, including OAI-SearchBot, ChatGPT-User and GPTBot.
Each one introduces itself by name when it arrives. Your site can welcome it, ignore it or turn it away based on that name.
That is the whole mechanism. If the visitor is turned away, or arrives and finds nothing readable, you have nothing to show it.
What a crawler setting can quietly turn away
Every site can have a small text file at yourdomain.com/robots.txt. It lists which visitors may go where.
One short line can block a visitor by name. Another can block everyone from everything. The second one is common on sites built in a staging area and then launched, because nobody removed the "keep everyone out while we build" rule.
Look for two kinds of problems:
- A blanket rule that disallows the whole site for all visitors
- A named rule that disallows the crawlers you actually want
Some people add rules against AI crawlers on purpose, usually over worries about training. That is a legitimate choice. Just make sure it was a choice, and that it did not also block the crawler that finds you for search.
Could your platform or plugin be blocking assistants by default?
Often, nobody on your team touched the file at all.
Website platforms, security plugins, hosting firewalls and content delivery networks can each decide on their own to treat unfamiliar automated visitors as threats. A tool built to stop spam bots may not distinguish a scraper from a legitimate assistant.
So the file can look perfectly fine while the server still returns an error to the visitor. The setting that matters may live in a dashboard you have never opened.
Ask three people the same question:
- Your host: "Do you block automated visitors by default?"
- Whoever manages your security plugin or firewall
- Whoever built the site
Do not accept "I think it's fine." Ask them to look.
How to check in ten minutes whether your pages can be read
You do not need a developer to get a first read. Try this:
- Type your domain followed by
/robots.txtinto a browser. Read what it says. Look for a line that blocks everything, or one that names an assistant crawler. - Open one of your important pages, such as a service page, and view the page source. Search for a sentence from the visible text. If you can find it in the source, a machine can too.
- Ask a free tool or your developer to request the page while identifying as a crawler, and report the status that comes back. A healthy page answers with a normal success. A block usually shows up as a refusal or a challenge page.
- Try your homepage and one deep page. Blocks are sometimes selective.
Write down what you find. Even "everything looks fine" is useful, because it tells you to stop worrying about this and move on to the slower causes.
Pages that only work for people: what to look for
A page can look complete to you and be nearly empty to a machine.
This happens when the text is loaded by a script after the page opens. It happens when content sits behind a tab or a "read more" button that needs a click. It happens when key details are baked into an image, and when a login or a cookie pop-up stands in front of everything.
A few signs:
- The page source has almost no words in it, even though the page looks full
- Your prices, hours or service areas appear only in pictures
- A pop-up must be dismissed before anything loads
The fix is usually structural. The important words should be in the page itself, in plain text, in the first response from the server.
Why do broken and redirected addresses make a site harder to read?
Every page on your site has an address. Over the years, addresses change. Sites get rebuilt, platforms get swapped, and old addresses get sent somewhere else, or nowhere.
Done carelessly, this leaves a mess. Old addresses that all land on the homepage. Chains of redirects that bounce a visitor through three stops. Pages that still get listed but now return an error.
A person gives up on those. A machine does too.
A New Leaf Painting Contractors, a Jacksonville company that has painted since 2010, hit exactly this after a platform move. Hundreds of old addresses had been redirected to the homepage, and pages that search engines still listed were gone.
When we rebuilt the site, we traced every address it had ever published, back through its older platforms. Each one now answers in a single hop, either to the page that replaced it or to a clean "gone" for thin, near-identical pages that should never have existed.
That project was about search engines and customers. We are not claiming it changed what any assistant says about the company. The point is narrower. A site where every address answers cleanly is a site any visitor, human or automated, can read without guessing.
What does unblocking promise, and what doesn't it?
Plainly: it does not promise a recommendation.
Removing a block means an assistant is allowed to read you. It does not mean an assistant will choose you. That depends on whether your pages say clearly what you do, where you do it, and why someone should trust you. It depends on what the rest of the web says about you.
Still, unblocking is where you start, for a simple reason. It is the only cause on the list you can rule out in an afternoon. Everything after it is slower and harder to measure.
Think of it as unlocking the front door of a shop. A customer still has to want what is inside. But a locked door guarantees an empty shop.
What to hand your web developer this week
Skip the vague request. Give them a short, specific list:
- Show us the current robots.txt and explain every rule in it
- Confirm whether our host, firewall or security plugin challenges or blocks automated visitors, and which ones
- Check that the text on our five most important pages appears in the page source without a script running
- List every address that redirects, and confirm each goes to the right page in one step
- Tell us which blocks were deliberate and which are leftovers
If you want a second set of eyes on any of it, our SEO and analytics work starts with exactly this kind of audit. If the problem turns out to be how the site was built, that is a custom web development conversation.
Your next step is smaller. Open yourdomain.com/robots.txt right now and read it. It takes thirty seconds, and it may be the whole answer.