Hallucinated features: how to spot and correct false AI statements about your software

Why AI likes to hallucinate about SaaS of all things

Software products are a special case: your feature list changes every two weeks, your pricing is revised quarterly, and integrations come and go. AI models, however, were trained on a data state from months or years ago. When a user today asks whether your tool offers an API or SSO, the model answers from a frozen state of knowledge - or it guesses. And with language models, guessing looks like a factual statement.

On top of that: SaaS categories are filled with similar products. A model that doesn't know your project management tool exactly fills the gap with attributes that are common at Asana, Trello or Jira. This is how plausible-sounding false statements arise. It claims Gantt charts, time tracking or Zapier support, because that is standard at competitors - not because your product can do it.

The tricky part: the answer sounds competent. No question mark, no subjunctive. The potential customer adopts the statement as truth and makes a purchase or rejection decision on this basis, without you ever learning about it.

The four hallucination types that cost you money

First the invented function: the model claims a feature you don't have. Sounds harmless, but leads to disappointed trials and bad reviews, because users start with wrong expectations. Second the denied function: the model says you lack something you long ago built. Here you lose deals right at the start of the funnel, without ever making the shortlist.

Third the price hallucination: the model names outdated tariffs, invented per-user prices or claims a free plan you abolished. This creates friction in the sales conversation and undermines trust when the real bill looks different. Fourth the integration lie: the model claims native connections to Salesforce, HubSpot or Slack that don't exist - or conceals the ones you have.

All four types have in common that they arise during purchase research. B2B software buyers today use ChatGPT and Perplexity the way they used to use Google, to build shortlists. Whoever is misrepresented there falls out of the shortlist before a human ever sees your website.

How to test systematically what the AI says about you

Start with a prompt list that maps real purchase questions. For a SaaS product these are typically: "Does [product] have an open API?", "Does [product] offer SSO and SCIM?", "What does [product] cost for 50 users?", "Which integrations does [product] support?", "Is [product] GDPR-compliant and where are the servers?". Write down 15 to 25 such questions that your target group really asks.

Then ask each question in several systems: ChatGPT, Gemini, Perplexity, Claude and Microsoft Copilot. It is important to do this without logged-in personalization, so that you see what a stranger without context gets. Document each answer verbatim in a table with date, model and the concrete statement. Only this way do you recognize patterns and can later measure whether your corrections take effect.

Repeat this test at fixed intervals, roughly monthly, and always after a larger release. Models are updated, answers shift. A feature that is correctly named today can disappear again after the next model update. GEO is not a project with an end date, but ongoing monitoring.

Where the false statement comes from - and why that matters

Before you correct, you have to understand the source. Broadly there are three causes. The training data is outdated: the model only knows your state from before the cut-off. That cannot be changed directly, but it can be influenced via retrieval systems like Perplexity or ChatGPT Search, which search the web live.

The second cause is contradictory or thin sources. When your website is vaguely worded, an old blog article names outdated prices and a comparison portal lists wrong features, the model picks some one of these versions. The clearer and more machine-readable your own facts are, the more readily your version wins. The third cause is pure confabulation: the model has no source and guesses on the basis of the category.

Check the sources given by Perplexity and ChatGPT Search. Often you see there directly that an outdated G2 page, an old Reddit thread or a competitor comparison is being cited. This concrete source is your point of attack, because you can influence it or override it with better signals.

Making your website machine-readable

Language models and their retrieval systems love unambiguous, structured statements. Build a feature page that contains one clear sentence per function: "[Product] offers a REST API with OAuth 2.0 authentication." Avoid marketing fog like "boundless possibilities". Add a public, current integrations overview as a list, ideally with one sentence per integration that a model can adopt verbatim.

Use structured data. A SoftwareApplication schema with a feature list, price and operating system helps crawling systems extract your facts cleanly. A well-maintained FAQ section that answers exactly the purchase questions from your prompt test is especially effective, because its question-answer structure corresponds exactly to what models like to cite.

Keep a public changelog and pricing page under a stable URL. When your prices lie only behind a "contact sales" button, the model has nothing concrete and invents numbers. A transparent, dated pricing section gives it a reliable source and noticeably reduces price hallucinations.

Correcting external signals, not just your own page

Models often trust third-party sources more than your self-presentation. That is why you have to maintain the places where others write about you. Update your profile on G2, Capterra and Product Hunt with correct features and prices. These portals are frequently cited and influence the AI perception disproportionately strongly.

Pay attention to communities. Reddit threads, Hacker News discussions and Stack Overflow answers flow into training data and live retrieval. You can't manipulate there, but you can correct false statements factually and transparently as an employee. A single corrected thread often has more effect than ten changes to your own landing page.

Comparison and "alternative to" articles are another lever. When your product is wrongly classified in a widely read comparison, contact the author with evidence. Many gladly update, because correct data also supports their credibility. This way you correct the source instead of just the symptom.

Making success measurable and catching regressions

Define a target state per prompt. For "Does [product] have SSO?" the correct answer is a clear yes with the correct standard. Rate each model answer as correct, partly correct or wrong and enter it over time into a curve. This way you see whether your measures take effect and in which model you are still losing.

Reckon with a delay. Changes to training data only take effect with the next model update, which can take months. With retrieval systems like Perplexity you often see effects after just days, as soon as your updated page has been re-crawled. Separate these two worlds in your evaluation, otherwise you draw wrong conclusions about the effect.

Set up lightweight alerting. A monthly automated run of your prompt list, whose results you compare with the previous month, catches regressions. When after a model update it is suddenly claimed again that you lack a core function, you want to know that within days, not when revenue collapses.

A realistic 30-day roadmap

Week one: build your prompt list and run the first full test across all five systems. Document every false statement and categorize it according to the four hallucination types. By the end of the week you have a prioritized list of which errors endanger the most revenue - usually denied core features and wrong prices.

Week two and three: work on the sources. Bring your feature, integrations and pricing pages into an unambiguous, machine-readable state, add schema markup and an FAQ. In parallel update G2, Capterra and other cited third-party sources. Correct the two or three most influential community threads factually.

Week four: second measurement run, at least on the retrieval systems that react quickly. Compare with the initial state, record what has moved, and plan the next monthly cycle. From now on the whole thing is routine: measure, improve sources, measure again. It is exactly this continuity that distinguishes providers who win in AI research from those who remain at the mercy of the chance of the training data.

When support tickets are the first warning signs

Your test routine runs monthly, but the AI sometimes changes its statements overnight, for instance after a model update. Your support team often notices first: a prospect asks about an SSO login you never had, or about an on-premise variant that exists only in the cloud. It is exactly these tickets that are gold dust, because they show which hallucination is currently actively distorting purchase decisions and where your sales team is losing trust.

So set up a lightweight feedback channel. A mandatory field in the ticket tool like 'Where does this expectation come from?' often already suffices. When three customers in a week mention the same invented feature, you have a reliable indication of which false statement should be corrected as a priority, long before your next scheduled test run would even capture it.

Changelog and pricing model: the two most common hallucination sources

With SaaS, knowledge ages especially fast. You deprecate a feature, change your pricing from seat-based to usage-based or drop a free tier, but the AI still knows the state from eighteen months ago. The result: it names prices you no longer charge, or recommends functions that were long ago discontinued. To prospects this feels like a breach of promise as soon as they land in the real product.

So keep a public, dated changelog with clear 'as of date' information and a canonical pricing page without contradictory old versions. Remove or mark outdated blog articles and comparison pages that cement old prices. Every leftover PDF price list on the web is a potential source from which the AI happily cites and overwrites your current truth.

Frequently asked questions about feature monitoring

'How often should I test?' With a stable product, monthly suffices; around releases or pricing changes, rather weekly. 'Is it enough to check only ChatGPT?' No, because Perplexity, Gemini and Copilot draw on other sources and thereby hallucinate differently. Check the assistants your target group really uses, instead of relying on a single one.

'What if the AI stubbornly sticks to the false statement?' Then usually a strong external signal is missing. A single corrected landing page doesn't convince the model when ten third-party sources claim the opposite. Work on G2, Wikipedia, directories and specialist articles until the factual situation in the open web unambiguously supports your version.