Skip to content

Guide

Review AI assistant answers and fill gaps

AI features · 6 min read

The Questions page (previously Hit Logs) shows what members ask your community's AI assistant (AI Copilot) and how well it answered. Use Admin → AI Copilot → Questions to check answers and their sources, see how members rated them, find questions the assistant struggles with, and add the missing answers to your knowledge base.

Who can do this: Owners and admins. · Where: Web (admin console). · Plan: Requires a platform plan that includes AI Copilot.

See how the AI assistant is doing on the Overview page

Admin → AI Copilot → Overview shows whether the AI assistant is working. Questions, Low confidence, and Rated helpful cover the last 7 days. Needs content counts open questions from the last 30 days, and Sources with errors counts knowledge sources whose last sync failed. Click a card to open the list behind it.

What the AI assistant's Questions page records

Go to Admin → AI Copilot → Questions. The page has two tabs: All questions and Needs content (previously Content Gaps).

All questions lists each question members ask the AI assistant:

Column What it shows
Question The member's question. Under it, the start of the answer, or why the assistant couldn't answer
Confidence High, Medium, or Low
Citations How many sources the answer cited
Rating The member's thumbs up or thumbs down, if they rated the answer
Time When it was asked

The list doesn't show which member asked; open a question to see that. AI writing tools, AI course outlines, and Try it tests aren't logged here.

Filter the AI assistant's questions

Use the controls above the table to narrow All questions:

  • Search questions: find words in a question.
  • A date range: Last 7 days, Last 30 days (the default), Last 90 days, or All time.
  • Confidence: Low, Medium, or High.
  • Member's scope: the source scope the member picked: Knowledge Base only, Community posts only, or Knowledge Base + Community.
  • Member rating: Helpful, Not helpful, Rated, or Not rated.
  • A sort order: Newest first or Lowest confidence first.

Click Clear Filters to reset them, and Refresh to load new questions.

Check the details of an AI answer

Click a question to open Question Detail. It shows the confidence, then:

  • Asked by: the member who asked, with a link to their member page, or Deleted account.
  • The member's rating, with any reason and comment they added, or The member didn't rate this answer.
  • Answer: the full answer the member received.
  • Citations: the sources listed under the answer, with a snippet of each.
  • Diagnostics: collapsed by default, for troubleshooting.

Diagnostics shows the Confidence and Coverage scores (how much of the answer is backed by its citations, from 0 to 1), Latency, Mode (Keyword or Semantic search), Knowledge base / community hits, Post-check (the final check on the answer: passed, failed, or rejected_low_confidence), and Member's chosen scope. Retrieved fragments lists everything the AI assistant found while searching, including results it didn't cite. Each fragment has a score, which only ranks fragments against each other; it isn't a percentage.

If a good source was retrieved but not cited, or the wrong post ranks first, adjust your community content settings. See Choose which spaces your AI assistant can search.

Save an AI answer as a Q&A entry

To correct or reuse an answer, click Save as Q&A entry in Question Detail. Add Q&A to Knowledge Base opens with the question and the AI's answer filled in, and Mateflow creates a Q&A knowledge source for you if you don't have one yet. Correct the Answer, then click Add to Knowledge Base.

Fill content gaps with Q&A entries

The Needs content tab lists questions the AI assistant couldn't answer well, so you can see which topics your sources don't cover. Questions that mean the same thing share one row: it shows the most-asked wording, with the others under Also asked as. When there are only a few questions, each wording gets its own row. Each row shows the Question, how often it was asked (Count), Found in (Nothing found, Knowledge base, Community posts, or Both), and when it was Last seen.

To add an answer:

  1. Click Add to KB next to the question. Mateflow creates a Q&A knowledge source for you if you don't have one yet.
  2. In Add Q&A to Knowledge Base, review the Question, and write the Answer. Markdown works. To start from a draft, click Draft with AI, then check it carefully.
  3. Optionally, add Alternative phrasings, one per line.
  4. Click Add to Knowledge Base.

The entry is indexed right away, and the whole row is marked as resolved. To set aside a question you don't want to answer, click Dismiss.

Track handled questions on the Needs content tab

Switch between To do, Resolved, and Dismissed, and choose a date range, to see what's left and what's been handled. Click Reopen on a resolved or dismissed question to move it back to To do. If members keep asking a resolved question, it returns to To do with the badge Still asked after a fix, and its count only includes questions asked since then.

Other ways to improve AI answers

Q&A entries give exact answers to single questions. For wider gaps:

Member ratings are another signal of answer quality, alongside Confidence, Post-check, and Needs content. A thumbs down can include a reason (Not accurate, Didn't answer my question, Out of date, or Something else) and a comment. After you add content, check All questions again to see whether confidence improves.

Related articles

Was this guide helpful?

Back to guides

Can't find what you need?

If you're a member of a community, its admins are the right people to ask. If you're building or running one on Mateflow, our team can help.