Can AI answer staff questions from our own policies? Yes, if every answer shows its source and a person owns the documents.
Short answer. Yes, when the AI is limited to your approved policy documents and every answer shows the passage it came from. Without that, a model answers from memory and sounds equally confident when it is wrong. In February 2024 a British Columbia tribunal ordered Air Canada to pay a passenger after its website chatbot gave wrong refund information, and rejected the argument that the chatbot was a separate entity: the airline is responsible for all the information on its website. An office bot is no different, because staff will act on what it says. Keep policies versioned, make not in the documents an acceptable answer, and send unclear cases to a named person.
01
How it works
The method is retrieval-augmented generation, first described by Lewis and colleagues in 2020. The policy documents are split into passages and indexed. When a staff member asks a question, the system retrieves the closest passages, hands them to the language model, and instructs it to answer only from them, naming the passage. The model supplies the wording; the documents supply the facts.
A source line shown with the answer lets a reader check the claim in seconds. It does not make the answer right, as the next section shows.
02
Why a cited source is not a guarantee
A source line shown beside an answer lets a reader check it in seconds. Published studies show that the check is needed, because the source is not always what the answer says it is.
Study
What it found
What it means for a policy bot
Magesh et al., 2024, retrieval-based legal research tools
Retrieval-based legal research tools from LexisNexis and Thomson Reuters each hallucinated between 17% and 33% of the time.
Retrieval reduces invented answers but does not remove them.
Liu, Zhang and Liang, 2023, four generative search engines
On average 51.5% of sentences were fully supported by their citations, and 74.5% of citations supported their sentence.
A citation can be present and still not back the sentence beside it.
Gao et al., 2023, the ALCE benchmark
On the ELI5 questions even the best models lacked complete citation support 50% of the time.
Check that the clause quoted says what the answer claims.
03
Who answers for a wrong answer
The Air Canada decision (Moffatt v. Air Canada, 2024 BCCRT 149, issued 14 February 2024) concerned a chatbot that told a passenger they could apply for a bereavement fare after travelling, within 90 days. The airline’s own policy page said otherwise. The tribunal found that Air Canada had not taken reasonable care to ensure its chatbot was accurate, and ordered it to pay C$812.02 in total. It said it should be obvious that a company is responsible for all the information on its website, whether it comes from a static page or a chatbot. That decision concerned a consumer, not staff, and this page is general information, not legal advice.
04
What a person must still check
Who owns each policy document, and when it was last reviewed.
Which version applies. Superseded policies must be removed from the index or clearly dated.
Exceptions and approvals. The policy may say ask HR for cases that the bot should never decide.
Matters involving a named person, such as a disciplinary case or a leave dispute. Route them to a human.
Answers where two documents disagree. The bot should say so, not pick one.
05
How an office would set this up
List the 50 questions staff really ask HR, IT and accounts. Take them from the inbox, not from imagination.
Gather the current policy documents, retire old versions and add a date to each.
Index them and instruct the model: answer only from the passages, quote the clause, and otherwise say it is not in the documents and name who to ask.
Test with the 50 questions plus 20 whose answers are not in the documents. The bot must decline the 20.
Start with a pilot group and log every question and every thumbs-down.
Review the log weekly and fix the documents rather than the prompt wherever you can.
Show the document name and its last-updated date beside every answer.
06
What ANVE does today
ANVE’s documents function answers questions over the office’s own documents and shows the source passage. Private GPTs are assistants built on the office’s own material. A policy helpdesk is one of the agents that can be built with a paid pilot; it is not a shipped product, and the pilot fee is agreed in writing before installation.
The documents function and Private GPTs run today in the development build. No sealed unit exists yet.
07
Questions people ask
Who is responsible if the bot gets a policy wrong?
Assume the office. In the Air Canada case the tribunal held the company responsible for what its chatbot said. The position for a staff-facing tool can differ, so take legal advice; this is general information.
What should the bot say when the answer is not in the documents?
Something like: this is not in the policy documents, please ask the named person. Test that it really does this, using questions you know are not covered.
Can staff trust the clause the bot quotes?
Treat it as a pointer. Studies of citation quality find that a citation often does not fully support the sentence beside it, so staff should open the clause for anything that matters.
How often should the documents be refreshed?
Whenever a policy changes. Give each policy an owner and make re-indexing part of the approval step.
08
What is not real yet
No sealed ANVE unit exists; every image of the appliance on this site is a labelled render. The software runs today on a development machine. No customers yet, no certifications, no delivery date, and the final price is not announced.
Reservations are ₹999 in India or US$12 elsewhere, refundable in full until you confirm an order in writing (terms). Offices that cannot use cloud AI can apply to be one of ten design partners.
