Almost every company has the same problem lying around, just under different names: the quality manual on the intranet, the vacation policy on the HR department's drive, the project documentation in SharePoint, the work instruction as a PDF in an email attachment from 2021. The knowledge is there. But whoever has a specific question has to search. And whoever doesn't find anything asks the one colleague who always knows, or decides by gut feeling.
An internal knowledge system turns this around: instead of searching, you ask a question in plain language, and the system answers, with a citation, from your own documents. We build systems like this, and in this article we explain how they work technically, what really matters, and where the pitfalls lie. No black magic involved, because there is none here.
The documents stay where they are
Let's clear up a common misconception first: a knowledge system does not require you to first move all your documents around, restructure them, or convert them to a new format. That would be a project that never finishes, and it is exactly what has sunk many intranet initiatives.
Instead, the content gets indexed. The system reads the existing sources where they live: file server, SharePoint, wiki, document management system. Each document is broken into meaningful sections, such as chapters or groups of paragraphs, and for each section a mathematical representation of its meaning is computed and stored in an index. That sounds more abstract than it is: think of it as a very good table of contents that doesn't just know words, but understands what a section is about. "Special leave", "paid time off", and "day off for moving house" end up close together in the index, even though the words are different.
Importantly, the index is a copy for search purposes, not the new source of truth. The original document remains authoritative. When a manual is updated, the index follows. The familiar filing structure stays in place, and nobody has to change how they work for the system to function.
How a question becomes an answer
When someone asks a question, two things happen in sequence. First: the system searches the index for the sections that match the question, typically a handful of passages from different documents. Second: a language model is presented with exactly these passages together with the question and formulates an answer from them. In the technical literature, this pattern has its own name, but that is irrelevant for understanding it. What matters is the division of labor: the search delivers the facts, the language model delivers the wording.
This separation is the heart of the whole thing. The language model is not trained on your documents and is not supposed to answer from its general knowledge either. It works like a clerk who is handed three pages from a file and told: answer the question using only these pages. If the answer is not on the pages, the correct response is not guessing, but the sentence: "I can't find anything on this in the sources."
Every answer carries its citation
Here we have to be honest: language models can produce convincing-sounding statements that are simply wrong. This is not a teething problem that will soon disappear, but a property of the technology. A system that provides information about policies and procedures within a company cannot afford that. A wrong but confidently worded answer about workplace safety is worse than no answer at all.
That is why we apply a hard rule to systems like this: what is not in the sources is not claimed. And every statement in the answer points to the passage it comes from, not vaguely to "the manual", but to the specific section: document, chapter, ideally with one click to the original. Anyone who doesn't trust the answer can verify it in ten seconds. Without this verifiability, such a system does not deserve trust.
Technically, this means several things at once: the language model is strictly instructed to answer exclusively from the presented sections. The answer is checked against the source sections. And if the search finds nothing suitable, that is said openly instead of dressing up a thin answer. You can recognize a good knowledge system by how it behaves when it doesn't know something.
The most important rule in one sentence: an internal knowledge system is only as trustworthy as its weakest answer. Section-level citations and an honest "the sources don't cover this" are therefore the precondition for employees taking the system seriously at all.
Permissions apply before the search, not after
Now comes the point that demos like to gloss over and that decides between success and disaster in practice: access rights. In your company, not everyone is allowed to read everything. Salary bands, personnel files, draft contracts, M&A documents, all of this sits behind permissions for good reason. A knowledge system that indexes across all repositories potentially erases these boundaries if you are not careful.
The naive approach would be to search first and then filter the answer. That is not enough. If confidential sections flow into answer generation at all, content can leak, even if the document itself is never displayed. Even the statement "a document on this exists that you are not allowed to see" can reveal more than it should.
That is why, in systems like this, we make sure the permission check applies before retrieval: whoever asks determines which sources are read in the first place. The system adopts the existing access rights from the source systems, and the search runs from the outset only over the sections that the person asking would also be allowed to open in the original. For an employee on the shop floor, the salary lists simply do not exist in this system, not as an answer, not as a source, not as a hint. If the head of HR asks the same question, she sees more, because she is allowed to see more. Same question, different sources, different answers. That is exactly how it has to be.
Where does it all run? In Germany or the EU
A system that processes manuals, HR policies, and project documents does not belong on just any server. For SMEs, in our view, that means: hosting in Germany or the EU, with a clear data processing agreement, and this applies to all components, including the language model. Solid options now exist for this, from European cloud offerings to models that run on your own infrastructure. Which variant fits depends on your data situation, budget, and IT landscape. What matters is that the decision is made deliberately, before the first document is indexed.
What changes in everyday work
What does this feel like in daily operations? Take an unspectacular, everyday question: "How many days in advance do I have to request special leave, from whom, and which rule applies to my department?" Today, this question means: search the intranet, pick the right one of three similarly named PDFs, discover that the works agreement contains a special rule for production, and, when in doubt, call HR anyway.
With a knowledge system, the person types the question exactly as they would ask a colleague. The answer states the deadline, the contact person, and the rule that applies to their department, with a reference to the relevant section in the works agreement and the personnel manual. Two clicks, and they are reading the original passage. HR no longer answers the same question twenty times a month; instead, it maintains the one document that all the answers come from.
The pattern repeats wherever knowledge is documented but hard to find: onboarding new employees, audit preparation in quality management, the question of which version of a work instruction is current. The system makes nobody redundant. It finally makes the documents your company writes and maintains anyway queryable.
Whether this is worthwhile for your company depends less on the technology than on the state of your documents: what is documented, where it lives, how current it is, and how cleanly the permissions are maintained. If you want to know whether your materials are suitable for this, let's take a look together. Often, an honest look at what you have is already half the answer.
