Content Creation

Creating an expert system’s knowledge base is the heart of knowledge engineering. This description covers multiple steps in the content creation process.

This part will cover:

Collecting expert knowledge
Using question-based rules to deliver expert knowledge with machines
The Importance of Answers in Expert Systems
Additional Supports in Expert System Knowledge Bases
Patterns for Modelling Expert Knowledge

Collecting Expert Knowledge

This post introduces the structure and format for meetings with subject matter experts. These meetings become the “live” facilitation sessions used to collect knowledge for the expert system’s database.

Acquiring expert knowledge

Expert knowledge is collected by knowledge engineers through live facilitation sessions.

Knowledge engineers acquire domain knowledge from subject matter experts through a range of facilitation techniques, including:

Random generation: asking subject matter experts to randomly identify types of problems, sub-types, information and outcomesWhat types of problems do you deal with all the time?
Direct description: directly asking subject matter experts to describe the problems, information, potential actions, best outcomes, etc.Tell us about a problem, about the information people need to deal with it, and what the best outcome would be.
Interviews: asking the interviewer a series of questions intended to elicit content for the systemWhat problem have you dealt with the most in the last year? Still on the topic of that problem, what should people know about it right away? What is something you wished people with this problem would do before anything else? What else can this person try? What is a good outcome or conclusion for this problem? What are some other problem types similar to this one?…
Use cases: starting with a use case of a typical non-expert user, and working through from the earliest step of problem identification, through to an outcomeLet’s talk about a user who has the characteristics of people you often help. We’ll call that user Bob. Let’s say Bob has problem X. What is something you think Bob needs to know right away? Will Bob likely have any misunderstandings or mistaken beliefs that you would want to dispel right away? What action would you want Bob to take?…
Domain division: bisecting the domain, often by an actor’s role any number of times into manageable sections or sub-domainsIf we’re talking about the consumer law domain, let’s start by dividing it up between buyers and sellers. Let’s start by dealing with the buyers. Now, let’s focus only on buyers of goods, rather than services. Now let’s narrow it down to people who bought goods online…
Branching: adding branches to existing logic to move into more specific or granular contentLet’s add a problem type to the branch we already started, then add multiple problem sub-types, then add sub-types to those…

Any combination of these approaches can be used in the same project.

Capturing live facilitation

The knowledge engineering team must capture as much of the subject matter expert’s knowledge as they can during a live facilitation session.

The knowledge can be captured in a rough format. Shorthand and abbreviations are fine. A series of notes or bullets can stand in for a first pass on longer text-based documents.

Because the subject matter expert is will likely have limited time (and brainpower) available, the rough work approach to capturing live facilitation sessions will make the most of it.

Knowledge content elaboration

Between live facilitation sessions with the subject matter expert, the content can be revised and reworked. These efforts include basic things like:

turning shorthand into more complete text
converting bullets and random notes into more complete informational resources
completing any steps spelled out by notes the team took for itself during the facilitation session

More advanced revision and rework can include things like:

moving concepts and facts into a question and answer format
mocking up new logic branches that appear to be similar, but slightly different, than others in the area of the sub-domain, for future validation with the subject matter expert
experimenting with new logic structures to improve the flow or accuracy of the expert content

Work breakdown

Live facilitation sessions can be scheduled for any length and frequency. In my experience, a 1.5 hour or 2 hour long session is a good target. Any shorter, and the team may not be able to achieve a good flow of work. Any longer, and the team may get tired under the heavy cognitive load of the work.

A maximum of 1.5 or 2 hours of live facilitation will also limit the amount of time the team can spend in a single session struggling with a particularly challenging problem.

For every hour of live facilitation, the elaboration process should take at least 1 to 3 hours longer. Teams who are more productive with this phase of work will make better use of their subject matter experts’ limited time.

Using Question-Based Rules to Deliver Expert Knowledge with Machines

An expert system built on logic-based rules needs input from to make it work. Questions can elicit this input from users.

Adding action to conditional logic

The rules in the expert system’s knowledge base depend on a conditional “if…,then…” or “if X, then Y” structure. The “if X” represents the condition and the “then Y” represents a resulting action. This structure can ultimately be stated to be “if (condition), then (action).

The expert system is designed to collect input from the user by asking questions about the condition in his or her circumstances. By answering the questions, the user provides the input the expert system needs to take the next action. These actions can include reasoning, providing information or directing users to other tools and actions.

Here are some examples:

If (user indicates (s)he wants dessert), then (the system will show a list of dessert choices).
If (the user indicates (s)he suffered a personal injury), then (the system will ask whether the injury was the result of a motor vehicle accident).
If (the user confirms the victim has suffered a poisonous snake bite), then (the system will display a short video explaining how to clean and dress the wound).

These conditional statements make it possible for a non-expert user to interact with the expert database.

User-focused questions

Conditional statements aren’t very interactive on their own. They can be turned into much more intuitive questions.

Expert system users aren’t experts in the domain. The questions have to be easy for non-experts to answer. It’s up to the knowledge engineering team to construct them in a way that walks them through the knowledge base the captured expert knowledge in the domain period

Users will usually be able to answer questions about their circumstances or about other facts that are available to them or discoverable by them, with a little help.

Binary vs multiple choice structures

Using one of the examples above, a conditional statement can be turned into questions into a dichotomous or “Yes / No” format:

Do you want dessert?

[ ] Yes

[ ] No

Yes / no questions can be very simple for users to answer. However they can only engage two possible rule outcomes and require the system to ask more questions. Consider if the user answers the dessert question in the negative. Then you might have to ask:

Do you want a main course?

[ ] Yes

[X] No

And:

Do you want an appetizer?

[ ] Yes

[ ] No

And so on.

Now consider questions that engage more answers and more rule outcomes:

What do you want to eat?

[ ] an appetizer

[ ] a main course

[ ] dessert

Depending on the expert system, the user may be required to pick only one answer at a time in a multiple choice structure, or as many as apply to their facts or circumstances.

These question structures can be a little more complicated to answer, but the gains are often worth it in terms of reducing the number of questions and rules chained in a linear sequence.

The Importance of Answers in Expert Systems

Expert systems use logic-based rules to make conclusions in a given domain. Offering these conclusions in the form of ‘answers’ to questions makes it easy for users to navigate through the system’s knowledge base without having to worry about rules and logic. Answers are also the key to an expert system’s actions.

Answers into actions

Answers do 2 important things for an expert system.

First, they enable the system to reason about a problem in the domain. In turn, this reasoning takes the form of output provided to users in response to input.

Second, they help the system take action – to make the reasoning happen. In a rule-based expert system, these answers cause rules to “fire”. It’s the firing of these rules that makes the system do things.

As explained earlier, the answers represent the consequent to the antecedent provided in the form of a question. They are modeled in a conditional “If…, then…” structure.

Let’s do a thought experiment.

Imagine you’re an expert firefighter. You’re standing at the end of a long hallway next to a cabinet that contains a fire hose and a first aid kit.
Suddenly you hear someone shouting “Help! Help!”
You cup your hands to your mouth and shout “What’s wrong?”
The other person says “There’s a fire burning here!”
You shout “Is anyone hurt?”
The other person shouts “No!”
You grab the fire hose next to you.
You grab the hose, leave the first aid kit, and run down the hall.
You get to where the other person’s standing and see there are 3 black doors and 3 white doors.
You ask the person “Behind which colour door is the fire?”
The other person says “It’s behind one of the white doors”.
You ask “Which door? The 1st, 2nd or 3rd?”
The other person say’s “It’s the 3rd”.
You turn on the hose and rush through the 3rd door with the hose…

This sequence uses questions to gather input from the person who needs help. You’re the expert who needs input from this person before you can help. The answers help you decide that it’s a type of problem you can help with. They also help you decide which tools to bring, and where to take them.

If you’re actually a robot firefighter, your actions in our thought experiment could be based on a set of rules. Each answer to your question caused you to ask another question – and to take some other actions. But it was the user’s input in the form of answers that caused your robot firefighter rules to fire for each action.

The rules in your system that cause you to take action might look like this:

Restricting answers

With a few exceptions, rules will likely be restricted to a certain set of answers – like multiple choice. The system needs some way to know what to do.

It’s up to knowledge engineers to build the rules, based on input from subject matter experts in the domain covered by the system’s knowledge base.

Additional Supports in Expert System Knowledge Bases

In addition to the conditional rules that turn a knowledge base into an expert system capable of providing reasoning and guidance, it’s possible to provide other things. These other elements are also in the knowledge base, and are meant to benefit users.

Information

There are a few ways to provide information to users, depending on the system. For explanation purposes, it’s easiest to divide it into “inline” information and “summary” information.

Inline information is delivered to users while they are working their way through the logic-based rules or branches of the system. This form of delivery breaks information into bite-sized chunks that don’t overwhelm users. It can also allow the system to inform or educate users about the domain to put them in a better question to answer subsequent questions or perform actions necessary to keep using the system.

It might look like this:

Question -> Answer -> Information -> Question -> Answer -> Information ->…

Summary information is delivered to users once they are finished. A question & answer approach might look like this:

Question -> Answer -> Question -> Answer -> Question – Answer -> Information

Logic rules that have a series of questions chained together all firing in sequence will likely have information at the end:

Fact 1 + Fact 2 + Fact 3 + Fact 4 -> Conclusion A -> Conclusion B -> Conclusion C -> Information about the conclusion

Tools

Tools are really action-oriented resources users can use to work toward a certain goal or objective. They are often self-help oriented.

For example, imagine the expert system is meant to emulate the help a human expert might provide to help a user determine their monthly disposable income. A specialized calculator where users input income and expenses could calculate the disposable income amount.

Off ramps

Because expert systems can’t help every user, it might be necessary to quickly move some people out of the system and into the hands of another expert or service provider. These avenues can be called “off-ramps” because they move the user out of the system instead of letting them continue.

A good off ramp will be as seamless as possible for users. Optimally, it will become a “warm hand-off” where the other service provider is alerted to and prepared for the user, and where the user is prepared to receive the service.

FAQs and help text

Users may need some help to figure out the interface or domain-related issues while they are using the system. Frequently asked questions and other forms of help text will support users with these challenges.

Feedback collection points

Some systems may allow users to provide feedback about their experience using the system. This feedback can cover things including:

satisfaction with resources
usability of the interface
identification of errors or things that appear to be missing
things that are unclear
general suggestions for improvement

System feedback collected directly from users can be used to support a continuous improvement process. The more the expert system gets used, the better it gets.

Patterns for Modelling Expert Knowledge

There are different ways to model or structure the expert knowledge in a technology-based system. Some of the approaches are actually quite complicated and theoretical. At least they seem that way to me.

The approach I take to knowledge engineering is what I believe to be quite a simple approach. In my view, knowledge engineering for an expert system that actually “ships” as opposed to spending its life in a research or development state, is already hard enough without adding complex structure to the knowledge base. That’s one of the reasons why I prefer a very simple, rule-by-rule approach.

Even simple rules will form a general pattern or approach in the logic tree can become detailed and quite sophisticated as a whole.

High level pattern

Let’s start with a high level description of a common pattern I use:

Diagnose the problem
Give information about the problem
Narrow the problem (deeper diagnosis)
Give more information about that narrowed problem
Repeat #3 and #4 as required
Optional: provide self-help guidance or tools
Optional: provide more than one option for an outcome, whether it’s a conclusion or guidance
End the process

In more natural language, this approach moves from general to specific. Each branch of the logic tree leads to the most specific conclusion or outcome possible, to the degree or granularity that knowledge engineers are able to create.

Here’s an alternative way to look at this pattern:

Identify the problem the expert system has to solve or the question it has to answer
Apply factual knowledge from the domain (that’s available to everyone, even if they haven’t looked it up) and heuristic knowledge (judgement of experts in the domain) to the problem or question
End the process

Reuse and Recycle

Knowledge engineers may notice a particular pattern or set of outcomes crystallizing within the domain. They may see similar things happening again and again. This is a sign that the pattern may work across the domain.

Don’t force it

The goal of knowledge engineering is to capture and formalize or structure knowledge from domain experts and deliver it to non-experts through an expert system. The domain, subject matter experts and users all come first. Knowledge engineers may find it difficult to take a pre-set pattern and force the domain into it.*

In other words, a pattern shouldn’t take precedent over the domain or what users need from your expert system.

*I’m working on ways to make this happen. It could make knowledge engineering much faster, simpler and more predictable.

Back to Table of Contents