Before knowledge engineering moves into the content creation phase, it’s important to do a tactical analysis. This work involves:
- Problem identification
- Domain identification
- Subject matter experts
- Defining problems and responses
- General scope determination
Problem Identification
There’s no real methodology involved with identifying a problem. Problems just happen.
The knowledge engineer’s job is to examine the problem and ask 3 key questions:
- Is this a problem for non-experts that wouldn’t be a problem for experts?
- Is there a relatively uniform knowledge domain here?
- Would expert reasoning or guidance help non-expert users with the problem?
First, an expert system isn’t the right tool for every problem. It helps to put expert knowledge and guidance in the hands of non-expert users. So if the problem is affecting the world’s top experts in a domain, an expert system may not help. If, on the other hand, experts don’t suffer from the same problem as non-experts, an expert system might be an effective response to the problem.
Second, knowledge engineers need to tap into expert knowledge. Experts usually have heightened levels of knowledge and expertise in a given domain. Polymaths aside, expertise will be specialized, at least in contrast with things outside the expert’s domain. In addition, the knowledge engineer will be looking to extract recognizable and repeatable patterns of reasoning within a relatively uniform domain. The knowledge base will have boundaries to it. So too must the domain.
Third, the problem has to be one that can be eliminated or mitigated for non-experts if they receive expert reasoning and guidance. An expert system can become a tool for disintermediation if it removes the middle-man standing between the expert knowledge and the non-expert who needs it but can’t access it because of resource problems such as a lack of experts or a lack of resources to pay for experts. But if users would be better off with access to this knowledge or expertise, an expert system could be an appropriate tool.
Positive answers to these 3 questions won’t guarantee success. But they will get a knowledge engineer off to a good start.
Domain Identification
A knowledge engineer’s job is to mine domain knowledge from human subject matter experts. The domain knowledge is the knowledge in a particular field of expertise.
The Ins and Outs of Domains
The domain can’t just be “everything” or “all knowledge.” It would be too difficult to cover everything in the knowledge base. You don’t have to depend only on one subject matter expert; you could keep using more and more to try and cover “all knowledge.” But even if you had the time and the resources to capture all this knowledge, it would also be impractical to connect users with particular problems to the right reasoning and guidance offered by the system if it covered all domains.
For example, let’s assume you’re trying to help a user who needs to make a magical potion. There’s a potential domain here: magic (or in J.K Rowling’s universe, witchcraft and wizardry). But this domain is still too broad, at least for the purposes of our expert system and the problem it seeks to solve.
So let’s narrow it down to subjects you can take at the Hogwarts School of Witchcraft and Wizardry. (I’m willing to bet this example will be easier to understand and remember than the one I’ll include after it)
To solve the problem for our non-expert who needs to make a magical potion, we can call “potions” our domain. Our knowledge engineer is the one who has the domain knowledge.
A less whimsical example could involve a non-expert who needs guidance to obtain a grant of probate for an estate. “Law” is a potential domain, but it’s probably too broad to be helpful for our problem. Law can be broken into subjects like criminal prosecution, real estate, intellectual property and so on. The specialized subject of “probate & estates” will be the right domain for our problem.
Subject matter experts
The knowledge engineering process will depend on the availability of expert knowledge. At least one subject matter expert with specialized domain knowledge must be available and willing to participate in a potentially lengthy process to construct the expert system’s knowledge base.
What makes a good subject matter expert?
By now, it should be clear that a good subject matter expert has a high level of domain knowledge. If you can get the top subject matter experts in the world in a particular domain, you should get them.
An expert also has contact with lots of non-experts in his or her domain should be able to share or appreciate these perspectives – and help to craft a knowledge base that’s easier for non-experts to use.
Subject matter experts aren’t knowledge engineers
Your subject expert doesn’t have to know how knowledge engineering works. The knowledge engineer facilitates the process according to the methodology. By the end of the process, experts will often gain considerable familiarity with the process, and make suggestions for improvements.
Knowledge engineers aren’t researchers
A lack of strong subject matter experts will frustrate the knowledge engineering process. Knowledge engineers cannot and should not be researching the domain on their own. The objective is not to repackage information that’s already available to non-experts (even though this can help). True success depends on making expert knowledge directly available to users through the expert system.
Knowledge engineers attempting to research the domain will miss important nuances that only a true subject matter expert will appreciate. This problem is particularly significant when it comes to the subject matter expert acting as a proxy for non-experts, describing how they see the problem, what their limitations may be, common myths and misconceptions, and what shape a good outcome or result will take for them.
Knowledge engineering can be time consuming to begin with. Requiring knowledge engineers to research the domain during the process can make the process unbearably long, in addition to making it less successful.
Drawing on multiple experts
Expert system knowledge bases can contain knowledge collected from multiple sources. Knowledge engineers can draw on the domain knowledge from one, two or several subject matter experts.
It’s not up to a single subject matter expert to know all the knowledge in the domain. Knowledge engineers can draw on several people to “fill in the blanks” in the knowledge base.
Reliance on multiple experts can also help to provide more balanced content in a knowledge base where users might be in adversarial problems and circumstances. For example, a knowledge base designed to help people with consumer debts can include contributions from lender-oriented experts, as well as borrowers, debt counselors, poverty advocates and more.
Collective intelligence can exist in an expert system’s knowledge base. It just needs to be sourced and created through the knowledge engineering process.
Defining Problems and Responses
Expert systems can respond to real world problems. It’s important for knowledge engineers to decide on the right problem before work begins. If subject matters have been engaged by this point, they can help with this analysis too.
Defining the problem and the response
Earlier, we talked about 3 key questions a knowledge engineer should consider:
- Is this a problem for non-experts that wouldn’t be a problem for experts?
- Is there a relatively uniform knowledge domain here?
- Would expert reasoning or guidance help non-expert users with the problem?
A next step is defining the problem and a proposed response. The problem definition will help knowledge engineers understand what they are working to solve and who it’s affecting. The response refers to the proposed outcomes for users.
The problem definition & response can serve as vision statement to communicate the objectives of the knowledge engineering work. It also keeps the work on track. Because knowledge engineering can take a long time, people might forget what they set out to do in the first place.
It’s easy to get bogged down on very small, but very difficult issues in knowledge engineering. A clear focus on the problem you meant to solve and the response you intended to provide can serve as a reflection point and problem solving tool if this happens. And it will happen.
Elements of a problem definition & response can include:
1. A one sentence description of the problem, identifying the people who are suffering as because of it:
Problem: People with fitness club membership contract disputes aren’t aware of the special rights they have under consumer laws and regulations.
2. A one sentence description of the positive impact or outcome that will be provided through the expert system:
Response: The expert system will diagnose fitness club membership contract disputes, provide information and guidance to consumers, and empower them to exercise their specific legal rights to achieve better outcomes.
Response & Outcome types
Expert systems can provide several types of responses, including any combination of the following:
- Diagnosing different types of problems to various degrees of specificity or granularity
- Providing specific information about a diagnosed problem
- Providing self-help tools and steps to address the problem
- Guidance about a user’s options for dealing with the problem and how to access them
Problems that can be addressed by these types of responses will make good candidates for expert systems. Remember: expert systems can’t pick up your dry cleaning or provide answers beyond the grasp of human experts.
General Scope Determination
While work is under way to define the problem and potential responses, knowledge engineers must work with subject matter experts to define and manage scope. This effort will be relatively general since it is difficult precisely define scope at such an early stage of work.
An expert system can solve multiple problems within a domain, but it’s important to avoid taking on too many challenges at once. If multiple problems are within the defined scope of work, they can be parceled up and knowledge engineered discretely.
Managing scope of responses
Scope must also be managed with respect to the potential responses to the problems. Knowledge engineers should manage expectations by being specific about what is in and out of scope with respect to responses.
The over-promise | under-deliver dynamic
The temptation to over-promise in terms of scope of problems and responses will be the strongest when approval is being sought for a project, or when a proposal is being discussed with stakeholders. Knowledge engineers may not fully appreciate the importance of a manageable scope before they actually begin working in the domain. Many stakeholders won’t ever really appreciate issues around scope – especially if they’re the type to focus primarily on the negative aspects of new projects.
In my experience, the final deliverable will often be slightly less ambitious than the vision shared by the team before knowledge engineering is under way.
Staging knowledge base development
It’s perfectly acceptable, and surprisingly practical, to plan on an expanded scope within a single domain in a “version 2” of the knowledge engineering work. After version 1 is loaded into the system, knowledge engineering can start on subsequent versions of content that will be added to the knowledge base in the future.
Back to Table of Contents