The people who make the machines smart: anatomy of a knowledge engineering team

It takes a lot of humanity to make machines intelligent. At least, that’s true for expert systems.

An expert system with an empty knowledge base is actually quite dumb. It depends completely on people to collect its knowledge, put it in the right form, load it up and then look after its ongoing care and feeding.

This rough list describes some specialized team roles  who make it happen.

1. Subject matter expert:

The ‘SME’ shares his or her expert knowledge in the domain addressed by the expert system. You need to find a really good expert and upload their mind into the system.

2. Knowledge engineer:

This person does the things described here. Essentially, the knowledge engineer facilitates the acquisition of expert knowledge from the subject matter expert in a form that can be used by non-experts through the expert system.

3. Knowledge content specialist:

This team member collects expert knowledge as it comes spilling out of the SME. A good knowledge creation specialist won’t waste a single drop of it. The knowledge must be collected and packaged in the right format to eventually be used by the expert system.

4. Knowledge refinement specialist:

Before expert knowledge is loaded into the system, it has to be refined for everything from style, grammar or punctuation to the exact layout. It may need special tags, file names and other identifiers in hundreds or thousands of places.

5. Knowledge approver:

In domains like law, it’s critical for every bit of expert knowledge to be reviewed and approved for things like legal accuracy before it’s loaded into the system. Omissions can be just as important as errors if the expert system is built for non-expert users.

6. Knowledge loader: 

Once the knowledge is ready and approved, this team member has to put it into the system. We call this an ETL or extract – transform – load process. First, we extract all the useful data from the knowledge content. Next, transform or convert it into the format the expert system can use. Finally, it’s loaded into the system.

7. Quality assurance testing lead:

When the knowledge is first loaded into the knowledge base, it has to be tested before anything else happens. This team member coordinates a group of testers (usually project ‘insiders’) to do this first round of review. Any initial bugs or errors go back to the loader to be corrected.

8. User testing lead: 

Once early testing is complete, the circle of testers will gradually expand to include more and more testing with people who are less and less familiar with the project or the knowledge domain. The user testing lead collects and compiles feedback from this testing and presents it to the team who will identify necessary changes and improvements to the knowledge base.

9. Continuous improvement specialist:

You can’t just load the knowledge into the system and consider it “done”.  A well designed system will have ongoing sources of user feedback to support this process. Analytics and other data collected from use of the system must also be reviewed and analyzed.  The goal is to keep improving the system over time. The more it gets used, the smarter it gets.

10. Knowledge base curator:

Have you ever been sure you had an email with some important information in it but just couldn’t seem to find it when you need it? Or how about that thing where you’re editing a document with a couple of other people and completely lose track of the latest version with the best changes? Expert system knowledge bases can contain tens or hundreds of thousands of data objects. What they don’t need are version control problems and confusion. What we do need is a knowledge base curator to keep it all straight.

11. Subject matter expert networks:

Want to keep your knowledge base from going stale? Keep it fresh by creating networks of subject matter experts who periodically report on new developments or insights in the domain (e.g. new legislation and court decisions in the legal domain). These experts can also continually provide new knowledge that can be engineered and loaded into the database to make it deeper or wider.

Smart machines start with smart teams

I’ve never actually worked on a knowledge engineering project where each one of these roles was held by a different person. In many cases, people will look after multiple roles. But the tasks are all quite important, and the demands can be surprisingly high.

Want to create a smart machine? It helps to have a good team.


Comments are closed.

Proudly powered by WordPress | Theme: Baskerville 2 by Anders Noren.

Up ↑