| By: | Lars Marius Garshol |
|---|---|
| Affiliation: | Ontopia A/S |
| Date: | $Date: 2001/07/17 13:12:30 $ |
| Version: | 1.1 ($Revision: 1.21 $) |
| |
This whitepaper gives a quick introduction to version 1.1 of the Ontopia Knowledge Suite (OKS), describing the architecture, functionality, and composition of the suite. There is also some discussion of the possible usage areas of the suite.
Basic understanding of the main concepts of topic maps is assumed; for explanations of these, please see the topic map introductions on our web site.
The Ontopia Knowledge Suite is a well-designed suite of products that together enable the development of diverse types of applications. The suite is designed to be integrated into any application to provide it with topic map capabilities. It is stable, performant, and thread-safe, and handles all the difficult aspects of developing with topic maps for you.
In version 1.1 the Suite consists of four products:
This is the heart of the suite: a Java SDK which has all the functionality needed for writing software that works with topic maps. The engine loads, stores and keeps track of the topic maps, and provides interfaces through which applications can access and manage the topic maps. The other products in the OKS are built on the engine and use its functionality.
This web application SDK greatly simplifies the development of topic map web applications. It is based on Java Servlets and Java Server Pages (JSP), and provides a set of tag libraries and Java components that make developing scalable topic map web applications with advanced functionality fast and easy.
This add-on to the engine enables it to store topic maps in relational databases, and to access and modify topic maps stored in such databases. Most RDBMS servers are supported.
This product contains a framework for indexing and searching for text in topic maps. This makes it very easy to do fulltext searches on topic maps, by topic names and the contents of occurrences. A fulltext search engine, integrated with the topic map engine, is provided with the product, but other engines can also be integrated, and the existing integration can easily be modified.
Below is a figure that shows how the Ontopia Knowledge Suite might be used to build a web site based on topic maps, storing the topic map data in a relational database.
Web application built on the OKS
In this example the web application is deployed in a Java 2 Platform, Enterprise Edition (J2EE) web server, together with the Ontopia Engine and Navigator. The topic map is stored in a relational database server, managed by the engine, and the web application is built using the navigator framework.
The Ontopia Knowledge Suite is not an end-user product, but a set of tools useful for building such products. In general, the Suite can be used to advantage in nearly any application that manages information. The problem solved by the Ontopia Knowledge Suite is the perennial problem of information management: the problem of navigating the information ocean to find the knowledge you seek.
We all know how difficult it can be to find the information you are looking for. The reasons for this are many and deep-rooted, and not the least of them is that creating appropriate information structures is in itself difficult. Topic maps, the product of decades of work and experience in information management, provide a model for the organization of information that can make creating appropriate structures much easier. If your information repository is organized with a topic map, structuring it becomes much easier, and so locating the document or piece of information you want also becomes much easier.
An important aspect of topic maps is that they do not require you to replace your information repository with one based on topic maps. Instead, you can maintain a topic map separately from your repository and have the topic map refer into the repository. The topic map can then be navigated to locate the information you need from the repository.
With the Ontopia Knowledge Suite you have what you need to create and manage topic maps for your repositories, as well as to build the interfaces used to navigate the resulting topic maps. Using the Ontopia Knowledge Suite all the difficult problems of topic map management, persistence, concurrency, transactions, and scalability are taken care of for you.
Some of the most important and interesting areas where the Knowledge Suite can simplify development and improve the result are:
Portals, being essentially large information repositories, are perfect use cases for the OKS. Using topic maps for the portal contents you will find yourself guided by the topic map model into creating a well-organized portal. Using the OKS developing the actual portal is greatly simplified, thanks to the Ontopia Topic Map Navigator SDK.
Using topic maps you can capture and manage the knowledge that constitutes the corporate memory in a kind of semantic network that represents people, products, roles, procedures, and other artefacts of your organization; the relationships between these entities; and the links to the documentation that describe them in detail. With the OKS you can enrich your intranet with topic maps and turn it into a real collaborative knowledge base.
Any content management system can benefit from a better and more flexible way of organizing information, which is what topic maps provide. Using the Ontopia Topic Map Engine you can integrate topic mapping capabilities into your content management system. The integration could take the form of making the entire system topic map-based, or it could create a virtual topic map of the system. With the Ontopia Topic Map Navigator you can easily build a web interface to the system.
The topic map data model is designed to allow automated merging of information from diverse sources. Using the Ontopia Topic Map Engine's support for merging, you can easily integrate information from many different sources into a coherent whole. This can be used to integrate the products in a product suite, as well as to finally allow the many separated applications of an enterprise to appear as a single whole. A web interface to this unified information can be developed using the Navigator framework.
The list of possible application areas for this technology is endless, and as topic map technology is still new and fresh exciting new uses for it will undoubtedly continue to be found.
To help you understand more clearly how these products can be used, and how topic maps can be made use of in practice, this section presents an example topic map application.
Dingbat Inc., invented for the purposes of this example, is a manufacturing company that produces construction equipment like drilling rigs, trucks, and so on. It has a large product catalog, consisting of not only the machinery, but also chemicals used by the machinery, spare parts, and so on. Keeping track what variations of a particular drilling rig exist, which pieces of documentation go with what variant, what spare parts with what variant and so on has proven to be a daunting task.
Dingbat already uses a product data management (PDM) system, and so can manage the CAD data, technical properties, and inter-product relationships using this system. Much of the related information, however, is not managed in the PDM system. For example, the product documentation is in a separate content management system (CMS), the information about which customers have bought what products are in the customer relationship management (CRM) system, the information about which branches of the company that can sell and provide training for what products is in people's heads, and so on.
In short, Dingbat Inc. is in the same position as countless other companies around the world. The structure of their IT systems mirrors the organization of the company, but while the departments communicate their IT systems do not. And, as usual, some information that ought to be stored electronically is not. This is the situation illustrated in the figure below.
Dingbat Inc., before
Enter topic maps.
Dingbat decides to tackle the problem of managing their information assets head-on, and decides to use topic map technology to do so. The engineering department is happy with its PDM system, and does not wish to replace it, so the IT department decides to use the Ontopia Topic Map Engine to create a virtual topic map view of the PDM system. This results in a topic map containing the product catalog of the company, mirrored live from the PDM system, with information such as the products, their relationships, their most important technical properties, and so on.
The Ontopia Topic Map Navigator is then installed and made to display the virtual topic map created from the PDM system. This provides instant gratification to everyone in the company, who can now browse their product catalog on the web. At the same time, no revolution has been necessary, and the engineering department continues to use its already-established PDM system. The architecture of this solution is shown below.
Topic mapping the PDM system
The next step is to apply the same formula to the CMS system of the documentation department. Another virtual topic map backend is created, this time for the CMS system. The CMS system really only contains the documentation of the company products, and product codes are used to identify which document documents which product. The resulting topic map for the CMS system consists of a topic for each product variant, with the documents attached to the products as occurrences of various types, and some metadata about each document.
This may not sound very interesting, since the topics in the CMS topic map are unrelated to each other and do not even have names. What the products do have, though, are subject indicators (topic map-speak for globally unique identifiers) created from the product codes. The PDM topic map also has subject indicators for its product topics. This means that these two topic maps can be merged automatically, on the basis of the product codes.
Dingbat Inc. still has a single topic map describing its product catalog, but this has now been enriched with connections from the products to their documentation. Suddenly, figuring out what documentation exists in what state for what variants of the same product can be done in seconds using a simple web interface. It is about this time that the employees of Dingbat Inc. start to get really excited about topic maps.
On the insistence of the sales department, the procedure is now repeated with their CRM system, yielding a topic map that has information about which customers have bought which variants of what products, the internal relationships of the customer companies and branches, as well as the responsible customer managers for each customer company. On the basis of the product codes, this topic map is merged with the PDM+CMS topic map.
Suddenly, the sales representatives of Dingbats Inc. can quickly discover information such as that there is no training material or documentation available for certain products. Could there be a connection between this and the low sales figures for auxiliary items to those products?
The overall IT system at Dingbats Inc. now looks as shown in the figure below.
Dingbats Inc., after
At this point Dingbats Inc. was very pleased indeed, and trying to decide whether to first integrate more systems into their topic map, or whether to start building applications on top of the topic map right away. Wouldn't it be nice, for example, if the system discovered when a product variant had no maintainance manual, and automatically started a workflow task in the documentation workflow system with the purpose of creating one? And so on.
By now you would probably like to know more about the products that enabled this system, and so we will return to the details of the Ontopia Knowledge Suite.
The Ontopia Topic Map Engine is what topic map applications and products use to work with topic maps. This SDK lets applications load topic maps from XML documents, store topic maps in databases, modify and access topic maps, and generally do all an application may need to do with a topic map. The engine has a core topic map API which all applications use to access topic map data, regardless of where those data are stored. Thus, whether the topic map is in-memory, in a database, or a virtual view is all the same to the application.
The Ontopia Topic Map Engine is the product of two years of topic map software development and sports a number of impressive features, such as:
The core API of the engine has been carefully designed to be consistent, intuitive, and scalable. A number of well-considered policies describing the rules topic map data must abide by have been designed and documented. The result is an API that is quickly learned, and that protects the integrity of the underlying topic map data. The core API consists only of data objects, with all action objects separated out. This separation means that the core API is essentially only a thin layer over the underlying data.
The core APIs provide an object model that fully supports both the ISO 13250 standard and the XTM 1.0 specification. All aspects of the models of both standards are supported, as are extensions for other addressing syntaxes than URIs.
The engine has APIs for importing and exporting topic maps to and from XML documents. There is full support for the XTM 1.0 format and all its features, as well as for an XML version of the ISO 13250 format. Topic maps can easily be imported from one format and exported into the other. There is also support for validating XTM topic maps against a DTD and verifying that there are no dead references.
There is also support for importing topic maps from the compact and easy-to-author LTM format developed by Ontopia. This can be used to author and demo simple topic maps very quickly.
To support the development and maintainance of the engine we have developed an extensive automated test suite for the engine. This test suite consists of about 650 test cases and verifies that all aspects of the engine are working correctly. All aspects of the core APIs, the utilities, the XML import/export, and the index API are tested. This ensures that the quality of the engine is high and that it remains so over time.
The test suite is shipped with the product, which enables customers to quickly verify that the engine works correctly in their particular environment. The test suite is also used to verify that the various implementations of the core APIs behave identically.
As is shown in the diagram below topic map applications access the topic map data through the core APIs and the topic map utilities. The utilities also use the core APIs, and so all interaction with the topic map data happen passes through the core interfaces. Import and export of topic maps serialized as XML, whether in XTM 1.0 or ISO 13250 format, is also done through the core interfaces.
The engine architecture
This architecture allows applications and utilities to be implemented independently of how topic map data are represented. The data can be an in-memory object structure (which is what this diagram shows), a special implementation that stores its data in a relational database, or even a virtual view onto another data source. All applications and utilities will still work with the topic map in the same way.
The RDBMS backend is an add-on to the Ontopia Topic Map Engine that uses the architecture described in the previous section to store topic maps persistently in relational databases. The applications access the topic map data through a different implementation of the core interfaces, which means that applications (as well as the engine utilities) need not care how the topic map data are stored. The test suite is used to verify that the two API implementations behave in precisely the same way.
The diagram below shows the architecture of the engine with the RDBMS backend.
The engine architecture (with RDBMS)
Using this backend topic map application can scale to handle enormous topic maps, and also get benefits such as transactions and concurrency management between different processes. The architecture of the engine allows prototypes to be developed quickly using topic maps generated by scripts and stored in XML files. The prototype can then be upgraded to production quality by switching to the RDBMS backend and upgrading the generation scripts. The applications themselves do not need to change, since they are independent of where the topic maps are stored.
The RDBMS backend is based on the TopLink object-relational mapping tool from WebGain. This tool is a mature and well-known product, and thus provides very good performance as well as many opportunities for performance tuning. Since TopLink supports a number of relational databases, including Oracle, Sybase, and MS SQL Server, this enables the Ontopia Topic Map Engine to support many different databases. Oracle was used as the development platform, but using the test suite we can easily verify that the engine works with any given RDBMS server. PostgreSQL 7.1 and InstantDB 4.0 have also been tested and are known to work. MySQL is known not to work (it does not support subqueries). TopLink is certified as 100% Pure Java.
The most natural way to access the information in a topic map today is through the web, and the Ontopia Topic Map Navigator greatly simplifies development of topic map web applications. The Navigator is a general development framework based on the engine that lets you create your own web interfaces to your topic maps.
The framework consists of a set of Java utility APIs designed for web development, and a set of JSP tag libraries. The tag libraries allow much of the logic in each web page to be written using HTML-like tags, thus dramatically simplifying the development process. The tag libraries are very high-level and allow developers to do scope filtering and other advanced operations simply by tweaking the attributes of tags.
Another important benefit to the tag libraries is that since they operate at a high level of abstraction, they allow for many advanced optimizations, like caching of the most frequently retrieved topic names, object pooling, and so on. Thread-safety, session management, and so on is also handled automatically by the framework. Applications using the tag libraries get these benefits with no extra effort.
The framework is fully internationalized and can correctly render topic maps written in any script. We have successfully viewed both Chinese and Japanese topic maps in our navigator applications. Applications built using the navigator inherit these features with no extra effort required.
The navigator framework is based on the Java 2 Platform, Enterprise Edition (J2EE), and applications developed with it can be deployed into any J2EE container. The Tomcat web server was used during development, but using an automated test suite it is easy to verify that the navigator works in other containers as well.
As a demonstration of the power of the navigator framework we have developed the Omnigator, which is short for 'the omnivorous topic map navigator'. This is a topic map web application that can be used to navigate any topic map. You can access an online demo of the Omnigator at the Ontopia web site.
Although written for demonstration purposes, the Omnigator is a useful tool in its own right, since it turns any topic map into a web site with no need for programming at all. This makes it very useful as a tool for quickly creating prototypical topic map applications that can be used to test out the applicability of topic maps to specific problems and also to convince management that topic maps are the correct approach. All that is needed is to generate your topic maps as XML documents, load them into the navigator, and you have a simple topic map demo.
Below is shown a screenshot from the Omnigator, showing the automatically generated page for a topic in Steve Pepper's Italian Opera topic map. The topic shown is Aida, a well-known opera.
The Omnigator
The Omnigator is an advanced topic map application, supporting all aspects of both the XTM 1.0 and ISO 13250 standards. Some of the more advanced concepts supported are listed below.
Metadata occurrences allow you to attach simple data values to your topics. In the screenshot above you can see that the premiére date of the opera Aida has been included. This mechanism allows any data value to be attached to a topic, and can be used to provide vital information about topics inside the topic map itself. The navigator recognizes metadata occurrences through the use of a Published Subject Indicator (PSI), which identifies occurrence types as metadata occurrence types.
Refification allows topics to represent identifiable resources within a computer, such as a web resource or a construct in a topic map. This can be used to make statements about associations, topic names, resources, and even occurrences (that is, the relationship between a topic and a resource). The navigator recognizes reification and uses it to optimize its presentation of the topic map. (See The Omnigator User's Guide for examples of reification.)
Scope can be attached to occurrences, names and associations to say what contexts they are valid in. This can be used to provide topics with names that are appropriate in different situations, and have the navigator automatically select the appropriate name for rendering. In the screenshot you can see a 'contains aria' association, for example, which associates the opera with the arias (songs) it contains. Another name for the same association is 'found in opera'. Scope is used to describe what situations each name is suitable for, and the navigator automatically shows 'contains aria' when on an opera topic, and 'found in opera' when on the aria. The name 'aria in opera' is used when there is no particular context.
These concepts are also supported by the underlying framework, which makes it easy to develop topic map web applications using these constructs.
The navigator framework, and the Omnigator built using it, both support a context filter which each user can set to control what parts of the topic map the web application is to show them. This filter uses the scopes defined in the topic map, and lets users decide which names they prefer to see, and what occurrences and associations they do not wish to see.
This feature can be used to let users decide which language they want to see a topic map in, if the topic map has names in several languages for each topic. They can also decide whether they want to see the full names of topics, for example, or just the abbreviated forms.
In very information-rich topic maps, with many occurrences and associations for each topic, the filter may be used to remove occurrences and associations that are less interesting to the user, in order to keep the amount of information presented to a manageable level.
Users can control the settings of the user context filter by following the 'Filter' link shown in the screenshot above.
The Navigator Framework uses a Model-View-Skin approach to rendering topic maps, cleanly separating graphical layout from information extraction. This simplifies application development, and allows application developers and graphic designers to collaborate more easily.
A view is essentially a web page template, written as a JSP page, with a set of named slots where content can be filled in. This makes it easy for graphic designers to create a single template for all pages in an application, thus simplifying the development, and at the same time separating application development from graphic design.
The model is a JSP page that generates blocks of content and attaches them to the named slots defined by the view. The model pages are generally written by application developers, and do all the hard work of extracting information from the topic map and other sources, and turning it into web page content.
The last piece of the puzzle, the skin, is simply a CSS stylesheet that is applied to the generated page to produce the page style. This will generally define aspects such as the colours, background, fonts and text size.
The end-user using the web application sees the combination of a model, a view and a skin, and each user can freely select whichever combination is found the most appealing. For example, the Omnigator has both a frame-view and a no-frames-view. Each user can freely select which view to use. Similarly, the Omnigator has a detail model, displaying as much information as possible, and a simple model, showing only essential information.
A set of skins are provided with the Omnigator, ranging from the tasteful to the less so. New skins can be added simply by copying them into a specific directory. Users can freely combine views, models, and skins as desired.
The Navigator has a concept of plugins, which allows extensions to topic map web applications to be developed separately from the applications and easily plugged in. The plugins can be deployed by simply dropping them into a directory, and the navigator will automatically detect their presence and include them in the interface.
Essentially, a plugin is a web page that is linked to from every page of a topic map web application. When users follow this link, the web page is informed about the current topic map and topic, and can thus present relevant information. The topic map statistics plugin, for example, will use these parameters to display statistics about the current topic map.
This mechanism allows plugins to be developed separately from specific topic map web applications and yet to be easily integrated into them. The result is greatly simplified development, installation, and administration.
The Omnigator comes with a set of plugins which can be helpful when creating topic maps and developing web applications. These plugins are:
The statistics reporter is a simple plugin that provides statistics about a topic map, such as the number of topics, the number of topics of each type, the association structure of the topic map, and so on.
The merge plugin allows you to merge any topic map in the navigator's topic map registry with the current topic map, and enters the merged topic map into the navigator as a new topic map. You can then browse the resulting topic map in the navigator. This allows you to easily and quickly test the effects of automatically merging topic maps.
The export plugin allows topic maps to be exported in either ISO 13250 or XTM 1.0 format, and to be viewed or downloaded. Two topic maps can be merged using the merge plugin and then exported in merged form using the export plugin. Topic maps can also be loaded in one format (XTM, ISO, or LTM) and exported in another.
This plugin comes with the fulltext integration product and makes it very easy to do fulltext searches on the topic map. Results are displayed in an intuitive manner, with links both into the topic maps and to any external resources in which matches were found.
More plugins are likely to be developed in the near future, since the simplicity of the plugin concept makes it very easy to develop new plugins. A number of trivial examples in addition to those mentioned here are distributed with the Omnigator.
The Ontopia Fulltext Search Integration is a framework which can be used to integrate fulltext search engines with topic maps. The framework is highly flexible, allowing any search engine to be integrated, allowing applications to customize integration in any way they wish, and providing simple defaults for those who have simpler needs.
Using the fulltext search you can search for topics by their names and the contents of their occurrences. This can be very helpful for users new to topic maps who need to find something specific in a topic map. Using the fulltext search they can type the name of what they are looking and immediately be presented with a list of alternative topics. From the topics they can then easily find the information they are looking for.
Using fulltext searching in this way gives users an entirely different user experience, since although they are doing an unstructured search the users get a result set that is anything but unstructured. Since users are shown their search results in terms of topic map constructs it is much easier to judge what they have found, and one can more easily search for obscure or vague concepts by searching for related topics and then navigating around.
The main features of the framework are:
Ontopia will continue to improve the Ontopia Knowledge Suite, intending to make it a complete solution for all aspects of topic map application development. Upcoming releases will enhance the Engine and Navigator products, and also extend the Suite with new products that further simplify the creation of topic maps.
Among the products scheduled for release are:
The Ontopia Topic Map Query Engine will provide an easy-to-use query language developed by Ontopia. The query engine will make it easy to develop search interfaces in topic map applications, but also simplify the development of applications, since topic maps can then be navigated much more easily.
The query engine will build on the engine and be integrated with the Navigator Framework, so that web applications can use the query language to locate the information they wish to display.
The query language used will not be the standard TMQL in the first release, but once a topic map query language has been standardized Ontopia will support it.
Products and features beyond what is mentioned here are also planned. Ontopia is dedicated to providing a complete suite of topic map products ready to revolutionize application development, and what is described in this whitepaper is only the first stage.
