Issues in the Use of External and Remote Services in Hypermedia Systems

Issues in the Use of External and Remote Services in Hypermedia Systems Helen Ashman*, Tim Cawley**, Scott Davis* and Greg Chase*

* Information Technology Division,
Defence Science and Technology Organisation, Australia

** Flinders University of South Australia

ABSTRACT

This paper describes a design for a distributed hypermedia system and discusses some related issues. The growth and accessibility of distributed information environments enables the sharing of information and computation resources over wide area networks, placing greater demands on hypermedia systems than most existing systems can easily handle. Current hypermedia systems frequently make use of the display services of external applications but rarely use external computation and retrieval services. The distributed hypermedia system proposed by this paper will be able to make use of computation services as well, including services provided by remote sources. The problem of identifying and locating services (display, retrieval or computation), either on local- or wide-area networks, can potentially be addressed.

1. Introduction

Hypermedia systems designers are of necessity becoming increasingly aware of the need to integrate information from the diverse repositories on the global network. Many hypermedia systems have been created which can address information from remote sources, including Hyper-G [2], Konstanz Hypermedia System [3] and the World Wide Web (WWW) [14]. However most hypermedia systems cannot use the computation services of external computation agents on a wide area network. This paper investigates the use of retrieval and computation services from external applications on a wide area network.

The use of display services of external applications is a research area well-covered in the open hypermedia systems literature, but these display services are generally available only on a single machine or at best on a local area network. The first moves towards providing display functionality over wide area networks is being experimented with by WWW browsers which have started using platform-independent machine code F1 to exchange display software between remote sites [13].

WWW software now uses the Common Gateway Interface (CGI) scripts concept [10], which permits the use of retrieval and computation services from arbitrary external applications. However, CGI scripts are limited in that they represent a fixed functionality, applicable to a fixed data repository. There is no notion of permitting remote users to submit personalised information retrieval or processing requests, although users can supply parameters for existing request scripts. One solution suggested to this problem was to create forms which represent the full available functionality of the computation agent and from which the user selected functions, however this could be a very large form and furthermore the functionality represented may be applicable to more than one document. An alternative solution is for a computation agent to ‘advertise’ its capabilities to a services broker in a distributed information environment, allowing remote clients to request these services.

A services broker is an external agent which provides information about the location and provision of services available from applications anywhere on the local- or wide-area networks. It gains information about local services when they register with the services broker and about remote services via other services brokers. Alternative names for services brokers include trader [6] and the object request brokers of the Common Object Request Broker Architecture [11].

Another useful feature is to permit client-end link computation. CGI scripts can only compute links of the sort that the script-writer predicted may be useful to the user. Links supplied this way are valuable but users may also require further link computation, such as linking received information to their local information resources. Client-end link computation is one solution to this.

Allowing client-generated service requests enables the inclusion of user preferences and local environmental variables into the computation. These environmental variables may include such things as the user’s identity, security clearance, institutional affiliation or access permissions. User preferences can include ranking on preferred link types or stop (or go) lists. The necessary structure for stop lists exists in WWW browsers already as they record what links the user has already followed, and this could easily be extended into not highlighting links that are on a user’s stop list. Server-end link computations could include some of these, although at the cost of increased communications overhead, but the server might need to verify information provided by the user, also security-related information should not leave the host environment.

The design proposed in section 3 is based upon the computation of links by external computation agents. The link manager makes use of the retrieval and computation functionality of external computation agents, making it possible for link sources to be computed by, for example, specialised string-matching software or graphics interpreters and for link destinations to be the result of, for example, a retrieval from a database management system or a text retrieval system. Furthermore, these link computations are performed independently of the display tool, so that the link computation functionality is not restricted to that of the display tool but can be any function from any computation agent that the link manager can reach. These issues are expanded in [4] and [12].

Discussion points relating to the design include:

In the two last cases, approaches are compared and situations which favour the use of each approach are discussed.

2. Requirements Arising from Distributed Hypermedia Systems

The World Wide Web has given greater visibility to distributed hypermedia systems as well as highlighting the specific problems arising from the use of hypermedia in a distributed information environment. This section discusses some of these problems and suggests some possible approaches.

3. Basic Design of the Distributed Hypermedia System

A distributed hypermedia system (DHS) will comprise a number of operationally independent applications. The design below includes display tools, a local services manager tool (LSM), a link engine, services brokers for locating and/or providing services and external computation agents providing data and specialised computation capabilities. A particular system may elect to fold some or all of these services into one application.

3.1 The User Workspace

The design of the DHS provides each user with a personal workspace which can be configured to personal needs. Display functionality, link preferences and other user-specific details form part of the user workspace, and so also do environmental variables such as user identity and security classification. These aspects of the user workspace are assimilated into all communications directed at the link engine, influencing the results of the link engine’s actions.

Figure 1 : The user workspace and its interactions

Figure 1 details how information is exchanged between the display tool and the link engine. Each user has one or more display tools and editing tools, a cut buffer and a LSM application which controls communications with the world outside the user workspace (particularly to the link engine component of the DHS) and is responsible for providing information about the user’s preferences and the environmental variables.

The LSM is responsible for packaging and sending the information to be acted upon (a andidate source ‘anchor’) to the link engine, including the addition of user-, context- and environment-specific information and any relevant state information. It receives a piece of information from any display tool in the user workspace (either directly or via the cut buffer), embeds it into a message along with all relevant user preferences and environmental variables and then sends this message to the link engine for action. It then receives any responses (this is either links to follow from a given source, or destinations to display) and displays results in an outline window. The user can then select the preferred result from this window – this choice is usually which link to follow or which destination to display. When a destination selection is made, the LSM orchestrates the display of the selected destination by invoking whatever application has the appropriate functionality, taking into account the user’s display preferences.

3.2 The Link Engine

Figure 2 : The link engine and its interactions

Figure 2 depicts the link engine’s interactions with the user workspaces on the local area network, the computation agents, and services brokers which supply the link engine with information about available services on the local- or wide-area network.

The link engine is responsible for receiving all requests for link services from each LSM, for orchestrating the computation of link sources and destinations by external applications, and for returning the results of computations to the appropriate LSM. Each LSM will connect to the link engine in much the same way that applications connect to a relational database management system. This will allow the communications some notion of state, that is, information about previous communications between the LSM and the link engine is preserved and can influence subsequent services. Note that some communications protocols such as http are stateless protocols, with a connection being established anew each time a service is requested [7] (some WWW browsers maintain state, including the user’s navigation history). In this case, the entire state would be transmitted upon initialisation, and state updates transmitted as they occur.

There is one link engine (perhaps more) supplying links to a number of users (represented by user workspaces) via the LSM in each user workspace. Computations that the link engine cannot perform for itself are requested from services brokers. This can include services provided by other link engines and services provided by retrieval and computation applications across local- and wide-area networks.

Services brokers could supply an application with services from other applications in two ways. Firstly, there are referred services where the requesting application asks for information about the service provider and is supplied with details enabling it to communicate directly with that service. Secondly, there are proxy services where the requesting application sends a request for a particular service and the services broker invokes that service on behalf of the requesting application. In each case, the services broker dynamically binds the name and location of a service provider to the request for the service F3.

Link (computation) specifications are stored as rule pairs of computations, in an external file or database table. This external store of specifications is read into memory upon startup of the link engine. Each rule pair constitutes a complete link type from which many individual links can be generated. The first rule of the pair represents the conditions that an object must satisfy in order to be a source anchor for that link type, and the second rule describes the computation which derives the destination anchor from the source anchor. The two rules may be expressed in different languages [12].

It is possible for a link manager to involve more than one computation or retrieval agent in the following of one link. The LSM can send a message to the link manager with the request “is this object the source of any links?”. The link manager will attempt to substitute the object details into its list of link computation specifications and for any successful substitutions, sends the source computation part of the specification to the appropriate external agent for computation. There may be more than one such source specification to be computed. The result returned for each specification is a “true” or “false”. When the link is to be followed, the destination part of the selected link specification is then sent to the appropriate computation agent (quite possibly a different computation agent to that which evaluated the source part of the same link computation specification) and this returns a number of destinations.

4. Interprocess Communications

There are two aspects of interprocess communications discussed in this section. Section 4.1 describes the use of the cut buffer for the exchange of data between display tools and the LSM. Section 4.2 describes briefly how the hypermedia system can send and receive service requests and information using the protocol most appropriate to the target application.

4.1 Exchange of Data Using the Cut Buffer

The DHS will support both the usual hypermedia system direct communications as well as using the cut buffer for the exchange of data between display tools and the LSM. Direct communications are likely to operate faster and offer more sophisticated linking but the cut buffer exchange can support linking out of many of those applications which could not otherwise communicate with the link engine. Another important issue is that direct communications require some addition to the source code (or a macro) of every application to be integrated, in order to support the packaging and sending of communications to the link engine, plus the associated software verification. In contrast, the cut buffer exchange makes use of the existing cut buffer or clipboard facilities of an application, so that integration can be managed with no new source code or new applications, no addition to the DHS and no alterations to any application or the operating system.

4.1.1 What are Cut Buffer Exchange and Direct Communications?

Direct communications are employed by most hypermedia systems which separate information display from link engines. A display tool and the LSM communicate with each other using an agreed protocol. Messages are targeted to the receiving application and are not visible to any other application. Direct communications require an application to be cognisant of the other applications which are to make use of its data, and to actively participate in the exchange of data.

The alternative to direct communications is the cut buffer exchange, which uses the cut buffer as a global variable within the user workspace. The LSM then retrieves this data for itself. The advantage of this is that no communications software need be supported, and many display tools already support use of the cut buffer for other purposes. With cut buffer exchange, the user selects a piece of information and that information is automatically copied by the display tool to its usual cut buffer. The LSM then collects the contents of the cut buffer and packages this information up along with environmental information, user preferences and other user-specific details and sends the message (by direct communications) to the link engine.

The cut buffer can be either the cut buffer or a clipboard, both belonging to a user workspace. The distinction between the cut buffer and the clipboard is that highlighting data automatically places it in the cut buffer, whereas an explicit ‘copy’ or ‘cut’ procedure is needed to place highlighted data into the clipboard. The cut buffer is usually supported by X11 windowing environments, whereas the Macintosh and Windows environments tend to only support the clipboard. However, the principles are the same, namely that links can be provided at a very general level to any application that uses a cut buffer or clipboard [5].

Links can be provided to data in almost any application, once it has been displayed. This is achieved by enabling two characteristics in a hypermedia engine, the first being cut buffer exchange and the second being non-advertised linking. The majority of links in hypermedia systems are advertised, that is, links that are found have buttons put around the anchor, the anchor is underlined or highlighted, or perhaps the anchor is displayed in a different colour. When links are externalised (i.e. not already embedded in documents), the LSM will need to reconcile the link anchors references that it possesses with the document being displayed. If these links are to be advertised, the LSM must either communicate these details directly to the display tool, alternatively a mock button can be overlaid on top of the display application without its knowledge and actions on that button are intercepted by the hypermedia management system. This first option is common among the ‘open hypermedia systems’. The second option is used in “shim” or “proxy” programmes [1] [5] (see 4.1.4).

If links are not to be advertised, then there is no need for either of these two options. Details of the candidate source anchor are passed to the hypermedia management system which then immediately attempts to find any destinations for links that the anchor may have attached. There is no need to return linking information to the display tool or to a proxy programme.

The combined use of non-advertised linking and cut buffer exchange makes it possible to provide links coming out of a totally unaware display tool, although these links will not be advertised. It is not necessary to make any changes to the display tool software or the windowing environment, or to create any new software in order to support this. For this mechanism to function effectively, the users must become accustomed to the idea that almost every selectable object in the display is potentially the source of a link.

4.1.2 Activating Link Service Requests

When using direct communications to notify a LSM that services are required, the receipt of the request is sufficient to authorise the LSM to act. However, cut buffer exchange requires that the LSM be notified or discover for itself that action is required. Two possibilities are Combined use of these two options should prevent the use of cut buffer data for links when not intended. Occasionally there may be errors with the first option, such as if the user is performing a cut and paste operation on some link-specific information.

4.1.3 Link Specifications for Cut Buffer Contents

One problem arising from use of the cut buffer to exchange information is that it may be difficult to determine the format or structure of the contents of the cut buffer. Of course, only information specific to the contents of the cut buffer needs to be considered here, as all user-specific details and environmental variables are handled by the LSM.

In the worst case, the LSM will have nothing more than the actual contents of the cut buffer with which to work. In such a case, one option is for the LSM to assume some default format (such as plain text) and pack it up accordingly. Alternatively the LSM can pack the contents into a message and tag the contents with some designation that indicates that the contents are of an unknown format retrieved from a cut buffer. Link computation specifications can be defined especially to deal with such information.

However in some cases there will be more than just the contents of the cut buffer available. Some operating systems will be able to provide information about the contents of cut buffers or clipboards, such as the name of the application which placed the contents there, or the format of the contents. A fairly sophisticated level of format information is already provided by platforms such as the Macintosh or Windows, in which the structure of certain types of information is preserved across applications.

4.1.4 Related Work

Cut buffer exchange is not yet widely-used to pass information between applications for hypermedia purposes, although it has been used in [5] for some time. Cut buffer exchange is the obvious approach to use when there is a need to incorporate display facilities of legacy applications.

Similar work with the use of the clipboard to exchange information was done in the creation of proxy programs [1] [5]. The primary difference between the proxy program approach and the approach above is that proxy programs are attached to each participating display tool, by editing the resource information of the application to create a menu of hypermedia-specific operations and some buttons denoting static links. The advantage of proxy programmes over the approach outlined above is that it is possible to create manually-specified links and to advertise these links (although not always in situ). However the disadvantages are that it can be difficult to add a menu to another application. The approach outlined in this paper requires no changes to existing software, no changes to the working environment and no additional new software – that is, it requires no preparation at all in order to permit linking. It will work for any application that uses a cut buffer, however it is not able to advertise links but can only provide non-advertised links.

The most pragmatic solution will be to use both approaches as they offer complementary functionalities. The proxy programme approach has much to offer but for practical reasons its scope may be limited. The additional level of cooperation has less sophisticated functionality but can be used in a wider range of situations.

4.2 Multilingual DHS

A link engine should be able to communicate with the LSM and with other services brokers and computational agents using a number of different communications protocols, For example, the Hyper-G information server can both serve and request information using protocols of the WWW and gopher [2].

For services by proxy, there is no need for the link engine to be able to communicate using any protocol other than the one it shares with the services broker which interfaces external services. In this case, the services broker will be responsible for being able to communicate with all service providers (such as retrieval engines or computation agents) using the appropriate protocol. When services are tendered by referral, the link engine will itself need to communicate with the external applications and hence will need to use the appropriate protocols. At present, services brokers offer services by referral only, but in the future, they may offer services by proxy.

Support of multiple protocols also enables the link engine to communicate with applications having various levels of accessibility. For example, there are many isolated sites which do not have http, telnet or ftp connections (perhaps blocked by a firewall) but which do support email – in this case, email would be the means by which the DHS requests services and receives information from these sites.

The DHS will have a library of functions which provide an interface to a number of protocols. To enable the DHS to communicate with a new protocol, a new set of underlying procedures is linked to the existing interface library. Hence the DHS software does not require recompilation when new protocols are added to its suite.

5. Link Specifications and the Services they request

The DHS will have externalised link computation specifications which are not part of the link engine source code [4] [12]. There are two possible classes of language for link computation specifications with a total of four different ways to request services.

An universal hypermedia language (UHL) may be defined to be a widely-recognised language for the specification of a well-defined set of hypermedia services. These services will support hypermedia linking and possibly also information display. Hypermedia linking services will include testing of whether an object is the source of a link, finding destinations of a link, computing links whose anchors are to be displayed and so on. When links are dynamically determined by computations carried out by external applications [4] [12], services required will also include searches and queries, text string matching and any number of operations which are commonly used to automatically define links.

With direct representation langauges, link computation specifications are expressed in the language that can be directly interpreted by one or more computation agents. For example, specifications could be a Prolog predicate, an SQL query or a C-shell script.

With UHL and DRLs, link computation services can be requested in four ways.

Choosing from the four service request options must take into account whether local artifacts are a necessary part of the computation. For example, an SQL query usually is dependent upon the local database schema, or a full-text retrieval system may have local keyword lists. Whenever the computation involves site-specific artifacts, service by artifact name is the correct approach.

Another consideration is that UHL will likely only support fairly widely-used functions, such as string-matching or image display. Less common functions will usually belong to the DRL category, although any service not requiring local artifacts could theoretically be specified in UHL. However this requires that UHL be able to designate an enormous range of services, and it would need to be continually updated as new services became available. In practice, the newer or more specialised services will probably be specified in one of the DRLs.

There is some overlap of service by language name and service by application name because some interpreted languages are unique to an application. But this is not always the case, for example, there are numerous interpreters for the Prolog and Lisp languages. Service by application name can be useful for supporting user preferences, such as preferences for display tools or different versions of applications.

6. Conclusion

There is no one correct design for a hypermedia system. There are hypermedia systems which address certain problem domains, and in doing so, they may lose flexibility or functionality or some which may involve high running costs.

The design outlined in this paper should be seen as a ‘pure’ version of a distributed hypermedia system. During the design and implementation phases, we are discovering that the pure system is not the most efficient. We are still experimenting and have no final recommendations yet, but we foresee that the optimal distributed hypermedia system will have a combination of inbuilt and external link computation functionality, where the external link computation is used at those times when the inbuilt functionality is inadequate or inappropriate for the user’s requirements. Inbuilt link computation functionality can be either link computation built into the display tools or functionality built into the link manager. Functionality inbuilt into the link manager might be restricted to widely-used functions on common data formats such as simple string matches on plain text. Functionality inbuilt into display tools could include specialised computations for information proprietary to that display tool.

Our preliminary recommendation would be to use the hypermedia functionality of this design to supplement whatever hypermedia functionality alternative applications are unable to provide.

References

[1] K.M. Anderson, R.N. Taylor, E.J. Whitehead, Chimera: Hypertext for Heterogeneous Software Environments, Proceedings of European Conference on Hypermedia Technologies, ACM, 1994, pp 94-107.

[2] K. Andrews, F. Kappe and H. Maurer, Serving Information to the Web with Hyper-G, Proceedings of the Third International World Wide Web Conference, Darmstadt Germany, April 1995, http://www.igd.fhg.de/www/www95/papers/105/hgw3.html.

[3] R. Assalg, R. Hammwoehner and M. Rittberger, The hypertext Internet connection: E-mail, online search, Gopher, Proceedings of 17th Online Information Ô93, Learned Information, December 1993, pp 453-464.

[4] M. Bieber and C. Kacmar, Designing Hypertext Support for Computational Applications, CACM, Vol 38 no. 8, August 1995.

[5] H.C. Davis, S. Knight and W. Hall, Light Hypermedia Link Services: A Study of Third-Party Application Integration, Proceedings of European Conference on Hypermedia Technologies, ACM, 1994, pp 41- 50.

[6] Distributed Systems Technology Centre, Interworking Trader Project, http://www.dstc.edu.au/AU/research_news/odp/trader/trader.html.

[7] M. Gleeson and T. Westaway, Beyond Hypertext: Using the WWW for Interactive Applications, Proceedings of the 1st Australasian World Wide Web Conference, Australia, May 1995, http://www.scu.ed u.au/ausweb95/papers/hypertext/gleeson/.

[9] National Centre for Supercomputing Applications (University of Illinois), The NCSA Mosaic Homepage, http://www.ncsa.uiuc.edu/SDG/Software/Mosaic/NCSAMosaicHome.html

[10] National Centre for Supercomputing Applications (University of Illinois), Common Gateway Interface (CGI) , http://hoohoo.ncsa.uiuc.edu/cgi/overview.html.

[11] Object Management Group, The Object Management Architecture (OMA) Executive Overview, http://www.omg.org/omaov.htm.

[12] J.L.M. Verbyla and H.L. Ashman, A User- Configurable Hypermedia-based interface via the functional model of the link, Hypermedia 6(3), 1994, pp 193-208.

[13] World Wide Web Consortium, Mobile Code, http://www.w3.org/hypertext/WWW/MobileCode/, 1995.

[14] World Wide Web Consortium, The World Wide Web, http://www.w3.org/hypertext/WWW/TheProject.html.


HTML version of ths paper by Helen Ashman, January 1996.

Footnotes

F1 The principle of architecture-neutral software is not unique to the WWW or to hypermedia. In the early 1980s, UCSD Pascal could be compiled to assembler code for an abstract machine called the ‘p-machine’. Intepreters for this assembler code (called ‘p-code’) were written for the Apple II, Vax and Corvus Concept machines among others.

F2 For example, World Wide Web links frequently point to documents which are no longer available, leaving the reader with an access error message.

F3 There is a third way in which services brokers could provide services to requesting applications – the services broker could send the actual computation agent to the client in machine-independent code form [13]. This may only be useful for services that do not require access to remote databases.


Copyright

Defence Science and Technology Organisation, Australia and Flinders University of South Australia, © 1996. Permission is hereby granted to use this document for personal use and in courses of instruction provided that the article is used in full and this copyright statement is reproduced. The Defence Science and Technology Organisation, Australia and Flinders University of South Australia also grant a non-exclusive licence to New Jersey Institute of Technology to publish this document in full on the World Wide Web and in printed form with the workshop papers, and for the document to be published on mirrors on the World Wide Web. Any other usage is strictly prohibited without the express permission of the Defence Science and Technology Organisation, Australia and the Flinders University of South Australia.
This paper appears in the Proceedings of the Second International Workshop on Incorporating Hypertext Functionality into Software Systems, held in conjunction with the ACM Hypertext ’96 conference, Washington, U.S.A.