Evaluation
This paper reviewed a selection of the wide range of search features provided by CSA, ERIC WEB and DIALOG. In spite of only looking at a small fraction of the available features, it offers an overview of the problematic of how systems' search features affect effective searching. DIALOG allows users to perform sophisticated searches and retrieves results at great speed. However, ERIC WEB and CSA interfaces outperformed DIALOG in design and overall usability.
Among the reviewed features, Boolean searching was present on all three systems, but it was differently implemented. ERIC WEB allows Boolean searching and nesting of operators but their search field is so small that users would find it hard to visualize a complex query. On the other hand, DIALOG’s raw interface permits flexibility and nesting of operators. CSA offers both a basic interface similar to ERIC WEB's and a Command Mode similar to DIALOG's. The Basic Search interface provides the user with the ability to do field searches that can be connected by Boolean operators from a drop-down menu, whereas in the Command Mode advanced users can simply type in the query.
The three systems performed differently in terms of supporting the refinement of an initial search. This feature is important for supporting the “building blocks” search strategy, in which “a heuristic process takes place, with the searcher refining their search in response to the results obtained” (Smith par 67). DIALOG, briefly displays a query with its results and gives it a set number. In this way, it is easy to modify or add on by just selecting a set number and combining it via Boolean operators. Similarly, CSA offers a search history page, with friendly links to “edit”, “delete” or “combine” searches. ERIC WEB offers a “search within results” feature, which allows users to modify a few aspects of a set, such as refining the type of publication, publication dates or adding related terms from the thesaurus. Surprisingly, ERIC WEB's “save this search” feature only allows searchers to re-run a search but does not offer a way to combine sets, forcing searchers to re-do a whole query.
In terms of help offered to the user during a search, both CSA and ERIC WEB provide links to a help screen in various points on the interfaces. CSA’s help is browsable and searchable which may make it easier for users to find help. Instead, ERIC WEB’s help is only browsable. DIALOG’s help command is very limited in assisting users with problems that they may encounter during their search. DIALOG documentation on the web is extensive but overwhelming at the same time because its interface is too crowded with links. Finally, DIALOG Blusheet is of the other extreme, by being brief and to the point, it lacks the clarity and the much needed examples.
DIALOG has impressive search features which allow information professionals to retrieve accurate results. However, it lacks all the add-on functionality that web-based systems offer. In Péter Jacsó’s own words, “DIALOG increasingly looks like the once-impressive basketball player of the 1970s who still has a few nice shots and blocks--such as the RANK, MAP, and duplicate detection and removal commands--but who can't follow the impressive, engaging pace and rhythm of the street-hoops style adapted by all of the major players in the Web-ified online information services” (par 4).
The author goes further on pointing out the biggest drawback of the
system, its lack of links. Links offer a wide range of beneficial features:
from enabling the connection with other databases (par. 7) to browsing
a thesaurus, to finding full-text sources for an article. In terms
of linking, ERIC WEB offers not only internal links but also external
links to sources where full text articles can be found.
In terms of usability, defined as "The extent to which a product
can be used by specified users to achieve specified goals with effectiveness,
efficiency, and satisfaction in a specified context of use" (ISO
9241-11), DIALOG can only accommodate the needs of trained users.
Conclusion
During this study, different search strategies were explored, which was useful to discuss the concepts of precision and recall. Another topic covered was controlled vocabulary and the advantages and disadvantages of using it. The context of a search was also reviewed, as well as the importance of user’s feedback in evaluating search results.
As mentioned above, this study observed only a limited array of all the features available on the three systems. Though these findings can only be considered limited, the process of comparing the different systems is illustrative of how the systems work on the back end. As Carol Tenopir explains, “…most online services […] have layers of interfaces, meant to make the systems easier to use, but also hide how the search process really works. The systems may look like a Cadillac on the outside, but you can’t tell how the engine works” (par. 7). It is through the meticulous study of search/retrieval features combined with the observation of users’ searching behaviour that one can hope to design better information retrieval systems.