Challenges
Relevance Ranking
Although relevance ranking is a desirable feature, it can be difficult to determine exactly what properties should be used to determine relevance ranking. Relevance, in at least some cases, “depends on context. Relevant to whom? For what purpose? Does the same relevance apply to an undergraduate who is looking for material for an introductory course in physics and a scientist who is searching for recent publications related to current research? The student might need a well known article that is not new, but the scientist is almost certainly not looking for that article†(Sadeh, Google Scholar: pros and cons searching section, para.12). While it might be desirable to determine the right relevance ranking for every possible patron (such as students from different classes, students who choose a certain subject to search, etc.) it is not feasible for most libraries to spend that much time changing relevance results. In fact, it is likely that many librarians leave the results as the vendor set them.
Another relevance ranking related problem is that metasearch engines today cannot usually present true relevance ranked results. In order to present the user with results in a timely manner, the database starts ranking the first results it gets back before all the information is pulled. This is necessary because some databases are very slow to search, and most users are not willing to wait a long time for their search to complete. However, this also means that the items pulled up first have a better shot at being listed highly in relevance ranking. There is also an issue with the amount and type of metadata each database provides. How can a metasearch decide how to rank an abstract verses full text, for instance? Pre-indexing of content would solve or greatly reduce these problems, but pre-indexing cannot be thoroughly implemented at this time.
Maintenance
Many of the problems associated with metasearches have to do with maintenance. Because the system relies on connections to so many databases, and those connections are fragile and easily broken, it is a burden to maintain the systems. Even a library that uses a hosted solution where the vendor maintains the links, librarians must still check to make sure the metasearching is working properly. Upgrades and system changes can wreak havoc on a metasearch system, breaking the connection to many databases. Charles Lockwood and Patricia MacDonald (2007) detail this, saying “Maintenance of the system would be a complex and labor intensive process. Library staff must constantly monitor, fix or report bugs in database connections (“targetsâ€) and often these targets change or become incompatible over time†(p. 85). This unreliable behavior may mean that staff are reluctant to promote the metasearch: “Due to the frequent upgrades and occasionally unpredictable behavior of the system, reference staff are still reluctant to promote or use it when other resources were more reliable†(p. 89). Users, too, may come to see metasearch as unreliable, especially compared with internet search engines’ nearly 100% uptime.
Loss of specialty searching
One of the downsides of metasearching often pointed to is the loss of specialty searching techniques found in individual databases. Though an advanced search screen often allows for simple boolean searching and sometimes searching of specific fields, most of the time the metasearch engine broadcasts the search as a keyword search. Subject searching, metadata searching, and other specialized techniques are dropped in favor of simplicity (searching the widest array of databases possible) and speed (completing the search in a reasonable amount to time). In some databases, the ability to search special fields is a huge advantage, so losing that ability can be a big drawback in a metasearch engine.
Until specialty searching and other issues in metasearching are perfected, it will be unusable in certain applications. In medicine, for instance, it is very important that the most relevant rankings are returned first. Sandra L. De Groote and Kristina Appelt (2007) observed that WebFeat “does not offer the advanced search features including limits and mapping to subject headings offered by directly searching some of the health sciences databases†(p. 39).
kmd :: Nov.27.2007 :: Uncategorized ::