The verity federated infrastructure Kiam Choo, Rajat Mukherjee, Rami Smair, and Wei Zhang   View Paper (PDF)   Return to Industry session 2: data mining and federated systems Abstract In the course of researching a subject, it is often necessary to submit the same search request to multiple heterogeneous information sources in order to (a) aggregate as much information as possible, and (b) integrate different aspects of the subject into a coherent report. While it is clear that there is value in providing a federated search solution to make dealing with multiple sources less time-consuming, not all organizations aggregate from the same sources, and once the information has been retrieved, not all organizations want them to be integrated in the same way.The Verity Federated Infrastructure addresses this problem by providing a flexible framework for adding new sources and customizing the way in which results are integrated, post-processed and presented. A new source is made available by writing a Java module called a worker that abides by the search interface of the source. Sources can range from simple information feeds to more complex applications, e.g., CRM systems, relational databases, etc. Workers also perform post-processing on the results returned by other workers, e.g., to provide uniform scores for results from different sources, filtering, etc. This post-processing enables different results to be integrated into a coherent report. Post-processing is triggered by events that propagate between workers and is done asynchronously in the background while results are being viewed. This ability to do background post-processing allows execution of time-consuming operations that provide substantial value without adversely affecting user experience. Finally, search results are returned and viewed incrementally, which enables searching of peer-to-peer networks via peer workers that we have developed. DiSC'03 © 2003 Association for Computing Machinery