Despite the significant achievements in multimedia search technologies, the existing solutions still lack several important features, which could guarantee high-quality search services and improved enduser experience. More specifically the following technological/scientific research topics will be addressed by the I-SEARCH project:
- A unified framework for multimodal content search and retrieval
- Sophisticated mechanisms for interaction with content
- Efficient presentation of the retrieved results
The overall conceptual architecture of the project is shown in the picture below and consists of the following three distinct layers which correspond to the three main advances of the I-SEARCH project:
Layer 1 (Descriptor Extraction – RUCoD Formulation): This layer includes all the descriptor extraction mechanisms that will lead to the novel RUCoD descriptor. Three main types of descriptors constitute the unified RUCoD descriptor:
- i. L(ow-level), content-related descriptors: L-descriptors are directly extracted from the networked media (text, audio, image, video and 3D), by utilizing low-level feature extraction mechanisms.
- ii. R(eal world) – related descriptors: R-descriptors refer to the real world information captured from various sensors integrated in the environment. Such sensors include GPS, temperature, time, weather sensors, RFID objects, etc.
- iii. U(ser), user-related descriptors: U-descriptors include non-verbal expressive, emotional and social descriptors. They are called user-related because they describe the user behavior associated with the content.
Layer 2 (Interaction): Involves the novel sophisticated mechanisms for interaction with content. It consists of the following three modules:
- i. Recommendations module: it deals with the feedback added by experts that are the most appreciated in a community upon a define topic.
- ii. Relevance Feedback module: relevance feedback captures the user satisfaction upon retrieval of results and can be either individual or social.
- iii. User interfaces, available for several types of end-user devices.
Layer 3 (Visualization): offers the mechanisms for efficient presentation of the retrieved results. It consists of Visual Analytics technologies, which provide an efficient way of presenting the retrieved data with respect to:
- i. Data management
- ii. Data analysis
- iii. Data visualization