- A unIfied framework for multimodal content SEARCH

FP7 Logo

RUCoD Schema

Rich Unified Content Description (RUCoD)

Introduction

The EU-funded project I-SEARCH aims to provide a novel unified framework for multimodal content indexing, search and retrieval. The I-SEARCH framework will be able to handle specific types of multimedia and multimodal content (text, 2D image, sketch, video, 3D objects and audio) along with real-world and user-related information, which can be used as queries and retrieve any available relevant content of any of the aforementioned types.

The searchable items within I-SEARCH will span from very simple media items (e.g., a single image or an audio file) to highly complex multimedia collections (e.g., a 3D object together with multiple 2D images and audio files) along with accompanying information. All the above multimedia collections are called Content Objects (CO). For a formal representation of COs, a novel description framework is introduced by I-SEARCH: the Rich Unified Content Description (RUCoD).

The RUCoD is a data representation of a CO consisting of descriptions (features/characteristics) of various multimedia types that are somehow associated to each other. A RUCoD will serve as a generic multimedia content descriptor, enhanced with real-world information, expressive and emotional descriptions, in order to facilitate the retrieval of different types of media irrespective of the query format. The goal is to enable the development of very heterogeneous applications, ranging from pure search and retrieval to personalised, user interaction specific and/or context-aware search. Therefore, this unified approach is proposed, where the actual metadata and the real world and user-related parts reside in the same format.

RUCoD Schema v1.4.1

The latest version of RUCoD schema (v1.4.1) is now available. It consists of the following XSD files:

The RUCoD.xsd Schema file

The RUCoD_Descriptors.xsd Schema file

Documentation

  • A more detailed description of the RUCoD framework is available at the following paper: P. Daras, A. Axenopoulos, V. Darlagiannis, D. Tzovaras, X. Le Bourdon, L. Joyeux, A. Verroust-Blondet, V. Croce, T. Steiner, A. Massari, A. Camurri, S. Morin, A-D. Mezaour, L. Sutton, S. Spiller, “Introducing a Unified Framework for Content Object Description”, International Journal of Multimedia Intelligence and Security, Special Issue on “Challenges in Scalable Context Aware Multimedia Computing”, Volume 2, Number 3–4/2011, DOI 10.1504/IJMIS.2011.044765, Pages: 351-375, January 2012 (http://www.iti.gr/iti/files/document/work/IJMIS0203-0409%20DARAS.pdf)
  • An in-depth description of the RUCoD schema is available at the I-SEARCH deliverable D3.3, which is publicly available here.
  • A presentation of RUCoD is available here.

Related Links

The RUCoD format has been adopted in the EU-funded project CUBRIK. RUCoD will be used as a metadata model in CUBRIK.

A presentation of RUCoD was given in the 2nd International Workshop on Standards and Technologies in Multimedia Archives and Records (STAR 2013), 18th – 19th January 2013, EPFL Lausanne, Switzerland. (http://mmspg.epfl.ch/page-58398.html)

Use case

A multimodal dataset has been created in I-SEARCH to demonstrate the Generic Multimodal Search Use Case. The dataset consists of 10305 COs classified into 51 categories. The COs consist of images, 3D objects, sounds and videos accompanied by textual information, tags and location information (if available). The RUCoD descriptors (XML documents) of the entire dataset are available for download below. The links to actual media files are available within the corresponding RUCoD XML document (in the <MultimediaContent> tag).

Low-level descriptors have been extracted for the 3D objects and images of the dataset. The links to descriptors are available at the RUCoD XML files (<L_Descriptor type=”ImageType”> for the image descriptors and <L_Descriptor type=”Object3D”> for the 3D object descriptors).

Download the I-SEARCH Multimodal Dataset
(19M Zip file, MD5 checksum: fd4e6bbd896063814d024538a5d2f738)

Creative Commons License
This work is licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License

  • Twitter
  • Facebook
  • LinkedIn
  • Share/Bookmark
Bookmark the permalink. Follow any comments here with the RSS feed for this post. Both comments and trackbacks are currently closed.

© 2010 I-SEARCH Project Consortium