The following working versions of the SCHEMA Reference System are available:
- The first version using proprietary visual descriptors can be found here
- The latest version integrating a non-normative part of the MPEG-7 standard, the MPEG-7 XM software, for extracting, coding and storing in the database standardized descriptors based on the output of the analysis modules (also known as SchemaXM) can be found here
- A future version integrating keyword search will appear soon.
One of the main results of SCHEMA will be the design of a general architecture for content-based analysis, representation, content protection (watermarking), indexing and retrieval systems. The architecture will be module-based, distributed and expandable. It will define the interfaces between different modules and each partner will be able to use its own module. It will take into account the requirements produced from WP2.
The overall system diagram demonstrates how visual content is analyzed by independent modules belonging to three categories:
- audio-visual analysis modules producing visual segments (regions, objects) for which low-level features can be extracted (Qimera
modules belong to this category),
- modules extracting higher-level descriptors characterizing content at a semantic level, e.g. modules determining whether an image or a clip is outdoor or indoor, whether it contains a face or not, whether it contains a specific soccer event such as goal, etc.,
- modules exploiting other modalities such as audio and text associated with the visual content.
Visual information retrieval can then be performed by submitting an example image. The system will analyze the submitted image in order to extract the relevant features and subsequently match these against the features already stored in the database for the entire collection of visual content. Most similar matches are returned to the user. Alternately, high-level information already stored in the database can be employed for semantic-level queries, such as "I want images which look like this and are of outdoor type".