|
DREAM will develop efficient storage, management, indexing, search, use and re-use of digital media assets for the postproduction industry. The R&D focuses on the creation and use of metadata generated from low-level content analysis and high-level semantic awareness, and the incorporation of semantic metadata into the workflow process. A novel Metadatabase and semantic intervention will support data analysis, indexing and retrieval. DREAM will create media industry products in the mid-term, and enable the subsequent development of applications in other sectors including security, medical, corporate, and consumer. The project is led by award-winning UK postproduction technology developer FilmLight, working with a company specialising in media software plug-ins (The Foundry), a major user partner (Double Negative) and a university partner (University of Reading IMSS) specialising in semantic technologies. There are three technical strands. The first will define the semantic infrastructure, involving metadata design and domain ontologies and develop tools for semantic video indexing, query formulation and retrieval. In this we will address the 'semantic gap' by combining metadata extracted by image processing with semantic descriptions derived from a network of ontologies, behavioural models and interactive techniques that recognise user behaviour and context. Where automatic detection from low-level processes gives insufficient clues, iterative cycles of image processing may be interleaved with feature extraction and semantic labelling until there is resolution. The second strand will develop new algorithms and tools to extract very rich, persistent metadata based on image processing and motion analysis. The image analysis will be linked to semantic knowledge to improve the accuracy of metadata extraction, and the resulting metadata will in turn improve the object recognition for information retrieval (by for example producing much more accurate human maps than previously possible). The third strand addresses the database level and data management, associating the metadata and data as they move through the system. In an image-based data world, such as digital cinema, the objects are too large and numerous to hold in a database. DREAM will create a Metadatabase to track the metadata associated with files held in any kind of distributed file system. The Metadatabase may be both physical and virtual, since some of the metadata types (motion vector analysis and derivatives) are large files that may be stored in the file system itself. The resulting system may be based on a single site or on multiple sites connected by a secure VPN. The project will result in new scientific and technological knowledge that forms the basis for a range of media industry products in the mid-term, and for further R&D leading to their long-term application to other sectors. The principal deliverables will be: - Ontologies and semantic models of postproduction processes, domain specific models relating to genres (such as action, drama) and objects appearing in them - A semantic resolution engine for labelling and query support - Smart proxy enabled tools for detection, labelling and transcoding data - Templates and role-specific interfaces for semi-automatic semantic labelling, query and retrieval - Algorithms for metadata extraction from low-level features of moving images, with semantic links to higher-level features - Metadata format definitions and interface for the OFX plug-in standard - New tools and plug-ins for motion estimation, object and character matting, feature extraction and modification based on the use of persistent metadata and semantic analysis - A metadatabase linked to a semantically enabled data management application for very large distributed file systems - An integrated prototype solution combining semantic retrieval tools, image analysis and metadata extraction, metadatabase and data management system
|