Multi-Device Media Analysis and Summarization for High Bandwidth Connected Environment


This PhD thesis collects all the research work done by the PhD candidate in the Joint Open Lab for Wireless Applications in multi-deVice Ecosystems (JOL WAVE CATANIA) of TIM Telecom Italia, which sponsored his doctoral fellowship. These applications, in which a big amount of multimedia data is analyzed and summarized, may be the enabling technology for LTE-based multimedial services. Three main categories of media have been treated in this dissertation: images, videos, and 3D data. For images and videos we realized two frameworks: The Social Picture and RECfusion. The Social Picture is a framework designed to collect and explore huge amount of crowd-sourced social images about public events, cultural sites and other customized private events. RECfusion is designed for automatic video curation driven by the popularity of the scenes acquired by multiple devices. Through these two frameworks the following topics are discussed: Image Matching, Saliency Estimation, and Video and Scene Summarization. Particularly, we investigated advanced image matching techniques (e.g., Compact Descriptors for Visual Search - CDVS). We detailed Content Based Image Retrieval (CBIR) methods, and we described how to compute the heatmap, which represent a valuable tool for analysis and summarization of large images collections. Then, we reach a novel definition of saliency model that we named Social Saliency. This name has been chosen because the model of attention is obtained querying image databases of social media (e.g., Flickr, Instagram, Panoramio), that definitely represent a social environment. We describe how media collections can be employed for 3D reconstruction and social saliency estimation using images, scene and context tracking using videos, parametrization using 3D medical data, and preservation and restoration using 3D Cultural Heritage data. In all of these applications, we described analysis and summarization methods to be used in high bandwidth connected environments. We present 4 real use-cases, co-authored and published in international journals and conferences: The Social Picture, RECfusion, 3D Data Analysis for Cultural Heritage, and 3D Data Analysis for Medical Research. [Note: PDF of PhD thesis is available under request]

PhD thesis