el   en  

Early and Late Fusion of Multiple Modalities in Sentinel Imagery and Social Media Retrieval

Discovering potential concepts and events by analyzing Earth Observation (EO) data may be supported by fusing other distributed data sources such as non-EO data, for instance, in-situ citizen observations from social media. The retrieval of relevant information based on a target query or event is critical for operational purposes, for example, to monitor flood events in urban areas, and crop monitoring for food security scenarios. To that end, we propose an early fusion (low-level features) and late fusion (high-level concepts) mechanism that combines the results of two EU-funded projects for information retrieval in Sentinel imagery and social media data sources. In the early fusion part, the model is based on active learning that e ectively merges Sentinel-1 and Sentinel-2 bands, and assists users to extract patterns. On the other hand, the late fusion mechanism exploits the context of other georeferenced data such as social media retrieval, to further enrich the listof retrieved Sentinel image patches. Quantitative and qualitative results show the efectiveness of our proposed approach.

Συνημμένα

  • Early and Late Fusion of Multiple Modalities in Sentinel Imagery and Social Media Retrieval (6.40MB)