The increasing spread of misinformation is one of the major problems facing society. In particular, misinformation poses a significant challenge to journalists, as their work increasingly depends on information from web sources and social media. In our work, we focus on the problem of supporting journalists in identifying visual misinformation, for example, images that have been manipulated or taken out of context. Based on informal interviews with journalists, we identified an opportunity to support them in collaboratively detecting visual misinformation.

To this end, we develop computer vision similarity matching process that aims to retrieve previously published versions of a query image, which may have been manipulated. An empirical evaluation of the matching process on a large-scale dataset collected from various sources including social media and news, shows that this process can successfully retrieve a wide range of image manipulations, but that some manipulations remain difficult to detect.