Predicting Saliency in Different Graphic Designs - Souradeep Chakraborty
Abstract: Well-designed graphic designs (such as flyers, Web pages or comics) often direct the user's attention with a mixture of captivating fonts, human faces and visually discriminating regions. There has been an array of work on predicting how these designs will direct the viewer's attention automatically. However, a majority of these models lack interpretability, i.e., the contribution of the underlying factors to saliency are not well studied. Also, these models do not leverage the common image features among these designs for saliency prediction, such as the more frequent texts, faces, headlines, etc. Therefore, a single unified model that can predict saliency in different graphic designs is yet to be proposed. In this report, we first review the existing works on saliency prediction in natural images as well as graphic design images and study the factors influencing saliency prediction in these images. Next, we present a simple yet effective method to predict saliency in different graphic designs. We use Web pages as the illustrative example for our method, but demonstrate that it applies to other forms of layouts. We predict the saliency of multiple visual components in a Web page that are known to attract attention and linearly combine these component saliency maps and page biases created by particular layouts. As it is known that both page type and content influence human viewing behavior, we discover the clusters of different Web page layouts that correlate well with the variety of viewing patterns, based on page layout and semantics, in an unsupervised manner. We use the layout-specific combination weights for the visual components to predict Web page saliency. To test our model, we collected a new free-viewing dataset consisting of 450 Web pages and eye tracking data from 41 subjects. Experiments show that the proposed model outperforms previous Web page saliency methods and compares well with saliency methods for other designs such as posters and comics. Also, the model shows promise for predicting saliency of dynamic Web pages.