Abstract: Visual Counting is defined as the task of counting objects of interest in an image or video, and it is an important Computer Vision task with several applications. Visual counters typically predict the density map given an input image, and there are several challenges with building such visual counters: 1) Predicting an accurate density map is a challenging task because of the large variations in scale and density; it is expensive to collect the labeled data needed to generalize visual counters to 2) various domains and 3) visual categories.

We propose solutions to address all of these challenges. To predict accurate density maps, we propose an iterative coarse to fine approach.
To reduce the labeled data needed to generalize visual counters to novel target domains, we propose to sample and label only a few informative samples from the target domain by making use of visual counters trained on some data rich source domain. These informative labeled samples are used to adapt the visual counter trained on the source domain to the desired target domain.

To reduce the labeled data needed to generalize visual counters to novel classes, we pose counting as a novel few shot regression task. We propose a novel neural network architecture for tackling the few-shot counting task, and collect the first dataset suitable for training few-shot counters.
Our few-shot counter requires a human user to provide a few examples of the novel categories, which makes it unsuitable for any fully automated systems. To address this limitation, we propose the first zero-shot visual counter which does not require any labels to generalize to novel categories at test time.