Abstract: We hypothesized that the parts of scenes identified by human observers as "objects" show distinct color properties from backgrounds, and that the brain uses this information towards object recognition. To address this hypothesis, we examined the color statistics of naturally and artificially colored objects and backgrounds in a database of over 20,000 images annotated with object labels. Objects tended to be warmer colored (L-cone response > M-cone response) and more saturated compared to backgrounds. That the distinguishing chromatic property of objects was defined mostly by the L-M post-receptoral mechanism, rather than the S mechanism, is consistent with the idea that trichromatic color vision evolved in response to a selective pressure to identify objects. We also show that classifiers trained using only color information could distinguish animate versus inanimate objects, and at a performance level that was comparable to classification using shape features. Animate/inanimate is considered a fundamental superordinate category distinction, previously thought to be computed by the brain using only shape information. Our results show that color could contribute to animate/inanimate, and likely other, object-category assignments. Finally, color-tuning measured in two macaque monkeys with functional magnetic resonance imaging, and confirmed by fMRI-guided microelectrode recording, supports the idea that color contributes to the global functional organization of inferior temporal cortex, the brain region implicated in object vision. More strongly in IT than in V1, colors associated with objects elicited higher responses than colors less often associated with objects.
The following files are available:
- Original Images (408 MB) - contains 20840 images (.jpg) that were downloaded from the Internet (these original images were collected by Microsoft Research Asia-MSRA).
- Foreground Mask (120 MB) - contains the mask of the foreground object. 20840 images (.tiff). These masks were created in Photoshop tools by hand segmenting the foreground object from the background in each image. The foreground object is natural if the color of the mask is red. The foreground object is manmade if the color of the mask is blue. If the color of the mask is black, then the fourth column of the Object Info file determines whether the object is (n) natural or (m) manmade.
- Object Info (850 KB) - contains a Microsoft Excel file with a row for each image in the data set (20840 images). The first column holds the name of the original image. The second column holds the name of its corresponding mask image. The third column contains the text label(s) describing the original image. The fourth and final column contains the (n)atural/(m)anmade group designator. The data are replicated in a CSV .txt file.
Citation: Rosenthal et al, Journal of Vision, 2018
* We do not assert copyright of the original images.