We introduce a novel thought for integrating artists perceptions on the real world into neural image style transfer process. Conventional approaches commonly migrate color or texture patterns from style image to content image, but the underlying design aspect of the artist always get overlooked. We want to address the in-depth genre style, that how artists perceive the real world and express their perceptions in the artwork. We collect a set of Van Goghs paintings and cubist artworks, and their semantically corresponding real world photos. We present a novel genre style transfer framework modeled after the mechanism of actual artwork production. The target style representation is reconstructed based on the semantic correspondence between real world photo and painting, which enable the perception guidance in style transfer. The experimental results demonstrate that our method can capture the overall style of a genre or an artist. We hope that this work provides new insight for including artists perceptions into neural style transfer process, and helps people to understand the underlying characters of the artist or the genre.

Genre-based style transfer results. The top row is real world photo, the bottom row is the stylized neural artworks based on reality-perception correspondence in Van Gogh’s style and cubism respectively

We have also released our dataset used in this paper [Download from China][Download Outside from China]

We collected 309 Van Gogh photo-style pairs and 34 cubism photo-style pairs. The Van Gogh-photo dataset consists of 309 Van Goghs painting and the semantically corresponding real world photo pairs. The cubismphoto dataset has 34 cubist painting and real world photo pairs. We choose these two genres on account of two reasons: Firstly, Van Goghs paintings and cubism artworks are commonly used as reference images in style transfer methods. Secondly, their paintings have distinctive genre characters, such as the bright colors in Van Goghs paintings and the broken geometry patches in cubism artworks. 

Detail information can be found from our IJCAI 2018 paper. If you are interested in this dataset or our work, please cite this paper as follows:

Zhuoqi Ma, Nannan Wang, Xinbo Gao, Jie Li. From Reality to Perception: Genre-Based Neural Image Style Transfer, in Proceedings of of the Twenty-Seventh International Joint Conference on Artificial Intelligence (IJCAI-18), 2018, pp. 3491-3497.