An interesting thing I learned today about CLIP is that it knows the styles of different artists! Here are three studying wizards, all from the same random seed, but guided with the names of the artists James Gurney, Paul Kidby, and Jean “Moebius” Giraud (respectively):
It seems like images that imitate painters are much more coherent than images that try to imitate photographs or renders, because people photograph things from every angle but only paint them from the side, so it knows what perspective to use, and doesn't end up in conflict.
— Ryan Moulton (@moultano) July 21, 2021
This is a great example of the compositionality of language. Style transfer has been done before, but you would need an example image for the style. Here CLIP has conflated knowledge of multiple works by each artist into a general knowledge of their aesthetic. I changed 3 words!!