Optical Illusions in CLIP — Embedding Space

adjust region or rotation

Region focus

Rotation

-180° +180°

Region and rotation both shift CLIP's classification — just like directing human attention does.

CLIP Embedding Space (PCA)

duck rabbit PC1 PC2
duck
0.2500
rabbit
0.2500

adjust the controls to see classification shift