My father makes a picture of the family dog Fikkie every evening. This resulted in 3500 images that I vector embedded to map this 10 year effort.

Look through it here!
fikkie.亼.net

how?

I did this by vector embedding all images. Embedding an image means that it converts the visual image into a 512-dimensional vector using OpenCLIP with ViT-B-32. This vector does not look at individual pixels, but at high level features like, color, subject, composition and even camera angles. Once we have this 512-dimensional mapping we can flatten it to 2D or 3D using UMAP. This allows us to explore semantically close images. This is called the latent space. The mapping does almost feel like magic, some groupings make total sense but for some areas you almost have to figure out what the similarities are.

Since we now have this map we are also able to embed new images quite quickly because we only need to embedded the new image. after that we can place it in our world and see where they land within this space. this allows for finding similar images.

Extra Data

In combination with vector embedding I also combined it with some traditional data. This all comes from twitter (x.com) where my father posts the images. Getting this data is quite tough to do since the API is removed but gallery-dl worked well in my experience. It simulates regular browsing by using your installed browsers cookies. With this I was able to get date of posting, likes, retweets & views.

After 10 years of posting the most popular picture is when Fikkie was lost and luckily got found again.

Findings

Personally, I find it super fascinating to explore through this map. You can see what at some point in 2018 the lights in the street turned from the warm fluorescent streetlight to a more colder LED light. There is also a whole genre of blurry moving images. which seem to be at the total opposite side of the map in comparison to the close up Fikkie shots. I also love the pictures with various politicians that were running for the election. The various shadow play shots are also a favorite of mine! It also turns out to be a lovely tool to find specific pictures. By just opening similar pictures I can find the image I am looking for quite quickly.

Most similar images.

you would think that after 10 years of dog pictures pretty much all possible shots have been taken. I can also query the most similar images and most common groupings. Here are a few groups: (look at cos for similarity rating)

and the winner for most similar image!

#3 – cos=0.939366

#2 – cos=0.94088

#1 – cos=0.944922 the winner!

Conclusion

Vector embedding is kind of amazing. If you have a dataset it becomes so much easier and fun to explore it. To find duplicates in your data but also find pictures that could exist but have not been made yet.