Sometimes it takes two triggers to learn something new. The first one was an interactive kiosk application about the Valdivia Expedition, the first German deep sea expedition that took place about one hundred years ago. The application was conceived by one of our former students who developed it as part of her bachelor thesis. Whilst her work is a little masterpiece, she also drew our intention to the vast amount of black-and-white images from this expedition. Here is a short excerpt of her work:
The second trigger was a blog post by Adrian Rosebrock. He usually sends me an email once a week and it goes like this: "Hey Jens, Why not do this..." or "Hi Jens, Today's tutorial is about...". In one of his tutorials he suggested to colorize black-and-white images with OpenCV and deep learning and I was instantly intrigued by the idea to apply this to our images from the Valdivia collection.
So, we took the rest of the day off from our daily work and dedicated our time for colorizing images. The first results by my colleague Melanie were promising, but we strived for more. We wanted it to be part of our new web-based research portal. Unfortunately, our current IIIF server we were running at this time couldn't be easily extended, so we grabbed some sources from the net and started our own implementation.
Here are two slides from my presentation held at the 2019 IIIF Conference in Göttingen. The first slide shows on the top half some black-and-white images from the Valdivia Expedition. In the bottom half you can see them colorized. The colorized images are rendered on the fly and you can see some funny color defects the more you zoom into the images. In hindsight, it was not a good idea to go for tile-based rendering as this activity is very hungry for resources. Furthermore, the more you zoom into the images the less the neural network has enough contextual information from the image and hence gets the colors wrong.
The second slide shows three different images of Gänseliesel or Goose Lizzy, a must-see in Göttingen. The first image I took in full color, for the second image I activated black-and-white mode on my smartphone and the third image shows the colorized image of the black-and-white image. Can you spot the differences between the first and third image?
As you can see, applying the algorithm to the whole picture yields pretty good results, though many details aren't colored that accurate.
Have a look at our source code repository. There you will find a detailed description on how you can setup our experimental server at home. So, dig up your old photos and colorize them. We hope you'll enjoy it as much as we did.