Go JibJib

A recent article in iX - Magazin für professionelle Informationstechnik - highlights the DevOps approach for a machine learning project for birdcall recognition.

The project originated from the Coding Da Vinci OST 2018, a culture hackathon in Leipzig in early 2018, where we decided to develop a birdcall classifier based on the animal voice archive using machine learning. The classifier should use a smartphone or REST API to identify birds based on their song. Almost three months later, we presented JibJib - an Android app for bird identification.

To train a classifier with machine learning, we first needed data. For this, we used the nearly 4.000 licence-free recordings from the animal voice archive, as well as additional audio files from the free bird song database xeno-canto.org. The combination of both resources provided us with about 80.000 audio files of almost 200 birds, which were finally used for the training of the machine-learning model. For the actual model, we used a Google-developed neural network ( VGGish), which was adapted and re-trained for our birds.

The query of the model hosted in the cloud is done either via our Android App, or directly via REST API call. This requires an MP4 audio file of a bird call, which can be sent directly to the API via curl:

$ curl -H 'Content-Type: application/octet-stream' -X POST --data-binary @bird_voice.mp4 https://gojibjib.org/api/detect/binary

The JSON message that follows in response contains the IDs of the three birds with the highest hit probabilities: at a value of 0.72, the model has a 72% confidence that the bird with the returned ID is the one recorded. This ID can then be used to obtain more information about the bird:

$ curl "https://gojibjib.org/api/birds/2?desc_en=false&desc_de=false"

More information about the source code and the API can be found on GitHub.

Now we hope to expand and develop the app further. The goal is to improve the model and make bird recognition more accurate. Furthermore, a refactoring of the backend and a standardization of the data science process is planned (more details about the methods of our development can be found in my article in the current iX ). We also hope to add more gamification elements to the Android app. If you have any ideas, questions or feedback just send us an email to gojibjibp[at]gmail.com.

Literatur:

[1] Alexander J. Knipping; Gezwitscher entziffern; Machine-Learning-Projekte umsetzen mit dem DevOps-Ansatz; iX 02/2019, S.92 ff.