A guest contribution
By
08.06.2022 | 15 minutes reading time
Tweet
In the warm seabeds of this world live unique predators. Despite their slow crawl they hunt for prey with poisonous spikes. With all kinds of dots, stripes and fractals they paint their shells. We are talking about the cone and olive snails, which have fascinated people for thousands of years until today.
In this blog post, we will explore these patterns and much more. We will show how we built our own snail music box, what we learned about genetics, biology and cell chemistry in the process, and how we applied methods from music theory and computer science to make the music of sea snails audible. It all started with a dreamy imagination….
What do snail patterns actually sound like?
As in the horns of the ibex or in our hair, cells work nonstop at the outer lip of the snail shell to add new tissue so that the snail can grow in a spiral. In manufacturing, us humans use similar processes: Patching carpets, inkjet printers, old punch cards from computers, reading musical scores, and even plucking the metal pins on jukeboxes. So we imagined using a cone snail as the rotating piece of a music box. We could almost imagine the music the snail produces as it grows very slowly.
As part of
But first, we wanted to understand the mechanism behind the formation of these patterns. How can millions of cells coordinate to create such complex patterns?
From genes to patterns: How does the diversity in snail shells come about?
A genetic change is responsible for creating two heels for further development of the organism’s shell in early development. Similarly, genetic mutations can also affect the initial concentration and production output of messenger and pigment substances. So that these rough changes produce a complex pattern variety, only by local communication between the cells, so-called Turing mechanisms are particularly suitable. These arise from several oscillating ratios between messenger substances, which influence each other and only indirectly affect pigmentation.
From the multitude of patterns described mathematically in the book “The Algorithmic Beauty of Sea Shells” by Hans Meinhardt, we would like to dedicate ourselves here to the one of the Oliva porphyria, as it shows one of the most exciting patterns. The pattern of Oliva porphyria is characterized by oblique lines, which are formed by the activation and inhibition of pigment production. Pigment-producing groups of cells are activated, which in turn activate their neighboring cells so that after some delay they also produce pigment. This creates a temporal record of the traveling wave of pigment production along the pigment-producing mantle gland at the growth margin. This chain of reactions goes in only one direction, since after their activation the cells enter a refractory period during which they must recover.
Spontaneous activations of smaller cell groups can activate neighboring cells on both sides. Now, two lines run in opposite directions. If two lines meet, they cancel each other out, because these regions have already entered the refractory period by the activation of the other line. The V-shape in the patterns is created.
That the wavy lines are based on a global system is evident from the fact that, if the accidental mutual cancellation of the waves results in a larger white area (1) at one location, a higher wave frequency occurs at another location in return, which leads to the dark areas (2) of O. porphyria.
Branching is the most common recurrent pattern of O. porphyria. It is based on the occurrence of retrograde waves, which thus spread in areas that have just become refractory (3). But how does this even work? On the basis of the V-shaped pattern of O. porphyria we can say that this snail also has refractory periods. Here we can assume that there is an underlying system that controls the number of pigment wave triggering processes. An explanatory approach is offered by Meinhardt: the pigment deposition in the snail shell could be regarded as a kind of waste system and by the fact that after each annihilation the number of migrating pigment waves becomes smaller, it seems logical that this process is regulated by an overarching system which keeps the waste disposal constant. This system then allows a higher probability of branching to compensate for the loss of traveling waves, so that simultaneous branching of lines occurs at several points
Usually the appearance of organisms is closely linked to an evolutionary competitive advantage. The maxim of evolutionary theory also applies to the relief structure of marine snails, but it cannot explain the diversity of patterns of snail shells. In this regard Hans Meinhardt writes:
"[...] it can be assumed that there is no high selection pressure on the type of pattern. The diversity suggests that the pattern can be changed drastically without threatening the species. Nature is allowed to play."
—Hans Meinhardt
Sonification process: How were the snail patterns made audible?
Snail selection
Thirteen snails were selected from the cone snail (Conus) and olive snail (Oliva) families.
First, we selected cone snails (Conus) whose shells had suitable patterns. Intrigued by the diversity of snail patterns, we additionally selected patterns with less complex stripes as well as chaotic patterns.
The large collection of the Museum für Naturkunde, which houses an extensive variety of species as well as different specimens of the same species, was crucial to be able to select the most suitable patterns for photogrammetric scanning.
Photogrammetry (AgisoftMetashape)
Photogrammetry is a process in which the position and shape of an object can be determined by stitching together individual photographs using specialized software. Similar to how we can perceive depth with our eyes through triangulation, the Agisoft Metashape program compares between 40 and 150 photos taken from different perspectives per snail. Characteristic points, so-called keypoints, which are recognized in several photos, are used to create the 3D model of the snail.
The snails are photographed all around from several angles with the help of a turntable. It is important to avoid reflections and to use a background of the same tone when rotating the snails, so that the program only recognizes keypoints of the snail from different perspectives. The images are then used to create 3D objects with the texture of the snails.
This part requires the most computing power, but the resulting 3D models can then be processed as desired and will also be uploaded to the Museum für Naturkunde’s
3D modeling and projection (Houdini and Blender)
The color texture of the 3D model is projected onto a cylinder around the snail so that it can then be rolled out into a flat 2D image. In this process, all the snails are placed in the same horizontal orientation so that the intersection of the cylinder-projected texture also occurs at the lip of the snail, which has no pattern. The result is a square PNG image file.
Image editing (GIMP)
Using the GIMP image editing program, edge detection filters were applied to the image to highlight the dot and stripe patterns regardless of scraping marks or discolorations in the background of the shell. This image is then converted to a black and white image using the thresholding function to obtain maximum contrast, which makes it easier to see the outlines of the dots in the next step. Interesting parts of the whole pattern were selected and cut out, because the whole pattern contains too many points (notes) for a clear sound. We thus obtain a score for the pattern of the snail.
Computer Vision (openCV in Python)
OpenCV is an open source library that also contains several different computer vision algorithms for the Python programming language. The OpenCV algorithms used in our Python project allowed us to automatically recognize structures, such as points, in the input images. A list of all points and their coordinates is created by drawing a bounding box around each point. The x and y coordinates of the center of the bounding box are each taken as the pitch for a note generated at a particular time. Other properties, such as the vertical length of the box, determine the duration of the generated note.
Fitting notes to a scale (Numpy in Python)
If the pixel coordinates of the image were taken directly as notes, almost all notes would deviate from the small number of notes that make up a scale. However, we know that different cultures each use different scales that vary in their harmonization and other acoustic effects, but they always represent only a small part of the possible notes in an octave. We therefore decide to adjust the x-coordinates (pitch) of the snail score so that it snaps to the pitch of one of the notes in the scale. By snapping the y-coordinates (time) to fewer possible time steps, we can set note patterns that occur at approximately the same time as chords.
Following the widespread habitat of cone and olive snails, different scales beyond the European heptatonic system were considered in the setting of the snails. The following scales were selected:
Asmaroneng (Pelog)*
Cirebonese tuning of the Pelog scale from Java. Pelog scales show a high variation between different regions of Indonesia, but mostly divide the octave into 9 intervals.
Balafon*
This scale was empirically derived from the tuning of balafon instruments in the Patna region.
Western 12 TET A Minor*
The minor scale, derived from the Greek scale of the Aeolian mode has a melancholic character compared to the major scale. However, it is still debatable whether this feeling is biologically anchored or merely culturally learned.
Arabic Rast on C**
Arabic scales are made up of 24 equal temperament notes, so that the Rast scale, although consisting of the same number as the western major scale, two of its notes are shifted by a quarter-tone, which does not exist in the Western system.
*from Leimma. A browser-based tool for exploring, creating, hearing, and playing microtonal tuning systems. Created by Khyam Allami and Counterpoint.
** derived from the application scale
Conversion to MIDI file (MIDIutil in Python)
The tabulated coordinates of the point patterns are converted to a MIDI file using the MIDIutil Python package.
MIDI files only have a range of 127 notes and intervals between them are interpreted to be the ones of the twelve tone equal temperament scale, making it the default for electronic music production. The selection of non-Western then needs to specify the map from MIDI note values to pitch values into the MIDI format, or use 3rd party plug-ins in digital audio workstations.
Instrumentation (GarageBand)
The last step contains the greatest room for interpretation, as the received MIDI note events were sonified with instruments and modulations in GarageBand according to subjective taste. Both traditional and modern synthetic sounds were used, which can best reproduce the pattern of the shell artistically and pedagogically. For fine patterns with a high density of dots, we chose staccato sounds to more suitable; for coarser patterns, flowing sounds.
Final result: music videos of 4 selected snails
Structure of the music boxes
The resulting snail compositions were turned into music boxes for the Soft Encounters exhibition at Floating University, where similar Conus species could spin in tune with their own music. The notes played at each moment were displayed with LEDs at the front of the music box.
At
Conclusion
At the Soft Encounters exhibition, the snail music boxes were exhibited on a smaller scale with three cone snails we purchased and were met with great interest and many inquiries about following up on the project.
During the exhibition, the sounds of a Conus nussatella were already processed in a binaural jam session. Further, the composition Conus imperialis and the composition Conus interruptus were the starting point for a performative exploration of the successful evolutionary adaptability of snails. In the performance Snail Trail, Christel Clerc, student assistant at the Museum für Naturkunde, explores a non-anthropocentric choreographic perspective on the theme of “being at home” and makes use of elements of contemporary and transdisciplinary physical theater. The production was created as part of SHADOWLIGHT, a two-month intensive exploration of light and shadow with
Although the amazing sounds of the snails excited us, in the process of setting them to music we also became aware of the limitations that the tools of digital music production have. The 127 different steps of a MIDI file are read in by most DAWs (Digital Audio Workstations) as notes of the western 12 TET system. This standard behavior can only be changed with specialized plugins for microtonal music and forced us to adapt the scales to the 12 TET system and showed how aligned the conception of even modern music production is.
Lastly, we would like to express our sincere gratitude to the Mediasphere for Nature team for allowing us to select the most appropriate specimens for digitization from the Museum für Naturkunde’s diverse collection of molluscs.
