Recently, I visited Detroit, Michigan with the Structure Sensor and my iPad mini. During this trip, I wanted to visit the Detroit Institute of Arts (DIA) and see how I could capture works of art with a 3D scanner as a normal visitor. Since I grew up in Michigan, the DIA has had a large impact on my perception of art and also on the way I developed as both a scientist and an artist. The murals of Diego Riviera imprinted on my young mind imagery of industrialization while the works of Warhol and Rothko gave me a sense for abstraction and modern art. Capturing 2D images of paintings is easy with a digital camera (allowed in the DIA), but sculptures are really more interesting with a 3D imaging solution. I’ve used 2D images from the DIA (and other museums via OpenGLAM) in digital remixing projects and was curious to see what could be done with the sculptures. So, with the Structure Sensor (3D depth camera) I began making my way through the museum scanning statues from Rodin and ancient Greece as well as a few rooms.
3D Scanning and Society
I had previously experimented with scanning through shop windows in Zurich, so I knew that I would be able to produce 3D scans from statues even if they were behind protective glass. This is a big advantage of using a 3D scanner; it allows you to really interact with statues in a new way in a museum setting. You can produce a scan and then really move in close on the model to appreciate the beauty of the form and texture of the works of art. Theoretically, the scan then allows you to print off your own copy or remix the form using programs such as Zbrush, Blender, MeshMixer, etc. Naturally, many pieces of art in different museums are protected by copyright, so it will be interesting to see how 3D scanning and intellectual property are addressed in the future. Recently 3Ders.org reported on Man accused of copyright infringement after 3D scanning a Michelangelo statue where the artworks are full-sized bronze casts of Michelangelo’s David & Moses (so not original pieces from the original artist, but rather castings of original representations). The introduction of new technologies into areas of society is always an interesting experiment. The Explorer program from Google Glass, for example, has shown that we’re not quite ready yet as a society for the widespread use of consumer-based smart glasses. While scanning statues in the DIA I got a few inquiring looks from people. Some of them understood immediately what I was doing and one museum patron struck up a friendly conversation, asking what I planned to do with the scans. I also heard a German mother tell her son to look at what I was doing with my Geräte (devices in German). This trip allowed me to show the DIA to my colleague Elizabeth Henry (henrygeneral.com), and when she was tired and resting on one of the benches, I scanned her as well.
3D Scan Quality
Overall, the Structure Sensor worked as I had expected. For most statues made of non-reflective surfaces, such as bronze or stone, I was able to produce good quality scans and surface texture maps. The statues would need to be smaller than I am, so ideally the size of an average adult male was the maximum size I could scan. This would allow me to stretch with my arms and place the iPad above my head to scan the top of a statue if needed. For statues behind protective glass, there was generally no issue with scanning, except for the corners of the protective structures. On the corners where two pieces of glass were joined, the image stitching process of the scanner could be thrown off due to the unexpected scattering of the depth camera data. This would also lead to artifacts or shadows in the color map of the final scan. The solution was to move quickly over the areas where a corner existed; otherwise scanning through protective glass was not an issue.
Of course, scanning small features or objects was a problem; in particular, a bronze casting of Hercules and the Stag could only be roughly captured. One of my favorite scans was of a Sealed Bather from the contemporary art section, with gentle slopping forms and no small angles, the statue’s surface was perfect to scan, even considering that another statue was in the same protective case and obstructed part of the viewing/scanning angle. I also scanned some of the rooms from the American and European art sections of the museum. The results were very rough since the Structure Sensor is simply not powerful enough to accurately measure boundaries far away. Still, I was impressed with the ease of room scanning and the ability to quickly capture the 3D volume of a room.
When using the Structure Scanning app, you must choose the size of the model you intend to scan before you start, and this then defines the size of the mesh elements that will be created. Therefore, scanning a large statue means the resolution will be rather low. However, if you only scan the head, you can expect a good quality scan with better defined features. Of course, the end application is most important when determining the ideal scan size. If you only want to print off a small reproduction of a statue, then the resolution can be small. If however, you would like to print off a full-sized head, you will want the highest resolution possible with the fewest errors during scanning. This was evident with the statue of a Greek Slave, which I did for the full statue first and then only for the head. The scan of only her head provided a much better definition of her facial structure as well as hair and around the eyes.
Custom Scanning App
We started out using the example app provided for use with the Structure Sensor. However, it has various limitations, such as not being able to save model files directly (they need to be emailed), and additionally it would be useful to include presets for different sized objects. Therefore, we have started developing a custom scanning app for iOS with the Structure Sensor, which will integrate more features and provide a better user experience specifically for scanning museum objects.
Future Projects
What’s coming next? Google’s Project Tango (table with integrated depth camera) is on the horizon, allowing people to interact more fluidly within 3D spaces. Additionally, the Metaio Augmented Reality SDK now allows the inclusion of depth cameras (like the Structure Sensor) for an integrated augmented reality experience. I had the opportunity to pick one up while in the US, but decided to hold off until they are officially shipping to Europe. For 3D imaging and augmented reality applications, museums offer a defined use case for the technologies, which allows for easier adoption by consumers. In the DIA, people intuitively understood what I was doing without additional explanation, so they were at ease with the technology (not always so with Google Glass). The pilot release of Google Glass suggested that the general population (San Francisco, in particular) isn’t ready yet to accept face-mounted computers as a normal personal computing product. However, industrial and defined use cases (like museums) are a completely different story.
At idezo we’re continuing to experiment with the Structure and other 3D scanning technologies for interactive museum applications. We see great potential for integrating 3D scans into projects such as OpenGLAM and providing 3D scanning services for cultural projects and museums.
If you’d like to discuss interactive museum projects, feel free to contact us directly.
