Imagine walking in a garden and appreciating a flower that you passed by. So, you take a photo of it, and you go back to your computer to upload the photo and transform your photo into a 3D flower that you can add to your favorite farming game. It’s technologically possible!
Feilong Yan, Minglun Gong, Daniel Cohen-Or, Oliver Deussen, Baoquan Chen
We present a semi-automatic method for reconstructing flower models from a single photograph. Such reconstruction is challenging since the 3D structure of a flower can appear ambiguous in projection. However, the flower head typically consists of petals embedded in 3D space that share similar shapes and form certain level of regular structure. Our technique employs these assumptions by first fitting a cone and subsequently a surface of revolution to the flower structure and then computing individual petal shapes from their projection in the photo. Flowers with multiple layers of petals are handled through processing different layers separately. Occlusions are dealt with both within and between petal layers. We show that our method allows users to quickly generate a variety of realistic 3D flowers from photographs and to animate an image using the underlying models reconstructed from our method.
I enjoyed reading this paper so much. Although my basic knowledge of computer graphics, I was able to understand most of it. The premise that makes generating a model from one photo for flowers possible is the fact that the petals of a flower are usually the same. They look different due to the direction of bends and the photo angle. This makes it possible to use all the petals to average a canonical shape of a petal to resemble all the petals. This canonical petal along with each petal in the image are intersected to create the final mesh. Furthermore, occlusion calculations are used to move the petal and rotate them.
The paper states that the whole process can be done in a minute! That’s really a short time compared to a professional 3D modeler that may take half an hour to achieve the same results.
The process, of course, is not without limits. First, it’s unable to model the complex patterns and bents drawn on the petal. Moreover, it’s unable to model correctly when an image is taken from an angle that makes it hard to detect intersections between the petals.
I wonder what other things we can model from a single photo. If I had to guess, I would probably say that we can model snowflakes. Regardless, I wish such research is further investigated because it will be a great achievement to transform concept art in game studios to 3D models immediately one day!
Unfortunately, you need academic access to be able to read the paper: https://onlinelibrary.wiley.com/doi/full/10.1111/cgf.12332