Sketch Based 3D Modelling - Drawing in 3D

Sketch based modelling is a method of creating 3D models by drawing 2D strokes.

The idea is appealing to me since I come from a 2D art background and I love drawing. There are many research papers on the subject and tantalising examples of digital analytic drawing which look like sorcery. ~~Spoiler alert, digital analytic drawing is not available publicly (or it is so top secret I can’t find anything that does it).~~

Edit March 2023:

SketchLab, a group of engineers and designers have released a scientific paper and presentation they talked about at Siggraph 22 which you may be interested in.

Edit March 2021: for something that looks like digital analytic drawing scroll to the bottom of the article to the AI section.

Sketch Based Modelling

When I started learning about 3D software I created my first 3D models with a web based software called Smoothie 3D.

Smoothie 3D lets you trace over an image and then makes the image into a 3D mesh using the image as a texture. This is a fast and easy way to make very basic 3D models using 2D artwork or photos as a guide. It is easy to make something relatively quickly for 3D printing or visualisation with this method compared to traditional 3D modelling techniques.

I’ve also used a similar software called Archipelis Designer which is a desktop based software made by the same developer.

This is a very like Smoothie 3D but it had a few more features I found handy, especially exporting combined meshes into Blender for quick base meshes for sculpting. I found this method a very fast way to sketch out basic 3D shapes for use in other 3D applications. I used Archipelis in this way when I was creating this quick (ish) squirrel character.

Squirrel Mascot for the Art Squirrel Blog
by Christine
on Sketchfab

I created the torso, arms and tail in Archipelis and then exported the mesh parts as obj files. The parts were imported into Blender where I sculpted a bit more detail and painted them before rigging the model and posing it.

There were some useful features from Smoothie 3D I wish had been included in Archipelis but it is useful if you combine it with Blender. Archipelis is limited to creating simple symmetrical models, whereas I was able to lock images to parts of meshes in Smoothie 3D and move those mesh parts around to create non-symmetrical models. This limitation was overcome by exporting mesh parts and using them in Blender though.

I did some research and also came across some other sketch based modelling software. A software developer called Ryan Schmidt worked on a project called ShapeShop. If you are interested in the science of the technique behind the software there is lots of information here: http://www.rms80.com/shapeshop.

Unfortunately the actual software is pretty outdated, but it was ahead of its time.

I came across an experimental project called Galumph. The developer Andrea Interguglielmi used to work at Dreamworks. It is a software that creates 3D objects from 2D strokes.It has since been discontinued because the developer has been working on a newer tool for Unity called Clayxels.

The other day I also came across a Japanese software called MagicalSketch 3D.

It is aimed toward children for 3D printing use. It looks like it works similarly to Smoothie 3D and Archipelis, except that the objects created don’t have to be symmetrical and you can paint directly onto the models you create in the program rather than having the image reference mapped as the texture. I think it is only one step up or similar to Paint 3D however (a free Windows 10 app).

A similar commercial software is called Teddy. This is available as a Unity plugin rather than a stand-alone program. I think it is based upon Teddy 3D which is an old discontinued software.

Possibly the most mainstream or well known type of this software for artists is Aartform Curvy3D

Curvy 3D uses sketch based modelling techniques and has digital sculpting capability. It has more features than the software I’ve listed above and has a higher price tag to reflect that, but it is on sale at the moment.

I’ve bought Curvy 3D 4 and it is great. Once you get the hang of it you can build some quick base meshes for sculpting and it has an inflate image to mesh feature which is useful. The developer is friendly and is working on improvements to the software for a 4.1 version.

Because I tried similar software such as Smoothie 3D and Archipelis Designer first I understood the working methods for Curvy 3D a little better than I would otherwise have with it being sketch / curve based.

There is also a mini version of Curvy 3D called Curvy 3D Go which is in early access on Steam.

A future version of a software called Dust3D may incorporate a “Marker Pen” sketch based modelling option as well as an interesting node based creation method that is designed to be easy to use for non-modellers.

Taya Conceptor is a 3D kitbashing sketch software. It used to be called Polybrush when it was in early development. HR Giger would have loved it. But seriously it may be one of the best looking 3D sketching applications I’ve come across so far. I may have to give it a go one day if I need to do Sci Fi squirrels or ornate detailed things.

Some addons for Blenders sculpting mode utilise the Grease pencil to help carve out parts of existing 3D models. These are still fairly experimental and in the early stages.

The 2.80 update will probably bring more innovation in this area. There are new addons for Grease Pencil being developed as we speak such as this one called Bumarin for drawing 3D meshes with grease pencil:

Reminds me a little of the ZSpheres approach in ZBrush.

Incidentally the ZBrush 2021.6 update coming on March 2nd will introduce some new sketch based modeling brushes and curve based snake hook brushes which will make blocking out shapes in 3D even more intuitive in the program than using ZSpheres. It looks very similar to the Curvy 3D and Smoothie 3D workflow with the balloon shape mask block-out tools.

Another new one I’ve found is QuickDRAW by a concept artist called Aleksandr Kilimnik.

Aleksandr Kilimnik and Jama Jurabaev are working on Blender add-ons that are aimed at making 3D modelling more intuitive, including some sketch based ones. Check out Quick Shape and QuickDeform. I think Quick Shape might be the successor to QuickDraw, but I’m not sure.

The Speedsculpt addon by Pitiwazou — Cédric Lepiller also has basic 3D sketching features in the latest update for Blender 2.8. Perhaps another one to try out.

SketchN’Carve is a new addon for Blender that uses grease pencil tools to convert drawings into 3D geometry.

Grid Modeler may be useful for an intuitive modeling workflow as well, especially for those used to a vector art like workflow with drawn shapes and boolean functions.

Voxels: 3D Pixel Art

There is another way you can utilize 2D pixel art skills and turn them into 3D models with programs such as SpriteStack and Magica Voxel.

This is a fun way to create 3D models which can also be exported and used in other 3D software.

There is also an add-on for Blender called Sprytile that lets you create 3D meshes using pixel tilesets.

Drawing in 3D Software

Nowadays artists are also exploring ways of bringing their drawings into a 3D space. This is useful for interactive presentations and animation work visual development. 2 ways I currently know of are a software called Mental Canvas and with the Grease Pencil feature in Blender.

Mental Canvas is aimed toward animation visual development artists (such as Sam Nassour) and studios.

Blender Grease Pencil is also being used for visual development and animation. The advantage with Grease Pencil is that it is free and available now, whereas Mental Canvas is not, you have to schedule a demo.

Blenders Grease Pencil strokes can also be converted into 3D mesh objects as I mentioned earlier, so it is being used as a conceptual 3D tool visualisation tool combined with the new physically based renderer known as Eevee.

Another interesting use of 2D images and drawings in 3D software is to help create poses for 3D models. Software such as Clip Studio Paint uses AI features to analyse drawings and photographs of poses and apply them to 3D rigged models.

Morpholio Trace and Shapr3D are tools on the iPad designed to combine sketch based 3D modeling with drawing tools. They are designed to be intuitive for designers and Illustrators to use with the apple pencil. These apps are primarily aimed toward engineering and industrial design sectors.

A new tool I’ve seen recently is web based and on iPad called Penzil. I saw 80 Level blogging about it here. It is very basic but might be a fun free introduction to the concept of drawing in 3D (although Blender’s Grease Pencil is free and more advanced).

2D images to 3D Meshes

If you are losing hope in sketch based 3D software there may be an alternative. Draw your image and turn it into a mesh. I recently figured out a way to do this to an extent using software that generates normal maps from sprites. I discovered that if you convert the normal map into a depth map (sometimes also called a height map or alpha image), you can use the depth map to generate a 3D mesh.

AI

Some work is also being done with AI to generate full 3D figures from photos which is interesting.

It is called PiFuHD: Multi-Level Pixel-Aligned Implicit Function for High-Resolution 3D Human Digitization. Currently there is a demo available on Google Research. I think if they made it into a usable program I would buy it, but I don’t know what their plans are. It would be a useful feature to make quick base meshes for sculpting, fashion, concept art, or 3D reference for artists.

Plasmo

This is in Beta but remember that digital analytic drawing I mentioned at the start of the article? Well this looks like the closest to that so far. Website: plasmo.ai

Please help support my blog

Liberapay . Buy me a coffee . Kofi . Payhip . Gumroad . PayPal