Cel Shading concepts

First off, I'd like to say a big 'hi' to all my friendly rivals participating in the current (as of May 1st) competition on the 3Dtotal forums. This tut, although possibly not that viable for your final entries, is for you ;D
Oh, and Tom, I've promised you some tuts in the past, so this should make up for it ;D

I came up with this cel shading technique while in a period of boredom and a concept art drought. it concentrates on the issues of other cel effects (like not being able to cast shadows, or having an intelligant outline). This technique should be easily ported to most other 3D packages if they have an algorithmic, light/shadow sensitive falloff routine. I can't port it, since I'm not rich enough to to be able to afford LW, or Maya, etc and can't really do it at 'work' or school. Anyway, on with the tut!

1. - model: here we have our base model, created concentrating on silhouette, and body mass, as opposed to topology - just a slightly different way of making a cartoony mesh; nothing special. It's subdivision btw.

2. - lights: I've rigged a simple lighting setup, making sure I've got some shadow and nice luminant gradients from 0 (black) to 255 (white). this is just to test out all the elements in this technique.

3. - base material: OK, this is the first real step in this technique. Seeing that you'll be wanting solid bands of shading, and maximum control (especially if you're doing a large scene) this will minimise hassle later on. Go into the material editor ('m') and create a stand-alone falloff named 'shading luminance base'. This will be used to create all the solid blocks of shading. Change it to 'shadow/light' falloff type and make sure that black is in the top slot, and white is on the bottom.

Now change the mix curve so that there's 4 points going like this:

1- x:0 y:0
1- x:0.45 y:0
1- x:0.55 y:1
1- x:1 y:1

now insert a copy of this falloff into it's own 2 slots. in the black slots falloff make the colours top = black and bottom=dark to medium gray. the mix curve should be:

1- x:0 y:0
1- x:0.2 y:0
1- x:0.3 y:1
1- x:1 y:1

for the black slots falloff make the colours top = light gray and bottom=near white. the mix curve should be:



1- x:0 y:0
1- x:0.7 y:0
1- x:0.8 y:1
1- x:1 y:1

Of course, you can change all this at your discretion, it's just a guide. You can add more or less shading bands, or change whatever the you want. One thing to try out is using much sharper gradients and enabling a supersampler to filter the aliased shading bands. ups the render time, but you can use more extreme forms of shading (huge long shadows for instance). I ended up switching to this later to get more of an authentic solid band than the softer look; it's all to do with what style you want.
blech! enough of the numbers already...

4. - shading materials: Now create a new standard material (mine is 'skin base shader'). Put up self illumination to 100%, no gloss/spec and put the diffuse colour to a medium/average of what you imagine the final will look like (this is just to help you out in the viewport).

next insert an instance of the 'shading luminance base' into the diffuse slot. There's two ways of colouring here. An easy to control globally way, or a more tweakable but no global control way. For the easy to control one, insert this falloff into a RGB multiply mat and simply tint it with the desired colour. you can then change the luminance values on all the materials in the scene by changing the base luminance shader and they keep their respective colours. For things like skin though, where you want to have some meaty red translucency where it's a base skin colour at the light source and more pinkish towards the darker areas or the such, then you use the second way which is simply using a copy of the luminance base and manually changing the colours in it. Your first shading material should look something like this:
repeat for any other materials.

5. - black outline: Make a simple material that is jet black in the diffuse, no gloss/spec and 100% self illum. OK, the trick in this technique is so simple, but surprisingly very hard to visually get a stuff up it's frightening ;D unlike virtually all the other cel shading techniques out there that use visually incorrect methods to create the outline in the material, mine uses a mesh base. Create a referance clone of you object that your cel shading at the same position of the normal mesh and apply the black material to it. On this referance, add a push modifier then a normals modifier. My push is set to around 5.5 but of course this is subject to scale differences. Set the normals modifier to flip normals, and looky there; you've got an outline - even in the viewport! Make sure you turn off cast shadows on this referance too so you don't get shadow artifacts - unless you've got a huge outline and the shadow wont fit properly that is.

 


6. - conclusion: now then, what makes this so great in comparison with all those other cel shading methods out there? well it's near as perfect as you can get in visual output with a clean, default max. It's simple to use, easy to create, and not that intensive to render out. It has the least amount of limitations, and the most flexible and versatile methodolagy meaning you can add hatching, doodling or any other type of effect easily (as seen in the last picture). Why not use a gradient ramp set to lighting you say? well some tuts do, but that means it doesn't output shadows, and is therefore useless for most applications. Why not use a secondary material or another falloff as the outline instead of another mesh? well that ALWAYS screws up on anything besides a sphere; try doing that on a cylinder and looking diagonally down on it - see, screwed ;D
Like anything in the world, there's something that's not quite right, making it prone to some form of abnormality. There's a whole heap of scientific and philosophic laws conspiring against the world to make that happen - so blame it on the scientists and philosophers ;D . Although minimal, in this case, it's that sometimes the outline can make thin antialiased lines when looking at extreme angles (this isn't really a stuff-up, just something that you may not be wanting for the style you're going for. It needs a minimum thickness cut off). umm... oh yeah, another is that the border mesh should be closed/capped and shouldn't have any holes as these make inverse lines (they curve the wrong way and you can see it's a normals flipped mesh). that's all I can think of and that I've seen... no real drawbacks, or limitations if your mesh is made 'correctly'.

What about animation you say? well because it's a reference, it'll deform to whatever you change the original to + then apply the normal and push modifiers. An easy rig is to either link, group, or even attach it (although I don't recommend attaching... too much work in most scenarios if you're planning on doing skeletal animation or similar).

Here's some other examples of what this can be used as a basis for (I thought I'd put the wireframe and bitmap based ones there as well to prove it's not a photoshop filter):

Comments, criticism, and sexual advances can be directed to mathius@geeklife.com If anyone discovers any other problems or limitations, please don't hesitate to contact me... oh, and my spelling isn't wrong, I'm Australian - and that's not an illness ;D

- thanks, and g'bye!

Fetching comments...

Post a comment