The Ambient Occlusion Pass
For the rendering the occlusion pass, a mental ray surface shader named "Ambient/Reflective Occlusion(base)” should be assigned to the scene geometry. As surface shaders behave like regular 3dsmax maps, they should be attached to a material in order to be assigned to geometry. So it would be appropriate to use the Ambient/Reflective Occlusion(base) map in a native mental ray material's Basic Shaders>Surface slot. Now the mental ray material can be assigned to the scene geometry to render the occlusion pass. Before assigning the material let's take a look at the shader's parameters.
I will be dealing with Samples and Type sections of the shader. Default sample amount of 16 would not be enough for a high quality image and will tend to flicker in animated sequences so it should be cranked up to at least 64 for smooth GI look for this image. Of course it all depends on your scene's needs, how much quality GI you want and how much time you have for rendering your scene. Spread parameter determines how the GI samples are scattered and default values would suffice in this case.
You only need to set the type of the shader which is 0 for standard occlusion, 1 is for occlusion with environment map and 2 is for bent normals mode with RGB values as occlusion which can only be used in an appropriate compositing software.
If Type is set to mode 0 the occlusion render will be a grayscale image based on the default Bright and Dark values of the shader which can be modified in RGB color space to obtain a tinted rendering in case you need one.
I want the rendering to reflect the environment colors of the scene so I will set the shader type to mode 1.
Tip: By the way if you do not want to loose the original materials in the scene, a quick way to assign a single material to all scene objects is by enabling material override from mental ray's Processing Panel and dragging the desired shader to designated slot.
After making the adjustments and assigning the material to all the scene members, the pass takes a minute and a half to render. The longest rendering time for a pass so far. The pass looks like:
Moving on to compositing, I'll be using Photoshop here but the method is similar between all the applications that support various blending or transfer modes between layers like Multiply, Overlay, Screen and so on..
We start the compositing by placing the Ambient Color Pass as the base layer:
Then we overelay the Ambient Occlusion Pass with "Overlay” blending mode:
The image looks a bit dull without the Specular Pass layer. Put it on top with the "Screen” transfer mode to complete