Wednesday, December 11, 2013

First Foray Into Physically-Based Shaders

Yesterday I made my first foray into physically-based shading (with some minor fixes and tweaks applied today).
For those of you who don't know, physically based shading is based on the principle that a surface can never reflect more light energy than it receives. It's actually much simpler than it sounds.
For diffuse, you can get away with just using regular Lambert (some games use Oren-Nayer, but I don't think that's at all necessary). For specular, things get much more complicated, and I won't get into details in this post.

One thing physically-based shaders are REALLY good at is having a wide range of materials - with a few parameters, you can fairly easily reproduce a lot of hard surfaces.
Anyway, here's a screenshot of my hard surface shader in action. On the left, three plastic spheres, in the middle, three polished metal spheres, and on the right three unpolished metal spheres. All nine spheres use the same shader (as does the ground underneath them).
I'll post more on the technique in a later post.

Tuesday, December 10, 2013

Whitepapers Must Die: A Rant

There's something I've always found infuriating about new game graphics techniques. The moment a technique is discovered, someone goes and writes up a whitepaper about it. Then, everyone else just links to the whitepaper as the gold standard for learning that technique.
I hate this. I don't know about everyone else, but I don't speak whitepaper.
The thing about academic texts is that there's almost as much of a focus on sounding professional as there is on conveying useful information. Clearly, there are plenty of people out there that find these useful - I for one, do not.
In my opinion, something like a personal blog is actually a far better place to convey these kinds of techniques.
On a personal blog, there is no pressure to maintain professionalism. When you're trying to convey information on a blog, you often go for a friendly or even humorous tone, and try to boil things down to a format most of your readers can easily understand - usually relaying useful anecdotal information such as engine specifics that the author encountered which would otherwise be considered too unrelated for a whitepaper. It's always been on personal blogs and websites that I'm able to gain a much deeper understanding of a subject.
Anyway, those are my two cents on the subject. Whitepapers should die. Rant over.

Monday, December 9, 2013

Better Skin Shading in Unity

In CG (both games and movies), there's something called the "uncanny valley". This refers to the trend where as human rendering becomes more and more realistic, at some point it suddenly stops looking realistic and instead starts looking creepy.
One technique seeing a rise in the AAA games industry is known as "sub surface scattering". I won't go into details, but the basics of it are this: light enters the skin, is scattered about, absorbed depending on how far it travels, and then exits (because skin is not actually opaque - it's semi-transparent).
There are many ways to go about this effect. There's texture space diffusion, screen space scattering, and (my personal favorite) pre-integrated skin shading.
What is pre-integrated skin shading? It's a very fast method of faking subsurface scattering, by baking the results into a lookup texture, and using the lookup texture as a lighting ramp.
Why is it important? Why can't we just use our old friends, Lambertian lighting and Blinn-Phong for shading our characters?
Because they end up looking like mannequins. Remove the skin color, and you're left with a statue.
Let's take a look at some pictures.
Here's the arm of a soldier character I'm using for testing. On the left, Blinn-Phong shading is used. On the right, pre-integrated skin shading is used.
I see the one on the left far too often, especially in indie/Unity games.

So how is it done?
First, you have your lookup texture. Here's your "realistic" lookup texture, using the lookup texture generator on an AltDevBlogADay article about it.
The U coordinate is lighting (from 0 to 1), while the V coordinate is basically skin depth (from 0 at the bottom, allowing no light through, to 1 at the top, allowing the most light through)
I don't like this one, however. In my opinion, realism is always trumped by what looks right. And to me, fiddling with that texture in Photoshop is what looks right.
Exaggerated? Definitely. Looks good? Hell yeah.
The article as linked above uses DDX and DDY to calculate skin depth, as well as tex2Dlod to sample a "blurred" version of the normal map. This imposes a lot of restrictions on the shader - too many, in my opinion.
Instead of using DDX/DDY, I'm going to use a plain old texture map for the same purpose. This puts the burden on the artist to paint skin depth, which is fine by me since that affords a lot more control over the final result anyway (and makes our shader more versatile, since a skin depth of zero gives us regular Lambert shading - we might give a character's clothes a value of zero, for instance). I'm also going to skip the tex2Dlod step, just using the normal map as-is.
And what does this give me? A realistic character shader which is miles ahead of your average blinn-phong shading, and is efficient enough to run on mobile platforms without breaking a sweat.

Hope this gives some people some ideas.
Happy gaming, everybody.

Sunday, December 8, 2013

Motion Blur in Unity Part 3: Rendering Motion Vectors

Today we're going to be rendering the motion vector buffer used for motion blur. To do this, we'll set up a special camera to render the scene with a replacement shader.

Image Effect Script

Before we continue, let's create an image effect skeleton.

using UnityEngine;
using System.Collections;

public class MotionBlurEffect : MonoBehaviour
{
 // the shader used to render motion vectors
 public Shader MotionVectorShader;
 
 // the shader used to render motion blur
 // will do so in two passes:
 // first pass: render motion blur
 // second pass: blend half-resolution blurred image with full resolution screen buffer
 public Shader MotionBlurShader;
 
 private Material motionBlurMaterial;
 
 void Awake()
 {
  motionBlurMaterial = new Material( MotionBlurShader );
 }
 
 void OnRenderImage( RenderTexture src, RenderTexture dest )
 {
  Graphics.Blit( src, dest );
 }
}
This sets up the material we will use during Blit, and implements OnRenderImage (at the moment it does nothing, simply blits the source texture directly to the destination).
Next, we'll set up the camera to render motion vectors.

private Camera motionVectorCamera;
 
void Awake()
{
 motionBlurMaterial = new Material( MotionBlurShader );
 
 // set up the motion vector camera
 motionVectorCamera = new GameObject("MotionVectorCamera").AddComponent<Camera>();
 motionVectorCamera.transform.parent = transform;
 motionVectorCamera.transform.localPosition = Vector3.zero;
 motionVectorCamera.transform.localRotation = Quaternion.identity;
        motionVectorCamera.backgroundColor = new Color( 0f, 0f, 0f, 0f );
        motionVectorCamera.renderingPath = RenderingPath.Forward;
// use motion vector replacement shader // we only want to render objects with matching 'RenderType' shader tag motionVectorCamera.SetReplacementShader( MotionVectorShader, "RenderType" ); // disable camera // we'll manually render it when needed motionVectorCamera.enabled = false; }
Let's also add a LayerMask field to this script. This LayerMask will affect which objects are to be motion blurred.

// affects which layers are to be motion blurred (default is everything)
public LayerMask MotionBlurLayers = ~0;

// ...

void Awake()
{
 motionBlurMaterial = new Material( MotionBlurShader );
 
 // set up the motion vector camera
 motionVectorCamera = new GameObject("MotionVectorCamera").AddComponent<Camera>();
 motionVectorCamera.transform.parent = transform;
 motionVectorCamera.transform.localPosition = Vector3.zero;
 motionVectorCamera.transform.localRotation = Quaternion.identity;
 motionVectorCamera.backgroundColor = new Color( 0f, 0f, 0f, 0f );
        motionVectorCamera.renderingPath = RenderingPath.Forward;
 motionVectorCamera.cullingMask = MotionBlurLayers;
 
 // use motion vector replacement shader
 // we only want to render objects with matching 'RenderType' shader tag
 motionVectorCamera.SetReplacementShader( MotionVectorShader, "RenderType" );
 
 // disable camera
 // we'll manually render it when needed
 motionVectorCamera.enabled = false;
}
And finally, let's render this camera to a half-resolution render texture in preparation for motion blur.

void OnRenderImage( RenderTexture src, RenderTexture dest )
{
 // make sure motion vector camera matches current camera
 motionVectorCamera.fieldOfView = camera.fieldOfView;
 motionVectorCamera.nearClipPlane = camera.nearClipPlane;
 motionVectorCamera.farClipPlane = camera.farClipPlane;
 
 // get temporary render texture for motion vectors
 RenderTexture motionVectors = RenderTexture.GetTemporary( src.width / 2, src.height / 2, 0, RenderTextureFormat.ARGBFloat );
 
 // render motion vector camera
 motionVectorCamera.targetTexture = motionVectors;
 motionVectorCamera.Render();
 
 Graphics.Blit( src, dest );
 
 // release temporary render textures
 RenderTexture.ReleaseTemporary( motionVectors );
}
Now that we have much of the framework in place to render our motion blur, let's get started on the actual motion vector shader.

Motion Vector Shader

We'll start with a simple vertex/fragment skeleton for our shader.


Shader "Custom/RenderMotionVector" {
 Properties {
 }
 SubShader {
  Tags { "RenderType"="Opaque" }
  LOD 200
  
  Pass {
  CGPROGRAM
  #pragma vertex vert
  #pragma fragment frag
  
  #include "UnityCG.cginc"

  struct v2f {
   float4 position : SV_POSITION;
  };
  
  v2f vert( appdata_full v )
  {
   v2f o;
   
   o.position = mul( UNITY_MATRIX_MVP, v.vertex);
   
   return o;
  }
  
  float4 frag( v2f i ) : COLOR
  {
   return float4( 1,1,1,1 );
  }
  
  ENDCG
  }
 }
}
This does hardly anything yet, simply outputting White.
Let's add to our v2f structure the current and last positions of the vertex. We'll calculate the last position via the matrix our helper script is passing to the shader.

Shader "Hidden/RenderMotionVector" {
 Properties {
 }
 SubShader {
  Tags { "RenderType"="Opaque" }
  LOD 200
  
  Pass {
  CGPROGRAM
  #pragma vertex vert
  #pragma fragment frag
  
  #include "UnityCG.cginc"
  
  // the model*view matrix of the last frame
  float4x4 _Previous_MV;

  struct v2f {
   float4 position : SV_POSITION;
   float4 curPos : TEXCOORD0;
   float4 lastPos : TEXCOORD1;
  };
  
  v2f vert( appdata_full v )
  {
   v2f o;
   
   o.position = mul( UNITY_MATRIX_MVP, v.vertex);
   o.curPos = o.position;
   o.lastPos = mul( UNITY_MATRIX_P, mul( _Previous_MV, v.vertex ) );
   
   return o;
  }
  
  float4 frag( v2f i ) : COLOR
  {
   return float4( 1,1,1,1 );
  }
  
  ENDCG
  }
 }
}
At first glance, it might seem odd that we're duplicating the 'position' field. This is done because we cannot read from the SV_POSITION semantic. We can, however, read from the TEXCOORD semantic, so we duplicate the position and pass it as TEXCOORD0 (the previous position is passed as TEXCOORD1).
Next, per pixel we will calculate the difference between curPos and lastPos and output this.

float4 frag( v2f i ) : COLOR
{
 float2 a = (i.curPos.xy / i.curPos.w) * 0.5 + 0.5;
 float2 b = (i.lastPos.xy / i.lastPos.w) * 0.5 + 0.5;
 float2 oVelocity = a - b;
 return float4( oVelocity.x, -oVelocity.y, 0, 1 );
}
This is the basis of our motion vector shader. However, remember that our SetReplacementShader call specified the "RenderType" tag. Since this subshader was set to "Opaque", our shader will only replace objects with a shader set to "Opaque". In order to replace other types, you must write more than one subshader for these types. Many of these are simply minor variations. Here is the full shader code with four variations: Opaque, TransparentCutout, TreeBark, and TreeLeaf

Shader "Hidden/RenderMotionVector" {
 Properties {
 }
 SubShader {
  Tags { "RenderType"="Opaque" }
  LOD 200
  
  Pass {
  CGPROGRAM
  #pragma vertex vert
  #pragma fragment frag
  
  #include "UnityCG.cginc"
  
  // the model*view matrix of the last frame
  float4x4 _Previous_MV;

  struct v2f {
   float4 position : SV_POSITION;
   float4 curPos : TEXCOORD0;
   float4 lastPos : TEXCOORD1;
  };
  
  v2f vert( appdata_full v )
  {
   v2f o;

   o.position = mul( UNITY_MATRIX_MVP, v.vertex);
   o.curPos = o.position;
   o.lastPos = mul( UNITY_MATRIX_P, mul( _Previous_MV, v.vertex));
   
   return o;
  }
  
  float4 frag( v2f i ) : COLOR
  {
   float2 a = (i.curPos.xy / i.curPos.w) * 0.5 + 0.5;
   float2 b = (i.lastPos.xy / i.lastPos.w) * 0.5 + 0.5;
   float2 oVelocity = a - b;
   return float4( oVelocity.x, -oVelocity.y, 0, 1 );
  }
  
  ENDCG
  }
 }
 SubShader {
  Tags { "RenderType"="TreeBark" }
  LOD 200
  
  Pass {
  CGPROGRAM
  #pragma vertex vert
  #pragma fragment frag
  
  #include "UnityCG.cginc"
  #include "TerrainEngine.cginc"
  
  // the model*view matrix of the last frame
  float4x4 _Previous_MV;

  struct v2f {
   float4 position : SV_POSITION;
   float4 curPos : TEXCOORD0;
   float4 lastPos : TEXCOORD1;
  };
  
  v2f vert( appdata_full v )
  {
   v2f o;

   v.vertex.xyz *= _Scale.xyz;
   v.vertex = AnimateVertex( v.vertex, v.normal, float4( v.color.xyz, v.texcoord1.xy));
   v.vertex = Squash( v.vertex );

   o.position = mul( UNITY_MATRIX_MVP, v.vertex);
   o.curPos = o.position;
   o.lastPos = mul( UNITY_MATRIX_P, mul( _Previous_MV, v.vertex));
   
   return o;
  }
  
  float4 frag( v2f i ) : COLOR
  {
   float2 a = (i.curPos.xy / i.curPos.w) * 0.5 + 0.5;
   float2 b = (i.lastPos.xy / i.lastPos.w) * 0.5 + 0.5;
   float2 oVelocity = a - b;
   return float4( oVelocity.x, -oVelocity.y, 0, 1 );
  }
  
  ENDCG
  }
 } 
 SubShader {
  Tags { "RenderType"="TransparentCutout" }
  LOD 200
  
  Pass {
  ZWrite Off
  AlphaTest Greater [_Cutoff]
  CGPROGRAM
  #pragma vertex vert
  #pragma fragment frag
  
  #include "UnityCG.cginc"

  sampler2D _MainTex;
  
  // the model*view matrix of the last frame
  float4x4 _Previous_MV;

  struct v2f {
   float4 position : SV_POSITION;
   float4 curPos : TEXCOORD0;
   float4 lastPos : TEXCOORD1;
   float4 tex : TEXCOORD2;
  };
  
  v2f vert( appdata_base v )
  {
   v2f o;
   
   o.position = mul( UNITY_MATRIX_MVP, v.vertex);
   o.curPos = o.position;
   o.lastPos = mul( UNITY_MATRIX_P, mul( _Previous_MV, v.vertex));
   o.tex = v.texcoord;
   
   return o;
  }
  
  float4 frag( v2f i ) : COLOR
  {
   float2 a = (i.curPos.xy / i.curPos.w) * 0.5 + 0.5;
   float2 b = (i.lastPos.xy / i.lastPos.w) * 0.5 + 0.5;
   float2 oVelocity = a - b;
   return float4( oVelocity.x, -oVelocity.y, 0, tex2D( _MainTex, i.tex.xy ).a );
  }
  
  ENDCG
  }
 }
 SubShader {
  Tags { "RenderType"="TreeLeaf" }
  LOD 200
  
  Pass {
  ZWrite Off
  AlphaTest Greater [_Cutoff]
  CGPROGRAM
  #pragma vertex vert
  #pragma fragment frag
  
  #include "UnityCG.cginc"
  #include "TerrainEngine.cginc"

  sampler2D _MainTex;
  
  // the model*view matrix of the last frame
  float4x4 _Previous_MV;

  struct v2f {
   float4 position : SV_POSITION;
   float4 curPos : TEXCOORD0;
   float4 lastPos : TEXCOORD1;
   float4 tex : TEXCOORD2;
  };
  
  v2f vert( appdata_full v )
  {
   v2f o;
   
   ExpandBillboard (UNITY_MATRIX_IT_MV, v.vertex, v.normal, v.tangent);
   v.vertex.xyz *= _Scale.xyz;
   v.vertex = AnimateVertex( v.vertex, v.normal, float4( v.color.xyz, v.texcoord1.xy));
   v.vertex = Squash( v.vertex );

   o.position = mul( UNITY_MATRIX_MVP, v.vertex);
   o.curPos = o.position;
   o.lastPos = mul( UNITY_MATRIX_P, mul( _Previous_MV, v.vertex));
   o.tex = v.texcoord;
   
   return o;
  }
  
  float4 frag( v2f i ) : COLOR
  {
   float2 a = (i.curPos.xy / i.curPos.w) * 0.5 + 0.5;
   float2 b = (i.lastPos.xy / i.lastPos.w) * 0.5 + 0.5;
   float2 oVelocity = a - b;
   return float4( oVelocity.x, -oVelocity.y, 0, tex2D( _MainTex, i.tex.xy ).a );
  }
  
  ENDCG
  }
 }
}
This should support most, if not all, cases.
If you don't really know what's going on in the Tree variations, don't worry. I'm not sure I understand, either :). The vertex shader code was copied over from the builtin tree shaders.

We're done with our replacement shader. As a final step, simply drag the shader onto the Motion Vector Shader field of the image effect.
In the next post, I'll cover creating the two-pass motion blur shader, and hooking it up to our image effect script.

Saturday, December 7, 2013

Motion Blur in Unity Part 2: The Model*View Matrix

The way our motion vectors are typically calculated is that, instead of just projecting a vertex with the model*view*projection matrix, we project it twice - once with the MVP matrix of the current frame, and once with the matrix of the last frame. Then, we calculate the displacement between them.
Unity, however, does not provide the model*view matrix of the last frame. So, we'll need to calculate it ourselves.
On the surface, this sounds easy. You can easily calculate the projection matrix of any camera in Unity, and a model matrix is just basically a local-to-world matrix, right? Yes. Usually. But in Unity, it's not so simple.
In Unity, how the model matrix is calculated depends on a wide array of factors. Whether it's a skinned or non-skinned mesh, whether it's uniform or non-uniformly scaled, whether it's static, whether it's statically batched, etc. The "local-to-world matrix" concept quickly breaks down.
I should note before we continue, that the following cases were discovered through personal trial and error.

Model Matrix Cases

So when is a model matrix just a local-to-world matrix? As it turns out, there is one case when this is true - when the model is not skinned, isn't statically batched, and is uniformly scaled.
In all other cases, it will not be.
So first, what happens if the model is not uniformly scaled? In this case, our model matrix considers the scale of the object to just be (1,1,1).
Also note that we need to recursively multiply scale by each parent.
With this in mind, let's build a function to calculate scale for our matrix.

Vector3 calculateScale()
{
 Vector3 scale = Vector3.one;
 
 // the model is uniformly scaled, so we'll use localScale in the model matrix
 if( transform.localScale.x == Vector3.one * transform.localScale.x )
 {
  scale = transform.localScale;
 }
 
 // recursively multiply scale by each parent up the chain
 Transform parent = transform.parent;
 while( parent != null )
 {
  scale = new Vector3( scale.x * parent.localScale.x, scale.y * parent.localScale.y, scale.z * parent.localScale.z );
  parent = parent.parent;
 }
 return scale;
}
This function starts with a scale value which depends on whether the transform is uniformly or non-uniformly scaled. Then, it recursively multiplies by the scale of each parent.
Let's also start putting together the function to calculate our model matrix.

Matrix4x4 calculateModelMatrix()
{
 return Matrix4x4.TRS( transform.position, transform.rotation, calculateScale() );
}
This now handles non-static, non-skinned meshes.
Now, what about static meshes?
Well, unfortunately here's where we get a bit project-specific. If your project has static batching disabled, then you don't have to worry about this. However, if your project does have static batching enabled, for static meshes Unity actually pre-transforms the geometry and simply uses identity as the model matrix.
I'm going to assume your project has static batching enabled (it is enabled by default in Unity). If it doesn't, you can skip this portion.

Matrix4x4 calculateModelMatrix()
{
 if( renderer.isPartOfStaticBatch )
 {
  return Matrix4x4.identity;
 }
 return Matrix4x4.TRS( transform.position, transform.rotation, calculateScale() );
}
Note that isPartOfStaticBatch does not seem to work as expected. That is, I found it reports true even if static batching is disabled. I do not know whether this is a bug or not.
So now our code handles regular and static meshes. But what about skinned meshes?
This one took me a while to figure out, but eventually I did. Skinned meshes for one act a bit like non-uniformly scaled meshes for one, and for another they do not use the transform the skinned mesh renderer component is attached to. Instead, they use the root bone transform.

Matrix4x4 calculateModelMatrix()
{
 if( renderer is SkinnedMeshRenderer )
 {
  Transform rootBone = ((SkinnedMeshRenderer)renderer).rootBone;
  return Matrix4x4.TRS( rootBone.position, rootBone.rotation, Vector3.one );
 }
 if( renderer.isPartOfStaticBatch )
 {
  return Matrix4x4.identity;
 }
 return Matrix4x4.TRS( transform.position, transform.rotation, calculateScale() );
}
And now, we've finally got an accurate model view matrix for all cases.
So, now that we can calculate the model matrix, let's create a helper script.

The Helper Script

Our script will keep track of the current and last model*view matrices, and will pass the matrix of the last frame to our material. Note that even if our shader does not have the matrix as a property, it will still store the matrix. We'll use this to our advantage when we go to render it with a replacement shader, which will be able to make use of this stored value.
We'll use OnWillRenderObject to do this. Later on, our image effect will create a secondary camera to render the motion vector. We'll call it "MotionVectorCamera" so that our matrix helper script can detect which camera is rendering it (we won't update the matrix if it isn't being rendered by our motion vector camera).

using UnityEngine;
using System.Collections;

public class MotionBlurMatrixHelper : MonoBehaviour
{
 private Matrix4x4 lastModelView = Matrix4x4.identity;
 
 void OnWillRenderObject()
 {
  if( Camera.current.name != "MotionVectorCamera" ) return;
  
  foreach (Material mat in renderer.materials)
  {
   mat.SetMatrix("_Previous_MV", lastModelView);
  }
  
  Matrix4x4 view = Camera.current.worldToCameraMatrix;
  Matrix4x4 model = calculateModelMatrix();
  
  lastModelView = view * model;
 }
 
 Matrix4x4 calculateModelMatrix()
 {
  if( renderer is SkinnedMeshRenderer )
  {
   Transform rootBone = ((SkinnedMeshRenderer)renderer).rootBone;
   return Matrix4x4.TRS( rootBone.position, rootBone.rotation, Vector3.one );
  }
  if( renderer.isPartOfStaticBatch )
  {
   return Matrix4x4.identity;
  }
  return Matrix4x4.TRS( transform.position, transform.rotation, calculateScale() );
 }
 
 Vector3 calculateScale()
 {
  Vector3 scale = Vector3.one;
  
  // the model is uniformly scaled, so we'll use localScale in the model matrix
  if( transform.localScale.x == Vector3.one * transform.localScale.x )
  {
   scale = transform.localScale;
  }
  
  // recursively multiply scale by each parent up the chain
  Transform parent = transform.parent;
  while( parent != null )
  {
   scale = new Vector3( scale.x * parent.localScale.x, scale.y * parent.localScale.y, scale.z * parent.localScale.z );
   parent = parent.parent;
  }
  return scale;
 }
}

Now that our shader has access to the model*view of the previous frame, we can get started on the motion vector shader. In the next post, I will cover creating the replacement shader and using it in an image effect script.

Friday, December 6, 2013

Unity Multiplayer Games by Packt Publishing Available For Preorder

Hey everybody!
My new book, Unity Multiplayer Games, published by Packt Publishing, is now available for preorder and is expected sometime this month.
This book covers everything you wanted to know about multiplayer games in Unity, with sample applications for every chapter to teach you by concrete example. Learn to use Unity Networking, Photon, Player.IO, and more, and even delve into some of the deeper topics like server-authoritative networking.
Preorder it here!
Happy gaming everyone.

Motion Blur In Unity Part 1: Overview

Welcome to my series on implementing motion blur in Unity. This is something I managed to create after days of trial and error, and I must say it's an incredibly nifty effect.
Before we get into the code, I'll start by talking about what I mean when I talk about motion blur, and how it works.

Motion Blur? Doesn't Unity Have That?

The answer is yes. Kind of. Not really.
Unity has a few motion blur effects. There's your traditional accumulation motion blur (which, despite the name, doesn't really look like motion blur - in the end, it ends up looking very dream-like), and a variety of so-called "camera motion blur" effects (none of which I could get working correctly) which try to simulate motion blur of the camera moving.
What it doesn't have is a special kind of motion blur employed in games you've likely poured hours into and not even noticed. I'm not sure what the technical term is (UDN refers to it as "per-object motion blur"), so I'll be referring to it as Screen Space Velocity Blur from this point on. This is not the technical term, but it makes for an easy term to remember (SSVB).
SSVB is used in a lot of games. Unreal Engine 3 features this technique, and therefore is used in nearly every Unreal game on the market. CryEngine also features this technique.
SSVB does two things: it produces realistic motion blur when the camera moves, and also features plausible per-object motion blur.

So How Does It Work?

SSVB works by rendering a velocity buffer. This velocity buffer contains screen-space velocity, rendered per pixel. In order to produce this velocity, the velocity buffer shader needs access to the model view of the current frame, and of the last frame. It then projects each vertex using both matrices, finds the displacement, and uses this as the velocity. The velocity is generally encoded in a floating-point buffer in the Red and Green channels (X and Y velocity).
Finally, the motion blur shader samples the velocity per-pixel. It uses this to step along a line, sampling pixels along the velocity vector. The final pixel output is the average of each sampled pixel.
There are a number of changes which can be made here, and which I implemented for performance. In Unreal, they render velocity to a half-resolution image, perform motion blur in half-resolution, and then blend the motion blurred image per-pixel with the full resolution image based on velocity magnitude (so areas which are motion blurred use the half-resolution image, while areas which are not blurred use the full resolution image)

It's a lot of work to set up in Unity, but it's definitely possible.
In the next post, we'll be deconstructing the many quirks of the model*view matrix in Unity, and creating a helper script to feed the current and last model*view matrices to our shader.

Thursday, December 5, 2013

Unity Optimization Myths

Something has been bugging me as of late. It comes from the darkest corners of the Unity development community.
I'm talking about misguided optimization.

It astounds me just how many bizarre "tips" I've seen for making Unity games run faster or smaller. Today I'm going to be listing a few, and talking about why I disagree with them.

Mipmaps Are Bad

I've seen a blog or two mentioning that mipmaps take up more memory than just the texture itself, and therefore disabling mipmaps is better.
The truth is, yes mipmaps do take up memory. They do not, however, take up much.
With the next generation of consoles released, the now-last generation of consoles seem dated now - and it's because they have been for years. For instance, the Xbox 360 features 512MB total RAM - this RAM is shared by CPU and GPU.
As consoles become more dated, mobiles are quickly catching up. For instance, the Nexus 7 features 1GB of RAM. While much of it is taken up by the OS, that still leaves a bit more than the 512MB offered by the Xbox 360.
Despite the 512MB limitation, mipmaps are still used quite heavily on the Xbox 360, and even megatextures as seen in the first-person shooter RAGE.
Mipmaps can also speed up rendering, as they can increase locality of texel access and reduce cache thrashing (especially important as GPUs tend to have small caches).
In general, if you need to reduce VRAM usage, reducing texture size or reusing textures is what you'll want to try first, before you consider disabling mipmaps.

Lower Poly Is Better

Many people love to advocate low poly counts. It sounds like it would make sense - fewer triangles and vertices is less for the GPU to process, right?
However, these days, GPUs are incredibly fast at processing raw geometry, even mobile GPUs. That's not to say you should go wild with polygons - you'll still want to keep polycounts reasonable, but don't break your brain trying to reduce polys - you're far more likely to encounter issues with draw calls and fillrate long before you hit poly limits.

Always Keep Temporary Variables

This one I suppose is borne out of a misunderstanding of the GC and how .NET handles memory. I've seen people suggest to keep temporary variables whenever possible to reduce memory allocations, such as turning this code:

void Update()
{
 Vector3 x = calculateSomething();
}
Into this:

Vector3 x;
void Update()
{
 x = calculateSomething();
}

Not only is this untrue, it can also make code significantly less readable. Atomic types like floats, ints, and bools, are not garbage collected. They are allocated on the Stack, whereas Objects (instances of a Class) are placed on the Heap. The Stack is not garbage collected, but the Heap is. Allocating value types will never trigger garbage collection, allocating objects will.
Keeping value types as temporary variables will not afford any performance improvement whatsoever.

Use FixedUpdate

This one is kind of mixed. The idea is good, the implementation is not.
Some people have mentioned getting performance improvements by moving some logic, such as AI, to FixedUpdate rather than Update, and modifying the FixedUpdate timestep.
Thing is, FixedUpdate is where physics is supposed to take place. That's why it runs on a fixed timestep - so that you don't get odd behavior like tunneling when framerate drops.
In order to afford any benefit, your fixed timestep has to be smaller than your frametime. So if our game is running at 30 FPS, our FixedUpdate would have to run less than 30 FPS in order to afford any improvement whatsoever. If you use physics, this increases the likelihood of tunneling and other odd behaviour.
The better option, if you absolutely need to reduce how often code executes, is to keep track of a frame counter and only execute your update code once every 'n' frames (which is perfectly acceptable for AI routines). This keeps physics timestep independent from AI timestep, as it should be.

Declare Foreach Iteration Variables Beforehand

And lastly, I just today encountered a discussion on foreach and how it allocates memory. Someone suggested that, in order to eliminate the allocation, you should declare the iteration variable before the loop.
That is completely and utterly false (luckily someone more knowledgeable was present to correct the original poster of the suggestion).
The reasoning was that this:

foreach( SomeClass x in SomeCollection )
{
 // do stuff
}
Would allocate "SomeClass x" for each entry in the collection (it doesn't). And that this:

SomeClass x;
foreach( x in SomeCollection )
{
 // do stuff
}
Would fix the allocation (it doesn't).
Consider the situation where we have this instead (a perfectly plausible situation):

foreach( Vector3 v in SomeCollection )
{
 // do stuff
}
This still triggers a heap allocation. Not because it's allocating Vector3's (remember, those are stored on the Stack, not the heap), but because it's allocating an enumerator to iterate over the collection.
The correct solution is to use a regular for loop instead of a foreach. Note that you don't have to do this everywhere - just where it counts.

And that's biggest secret of optimization. You should focus your efforts on optimizing where it counts. If your login routine generates GC calls, it doesn't matter that much - because login doesn't happen regularly during the lifespan of the game, certainly not every frame, but AI or gameplay for example are much better candidates as they run frequently.

Anyway, those are my thoughts for today. I'm tired.
Happy game development, everybody.

New Blog!

Woot, new blog!
After days of thoughts and words popping up in my head, I couldn't take it anymore. Twitter, with the 140 character limit, barely provides room for a few words, Google Plus just doesn't feel the right place for what I have to say, and Facebook - ugh, Facebook.

Here I'll be posting whatever thoughts come to mind about game development, Unity, occasional gaming, and whatever else I want to throw up on the internet.

Looking forward to it :)