Vertex texture sampling notes

Vertex Shader 3.0 supports texture sampling. Several books and articles I’ve come across reference this feature and provide HLSL example code for how to do it, but none explain how you’re supposed to set up you DirectX pipeline to be able to make it work. It’s taken me hours and hours to get things working, so I thought I’d share what I had to do to make it happen.

Why sample a texture in a vertex shader? Displacement mapping, passing large arrays of data to a VS so you can deform geometry (something you could only previously do in a pixel shader). Why NOT sample textures in a vertex shader? The performance is much worse than it would be with a pixel shader. Beware.

The HLSL function that performs texture sampling in VS is tex2Dlod(). It takes two arguments, your sampler and a float4 in the format (u, v, mip1, mip2). Set mip1 and mip2 to 0 unless you want to do mipmap blending. Most of the time you’re going to call this function like this:

float4 foo = tex2Dlod(DisplaceSampler, float4(vTexCoord0.x, vTexCoord0.y, 0.0, 0.0));

When you declare your sampler in your HLSL file you don’t need to do anything special. Typically tho you don’t want to do any blending between points in the texture, so specify your sample with POINT sampling:

texture g_Texture;
sampler DisplaceSampler = sampler_state
{
texture = ;
MinFilter = POINT;
MagFilter = POINT;
MipFilter = NONE;
AddressU = CLAMP;
AddressV = CLAMP;
};

Current hardware only supports floating point texture formats for vertex texture sampling. When you load the texture that you’re going to sample in the VS pass D3DFMT_A32B32G32R32F or D3DFMT_R16F as the format.

HRESULT hres = D3DXCreateTextureFromFileEx(
d->GetDevice(),
name,
D3DX_DEFAULT, D3DX_DEFAULT,
D3DX_DEFAULT,
D3DUSAGE_RENDERTARGET,
D3DFMT_A32B32G32R32F,
D3DPOOL_DEFAULT,
D3DX_DEFAULT,
D3DX_DEFAULT,
NULL,
NULL,
NULL,
&texture);

The docs say to use RENDERTARGET for vertex sampled textures but I’ve monkeyed with this value and can’t get it to change anything.

I could only get tex2Dlod() working by using hardware vertex processing with the HAL renderer. Software vertex processing is returning 0 for the texture every time, so I guess I still have some bugs to work out. I had an email exchange with Wolfgang Engel about this and he said that the HLSL compiler at one time had some bugs with how it handled the tex2Dlod() function and that they might still be in there. In the mean time I’m just doing without vertex debugging I guess.

g_pD3D->CreateDevice(
D3DADAPTER_DEFAULT,
D3DDEVTYPE_HAL,
hWnd,
D3DCREATE_HARDWARE_VERTEXPROCESSING,
&g_d3dpp,
&g_pd3dDevice );

To pass the texture to the vertex shader you give it to the ID3DXEffect object like you would any other texture:

hr = g_pEffect->SetTexture( “g_Texture”, texture);

That’s about it. The biggest sticking point for me was finally trying it on the hardware, because for whatever reason my REF renderer isn’t doing tex2Dlod() correctly. Usually things are the other way around right? It works in the debugger and fails outside the debugger? Oh well. 😛