Basics

You can find the vector fx in the layer settings. Vector FX can affect the entire Layer or just one channel, meaning you can choose what channel you want to add vector fx to by choosing it from the channel dropdown. Vector FX are basically effects applied to the mapping of the textures you have in your channel, meaning you need to add one to see the effects.

Adding

You add a vector effect by choosing it from the dropdown or the thumbnail preview. Now you can change its settings, move the effects up and down in the stack and hide or remove them.