Basics
Sub-Materials are basically folders with layers inside them. They should have their most important controls exposed to the folder so you can change them easily. Sub-Materials can be imported into a material to quickly get a material down.
You can import a Sub-Material by clicking the corresponding button. This will open a popup where you can select a material from one of the categories. Then click OK to import it.
Creation
In the user preferences you can click the Sub-Material button. This will prompt you to open a new file. This is the file where the Sub-Materials are stored. You will find an explanation on how to add your own Sub-Materials in the file itself.