Remove the board, the line on the table in relation to the front of the table is the angle that the blade will cut best. Using a t-bevel get this angle and use it to align your fence. Every blade will be different and many will have very minimal or no drift. The blade guides are a very essential part of the proper operation of the bandsaw. The quality of these play a major role in the type of cut you will get.
Most small bandsaws have a metal guide on each side of the blade and a bearing on the backside.
The side guides keep the blade from wandering, and the bearing in back keeps the blade from running off the tires. It is best to have the side guides as close as possible to the blade, unfortunately the metal blocks cannot be positioned as close as should be. Many people make wooden blocks out of Lignum Vitae or purchase after market guides called Cool Blocks which can be made to just touch the blades. This is highly reccommended. The better quality saws have bearings on the side also.