Yes, it is. CloudFormation is free. AWS only charges for those services which you provision using CloudFormation.