Tweaked the clock enable logic to fix some wierdness

Change-Id: I3d112f28ab5f3293f3d19c6ae418d981bb3c1b93