I agree. This complicates things. 

The particular problem that you brought up: we can likely classify it as bad usage of naarad APIs. If the user has already tagged tests using signal_start and signal_stop, should they be allowed to run another workload?  

Do you want to replicate the current functionality of bin/naarad for now and then we fix the complications going forward? Or we can sit together and think about all the conflicting functionalities now and figure out the correct fix.