If your employer or agency disciplines you for refusing to work, you can apply for compensation from an Employment Tribunal. If there is union recognition where you work you should see your union safety rep. If there is no safety representative you should raise your concerns directly with your employer. There is no union where I work. You have the right to join a union, whatever the boss or agency may think, and in any case you do not need to tell them.
Good agencies recognise unions and encourage their staff to join.
As a union member you will be able to get help and advice if things do go wrong at work. My boss keeps making passes at me. You should inform your employment agency. A good agency will not want their workers to be subjected to sexual harassment, bullying or other bad treatment. You may also have legal remedies such as claiming sex discrimination at an Employment Tribunal or sueing for breach of contract if the employer has failed to provide a safe workplace for you.