No, starting work without a party wall agreement can lead to legal action from your neighbour. You must obtain either written consent from your neighbour or a party wall award before commencing any work.