Yes and no. This would not particularly be suitable if you're thinking of using tempura in a rolled sushi (in fact, we rarely put tempura for rolled sushi in Japan). If you want to use it for rolled sushi, you would need thicker batter like ebi furai. However, if you put the tempura on top of rice (nigiri style), it will work and that's how we usually combine tempura and sushi in Japan.