An entrance to the shrine complex is just across the road, opposite the bridge and up a flight of stone steps that brings you first to Shodo's temple.
In order to get to Shodo's temple, you have to go up on some stone steps.