Neidje was born at Alawanydajawany, on the East Alligator River, around 1920 into the Bunitj clan of the Gagudju people. His father was Nadampala and his mother was Lucy Wirlmaka, from the Ulbuk clan of the Amurdak people. He grew up leading a traditional lifestyle in the East Alligator region, and was taught by his father, his grandfather, and others how to hunt and manage his environment. From about the age of 20 he worked first with buffalo hunters, then at a timber mill, and then on board a lugger transporting people and goods along the North Coast of the Northern Territory and to remote island communities. He was initiated at a ceremony at Ubirr in the early 1940s. During the Second World War he helped in the defence of Australia, working at the radar station at Cape Don. He was in Darwin during the Japanese bombings in 1942 and helped Aboriginal people during and after the devastation.