The most common organizers use keyboards for input and control. This both determines their shape and their minimum size. Doing anything else different for a device mostly interested in letters and not numbers is silly. The keys need to be some minimum size and spacing or else they won't be reliable in use. With the pen interface, the Palm Pilot fits into a standard shirt pocket and many other places that a regular PDA would have difficulty with.
Size becomes determined by the screen display quality more than anything else.
The Palm Pilot is like a small pad of paper in size, shape, and weight. Although part of the limited acceptance PDAs was user interface and part was technical, the largest barrier was probably cultural. By proportion, far fewer people used computers of any kind to organize and schedule their lives. The Daytimer and its ilk was how most busy professionals kept track of the details in their lives. Email and scheduling systems still tended to be on the main- frame if it was on the computer at all, or run by the traditional appointment book on a secretary's desk.