It does not refer to the visual design, though it often may include visual design elements.
Information architecture is a major new field which has seen very significant growth.
Very schematically it is how to visualise the information needing to be conveyed by all the content of a web site and developing a successful interface and map for that design. This map refers as much to the relations amongst the information a site is to contain as to the navigation design to be employed. Information architecture assumes, examines, and evaluates who the users of a site will be, what their needs are, what information is being provided and the relations between all these. Information architects also consider issues of accessibility, which includes speed of access, presentation, clarity, link clarity, and consistency. Rarely will you find a site designed by an information architect that uses frames, largely because frames run counter to issues of access, design, and usability.