An information processing program that causes a computer of an information processing apparatus to operate as: image obtaining means for sequentially obtaining a taken image that is taken by a camera; orientation information obtaining means for obtaining orientation information indicating an orientation of the camera with respect to the direction of gravity, from an orientation sensor; specific object detection means for detecting a specific object from the taken image obtained by the image obtaining means; positional relation calculation means for calculating a relative positional relation between the specific object and the camera; display determination means for determining at least any of a display form of a virtual object, a type of the virtual object, and the number of the virtual objects, on the basis of at least the orientation information; image generation means for generating an image of the virtual object corresponding to the determination by the display determination means, on the basis of the positional relation calculated by the positional relation calculation means; display control means for displaying the image of the virtual object on a display device; wherein, the display determination means determines the display form of the virtual object in accordance with an orientation of the specific object with respect to a predetermined direction indicated by the orientation information; the orientation information obtaining means obtains, as the orientation information, orientation information indicating a predetermined direction in a coordinate system based on the camera, the display determination means calculates a predetermined direction in a coordinate system based on the specific object on the basis of the positional relation and the orientation information obtained by the orientation information obtaining means, and determines the display form of the virtual object on the basis of the calculated predetermined direction, when the predetermined direction in the coordinate system based on the specific object is in a first range, the display determination means determines the display form of the virtual object as a first form, when the predetermined direction in the coordinate system based on the specific object is in a second range that is separated from the first range by a predetermined boundary, the display determination means determines the display form of the virtual object as a second form, and when the display form of the virtual object is changed from the first form to the second form, the display determination means sets the predetermined boundary on the first range side as compared to that when the display form of the virtual object is changed from the second form to the first form. image obtaining means for sequentially obtaining a taken image that is taken by a camera; orientation information obtaining means for obtaining orientation information indicating an orientation of the camera with respect to the direction of gravity, from an orientation sensor; specific object detection means for detecting a specific object from the taken image obtained by the image obtaining means; positional relation calculation means for calculating a relative positional relation between the specific object and the camera; display determination means for determining at least any of a display form of a virtual object, a type of the virtual object, and the number of the virtual objects, on the basis of at least the orientation information; image generation means for generating an image of the virtual object corresponding to the determination by the display determination means, on the basis of the positional relation calculated by the positional relation calculation means; display control means for displaying the image of the virtual object on a display device; the display determination means determines the display form of the virtual object in accordance with an orientation of the specific object with respect to a predetermined direction indicated by the orientation information; the orientation information obtaining means obtains, as the orientation information, orientation information indicating a predetermined direction in a coordinate system based on the camera, the display determination means calculates a predetermined direction in a coordinate system based on the specific object on the basis of the positional relation and the orientation information obtained by the orientation information obtaining means, and determines the display form of the virtual object on the basis of the calculated predetermined direction, when the predetermined direction in the coordinate system based on the specific object is in a first range, the display determination means determines the display form of the virtual object as a first form, when the predetermined direction in the coordinate system based on the specific object is in a second range that is separated from the first range by a predetermined boundary, the display determination means determines the display form of the virtual object as a second form, and when the display form of the virtual object is changed from the first form to the second form, the display determination means sets the predetermined boundary on the first range side as compared to that when the display form of the virtual object is changed from the second form to the first form.