A tele-immersive collaboration system comprising: a local first tele-immersive collaboration set-up and a remote second tele-immersive collaboration set-up, each set-up comprising: one or more cameras for producing camera information that can be used to construct a depth image of the real space of the tele-immersive collaboration set-up, and means for constructing said depth image from said camera information; a display device, said display device being a stereo display device; and a physical semi-transparent mirror positioned in front of the display device; said system further comprising: a processing system configured to: receive, from a remote tele-immersive collaboration set-up, remote input information, said remote input information comprising a three-dimensional representation of objects in the real space of the remote set-up, said three-dimensional representation of objects being produced based on said remote camera information; receive, from the local set-up, local camera information; process available depth information to generate local three-dimensional object information based on said local camera information, said local three-dimensional object information corresponding to a three-dimensional representation of objects in the real space of the local set-up; compose three-dimensional scene information by combining the local three-dimensional object information and the remote input information; and send the three-dimensional scene information to said stereo display device. a local first tele-immersive collaboration set-up and a remote second tele-immersive collaboration set-up, each set-up comprising: one or more cameras for producing camera information that can be used to construct a depth image of the real space of the tele-immersive collaboration set-up, and means for constructing said depth image from said camera information; a display device, said display device being a stereo display device; and a physical semi-transparent mirror positioned in front of the display device; one or more cameras for producing camera information that can be used to construct a depth image of the real space of the tele-immersive collaboration set-up, and means for constructing said depth image from said camera information; a display device, said display device being a stereo display device; and a physical semi-transparent mirror positioned in front of the display device; said system further comprising: a processing system configured to: receive, from a remote tele-immersive collaboration set-up, remote input information, said remote input information comprising a three-dimensional representation of objects in the real space of the remote set-up, said three-dimensional representation of objects being produced based on said remote camera information; receive, from the local set-up, local camera information; process available depth information to generate local three-dimensional object information based on said local camera information, said local three-dimensional object information corresponding to a three-dimensional representation of objects in the real space of the local set-up; compose three-dimensional scene information by combining the local three-dimensional object information and the remote input information; and send the three-dimensional scene information to said stereo display device. a processing system configured to: receive, from a remote tele-immersive collaboration set-up, remote input information, said remote input information comprising a three-dimensional representation of objects in the real space of the remote set-up, said three-dimensional representation of objects being produced based on said remote camera information; receive, from the local set-up, local camera information; process available depth information to generate local three-dimensional object information based on said local camera information, said local three-dimensional object information corresponding to a three-dimensional representation of objects in the real space of the local set-up; compose three-dimensional scene information by combining the local three-dimensional object information and the remote input information; and send the three-dimensional scene information to said stereo display device. receive, from a remote tele-immersive collaboration set-up, remote input information, said remote input information comprising a three-dimensional representation of objects in the real space of the remote set-up, said three-dimensional representation of objects being produced based on said remote camera information; receive, from the local set-up, local camera information; process available depth information to generate local three-dimensional object information based on said local camera information, said local three-dimensional object information corresponding to a three-dimensional representation of objects in the real space of the local set-up; compose three-dimensional scene information by combining the local three-dimensional object information and the remote input information; and send the three-dimensional scene information to said stereo display device.