toji/gl-matrix — Issue #15: Creating a screen to ray function
Author: ceptri
Created: 2012-02-03 05:48:17+00:00
State: open

I love the library and have been using it a lot of the past six months. I've built my matrices for open gl using:

mat4.perspective(manta.m_camera.m_fov, manta.m_camera.m_aspect, manta.m_camera.m_near, manta.m_camera.m_far, this.m_projectionMatrix);
...
mat4.lookAt(vEye, vCameraTarget, vUp, cameraMatrix);
...
mat4.multiply(this.m_projectionMatrix, this.m_cameraMatrix, this.m_projectionViewMatrix);

and

gl_Position = uWorldViewProjectionMatrix * vec4(aVertexPosition, 1.0);

in my shader.

But I am having absolutely no luck trying to build a ray from a screen coordinate. I tried:

var invViewMatrix = mat4.create();
mat4.inverse(manta.m_render.m_projectionViewMatrix, invViewMatrix);

var dx = (x / (this.m_viewWidth / 2)) - 1;
var dy = -((y / (this.m_viewHeight / 2)) - 1);

var t1 = [dx, dy, -1, 1.0];
var t2 = [dx, dy, 1, 1.0];

mat4.multiplyVec4(invViewMatrix, t1);
mat4.multiplyVec4(invViewMatrix, t2);

var p1 = vec3.create( [ t1[0] / t1[3], t1[1] / t1[3], t1[2] / t1[3] ]);
var p2 = vec3.create( [ t2[0] / t2[3], t2[1] / t2[3], t1[2] / t2[3] ]);

But this just seems to give me garbage results. Any ideas?

Thanks