In Hamiltonian mechanics each dynamical variable f generates some transformations in the phase space via Poisson brackets. Due to axiom 2, the transformations, generated by classical variables and quantum operators, have the same algebra . Look here for details. For each pair of states their scalar product is introduced, which is 1. Complex conjugation of classical variables is represented as Hermitean conjugation of operators.
Operators are matrices, acting on such vectors.
They are motions - scalar product conserving transformations in the space of states. In physical applications the space of states usually is infinitely-dimensional. Completeness means that all Cauchy sequences are convergent. Look here for details. Actual set of states is , projectivization of Hilbert space by multiplication on complex numbers.