A method for a neural network, comprising: generating, using a processor, a codebook (406, 606, 704, 802, 1004) from a weighting matrix (404, 602, 702) of the neural network, wherein generating the codebook (406, 606, 704, 802, 1004) includes: creating a plurality of sub-vectors (408, 604, 708-722) by chopping each row of the weighting matrix (404, 602, 702), performing a clustering operation on each of the plurality of sub-vectors (408, 604, 708-722) to group the sub-vectors (408, 604, 708-722) with other sub-vectors (408, 604, 708-722) having similar weight values, and determining one or more centroids from the clustering operation performed on each of the plurality of sub-vectors (408, 604, 708-722); building, using a processor, an index matrix (608, 706, 1006) from the codebook (406, 606, 704, 802, 1004), wherein the index matrix (608, 706, 1006) includes one or more index values, and each of the one or more index values corresponds to a location in the codebook (406, 606, 704, 802, 1004), the index matrix (608, 706, 1006) having dimensions such that the position of each index value corresponds to the position of one of the sub-vectors (408, 604, 708-722) created from the original weighting matrix (404, 602, 702), such that the index matrix (608, 706, 1006) labels the sub-vectors (408, 604, 708-722) of the weighting matrix (404, 602, 702) with the corresponding index of the codebook (406, 606, 704, 802, 1004) where centroids are stored for each of the sub-vectors (408, 604, 708-722) of the weighting matrix (404, 602, 702) used to create the centroids in that particular index of the codebook (406, 606, 704, 802, 1004); and generating, using a processor, a plurality of hash tables (804, 806, 808, 1008), one hash table (804, 806, 808, 1008) for each possible input value of a plurality of inputs of an input vector (402, 1002) for applying against the weighting matrix (404, 602, 702) to obtain an output vector, wherein generating each hash table (804, 806, 808, 1008) for each input value of the possible input values of the plurality of inputs of the input vector (402, 1002) includes multiplying each of the one or more centroids of the codebook (406, 606, 704, 802, 1004) by that input value. generating, using a processor, a codebook (406, 606, 704, 802, 1004) from a weighting matrix (404, 602, 702) of the neural network, wherein generating the codebook (406, 606, 704, 802, 1004) includes: creating a plurality of sub-vectors (408, 604, 708-722) by chopping each row of the weighting matrix (404, 602, 702), performing a clustering operation on each of the plurality of sub-vectors (408, 604, 708-722) to group the sub-vectors (408, 604, 708-722) with other sub-vectors (408, 604, 708-722) having similar weight values, and determining one or more centroids from the clustering operation performed on each of the plurality of sub-vectors (408, 604, 708-722); creating a plurality of sub-vectors (408, 604, 708-722) by chopping each row of the weighting matrix (404, 602, 702), performing a clustering operation on each of the plurality of sub-vectors (408, 604, 708-722) to group the sub-vectors (408, 604, 708-722) with other sub-vectors (408, 604, 708-722) having similar weight values, and determining one or more centroids from the clustering operation performed on each of the plurality of sub-vectors (408, 604, 708-722); building, using a processor, an index matrix (608, 706, 1006) from the codebook (406, 606, 704, 802, 1004), wherein the index matrix (608, 706, 1006) includes one or more index values, and each of the one or more index values corresponds to a location in the codebook (406, 606, 704, 802, 1004), the index matrix (608, 706, 1006) having dimensions such that the position of each index value corresponds to the position of one of the sub-vectors (408, 604, 708-722) created from the original weighting matrix (404, 602, 702), such that the index matrix (608, 706, 1006) labels the sub-vectors (408, 604, 708-722) of the weighting matrix (404, 602, 702) with the corresponding index of the codebook (406, 606, 704, 802, 1004) where centroids are stored for each of the sub-vectors (408, 604, 708-722) of the weighting matrix (404, 602, 702) used to create the centroids in that particular index of the codebook (406, 606, 704, 802, 1004); and generating, using a processor, a plurality of hash tables (804, 806, 808, 1008), one hash table (804, 806, 808, 1008) for each possible input value of a plurality of inputs of an input vector (402, 1002) for applying against the weighting matrix (404, 602, 702) to obtain an output vector, wherein generating each hash table (804, 806, 808, 1008) for each input value of the possible input values of the plurality of inputs of the input vector (402, 1002) includes multiplying each of the one or more centroids of the codebook (406, 606, 704, 802, 1004) by that input value.