A semiconductor apparatus comprising: one or more substrates; and logic coupled to the one or more substrates, wherein the logic is implemented at least partly in one or more of configurable logic or fixed-functionality hardware logic, and the logic is configured to: detect at least one tensor operation in an application, wherein the tensor operation has an unspecified input tensor size, wherein the input tensor size is defined in terms of the input column and row length of a tensor for the tensor operation; determine the input tensor size at runtime; and select a partition configuration for the tensor operation, based at least in part on the input tensor size and one or more runtime conditions, wherein the one or more runtime conditions include one or more of an expected power consumption, a matrix sparsity or a hardware resource availability, and wherein the partition configuration is configured to define a set of matrix shapes for the tensor operation, and to further define a set of hardware resources for executing the tensor operation; and wherein to select the partition configuration, the logic is further configured to search a lookup table for the input tensor size and at least one of the one or more runtime conditions; and wherein the lookup table is configured to include benchmarking data to enable retrieval of an optimal execution/partitioning plan along with any optimized intra-tensor-core optimized code generation for executing the tensor operation based on the selected partition configuration with reference to the one or more runtime conditions. one or more substrates; and logic coupled to the one or more substrates, wherein the logic is implemented at least partly in one or more of configurable logic or fixed-functionality hardware logic, and the logic is configured to: detect at least one tensor operation in an application, wherein the tensor operation has an unspecified input tensor size, wherein the input tensor size is defined in terms of the input column and row length of a tensor for the tensor operation; determine the input tensor size at runtime; and select a partition configuration for the tensor operation, based at least in part on the input tensor size and one or more runtime conditions, wherein the one or more runtime conditions include one or more of an expected power consumption, a matrix sparsity or a hardware resource availability, and wherein the partition configuration is configured to define a set of matrix shapes for the tensor operation, and to further define a set of hardware resources for executing the tensor operation; and wherein to select the partition configuration, the logic is further configured to search a lookup table for the input tensor size and at least one of the one or more runtime conditions; and wherein the lookup table is configured to include benchmarking data to enable retrieval of an optimal execution/partitioning plan along with any optimized intra-tensor-core optimized code generation for executing the tensor operation based on the selected partition configuration with reference to the one or more runtime conditions. detect at least one tensor operation in an application, wherein the tensor operation has an unspecified input tensor size, wherein the input tensor size is defined in terms of the input column and row length of a tensor for the tensor operation; determine the input tensor size at runtime; and select a partition configuration for the tensor operation, based at least in part on the input tensor size and one or more runtime conditions, wherein the one or more runtime conditions include one or more of an expected power consumption, a matrix sparsity or a hardware resource availability, and wherein the partition configuration is configured to define a set of matrix shapes for the tensor operation, and to further define a set of hardware resources for executing the tensor operation; and wherein to select the partition configuration, the logic is further configured to search a lookup table for the input tensor size and at least one of the one or more runtime conditions; and wherein the lookup table is configured to include benchmarking data to enable retrieval of an optimal execution/partitioning plan along with any optimized intra-tensor-core optimized code generation for executing the tensor operation based on the selected partition configuration with reference to the one or more runtime conditions.