This mainly follows from the differences between how we and machines see the world. All data captured by cameras is stored in a multi-channel digital way. Therefore all the work, all pattern recognition by any Computer Vision algorithm is based on comparing natural tendencies in numbers. And while it still can find many patterns, the algorithm has no empirical idea what that could mean. It compares numbers until it finds suitable ones without additional context. So from this, it is the hardest for algorithms to grasp image context.