Unity mostly loads assets when they are used in the editor and assets are generally speaking only unloaded every once in a while when an asset garbage collection is running. This means that running an operation for the first time is generally speaking slower than when you repeat it.