No, Mastery is permanently applied to your weapons. Even if you remove and then bring back a weapon or fruit, it will retain the same level of Mastery it had before it was removed.