A system for generating an optimized parametrized conversion function T for converting an original medical image of a first image type into a converted medical image of a second image type, the system comprising at least one processing unit configured to: obtain original medical images of the first and second image types; obtain an initial parametrized conversion function G to convert original medical images of the first image type into converted medical images of the second image type; calculate a first penalty P 1 based on at least one comparison of a first original medical image of the second image type with a first converted medical image of the second image type, that has been generated by applying a first parametrized conversion function G 1, which is based on the initial parametrized conversion function G, to a first original medical image of the first image type that forms an image pair with the first original medical image of the second image type and has thereby been determined to show the same part of the same patient; calculate a second penalty P 2 based on at least one comparison of an original medical image of the first image type and a converted medical image of the second image type that has been generated by applying a second parametrized conversion function G 2, which is based on the initial parametrized conversion function G, to the original medical image of the first image type, after converting the original medical image of the first image type and/or the converted medical image of the second image type into images of the same image type; and generate the optimized parametrized conversion function T based on the parameters of the initial parametrized conversion function G and at least said first and second penalties P 1 and P 2. obtain original medical images of the first and second image types; obtain an initial parametrized conversion function G to convert original medical images of the first image type into converted medical images of the second image type; calculate a first penalty P 1 based on at least one comparison of a first original medical image of the second image type with a first converted medical image of the second image type, that has been generated by applying a first parametrized conversion function G 1, which is based on the initial parametrized conversion function G, to a first original medical image of the first image type that forms an image pair with the first original medical image of the second image type and has thereby been determined to show the same part of the same patient; calculate a second penalty P 2 based on at least one comparison of an original medical image of the first image type and a converted medical image of the second image type that has been generated by applying a second parametrized conversion function G 2, which is based on the initial parametrized conversion function G, to the original medical image of the first image type, after converting the original medical image of the first image type and/or the converted medical image of the second image type into images of the same image type; and generate the optimized parametrized conversion function T based on the parameters of the initial parametrized conversion function G and at least said first and second penalties P 1 and P 2.