changes from anchored teeth force application [8]. Among the different therapies, bone-anchored maxillary protraction (BAMP) is considered one of the most reliable methods for Class III malocclusion; it eliminates dentoalveolar effects and facilitates patient full-time wear compliance by using only intraoral skeletal anchorage. BAMP with miniplates is used in growing patients, with multiple miniscrews to ensure anchor stability and avoid any possible damage to the tooth germ [9]. A recent systematic review and clinical study showed reliable results for the treatment of skeletal Class III malocclusion, in which BAMP was reported to enhance maxillary growth and inhibit mandibular growth [10].

Despite its advantages, there are limitations to BAMP treatment, such as the need for additional surgery, possible tooth germ injury, and irritation of the adjacent tissues by elastics or miniplates. The miniplates can also loosen due to insufficient bone density at early ages. Therefore, the precise application of ready-made miniplates of sufficient bone quality following the contour of the cortical surface while avoiding the tooth germ is often surgically challenging [11,12].

Recent developments in 3D printing technology and the introduction of digital dentistry have made it possible to fabricate patient-customized simulations and individualized dental devices. In addition, metal 3D printing using selective laser sintering (SLS) allows the fabrication of individualized bone fixation plates and bone reconstruction material. BAMP is an effective treatment modality for skeletal Class III treatment; however, it is challenging to apply conventional plates onto the correct location. Therefore, the clinical feasibility of individualized BAMP plates using preoperative simulation and 3D titanium printing was tested in this pilot study.

## 2. Methods

### 2.1. Study Sample

This pilot prospective study was performed from 2019 to 2021 in departments of orthodontics and oral and maxillofacial surgery at Ewha Womans University Mokdong Hospital. This study was approved by the institutional ethics committees (EUMC 2019-06-014). Patients who were referred by the orthodontics department for four BAMP miniplates were included in the study. Any syndromic patients and cleft patients were excluded.

### 2.2. Data Acquisition

Cone beam computed tomography (CBCT) and intraoral scanning were performed to collect essential pre-surgical information to fabricate the individualized 3D-printed BAMP device. Since the BAMP device itself is placed on the maxillary and mandibular bone surface, it can be manufactured solely with CBCT, without digital information of the dentition. However, we previously experienced inaccurate placing of the BAMP device without dentitional information during the actual surgery. Therefore, an additional tooth-guided jig that can deliver the BAMP device to the precise simulated position was manufactured, and intraoral scanning was performed for an accurate tooth-guided transfer jig for properly positioning the plate.

Dicom data of CBCT were extracted into STL format and merged with intraoral scanning STL data. The CBCT dataset obtained 2 weeks before surgery was surface-rendered in the 3D model (STL format) of the bone. Intraoral Scanning with Trios3 (3 shape, Copenhagen, Denmark) started with the most distal tooth in the third quadrant and continuing to the anterior teeth. Next, the fourth quadrant was scanned, again beginning with the most distal tooth. Scanning of the maxilla started with the most distal tooth in the second quadrant and ended at the central incisor. The first quadrant was recorded starting with the most distal tooth. The camera was positioned at 45 degrees (or as close as possible to the axis of the tooth) to the buccal and lingual scans. The scanning device works by means of confocal microscopy, with a fast scanning time; the light source provides an illumination pattern to cause light oscillation on the object. As for the superimposition and merger of DICOM+STL (including software info), 3D Slicer.extension slicer RT, ver.4.11,
open source) was used to create a 3D bone model file (stl format), and Meshmixer (ver.3.5, Autodesk) was used to edit the surface model created by 3D slicer.

## 2.3. Preoperative Simulation

Using Ondemand® CAD/CAM software (Cybermed Co., Seoul, Korea) and Doctor Check software® (Cusmedi Co., Seoul, Korea), virtual miniplates and an additional transfer jig were designed with respect to the patient's bone contour, location of the hook, and shape of the surgical site. The maxillary plates for BAMP are preferably positioned on the zygomatico-maxillary junction and maxillary first molar. Positioning as close to the teeth as possible is more advantageous for surgical convenience and minimal surgical invasion. When located in the alveolar bone vertically too close to the teeth, there is a risk of tooth germ injury and screw loosening due to relatively more active bone remodeling and the low density of the maxillary bone during treatment. When positioned higher, near the zygomatic bone, permanent tooth germ injury can be avoided, and fixation strength can be obtained due to the greater bone density. The proper position, considering bone density for fixational support and avoiding tooth germ injury, could be determined preoperatively on the CBCT.

## 2.4. Surgical Procedure

The BAMP device consisted of a closed subperiosteal part for bone fixation and an open intraoral part for applying the elastics to deliver orthopedic force. The conventional flat-design bone fixation miniplate does not follow the bone surface contour nor does it form a tight contact with the bone, leaving small gaps that could cause biological complications. To minimize these complications, the BAMP plate is designed to be positioned to emerge intraorally at the mucogingival junction 5 mm above the gingival margin so that it can be placed along the anatomical contour when it is opened intraorally in the bone fixation part. This design minimizes the biological complications of the device maintained in the oral cavity for at least 6 months and up to 3 years. The elastics can be applied between the maxillary premolar and the first molar to maximize the patient's treatment adherence.

Accurate application of the simulated BAMP device to the determined location during actual surgery is a challenge. An additional transfer jig can be fabricated using precise intraoral scanning data to minimize the error between simulation and actual surgery. The transfer jig is suitable for being referenced to adjacent maxillary teeth and is manufactured to carry and deliver the BAMP device. The transfer jig is manufactured using resin or titanium metal printing (SLA type, EP-3500, Shinin). The BAMP device to be placed on the mandibular anterior teeth is designed in the same manner. The mandibular device is placed on the mandibular symphysis; therefore, tooth germ injury and bone density are less of a concern than with the maxillary device.

The design of the customized BAMP device through preoperative simulation is exported as an STL file from a CAD software program (Magics, Materialise, Leuven, Belgium) and submitted for 3D printing (Metalsys150, Winforsys Co., Seoul, Korea; laser power 120-200 W) using titanium alloy (Ti6Al4V ELI, medical grade in accordance with ASTM F136, AP&C, Quebec, QC, Canada) following the SLS additive manufacturing technique. Titanium powder was melted into thin layers, which were used in the 3D printer. Post-processing of the plate involved removal of the loose power and the support structures and polishing to improve the surface quality and mechanical properties and reduce errors during CBCT and scanning data acquisition.

A rapid prototyping model of the patient was manufactured before surgery, and a virtual surgery was performed through the process of pre-fitting. In the actual surgery, customized plates were installed at the surgical site using the transfer jig as a guide. An incision (1.5-2.5 cm) was made above the bone where the fixation plate had to be placed, and minimal dissection was performed. Then, the jig was precisely matched to the reference point, indicating accurate installation of the BAMP device. After the correct position of the BAMP device was confirmed, fixation was performed using 1.5 mm screws. After
confirming that the BAMP device was sufficiently fixed to the bone, the incision was sutured using vicryl 4-0 and 5-0.

Postoperatively, patients were instructed to use a chlorhexidine mouth rinse three times a day for 7 days. Analgesics and non-steroidal anti-inflammatory drugs were prescribed. After 7–10 days of soft tissue healing, Class III elastics were applied with an initial force of ~70 g on each side. The force of the elastics was increased to 150 g after 1 month of traction and again increased to 200–250 g after 3 months. Patients were instructed to wear the elastics full-time, except for meals and for brushing. They had to replace the elastics daily, with regular follow-up. Patients were also instructed to follow-up once a month postoperatively.

## 2.5. Cephalometric and Superimposition Analysis

Simulated positioned digital images and postoperative CT data were superimposed for comparison to evaluate the accuracy of preoperative simulation and actual application of the fabricated BAMP device. In addition, lateral cephalometric analysis was performed before BAMP treatment and 2 years postoperatively to evaluate the treatment effects of BAMP's skeletal Class III correction. The superimposition of lateral cephalograms was analyzed, and changes in SNA, SNB, ANB, and Wits were compared to determine the effect of growth modification.

The 3D deviation between the position of the BAMP designed with CBCT and the position of the BAMP after surgery was investigated to evaluate the accuracy between virtual simulation and actual surgery. Twenty points were randomly selected from BAMP and used as reference points to investigate the single linear deviation, reported in mm, along the x-, y-, and z-axis. The x-axis is the transverse (lateral/medial) direction, the y-axis is the sagittal (anterior/posterior) direction, and the z-axis is the axial (cranial/caudal) direction. The deviation in the cranial, anterior, and lateral directions of the BAMP was considered positive (+), and the deviation in the contralateral direction was considered negative (−). The 3D average distance is expressed in mm, and the difference is indicated by color (Supplementary Figure S1). The 3D geometrical deviation was assessed after color coding high or low geometrical deviation using Geomagic software (Freeform Plus, 3D Systems, Morrisville, NC, USA) (Supplementary Figure S2).

# 3. Results

Four patients were screened, and two were included in this study for BAMP therapy. The two included patients were both male, 9 and 10 years old. All operations were performed under local anesthesia, and the average time from incision to the upper bone of the BAMP device through the transfer jig, from screw fixation to sutures, was 8.3 min per site, on average. Unlike for conventional plate surgery, the time to consider tooth germ injury and sufficiently dense bone and application work through screw bending were not relevant; therefore, the surgery time was dramatically reduced. In addition, since the plate fixing position was accurately known in advance, minimally invasive surgery, including incision and dissection, was possible.

During the follow-up, the customized miniplates showed no loosening or inflammation. The miniplates were well-retained for over a year, had sufficient retention force, and maintained good oral hygiene in the surrounding tissue.

Skeletal measurements revealed that BAMP decreased ANB difference (−4.56 to −1.09) and Wits appraisal (−7.52 to −3.26). Other measurements indices for sagittal and vertical growth were also in the normal range, indicating successful growth modification with customized BAMP therapy for skeletal Class III malocclusion (Table 1).

The accuracy between simulation and actual surgery was satisfactory. In the measurement of the linear deviation between preoperative simulation and actual surgery, the average difference was −0.0154 mm on the x-axis, −0.0946 mm on the y-axis, and 0.0579 mm on the z-axis, with an average distance of 0.1081 mm (Table 2).
<table><caption>**Table 1.** Skeletal measurements: preoperative and postoperative (24 months) *.</caption><thead><tr><th>Measurement</th><th>Reference Value</th><th>Preoperative</th><th>Postoperative 24 Months</th></tr></thead><tbody><tr><td>Maxillo-mandibular measurement</td><td></td><td></td><td></td></tr><tr><td>SNA</td><td>81.6 (3.1)</td><td>77.98</td><td>84.63</td></tr><tr><td>SNB</td><td>79.1 (3.0)</td><td>82.54</td><td>85.72</td></tr><tr><td>ANB difference</td><td>2.4 (1.8)</td><td>-4.56</td><td>-1.09</td></tr><tr><td>Wits appraisal</td><td>-2.7 (2.4)</td><td>-7.52</td><td>-3.26</td></tr><tr><td>Angular measurement</td><td></td><td></td><td></td></tr><tr><td>FMA</td><td>29.63 (5.66)</td><td>30.34</td><td>28.98</td></tr><tr><td>SN-GoMe</td><td>36 (4)</td><td>37.1</td><td>32.72</td></tr><tr><td>A point—N Perpend</td><td>0.4 (2.3)</td><td>-5.37</td><td>-1.86</td></tr><tr><td>Facial height measurement</td><td></td><td></td><td></td></tr><tr><td>Facial Height Ratio</td><td>65 (9)</td><td>61.7</td><td>66.37</td></tr><tr><td>Posterior Facial Height</td><td>85 (5.5)</td><td>76.84</td><td>90.76</td></tr><tr><td>Anterior Facial Height</td><td>127.4 (5.6)</td><td>124.52</td><td>136.74</td></tr><tr><td>Dentoalveolar measurement</td><td></td><td></td><td></td></tr><tr><td>U1 to SN</td><td>107 (6)</td><td>123.45</td><td>120.62</td></tr><tr><td>IMPA</td><td>95.9 (6.3)</td><td>78.2</td><td>78.3</td></tr><tr><td>Interincisal angle</td><td>124 (8.3)</td><td>121.2</td><td>128.32</td></tr><tr><td>Soft tissue measurement</td><td></td><td></td><td></td></tr><tr><td>Upper Lip E-plane</td><td>0.1 (2)</td><td>0.44</td><td>-1.58</td></tr><tr><td>Lower Lip E-plane</td><td>0.1 (2)</td><td>2.69</td><td>0.26</td></tr><tr><td>Nasolabial angle</td><td>84.9 (5)</td><td>93.86</td><td>101.9</td></tr></tbody></table>

<table><caption>**Table 2.** Linear deviations in the x-, y-, and z-axis at 20 selected reference points.</caption><thead><tr><th rowspan="2"></th><th colspan="4">Linear Deviation</th></tr><tr><th>x-Axis<br>(Lateral/Medial)</th><th>y-Axis<br>(Anterior/Posterior)</th><th>z-Axis<br>(Cranial/Caudal)</th><th>Distance Difference</th></tr></thead><tbody><tr><td>N</td><td>20</td><td>20</td><td>20</td><td>20</td></tr><tr><td>Median</td><td>-0.0154</td><td>-0.0946</td><td>0.0579</td><td>0.1081</td></tr><tr><td>Range</td><td>-0.1729-0.2448</td><td>-1.1643-1.5076</td><td>-0.9189-0.7126</td><td>0.0211-1.3772</td></tr><tr><td>SD</td><td>0.1463</td><td>0.9012</td><td>0.5515</td><td>0.5074</td></tr></tbody></table>

## 4. Case Presentation

A 10-year-old boy presented with maxillary retrognathism, mandible prognathism, and dentofacial deformity and was diagnosed with skeletal Class III malocclusion (Figures 1 and 2). The patient had no systemic diseases and no signs of temporomandibular joint dysfunction. Cephalometric analysis showed a concave facial profile and hyperdivergent growth pattern. Considering the patient's age and growth patterns, treatment with customized BAMP miniplates was considered as the most optimal therapy.

For the left maxilla, a 'Y' shaped plate was used and fixed at the zygomatic process to avoid possible disruption of the molar roots. An 'L' shaped plate was designed for the mandible and fixed at the symphysis of mandible. After preoperative simulation and fabrication, BAMP plates were used for the individualized rapid prototyping model (Figure 3). Under local anesthesia, customized BAMP plates were installed at the simulated location using the transfer jig as a guide. The jig was precisely matched with the reference point for accurate installation of the miniplates (Figure 4). The mean surgical time from incision to suture was 6.4 min for each side.

<footnote>* Values are presented as mean (standard deviation).</footnote>
<figure><img src="image_39.png" /><figcaption>**Figure 1.** Facial photographs showing lateral and 45 degree profiles (A,D) at initial, (B,E) 12 months after customized bone-anchored maxillary protraction (BAMP) therapy, and (C,F) 24 months after customized BAMP therapy.</figcaption></figure>

<figure><img src="image_40.png" /><figcaption>**Figure 2.** (A) Initial intraoral photographs. (B) Intraoral photographs 24 months after customized bone anchored maxillary protraction therapy showing Class I molar key.</figcaption></figure>
<figure><img src="image_41.png" /><figcaption>**Figure 3.** Preoperative simulation and application of fabricated customized plates and transfer jig on the individualized rapid prototyping model.</figcaption></figure>

The patient was advised to maintain good oral hygiene after the operation. After soft tissue healing, Class III elastics (5/8 3.50 oz) were applied with an initial force of ~70 g on each side. The force of elastics (left: 5/16 3.50 oz; right: 3/16 3.50 oz) was increased to 150 g after 1 month of traction and again increased to 250 g after 3 months. The patient was instructed to wear the elastics full-time, except during meals and brushing. The patient also had to replace the elastics daily, with regular follow-up (Figure 4). During the follow-up, the customized miniplates showed no loosening or inflammation. The miniplates were well-retained for over two years, showing sufficient retention force, and maintained good oral hygiene in the surrounding tissue. Superimposition of lateral cephalograms indicated that BAMP therapy successfully protracted the maxilla and prevented mandibular growth and hyperdivergent vertical growth (Figure 5).
<figure><img src="image_42.png" /><figcaption>**Figure 4.** Surgical photographs. (A,B) Customized BAMP miniplates were transferred to the pre-simulated location using a transfer jig. (C,D) Postoperative photographs.</figcaption></figure>

<figure><img src="image_43.png" /><figcaption>**Figure 5.** Superimposition of lateral cephalograms at initial stage, 12 months after BAMP therapy, and 24 months after BAMP therapy. The superimposition was based on the cranial base.</figcaption></figure>
## 5. Discussion

Recently, bone-anchored maxillary protraction has been regarded as an effective treatment method for growth modification in growing skeletal Class III patients; however, an individualized approach, especially using 3D metal plate printing, has not been previously reported. The authors believe sharing our experience in the study will contribute to the acceleration of the medical use of individualized metal printing to the field of dentistry.

Skeletal Class III malocclusion correction in growing patients is considered one of the most challenging treatments in orthodontics. It is conducted using either an extraoral or intraoral device. The use of a facemask has been one of the most common approaches for maxillary traction, but it may result in undesired dentoalveolar outcomes, such as upper incisors or extrusion of the upper molars. A chin cup is effective for mandibular development restriction, but it may cause anterior teeth inclination in the mandible due to the pressure applied on the bone and soft tissue [13]. Wearing an extraoral appliance may be esthetically unpleasing and may cause social discomfort in growing patients, oftentimes leading to noncompliance [14]. Miniscrews or miniplates have replaced the use of extraoral devices for such orthopedic force transfer. Intraoral miniscrews and miniplates have shown sufficient biocompatibility and durability for the loading of orthodontic force, significantly influencing the advancement of maxillary structures [9,11].

However, the use of bone anchorages has been increasing, and various factors relating to the failure of implants are being studied. The location of the implant, diameter of the screw, shape of the miniplate, inflammation, and keratinization of peri-implant tissues affect the stability of bone anchorage [15,16]. Age is also a crucial factor in adolescent patients. The stability of BAMP therapy can be attributed to mechanical retention and the density and thickness of the cortical bone. In our case, the plate on the left maxilla and right mandible were found mobile after 5 months of installation and eventually failed. As a child, the patient's developing bone may have not been strong enough to endure the force applied for anchorage. Applied orthopedic force for bone correction, which is relatively higher than that applied for tooth movement, may also have influenced the stability of bone anchorage. Bone remodeling occurs quickly and plays a significant role in young patients during their growing period. When excess force is applied on the anchorage device, its loading can significantly influence the density and turnover of the alveolar bone, loosening the BAMP correction device and leading to eventual failure.

Oral hygiene and plaque accumulation may also affect the success of plate insertion. Microbial flora at the interface between the artificial material and mucosal epithelium influence plate fixation. Anaerobic bacteria at the insertion site may cause inflammation. In our case, only mild infection was observed in the right mandible. Keeping the emergence point within the attached gingiva is essential to avoid bacterial invasion [17].

A customized titanium miniplate was manufactured after the failure of anchorage on the left maxilla and right mandible. The transfer jig created for each surgical site allowed clinicians to precisely determine the final location and delivery of the plate during the surgery [18,19]. CBCT and oral scan data were used to evaluate and design the customized miniplates accordingly. In the second operation, the customized miniplates were to be fixed at the zygomatic process and symphyseal to avoid the previous insertion site and any possible damage to the surrounding structures, such as the root and tooth germ. Bending and adjusting the plate contour not only alters the mechanical strength of the plate but also causes its inevitable deformation. Customized plates manufactured according to the skeletal contour of the patient eliminate the need for adjustment during the surgery. The use of customized devices allows clinicians to make smaller but more accurate incisions and deliver the plates in their ideal locations within a shorter period. There was a report of a minimally invasive approach for BAMP placement without surface incisions in the gingival margins or papillae. This potentially minimizes the gingival recession that sometimes accompanies flap surgery [20]. A combination of maxillary protraction, expansion and contraction would be beneficial in this case.
A BAMP device ideally fixes to the jawbone and directly contacts the oral surface through the open mucosa. There are inherent risks of plaque accumulation and chronic inflammation, but BAMP therapy is shorter and more efficient than extraoral orthopedic devices such as a chin cup.

**6. Conclusions**

Preoperative simulation and a 3D titanium printing technique allowed the fabrication of customized BAMP devices and their precise application at the planned locations. The clinical results demonstrated the stability and effectiveness of this modality for the treatment of skeletal Class III malocclusion.

**Supplementary Materials:** The following are available online at https://www.mdpi.com/article/10.3390/jpm1111087/s1, Figure S1: Distance measurement of 20 selected reference points, Figure S2: A color-coded deviation figure that visualizes the high or low geometrical deviation.

**Author Contributions:** S.K., S.-H.K., M.K.—Conception and design of the work, analysis, interpretation of data, critical revision of draft; S.-M.L., Y.L.P., S.Y., J.L.—Conception and design of the work, analysis, interpretation of data, original draft preparation; W.K.—Analysis, interpretation of data, critical revision of draft; J.-W.K.—Conception and design of the work, analysis, interpretation of data, critical revision of draft. All authors have read and agreed to the published version of the manuscript.

**Funding:** This research was supported by the KIAT (Korea Institute for Advancement of Technology) grant funded by the Korea Government (MOTIE:Ministry of Trade Industry and Energy). (P0008799), and the Korea Medical Device Development Fund grant from the Korea Government (the Ministry of Science and ICT, the Ministry of Trade, Industry and Energy, the Ministry of Health & Welfare, the Ministry of Food and Drug Safety) (Project Number: 9991006713, KMDF_PR_20200901_0040).

**Institutional Review Board Statement:** The study was conducted according to the guidelines of the Declaration of Helsinki and approved by the Institutional Review Board (or Ethics Committee) of Ewha Womans Unviersity Mok-Dong Hospital (EUMC 2019-06-14).

**Informed Consent Statement:** Informed consent was obtained from all subjects involved in the study.

**Data Availability Statement:** Data is available with permission of corresponding author.

**Conflicts of Interest:** The authors declare no conflict of interest. The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript, or in the decision to publish the results.

**References**

1. Proffit, W.R.; Fields, H.W., Jr.; Sarver, D.M. *Contemporary Orthodontics*; Elsevier Health Sciences: Amsterdam, The Netherlands, 2006.

2. Litton, S.F.; Ackermann, L.V.; Isaacson, R.J.; Shapiro, B.L. A genetic study of Class 3 malocclusion. *Am. J. Orthod.* **1970**, *58*, 565–577.
[CrossRef]

3. Borzabadi-Farahani, A.; Borzabadi-Farahani, A.; Eslamipour, F. Malocclusion and occlusal traits in an urban Iranian population. An epidemiological study of 11- to 14-year-old children. *Eur. J. Orthod.* **2009**, *31*, 477–484. [CrossRef] [PubMed]

4. Eslamian, L.; Borzabadi-Farahani, A.; Badiee, M.R.; Le, B.T. An Objective Assessment of Orthognathic Surgery Patients. *J. Craniofac. Surg.* **2019**, *30*, 2479–2482. [CrossRef]

5. White, L. Early orthodontic intervention. *Am. J. Orthod. Dentofac. Orthop.* **1998**, *113*, 24–28. [CrossRef]

6. Zhang, H.; Deng, F.; Wang, H.; Huang, Q.; Zhang, Y. Early orthodontic intervention followed by fixed appliance therapy in a patient with a severe Class III malocclusion and cleft lip and palate. *Am. J. Orthod. Dentofac. Orthop.* **2013**, *144*, 726–736. [CrossRef] [PubMed]

7. Azamian, Z.; Shirban, F. Treatment Options for Class III Malocclusion in Growing Patients with Emphasis on Maxillary Protraction. *Sci. Cairo* **2016**, *2016*. [CrossRef] [PubMed]

8. Sugawara, J.; Baik, U.B.; Umemori, M.; Takahashi, I.; Nagasaka, H.; Kawamura, H.; Mitani, H. Treatment and posttreatment dentoalveolar changes following intrusion of mandibular molars with application of a skeletal anchorage system (SAS) for open bite correction. *Int. J. Adult Orthodon. Orthognath. Surg.* **2002**, *17*, 243–253. [PubMed]

9. De Clerck, H.; Cevidanes, L.; Baccetti, T. Dentofacial effects of bone-anchored maxillary protraction: A controlled study of consecutively treated Class III patients. *Am. J. Orthod. Dentofac. Orthop.* **2010**, *138*, 577–581. [CrossRef]

10. Rodríguez de Guzmán-Barrera, J.; Sáez Martínez, C.; Boronat-Catalá, M.; Montiel-Company, J.M.; Paredes-Gallardo, V.; Gandía-Franco, J.L.; Almerich-Silla, J.M.; Bellot-Arcís, C. Effectiveness of interceptive treatment of class III malocclusions with skeletal anchorage: A systematic review and meta-analysis. *PLoS ONE* **2017**, *12*, e0173875. [CrossRef] [PubMed]
11. Cevidanes, L.; Baccetti, T.; Franchi, L.; McNamara, J.A., Jr.; De Clerck, H. Comparison of two protocols for maxillary protraction: Bone anchors versus face mask with rapid maxillary expansion. *Angle Orthod.* **2010**, *80*, 799-806. [CrossRef] [PubMed]

12. Cornelis, M.A.; Scheffler, N.R.; Mahy, P.; Siciliano, S.; De Clerck, H.J.; Tulloch, J.F. Modified miniplates for temporary skeletal anchorage in orthodontics: Placement and removal surgeries. *J. Oral Maxillofac. Surg.* **2008**, *66*, 1439-1445. [CrossRef] [PubMed]

13. Kim, J.H.; Viana, M.A.; Graber, T.M.; Omerza, F.F.; BeGole, E.A. The effectiveness of protraction face mask therapy: A meta-analysis. *Am. J. Orthod. Dentofac. Orthop.* **1999**, *115*, 675-685. [CrossRef]

14. Sergl, H.G.; Klages, U.; Zentner, A. Functional and social discomfort during orthodontic treatment—Effects on compliance and prediction of patients’ adaptation by personality variables. *Eur. J. Orthod.* **2000**, *22*, 307-315. [CrossRef] [PubMed]

15. Miyawaki, S.; Koyama, I.; Inoue, M.; Mishima, K.; Sugahara, T.; Takano-Yamamoto, T. Factors associated with the stability of titanium screws placed in the posterior region for orthodontic anchorage. *Am. J. Orthod. Dentofac. Orthop.* **2003**, *124*, 373-378. [CrossRef]

16. Cheng, S.J.; Tseng, I.Y.; Lee, J.J.; Kok, S.H. A prospective study of the risk factors associated with failure of mini-implants used for orthodontic anchorage. *Int. J. Oral Maxillofac. Implant.* **2004**, *19*, 100-106.

17. Murakami, S.; Mealey, B.L.; Mariotti, A.; Chapple, I.L.C. Dental plaque-induced gingival conditions. *J. Clin. Periodontol.* **2018**, *45*, S17-S27. [CrossRef] [PubMed]

18. Kim, S.H.; Choi, Y.S.; Hwang, E.H.; Chung, K.R.; Kook, Y.A.; Nelson, G. Surgical positioning of orthodontic mini-implants with guides fabricated on models replicated with cone-beam computed tomography. *Am. J. Orthod. Dentofac. Orthop.* **2007**, *131*, S82-S89. [CrossRef] [PubMed]

19. Yu, J.J.; Kim, G.T.; Choi, Y.S.; Hwang, E.H.; Paek, J.; Kim, S.H.; Huang, J.C. Accuracy of a cone beam computed tomography-guided surgical stent for orthodontic mini-implant placement. *Angle Orthod.* **2012**, *82*, 275-283. [CrossRef] [PubMed]

20. Zadeh, H.H.; Borzabadi-Farahani, A.; Fotovat, M.; Kim, S.H. Vestibular Incision Subperiosteal Tunnel Access (VISTA) for Surgically Facilitated Orthodontic Therapy (SFOT). *Contemp. Clin. Dent.* **2019**, *10*, 548. [PubMed]
<figure><img src="image_44.png" /></figure>
<figure><img src="image_45.png" /></figure>

Article

# A Pan-Cancer Atlas of Differentially Interacting Hallmarks of Cancer Proteins

Medi Kori <fnref n="1" />,† Ⓓ , Gullu Elif Ozdemir <fnref n="1" />,† , Kazim Yalcin Arga <fnref n="1" />,<fnref n="2" />,* Ⓓ and Raghu Sinha <fnref n="3" />,* Ⓓ

**Abstract:** Cancer hallmark genes and proteins orchestrate and drive carcinogenesis to a large extent, therefore, it is important to study these features in different cancer types to understand the process of tumorigenesis and discover measurable indicators. We performed a pan-cancer analysis to map differentially interacting hallmarks of cancer proteins (DIHCP). The TCGA transcriptome data associated with 12 common cancers were analyzed and the differential interactome algorithm was applied to determine DIHCPs and DIHCP-centric modules (i.e., DIHCPs and their interacting partners) that exhibit significant changes in their interaction patterns between the tumor and control phenotypes. The diagnostic and prognostic capabilities of the identified modules were assessed to determine the ability of the modules to function as system biomarkers. In addition, the druggability of the prognostic and diagnostic DIHCPs was investigated. As a result, we found a total of 30 DIHCP-centric modules that showed high diagnostic or prognostic performance in any of the 12 cancer types. Furthermore, from the 16 DIHCP-centric modules examined, 29% of these were druggable. Our study presents candidate systems' biomarkers that may be valuable for understanding the process of tumorigenesis and improving personalized treatment strategies for various cancers, with a focus on their ten hallmark characteristics.

**Keywords:** hallmarks of cancer; differential interactome; system biomarkers; druggability; personalized treatments

## 1. Introduction

Cancer is one of the leading causes of death in almost every country in the world. According to GLOBOCAN, there were about 19.3 million new cancer cases and 10 million cancer deaths in 2020 [1], and these numbers make the fateful picture clear. Cancer is an uncontrolled cell-division process that leads to cell transformation through the occurrence of various and successive genetic alterations. This transformation is the result of a complex process, and this complexity makes the disease an enigma shrouded in mystery and incurable [2].

In 2000, Hanahan and Weinberg wondered, as there were a variety of cancers, how many regulatory mechanisms were affected, or were the same regulatory mechanisms destroyed in the cell to become cancerous? Additionally, were there common signals? After asking these questions, they proposed that cancer cells exhibited six biological phenomena, which they called “hallmarks of cancer”. They proposed that these six hallmarks play a critical role in cancer development and are likely to occur in all cancers [3]. In the coming years, Hanahan and Weinberg added four more cancer features to the list of characteristics with the ongoing studies. Subsequently, with this update, the ten hallmarks of cancer approved today (eight trait characteristics and two enabling characteristics) were formed.

**Citation:** Kori, M.; Ozdemir, G.E.; Arga, K.Y.; Sinha, R. A Pan-Cancer Atlas of Differentially Interacting Hallmarks of Cancer Proteins. *J. Pers. Med.* **2022**, *12*, 1919. https://doi.org/10.3390/jpm12111919

Academic Editor: Anne-Marie Caminade

Received: 23 October 2022
Accepted: 15 November 2022
Published: 17 November 2022

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

<figure><img src="image_46.png" /></figure>

**Copyright:** © 2022 by the authors.
Licensee MDPI, Basel, Switzerland.
This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

<footnote n="1">¹ Department of Bioengineering, Marmara University, Istanbul 34854, Turkey</footnote>

<footnote n="2">² Genetic and Metabolic Diseases Research and Investigation Center, Marmara University, Istanbul 34854, Turkey</footnote>

<footnote n="3">³ Department of Biochemistry and Molecular Biology, Penn State College of Medicine, Hershey, PA 17033, USA</footnote>

<footnote>* Correspondence: kazim.arga@marmara.edu.tr (K.Y.A.); rus15@psu.edu (R.S.)</footnote>

<footnote>† These authors contributed equally to this work.</footnote>
These hallmarks include: (i) activating invasion and metastasis; (ii) enabling replicative immortality; (iii) evading growth suppressors; (iv) avoiding immune destruction; (v) genome instability and mutation; (vi) inducing angiogenesis; (vii) deregulating cellular energetics; (viii) resisting cell death; (ix) sustaining proliferative signaling; and (x) tumor-promoting inflammation [4]. Hanahan suggested, in his latest article, that potential features such as cellular plasticity, non-mutative epigenetic reprogramming, polymorphic microbiomes, and impaired differentiation could be added to the list in the future. However, he believes that the definition of these conceptual features needs to be discussed and experimentally validated with cancer biology studies [5].

Since the hallmarks of cancer genes and proteins orchestrate and drive carcinogenesis to a large extent, it is noteworthy to study these features in different cancers to understand the process of tumorigenesis and discover measurable indicators of altered biological states such as biomarkers. A study which was conducted by Nagy et al. [6] performed a pan-cancer survival analysis of cancer hallmark genes by not considering the associations among them. However, the development of cancer is generally induced by a combination of genes, proteins, metabolites, and other factors. In 2019, Yu and co-workers, by considering the association between genes, identified cancer hallmarks based on the gene co-expression networks for seven cancers [7]. Nevertheless, deciphering the changes at the protein level (protein interactome) is essential to understand tumorigenesis at the systemic level. Therefore, deciphering the hallmarks of different cancers using protein-protein interactions (PPIs) within the protein interactome is a promising approach to understand the mechanisms of cancer and to propose diagnostic, prognostic or therapeutic biomarker targets.

In our study, we performed a pan-cancer analysis and aimed to map differential hallmarks of cancer-associated protein-protein interactions (Figure 1). We examined the Cancer Genome Atlas (TCGA) transcriptome data from 12 different cancers: breast invasive carcinoma (BRCA); colon adenocarcinoma (COAD); head and neck squamous cell carcinoma (HNSC); kidney renal clear cell carcinoma (KIRC); kidney renal papillary cell carcinoma (KIRP); liver hepatocellular carcinoma (LIHC); lung adenocarcinoma (LUAD); lung squamous cell carcinoma (LUSC); prostate adenocarcinoma (PRAD); stomach adenocarcinoma (STAD); thyroid carcinoma (THCA); and uterine corpus endometrial carcinoma (UCEC), having sufficient samples in both tumor and control groups (n > 30) and applied a differential protein interactome algorithm [8], to determine differentially interacting hallmark of cancer proteins (DIHCPs) and DIHCP-centered modules (i.e., DIHCPs and their interacting partners) that represent significant changes in their interaction patterns between the tumor and control phenotypes. The diagnostic and prognostic capabilities of the identified DIHCP-centered modules were assessed in order to identify the modules' potential ability to function as system biomarkers. In addition, the druggability of the prognostic and diagnostic DIHCPs were investigated. Ultimately, this study presents candidate system biomarkers that may be useful for understanding tumorigenesis, developing novel diagnostic tools, and improving personalized treatment strategies for various cancers, with a focus on the hallmarks of cancer.
<figure><img src="image_47.png" /><figcaption>**Figure 1.** The systematic approach for the study. The steps of the applied systems' biology methodology for the study. After collecting the data from different repositories, the differential interactome algorithm was applied. This algorithm was used to identify differentially interacting hallmark of cancer proteins (DIHCPs) and DIHCPs-centered modules. The prognostic and diagnostic performance of the DIHCPs-centered modules was evaluated. The enrichment analyses were performed and finally the components of the modules' druggability were discovered.</figcaption></figure>

## 2. Materials and Methods

### 2.1. *Collecting of Gene Expression Data*

RNA-sequencing (RNA-seq) fragments per kilobase of transcript per million fragments mapped (FPKM) normalized gene expression data were collected from the Cancer Genome Atlas (TCGA) [9]. For this study, gene expression data of 12 cancers with at least 30 normal
and tumor samples of 33 different cancers were collected from the TCGA. In total, gene expression data were collected from 6239 tumor and 637 matched normal tissue samples. The 12 cancer types studied and their sample numbers are represented in Figure 2.

<figure><img src="image_48.png" /><figcaption>**Figure 2.** Distribution of samples from 12 human cancers. The diagram shows control and tumor samples from 12 human cancers analyzed in the study.</figcaption></figure>

## 2.2. Collection of Cancer Hallmark Genes

Genes representing all the ten hallmarks of cancer [4] were collected from two publicly available biological repositories which are Cancer Hallmark Genes (CHG) da-tabase [10] and the Catalogue of Somatic Mutations in Cancer (COSMIC) database [11] (v.95). A total of 1906 different cancer hallmark genes were collected from these two repositories. The GeneCards: The Human Gene Database [12] was used to define proteins encoded by the cancer hallmark genes.

## 2.3. Collection of Human Protein Interactome Data

Human PPI data were extracted from the BioGRID database [13] (MV-Physical-4.2.191), which contains 51,745 physical and experimentally detected PPIs among 10,177 human proteins. Integration of the obtained PPI interaction data with proteins encoded by genes for which gene expression data are available in TCGA and with proteins encoded by cancer hallmark genes resulted in a network consisting of 7422 PPIs among 1906 proteins.

## 2.4. Identification of Differentially Interacting Hallmark of Cancer Proteins

The differential protein interactome algorithm [8] was applied to the gene expression profiles of all 12 cancer types using the R platform [14] (version 4.0.2). Briefly, differential hallmarks of cancer-associated PPIs (dHCPIPIs) were determined to detect changes in PPI patterns between tumors and controls. To this end, the algorithm uses gene expression profiling to predict the relative frequency of observation (q-value) for each PPI to represent the dHCPIPIs.

The following criteria were chosen to determine dHCPIPIs: (i) if the predicted q value for PPI is less than 0.10, the interaction is significantly suppressed in the tumor state; (ii) if the predicted q value for PPI is greater than 0.90, the interaction is significantly activated in the tumor state; (iii) the normalized frequency of observation in the tumor or normal phenotype is greater than 20%.

Application of the differential protein interactome algorithm yielded DIHCPs representing significant changes in their interaction patterns during the transition between the normal and tumor phenotypes. The DIHCPs were classified into 2 groups according to their interaction partners: (i) proteins with suppressed interactions in the tumor state and (ii) proteins with activated interactions in the tumor state. DIHCPs (the hub hallmark of cancer proteins) together with their interacting protein partners were designated as DIHCP-centered modules (modular structures around hubs). Further analyses were performed using the DIHCP-centered modules with at least 10 proteins. The dHCPIPIs and DIHCP-centered modules were visualized using Cytoscape (v3.5.0) [15].
## 2.5. Diagnostic Performance Analyses of Differentially Interacting Hallmark of Cancer Protein Modules

Principal component analyses (PCA) were performed based on TCGA-derived gene expression profiles of the genes encoding DIHCPs in each DIHCP-centered module. Each simulation was performed with at least 30 randomly selected normal and 30 tumor samples, and the first 3 principal components representing the highest variances (at least 80% of the total variance) were considered in determining the sensitivity (the proportion of positive test results among diseased individuals) and specificity (the proportion of negative test results among healthy individuals) metrics. Simulations were repeated until the robustness of the average value of the sensitivity and specificity metrics was ensured. The DIHCP-centered modules with at least 90% of the sensitivity and specificity values were considered statistically significant in this study and were accepted as a diagnostic DIHCP-centered module.

## 2.6. Prognostic Performance Analyses of Differentially Interacting Hallmarks of Cancer Protein Modules

To evaluate the prognostic performance of the DIHCP-centered modules, clinical information on 12 cancer types was collected from TCGA and used in the prognostic performance analyses. Prognostic abilities were evaluated by Kaplan-Meier plots and the log-rank test. All analyses were performed using the Survival package in R [14] (version 4.0.2). Samples were analyzed according to the prognostic index (PI), which is the linear component of the Cox model ($PI = \beta_1 \times 1 + \beta_2 x_2 + ... + \beta_p x_p$, where $x_i$ is the expression value of each gene, $\beta_i$ is the coefficient obtained from the Cox fit). The hazard ratio ($HR = (O_1/E_1)/(O_2/E_2)$) was calculated using the ratio between the relative mortality rate in group 1 and the relative mortality rate in group 2, where O and E are the observed and expected number of deaths, respectively. DIHCP-centered modules with a log-rank $p$-value < 0.01 were considered as statistically significant and accepted as a prognostic DIHCP-centered module in this study.

## 2.7. Enrichment Analyses of Diagnostic and Prognostic Modules

To obtain clues about the biological characteristics of the diagnostic and prognostic modules, over-representation analyses were performed using the bioinformatics tool Database for Annotation, Visualization and Integrated Discovery (DAVID) [16] to identify functional annotations (i.e., biological pathways) significantly associated with DIHCPs. Pathway p-values were determined using Fisher's exact test, and the Benjamini-Hochberg correction was used as a correction technique for multiple testing. Pathways with adjusted $p < 0.01$ were considered statistically significant.

Moreover, although the DIHCPs were already associated with hallmarks of cancer and to determine which cancer hallmarks are prominent and to further examine the distribution of the hallmarks, the components of the DIHCP-centered modules (i.e., DIHCPs) were subjected to enrichment analysis. For the analyses, we used the obtained cancer hallmark genes from the two repositories [10,11] and integrated them with the diagnostic and prognostic DIHCPs.

## 2.8. Druggability of the Diagnostic and Prognostic Differentially Interacting Hallmark of Cancer Protein Modules

The DIHCP-centered modules that were (i) accepted as diagnostic in this study, (ii) accepted as prognostic in this study, and (iii) all dHCPPIs that were suppressed or activated in the tumor state were included in the druggability analysis. Components of the DIHCP-centered modules that met the established criteria were screened for druggability using the Drug Gene Interaction Database (DGIdb v4.2.0) [17]. Only FDA-approved drugs were considered throughout the screening process. The types of protein-drug interaction (activator, suppressor, inhibitor, etc.) were also considered. Namely, if all dHCPPIs in the modules were suppressed, the drug candidates with activator activity were considered, or if all dHCPPIs in the modules were activated, the drug candidates with inhibitor activity were considered.
## 3. Results

### 3.1. Interpreting Differential Protein Interactome Algorithm in Human Cancers

To identify dHCPIPIs between tumor and normal tissue samples, the differential protein interaction algorithm [8] was independently applied to the gene expression profiles of the 12 cancer types and integrated with the reconstructed human protein interactome (7422 PPIs between 1906 proteins). Application of the algorithm yielded the hub proteins representing significant changes in the interaction patterns between “tumor phenotype” and “normal phenotype”, which we named “DIHCPs”.

We identified a total of 4405 dHCPIPIs among 832 different hallmarks of cancer proteins (i.e., DIHCPs) for 12 cancer types. While HNSC had the highest number of dHCPIPIs, KIRP had the lowest dHCPIPIs. The tumor specificity of dHCPIPIs varied by cancer type. Of all the dHCPIPIs found, 786 (18.5%) of the dHCPIPIs were specific to a cancer type. COAD had the highest number of specific interactions (230 specific dHCPIPIs), whereas there was one specific dHCPI for LUAD (0.2%) (Figure 3A). In addition, HNSC had the highest number of nonspecific dHCPIPIs. On the other hand, none of the dHCPIPIs was the same for all cancer types studied, and five interactions were the same in at most seven different cancers (i.e., CAV1-CTNNB1, COL1A1-IGFBP3, LDHA-LDHB, PCNA-GAPDH, and PSMB7-PSMB3). Comparative analysis of the interaction type (i.e., activated or suppressed) of these dHCPIPIs revealed that the dHCPIPIs were generally activated in the tumor state (88% activated).

<figure><img src="image_49.png" /><figcaption>**Figure 3.** Distribution of differential hallmark of cancer-associated protein-protein interactions and differentially interacting hallmark of cancer proteins. (A) The pie-donut chart showing the percentage of specific and non-specific differential hallmark of cancer-associated protein-protein interaction (dHCPIPIs) between cancer types. (B) The pie-donut chart showing the percentage of specific and non-specific differentially interacting hallmark of cancer proteins (DIHCPs) between cancer types.</figcaption></figure>

### 3.2. Differentially Interacting Hallmark Proteins and Modules in Human Cancers

Of the 832 DIHCPs identified, 26% of the DIHCPs were specific for one cancer type. There were two common proteins (EGFR and ESR1) that had DIHCP features in all cancers
studied. COAD had the highest number of specific DIHCPs. Namely, 16.9% of the DIHCPs of COAD were COAD-specific (Figure 3B).

The differential interactome showed a network topology with modular organization. Therefore, the DIHCPs (hub cancer hallmark proteins) together with their interacting protein partners, were referred to as modules, which we called “DIHCP-centered modules”. The DIHCP-centered modules with at least 10 components were considered, and each DIHCP-centered module was named after the name of the hub protein of the module.

In the comparative analysis of DIHCP-centered modules, we found that modules whose hubs belong to the protease or 14-3-3 protein complex shared the vast majority of DIHCPs. Therefore, DIHCP-centered modules sharing at least 70% of the common proteins were pooled based on cancer type to increase the robustness of DIHCP-centered modules. These pooled modules were designated as mPSMcomp or mYWHAcomp. For BRCA, four proteasome complex-associated hub modules were pooled. For COAD, two proteasome and two 14-3-3 protein complex-associated hub modules were pooled separately. In HNSC, 25 proteasome-associated and 3 of the 14-3-3 protein complex-associated hub modules were pooled. In LUAD and UCEC, five protease complex-associated hub modules were pooled independently. In LUSC, 31 proteasome complex-associated hub modules were pooled.

As a result, a total of 111 DIHCP-centered modules were identified (Table S1 (Supplementary Materials)). HNSC provided the highest number of DIHCP-centered modules (21 DIHCP-centered modules), while KIRP provided only 2 DIHCP-centered modules (Figure 4).

### 3.3. Diagnostic and Prognostic Power of Differentially Interacting Hallmarks of Cancer Protein-Centered Modules

In this study, the DIHCP-centered modules have the ability to be potential systems' biomarkers because we believe that the potential disease differences depending on diseased and control status are mainly due to the coordinated action of a group of biological entities. We further hypothesize that the DIHCP-centered module, as a systems' biomarker, would have high diagnostic and prognostic capabilities.

The diagnostic feature of each module was analyzed by PCA. Considering the most significant principal components (accounting for at least 80% of the total variance), sensitivity and specificity metrics were calculated, and modules with at least 90% of the sensitivity and specificity values were considered as diagnostic DIHCP-centered modules. Of the 111 DIHCP-centered modules, 39 had significantly high diagnostic performance (sensitivity ≥ 90% and specificity ≥ 90%), and some of these are shown in Figure 5. No diagnostic DIHCP-centered module could be observed for PRAD and THCA (Table S2).

The prognostic performance of the modules was assessed using Kaplan-Meier survival plots. The log-rank p-value and hazard ratios were considered to determine whether the DIHCP-centered modules had a high impact on overall patient survival. A total of 88 DIHCP-centered modules had a high impact on patients' overall survival (log-rank p-value < 0.01) in different cancer types (Figure 6), except for PRAD that had no effect on the prognostic DIHCP-centered modules (Table S2).
<figure><img src="image_50.png" /><figcaption>**Figure 4.** Differential interactome networks in 12 human cancers. For each cancer type, a differential interactome network was constructed around a differential hallmark of cancer-associated PPIs (dHCPPIs). The number of differentially interacting hallmarks of cancer proteins (DIHCPs) for each cancer type was presented.</figcaption></figure>
<figure><img src="image_51.png" /><figcaption>**Figure 5.** Principal component analyses for different cancer types. PCA plots showing individual differences in protein expression profiles between cancer types comprising at least 30 individuals in each type.</figcaption></figure>
<figure><img src="image_52.png" /><figcaption>**Figure 6.** Prognostic analyses for different types of cancer. Kaplan-Meier Plots estimating patients' survival for 11 cancers with p-value and hazard ratio given for each curve.</figcaption></figure>

Among the DIHCP-centered modules, there were a total of 30 DIHCP-centered modules that showed a high diagnostic or prognostic performance in any of the cancer types: one module in BRCA (mPPP1CA), KIRC (mCCND1) and STAD (mHSP90AA1); two modules in HNSC (mCAV1 and mEGFR), KIRP (mHSPB1 and mMET) and LIHC (mHSP90AA1 and mHSPB1); three modules in UCEC (mGNB1, mPCNA and mSFN); four modules in LUAD (mNPM1, mPPP1CA, mSTAT1, and mVIM); seven modules in COAD (mCDC37, mCTNNB1, mGNB2, mJUN, mMYC, mNONO, and mYWHAcomp); and LUSC (mCAV1, mCDH1, mNONO, mPCNA, mPPP1CA, mPSMcomp, and mYWHAG) indicated both diagnostic and prognostic properties.

### 3.4. Enrichment Analyses of Diagnostic and Prognostic Protein Modules

Pathway and hallmark enrichment analyses were performed to obtain further biological characteristics of the diagnostic and prognostic 30 modules. The DIHCPs pathway over-representation analysis of modules based on annotations stored in the KEGG database revealed various pathways (Figure 7). For instance, cancer-associated pathways such as
bladder cancer, chronic myeloid leukemia, gastric cancer, glioma, and melanoma come into prominence. Signaling pathways such as Hippo, JAK-STAT, MAPK, PI3K-AKT, and WNT, which are known to be associated with cancer development and progression, were found to be statistically significant. Interestingly, viral carcinogenesis and viral infections associated with cancer (i.e., HBV, HPV, HTLV1, and KSHV) [18] were remarkable pathways associated with DIHCPs. Moreover, the hallmarks of cancer comprise ten distinct biological capabilities derived during the multi-staged development of tumors; however, to reveal the most fundamental trait of cancer cells, hallmark enrichment analyses were performed in di-agnostic and prognostic DIHCP-centered modules (Figure 8A). Among the ten hallmarks of cancer, DIHCPs were most enriched in sustaining proliferative signaling, presumably due to the fact that unlimited replication is essential in cancer cells for tumor development. In addition, genome instability and mutation and enabling replicative immortality hallmarks remain in the background when compared to others.

<figure><img src="image_53.png" /><figcaption>**Figure 7.** The pathway enrichment analyses results. The bubble plot indicates the diagnostic and prognostic 30 modules differentially interacting hallmark of cancer protein's (DIHCP's) pathway enrichment results according to q-value (−log₁₀) significance.</figcaption></figure>
<figure><img src="image_54.png" /><figcaption>**Figure 8.** The hallmark enrichment and druggability analyses results. (A) The heat-map represents the hallmark enrichment results of the module's differentially interacting hallmark of cancer proteins (DIHCPs). AID: avoiding immune destruction; AIM: activating invasion and metastasis; DCE: deregulating cellular energetics; ERI: enabling replicative immortality; EGS: evading growth suppressors; GIM: genome instability and mutation; IA: inducing angiogenesis; RCD: resisting cell death; SPS: sustaining proliferative signaling; TPI: tumor-promoting inflammation. (B) The druggability distribution of module's DIHCPs. The bar graph indicates the number of DIHCPs, which are druggable (light grey bars) and undruggable (dark grey bars).</figcaption></figure>

### 3.5. Druggability of Differentially Interacting Hallmarks of Cancer Proteins

The DIHCPs were promising therapeutic targets because they represent significant changes in the interaction patterns between tumor and normal phenotype. For this reason, we investigated the druggability of the modules. Of the 30 diagnostic and prognostic DIHCP-centered modules, the DIHCPs of 17 modules were examined according to the established third criteria, because 13 of the dHCPPIs modules had both suppressed and activated interactions that did not meet our third criteria (see Materials and Methods).

We found that of the 17 DIHCP-centered modules, 28% of the DIHCPs were druggable. The mPSMcomp belonging to LUSC had the highest percentage (66%) of druggable proteins, whereas the lowest number of druggable DIHCPs among all cancers analyzed belonged to mGNB2 (10%) of COAD (Figure 8B).

A total of 200 different drugs targeting DIHCPs were found. A total of 54% of the drugs were specific for only one cancer type. None of the drugs were specific for all cancers studied, and 25 of the drugs were specific for no more than 6 different cancers (Table S3).

## 4. Discussion

Complex diseases such as cancer usually arise as a result of interactions between biological entities. The biological molecules that trigger the development of a cancer or biological process never function alone; they work together in a complex, interconnected network to carry out these biological functions. The state of interconnectedness of molecules establishes the phenomenon of the "biological network". A biological network is an advanced concept that allows researchers to model, characterize, and decipher complex
interactions between different biologically relevant entities in a biological system [19]. Biological networks thus change our perspective and shed light on the internal organization of the cell, allowing us to understand the mechanisms underlying complex diseases. Moreover, predictive, preventive, and personalized medicine is a holistic healthcare strategy that aims to predict individual predisposition, provide targeted prevention, and provide personalized treatment [20]. To advance personalized medicine strategies, the discovery of new treatments is essential. However, with today's financial resources, it is nearly impossible to discover drugs from scratch using traditional methods. Using biological network analysis, researchers can also discover and/or predict anti-cancer drugs [21]. We believe that network-based systems biology is a new approach for discovering treatment and prevention strategies, especially for cancer, which is currently a major burden worldwide, and researchers are embracing this concept. For example, researchers have integrated gene expression profiling with multiple networks to identify new biomarkers and drug candidates for breast cancer [22], pancreatic cancer [23], cervical cancer [24], acute myeloid leukemia [25], thyroid cancer [26], and so on. Accordingly, these holistic analyses enable the discovery of potential biomarkers for diagnosis, prognosis, or therapeutic purposes. Therefore, biological networks hold great potential for the present and the future, especially for complex diseases such as cancer [27].

The protein-protein interactome represents a biological network that encompasses all physical protein interactions within the cell. These protein-protein interactions play a crucial role in the living organism. These are essential for almost all cellular functions, involved in the physiology and biochemistry of the organism. These can also influence cancer cell growth, transmit oncogenic signals, and even cause the development of typical cancer features. Targeting such interactions is a promising strategy for new drug development [28,29]. Thus, it is strongly suggested that significant changes in protein-protein interactions occur depending on the phenotype of the individual (i.e., diseased versus healthy) [30].

We applied our constructed differential interactome algorithm, which had proven to be a powerful tool in other studies [8,31,32], for the protein-protein interactome data on hallmarks of cancer. Our results revealed DIHCPs showing remarkable changes in the interaction patterns of patients during the transition from the “normal” phenotype to the “cancer” phenotype. As mentioned earlier, cancers occur and spread usually due to the coordinated action of a group of biological molecules. In this study, DIHCPs and the modular structures around these proteins (DIHCP-centered modules) were considered as potential systems’ biomarkers. However, we believe that a valid biomarker must provide prognostic or diagnostic information about the specific cancer. For this reason, to provide confidence in the precise diagnostic and prognostic capacity of the systems’ biomarkers we clearly demonstrated their diagnostic and prognostic performances by PCA and survival analyses and demonstrated high biomarker performances in multiple tumors (30 diagnostic and prognostic DIHCP-centered modules).

In this study, we only used hallmark of cancer proteins, while we integrated proteomics data to apply the differential interactome algorithm. Hallmarks of cancer proteins are considered as driver proteins of tumorigenesis. These proteins are responsible for the most basic phenotypic features of tumor initiation and progression [3]. Therefore, we hypothesized that we can highlight the principles and mechanisms of tumorigenesis by focusing only on the proteins essential for carcinogenesis (hallmarks of cancer proteins), we can reduce the complexity of the disease, and typify the process by characteristic and complementary features. In addition, if the potential system biomarker components (i.e., DIHCPs) presented in the study are all among the hallmark characteristics of human cancers, we believe we will identify highly robust cancer biomarkers. Moreover, since these hallmarks represent features of cancer cells, recent therapeutic approaches have been increasingly aiming at targeting hallmark proteins [33]. Therefore, the systems’ biomarkers identified in this study will have remarkable clinical value. For this reason, this study also evaluated the druggability of the diagnostic and prognostic systems’ biomarkers.
The hubs of the diagnostic and prognostic systems' biomarkers are mostly associated with the sustaining proliferative signaling involved as an important feature in the hallmarks of cancer. It is known that cancer cells can stimulate their own proliferation indefinitely and this cancer hallmark characteristic is chronically activated in all cancers. The well-known mechanism for sustaining proliferative signaling is to activate oncogenes [34]. Oncogenes encode proteins that stimulate cell proliferation and programmed cell death. An oncogene arises when a proto-oncogene is altered genetically (i.e., mutated) and over-expressed. For instance, in cancers, the most noticeable oncogenes are BRAF and RAS [35]. Moreover, resisting the cell-death hallmark feature comes into prominence for diagnostic and prognostic hubs. Resistance to cell death is a natural barrier to cancer development and is mostly regulated by programmed cell death. The best-known form of programmed cell death is apoptosis, in which cells are destined to die. In addition to apoptosis, necroptosis is referred to as the second important process for inducing cell death. Apoptosis and necroptosis cause different immune responses. Necroptosis releases molecules that promote inflammation, while apoptosis triggers silent immunological responses [36]. Autophagy is another process that leads to cell death [34]. The autophagy process allows cells to remove unnecessary or dysfunctional components. In cancer, autophagy may play a tumor-suppressor or oncogene role under certain conditions and at certain stages of carcinogenesis [37]. A recent review systematically described the hallmarks of different cell death modes [38].

The major limitation of the study is the lack of experimental validations of the diagnostic and prognostic modules with relevant tissue samples or cell lines. Therefore, the most important aspect of this study is to translate these computational findings into experimental approaches. For example, future in vitro studies need to be performed to investigate the impact of the identified modules in terms of their response to cell viability, cell migration and disease progression. In addition, the accuracy, consistency, reproducibility, and reliability of the biomarkers presented in this study should be experimentally validated if they prove to be clinically useful. In addition, the mechanism of action of these modules needs to be experimentally evaluated to clearly elucidate their effects on the hallmarks of cancer characteristics. We believe that computational analysis is an important and first step in biomarker development. However, to address the broad medical and scientific audience, the need for experimental validation is inevitable.

In conclusion, the knowledge of specific proteins being impacted within a given hallmark of the patient's cancer type could offer the opportunity to personalize treatments. One such approach would be to target proteins from the proteins in the enrichment analysis that match the patient's specific tumor type and prioritize repurposing the drugs proposed from our analysis. Furthermore, our investigation on the hallmarks of cancer protein provides valuable data for further experimental and clinical efforts in a variety of cancer types since the proposed systems' biomarkers have the potential to be diagnostic and/or prognostic. Moreover, the protein-protein interactions could be utilized as therapeutic targets.

**Supplementary Materials:** The following supporting information can be downloaded at: https://www.mdpi.com/article/10.3390/jpm12111919/s1, Table S1: Differentially interacting hallmark of cancer protein (DIHCP)-centered modules; Table S2: Diagnostic and prognostic analysis of differentially interacting hallmark of cancer protein (DIHCP)-centered modules; Table S3: Drugs targeting module's differentially interacting hallmark of cancer proteins (DIHCPs).

**Author Contributions:** Conceptualization, K.Y.A. and R.S.; Methodology, K.Y.A. and R.S.; Formal Analysis, M.K. and G.E.O.; Investigation, M.K. and G.E.O.; Writing—Original Draft Preparation, M.K.; Writing—Review and Editing, M.K., G.E.O., K.Y.A. and R.S.; Supervision, K.Y.A. and R.S. All authors have read and agreed to the published version of the manuscript.

**Funding:** This research received no external funding.

**Institutional Review Board Statement:** Not applicable.

**Informed Consent Statement:** Not applicable.
**Data Availability Statement:** Publicly available datasets were analyzed in this study. The datasets analyzed during the current study are available in The Genome Cancer Atlas (https://portal.gdc.cancer.gov/, accessed on 1 March 2022). Protein interactome data are available in Biological General Repository for Interaction Datasets (https://thebiogrid.org, accessed on 30 March 2022). The cancer hallmark gene data are available in the Catalogue of Somatic Mutations in Cancer (https://cancer.sanger.ac.uk/cosmic, accessed on 23 May 2022) and Cancer Hallmark Genes (http://www.bio-bigdata.com/CHG/index.html, accessed on 23 May 2022). Source codes for the differential interactome algorithm (implemented in R, version 4.0.2) are freely available at http://sysbio.bioe.eng.marmara.edu.tr/diff-int-ome, accessed on 25 July 2022.

**Acknowledgments:** The author(s) received no financial support for the research, authorship, and/or publication of this article.

**Conflicts of Interest:** The authors declare no conflict of interest.

## References

1. Sung, H.; Ferlay, J.; Siegel, R.L.; Laversanne, M.; Soerjomataram, I.; Jemal, A.; Bray, F. Global Cancer Statistics 2020: GLOBOCAN Estimates of Incidence and Mortality Worldwide for 36 Cancers in 185 Countries. *CA Cancer J. Clin.* **2021**, *71*, 209–249. [CrossRef] [PubMed]

2. Ravi, S.; Alencar, A.M.; Arakelyan, J.; Xu, W.; Stauber, R.; Wang, C.I.; Papyan, R.; Ghazaryan, N.; Pereira, R.M. An Update to Hallmarks of Cancer. *Cureus* **2022**, *14*, e24803. [CrossRef] [PubMed]

3. Hanahan, D.; Weinberg, R.A. The hallmarks of cancer. *Cell* **2000**, *100*, 57–70. [CrossRef]

4. Hanahan, D.; Weinberg, R.A. Hallmarks of cancer: The next generation. *Cell* **2011**, *144*, 646–674. [CrossRef]

5. Hanahan, D. Hallmarks of Cancer: New Dimensions. *Cancer Discov.* **2022**, *12*, 31–46. [CrossRef] [PubMed]

6. Nagy, Á.; Munkácsy, G.; Győrfffy, B. Pancancer survival analysis of cancer hallmark genes. *Sci. Rep.* **2021**, *11*, 6047. [CrossRef] [PubMed]

7. Yu, L.H.; Huang, Q.W.; Zhou, X.H. Identification of Cancer Hallmarks Based on the Gene Co-expression Networks of Seven Cancers. *Front. Genet.* **2019**, *10*, 99. [CrossRef]

8. Gulfidan, G.; Turanli, B.; Beklen, H.; Sinha, R.; Arga, K.Y. Pan-cancer mapping of differential protein-protein interactions. *Sci. Rep.* **2020**, *10*, 3272. [CrossRef]

9. Tomczak, K.; Czerwińska, P.; Wiznerowicz, M. The Cancer Genome Atlas (TCGA): An immeasurable source of knowledge. *Contemp. Oncol. (Pozn)* **2015**, *19*, A68–A77. [CrossRef]

10. Zhang, D.; Huo, D.; Xie, H.; Wu, L.; Zhang, J.; Liu, L.; Jin, Q.; Chen, X. CHG: A Systematically Integrated Database of Cancer Hallmark Genes. *Front. Genet.* **2020**, *11*, 29. [CrossRef]

11. Tate, J.G.; Bamford, S.; Jubb, H.C.; Sondka, Z.; Beare, D.M.; Bindal, N.; Boutselakis, H.; Cole, C.G.; Creatore, C.; Dawson, E.; et al. COSMIC: The Catalogue of Somatic Mutations in Cancer. *Nucleic Acids Res.* **2019**, *47*, D941–D947. [CrossRef] [PubMed]

12. Stelzer, G.; Rosen, N.; Plaschkes, I.; Zimmerman, S.; Twik, M.; Fishilevich, S.; Stein, T.I.; Nudel, R.; Lieder, I.; Mazor, Y.; et al. The GeneCards Suite: From Gene Data Mining to Disease Genome Sequence Analyses. *Curr. Protoc. Bioinform.* **2016**, *54*, 1.30.1–1.30.33. [CrossRef]

13. Oughtred, R.; Stark, C.; Breitkreutz, B.J.; Rust, J.; Boucher, L.; Chang, C.; Kolas, N.; O'Donnell, L.; Leung, G.; McAdam, R.; et al. The BioGRID interaction database: 2019 update. *Nucleic Acids Res.* **2019**, *47*, D529–D541. [CrossRef]

14. Huber, W.; Carey, V.J.; Gentleman, R.; Anders, S.; Carlson, M.; Carvalho, B.S.; Bravo, H.C.; Davis, S.; Gatto, L.; Girke, T.; et al. Orchestrating high-throughput genomic analysis with Bioconductor. *Nat. Methods* **2015**, *12*, 115–121. [CrossRef] [PubMed]

15. Shannon, P.; Markiel, A.; Ozier, O.; Baliga, N.S.; Wang, J.T.; Ramage, D.; Amin, N.; Schwikowski, B.; Ideker, T. Cytoscape: A software environment for integrated models of biomolecular interaction networks. *Genome Res.* **2003**, *13*, 2498–2504. [CrossRef] [PubMed]

16. Sherman, B.T.; Hao, M.; Qiu, J.; Jiao, X.; Baseler, M.W.; Lane, H.C.; Imamichi, T.; Chang, W. DAVID: A web server for functional enrichment analysis and functional annotation of gene lists (2021 update). *Nucleic Acids Res.* **2022**, *50*, W216–W221. [CrossRef]

17. Freshour, S.L.; Kiwala, S.; Cotto, K.C.; Coffman, A.C.; McMichael, J.F.; Song, J.J.; Griffith, M.; Griffith, O.L.; Wagner, A.H. Integration of the Drug-Gene Interaction Database (DGIdb 4.0) with open crowdsource efforts. *Nucleic Acids Res.* **2021**, *49*, D1144–D1151. [CrossRef]

18. Kori, M.; Arga, K.Y. Human oncogenic viruses: An overview of protein biomarkers in viral cancers and their potential use in clinics. *Expert Rev. Anticancer Ther.* **2022**, 1–14. [CrossRef]

19. Zhang, B.; Tian, Y.; Zhang, Z. Network biology in medicine and beyond. *Circ. Cardiovasc. Genet.* **2014**, *7*, 536–547. [CrossRef]

20. Goetz, L.H.; Schork, N.J. Personalized medicine: Motivation, challenges, and progress. *Fertil Steril.* **2018**, *109*, 952–963. [CrossRef]

21. Li, K.; Du, Y.; Li, L.; Wei, D.Q. Bioinformatics Approaches for Anti-cancer Drug Discovery. *Curr. Drug Targets.* **2020**, *21*, 3–17. [CrossRef] [PubMed]

22. Khan, A.; Rehman, Z.; Hashmi, H.F.; Khan, A.A.; Junaid, M.; Sayaf, A.M.; Ali, S.S.; Hassan, F.U.; Heng, W.; Wei, D.Q. An Integrated Systems Biology and Network-Based Approaches to Identify Novel Biomarkers in Breast Cancer Cell Lines Using Gene Expression Data. *Interdiscip. Sci.* **2020**, *12*, 155–168. [CrossRef] [PubMed]
23. Kaushik, A.C.; Wang, Y.J.; Wang, X.; Wei, D.Q. Irinotecan and vandetanib create synergies for treatment of pancreatic cancer patients with concomitant TP53 and KRAS mutations. *Brief Bioinform.* **2021**, *22*, bbaa149. [CrossRef] [PubMed]

24. Kori, M.; Arga, K.Y.; Mardinoglu, A.; Turanli, B. Repositioning of Anti-Inflammatory Drugs for the Treatment of Cervical Cancer Sub-Types. *Front. Pharmacol.* **2022**, *13*, 884548. [CrossRef]

25. Kelesoglu, N.; Kori, M.; Turanli, B.; Arga, K.Y.; Yilmaz, B.K.; Duru, O.A. Acute Myeloid Leukemia: New Multiomics Molecular Signatures and Implications for Systems Medicine Diagnostics and Therapeutics Innovation. *OMICS* **2022**, *26*, 392–403. [CrossRef]

26. Gulfidan, G.; Soylu, M.; Demirel, D.; Erdonmez, H.; Beklen, H.; Ozbek Sarica, P.; Arga, K.Y.; Turanli, B. Systems biomarkers for papillary thyroid cancer prognosis and treatment through multi-omics networks. *Arch Biochem. Biophys.* **2022**, *15*, 109085. [CrossRef]

27. Yan, W.; Xue, W.; Chen, J.; Hu, G. Biological Networks for Cancer Candidate Biomarkers Discovery. *Cancer Inform.* **2016**, *15*, 1–7. [CrossRef]

28. Lu, H.; Zhou, Q.; He, J.; Jiang, Z.; Peng, C.; Tong, R.; Shi, J. Recent advances in the development of protein-protein interactions modulators: Mechanisms and clinical trials. *Sig. Transduct. Target. Ther.* **2020**, *5*, 213. [CrossRef]

29. Ivanov, A.A.; Khuri, F.R.; Fu, H. Targeting protein-protein interactions as an anticancer strategy. *Trends Pharmacol. Sci.* **2013**, *34*, 393–400. [CrossRef]

30. Caldera, M.; Buphamalai, P.; Müller, F.; Menche, J. Interactome-based approaches to human disease. *Curr. Opin. Syst. Biol.* **2017**, *3*, 88–94. [CrossRef]

31. Ayyildiz, D.; Gov, E.; Sinha, R.; Arga, K.Y. Ovarian cancer differential interactome and network entropy analysis reveal new candidate biomarkers. *OMICS* **2017**, *21*, 285–294. [CrossRef] [PubMed]

32. Caliskan, A.; Gulfidan, G.; Sinha, R.; Arga, K.Y. Differential interactome proposes subtype-specific biomarkers and potential therapeutics in renal cell carcinomas. *J. Pers. Med.* **2021**, *11*, 158. [CrossRef] [PubMed]

33. Sameri, S.; Mohammadi, C.; Mehrabani, M.; Najafi, R. Targeting the hallmarks of cancer: The effects of silibinin on proliferation, cell death, angiogenesis, and migration in colorectal cancer. *BMC Complement. Med. Ther.* **2021**, *21*, 160. [CrossRef] [PubMed]

34. Hanahan, D.; Weinberg, R.A. Biological hallmarks of cancer. In *Holland-Frei Cancer Medicine*, 9th ed.; Bast, R.C., Hait, W.N., Kufe, D.W., Weichselbaum, R.R., Holland, J.F., Croce, C.M., Piccart-Gebart, M., Wang, H., et al., Eds.; Wiley-Blackwell: Hoboken, NJ, USA, 2017; pp. 7–16, ISBN 978-1-118-93469-2.

35. Botezatu, A.; Iancu, I.V.; Popa, O.; Plesa, A.; Manda, D.; Huica, I.; Vladoiu, S.; Anton, G.; Badiu, C. Mechanisms of Oncogene Activation. In *New Aspects in Molecular and Cellular Mechanisms of Human Carcinogenesis*; Bulgin, D., Ed.; IntechOpen: London, UK, 2016; pp. 1–52, ISBN 978-953-51-2253-1.

36. Heckmann, B.L.; Tummers, B.; Green, D.R. Crashing the computer: Apoptosis vs. necroptosis in neuroinflammation. *Cell Death Differ.* **2019**, *26*, 41–52. [CrossRef]

37. Li, X.; He, S.; Ma, B. Autophagy and autophagy-related proteins in cancer. *Mol. Cancer* **2020**, *19*, 12. [CrossRef]

38. Yan, G.; Elbadawi, M.; Efferth, T. Multiple cell death modalities and their key features (Review). *World Acad. Sci. J.* **2020**, *2*, 39–48. [CrossRef]
Article

# Dynamic Collaborations for the Development of Immune Checkpoint Blockade Agents

Arisa Djurian ¹D, Tomohiro Makino <fnref n="1" />, Yeongjoo Lim ²D, Shintaro Sengoku <fnref n="3" /> and Kota Kodama <fnref n="1" />,<fnref n="4" />,*D

**Citation:** Djurian, A.; Makino, T.; Lim, Y.; Sengoku, S.; Kodama, K. *Dynamic Collaborations for the Development of Immune Checkpoint Blockade Agents.* *J. Pers. Med.* **2021**, *11*, 460. https://doi.org/10.3390/jpm11060460

**Academic Editor:**
Anne-Marie Caminade

Received: 28 April 2021

Accepted: 18 May 2021

Published: 24 May 2021

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

<figure><img src="image_55.png" /></figure>

**Copyright:** © 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

**Abstract:** We studied the overview of drug discovery and development to understand the recent trends and potential success factors of interorganizational collaboration by reviewing 1204 transactions performed until 2019 for 107 anticancer drugs approved by the US Food and Drug Administration (FDA) from 1999 to 2018. Immune checkpoint blockade was found to be a significantly active area in interorganizational transactions, especially the number of alliances, compared with other mechanisms of action of small molecules and biologics for cancer treatment. Furthermore, the analysis of pembrolizumab and nivolumab showed that the number of approved indications for these two drugs has been rapidly expanding since their first approval in 2014. Examination of the acquisitions and alliances regarding pembrolizumab and nivolumab showed that many combination partners were developed by US-based biotechnology or start-up companies, the majority of which were biologics. These findings suggest that immune checkpoint blockade is a paradigm for cancer treatment, resulting in huge product sales and continuous indication expansion. Additionally, interorganizational collaboration, especially trial collaboration, is a strategic approach for the development of immune checkpoint blockade agents. The translation of these empirical practices to new drug candidates is expected for the research and development of innovative drugs in the future.

**Keywords:** cancer treatment; immune checkpoint blockade; PD-(L)1 inhibitors; interorganizational transaction; combination therapy

## 1. Introduction

Business in the pharmaceutical industry is unique in terms of product universality, where a company needs to research and develop each pipeline of drugs according to strict local regulations and requirements to obtain approval from the respective regulatory authority to launch their products. Therefore, drug research and development (R&D) is an extremely costly and time-consuming process [1-4]. In addition, business success in the pharmaceutical industry requires innovative products and expansion of product values to secure high pricing, grant of reimbursement and longer market exclusivities. Recently, R&D has become more complicated as modality becomes diverse by introducing new technologies and knowledge to address existing unmet medical needs in specific target populations. Thus, companies in the pharmaceutical industry have strived to enhance sustainability by adopting various strategic approaches, such as pursing globalized or region-oriented business models [5,6], selective therapeutic areas in R&D, optimizing R&D productivity [7], and open innovation, including external collaboration and interorganizational transactions [8-11]. In particular, the importance of external collaboration in R&D has been disseminated and many pharmaceutical companies have pursued interorganizational

<footnote n="1">¹ Graduate School of Technology Management, Ritsumeikan University, Osaka 567-8570, Japan; gr0434hi@ed.ritsumei.ac.jp (A.D.); makino.tomohiro@gmail.com (T.M.)</footnote>

<footnote n="2">² Faculty of Business Administration, Ritsumeikan University, Osaka 567-8570, Japan; lim40@fc.ritsumei.ac.jp</footnote>

<footnote n="3">³ School of Environment and Society, Tokyo Institute of Technology, Tokyo 108-0023, Japan; sengoku.s.aa@m.titech.ac.jp</footnote>

<footnote n="4">⁴ Center for Research and Education on Drug Discovery, The Graduate School of Pharmaceutical Sciences in Hokkaido University, Sapporo 060-0812, Japan</footnote>

<footnote>* Correspondence: kkodama@fc.ritsumei.ac.jp; Tel.: +81-726652448; Fax: +81-726652448</footnote>
transactions and established open innovation platforms to acquire external knowledge and
pipelines across organizations [8,10–13].

In the pharmaceutical industry, cancer therapeutics have a long history in the trans-
formation of treatment options, including chemotherapy [14,15], targeted therapy [16],
biologics [17], and combination therapy [18]. The recent hot spot is immuno-oncology [19,20],
in which immune checkpoint blockade is a paradigm. Currently available immune check-
point inhibitors are cytotoxic T lymphocyte-associated antigen 4 (CTLA4) inhibitors, pro-
grammed cell death protein 1 (PD-1) inhibitors, and programmed death-ligand 1 (PD-L1)
inhibitors. The first entry into the market for immune checkpoint blockade was the anti-
CTLA4 antibody ipilimumab (Bristol-Myers Squibb (BMS) (New York, NY, USA), and
Medarex (Princeton, NJ, USA)), approved for metastatic melanoma in 2011 by the Food
and Drug Administration (FDA) in the US. Subsequently, two anti-PD-1 antibodies, pem-
brolizumab (Merck) and nivolumab (BMS), were approved for metastatic melanoma in
2014 [21]. A number of PD-1 or PD-L1 inhibitors have been approved or are under devel-
opment [22,23], thereby expanding the target tumor types [24] not only in the US but also
in Europe, Japan, and other regions.

After the isolation of PD-1 by Dr. Honjo's laboratory in Japan in 1992 [25], there was a long lag period until its first approval in 2014. The approval process involved extensive collaborations, supported by academia, biotechnology companies, pharmaceutical companies, biomarker companies, non-profit organizations, and regulatory agencies [20,26]. Chen and Han reviewed several important histories regarding anti-PD-(L)1 therapy for human cancer [26]. For example, the discovery of the PD pathway resulted from several collaborations by scientists, belonging to different laboratories, who identified and isolated at least five interacting molecules, PD-1 [25], PD-L1 (B7 homolog 1 [B7-H1]) [27,28], PD-L2 (B7-DC) [29,30], CD80 (B7-1) [31,32], and molecule family member b (RGMb) [33], and explored their functions and discovered mutual interactions [28,34–37]. Hoos suggested that collaboration among Medarex, BMS, a community organization called the Cancer Immunotherapy Consortium (CIC) of the Cancer Research Institute (New York, NY, USA), and CIC's partners would generate breakthrough in the clinical evaluation of cancer immunotherapies by establishing new criteria and evaluation method. This led to the success of ipilimumab in clinical trials and eventually contributed to the acquisition of Medalex and responsible regulatory guidance by the FDA and the European Medicine Agency (EMA) (Amsterdam, The Netherlands). During the R&D of immune checkpoint blockade agents, there were important interorganizational transactions. For example, LifeArc (London, UK), which has a technology to generate a humanized clinical candidate, engineered pembrolizumab in collaboration with Organon in 2007 [38], and Merck acquired Organon later that year. Medarex, which was a biopharmaceutical company focused on the discovery, development, and potential commercialization of fully human antibody-based therapeutics and engineered ipilimumab and nivolumab, was acquired by BMS in 2009 [39]. Additionally, the development of combination therapy in this segment is quite active before and even after product launch [10,40,41] involving the collaboration of multiple companies. Given these previous successful practices, one of the effective approaches for sustainable business in the discovery and development of anticancer drugs may consider how a company that tries to generate a potential innovative drug can be a pioneer in terms of open innovation. Our interests were to understand whether immune checkpoint blockade is the active area in terms of interorganizational collaboration in anticancer drug development, and whether there are strategic collaborations with various external partners behind the recent successes of the indication and market expansion of immune checkpoint blockade agents. However, no previous study has investigated the trends in interorganizational collaborations for the R&D of immune checkpoint blockade that potentially led to successful drug development.

Therefore, the objective of this study was to understand the recent trends and success
factors, based on various experiences of active transactions, for the development of immune
checkpoint blockade agents. The definition of success in our study is a positive outcome de-
rived from drug development or interorganizational transactions which enable a company
to maintain sustainable growth, specifically drug approvals, new indication approvals, and market sales. We used the number of approved indications and market sales amount as proxies for success. We also investigated the important transactions in the discovery and development of immune checkpoint blockade agents, including extensive combination therapy development, especially pembrolizumab and nivolumab, which triggered the paradigm of standard of care in cancer therapeutics. In the present study, we investigated these from the angle of the mechanism of action (MOA), which defines how a drug or other substance produces an effect in the body; for example, how it affects a specific target in a cell, such as an enzyme, or a cell function, such as cell growth. Therefore, the efficacy and safety of drugs are highly dependent on the MOA [42]. Immune checkpoint blockade is the current major MOA in cancer therapy, and we expect that comparisons between immune checkpoint blockade and other MOAs of anticancer drugs are valuable for our research objectives. As an immune checkpoint blockade agent is a monoclonal antibody, that is, a biological anticancer drug, we precisely compared immune checkpoint blockade agents with other biologics. We also added small molecules for comparison, which are the most conventional therapeutic agents used in cancer treatment [14,15]. These small molecules were divided into two groups, namely kinase inhibitors (which are the major MOAs of small molecules) and other small molecules. Our goal was to uncover the potential key success factors of R&D for generating innovative and novel drugs to fulfill unmet medical needs in the future.

## 2. Materials and Methods

### 2.1. *Samples and Data Sources*

Sample data on cancer drugs were collected from the FDA's New Molecular Entity (NME) list of approved small molecules and the New Biological Entity (NBE) list of approved biologics [24] from 1999 to 2018. Target drugs were determined based on CenterWatch's list [43] of new cancer drug approvals, which was cross-referenced against the FDA's NME and NBE lists. Using this approach, we selected 77 small molecules and 30 biologics as samples for this study.

The data source of approved indications for pembrolizumab and nivolumab was a package insert published by the FDA [24] from 2014 to 2019. The data source of product sales in pembrolizumab and nivolumab was Form 10-K from 2014 to 2019. The data source of company type and development phase of combination partners tested with pembrolizumab or nivolumab in interorganizational transactions was the Biomedtracker Deal Search [44].

### 2.2. *Variables and Data Sources*

Information on the number of transactions per product was collected from the Informa database's Biomedtracker Deal Search [44]. We examined 1204 transactions related to the identified 107 cancer drugs approved by the FDA. The deal types defined by Informa included "acquisitions," "alliances," and "financing." Each concept of "acquisitions," "alliances," and "financing." are defined as follows:

* An acquisition is when one company purchases all or most of another company's shares to gain control of that company.

* An alliance is an agreement between two or more companies regarding a pharmaceutical product, technology, service, etc.

* A financing involves a company raising money publicly or privately through the sale of equity, debt, or royalty monetization.

The deal characteristics defined by in the alliances consisted of "co-promotions," "includes contract," "includes equity," "includes royalties or profit split information," "intra-biotech deals," "marketing-licensing," "product or technology swaps," "R+D and marketing-licensing," "trial collaborations," and "reverse licensing." The development phase is derived from each company's source information since the information on Biomed-
tracker Deal Search is obtained exclusively from publicly available sources such as company press releases, medical conference presentations, etc.

## 2.3. Statistical Analysis

SPSS Statistics 27 (IBM, Armonk, NY, USA) and Microsoft Excel 2016 (Microsoft, Redmond, WA, USA) were used for the statistical analysis. We performed multiple comparison Tukey–Kramer tests corresponding to 95% confidence intervals using a pre-set significance level (two-sided *p*-value of 0.05). The Tukey–Kramer test is the extension of Tukey’s test. Both Tukey’s test and the Tukey–Kramer test are a parametric multiple comparisons procedure and applies simultaneous to the set of all pairwise comparisons to find means that are significantly different from each other. We chose the Tukey–Kramer test which is used in the case of unequal samples sizes while Tukey’s test is used in the case of equal samples sizes. In the figures, the obtained *p*-values are presented as follows: *p* < 0.01 as **, *p* < 0.05 as *, and † *p* < 0.1 as †.

# 3. Results

## 3.1. *Interorganizational Transaction per Mechanism of Action*

We initially investigated the trend of interorganizational transactions for drug discovery and development in the oncology area by reviewing 1193 transactions performed until July 2019. We performed a multiple comparison Tukey–Kramer test to compare the number of transactions among four categories of MOA: immune checkpoint blockade (which is the largest number of interorganizational transactions in biological anticancer drugs approved under Biologics License Application (BLA)), other biologics approved under BLA, kinase inhibitor (which is the largest number of interorganizational transactions in small molecule anticancer drugs under New Drug Application (NDA)), and other small molecules approved under NDA. The number of interorganizational transactions in immune checkpoint blockade was significantly higher than that in other MOAs (immune checkpoint blockade: other biologics: kinase inhibitors: other small molecules’ total = 44.00, SD 31.97: 11.30, SD 16.21: 6.95, SD 7.70: 9.26, SD 7.09, *p* < 0.05; acquisition = 3.29, SD 3.30: 1.17, SD 1.95: 0.68, SD 1.12: 1.46, SD 1.77, *p* < 0.05; alliances = 35.43, SD 24.67: 8.65, SD 12.63: 5.45, SD 5.94: 6.08, SD 5.40, *p* < 0.05; financing = 5.29, SD 4.89: 1.48, SD 2.25: 0.82, SD 1.66: 1.72, SD 2.25, *p* < 0.05; Figure 1). It has also been suggested that alliances are the major category of interorganizational transactions in immune checkpoint blockade.

<figure><img src="image_56.png" /><figcaption>**Figure 1.** Multiple comparisons of the number of interorganizational transactions by the pairwise comparison procedure among immune checkpoint blockade agents, other biologics, kinase inhibitors and other small molecules. *n* = 77 (small molecules) and 30 (biologics). \* *p* < 0.05.</figcaption></figure>
We then explored the main purpose of transactions for alliances in immune checkpoint blockade. We classified the deal characteristics carefully. Some deals classified in deal types are classified two or more deal characteristics on Biomedtracker Deal Search, and we counted all deal characteristics in these cases. It was suggested that transactions for “alliance” were performed for trial collaborations most frequently, followed by R+D and marketing-licensing (Table 1).

<table><caption>**Table 1.** The average number of interorganizational transactions for seven immune checkpoint blockade agents in each deal characteristic defined in the alliances.</caption><thead><tr><td></td><td>Average</td><td>SD</td></tr></thead><tbody><tr><td>Co-promotion</td><td>3.571</td><td>2.225</td></tr><tr><td>Includes Contract</td><td>0.714</td><td>0.756</td></tr><tr><td>Includes Equity</td><td>2.143</td><td>2.116</td></tr><tr><td>Includes Royalty or Profit Split Information</td><td>9.000</td><td>5.916</td></tr><tr><td>Intra-biotech Deal</td><td>4.714</td><td>6.396</td></tr><tr><td>Marketing-licensing</td><td>0.714</td><td>1.496</td></tr><tr><td>Product or Technology Swap</td><td>0.429</td><td>0.535</td></tr><tr><td>R+D and Marketing-licensing</td><td>14.000</td><td>9.678</td></tr><tr><td>Trial Collaborations</td><td>24.429</td><td>18.636</td></tr><tr><td>Reverse Licensing</td><td>0.286</td><td>0.488</td></tr></tbody></table>

## 3.2. Market Landscape and Interorganizational Collaborations for Combination Therapy Development of Top Two Immune Checkpoint Blockades: Pembrolizumab and Nivolumab

To understand the landscape of immune checkpoint blockade in the market, we examined the history of approved indications and annual product sales of two immune checkpoint blockades in the US, namely pembrolizumab and nivolumab, which triggered the paradigm of immuno-oncology and are the top two successful products in terms of the number of indications and product sales. Both pembrolizumab and nivolumab have achieved rapid indication expansion since they were initially approved in 2014 (Figure 2). Both products target various tumor types, such as solid and hematological tumors, including microsatellite instability-high and mismatch repair-deficient colorectal cancer. We also confirmed that product sales of both pembrolizumab and nivolumab have increased very rapidly, and the total annual sales of pembrolizumab surpassed that of nivolumab in 2018.

Next, we investigated the empirical landscape of combination therapy of pembrolizumab and nivolumab, as trial collaboration is the most common area in which active interorganizational transactions are performed in immune checkpoint blockade (Table 1), and most of them are related to combination therapy development. In this investigation, we selected only interorganizational transactions that directly influence the drug discovery and development of pembrolizumab and nivolumab, hereinafter referred to as effective interorganizational transactions. These included 67 transactions in alliance and financing for pembrolizumab until 2019 and 59 transactions in acquisition, alliance, and financing for nivolumab until 2019.
<figure><img src="image_57.png" /><figcaption>**Figure 2.** Annual approved indications, annual product sales (line graph), and annual number of effective interorganizational transactions (bar graph) of pembrolizumab and nivolumab from 2014 to 2019. NSCLC: non-small cell lung cancer, SCCHN: squamous cell carcinoma of the head and neck, HL: Hodgkin lymphoma, UC: urothelial carcinoma, MSI-H: microsatellite instability-high, GC: gastric cancer, GEC: gastroesophageal cancer, CC: cervical cancer, PMBCL: primary mediastinal large B-cell lymphoma, HCC: hepatocellular carcinoma, MCC: Merkel cell carcinoma, RCC: renal cell carcinoma, SCLC: small cell lung cancer, EC: endometrial carcinoma, BC: bladder cancer, TMB-H: tumor mutational burden-high, cSCC: cutaneous squamous cell carcinoma, dMMR: mismatch repair deficient, CRC: colorectal cancer, Adj Melanoma: adjuvant treatment of melanoma, ESCC: esophageal squamous cell carcinoma. Effective interorganizational transactions means interorganizational transactions that directly influenced the drug discovery and development of pembrolizumab and nivolumab. Combination therapy: interorganizational transactions for combination therapy development. Others: interorganizational transactions for other purpose.</figcaption></figure>

Figure 2 shows the annual number of effective interorganizational transactions for pembrolizumab and nivolumab, sorted into transactions for combination therapy development and others from 2014 to 2019. Throughout the period, the main purpose of transactions was combination therapy development for both pembrolizumab and nivolumab. Transactions for companion diagnostics (CoDx) development have been observed for both pembrolizumab and nivolumab. Ono Pharmaceutical and Dako AS contracted for CoDx development in 2015, BMS contracted with Enterome Bioscience SA for CoDx development in 2016 and invested in GeneCentric Diagnostics for translational biomarker research in 2017 for nivolumab, and Merck contracted with NanoString Technologies for CoDx development in 2016 for pembrolizumab. Interestingly, Merck lost two cases in 2017, where Merck paid $625 million upfront and global sales royalties to BMS owing to patent infringement to obtain non-exclusive rights and paid $19.5 million to PDL Biopharma owing to patent infringement to obtain non-exclusive rights.

Outside of the period from 2014 to 2019, one of the outstanding transactions was the contract between Ono and Medalex for joint R&D in 2005, in which Medalex obtained exclusive rights in North America. Of note, the deal search based on our algorithm in the database did not detect the acquisition of Medalex by BMS in 2009.
We also examined the features of combination partners, including developing companies, which were identified in effective interorganizational transactions for combination therapy development. As transactions in financing involve collaborations not directly related to combination therapy development, we only used transactions in acquisitions and alliances; there were 46 transactions in alliances for pembrolizumab and 41 transactions, including 3 acquisitions and 38 alliances, for nivolumab. Figure 3 shows the number of products distinguished by the type of company that develops combination partners tested with pembrolizumab or nivolumab, displayed in each development phase from preclinical to market. We classified developing companies into pharmaceutical companies or biotechnology company/start-up companies, and classified development phases into preclinical, phase 1, phase 1/2 or phase 2, phase 3, initial regulatory filing, or marketed after the initial regulatory approval. Some transaction information did not indicate the development phase of the combination partners and was classified as not applicable. It was observed that the main partners were biotechnology/start-up companies for both pembrolizumab and nivolumab; 8 pharmaceutical companies and 35 biotechnology/start-up companies for pembrolizumab and 8 pharmaceutical companies and 24 biotechnology/start-up companies for nivolumab. In addition, different trends were observed, as mainly combination partners under phase 1, phase 1/2, or phase 2 statuses developed by biotechnology companies or start-up companies were tested with pembrolizumab, whereas nivolumab was combined with combination partners under broader development phases from preclinical to marketed phases.

<figure><img src="image_58.png" /><figcaption>**Figure 3.** Number of products that were tested with pembrolizumab or nivolumab for combination therapy development. The number of products is displayed by the type of developing company as Pharm or Biotech/Start-ups. Pharm: pharmaceutical company, Biotech: biotechnology company, Start-ups: start-up company, Ph1: Phase 1, Ph1/2 or Ph2: phase 1/2 or phase 2, Ph3: phase 3, Filed: initial regulatory filing, Marketed: marketed after the initial regulatory approval, n/a: not applicable.</figcaption></figure>

4. Discussion

We investigated the recent trends and success factors of interorganizational transactions in cancer therapeutics, especially immune checkpoint blockade. We also explored in depth transactions for the drug discovery and development of immune checkpoint blockade agents, including extensive combination therapy development, especially pem-
pembrolizumab and nivolumab. We collected data for each parameter from an open-source database and performed summary statistics and statistical analysis to study the trends of interorganizational transactions, combination therapies, especially with pembrolizumab and nivolumab, and the features of combination partners. Our findings suggest that immune checkpoint blockade is the most active area in terms of interorganizational collaboration in cancer drug development, where alliances related to combination therapy development are the center of practice. It was also suggested from a detailed investigation of transactions for pembrolizumab and nivolumab that different approaches were taken in indication expansion and phase of combination partners.

There were many more interorganizational transactions for drug discovery and development for immune checkpoint blockade than for other MOAs, and the main purpose of transactions was for trial collaboration, as shown in Figure 1 and Table 1, respectively. When we examined the details of interorganizational transactions for pembrolizumab and nivolumab, the majority of these were performed for combination therapy development. Based on these two observations, it was suggested that interorganizational collaborations are important in drug discovery and development of immune checkpoint blockade and alliances is the most effective means for activating combination therapy development which requires an in-licensing pipeline from another company or co-development with another company. These consequences may bring momentum to R&D activities in immune checkpoint blockade area by enabling partnering companies to achieve continuous regulatory authorization and lead business success in sales. In addition, indication expansions observed in Figure 2 may promote the potential combinations of immune checkpoint blockade with other agents for broader cancer types. Combination therapy is the key to expanding the use of pembrolizumab and nivolumab. This may be because combination therapy is expected to increase the number of responders who are non-responders with monotherapy, as previous studies argued that a combination strategy where various combination targets are tested with PD-(L)1 inhibitors is one of the most promising approaches to address treatment of these non-responders [21,40,45–47]. Although immune checkpoint blockade is a paradigm in cancer treatment, as shown by higher overall response rate, achievability of off treatment survival, and higher safety profile, there is clear challenge that only a fraction of cancer patients can derive clinical benefit from immune checkpoint blockade [21,47–50] thus much more research should be necessary for exploring further potential.

Both pembrolizumab and nivolumab are leading products in immune checkpoint blockade and both have common areas in drug discovery and development, including the same initial approval year for the same indication by the FDA. In addition, there are differences between pembrolizumab and nivolumab in terms of market landscape and interorganizational transactions. We anticipated that these differences could lead to successful drug discovery and development. Regarding the approval history and product sales transition of pembrolizumab and nivolumab, as shown in Figure 2, the indication expansion of nivolumab was initially more rapid than that of pembrolizumab from the first FDA approval in 2014 to 2017, indicating that the expansion of pembrolizumab increased from 2017 to 2019, whereas that for nivolumab has slowed down. This transition seems to link the transition of product sales, as initial product sales of nivolumab were higher than those of pembrolizumab until 2017, and this trend has been reversed since 2018. Both products are innovative and successful against various cancers. However, the difference in development strategies might have affected the result of indication expansion and product sales.

As shown in Figure 2, it was observed that the distribution of the effective transactions appears to present a “Gaussian” shape with the peak in 2017 while the sales trend keeps increasing even in 2019 [10]. Our previous study shows that the observed distribution with the peak in 2017 is unique to pembrolizumab and nivolumab while other biologics and small molecules show different distribution of interorganizational transactions with peaks in different years. We anticipate that the reason for this is because newer immune checkpoint blockades and new drugs using new technologies such as gene therapy have
been developed after the launch of pembrolizumab and nivolumab, and the interorgani-
zational collaboration has shifted in combination with those newer agents. On the other
hand, pembrolizumab and nivolumab still have solid positions with a large amount of
evidence for various indications, which could be associated with an increase in sales as
the previous research argued that the order of entry and strong efficacy are correlated
with product sales [51,52]. In addition, we confirmed that several interorganizational
transactions were performed for the R&D of CoDx and biomarkers. Although immune
checkpoint blockade is a remarkably effective approach, previous researchers have argued
that the overall response rate varies and is highly dependent on the microenvironment
or genetic variations in patients with cancer [47,53]. Therefore, predictive biomarkers are
very important [46,49,50,54] and PD-L1 expression is one of the most common biomarkers.
However, it has been previously reported that a single biomarker is insufficient to predict
clinical benefit or durability of the response to treatment in patients with cancer, and there
are no reliable predictive biomarkers [21,50]. Therefore, our observations reflect the fact
that further research in this area is necessary.

Regarding the observation of differences in the phase of combination partners between pembrolizumab and nivolumab, different strategies for partner selection can be expected. Although we did not investigate the correlation between strategic transactions and success in drug development and commercialization, the result shown in Figure 3 suggests that the selective approach of collaboration partners led to more successful indication expansion and increases in sales, as pembrolizumab was tested with many combination targets that were in phase 1, phase 1/2, or phase 2. It is also interesting to note that Merck's transactions for payment owing to patent infringement seemed to be decisive, which enabled Merck to obtain exclusive rights to develop and sell pembrolizumab globally. In addition to interorganizational transactions, different approaches can be observed in clinical development, as in the case of nivolumab, which was ahead of pembrolizumab in clinical development entry; the duration from initial investigational new drug application to initial BLA submission was 100 months, with data from 886 subjects in six clinical trials, whereas it was 56 months with data from 1577 subjects in five clinical trials for pembrolizumab, which eventually enabled BLA approval earlier than that of nivolumab. These different approaches in drug development are referable when companies in the pharmaceutical industry develop pipeline strategies.

These considerations, based on our findings, can be translated into R&D strategies when new innovative pipelines are developed. In the future study, we will perform similar analysis focused on the MOA of additional combination therapy approved in recent years in order to clarify the findings in this study which were specific for immune checkpoint blockade.

**5. Conclusions**

The results of this study confirmed that active research and development related to immune checkpoint blockade is extraordinary in terms of continuous indication expansion and interorganizational transactions, especially for trial collaboration after the initial success of regulatory approval. In particular, trial collaboration for testing combinations with other cancer drugs was main deal characteristic specifically in the cases of pembrolizumab and nivolumab, which also achieved continuous indication expansion and increases in market sales. In addition, we found a remarkable variety of interorganizational collaborations, from the discovery of potential molecules, such as PD-1, to the post-launch of products.

However, there are limitations in our study. We performed database research us-
ing an open-source database and investigated only anticancer drugs, especially immune
checkpoint blockade. In addition, detailed investigations were performed only for pem-
brolizumab and nivolumab. Further research is expected in the future to investigate
whether the results from our study are applicable to other MOAs or modalities or whether
the same kind of strategic collaboration can be seen in newer MOAs or modalities in the
future. We expect the results of future research to address the remaining unmet medical needs and sustainability of the pharmaceutical industry.

**Author Contributions:** Conceptualization, A.D.; methodology, A.D.; software, A.D.; validation, A.D.; formal analysis, A.D.; investigation, A.D.; resources, A.D.; data curation, A.D.; writing—original draft preparation, A.D.; writing—review and editing, T.M., Y.L., S.S. and K.K.; supervision, K.K. All authors have read and agreed to the published version of the manuscript.

**Funding:** This work was supported by Grants-in-Aid for Challenging Exploratory Research (grant numbers 20K20769, 26301022, and 23730336). The funding sources did not participate in the study design, data collection, analysis, interpretation, report writing, or the decision to submit this article for publication. In addition, this work was supported by the FFJ/Air Liquide Fellowship. The author gratefully acknowledges the generous support and assistance of the Fondation France-Japon (FFJ) de Ecole des Hautes Etudes en Sciences Sociales (EHESS) and Air Liquide. The authors would like to express their sincere thanks to all the respondents who participated in the survey of this study.

**Institutional Review Board Statement:** Not applicable.

**Informed Consent Statement:** Not applicable.

**Acknowledgments:** We are grateful to the editors and referees for their valuable comments, which helped to improve this paper.

**Conflicts of Interest:** Arisa Djurian is an employee of Takeda Pharmaceutical Company Limited, and Tomohiro Makino is an employee of Daiichi Sankyo Company Limited. Neither company has any direct relationship with the content of this article.

## References

1. Kola, I.; Landis, J. Can the pharmaceutical industry reduce attrition rates? *Nat. Rev. Drug Discov.* **2004**, *3*, 711–715. [CrossRef] [PubMed]

2. Adams, C.P.; Van Brantner, V. Market watch: Estimating the cost of new drug development: Is it really $802 million? *Health Aff.* **2006**, *25*, 420–428. [CrossRef] [PubMed]

3. DiMasi, J.A.; Hansen, R.W.; Grabowski, H.G. The price of innovation: New estimates of drug development costs. *J. Health Econ.* **2003**, *22*, 151–185. [CrossRef]

4. Ciani, O.; Jommi, C. The role of health technology assessment bodies in shaping drug development. *Drug Des. Devel. Ther.* **2014**, *8*, 2273–2281. [CrossRef]

5. Rugman, A.M.; Verbeke, A. A perspective on regional and global strategies of multinational enterprises. *J. Int. Bus. Stud.* **2004**, *35*, 3–18. [CrossRef]

6. Teramae, F.; Makino, T.; Lim, Y.; Sengoku, S.; Kodama, K. International strategy for sustainable growth in multinational pharmaceutical companies. *Sustainability* **2020**, *12*, 867. [CrossRef]

7. Sengoku, S.; Yoda, T.; Seki, A. Assessment of Pharmaceutical Research and Development Productivity With a Novel Net Present Value-based Project Database. *Ther. Innov. Regul. Sci.* **2011**, *45*, 175–185. [CrossRef]

8. Bianchi, M.; Cavaliere, A.; Chiaroni, D.; Frattini, F.; Chiesa, V. Organisational modes for Open Innovation in the bio-pharmaceutical industry: An exploratory analysis. *Technovation* **2011**, *31*, 22–33. [CrossRef]

9. Makino, T.; Sengoku, S.; Ishida, S.; Kodama, K. Trends in interorganizational transactions in personalized medicine development. *Drug Discov. Today* **2019**, *24*, 364–370. [CrossRef]

10. Djurian, A.; Makino, T.; Lim, Y.; Sengoku, S.; Kodama, K. Trends of business-to-business transactions to develop innovative cancer drugs. *Sustainability* **2020**, *12*, 5535. [CrossRef]

11. Wang, L.; Plump, A.; Ringel, M. Racing to define pharmaceutical R&D external innovation models. *Drug Discov. Today* **2015**, *20*, 361–370. [CrossRef] [PubMed]

12. Schuhmacher, A.; Germann, P.-G.; Trill, H.; Gassmann, O. Models for open innovation in the pharmaceutical industry. *Drug Discov. Today* **2013**, *18*, 1133–1137. [CrossRef]

13. Makino, T.; Lim, Y.; Kodama, K. Strategic R&D transactions in personalized drug development. *Drug Discov. Today* **2018**, *23*, 1334–1339. [CrossRef] [PubMed]

14. DeVita, V.T.; Chu, E. A History of Cancer Chemotherapy. *Cancer Res.* **2008**, *68*, 8643–8653. [CrossRef]

15. Chabner, B.A.; Roberts, T.G. Chemotherapy and the war on cancer. *Nat. Rev. Cancer* **2005**, *5*, 65–72. [CrossRef] [PubMed]

16. Sawyers, C. Targeted cancer therapy. *Nature* **2004**, *432*, 294–297. [CrossRef] [PubMed]

17. Lord, C.J.; Ashworth, A. Biology-driven cancer drug development: Back to the future. *BMC Biol.* **2010**, *8*, 1–12. [CrossRef]

18. Al-Lazikani, B.; Banerji, U.; Workman, P. Combinatorial drug therapy for cancer in the post-genomic era. *Nat. Biotechnol.* **2012**, *30*, 679–692. [CrossRef]
19. Finn, O.J. Immuno-oncology: Understanding the function and dysfunction of the immune system in cancer. *Ann. Oncol.* **2012**, *23*, 8–11. [CrossRef]

20. Hoos, A. Development of immuno-oncology drugs-from CTLA4 to PD1 to the next generations. *Nat. Rev. Drug Discov.* **2016**, *15*, 235–247. [CrossRef] [PubMed]

21. Sharma, P.; Allison, J.P. The future of immune checkpoint therapy. *Science* **2015**, *348*, 56–61. [CrossRef] [PubMed]

22. Postow, M.A.; Callahan, M.K.; Wolchok, J.D. Immune checkpoint blockade in cancer therapy. *J. Clin. Oncol.* **2015**, *33*, 1974–1982. [CrossRef]

23. Topalian, S.L.; Drake, C.G.; Pardoll, D.M. Immune checkpoint blockade: A common denominator approach to cancer therapy. *Cancer Cell* **2015**, *27*, 450–461. [CrossRef] [PubMed]

24. Drugs@FDA: FDA-Approved Drugs. Available online: https://www.accessdata.fda.gov/scripts/cder/daf/ (accessed on 5 December 2020).

25. Ishida, Y.; Agata, Y.; Shibahara, K.; Honjo, T. Induced expression of PD-1, a novel member of the immunoglobulin gene superfamily, upon programmed cell death. *EMBO J.* **1992**, *11*, 3887–3895. [CrossRef] [PubMed]

26. Chen, L.; Han, X. Anti-PD-1/PD-L1 therapy of human cancer: Past, present, and future Find the latest version: Anti–PD-1/PD-L1 therapy of human cancer: Past, present, and future. *J. Clin. Investig.* **2015**, *125*, 3384–3391. [CrossRef]

27. Dong, H.; Zhu, G.; Tamada, K.; Chen, L. B7-H1, a third member of the B7 family, co-stimulates T-cell proliferation and interleukin-10 secretion. *Nat. Med.* **1999**, *5*, 1365–1369. [CrossRef]

28. Freeman, G.J.; Long, A.J.; Iwai, Y.; Bourque, K.; Chernova, T.; Nishimura, H.; Fitz, L.J.; Malenkovich, N.; Okazaki, T.; Byrne, M.C.; et al. Engagement of the PD-1 immunoinhibitory receptor by a novel B7 family member leads to negative regulation of lymphocyte activation. *J. Exp. Med.* **2000**, *192*, 1027–1034. [CrossRef]

29. Tseng, S.Y.; Otsuji, M.; Gorski, K.; Huang, X.; Slansky, J.E.; Pai, S.I.; Shalabi, A.; Shin, T.; Pardoll, D.M.; Tsuchiya, H. B7-DC, a new dendritic cell molecule with potent costimulatory properties for T cells. *J. Exp. Med.* **2001**, *193*, 839–845. [CrossRef]

30. Latchman, Y.; Wood, C.R.; Chernova, T.; Chaudhary, D.; Borde, M.; Chernova, I.; Iwai, Y.; Long, A.J.; Brown, J.A.; Nunes, R.; et al. PD-L2 is a second ligand for PD-1 and inhibits T cell activation. *Nat. Immunol.* **2001**, *2*, 261–268. [CrossRef]

31. Butte, M.J.; Keir, M.E.; Phamduy, T.B.; Sharpe, A.H.; Freeman, G.J. Programmed Death-1 Ligand 1 Interacts Specifically with the B7-1 Costimulatory Molecule to Inhibit T Cell Responses. *Immunity* **2007**, *27*, 111–122. [CrossRef]

32. Park, J.J.; Omiya, R.; Matsumura, Y.; Sakoda, Y.; Kuramasu, A.; Augustine, M.M.; Yao, S.; Tsushima, F.; Narazaki, H.; Anand, S.; et al. B7-H1/CD80 interaction is required for the induction and maintenance of peripheral T-cell tolerance. *Blood* **2010**, *116*, 1291–1298. [CrossRef] [PubMed]

33. Xiao, Y.; Yu, S.; Zhu, B.; Bedoret, D.; Bu, X.; Duke-Cohan, L.M.F.; Umetsu, D.T.; Sharpe, A.H.; DeKruyff, R.H.; Freeman, G.J. RGMb is a novel binding partner for PD-l2 and its engagement with PD-l2 promotes respiratory tolerance. *J. Exp. Med.* **2014**, *211*, 943–959. [CrossRef] [PubMed]

34. Nishimura, H.; Nose, M.; Hiai, H.; Minato, N.; Honjo, T. Development of lupus-like autoimmune diseases by disruption of the PD-1 gene encoding an ITIM motif-carrying immunoreceptor. *Immunity* **1999**, *11*, 141–151. [CrossRef]

35. Dong, H.; Zhu, G.; Tamada, K.; Flies, D.B.; Van Deursen, J.M.A.; Chen, L. B7-H1 determines accumulation and deletion of intrahepatic CD8+ T lymphocytes. *Immunity* **2004**, *20*, 327–336. [CrossRef]

36. Chen, L. Co-inhibitory molecules of the B7-CD28 family in the control of T-cell immunity. *Nat. Rev. Immunol.* **2004**, *4*, 336–347. [CrossRef]

37. Zou, W.; Chen, L. Inhibitory B7-family molecules in the tumour microenvironment. *Nat. Rev. Immunol.* **2008**, *8*, 467–477. [CrossRef]

38. Lifearc, A. LifeArc: Case Study-Keytruda. Available online: https://www.lifearc.org/case-studies/keytruda-new-generation-cancer-treatment/ (accessed on 13 December 2020).

39. BMS Newsroom: Bristol-Myers Squibb to Acquire Medarex. Available online: http://news.bms.com/press-release/partnering-news/bristol-myers-squibb-acquire-medarex (accessed on 13 December 2020).

40. Yu, J.X.; Hodge, J.P.; Oliva, C.; Neftelinov, S.T. Trends in clinical development for PD-1 / PD-L1 inhibitors. *Nat. Rev. Drug Discov.* **2019**, *18*, 13–14.

41. Tang, J.; Yu, J.X.; Hubbard-Lucey, V.M.; Neftelinov, S.T.; Hodge, J.P.; Lin, Y. The clinical trial landscape for PD1/PD11 immune checkpoint inhibitors. *Nat. Rev. Drug Discov.* **2018**, *17*, 854–855. [CrossRef]

42. Definition of Mechanism of Action. Available online: https://www.cancer.gov/publications/dictionaries/cancer-terms/def/mechanism-of-action (accessed on 1 November 2020).

43. CenterWatch. Available online: https://www.centerwatch.com/directories/1067-fda-approved-drugs (accessed on 20 July 2019).

44. Biomedtracker Deal Search. Available online: https://www.biomedtracker.com/DealSearch.cfm (accessed on 26 November 2020).

45. Alsaab, H.O.; Sau, S.; Alzhrani, R.; Tatiparti, K.; Bhise, K.; Kashaw, S.K.; Iyer, A.K. PD-1 and PD-L1 checkpoint signaling inhibition for cancer immunotherapy: Mechanism, combinations, and clinical outcome. *Front. Pharmacol.* **2017**, *8*, 1–15. [CrossRef]

46. Pardoll, D.M. The blockade of immune checkpoints in cancer immunotherapy. *Nat. Rev. Cancer* **2012**, *12*, 252–264. [CrossRef]

47. Zappasodi, R.; Merghoub, T.; Wolchok, J.D. Emerging Concepts for Immune Checkpoint Blockade-Based Combination Therapies. *Cancer Cell* **2018**, *33*, 581–598. [CrossRef] [PubMed]
48. Gong, J.; Chehrazi-Raffle, A.; Reddi, S.; Salgia, R. Development of PD-1 and PD-L1 inhibitors as a form of cancer immunotherapy: A comprehensive review of registration trials and future considerations. *J. Immunother. Cancer* **2018**, *6*, 1–18. [CrossRef] [PubMed]

49. Taube, J.M.; Klein, A.; Brahmer, J.R.; Xu, H.; Pan, X.; Kim, J.H.; Chen, L.; Pardoll, D.M.; Topalian, S.L.; Anders, R.A. Association of PD-1, PD-1 ligands, and other features of the tumor immune microenvironment with response to anti-PD-1 therapy. *Clin. Cancer Res.* **2014**, *20*, 5064–5074. [CrossRef] [PubMed]

50. Meng, X.; Huang, Z.; Teng, F.; Xing, L.; Yu, J. Predictive biomarkers in PD-1/PD-L1 checkpoint blockade immunotherapy. *Cancer Treat. Rev.* **2015**, *41*, 868–876. [CrossRef]

51. Schulze, U.; Ringel, M. What matters most in commercial success: First-in-class or best-in-class. *Nat. Rev. Drug Discov.* **2013**, *12*, 419–420. [CrossRef]

52. Teramae, F.; Makino, T.; Sengoku, S.; Lim, Y.; Natori, T.; Kodama, K. Research on pharmaceutical product life cycle patterns for sustainable growth. *Sustainability* **2020**, *12*, 8938. [CrossRef]

53. Lawrence, M.S.; Stojanov, P.; Polak, P.; Kryukov, G.V.; Cibulskis, K.; Sivachenko, A.; Carter, S.L.; Stewart, C.; Mermel, C.H.; Roberts, S.A.; et al. Mutational heterogeneity in cancer and the search for new cancer-associated genes. *Nature* **2013**, *499*, 214–218. [CrossRef]

54. Patel, S.P.; Kurzrock, R. PD-L1 expression as a predictive biomarker in cancer immunotherapy. *Mol. Cancer Ther.* **2015**, *14*, 847–856. [CrossRef]
<figure><img src="image_59.png" /></figure>

Article

# A Retrospective Analysis of the De Ritis Ratio in Muscle Invasive Bladder Cancer, with Focus on Tumor Response and Long-Term Survival in Patients Receiving Neoadjuvant Chemotherapy and in Chemo Naïve Cystectomy Patients—A Study of a Clinical Multicentre Database

Victoria Eriksson <fnref n="1" />,†, Oscar Holmkvist <fnref n="1" />,†, Ylva Huge <fnref n="2" />, Markus Johansson <fnref n="3" />, Farhood Alamdari <fnref n="4" />, Johan Svensson <fnref n="5" />, Firas Aljabery <fnref n="2" />,‡ and Amir Sherif <fnref n="1" />,*,‡ ID

**Citation:** Eriksson, V.; Holmkvist, O.; Huge, Y.; Johansson, M.; Alamdari, F.; Svensson, J.; Aljabery, F.; Sherif, A. A. Retrospective Analysis of the De Ritis Ratio in Muscle Invasive Bladder Cancer, with Focus on Tumor Response and Long-Term Survival in Patients Receiving Neoadjuvant Chemotherapy and in Chemo Naïve Cystectomy Patients—A Study of a Clinical Multicentre Database. *J. Pers. Med.* **2022**, *12*, 1769. https://doi.org/10.3390/jpm12111769

Academic Editor: Anne-Marie Caminade

Received: 30 September 2022

Accepted: 25 October 2022

Published: 27 October 2022

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

<figure><img src="image_60.png" /></figure>

**Copyright:** © 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

**Abstract:** Background: A high pre-treatment De Ritis ratio, the aspartate transaminase/alanine aminotransferase ratio, has been suggested to be of prognostic value for mortality in muscle-invasive bladder cancer (MIBC). Our purpose was to evaluate if a high ratio was associated with mortality and downstaging. Methods: A total of 347 Swedish patients with clinically staged T2-T4aN0M0, with administered neoadjuvant chemotherapy (NAC) or eligible for NAC and undergoing radical cystectomy (RC) 2009–2021, were retrospectively evaluated with a low ratio < 1.3 vs. high ratio > 1.3, by Log Rank test, Cox regression and Mann–Whitney U-test (MWU), SPSS 27. Results: Patients with a high ratio had a decrease of up to 3 years in disease-free survival (DFS), cancer-specific survival (CSS) and overall survival (OS) (p = 0.009, p = 0.004 and p = 0.009) and 5 years in CSS and OS (p = 0.019 and p = 0.046). A high ratio was associated with increased risk of mortality, highest in DFS (HR, 1.909; 95% CI, 1.265–2.880; p = 0.002). No significant relationship between downstaging and a high ratio existed (p = 0.564 MWU). Conclusion: A high pre-treatment De Ritis ratio is on a population level, associated with increased mortality post-RC in endpoints DFS, CSS and OS. Associations decrease over time and require further investigations to determine how strong the associations are as meaningful prognostic markers for long-term mortality in MIBC. The ratio is not suitable for downstaging-prediction.

**Keywords:** urinary bladder neoplasms; neoadjuvant therapy; cystectomy; clinical decision rules; prognosis

## 1. Introduction

In 2020, 573,000 people were diagnosed with urinary bladder cancer (UBC), making it the 10th most common cancer worldwide and 6th most common in men [1]. The incidence varies between countries. Sweden has approximately 3200 new cases every year, with 700 deaths yearly. Men are three times more likely to be affected than women. At diagnosis, 70–75% are non-muscle-invasive bladder cancer (NMIBC; clinical stages Ta, T1 and Tis) and 20–25% are muscle-invasive bladder cancer (MIBC; clinical stages T2–T4) [2]. MIBC-patients have had a worse prognosis, with a five-year survival of less than 50% [3]. The addition of

<footnote n="1">¹ Department of Surgical and Perioperative Sciences, Urology and Andrology, Umeå University, 901 87 Umeå, Sweden</footnote>

<footnote n="2">² Department of Clinical and Experimental Medicine, Division of Urology, Linköping University, 581 83 Linköping, Sweden</footnote>

<footnote n="3">³ Department of Surgery, Division of Urology, Sundsvall-Härnösand County Hospital, 856 43 Sundsvall, Sweden</footnote>

<footnote n="4">⁴ Department of Urology, Västmanland Hospital, 721 89 Västerås, Sweden</footnote>

<footnote n="5">⁵ Department of Statistics, Umeå School of Business, Economics and Statistics (USBE), Umeå University, 901 87 Umeå, Sweden</footnote>

<footnote>* Correspondence: amir.sherif@umu.se</footnote>

<footnote>† These authors contributed equally to this work.</footnote>

<footnote>‡ These authors contributed equally to this work.</footnote>
Cisplatinum-based neoadjuvant chemotherapy (NAC) prior to radical cystectomy (RC) has improved survival. However, if patients are non-responders to NAC, this may result in delayed RC and adverse events with decreased downstaging. Downstaging is considered a surrogate marker for overall survival [4,5].

The classical staging systems of clinical tumor-node-metastasis (cTNM) from the American Joint Committee on Cancer (AJCC) and the World Health Organization (WHO) classifications of high and low tumor grade, is routine in classifying and evaluating prognosis in MIBC-patients. Still, the classifications have less significance in predicting 5-year progression-free survival (PFS) [6]. Finding pre-treatment factors with the potential to predict aggressive disease and survival post-RC, would be useful [7]. A more precise evaluation of prognosis could also affect treatment selection prior to RC [8]. No clinical markers concerning prognosis in MIBC are available from a simple blood test at present [9].

However, the De Ritis ratio has recently been described as having prognostic value in multiple individual cancer forms, such as renal cell carcinoma (RCC), myeloma, colonic, pancreatic and urothelial cancer [8,10–15]. The ratio consists of the well-known circulating enzymes aspartate aminotransferase (AST) and alanine aminotransferase (ALT) [3,10], which are retrievable by blood test. The AST/ALT ratio was initially presented by De Ritis in 1957 as a tool to assess liver disease [16]. The underlying mechanism between the AST/ALT enzymes and cancer is not fully clear, but aminotransaminases are expressed by both cancerous and noncancerous cells and found in the liver, heart, skeletal muscle, and kidney [13]. UBC is considered a glucose-dependent malignancy, and it is known that AST plays an essential role in glycolysis, which leads to a potential condition in which increased glucose metabolism induced by cancer, known as the Warburg effect, exists with an increase in AST/ALT ratio. Thus, a potential correlation between a high ratio and cancer disease has been hypothesized [7,8,13,17,18].

Retrospective studies on MIBC have reported findings in which a high ratio was prognostic for inferior PFS, lesser cancer-specific survival (CSS) and lower overall survival (OS). Similar methods to establish a cut-off value were used, including ROC curve with a high De Ritis ratio if >1.1–1.5. The studies may have made deeper reflections regarding potential weaknesses in the statistical methods that established the cut-off level [3, 7, 8, 10, 12, 19]. One study was based on a population where 30% had disseminated disease and there was no mention of NAC [19]. Another study had a mixed population of MIBC and NMIBC (non-MIBC) and only 18% of the patients received NAC. One of the most important factors for assessing long-term survival, lymph node status pre-treatment, was reported in a vague way, where it is not possible to determine whether it was overt cN+ or whether postoperative pN+ was intended [3]. One study presented survival predictions solely in OS [7], which may include, but is not limited to, death by other types of cancers than MIBC, noncancer-related death and death from treatment. Thus, many events in patients that were specifically unrelated to MIBC were included [20]. Two meta-analyses from 2020 concluded that further validation of the De Ritis is required and that previous retrospective studies from up to 2020, including UBC, were often single-centre and lacking a diverse study population such as patients of African or American origin [11].

We intended to evaluate a larger cohort solely on NAC-treated and NAC-eligible patients after the exclusion of pre-treatment disseminated disease, bacillus Calmette-Guérin (BCG)-resistant UBC and liver disease, as opposed to some previous studies that included patients with known lymph-node involvement. Thus, our aspiration was a cohort with potentially less bias due to high comorbidity and disseminated disease. The aim was to investigate if an association between a high De Ritis ratio > 1.3 and increased mortality existed, by evaluating endpoints in different time periods regarding survival outcomes post-RC and evaluating the proxy marker downstaging of primary tumor. To the best of our knowledge, downstaging has not yet been evaluated in this research area.
## 2. Materials and Methods

Study design and patient selection; Cystectomized patients were derived from an existing clinical, multi-centre database with all cystectomized patients between the years 2009 and 2021 (n = 973). The patients were treated at either four Swedish cystectomy centres; Norrlands Universitetssjukhus, Umeå; Länssjukhuset Sundsvall, Sundsvall; Västmanlands sjukhus, Västerås and Universitetssjukhuset Linköping, Linköping. Inclusion criteria were MIBC; cT2-T4aN0M0, urothelial histopathology and NAC-administered or NAC-eligible NAC-naïve patients. All patients were discussed after diagnosis at multidisciplinary conference and assigned NAC pre-RC if clinically acceptable. Exclusion criteria were liver disease, disseminated cancer, non-urothelial histopathology, and contraindications for NAC; i.e., age > 75, reduced kidney function, Charlson Age Comorbidity Index (CACI) > 6 and hearing impairment. Patients with missing data regarding AST/ALT, due to there being no available laboratory results, were excluded. Final analysis included 347 patients, all who had undergone RC (Figure 1).

<figure><img src="image_61.png" /><figcaption>**Figure 1.** Flowchart displaying selection process of patient cohort, including inclusion and exclusion criteria. Muscle invasive bladder cancer (MIBC); clinical tumor node metastasis (cTNM); neoadjuvant chemotherapy (NAC); aspartate aminotransferase/alanine aminotransferase (AST/ALT); radical cystectomy (RC).</figcaption></figure>

Study procedure: clinicopathological variables from patient medical records were individually collected and compiled into an existing clinical database. Laboratory data were updated from 1 January 2009 to 31 January 2022. Variables included age, Body Mass Index (BMI), American Society of Anaesthesiologists (ASA) score, CACI, cTNM, pTNM, NAC or NAC-legibility, active smoker yes/no, outcome data on cancer recurrence and disease-free survival (DFS), CSS and OS. DFS was chosen instead of PFS as the presented endpoint in this study, with the included events initially described by Punt et al. [21] and later by Birgisson et al. [20]. AST/ALT was documented within 30 days pre-NAC or pre-RC, depending on the treatment the patients were assigned. AST and ALT were routinely tested in lithium heparin plasma with upper reference level 0.75 and 1.1 µ/L, respectively. De Ritis cut-off value in analysis was set to 1.3 based on a frequent used cut-off in previous studies [3,7,19]. Patients were categorized into two groups based on high ratio ≥ 1.3 or low
ratio < 1.3. Downstaging of the primary tumor was divided into four different outcomes and ranked with values 1–4 on an ordinal scale; progressive disease (PD; pN+, pT4b), stable disease (SD; pT2–4aN0M0), partial response (PR; pTa, pTis, pT1N0M0) and complete response (CR; pT0N0M0). In analysis, variables on two levels were treated as nominal, such as age > 70 or age < 70, active smoker; yes or no, NAC; yes or no, ASA and pT-stage > 0; pT0 or pTis-T4b, considering the importance of pT0 as a survival benefit, according to Rosenblatt et al. [4]. The variables BMI and CACI were treated as interval variables.

Statistical analysis: Analysis on downstaging was performed separately on NAC-receivers, and the entire cohort of NAC and NAC-eligible NAC-naïve patients, to compare if the association with downstaging and a high ratio differed depending on the different NAC levels. The association between the pre-treatment De Ritis ratio and downstaging outcomes was evaluated by the Mann–Whitney U tests. Survival was predicted up to 3, 5 and 13 years with the study endpoints DFS, CSS and OS. The influence of the pre-treatment De Ritis ratio on the study endpoints was visualized and compared with the Kaplan–Meier estimator and the Log Rank test. Crude and adjusted Cox proportional regression analyses were carried out to determine the influence of patient gender, age, histological TN, pathological T stage, tumour grade and pre-treatment De Ritis ratios on DFS, CSS and OS. HRs estimated from Cox models are shown as the HR with the corresponding 95% CI. The Cox proportional hazard assumption were visually evaluated in STATA. All the used tests are two-sided with a significance level of 5%. The data were statistically evaluated in SPSS Statistics 27.0 for Mac (IBM Corporation, Armonk, NY, USA), and STATA version 15 (Stata Corp, Houston, TX, USA).

### 3. Results

The final cohort consisted of 347 patients (Figure 1). A total of 258 patients with NAC had pre-treatment data on AST/ALT and 89 NAC-eligible NAC-naïve patients had pre-treatment data on AST/ALT. A total of 76% of the total cohort were men, but a larger proportion of the women in the cohort existed within the high ratio group, compared to the low ratio group (Table 1, *p* = 0.044). A total of 35% of the patients in the cohort were aged over 70. Regarding comorbidity, median level of CACI was 5 and approximately 23% of 347 patients had ASA grade III. Additionally, 75% received some sort of NAC, of which 78% within the group of high ratios were NAC-receivers, compared to 73% NAC-receivers within the low-ratio group. The majority of the cohort-patients had initial stage cT2 (61%) (Table 1).

<table><caption>**Table 1.** Patient cohort characteristics within groups of low and high De Ritis ratio and total cohort.</caption><thead><tr><th>Variables</th><th>Low De Ritis Ratio < 1.3</th><th>High De Ritis Ratio ≥ 1.3</th><th>Total Cohort</th><th>p-Value</th></tr></thead><tbody><tr><td>Sex, n (%)</td><td>n = 218</td><td>n = 129</td><td>n = 347</td><td></td></tr><tr><td></td><td></td><td>Preoperative</td><td></td><td></td></tr><tr><td>Male</td><td>173 (79.4)</td><td>90 (69.8)</td><td>263 (75.8)</td><td>0.044 *</td></tr><tr><td>Female</td><td>45 (20.6)</td><td>39 (30.2)</td><td>84 (24.2)</td><td></td></tr><tr><td>Age, n (%)</td><td></td><td></td><td></td><td>0.055 *</td></tr><tr><td>&lt;70 years</td><td>149 (68.3)</td><td>75 (58.1)</td><td>224 (64.6)</td><td></td></tr><tr><td>&gt;70 years</td><td>69 (31.7)</td><td>54 (41.9)</td><td>123 (35.4)</td><td></td></tr><tr><td>BMI (mean, SD)</td><td>26.46 (3.9)</td><td>25.3 (3.9)</td><td>26.03 (3.9)</td><td>0.008 †</td></tr><tr><td>ASA, n (%)</td><td></td><td></td><td></td><td>0.592 *</td></tr><tr><td>I</td><td>37 (17)</td><td>21 (16.3)</td><td>58 (16.7)</td><td></td></tr><tr><td>II</td><td>134 (61.5)</td><td>74 (57.4)</td><td>208 (59.9)</td><td></td></tr><tr><td>III</td><td>47 (21.6)</td><td>34 (26.4)</td><td>81 (23.3)</td><td></td></tr><tr><td>CACI (median, IQR)</td><td>5 (4, 5)</td><td>5 (4, 5)</td><td>5 (4, 5)</td><td>0.275 <sup>f</sup></td></tr><tr><td>Smoker, n (%)</td><td></td><td></td><td></td><td>0.966 *</td></tr><tr><td>Yes</td><td>57 (26.1)</td><td>34 (26.4)</td><td>91 (26.2)</td><td></td></tr><tr><td>No</td><td>161 (73.9)</td><td>95 (73.6)</td><td>256 (73.8)</td><td></td></tr></tbody></table>
<table><caption>**Table 1.** *Cont.*</caption><thead><tr><th>Variables</th><th>Low De Ritis Ratio &lt; 1.3</th><th>High De Ritis Ratio ≥ 1.3</th><th>Total Cohort</th><th>p-Value</th></tr></thead><tbody><tr><td>cT-stage, n (%)</td><td></td><td></td><td></td><td>0.317 *</td></tr><tr><td>cT2</td><td>136 (62.4)</td><td>77 (59.7)</td><td>213 (61.4)</td><td></td></tr><tr><td>cT3</td><td>73 (33.5)</td><td>41 (31.8)</td><td>114 (32.8)</td><td></td></tr><tr><td>cT4a</td><td>9 (4.1)</td><td>11 (8.5)</td><td>20 (5.8)</td><td></td></tr><tr><td>NAC, n (%)</td><td></td><td></td><td></td><td>0.299 *</td></tr><tr><td>Yes</td><td>158 (72.5)</td><td>100 (77.5)</td><td>258 (74.4)</td><td></td></tr><tr><td>No</td><td>60 (27.5)</td><td>29 (22.5)</td><td>89 (25.6)</td><td></td></tr><tr><td>NAC; MVAC, n (%)</td><td></td><td></td><td></td><td>0.521 *</td></tr><tr><td>Yes</td><td>131 (60.1)</td><td>82 (63.6)</td><td>213 (61.4)</td><td></td></tr><tr><td>No</td><td>87 (39.9)</td><td>47 (36.4)</td><td>134 (38.6)</td><td></td></tr><tr><td>pT-stage n (%)</td><td></td><td>Postoperative</td><td></td><td>0.612 *</td></tr><tr><td>pT0</td><td>62 (28.4)</td><td>40 (31.0)</td><td>102 (29.4)</td><td></td></tr><tr><td>pTCis-T4b</td><td>156 (71.6)</td><td>89 (69.0)</td><td>245 (70.6)</td><td></td></tr></tbody></table>

Significantly increased mortality was found in patients with high De Ritis ratio; Log Rank-test, in endpoints DFS (p = 0.009), CSS (p = 0.004), and OS (p = 0.009) up to three years (36 months) post-RC (Figure 2a-c). Significantly increased mortality existed in patients with high De Ritis ratio in endpoints CSS (p = 0.019) and OS (p = 0.046) but not in DFS (p = 0.057) up to five years (60 months) post-RC (Figure 3a-c). No significant increased mortality existed in patients with high ratio, up to 13 years (160 months) post-RC in any endpoint; DFS (p = 0.090), CSS (p = 0.067), or OS (p = 0.261, Figure 4a-c).

Kaplan-Meier curves predicting survival endpoints up to 36 months.

<figure><img src="image_62.png" /><figcaption>**Figure 2.** *Cont.*</figcaption></figure>

<footnote>* Chi-square test; † t-test; ‡ Mann-Whitney U-test; BMI; Body Mass Index, ASA; American Society of Anaesthesiologists, CACI; Charlson Age Comorbidity Index, cT2-4a-clinical-tumor-stage; NAC; neoadjuvant chemotherapy, pT0-T4b; pathological-tumor-stage.</footnote>
<figure><img src="image_63.png" /><figcaption>**Figure 2.** Kaplan–Meier curves predicting 36 (3 years) and 60 months (5 years) survival endpoints (a) Disease-free survival; (b) Cancer-specific survival; (c) Overall survival.</figcaption></figure>

Kaplan–Meier curves predicting survival endpoints up to 60 months.

<figure><img src="image_64.png" /><figcaption>**Figure 3.** Kaplan–Meier curves predicting 60 months (5 years) survival endpoints (a) Disease-free survival; (b) Cancer-specific survival; (c) Overall survival.</figcaption></figure>

Kaplan–Meier curves predicting survival endpoints up to 160 months.
<figure><img src="image_65.png" /><figcaption>**Figure 4.** Kaplan–Meier curves predicting 5-13 years (160 months) post-RC survival endpoints (a) Disease-free survival; (b) Cancer-specific survival; (c) Overall survival.</figcaption></figure>

The Cox proportional hazard assumption was visually evaluated and the hazard ratio (HR) between low and high De Ritis ratio seems constant up to three years, and then decreases somewhat. After five years, the proportional hazard assumption can no longer be considered valid. Hence, HR is only evaluated up to three and five years (Table 2). Regarding NAC, in neither crude nor adjusted Cox analysis can a significant interaction effect be visualized between a high De Ritis ratio and NAC, indicating that a high De Ritis ratio equally affects mortality in NAC-receivers and those who are NAC-eligible in our material. We used this as an argument to evaluate the effect of the De ritis ratio in the entire cohort.

A high De Ritis ratio > 1.3 was associated with increased risk of mortality in endpoints DFS, CSS and OS up to three years post-RC, up to five years in CSS and OS, both crude and adjusted for pT-stage > 0, age > 70, active smoking, ASA, CACI, BMI, and NAC. In addition, pT-stage > 0 was associated with increased risk of mortality in all endpoints up to three- and five-years post-RC when adjusted for ratio > 1.3, age > 70, active smoking, ASA, CACI, BMI, and NAC. No individual association was seen between mortality and the other variables age > 70, active smoking, ASA, BMI, and NAC, in any timeframe or endpoint (Table 2).
<table><caption>**Table 2.** Cox analysis crude and adjusted on entire cohort; NAC and NAC-eligible NAC-naïve, n = 347, up to 3- and 5-years post-RC.</caption><thead><tr><th rowspan="2">Type of Survival</th><th rowspan="2">Variables</th><th colspan="3">Crude Estimates <sup>1</sup></th><th colspan="3">Adjusted Estimates <sup>2</sup></th></tr><tr><th>HR</th><th>CI</th><th>p</th><th>HR</th><th>CI</th><th>p</th></tr></thead><tbody><tr><td colspan="8">Survival up to 3 years after RC</td></tr><tr><td rowspan="2">DFS</td><td>AST/ALT > 1.3</td><td>1.573</td><td>1.117-2.215</td><td>0.010 *</td><td>1.673</td><td>1.175-2.380</td><td>0.004 **</td></tr><tr><td>pT-stage > 0</td><td></td><td></td><td></td><td>2.654</td><td>1.628-4.325</td><td>0.000 **</td></tr><tr><td rowspan="2">CSS</td><td>AST/ALT > 1.3</td><td>1.772</td><td>1.191-2.636</td><td>0.005 *</td><td>1.909</td><td>1.265-2.880</td><td>0.002 **</td></tr><tr><td>pT-stage > 0</td><td></td><td></td><td></td><td>3.913</td><td>2.059-7.435</td><td>0.000 **</td></tr><tr><td rowspan="2">OS</td><td>AST/ALT > 1.3</td><td>1.625</td><td>1.125-2.348</td><td>0.010 *</td><td>1.724</td><td>1.183-2.513</td><td>0.006 **</td></tr><tr><td>pT-stage > 0</td><td></td><td></td><td></td><td>2.493</td><td>1.485-4.186</td><td>0.001 **</td></tr><tr><td colspan="8">Survival up to 5 years after RC</td></tr><tr><td rowspan="2">DFS</td><td>AST/ALT > 1.3</td><td>1.372</td><td>0.989-1.902</td><td>0.058</td><td>1.485</td><td>1.060-2.080</td><td>0.021 **</td></tr><tr><td>pT-stage > 0</td><td></td><td></td><td></td><td>2.783</td><td>1.747-4.434</td><td>0.000 **</td></tr><tr><td rowspan="2">CSS</td><td>AST/ALT > 1.3</td><td>1.575</td><td>1.075-2.308</td><td>0.020 *</td><td>1.759</td><td>1.185-2.612</td><td>0.005 **</td></tr><tr><td>pT-stage > 0</td><td></td><td></td><td></td><td>4.334</td><td>2.292-8.193</td><td>0.000 **</td></tr><tr><td rowspan="2">OS</td><td>AST/ALT > 1.3</td><td>1.427</td><td>1.005-2.025</td><td>0.047 *</td><td>1.556</td><td>1.086-2.228</td><td>0.016 **</td></tr><tr><td>pT-stage > 0</td><td></td><td></td><td></td><td>2.564</td><td>1.567-4.194</td><td>0.000 **</td></tr></tbody></table>

¹ Crude estimates of the De Ritis ratio < 1.3 or ratio > 1.3; ² Adjusted estimates for variables ratio > 1.3, Age > 70, CACI; Charlson Age Comorbidity Index, ASA; American Society of Anesthesiologists, active smoking, BMI; Body Mass Index, pT-stage; pathological T-stage post-RC, NAC; neoadjuvant chemotherapy. DFS; Disease-free survival, CSS; Cancer-Specific Survival, OS; Overall survival. * Significant *p*-value in crude estimates suggest association between increased risk of mortality and a ratio > 1.3. ** Significant *p*-value in adjusted estimates age > 70, CACI, ASA active smoking, BMI, pT-stage, or NAC, suggest an association between increased risk of mortality and significant variable. Only variables with values of statistical significance any time in analysis are displayed in table.

There is no significant relationship between the downstaging of primary tumor and a high De Ritis ratio > 1.3 (p = 0.564 MWU) either in the entire cohort (Figure 5a) or in NAC-receivers only (Figure 5b) (p = 0.276 MWU). A visible difference can be seen in distribution between the downstaging grades, where NAC-receivers have a higher percentage of increased downstaging within the survival markers of CR and PR (Figure 5b), compared to the entire cohort (Figure 5a).

<figure><img src="image_66.png" /><figcaption>**Figure 5.** (a) Downstaging of primary tumor in patient cohort NAC and NAC-naïve n = 347, by the four downstaging categories with patients (%) within sub-groups of high or low De Ritis ratio. There is no significant relationship between downstaging and a high De Ritis ratio > 1.3 (p = 0.564 MWU); (b) Downstaging of primary tumor in NAC patients only n = 258, by the four downstaging categories with patients (%) within sub-groups of high or low De Ritis ratio. There is no significant relationship between downstaging and a high ratio > 1.3 (p = 0.276 MWU).</figcaption></figure>
Downstaging of tumor vs high AST/ALT ratio within all patients (left) and only NAC patients (right).

**4. Discussion**

A high pre-treatment De Ritis ratio was associated with higher mortality in both survival analysis (Figures 2a-c, 3a-c and 4a-c) and in crude and adjusted risk-analysis with increased HR if the ratio was >1.3 in patients, compared to low ratio <1.3. pT-stage >0 was also frequently associated with higher mortality in more endpoints than the De Ritis ratio (Table 2). Thus, a higher tumor stage with more severe cancer post-RC is most statistically associated with mortality than all of our other included variables, as was to be expected. We suggest that pT-stage post-RC that is of the current standard is still of higher prognostic value in predicting survival compared to a high De Ritis ratio pre-treatment. However, UBC is considered a glucose-dependent malignancy, and it is known that AST plays an essential role in glycolysis, which leads to a potential condition in which the increased glucose metabolism by cancer, known as the Warburg effect, exists with an increase in AST/ALT ratio and, therefore, possibly has a high ratio with cancer disease. However, the exact mechanism of the suggested condition has not been shown [8,13] and can only be hypothesized, which must be considered a weakness.

No significant relationship existed between downstaging of the primary tumor and a high De Ritis ratio in either the mixed cohort (p = 0.564 MWU) or in NAC-receivers only (p = 0.276 MWU) (Figure 5a,b). We, therefore, propose that the different NAC-levels did not affect the potential association between a high De Ritis ratio and downstaging, and that a high pre-treatment ratio may not be of prognostic value regarding downstaging response in either NAC- or NAC-eligible patients. To the best of our knowledge, these findings have not previously been described. The strength in this study is in the evaluation of a relatively large cohort (n = 347), considering extensive inclusion and exclusion criteria compared to other similar studies with higher heterogeneity in the study population. A selection bias could be argued regarding the selection process of patients described in the Methods, where patients with extra high comorbidity may have been excluded due to the inclusion criteria. However, our result may possibly reflect a healthier population than other, similar studies that made no such distinction between levels of disease pre-analysis. Consequently, we propose that our significant results are not primarily due to the high comorbidity in the study population, which may be considered a strength.

A potential weakness may be the HRs in Table 2. The Cox proportional hazard assumption was visually evaluated and the hazard ratio between low and high De Ritis ratio seems constant up to three years, and then decreases somewhat. After five years, the proportional hazard assumption can no longer be considered valid. Other unknown factors may influence the hazard around and after 3-5 years, which may be considered a weakness in the results displayed in Table 2. However, many recurrences were detected before three years post-RC, hence suggesting that the risk analysis utilizing De Ritis ratio has a clear interest for evaluation. Another weakness may lie in our cut-off level of a high De Ritis ratio > 1.3. This was selected from previous studies [3,7]. This may have influenced our results. However, we still had unique individual AST/ALT levels on our population, and can thus compare our results with studies that had the same cut-off level of 1.3, but different AST/ALT levels due to the different study population. We had similar results to several other studies regarding the association between higher mortality and a high ratio [7,10,19,22]. Finally, our results regarding an association between increased mortality and a high ratio lack statistical calculations regarding how strong that association may be. Therefore, we suggest that the strength in the association should be further investigated. This may provide more insight into whether said association between a high De Ritis ratio and mortality is so strong that the ratio can be considered a future prognostic factor for survival in MIBC.
**5. Conclusions**

A high pre-treatment De Ritis ratio is associated with an increased risk of mortal-
ity post-RC in endpoints DFS, CSS and OS. The association decreases over time and
requires further investigation to determine how strong the association is as a long-term
potential prognostic factor for mortality in MIBC. The De Ritis ratio is not suitable for
predicting downstaging.

**Author Contributions:** Conceptualization, A.S. and V.E.; methodology, A.S.; software, A.S. and J.S.; validation, V.E., O.H. and A.S.; formal analysis, V.E. and A.S.; investigation, V.E.; resources, A.S., F.A. (Farhood Alamdari) and F.A. (Firas Aljabery).; data curation, V.E., O.H., M.J. and Y.H.; writing—original draft preparation, V.E., O.H. and A.S.; writing—review and editing, V.E., A.S., O.H., J.S., Y.H., M.J., F.A. (Farhood Alamdari) and F.A. (Firas Aljabery).; visualization, V.E.; supervision, A.S.; project administration, V.E.; funding acquisition, A.S. All authors have read and agreed to the published version of the manuscript.

**Funding:** This research was funded by the Swedish Research Council funding for clinical research in medicine (ALF) in Västerbotten, VLL, grant number Bas-ALF/VLL RV-848051" and "The APC was funded by AS".

**Institutional Review Board Statement:** The study was conducted in accordance with the Declaration of Helsinki and approved by the Institutional Review Board (or Ethics Committee) of Etikprövn-ingsnämnden (EPN) Umeå, Sweden (protocol code 2013/463-31M, date of approval: 3 June 2014).

**Informed Consent Statement:** Patient consent was waived due to the study being retrospective and therefore no informed consent was required according to EPN.

**Data Availability Statement:** On reasonable request, the corresponding author can make available all codified data from the clinical database used for this study.

**Conflicts of Interest:** A.S., corresponding author, declares that he received Grant No. Bas-ALF/VLL RV-848051 as funding concerning this project, from The Swedish Research Council funding for clinical research in medicine (ALF) in Vaesterbotten, VLL, Sweden. All other authors declare no conflict of interest. The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript; or in the decision to publish the results.

**References**

1. Bladder Cancer Statistics: World Cancer Research Fund International. [Updated 23 March 2022]. Available online: https://www.wcrf.org/cancer-trends/bladder-cancer-statistics/ (accessed on 23 March 2022).

2. Epidemiologi—RCC Kunskapsbanken: Kunskapsbanken.cancercentrum.se; On Behalf of SKR (Sveriges Kommuner och regioner). Available online: https://kunskapsbanken.cancercentrum.se/diagnoser/urinblase-och-urinvagscancer/vardprogram/bakgrund-och-orsaker/#chapter-3--1-Epidemiologi (accessed on 23 March 2022).

3. Ha, Y.S.; Kim, S.W.; Chun, S.Y.; Chung, J.W.; Choi, S.H.; Lee, J.N.; Kim, B.S.; Kim, H.T.; Yoo, E.S.; Kwon, T.G.; et al. Association between De Ritis ratio (aspartate aminotransferase/alanine aminotransferase) and oncological outcomes in bladder cancer patients after radical cystectomy. *BMC Urol.* **2019**, *19*, 10. [CrossRef] [PubMed]

4. Rosenblatt, R.; Sherif, A.; Rintala, E.; Wahlqvist, R.; Ullén, A.; Nilsson, S.; Malmström, P.U. Pathologic downstaging is a surrogate marker for efficacy and increased survival following neoadjuvant chemotherapy and radical cystectomy for muscle-invasive urothelial bladder cancer. *Eur. Urol.* **2012**, *61*, 1229–1238. [CrossRef] [PubMed]

5. Eriksson, V.; Holmlund, J.; Wiberg, E.; Johansson, M.; Huge, Y.; Alamdari, F.; Svensson, J.; Aljabery, F.; Sherif, A. Adverse events during neoadjuvant chemotherapy for muscle invasive bladder cancer—A Swedish retrospective multicentre study of a clinical database. *Transl. Androl. Urol.* **2022**, *11*, 1105–1115. [CrossRef] [PubMed]

6. Xu, X.; Wang, Y.; Zhang, S.; Zhu, Y.; Wang, J. Exploration of Prognostic Biomarkers of Muscle-Invasive Bladder Cancer (MIBC) by Bioinformatics. *Evol. Bioinform. Online* **2021**, *17*, 117693432110492702021. [CrossRef] [PubMed]

7. Ghahari, M.; Salari, A.; Yazdi, M.G.; Nowroozi, A.; Fotovat, A.; Momeni, S.A.; Nowroozi, M.R.; Amini, E. Association between Preoperative de Ritis (AST/ALT) Ratio and Oncological Outcomes Following Radical Cystectomy in Patients with Urothelial Bladder Cancer. *Clin. Genitourin Cancer* **2022**, *20*, e89–e93. [CrossRef] [PubMed]

8. Yuk, H.D.; Jeong, C.W.; Kwak, C.; Kim, H.H.; Ku, J.H. De Ritis Ratio (Aspartate Transaminase/Alanine Transaminase) as a Significant Prognostic Factor in Patients Undergoing Radical Cystectomy with Bladder Urothelial Carcinoma: A Propensity Score-Matched Study. *Dis. Markers* **2019**, *2019*, 6702964. [CrossRef] [PubMed]

9. Batista, R.; Vinagre, N.; Meireles, S.; Vinagre, J.; Prazeres, H.; Leão, R.; Máximo, V.; Soares, P. Biomarkers for Bladder Cancer Diagnosis and Surveillance: A Comprehensive Review. *Diagnostics* **2020**, *10*, 39. [CrossRef] [PubMed]
10. Bezan, A.; Mrsic, E.; Krieger, D.; Stojakovic, T.; Pummer, K.; Zigeuner, R.; Georg, C.H.; Pichler, M. The Preoperative AST/ALT (De Ritis) Ratio Represents a Poor Prognostic Factor in a Cohort of Patients with Nonmetastatic Renal Cell Carcinoma. *J. Urol.* **2015**, *194*, 30–35. [CrossRef] [PubMed]

11. Su, S.; Liu, L.; Li, C.; Zhang, J.; Li, S. Prognostic Role of Pretreatment De Ritis Ratio (Aspartate Transaminase/Alanine Transaminase Ratio) in Urological Cancers: A Systematic Review and Meta-Analysis. *Front. Oncol.* **2020**, *10*, 1650. [CrossRef] [PubMed]

12. Lee, H.; Lee, S.E.; Byun, S.S.; Kim, H.H.; Kwak, C.; Hong, S.K. De Ritis ratio (aspartate transaminase/alanine transaminase ratio) as a significant prognostic factor after surgical treatment in patients with clear-cell localized renal cell carcinoma: A propensity score-matched study. *BJU Int.* **2017**, *119*, 261–267. [CrossRef] [PubMed]

13. Cheng, X.; Zhou, X.; Yi, M.; Xu, S.; Zhang, C.; Wang, G. Preoperative aspartate transaminase/alanine transaminase ratio as a prognostic biomarker in primary non-muscle-invasive bladder cancer: A propensity score-matched study. *BMC Urol.* **2021**, *21*, 1362021. [CrossRef] [PubMed]

14. Lindmark, G.; Gerdin, B.; Påhlman, L.; Bergström, R.; Glimelius, B. Prognostic predictors in colorectal cancer. *Dis. Colon Rectum* **1994**, *37*, 1219–1227. [CrossRef] [PubMed]

15. Stocken, D.D.; Hassan, A.B.; Altman, D.G.; Billingham, L.J.; Bramhall, S.R.; Johnson, P.J.; Freemantle, N. Modelling prognostic factors in advanced pancreatic cancer. *Br. J. Cancer* **2008**, *99*, 883–893. [CrossRef]

16. De Ritis, F.; Coltorti, M.; Giusti, G. An enzymic test for the diagnosis of viral hepatitis; the transaminase serum activities. *Clin. Chim. Acta* **1957**, *2*, 70–74. [CrossRef]

17. Liberti, M.V.; Locasale, J.W. The Warburg Effect: How Does it Benefit Cancer Cells? *Trends Biochem. Sci.* **2016**, *41*, 211–218. [CrossRef] [PubMed]

18. Sookoian, S.; Pirola, C.J. Alanine and aspartate aminotransferase and glutamine-cycling pathway: Their roles in pathogenesis of metabolic syndrome. *World J. Gastroenterol.* **2012**, *18*, 3775–3781. [CrossRef] [PubMed]

19. Gorgel, S.N.; Kose, O.; Koc, E.M.; Ates, E.; Akin, Y.; Yilmaz, Y. The prognostic significance of preoperatively assessed AST/ALT (De Ritis) ratio on survival in patients underwent radical cystectomy. *Int. Urol. Nephrol.* **2017**, *49*, 1577–1583. [CrossRef] [PubMed]

20. Birgisson, H.; Wallin, U.; Holmberg, L.; Glimelius, B. Survival endpoints in colorectal cancer and the effect of second primary other cancer on disease free survival. *BMC Cancer* **2011**, *11*, 438. [CrossRef] [PubMed]

21. Punt, C.J.; Buyse, M.; Köhne, C.H.; Hohenberger, P.; Labianca, R.; Schmoll, H.J.; Påhlman, L.; Sobrero, A.; Douillard, J.Y. Endpoints in adjuvant treatment trials: A systematic review of the literature in colon cancer and proposed definitions for future trials. *J. Natl. Cancer Inst.* **2007**, *99*, 998–1003. [CrossRef] [PubMed]

22. Hu, X.; Yang, W.X.; Wang, Y.; Shao, Y.X.; Xiong, S.C.; Li, X. The prognostic value of De Ritis (AST/ALT) ratio in patients after surgery for urothelial carcinoma: A systematic review and meta-analysis. *Cancer Cell Int.* **2020**, *20*, 39. [CrossRef] [PubMed]
<figure><img src="image_67.png" /></figure>
<figure><img src="image_68.png" /></figure>

Article

# Thromboembolic Events in Patients Undergoing Neoadjuvant Chemotherapy and Radical Cystectomy for Muscle-Invasive Bladder Cancer: A Study of Renal Impairment in Relation to Potential Thromboprophylaxis

Harriet Rydell <fnref n="1" />,†, Anna Ericson <fnref n="1" />,†, Victoria Eriksson <fnref n="1" /> ID, Markus Johansson <fnref n="1" />,<fnref n="2" />, Johan Svensson <fnref n="3" />, Viqar Banday <fnref n="1" />,‡ ID and Amir Sherif <fnref n="1" />,*,‡ ID

**Citation:** Rydell, H.; Ericson, A.; Eriksson, V.; Johansson, M.; Svensson, J.; Banday, V.; Sherif, A. Thromboembolic Events in Patients Undergoing Neoadjuvant Chemotherapy and Radical Cystectomy for Muscle-Invasive Bladder Cancer: A Study of Renal Impairment in Relation to Potential Thromboprophylaxis. *J. Pers. Med.* **2022**, *12*, 1961. https://doi.org/10.3390/jpm12121961

Academic Editor: Anne-Marie Caminade

Received: 30 September 2022

Accepted: 22 November 2022

Published: 27 November 2022

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

<figure><img src="image_69.png" /></figure>

**Copyright:** © 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

**Abstract:** Recent studies on patients with muscle-invasive bladder cancer (MIBC) undergoing neoadjuvant chemotherapy (NAC) have shown an association between NAC and thromboembolic events (TEE) prior to radical cystectomy (RC). Recent studies suggest that central venous access catheters (CVAs) may induce TEEs, and low-molecular-weight heparin (LMWH) has been mentioned as possible prophylaxis. However, other studies have shown a high incidence of decreased renal function in these patients. The purpose of this study was to determine the portion of MIBC patients with NAC-induced TEEs who had decreased preoperative renal function for whom LMWH potentially would not be beneficial as prophylaxis. We identified 459 cystectomized MIBC patients from two Swedish medical centers from 2009 to 2021. The inclusion criterion of cT2-T4aN0M0 resulted in 220 eligible patients, who were further divided into NAC-administered (n = 187) and NAC-eligible (n = 33), the tentative control group. Values of renal function before, during, and after each NAC cycle were retrospectively collected from individual medical records. Amongst the NAC-administered patients with TEE (n = 29), 41% (95% CI 23.5–61.1%) of patients had decreased renal function. Thus, a substantial portion of NAC-administered patients who developed TEEs had reduced renal function and would have been less likely to have benefited from renal clearance-dependent LMWH prophylaxis.

**Keywords:** complications; cystectomy; low-molecular-weight heparin; neoadjuvant therapy; thromboembolism; urinary bladder neoplasms

## 1. Introduction

Urinary bladder cancer accounted for around 570,000 new cancer cases and about 200,000 deaths worldwide in 2020, making it the tenth most commonly diagnosed cancer [1]. Bladder cancer manifests either as non-muscle-invasive cancer or a muscle-invasive form (MIBC); the latter form accounts for approximately 25% of cases at diagnosis. Urothelial MIBC was, prior to the introduction of neoadjuvant chemotherapy (NAC), associated with a poor prognosis, with a 5-year overall survival (OS) post-RC of 50%. The current treatment regimen in Europe for MIBC includes NAC in medically fit patients and radical cystectomy (RC) [2]. The inclusion of NAC pre-RC aims to eradicate micro-metastatic dissemination and has been shown to significantly increase survival compared with RC only [3,4]. We previously showed that chemo-responding patients who downstaged with

<footnote n="1">¹ Department of Surgical and Perioperative Sciences, Urology and Andrology, Umeå University, 90187 Umeå, Sweden</footnote>

<footnote n="2">² Department of Surgery, Urology Section, Sundsvall-Härnösand Hospital, 85186 Sundsvall, Sweden</footnote>

<footnote n="3">³ Department of Statistics, Umeå School of Business, Economics and Statistics, Umeå University, 90187 Umeå, Sweden</footnote>

<footnote>* Correspondence: amir.sherif@umu.se; Tel.: +46-705-229-104</footnote>

<footnote>† These authors contributed equally to this work.</footnote>

<footnote>‡ These authors contributed equally to this work.</footnote>
complete response (CR) to pT0N0M0) had an absolute risk reduction of 31% for death at the five-year median observation time (OS) [5].

The NAC regimen for MIBC patients in Sweden in most centers consists of a cisplatin-
based combination of methotrexate, vinblastine, doxorubicin, and cisplatin (MVAC), while
in patients who are ineligible for cisplatin, a carboplatin-gemcitabine combination is
used [6]. Eligibility for NAC, according to Swedish guidelines, are age ≤ 75 years, re-
nal function with eGFR > 50-60, no hearing impediments, and acceptable comorbidities [7].
NAC treatment is generally well tolerated; however, adverse events, including throm-
boembolic events (TEEs). have been shown to occur during and after NAC treatment [8,9].
Eriksson et al., using a multicenter clinical dataset, showed serious grade 4 TEEs in patients
treated with NAC. In addition, acute kidney injury and chronic kidney disease were seen
in 41% and 11% of patients, respectively [10]. Interestingly, 30% of patients receiving NAC
had to terminate the treatment prematurely. Among these patients, 62% of terminations
were due to acute kidney injury. There was a significant association between decreased
kidney injury and increased downstaging, possibly reflecting the effect of completed in-
tended NAC cycles on downstaging, as reported earlier [5]. Renal impairment is a fraught
consequence of cisplatin-based combinations and one of the key determinants for NAC
eligibility [7,11]. Mehrazin et al., in a recent study, showed a continued decline in eGFR
rates in MIBC-NAC patients after discharge. In their study, the rate of decline was related
to eGFR rates at discharge, with up to 43% of the patients showing a decline [12]. Deep
vein thrombosis (DVT) and pulmonary embolism (PE) are two lethal complications during
NAC and RC. Endothelial damage and the hypercoagulable state during cancer treatment
continues post treatment [13]. Thromboprophylaxis pre- and post-surgery has become a
standard in tertiary care centers and is highly encouraged as a quality-of-care indicator [14].
Low-molecular-weight heparin (LMWH) is frequently prescribed for the prevention and
treatment of TEEs in patients [15]. LMWH provides a reduction in the incidence of venous
thromboembolism and requires minimal monitoring [16,17]. However, combined with im-
paired renal function, the major mode of excretion for LMWH, there is an increased risk of
supratherapeutic accumulation of LMWH with an associated risk of major bleeding [12]. To
the best of our knowledge, no international or national guidelines exist advocating LMWH
as prophylaxis against TEEs amongst MIBC-NAC patients during the NAC period. In this
retrospective study, we investigate the incidence of NAC-induced renal impairment in our
research database. The objective was to assess the incidence of patients with NAC-induced
TEEs who may not have benefited from LMWH if prophylaxis had hypothetically been
used.

**2. Materials and Methods**

2.1. *Data Collection and NAC Eligibility*

The study population was derived from an extensive dataset based on records of radically cystectomized patients with urothelial carcinoma in the urinary bladder in the years 2009–2021. Patients’ data were retrospectively collected from individual medical records from the two participating Swedish centers: Norrlands Universitetsjukhus in Umeå and Länssjukhuset in Sundsvall. The inclusion criteria allowed for MIBC carcinomas categorized as cT2-T4bN0M0 with urothelial or mixed-type histopathology, thus excluding non-muscle-invasive cancers, disseminated disease before treatment initiation, and histopathology without any urothelial component. Hence, of the 459 radically cystectomized patients who were initially available for consideration, 220 were suitable for analysis, as 239 patients were excluded due to unfit histopathology, non-cT2-4aN0M0, or not fulfilling the criteria for NAC treatment and thus being deemed as non-NAC-eligible patients. For detailed information on patient selection, see Figure 1.
<figure><img src="image_70.png" /><figcaption>**Figure 1.** Flowchart of study population. MIBC; muscle-invasive bladder cancer. NAC; neoadjuvant chemotherapy. TEE; thromboembolic event.</figcaption></figure>

The majority of MIBC patients in Sweden today receive 3 or 4 cycles of chemotherapy before RC, and in 2021, 71% of all MIBC patients received NAC in Umeå. On the contrary, in 2009, only 17% received NAC in Umeå, as NAC was not recommended by clinical guidelines until more recent years. Hence, a portion of all MIBC patients in Sweden did not receive NAC and was solely radically cystectomized. As chemotherapy by time was introduced into modern MIBC guidelines, the number of NAC-administrated patients rose. However, a few patients still are untreated due to other circumstances; for example, some choose to reject NAC. Thus, non-NAC but NAC-eligible patients provide us with a suitable parallel control group, as these patients can be regarded as exhibiting the same clinical status as those treated with NAC (Table 1). Consequently, this study focused on the NAC-administered patients (n = 187) and NAC-eligible non-NAC patients (n = 33), who were considered being the tentative control group. By analyzing the NAC-eligible patients, any confounding factors regarding the carcinoma and its innate effects on the cohort's kidney function could be overviewed. The basic characteristics of both NAC-treated as well as NAC-eligible patients are shown in Table 1. To establish which patients would have been eligible for NAC, medical journals and preestablished NAC criteria were used with the guidance of a senior urologist and the national guidelines. To be considered NAC-eligible, patients had to be ≤75 years old with an eGFR > 50 or creatinine < 100 and a Charlson Age Comorbidity Index (CACI) of 6 or less. The CACI is used to determine a standardized 10-year survival prediction and was individually calculated for each patient. If a patient fulfilled the criteria to receive NAC but the multidisciplinary team recommended abstaining
from treatment, the latter judgement was regarded by the study being the final verdict regarding a patient's NAC eligibility.

<table><caption>**Table 1.** Baseline characteristics of the study cohorts.</caption><thead><tr><th rowspan="2">VARIABLE</th><th colspan="2">NAC-ADMINISTRATED n = 187</th><th colspan="2">NAC-ELIGIBLE n = 33</th></tr><tr><th>WITH TEE n = 29</th><th>NO TEE n = 158</th><th>TEE + NO TEE n = 187</th><th></th></tr><tr><th></th><th>Mean (SD)</th><th>Mean (SD)</th><th>Mean (SD)</th><th>Mean (SD)</th></tr></thead><tbody><tr><td>AGE</td><td>67 (7)</td><td>68 (7)</td><td>68 (7)</td><td>67 (9)</td></tr><tr><td>CACI</td><td>5 (1)</td><td>5 (1)</td><td>5 (1)</td><td>5 (1)</td></tr><tr><td>NO. NAC CYCLES</td><td>3 (1)</td><td>3 (1)</td><td>3 (1)</td><td>0 (0)</td></tr><tr><td></td><td>n (%)</td><td>n (%)</td><td>n (%)</td><td>n (%)</td></tr><tr><td>MEDICAL CENTER</td><td></td><td></td><td></td><td></td></tr><tr><td>UMEÅ</td><td>25 (86)</td><td>120 (76)</td><td>145 (77)</td><td>22 (67)</td></tr><tr><td>SUNDSVALL</td><td>4 (14)</td><td>38 (24)</td><td>42 (22)</td><td>11 (33)</td></tr><tr><td>GENDER</td><td></td><td></td><td></td><td></td></tr><tr><td>WOMAN</td><td>4 (14)</td><td>41 (26)</td><td>45 (24)</td><td>5 (15)</td></tr><tr><td>MAN</td><td>25 (86)</td><td>118 (75)</td><td>143 (76)</td><td>28 (85)</td></tr><tr><td>cT-STADIUM</td><td></td><td></td><td></td><td></td></tr><tr><td>T2</td><td>15 (51)</td><td>91 (57)</td><td>106 (57)</td><td>22 (67)</td></tr><tr><td>T3</td><td>12 (41)</td><td>54 (34)</td><td>66 (35)</td><td>11 (33)</td></tr><tr><td>T4 & T4a</td><td>2 (7)</td><td>13 (8)</td><td>15 (8)</td><td>0 (0)</td></tr><tr><td>THROMBOSIS PROPHYLAXIS</td><td></td><td></td><td></td><td></td></tr><tr><td>Anticoagulant</td><td>12 (41)</td><td>9 (6)</td><td>21 (11)</td><td>3 (9)</td></tr><tr><td>Antiplatelet</td><td>28 (96)</td><td>23 (14)</td><td>51 (27)</td><td>5 (15)</td></tr></tbody></table>

NAC: neoadjuvant chemotherapy. TEE: thromboembolic event. SD: standard deviation. CACI: Charlson comorbidity index.

Laboratory values regarding renal function were gathered, adding to the database's already large collection of clinicopathological variables, such as age, gender, smoking habits, BMI, ASA, CACI, and TEE. Values for renal function are monitored in MIBC patients but are rarely measured by one single variable. Thus, four measurements were obtained for the study to obtain a conclusive portrayal of each patient's clinical kidney status: serum/plasma creatinine (S/P-creatinine), eGFR by cystatin c, eGFR by creatinine and chrome-ethylenediaminetetraacetic acid (Cr-EDTA). The cut-offs for each measurement were in accordance with Swedish clinical practice and the standardized reference intervals: S/P-creatinine > 90 for females and > 100 for males, eGFR (creatinine) < 60, eGFR (cystatin c) < 60, and Cr-EDTA < 60. The reference intervals for all analyzed measurements are presented in Table 2. Values were collected at predetermined time intervals: pre-TUR-b, before initiation of NAC, after each provided NAC cycle, and, finally, before RC. Not all patients had an available value for each measurement in every time period, and if values were inaccessible, they were considered missing data. As no conclusion can be made from unavailable measurements, the study considered missing data to be non-pathological values. No patient had missing values in all 4 extracted measurements. Utilizing each exact value for every available measurement that was collected allowed the stratification of the cohort's population into two sub-categories: patients with or without reduced kidney function. As one patient could potentially have either one single or several pathological values, multiple groupings regarding the type of measurement of kidney function at different time periods were considered.

Another distinction in the study cohort was made by separating the NAC-administrated who did or did not suffer a TEE. The TEE incidences, presented in Table 3, were used for analysis, with PE, DVT, TEEs anatomically related to the central venous access catheters (CVA), angina/myocardial infarction (MI), and transient ischemic attack (TIA)/stroke used as included types of events. The events were registered if the time of the TEE occurred
during the period from TUR-b to RC. Four patients had a total number of 2 TEEs, and one patient had a total of 3 TEEs.

<table><caption>**Table 2.** Reference interval and cut-off values for pathology of included measurements.</caption><thead><tr><th>MEASUREMENT</th><th></th><th>REFERENCE INTERVAL</th><th>CUT OFF VALUE</th></tr></thead><tbody><tr><td rowspan="2">P/S-Creatinine</td><td>Male</td><td>60-100</td><td>&gt;100</td></tr><tr><td>Female</td><td>50-90</td><td>&gt;90</td></tr><tr><td rowspan="2">eGFR (creatinine)</td><td>18-50 years</td><td>&gt;80</td><td>&lt;60</td></tr><tr><td>&gt;50 years</td><td>&gt;60</td><td>&lt;60</td></tr><tr><td rowspan="2">eGFR (cystatin c)</td><td>18-50 years</td><td>&gt;80</td><td>&lt;60</td></tr><tr><td>&gt;50 years</td><td>&gt;60</td><td>&lt;60</td></tr><tr><td rowspan="3">Cr-EDTA</td><td>20-50 years</td><td>80-125 mL/min/1.73 m<sup>2</sup></td><td>&lt;60</td></tr><tr><td>51-65 years</td><td>60-110 mL/min/1.73 m<sup>2</sup></td><td>&lt;60</td></tr><tr><td>66-80 years</td><td>50-90 mL/min/1.73 m<sup>2</sup></td><td>&lt;60</td></tr></tbody></table>

P/S: plasma or serum. eGFR: estimated glomerular filtration rate. Cr-EDTA: chrome ethylenediaminetetraacetic acid.

<table><caption>**Table 3.** TEE incidences amongst NAC-administrated patients pre-cystectomy.</caption><thead><tr><th></th><th>NAC-ADMINISTRATED n = 187<br>WITH TEE n = 29</th></tr></thead><tbody><tr><td>TYPE OF TEE</td><td></td></tr><tr><td>DVT</td><td>2</td></tr><tr><td>THROMBOPHLEBITIS</td><td>3</td></tr><tr><td>PE</td><td>18</td></tr><tr><td>FROM CVA</td><td>11</td></tr><tr><td>ANGINA/MI</td><td>1</td></tr></tbody></table>

TEE, thromboembolic events; NAC, neoadjuvant chemotherapy; DVT, deep vein thrombosis; PE, pulmonary embolism; FROM CVA, anatomically related to the central venous access; MI, myocardial infarction.

## 2.2. Statistics

Population characteristics for interval variables are described as means and standard deviation, and those for nominal variables are described as frequencies and percentages. The confidence interval for the percentage of patients with reduced kidney function among the defined groups of the cohort was calculated with the Clopper-Pearson method. The statistical analyses were performed with IBM SPSS Statistics for Windows, Version 28.0. Armonk, NY, USA: IBM Corp.

## 2.3. Ethics

The study was approved by the regional ethics board in Umeå: EPN-Umeå, dnr: 2013/463-31M and amendment 2016/403-32. The study conformed to the provisions of the Declaration of Helsinki (as revised in Fortaleza, Brazil, October 2013). The regional ethics board specifically decided that informed consent from the participants was to be considered redundant, especially due to the high mortality of MIBC as well as the retrospective nature of the study.

## 3. Results

The clinicopathological variables for each of the 220 analyzed patients were extracted from the extensive official database charting all MIBC patients who have undergone RC in Sweden and are presented in Table 1. Amongst these variables, TEE incidences were collected, enabling division into subgroups depending on whether patients suffered postulated NAC-induced thrombosis or not. Twenty-nine patients in the NAC-administrated group suffered one or more TEEs. Out of them, 18 were PEs and 11 were thromboses
anatomically connected to the CVA. Three patients suffered thrombophlebitis as a TEE, two had DVTs, and, finally, one patient presented with angina. Four patients developed two individual TEEs, and one patient developed three.

The findings regarding pathological renal values before the initiation of chemotherapy are presented in Table 4, elucidating any existing kidney damage in the study population pre-NAC and thus establishing the baseline measurements in each group. Non-pathological values for P/S-creatinine and eGFR are predetermined requirements for NAC eligibility; consequently, no patients in this group had reduced kidney function. In addition, as no patient in the NAC-eligible group suffered any TEE pre-RC, only patients without TEEs are presented. The reduced kidney function in the study population during the entire period, TUR-b to RC, is presented in Table 5. In the NAC-administrated group, 98 patients (52%) out of the total number of 187 had reduced kidney function pre-RC. Pathological measurements were found in 12 patients belonging to the NAC-administrated subgroup who also suffered TEEs (n = 29), resulting in 41% (95% CI 23–61%).

<table><caption>**Table 4.** Reduced kidney function after TUR-B, before first NAC cycle.</caption><thead><tr><th></th><th colspan="2">NAC-ADMINISTRATED<br>(n = 187)</th><th colspan="2">NAC-ELIBIGLE<br>(n = 33)</th></tr><tr><th></th><th>WITH TEE<br>n (%)</th><th>NO TEE n (%)</th><th>TEE + NO TEE<br>n (%)</th><th>NO TEE n (%)</th></tr></thead><tbody><tr><td></td><td>n = 29</td><td>n = 158</td><td>n = 187</td><td>n = 33</td></tr><tr><td colspan="5"><strong>REDUCED KIDNEY FUNCTION PRE-NAC 1</strong></td></tr><tr><td colspan="5"><strong>MEASUREMENT</strong></td></tr><tr><td>P/S-Creatinine</td><td>6 (21)</td><td>29 (18)</td><td>35 (19)</td><td>0 (0)</td></tr><tr><td>eGFR (creatinine)</td><td>3 (10)</td><td>14 (9)</td><td>17 (9)</td><td>0 (0)</td></tr><tr><td>eGFR (cystatin c)</td><td>1 (3)</td><td>25 (16)</td><td>26 (14)</td><td>0 (0)</td></tr><tr><td>Cr-EDTA</td><td>3 (10)</td><td>21 (13)</td><td>24 (13)</td><td>0 (0)</td></tr><tr><td colspan="5"><strong>ALL MEASUREMENTS</strong></td></tr><tr><td></td><td>9 (31)</td><td>50 (32)</td><td>59 (31)</td><td>0 (0)</td></tr></tbody></table>

For NAC eligibility, non-pathological renal functional values were required as predetermined by the study; thus, no NAC-eligible patients present any kidney damage at diagnosis. All measurements: one or more of the four values were pathological in the same patient. TUR-B: transurethral resection of the bladder. NAC: neoadjuvant chemotherapy. TEE: thromboembolic event. P/S: plasma or serum. eGFR: estimated glomerular filtration rate. Cr-EDTA: chrome ethylenediaminetetraacetic acid.

<table><caption>**Table 5.** Reduced kidney function at any point during the entire period; pre-TUR-B to before cystectomy.</caption><thead><tr><th></th><th colspan="2">NAC-ADMINISTRATED n = 187</th><th colspan="2">NAC-ELIBIGLE<br>(n = 33)</th></tr><tr><th></th><th>WITH TEE<br>n (%)</th><th>NO TEE n (%)</th><th>TEE + NO TEE<br>n (%)</th><th>NO TEE n (%)</th></tr></thead><tbody><tr><td></td><td>n = 29</td><td>n = 158</td><td>n = 187</td><td>n = 33</td></tr><tr><td colspan="5"><strong>REDUCED KIDNEY FUNCTION PRE CE</strong></td></tr><tr><td colspan="5"><strong>MEASUREMENT</strong></td></tr><tr><td>P/S-Creatinine</td><td>9 (31)</td><td>69 (44)</td><td>78 (42)</td><td>9 (27)</td></tr><tr><td>eGFR (creatinine)</td><td>5 (17)</td><td>37 (23)</td><td>42 (22)</td><td>1 (3)</td></tr><tr><td>eGFR (cystatin c)</td><td>2 (7)</td><td>31 (20)</td><td>33 (18)</td><td>0 (0)</td></tr><tr><td>Cr-EDTA</td><td>3 (10)</td><td>22 (14)</td><td>25 (13)</td><td>0 (0)</td></tr><tr><td colspan="5"><strong>ALL MEASUREMENTS</strong></td></tr><tr><td></td><td>12 (41)</td><td>86 (54)</td><td>98 (52)</td><td>9 (27)</td></tr></tbody></table>

All measurements: one or more of the four values were pathological in the same patient. TUR-B: Transurethral resection of the bladder. NAC: neoadjuvant chemotherapy. TEE: thromboembolic event. RC: radical cystectomy. P/S: plasma or serum. eGFR: estimated glomerular filtration rate. Cr-EDTA: chrome ethylenediaminetetraacetic acid.
Table 6 presents the portion of the study population in which at least one of the four predetermined and analyzed measurements were accessible. Out of the 220 patients, 35 (16%) had no obtainable values between TUR-b and the first NAC cycle and were thus left as missing data.

<table><caption>**Table 6.** Patients with available values at any point in the entire time period: TUR-b–pre-cystectomy.</caption><thead><tr><th rowspan="3"></th><th colspan="2">NAC-ADMINISTRATED<br>n = 187</th><th colspan="2">NAC-ELIGIBLE<br>n = 33</th></tr><tr><th>WITH TEE<br>n (%)</th><th>NO TEE<br>n (%)</th><th>TEE + NO TEE<br>n (%)</th><th>NO TEE n (%)</th></tr><tr><th>n = 29</th><th>n = 158</th><th>n = 187</th><th>n = 33</th></tr></thead><tbody><tr><th colspan="5">MEASUREMENT</th></tr><tr><td>P/S-Creatinine</td><td>29 (100)</td><td>158 (100)</td><td>187 (100)</td><td>32 (97)</td></tr><tr><td>eGFR (creatinine)</td><td>20 (69)</td><td>109 (69)</td><td>129 (69)</td><td>10 (30)</td></tr><tr><td>eGFR (cystatin c)</td><td>0 (0)</td><td>15 (9)</td><td>15 (8)</td><td>1 (3)</td></tr><tr><td>Cr-EDTA</td><td>1 (3)</td><td>2 (1)</td><td>3 (2)</td><td>0 (0)</td></tr></tbody></table>

TUR-b: transurethral resection of the bladder, NAC: neoadjuvant chemotherapy. TEE: thromboembolic event. P/S: plasma or serum. eGFR: estimated glomerular filtration rate. Cr-EDTA: chrome ethylenediaminetetraacetic acid.

## 4. Discussion

According to our findings, a considerable portion of the NAC-administrated patients who suffered a TEE during treatment had reduced renal function, as 41% (95% CI 23–61%) had one or more pathological values throughout the entire time period, up until the day of cystectomy. Fifty-four percent of the NAC-administrated patients who underwent their chemotherapy without a thrombolytic adverse event also had reduced kidney function, presenting a similar result as the TEE group. The most frequent pathological value amongst the NAC-treated TEE patients was P/S-creatinine, with nine patients (31%) showcasing a higher value than the established reference interval. This is possibly in accordance with P/S-creatinine being the most frequently used measurement for kidney function in Sweden. No noteworthy differences were found between the NAC-administrated subgroups regarding kidney function before the initiation of chemotherapy. Hence, the two groups of TEE patients and those who did not suffer a TEE could be considered to have the same baseline renal function before the first NAC cycle.

The main objective of the study was to investigate renal function in NAC-administrated patients who had suffered a TEE and to assess their ability to potentially respond to LMWH-prophylaxis. Hence, the participants were divided into subgroups of those who did or did not develop thrombosis during the time between diagnosis and RC. NAC inherently contributes to an amplified TEE risk in MIBC patients undergoing RC, and recent findings have suggested that the choice of CVA for chemotherapy can influence the increased risk [8,18,19]. As LMWH regularly is used in TEE treatment and commonly as prophylaxis in other high-risk patient groups, suggestions have been made to also use LMWH in cases in which MIBC-NAC patients would be considered high-risk for TEE. Accordingly, Mehrazin et al. suggested recently that established reduced kidney function would render heparin-based prophylaxis sub-therapeutic by accumulation, which in turn would increase the risk of clinically significant bleeding. Our results showed that in NAC-administrated patients who did suffer a TEE, 41% would potentially not have responded to prophylactic treatment. Thus, the argument can be made to contemplate whether LMWH can be considered the single solution to the increased TEE risk in this patient group or whether instead a shift in the main focus should be made to the choice of CVA.

As the hypothetical LMWH treatment would be initiated at the start of chemotherapy (NAC), the distinction between the TEE and non-TEE patients would not be yet established. Therefore, there is relevance in also establishing the incidence of reduced renal function in NAC-administrated patients who have not suffered a TEE. In the NAC-administrated no-TEE group, 54% were found to have reduced renal function during the entire pre-RC
period. Hence, this group showed similar results to the TEE-positive subgroup, indicating that LMWH could be subtherapeutic in the NAC-MIBC group as a whole. The NAC-eligible non-NAC group presented with a lower prevalence of pathological renal values than the groups (w/wo TEE) treated with NAC, possibly reflecting the negative effect that MIBC might inherently possess on kidney function.

However, because of this study's numerically limited cohort population, a larger patient selection is needed to further establish the incidence of reduced kidney function amongst NAC-administrated MIBC patients. Further important associated subjects to evaluate are how the current clinical use of LMWH contributes to reduced renal function, as well as extending the time period of analysis both pre-RC as well as post-RC. A strength of this study is that the analyzed measurements are meticulously extracted from individual medical records; thus, the presented findings could be regarded as highly reliable.

**5. Conclusions**

Our findings show that a substantial portion of NAC-administered patients with TEEs had reduced renal function pre-RC, and therefore, they would potentially not be suited for renal clearance-dependent LMWH prophylaxis. However, further investigations are needed to fully comprehend the prevalence of reduced renal function amongst MIBC patients.

**Author Contributions:** Conceptualization, A.S., H.R. and A.E.; methodology, A.S.; software, A.S and J.S.; validation, H.R., A.E., J.S. and A.S.; formal analysis, H.R., V.B., A.E. and A.S.; investigation, H.R., A.E., V.E. and A.S.; resources, A.S.; data curation, V.E., H.R., A.E., M.J. and J.S.; writing—original draft preparation, H.R., A.E., V.B. and A.S.; writing—review and editing, H.R., A.E., V.B., J.S., M.J. and A.S.; visualization, A.E.; supervision, A.S.; project administration, H.R., V.B. and A.S.; funding acquisition, A.S. All authors have read and agreed to the published version of the manuscript.

**Funding:** This research was funded by the Swedish Research Council funding for clinical research in medicine (ALF) in Västerbotten, VLL, grant number Bas-ALF/VLL RV-848051, and the APC was funded by AS.

**Institutional Review Board Statement:** The study was conducted in accordance with the Declaration of Helsinki and approved by the institutional review board (or ethics committee) of Etikprövningsnämnden (EPN) Umeå, Sweden (protocol code 2013/463-31M, date of approval: 3rd of June 2014).

**Informed Consent Statement:** Patient consent was waived due to the study being retrospective, and therefore, no informed consent was required according to EPN.

**Data Availability Statement:** On reasonable request, the corresponding author can make available all codified data from the clinical database used for this study.

**Conflicts of Interest:** A.S., the corresponding author, declares that he received grant no. Bas-ALF/VLL RV-848051 as funding concerning this project from the Swedish Research Council funding for clinical research in medicine (ALF) in Västerbotten, VLL, Sweden. All other authors declare no conflicts of interest. The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript; or in the decision to publish the results.

**References**

1. Sung, H.; Ferlay, J.; Siegel, R.L.; Laversanne, M.; Soerjomataram, I.; Jemal, A.; Bray, F. Global Cancer Statistics 2020: GLOBOCAN Estimates of Incidence and Mortality Worldwide for 36 Cancers in 185 Countries. *CA Cancer J. Clin.* **2021**, *71*, 209–249. [CrossRef] [PubMed]

2. European Association of Urology. Muscle-Invasive and Metastatic Bladder Cancer. Available online: https://uroweb.org/guidelines/muscle-invasive-and-metastatic-bladder-cancer (accessed on 30 September 2022).

3. Hamid, A.; Ridwan, F.R.; Parikesit, D.; Widia, F.; Mochtar, C.A.; Umbas, R. Meta-analysis of neoadjuvant chemotherapy compared to radical cystectomy alone in improving overall survival of muscle-invasive bladder cancer patients. *BMC Urol.* **2020**, *20*, 158. [CrossRef] [PubMed]

4. Sherif, A.; Holmberg, L.; Rintala, E.; Mestad, O.; Nilsson, J.; Nilsson, S.; Malmström, P.U.; Group, N.U.C. Neoadjuvant cisplatinum based combination chemotherapy in patients with invasive bladder cancer: A combined analysis of two Nordic studies. *Eur. Urol.* **2004**, *45*, 297–303. [CrossRef] [PubMed]
5. Rosenblatt, R.; Sherif, A.; Rintala, E.; Wahlqvist, R.; Ullén, A.; Nilsson, S.; Malmström, P.U. Pathologic downstaging is a surrogate marker for efficacy and increased survival following neoadjuvant chemotherapy and radical cystectomy for muscle-invasive urothelial bladder cancer. *Eur. Urol.* **2012**, *61*, 1229-1238. [CrossRef] [PubMed]

6. Jerlström, T.; Chen, R.; Liedberg, F.; Andrén, O.; Ströck, V.; Aljabery, F.A.S.; Hosseini, A.; Sherif, A.; Malmström, P.U.; Ullén, A.; et al. No increased risk of short-term complications after radical cystectomy for muscle-invasive bladder cancer among patients treated with preoperative chemotherapy: A nation-wide register-based study. *World J. Urol.* **2020**, *38*, 381-388. [CrossRef] [PubMed]

7. Leiberg, F. Nationellt Vårdprogram Cancer i Urinblåsa, Njurbäcken, Urinledare och Urinrör. Available online: https://cancercentrum.se/samverkan/cancerdiagnoser/urinblasa-urinvagar/vardprogram/ (accessed on 28 June 2022).

8. Ottosson, K.; Pelander, S.; Johansson, M.; Huge, Y.; Aljabery, F.; Sherif, A. The increased risk for thromboembolism pre-cystectomy in patients undergoing neoadjuvant chemotherapy for muscle-invasive urinary bladder cancer is mainly due to central venous access: A multicenter evaluation. *Int. Urol. Nephrol.* **2020**, *52*, 661-669. [CrossRef] [PubMed]

9. Ording, A.G.; Nielsen, M.E.; Smith, A.B.; Horváth-Puhó, E.; Sørensen, H.T. Venous thromboembolism and effect of comorbidity in bladder cancer: A danish nationwide cohort study of 13,809 patients diagnosed between 1995 and 2011. *Urol. Oncol.* **2016**, *34*, e291-e298. [CrossRef] [PubMed]

10. Eriksson, V.; Holmlund, J.; Wiberg, E.; Johansson, M.; Huge, Y.; Alamdari, F.; Svensson, J.; Aljabery, F.; Sherif, A. Adverse events during neoadjuvant chemotherapy for muscle invasive bladder cancer-a Swedish retrospective multicentre study of a clinical database. *Transl. Androl. Urol.* **2022**, *11*, 1105-1115. [CrossRef] [PubMed]

11. Seng, S.; Liu, Z.; Chiu, S.K.; Proverbs-Singh, T.; Sonpavde, G.; Choueiri, T.K.; Tsao, C.K.; Yu, M.; Hahn, N.M.; Oh, W.K.; et al. Risk of venous thromboembolism in patients with cancer treated with Cisplatin: A systematic review and meta-analysis. *J. Clin. Oncol.* **2012**, *30*, 4416-4426. [CrossRef] [PubMed]

12. Mehrazin, R.; Piotrowski, Z.; Egleston, B.; Parker, D.; Tomaszweski, J.J.; Smaldone, M.C.; Abbosh, P.H.; Ito, T.; Bloch, P.; Iffrig, K.; et al. Is extended pharmacologic venous thromboembolism prophylaxis uniformly safe after radical cystectomy? *Urology* **2014**, *84*, 1152-1156. [CrossRef] [PubMed]

13. Sørensen, C.; Andersen, M.; Kristiansen, V.B.; Jensen, R.; Wille-Jørgensen, P. The occurrence of late thromboembolic complications after elective abdominal surgery. *Ugeskr. Laeger* **1990**, *152*, 1586-1587. [PubMed]

14. Lyman, G.H.; Carrier, M.; Ay, C.; Di Nisio, M.; Hicks, L.K.; Khorana, A.A.; Leavitt, A.D.; Lee, A.Y.Y.; Macbeth, F.; Morgan, R.L.; et al. American Society of Hematology 2021 guidelines for management of venous thromboembolism: Prevention and treatment in patients with cancer. *Blood Adv.* **2021**, *5*, 927-974. [CrossRef] [PubMed]

15. Geerts, W.H.; Bergqvist, D.; Pineo, G.F.; Heit, J.A.; Samama, C.M.; Lassen, M.R.; Colwell, C.W. Prevention of venous thromboembolism: American College of Chest Physicians Evidence-Based Clinical Practice Guidelines (8th Edition). *Chest* **2008**, *133*, 381s-453s. [CrossRef] [PubMed]

16. ENOXACAN Study Group. Efficacy and safety of enoxaparin versus unfractionated heparin for prevention of deep vein thrombosis in elective cancer surgery: A double-blind randomized multicentre trial with venographic assessment. *Br. J. Surg.* **1997**, *84*, 1099-1103.

17. Nicolaides, A.N.; Fareed, J.; Kakkar, A.K.; Comerota, A.J.; Goldhaber, S.Z.; Hull, R.; Myers, K.; Samama, M.; Fletcher, J.; Kalodiki, E.; et al. Prevention and treatment of venous thromboembolism-International Consensus Statement. *Int. Angiol.* **2013**, *32*, 111-260. [PubMed]

18. Browne, C.; Davis, N.F.; Nolan, W.J.; MacCraith, E.D.; Lennon, G.M.; Mulvin, D.W.; Galvin, D.J.; Quinlan, D.M. Neoadjuvant Platinum-Based Chemotherapy is an Independent Predictor for Preoperative Thromboembolic Events in Bladder Cancer Patients Undergoing Radical Cystectomy. *Curr. Urol. 2017*, *10*, 132-135. [CrossRef] [PubMed]

19. Duivenvoorden, W.C.; Daneshmand, S.; Canter, D.; Lotan, Y.; Black, P.C.; Abdi, H.; van Rhijn, B.W.; Fransen van de Putte, E.E.; Zareba, P.; Koskinen, I.; et al. Incidence, Characteristics and Implications of Thromboembolic Events in Patients with Muscle Invasive Urothelial Carcinoma of the Bladder Undergoing Neoadjuvant Chemotherapy. *J. Urol.* **2016**, *196*, 1627-1633. [CrossRef] [PubMed]
<figure><img src="image_71.png" /></figure>
<figure><img src="image_72.png" /></figure>

Review

# COVID-19, Mucormycosis and Cancer: The Triple Threat—Hypothesis or Reality?

Ishika Mahajan <fnref n="1" /> ID, Aruni Ghose <fnref n="2" />,<fnref n="3" />,<fnref n="4" />,<fnref n="5" /> ID, Deepika Gupta <fnref n="6" />, Manasi Manasvi <fnref n="7" />, Saisha Bhandari <fnref n="8" />, Aparimita Das <fnref n="9" />, Elisabet Sanchez <fnref n="4" /> and Stergios Boussios <fnref n="4" />,<fnref n="10" />,<fnref n="11" />,* ID

**Citation:** Mahajan, I.; Ghose, A.; Gupta, D.; Manasvi, M.; Bhandari, S.; Das, A.; Sanchez, E.; Boussios, S. COVID-19, Mucormycosis and Cancer: The Triple Threat— Hypothesis or Reality? *J. Pers. Med.* **2022**, *12*, 1119. https://doi.org/ 10.3390/jpm12071119

Academic Editor: Anne-Marie Caminade

Received: 16 June 2022

Accepted: 8 July 2022

Published: 10 July 2022

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

<figure><img src="image_73.png" /></figure>

**Copyright:** © 2022 by the authors.
Licensee MDPI, Basel, Switzerland.
This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

**Abstract:** COVID-19 has been responsible for widespread morbidity and mortality worldwide. Invasive mucormycosis has death rates scaling 80%. India, one of the countries hit worst by the pandemic, is also a hotbed with the highest death rates for mucormycosis. Cancer, a ubiquitously present menace, also contributes to higher case fatality rates. All three entities studied here are individual, massive healthcare threats. The danger of one disease predisposing to the other, the poor performance status of patients with all three diseases, the impact of therapeutics for one disease on the pathology and therapy of the others all warrant physicians having a better understanding of the interplay. This is imperative so as to effectively establish control over the individual patient and population health. It is important to understand the interactions to effectively manage all three entities together to reduce overall morbidity. In this review article, we search for an inter-relationship between the COVID-19 pandemic, emerging mucormycosis, and the global giant, cancer.

**Keywords:** cancer; mucormycosis; COVID-19; COVID-associated mucormycosis; diabetes; steroids; immunosuppression

## 1. Introduction

The Severe Acute Respiratory Syndrome Coronavirus-2 (SARS-CoV-2)-induced Coronavirus disease 2019 (COVID-19) pandemic has witnessed more than 545 million illnesses and 6.3 million fatalities worldwide. It affected all national healthcare systems at different levels [1,2]. The southeast Asian belt constitutes the world's second largest COVID-affected area. Second only to the USA, India boasts numbers larger than 43 million cases and 0.52 million deaths [2].

Bacterial and fungal co-infection alongside COVID-19 had been nil or minimal in Middle Eastern Respiratory Syndrome Coronavirus (MERS-CoV) and SARS-CoV-1 [1].

<footnote n="1">¹ Department of Medical Oncology, Apollo Cancer Centre, Chennai 600001, India; ishikaishan16@gmail.com</footnote>

<footnote n="2">² Department of Medical Oncology, Barts Cancer Centre, St. Bartholomew's Hospital, Barts Health NHS Trust, London KT1 2EE, UK; aruni.ghose@nhs.net</footnote>

<footnote n="3">³ Department of Medical Oncology, Mount Vernon Cancer Centre, East and North Hertfordshire NHS Trust, London KT1 2EE, UK</footnote>

<footnote n="4">⁴ Department of Medical Oncology, Medway NHS Foundation Trust, Windmill Road, Gillingham ME7 5NY, UK; elizabet.sanchez@nhs.net</footnote>

<footnote n="5">⁵ Division of Research, Academics and Cancer Control, Saroj Gupta Cancer Centre and Research Institute, Kolkata 700001, India</footnote>

<footnote n="6">⁶ Microbiology, College of Medicine and Sagore Dutta Hospital, Kolkata 700001, India; guptadeepika.587@gmail.com</footnote>

<footnote n="7">⁷ Internal Medicine, Kasturba Medical College, Mangalore 574142, India; manasimanasvi01@gmail.com</footnote>

<footnote n="8">⁸ Internal Medicine, Post Graduate Institute of Medical Education and Research, Chandigarh 133301, India; saisha393@gmail.com</footnote>

<footnote n="9">⁹ Faculty of Allied Health Sciences, Chettinad Academy of Research and Education, Chennai 600001, India; k.aparimita@gmail.com</footnote>

<footnote n="10">¹⁰ Faculty of Life Sciences & Medicine, School of Cancer & Pharmaceutical Sciences, King's College London, London SE1 9RT, UK</footnote>

<footnote n="11">¹¹ AELIA Organization, 9th Km Thessaloniki-Thermi, 57001 Thessaloniki, Greece</footnote>

<footnote>* Correspondence: stergiosboussios@gmail.com or stergios.boussios@nhs.net or stergios.boussios@kcl.ac.uk</footnote>