A population-based study to predict distant metastasis in patients with renal cell carcinoma
Original Article

A population-based study to predict distant metastasis in patients with renal cell carcinoma

Yong Li1, Peng Chen2, Zhi Chen2

1Department of Urology, The Second Affiliated Hospital, University of South China, Hengyang, China; 2Department of Urology, Xiangya Hospital, Central South University, Changsha, China

Contributions: (I) Conception and design: Z Chen, Y Li; (II) Administrative support: Z Chen; (III) Provision of study materials or patients: Z Chen, Y Li; (IV) Collection and assembly of data: Y Li, P Chen; (V) Data analysis and interpretation: Y Li, P Chen; (VI) Manuscript writing: All authors; (VII) Final approval of manuscript: All authors.

Correspondence to: Zhi Chen. Department of Urology, Xiangya Hospital, Central South University, No. 87 Xiangya Road, Changsha 410008, China. Email: czhi1011@126.com.

Background: Nomogram is potentially applied for quantitatively evaluating the probability of distant metastasis. The objective of our research was to establish a nomogram to predict distant metastasis in renal cell carcinoma (RCC) patients.

Methods: We conducted a retrospective analysis on 37,190 RCC cases diagnosed between 2010 and 2015 in the Surveillance Epidemiology and End Results (SEER) database. A multivariate logistic regression model-based nomogram was applied for predicting the risk factors concerning distant metastasis of RCC individuals. The concordance index (C-index) and calibration curves were utilized to internally validate the discrimination of nomogram. Decision curve analysis (DCA) was applied for comparing the predictive performance and clinically practical values between nomogram and conventional clinicopathologic risk factors.

Results: The nomogram incorporated seven clinical variables and achieved a predictive accuracy with a C-index of 0.863. The calibration plots illustrated optimal accordance between model prediction and practical observation. The DCA indicated the nomogram-based clinical utility. Receiver operating characteristic (ROC) curves also demonstrated an area under the curve (AUC) of 0.901 [95% confidence interval (CI): 0.894–0.908] in the training cohort and 0.892 (95% CI: 0.881–0.903) in the testing cohort.

Conclusions: Our proposed novel nomogram potentially serves as an accurate and user-friendly clinical tool to predict occurrence of distant metastases in RCC patients.

Keywords: Distant metastasis; nomogram; renal cell carcinoma (RCC); risk factors; Surveillance Epidemiology and End Results (SEER)


Submitted Dec 12, 2020. Accepted for publication Mar 05, 2021.

doi: 10.21037/apm-20-2481


Introduction

Kidney cancer represents the 7th most prevalent tumor in the developed countries (1,2), accounting for 3.7% of the global cancer burden (3). There were an estimated 63,990 newly diagnosed patients with kidney cancer in USA in 2017 (3). Renal cell carcinoma (RCC), the most frequent histopathological subtype, accounts for approximately 85% of kidney cancer and is one of malignancies derived from renal tubular cells (2,3). RCC is the 6th most commonly diagnosed cancer in men and the 10th in women (4) and the ratio of male to female morbidity is 1.7:1, with a mean age of 64 years (3,5). RCC primarily comprises three histopathologic subtypes, including clear cell RCC (ccRCC) (70%), papillary RCC (pRCC) (15%) and chromophobe RCC (chRCC) (5%) (6-8). The survival of RCC patients has been improving owing to significant progress in the surgical management and the use of novel immunotherapy agents (8,9). Nevertheless, a large proportion of RCC patients present with high-stage disease, with unsatisfactory prognosis because of the occurrence of distantmetastases (3,10).

Although the majority of detected renal neoplasms are small lesions, up to 17% of RCC patients are initially diagnosed with distant metastases (11-13). One-third of the localized RCC patients are treated by curative surgery and demonstrate a relapse at distant locations (10). In a retrospective study of 338 pathologic T1a (pT1a) RCC patients, 9 patients (2.7%) were reported with metastatic lesions (14). The regional lymph nodes are potentially the most common metastatic site, followed by the lung, brain, bone and soft tissue (15,16). Among metastatic RCC cases, approximately 75% demonstrate three or more metastatic locations (17). Distant metastasis adversely impacts the prognosis of the patients (18). The 5-year relative survivals of RCC of TNM stages I and II are 81% and 74%, respectively, and the overall 5-year survival of RCCs stage III and IV sharply diminishes to 53% and 10%, respectively, mostly due to distant metastases (19-21). Therefore, it is crucial to elucidate the clinicopathological risk factors promoting distant metastatic disease in RCC.

Nomograms represent user-friendly graphical mathematical models that generate a single numerical estimate of a clinical event of interest, thus helping physicians predict occurrence of a given event based on specific clinical and pathological variables (22-24). A growing number of nomogram models have been developed to improve clinical decision making during, such as hepatocellular carcinoma (23), breast carcinoma (25) and lung carcinoma (26) as well as colon carcinoma (27). To our knowledge, a promising nomogram model has not been established for predicting distant metastasis of RCC patients, using the Surveillance Epidemiology and End Results (SEER) database. Herein, we sought to design a robust nomogram by incorporating clinicopathological parameters extracted from the SEER database, to predicting presence of distant metastases in RCC patients that would be potentially clinically useful (23).

We present the following article in accordance with the TRIPOD Checklist (available at http://dx.doi.org/10.21037/apm-20-2481).


Methods

Data source

Original clinical data were extracted from the SEER public national database that incorporates 18 population-based tumor registries and covers approximately 28% of America population (http://seer.cancer.gov/) (23,28). The study was conducted in accordance with the Declaration of Helsinki (as revised in 2013). The clinical data in the SEER database, including patient demographics, tumor characteristics and survival status, are publicly accessible and anonymized. Therefore, our study did not require a separate ethical review or the patient consent (23). The study methods also conformed to the rules and regulations of this database.

Patient data extraction

The data of RCC patients diagnosed between 2010 and 2015 were identified from the SEER database through the SEER*Stat 8.3.2 software. A total of 37,190 eligible RCC individuals that have undergone a cancer-directed surgery were retrieved, and were further divided into a training set (n=24,793) and a testing set (n=12,397). Using the International Classification of Diseases for Oncology, third edition (ICD-O-3) categories, we searched the RCC-associated codes 8310/3, 8312/3, 8317/3 and 8318/3. The collected data included patient demographics (age at diagnosis, sex, ethnicity), tumor characteristics (laterality, pathological differentiation, T stage, lymph node status, metastasis status, TNM stage, tumor size, histological type, status of invasion beyond the capsule, Fuhrman nuclear grade), surgery type and survival status. The exclusion criteria included: (I) the age of patient at diagnosis less than 18 years; (II) individuals lacking sufficient information regarding the pathological differentiation, TNM stage, distant metastasis, cause of death and complete survival data. The flow diagram describing the patient selection is shown in Figure 1.

Figure 1 The flow chart of patient selection.

Nomogram construction and validation

The univariate and multivariable Cox regression analyses were utilized to investigate significantly independent risk factors associated with distant metastasis in RCC patients. The significant parameters identified on the multivariate analysis were incorporated into a nomogram to predicting distant metastasis. The predictive reliability and accuracy of the nomogram was evaluated through the SEER internal testing group. A bootstrapping method with 2,000 resamples was used to internally validate the nomogram. The receiver operating characteristic (ROC) and calibration curves were applied to measure the predictive efficiency and reliability of the nomogram.

Clinical utility

Decision curve analysis (DCA) represents a novel algorithm that was used to evaluate the clinical application value of the predictive nomogram, through estimating the net benefits at every risk threshold possibility (29). Based on the DCA, the curves were formulated to estimate the clinical effect and to identify the number of high-risk RCC subjects with distant metastasis under the condition of different threshold probabilities, thus assisting physicians to more visually conclude its significant performance (30). The most desirable model demonstrated greater net benefits throughout a range of threshold probabilities (31). DCA was applied to compare the clinical benefit of the nomogram model versus the other clinicopathological factors in the training set.

Statistical analysis

The R software version 3.3.4 was applied for our statistical analyses. We compared classification variables and continuous variables using the Chi-squared test and Student’s t-test (22). The nomogram was constructed and validated using the “rms” R package. The “survivalROC” R package was used to formulate ROC curve. The “rmda” R package was used to establish DCA (22). On the basis of the maximum Youden index in the training group, we estimated the cutoff values of the risk scores in the predictive nomogram. All RCC individuals were stratified into low- and high-risk sets. Statistically significant result was considered when P value was less than 0.05.

Ethical statement

The study was conducted in accordance with the Declaration of Helsinki (as revised in 2013). All information from the SEER program is available and free for public, so the agreement of the medical ethics committee board was not necessary.


Results

Clinical features

The clinicopathological features of the RCC cases are shown in Table 1. No statistically significant differences were found between the training set and testing set (P>0.05). Median patient age was 59 years and a large proportion of the patients were male (62.8%) in both cohorts. The ccRCC subtype was the most frequent histologic type in both cohorts. The rates of distant metastasis were 6.17% (1,530/24,793) and 6.04% (749/12,397) in the training and testing sets, respectively. Notably, patients administrated with nephrectomy displayed a higher rate (58.8%), compared with those performed by local excision (40.6%). In the correlative analysis (Table 2), 11 parameters were significantly associated with distant metastasis in both sets (P<0.001), including gender, pathological differentiation, T stage, lymph node status, TNM stage, tumor size, histological type, invasion beyond the capsule, Fuhrman nuclear grade, surgery type, overall survival (OS).

Table 1
Table 1 Clinical characteristics of RCC cases
Full table
Table 2
Table 2 Correlations between distant metastasis and clinicopathologic factors of RCC patients in the two sets
Full table

Predictive parameters associated with distant metastasis

The following parameters were associated with the occurrence of distant metastasis on univariate regression analysis: gender, race, pathological differentiation, T stage, lymph node status, tumor size, histological type, status of invasion beyond the capsule, Fuhrman nuclear grade and surgery type (P<0.05) (Table 3). These parameters were incorporated into a multivariate Cox proportional hazard regression model (Table 4). Seven parameters were considered as significantly independent risk factors associated with distant metastasis. Compared with the ccRCC subtype, papillary subtype [odds ratio (OR) =0.55, 95% confidence interval (CI): 0.42–0.70, P<0.001] and chromophobe subtype (OR =0.18, 95% CI: 0.10–0.30, P<0.001) had a lower rate of distant metastasis. The status of lymph node involvement N1 (OR =5.47, 95% CI: 4.56–6.57, P<0.001), larger tumor size (>30 mm, OR =2.49, 95% CI: 1.82–3.49, P<0.001) and the invasion beyond the capsule (OR =1.48, 95% CI: 1.25–1.75, P<0.001) were related to the presence of distant metastasis. Patients undergoing radical nephrectomy were associated with higher risk of distant metastasis than patients undergoing local excision (OR =4.17, 95% CI: 3.15–5.61, P<0.001). Notably, RCC patients with more advanced T stage had a higher rate of distant metastasis (T2: OR =2.85, 95% CI: 2.30–3.54; T3: OR =3.96, 95% CI: 3.21–4.89; T4: OR =11.94, 95% CI: 8.56–16.68; P<0.001 for all). Regarding pathological differentiation grade, compared with well differentiated RCC, poorly differentiated RCC (OR =2.76, 95% CI: 1.17–7.43, P=0.03) and undifferentiated RCC (OR =4.06, 95% CI: 1.69–11.16, P=0.003) were characterized with a higher possibility of distant metastasis.

Table 3
Table 3 Risk factors for distant metastasis in RCC patients through univariate analysis in the training group
Full table
Table 4
Table 4 Risk factors for distant metastasis in RCC patients on multivariate analysis in the training set
Full table

Construction and validation of predictive nomogram for distant metastasis

A nomogram model was constructed by incorporating seven significant risk factors identified on multivariate Cox regression analysis based on the training set (Figure 2A). Additionally, each parameter had a corresponding score in the nomogram. As revealed in Table S1, we estimated the point distributions and predictive scores for every parameter in our nomogram. As shown in Figure 2A, T stage made the largest contribution to the presence of distant metastasis, followed by pathological differentiation, histological type, the status of lymph node involvement, surgery type, tumor size and the status of invasion beyond the capsule in descending order. The discriminatory capacity of our nomogram model was satisfactory, and demonstrated a concordance index (C-index) of 0.863. The calibration plots to predict distant metastasis in the training group (Figure 2B) and in the testing group (Figure 2C) demonstrated the desirable concordance between model predictions and the actual observations.

Figure 2 Nomogram and calibration curves for predicting distant metastasis in patients with RCC. (A) Predictive nomogram for the presence of distant metastasis in RCC patients. (B,C) Calibration plots for evaluating the predictive efficiency of nomogram in (B) training and (C) testing group, respectively. RCC, renal cell carcinoma.

Additionally, we performed a comparison of the predictive nomogram with the conventional clinicopathologic risk variables through ROC analysis. The AUCs of the predictive nomogram were 0.901 (95% CI: 0.894–0.908) in the training set (Figure 3A) and 0.892 (95% CI: 0.881–0.903) in the testing set (Figure 3B), respectively, which were dramatically greater than the other clinical factors (such as pathological differentiation, histological type, lymph node status, invasion beyond the capsule and Fuhrman nuclear grade). These results highlighted the fact that the predictive nomogram achieved remarkable superiority over these additional risk factors.

Figure 3 ROC curves for comparing the predictive value between the distant metastasis nomogram and clinicopathological risk factors in the (A) training and (B) testing cohort, respectively. ROC, receiver operating characteristic; AUC, area under the curve; 95% CI, 95% confidence interval.

In accordance with the nomogram risk score, bar diagrams were formulated to assess the discriminatory performance. We further calculated the cutoff value of 238 for the distant metastasis nomogram via the maximum Youden index. All RCC individuals were stratified into low- and high-risk subgroups. Indeed, the RCC patients in the high-risk group exhibited a higher probability of distant metastasis than did those in the low-risk group in the training and testing cohorts (Figure 4A,B).

Figure 4 Risk classification and Kaplan-Meier survival curves for RCC patients. (A,B) The bar graphs for evaluating the discriminatory performance of the nomogram for distant metastasis in the training (A) and testing (B) cohort, respectively. (C,D) Survival plots for evaluating the OS of RCC patients with distant metastasis in the training (C) and testing (D) group, respectively. OS, overall survival; RCC, renal cell carcinoma.

Clinical application of the nomogram

RCC patients in the high-risk group were characterized by significantly shorter OS, compared with those in the low-risk group in the two cohorts (P<0.0001) (Figure 4C,D). DCAs further demonstrated that the clinical utility and the benefits of the nomogram were the superior to the conventional clinicopathological variables (including pathological differentiation, histological type, lymph node status, invasion beyond the capsule, and Fuhrman nuclear grade), both in the training cohort (Figure 5A) and in the validation cohort (Figure 5B). The threshold probabilities of 0–0.4 were the most beneficial in predicting distant metastasis using the nomogram. The clinical impact plots of the nomogram in the training set (Figure 5C) and in the testing set (Figure 5D) demonstrated that the model displayed a desirable predictive performance.

Figure 5 The decision curves and clinical impact curves of the nomogram for predicting distant metastasis for RCC patients. (A,B) DCAs of nomogram and additional clinical risk factors in the training (A) and testing (B) cohort, respectively. (C,D) Clinical impact curves of the predictive nomogram in the training (C) and testing (D) cohort. DCA, decision curve analysis; RCC, renal cell carcinoma.

Discussion

Distant metastatic disease is a conventional symbol of advanced stage and indicates unfavorable prognosis for RCC patients (32). It has been demonstrated that cancer-targeted surgical intervention provides an enhanced survival in RCC patients with metastatic disease (33,34). Immune checkpoint inhibitors (ICIs) and multitarget tyrosine kinase inhibitors (TKIs) such as cabozantinib can be used as the second-line drugs to improve prognosis (35,36). Therefore, timely and accurate detection of distant metastasis is crucial, and may guide an optimal treatment decision. Computed tomography (CT) remains the first-line method of examination of suspected metastatic disease with a good staging accuracy. Nevertheless, there are limitations of CT examination in evaluating the nodal status (37,38). It is important to predict the risk of distant metastasis of RCC patients using a noninvasive method. In this report, we aimed to generate a predictive nomogram model associated with distant metastasis, utilizing the data from the SEER database. The nomogram demonstrated a desirable discrimination and can be used as a clinical tool for individualized evaluation to predict the occurrence of distant metastasis in RCC patients.

Our predictive nomogram model associated with distant RCC metastasis incorporated seven clinicopathological risk factors: T stage, pathological differentiation, histological type, lymph node status, surgery type, tumor size and the status of invasion beyond the capsule. Hutterer et al. collected data from 2,660 RCC patients from 11 centers and developed a nomogram to predict the risk of distant metastases. They found that there were 10.1% and 10.5% patients with distant metastases in the initial group and in the external validation group, respectively, which was higher than in our data set with 6.17% and 6.04% in the training and testing groups, respectively (39). Multiple studies have also revealed that the tumor size was indeed significantly correlated with the presence of distant metastasis in RCC, while age was not an independent predictor, consistent with the findings in our study (39-41). They also demonstrated that the occurrence of metastatic lesions in RCC enhanced with increasing tumor size. Specifically, there was a statistically significant correlation between metastatic risk and the tumor size of less than 5 cm. In the current study, we found that renal neoplasms ≥3 cm in size had a higher risk of distant metastasis (42). When RCC was 23 mm (mean size), the proportion of metastatic tumor was approximately 2.0% (43). As shown in the current study, histologic RCC subtype had a crucial impact on the metastatic potential of tumors. Indeed, another study also found that ccRCC harbored the highest metastatic risk, followed by pRCC and chRCC (40). Additionally, the influence of regional lymph node involvement on the distant metastatic potential of RCC was also demonstrated in our finding. Another study found that regional lymph node involvement increased the occurrence of distant metastasis by 50% (44). Our nomogram highlighted the fact that the rate of distant metastasis increased with the RCC differentiation (from well to poorly differentiated). Other reports similarly showed that poor pathological differentiation of RCC was correlated with inferior prognosis (45-47). This is not surprising, as the pathological differentiation grade reflects the tumor biological behavior and represents an independent predictor for distant RCC metastasis.

In our report, we incorporated a large number of cases from the SEER database, arguably the best available source of clinical data. All cases were randomly stratified into a training and internal testing set to establish and validate a predictive nomogram, which makes the conclusion convincing. All variables included in our nomogram were readily available and prevalent in clinical practice, contributing to convenient evaluation of risk factors for physicians. Nevertheless, certain shortcomings should be acknowledged in our study. First, our report could not incorporate some potentially important indicators, such as BMI, family history of hypertension, smoking and drinking, genetic markers, vascular invasion, surgical margin status and the immunotherapeutic protocols, which were not available in the SEER database (23,48-50). Second, the training and validation groups in our study were both derived from the SEER database, and were not externally validated in an independent dataset or our hospital (51). Third, our study may have had some inherent bias because of the use of retrospective information from the SEER database. Prospective clinical trials with a large sample size are thus required to verify our predictive model in the general population (23,31).


Conclusions

In conclusion, a novel nomogram to predict distant metastasis in RCC patients was established, using a large population cohort from the SEER database. This tool may potentially help both physicians and RCC patients in make risk assessment of metastatic disease.


Acknowledgments

The authors of this study have no contribution to SEER data collection. We would like to thank the SEER database for its open access. We would like to thank Weiping Xia for his help in polishing the language of our paper.

Funding: This work was supported by Hunan Provincial Natural Science Foundation of China (No. 2020JJ4870).


Footnote

Reporting Checklist: The authors have completed the TRIPOD Checklist. Available at http://dx.doi.org/10.21037/apm-20-2481

Conflicts of Interest: All authors have completed the ICMJE uniform disclosure form (available at http://dx.doi.org/10.21037/apm-20-2481). The authors have no conflicts of interest to declare.

Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. The study was conducted in accordance with the Declaration of Helsinki (as revised in 2013). All information from the SEER program is available and free for public, so the agreement of the medical ethics committee board was not necessary.

Open Access Statement: This is an Open Access article distributed in accordance with the Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International License (CC BY-NC-ND 4.0), which permits the non-commercial replication and distribution of the article with the strict proviso that no changes or edits are made and the original work is properly cited (including links to both the formal publication through the relevant DOI and the license). See: https://creativecommons.org/licenses/by-nc-nd/4.0/.


References

  1. Ferlay J, Soerjomataram I, Dikshit R, et al. Cancer incidence and mortality worldwide: sources, methods and major patterns in GLOBOCAN 2012. Int J Cancer 2015;136:E359-86. [Crossref] [PubMed]
  2. Zhang Z, Tang H, Chen P, et al. Demystifying the manipulation of host immunity, metabolism, and extraintestinal tumors by the gut microbiome. Signal Transduct Target Ther 2019;4:41. [Crossref] [PubMed]
  3. Siegel RL, Miller KD, Jemal A. Cancer Statistics, 2017. CA Cancer J Clin 2017;67:7-30. [Crossref] [PubMed]
  4. Siegel RL, Miller KD, Jemal A. Cancer statistics, 2018. CA Cancer J Clin 2018;68:7-30. [Crossref] [PubMed]
  5. Choueiri TK, Motzer RJ. Systemic Therapy for Metastatic Renal-Cell Carcinoma. N Engl J Med 2017;376:354-66. [Crossref] [PubMed]
  6. Mai KT, Landry DC, Robertson SJ, et al. A comparative study of metastatic renal cell carcinoma with correlation to subtype and primary tumor. Pathol Res Pract 2001;197:671-5. [Crossref] [PubMed]
  7. Shanbhogue AK, Vikram R, Paspulati RM, et al. Rare (<1%) histological subtypes of renal cell carcinoma: an update. Abdom Imaging 2012;37:861-72. [Crossref] [PubMed]
  8. Liu S, Lai W, Shi Y, et al. Annotation and cluster analysis of long noncoding RNA linked to male sex and estrogen in cancers. NPJ Precis Oncol 2020;4:5. [Crossref]
  9. Jonasch E, Gao J, Rathmell WK. Renal cell carcinoma. BMJ 2014;349:g4797. [Crossref] [PubMed]
  10. Barata PC, Rini BI. Treatment of renal cell carcinoma: Current status and future directions. CA Cancer J Clin 2017;67:507-24. [Crossref] [PubMed]
  11. Capitanio U, Montorsi F. Renal cancer. Lancet 2016;387:894-906. [Crossref] [PubMed]
  12. Wang Z, Zhang B, Chen Z, et al. The long noncoding RNA myocardial infarction-associated transcript modulates the epithelial-mesenchymal transition in renal interstitial fibrosis. Life Sci 2020;241:117187 [Crossref] [PubMed]
  13. Wang Z, Chen Z, Li B, et al. Curcumin attenuates renal interstitial fibrosis of obstructive nephropathy by suppressing epithelial-mesenchymal transition through inhibition of the TLR4/NF-кB and PI3K/AKT signalling pathways. Pharm Biol 2020;58:828-37. [Crossref] [PubMed]
  14. Suzuki K, Mizuno R, Mikami S, et al. Prognostic significance of high nuclear grade in patients with pathologic T1a renal cell carcinoma. Jpn J Clin Oncol 2012;42:831-5. [Crossref] [PubMed]
  15. Kavolius JP, Mastorakos DP, Pavlovich C, et al. Resection of metastatic renal cell carcinoma. J Clin Oncol 1998;16:2261-6. [Crossref] [PubMed]
  16. Hoffmann NE, Gillett MD, Cheville JC, et al. Differences in organ system of distant metastasis by renal cell carcinoma subtype. J Urol 2008;179:474-7. [Crossref] [PubMed]
  17. Alt AL, Boorjian SA, Lohse CM, et al. Survival after complete surgical resection of multiple metastases from renal cell carcinoma. Cancer 2011;117:2873-82. [Crossref] [PubMed]
  18. Bruno JJ 2nd, Snyder ME, Motzer RJ, et al. Renal cell carcinoma local recurrences: impact of surgical treatment and concomitant metastasis on survival. BJU Int 2006;97:933-8. [Crossref] [PubMed]
  19. Porta C, Cosmai L, Leibovich BC, et al. The adjuvant treatment of kidney cancer: a multidisciplinary outlook. Nat Rev Nephrol 2019;15:423-33. [Crossref] [PubMed]
  20. Yoshiyama A, Morii T, Susa M, et al. Preoperative evaluation of renal cell carcinoma patients with bone metastases on risks for blood loss, performance status and lethal event. J Orthop Sci 2017;22:924-30. [Crossref] [PubMed]
  21. Abdel-Rahman O. Clinical correlates and prognostic value of different metastatic sites in metastatic renal cell carcinoma. Future Oncol 2017;13:1967-80. [Crossref] [PubMed]
  22. Yan Y, Liu H, Mao K, et al. Novel nomograms to predict lymph node metastasis and liver metastasis in patients with early colon carcinoma. J Transl Med 2019;17:193. [Crossref] [PubMed]
  23. Zhang Z, Xie H, Chen P, et al. Development and Identification of a Nomogram Prognostic Model for Patients with Primary Clear Cell Carcinoma of the Liver. Med Sci Monit 2020;26:e919789 [Crossref] [PubMed]
  24. Balachandran VP, Gonen M, Smith JJ, et al. Nomograms in oncology: more than meets the eye. Lancet Oncol 2015;16:e173-80. [Crossref] [PubMed]
  25. van den Hoven I, van Klaveren D, Verheuvel NC, et al. Predicting the extent of nodal involvement for node positive breast cancer patients: Development and validation of a novel tool. J Surg Oncol 2019;120:578-86. [Crossref] [PubMed]
  26. Zhang Y, Zheng D, Xie J, et al. Development and Validation of Web-Based Nomograms to Precisely Predict Conditional Risk of Site-Specific Recurrence for Patients With Completely Resected Non-small Cell Lung Cancer: A Multiinstitutional Study. Chest 2018;154:501-11. [Crossref] [PubMed]
  27. Hu T, Wang S, Huang L, et al. A clinical-radiomics nomogram for the preoperative prediction of lung metastasis in colorectal cancer patients with indeterminate pulmonary nodules. Eur Radiol 2019;29:439-49. [Crossref] [PubMed]
  28. Cronin KA, Ries LA, Edwards BK. The Surveillance, Epidemiology, and End Results (SEER) Program of the National Cancer Institute. Cancer 2014;120:3755-7. [Crossref] [PubMed]
  29. Vickers AJ, Cronin AM, Elkin EB, et al. Extensions to decision curve analysis, a novel method for evaluating diagnostic tests, prediction models and molecular markers. BMC Med Inform Decis Mak 2008;8:53. [Crossref] [PubMed]
  30. Kerr KF, Brown MD, Zhu K, et al. Assessing the Clinical Impact of Risk Prediction Models with Decision Curves: Guidance for Correct Interpretation and Appropriate Use. J Clin Oncol 2016;34:2534-40. [Crossref] [PubMed]
  31. Zhou W, Huang C, Yuan N. Prognostic nomograms based on log odds of positive lymph nodes for patients with renal cell carcinoma: A retrospective cohort study. Int J Surg 2018;60:28-40. [Crossref] [PubMed]
  32. Mitsui Y, Shiina H, Kato T, et al. Versican Promotes Tumor Progression, Metastasis and Predicts Poor Prognosis in Renal Carcinoma. Mol Cancer Res 2017;15:884-95. [Crossref] [PubMed]
  33. Walther MM, Alexander RB, Weiss GH, et al. Cytoreductive surgery prior to interleukin-2-based therapy in patients with metastatic renal cell carcinoma. Urology 1993;42:250-7; discussion 257-8. [Crossref] [PubMed]
  34. Franklin JR, Figlin R, Rauch J, et al. Cytoreductive surgery in the management of metastatic renal cell carcinoma: the UCLA experience. Semin Urol Oncol 1996;14:230-6. [PubMed]
  35. Choueiri TK, Escudier B, Powles T, et al. Cabozantinib versus everolimus in advanced renal cell carcinoma (METEOR): final results from a randomised, open-label, phase 3 trial. Lancet Oncol 2016;17:917-27. [Crossref] [PubMed]
  36. Motzer RJ, Escudier B, McDermott DF, et al. Nivolumab versus Everolimus in Advanced Renal-Cell Carcinoma. N Engl J Med 2015;373:1803-13. [Crossref] [PubMed]
  37. Capitanio U, Bensalah K, Bex A, et al. Epidemiology of Renal Cell Carcinoma. Eur Urol 2019;75:74-84. [Crossref] [PubMed]
  38. Diaz de Leon A, Pedrosa I. Imaging and Screening of Kidney Cancer. Radiol Clin North Am 2017;55:1235-50. [Crossref] [PubMed]
  39. Hutterer GC, Patard JJ, Jeldres C, et al. Patients with distant metastases from renal cell carcinoma can be accurately identified: external validation of a new nomogram. BJU Int 2008;101:39-43. [PubMed]
  40. Daugherty M, Sedaghatpour D, Shapiro O, et al. The metastatic potential of renal tumors: Influence of histologic subtypes on definition of small renal masses, risk stratification, and future active surveillance protocols. Urol Oncol 2017;35:153.e15-153.e20. [Crossref] [PubMed]
  41. Tirumani SH, Souza D, Krajewski KM, et al. Impact of histologic subtype and sarcomatoid transformation on metastasis in renal cell carcinoma: a single institute experience in 149 patients. Abdom Radiol (NY) 2016;41:295-302. [Crossref] [PubMed]
  42. Zastrow S, Phuong A, von Bar I, et al. Primary tumor size in renal cell cancer in relation to the occurrence of synchronous metastatic disease. Urol Int 2014;92:462-7. [Crossref] [PubMed]
  43. Smaldone MC, Kutikov A, Egleston BL, et al. Small renal masses progressing to metastases under active surveillance: a systematic review and pooled analysis. Cancer 2012;118:997-1006. [Crossref] [PubMed]
  44. Tadayoni A, Paschall AK, Malayeri AA. Assessing lymph node status in patients with kidney cancer. Transl Androl Urol 2018;7:766-73. [Crossref] [PubMed]
  45. Pichler R, Compérat E, Klatte T, et al. Renal Cell Carcinoma with Sarcomatoid Features: Finally New Therapeutic Hope? Cancers (Basel) 2019;11:422. [Crossref] [PubMed]
  46. Porfyris O, Alexandrou P, Masaoutis C, et al. Ovarian metastasis of renal cell carcinoma: Clinical and pathological presentation of a case. Turk J Urol 2018;45:150-3. [Crossref] [PubMed]
  47. Truong LD, Shen SS. Immunohistochemical diagnosis of renal neoplasms. Arch Pathol Lab Med 2011;135:92-109. [Crossref] [PubMed]
  48. Crona DJ, Skol AD, Leppanen VM, et al. Genetic Variants of VEGFA and FLT4 Are Determinants of Survival in Renal Cell Carcinoma Patients Treated with Sorafenib. Cancer Res 2019;79:231-41. [Crossref] [PubMed]
  49. Sanchez-Gastaldo A, Kempf E, Gonzalez Del Alba A, et al. Systemic treatment of renal cell cancer: A comprehensive review. Cancer Treat Rev 2017;60:77-89. [Crossref] [PubMed]
  50. Bao Y, Wang Z, Liu B, et al. A feed-forward loop between nuclear translocation of CXCR4 and HIF-1alpha promotes renal cell carcinoma metastasis. Oncogene 2019;38:881-95. [Crossref] [PubMed]
  51. He C, Zhong L, Zhang Y, et al. Development and validation of a nomogram to predict liver metastasis in patients with pancreatic ductal adenocarcinoma: a large cohort study. Cancer Manag Res 2019;11:3981-91. [Crossref] [PubMed]
Cite this article as: Li Y, Chen P, Chen Z. A population-based study to predict distant metastasis in patients with renal cell carcinoma. Ann Palliat Med 2021;10(4):4273-4288. doi: 10.21037/apm-20-2481