Skip to main content
Download PDF

Meta-analysis of prediction models for predicting lymph node metastasis in thyroid cancer

World Journal of Surgical Oncology volume 22, Article number: 278 (2024) Cite this article

Abstract

Background

The purpose of this systematic review and meta-analysis is to assess the efficacy of various machine learning (ML) techniques in predicting preoperative lymph node metastasis (LNM) in patients diagnosed with papillary thyroid carcinoma (PTC). Although prior studies have investigated the potential of ML in this context, the current evidence is not sufficiently strong. Hence, we undertook a thorough analysis to ascertain the predictive accuracy of different ML models and their practical relevance in predicting preoperative LNM in PTC patients.

Materials and methods

In our search, we thoroughly examined PubMed, Cochrane Library, Embase, and Web of Science, encompassing their complete database history until December 3rd, 2022. To evaluate the potential bias in the machine learning models documented in the included studies, we employed the Prediction Model Risk of Bias Assessment Tool (PROBAST).

Results

A total of 107 studies, involving 136,245 patients, were included. Among them, 21,231 patients showed central LNM (CLNM) and 4,637 had lateral LNM (LLNM). The meta-analysis results revealed that the c-index for predicting LNM, CLNM, and LLNM were 0.762 (95% CI: 0.747–0.777), 0.762 (95% CI: 0.747–0.777), and 0.803 (95% CI: 0.773–0.834) in the training set, and 0.773 (95% CI: 0.754–0.791), 0.762 (95% CI: 0.747–0.777), and 0.829 (95% CI: 0.779–0.879) in the validation set, respectively. A total of 134 machine learning-based prediction models were included, covering 10 different types. Logistic Regression (LR) was the most commonly used model, accounting for 81.34% (109/134) of the included models.

Conclusions

Machine learning methods have shown a certain level of accuracy in predicting preoperative LNM in PTC patients, indicating their potential as a predictive tool. Their use in the clinical management of PTC holds great promise. Among the various ML models investigated, the performance of logistic regression-based nomograms was deemed satisfactory.

Introduction

Over the past two decades, there has been a significant surge in the incidence of thyroid cancer. According to the 2020 Global Cancer Statistics, thyroid cancer is now ranked as the 9th most common cancer worldwide. With over 586,000 new cases diagnosed annually, its morbidity has been growing at a rate of approximately 6% per year [1,2,3,4]. This sharp increase in thyroid cancer poses a substantial challenge, especially for countries with high population densities. The latest data from the Chinese National Cancer Center reveals that thyroid cancer is the 7th most prevalent cancer in China and the 3rd most prevalent cancer in women [5]. Among all malignancies, the morbidity of thyroid cancer currently exhibits the highest growth rate. The predominant type of thyroid cancer is papillary thyroid carcinoma (PTC), accounting for approximately 85-90% of the cases [6,7,8,9].

The metastasis of papillary thyroid carcinoma (PTC) to the cervical lymph nodes is typically categorized into central lymph node metastasis (CLNM) and lateral lymph node metastasis (LLNM). CLNM is the more frequent site of metastasis, with an incidence of lymph node metastasis (LNM) ranging from 30 to 90% in all PTC patients [10,11,12]. The accurate prediction of LNM before surgery allows clinicians to effectively plan and select treatment strategies for PTC patients, determining the extent of surgical resection and serving as a crucial prognostic factor [13]. Ultrasound has been the primary method for predicting LNM in preoperative PTC patients. However, it has shown unsatisfactory accuracy in detecting CLNM, with detection rates ranging from 36.7 to 48.7% [14, 15]. The impact of LNM on the prognosis and mortality of PTC patients remains a topic of debate. Some studies have shown no association between LNM and overall survival in PTC patients [16]. However, a study by Zaydfudim et al. revealed that LNM is associated with increased mortality, particularly in patients over the age of 45 [17]. Other studies have suggested that LNM can be a risk factor for local recurrence and distant metastasis. PTC patients with LNM have a higher likelihood of experiencing local recurrence, resulting in shortened survival times [18,19,20]. Despite conflicting opinions, there are recommendations for prophylactic central lymph node dissection (PCLND) by some experts to ensure comprehensive treatment for PTC patients [21, 22]. PCLND can reduce the local recurrence rate and provide more accurate staging for PTC [22,23,24]. The American Thyroid Association guidelines (ATA) also recommend PCLND for PTC patients, especially those with advanced-stage PTC [25]. An expert consensus from the European Society of Endocrine Surgeons has similarly recommended performing PCLND on PTC patients with specific characteristics, such as T3 or T4 stage, age over 45 years or under 15 years, male patients, bilateral or multifocal tumors [26]. However, there are concerns about potential complications of this procedure, such as hypoparathyroidism and recurrent laryngeal nerve injury, leading to a recommendation against routinely performing PCLND [27, 28].

Arthur Samuel is credited with introducing the concept of machine learning (ML) in the 1950s. Since then, ML has been widely applied across various domains [29]. In recent years, as artificial intelligence has gained prominence in the medical field, ML has found extensive applications in disease diagnosis, prognosis, and recurrence prediction [30,31,32]. Notably, ML models have been employed to forecast preoperative lymph node metastasis (LNM) in patients with papillary thyroid carcinoma (PTC). Despite the construction of various prediction models, there is significant heterogeneity among them due to the incorporation of diverse modeling methods and variables. Consequently, the predictive value of these models remains a topic of controversy. Currently, no systematic review has comprehensively summarized and compared the predictive performance and clinical utility of these models. Therefore, a systematic review and meta-analysis have been conducted to evaluate whether ML-based prediction models yield superior results in predicting preoperative LNM of PTC. Additionally, this study aims to provide a comprehensive summary of evidence-based medicine in order to lay the groundwork for the future development of mature, straightforward, and visualized intelligent tools, akin to the nomogram developed by the Memorial Sloan Kettering Cancer Center for predicting LNM in breast and prostate cancer [33,34,35].

Materials and methods

The systematic review and meta-analysis followed the guidelines outlined in the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) Statement. The study protocol was registered in PROSPERO with registration No. CRD42022376082.

Inclusion and exclusion criteria

Inclusive criteria

  1. (1)

    Participants: Individuals diagnosed with PTC.

  2. (2)

    Types of Studies: This study includes different research designs, such as case-control studies, cohort studies, nested case-control studies, and case-cohort studies.

  3. (3)

    An extensive machine learning model was developed to predict the presence of LNM. Furthermore, studies that have built separate machine learning models using the same dataset will be included, regardless of whether external validation was conducted.

  4. (4)

    The literature used in this analysis specifically emphasizes publications written in the English language.

Exclusion criteria

  1. (1)

    Irrelevant study types such as meta-analysis, review, guideline, and expert comments are excluded.

  2. (2)

    The focus is solely on identifying risk factors without using a ML-based model.

  3. (3)

    Essential outcome indicators such as ROC, c-statistic, c-index, sensitivity, specificity, accuracy, recovery, confusion matrix, diagnostic four grid table, F1 score, and calibration curve are not available.

  4. (4)

    Studies with fewer than 50 cases are deemed insufficient.

  5. (5)

    The studies are specifically aimed at predicting the accuracy of individual factors.

Search strategy

We conducted a systematic search of 4 electronic databases, including PubMed, Cochrane, Embase, and Web of Science, from the commencement of the databases up to December 3, 2022. The search items included Medical Subject Headings (MeSH) and free-text terms, such as “Thyroid Cancer, Papillary,” “machine learning,” and “prediction model.” Additional file 1 contains the detailed search strategies.

Study selection and data extraction

The articles collected were imported into the Endnote software. Following a duplicate check, the titles and abstracts of these articles were reviewed to exclude any irrelevant studies. Subsequently, the full texts of the remaining articles were downloaded and thoroughly studied. Only the studies that satisfied the pre-defined inclusion criteria were ultimately incorporated.

The data included in the studies was obtained using a pre-designed spreadsheet. It encompasses the title, first author, publication date, author nationality, study type, patient source, criteria for diagnosing LNM, LNM sites, number of LNM samples, total number of submitted samples, number of LNM samples in the training set, total number of submitted samples in the training set, validation set generation method, method for addressing over-fitting, number of LNM samples in the validation set, total number of submitted samples in the validation set, method for processing missing values, method for variable screening/feature selection, type of model constructed, and variables used for modeling.

Two thyroid cancer researchers, LLF, a doctor with 8 years of clinical experience, and CYZ, a doctor with 5 years of clinical experience, independently carried out the study selection and data extraction. They will cross-check their findings upon completion. In case of any disagreements, a third researcher, HF, who has 20 years of clinical experience in thyroid cancer and expertise in machine learning, will independently conduct the extraction. The final decision will be reached through discussion.

Risk of bias assessment

The studies’ risk of bias was assessed using a comprehensive evaluation tool called PROBAST, which covers four domains: participants, predictive variables, results, and statistical analysis [36, 37]. Each domain includes a specific set of questions. The participants’ domain comprises 2 questions, the predictive variables’ domain includes 3 questions, the results’ domain contains 6 questions, and the statistical analysis’ domain consists of 9 questions. For each question, the response options are “yes/maybe yes,” “no/maybe not,” or “no information.” A domain is categorized as having a “high risk” of bias if any of its questions are answered with “no/maybe not” or “no information.” Conversely, a domain is classified as having a “low risk” of bias if all of its questions are answered with “yes/maybe yes.” If all four domains are graded as low risk, the overall risk of bias in the study is considered low. However, if more than one domain is graded as high risk, the overall risk of bias is considered “high.”

Two experienced doctors in the field of thyroid cancer, YJC and GZ, will independently conduct a risk of bias assessment and cross-check once they have completed their assessments. YJC has 10 years of clinical experience, while GZ has 6 years. If any disputes arise, a third researcher, LF, who has 10 years of clinical experience in thyroid cancer and expertise in meta-analysis or systematic reviews, will conduct an independent bias risk assessment. Ultimately, the final decision will be made through a discussion among all three researchers.

Data synthesis and statistical analysis

We assessed the prediction model’s performance using multiple outcome measures. The primary measure utilized was the c-index, which offers a comprehensive evaluation of the model’s accuracy. However, it is crucial to emphasize that the c-index might not completely capture the accuracy in the presence of a substantial disparity in the frequency of cases with LNM compared to those without LNM. Therefore, for further assessment of the model’s accuracy, we also incorporated Sensitivity (SEN) and Specificity (SPE).

We performed a meta-analysis to evaluate the performance metrics (c-index and accuracy) of the model. If the c-index did not have the 95% confidence interval (95% CI) and standard error, we estimated its standard error based on the research conducted by Debray TP [38]. Due to the variability in variables and inconsistent parameters across various ML models, we employed a random-effects model for the meta-analysis of the c-index. Furthermore, even within the same model, variances in the selection of modeling variables or parameters may result in heterogeneity. Taking these factors into consideration, a random-effects model was ultimately selected. Additionally, a bivariate mixed-effects model was utilized for SEN (sensitivity) and SPE (specificity). All statistical analyses were conducted using R4.2.0 software (R development Core Team, Vienna, http://www.R-project.org).

Results

Study selection

After completing a thorough search of the databases, a total of 1309 studies were discovered. After eliminating duplicates, 968 distinct studies were retained. Subsequently, 861 studies were excluded following the screening of titles and abstracts. Ultimately, after diligent evaluation, 107 studies were considered eligible and included in the analysis. The study selection process is visually illustrated in Fig. 1. For further details on the eligible studies, please consult Additional file 2.

Fig. 1
figure 1

Study selection process

Full size image

Characteristics of included studies

A total of 107 studies from four countries - China, Australia, Korea, and the USA - were included in this analysis. Among them, 105 were retrospective studies, while two were prospective studies. The publication dates ranged from 2014 to 2022, and the sample sizes varied from 503 to 22,719. In total, 136,245 patients were involved, with 46,610 of them having LNM. The study encompassed 134 models, representing 10 different types. Specifically, there were 134 predictive models for LNM, with 109 of them being based on LR, constituting 81.34% of the total. The remaining models consisted of 7 GBM models, 4 RF models, 3 ANN models, 3 DT models, 2 LASSO models, 2 SVM models, 2 KNN models, 1 BN model, and 1 DL model.

A total of 100 models were used for CLNM prediction, with 82 models based on LR, accounting for 82.00% of the total. The remaining models comprised 6 GBM models, 3 RF models, 2 ANN models, 2 DT models, 1 LASSO model, 1 SVM model, 1 KNN model, 1 BN model, and 1 DL model. In the case of LLNM prediction, there were 35 models in total, out of which 27 were based on LR, representing 77.14% of the total. The remaining models included 1 GBM model, 1 RF model, 1 ANN model, 1 DT model, 1 LASSO model, 1 SVM model, and 1 KNN model. Additional file 3 contains fundamental information about the 107 original studies included in the analysis.

This analysis included a total of 107 studies, out of which 86 studies provided details about their validation methods. Among these, 61 studies focused exclusively on internal validation. Of these, 28 studies utilized random sampling, 18 studies employed bootstrapping, 10 studies utilized k-fold cross-validation, and 5 studies combined k-fold cross-validation with bootstrapping. Additionally, 16 studies conducted both internal and external validation. Out of these, 6 studies utilized bootstrapping for external validation, 4 studies used k-fold cross-validation for external validation, and 1 study employed both k-fold cross-validation and bootstrapping for external validation. Moreover, 5 studies incorporated random sampling for internal validation alongside external validation. Furthermore, 9 studies were specifically dedicated to external validation, with 4 of them being multi-center external validation studies. More detailed information is available in Additional file 3.

Characteristics of included prediction models

The models provided are classified into the following categories: clinical characteristics-based model, radiomics and clinical characteristics-based model, radiomics characteristics-based model, and ultrasonic characteristics-based model. The clinical characteristics-based model uses variables such as age, gender, tumor diameter, tumor site, history of Hashimoto’s thyroiditis, multi-focal, and extracapsular, among others, for predicting CLNM. For the prediction of LLNM, in addition to the commonly used variables mentioned above, the number or ratio of CLNM and prelaryngeal LNM are also considered effective variables for modeling. Detailed information about the variables used for modeling can be found in Additional file 4.

Quality assessment

The study assessed the risk of bias and the applicability of the models using PROBAST [39]. Most of the studies included were of case-control design, leading to a high risk of bias primarily related to sample selection. In terms of statistical analysis, the high risk of bias was primarily attributed to the lack of an independent validation set. Several validation sets did not meet the requirement of having a sample size of at least 20 times the event per variable (EPV). The outcomes of the risk of bias assessment for the models included in our systematic review are presented in Fig. 2.

Fig. 2
figure 2

Results of the risk of bias assessment of the models included in our systematic review

Full size image

Results of meta-analysis

The ML-based models achieved a c-index of 0.762 (95%CI: 0.747–0.777), SEN of 0.71 (95%CI: 0.69–0.74), and SPE of 0.74 (95%CI: 0.72–0.76) in predicting LNM in the training set. Similarly, for the prediction CLNM, the models attained a c-index of 0.762 (95%CI: 0.747–0.777), SEN of 0.68 (95%CI: 0.65–0.71), and SPE of 0.74 (95%CI: 0.72–0.76). As for the prediction of LLNM, the models achieved a c-index of 0.803 (95%CI: 0.773–0.834) and a SEN of 0.78 (95%CI: 0.75–0.81).

The ML-based models in the validation set showed promising performance in predicting LNM. Specifically, the c-index, SEN, and SPE for predicting LNM were 0.773 (95% CI: 0.754–0.791), 0.74 (95% CI: 0.72–0.76), and 0.74 (95% CI: 0.70–0.77), respectively. Similarly, in the prediction of CLNM, the c-index, SEN, and SPE were 0.762 (95% CI: 0.747–0.777), 0.74 (95% CI: 0.71–0.765), and 0.72 (95% CI: 0.68–0.76), respectively. Lastly, for predicting LLNM, the ML-based models achieved superior results with a c-index, SEN, and SPE of 0.829 (95% CI: 0.779–0.879), 0.77 (95% CI: 0.72–0.82), and 0.82 (95% CI: 0.74–0.88), respectively (Table 1; Fig. 3).

Table 1 The c-index of machine learning models in papillary thyroid carcinoma (PTC) prediction applied to different subgroups and different variables used for model construction
Full size table
Fig. 3
figure 3

Forest map for the c-index of machine learning in detecting lymph node metastasis of papillary thyroid cancer

Full size image

Subgroup analysis of different model types

There were 10 types of ML-based models involved, and LR was the most frequently used.

LR models

The LR models demonstrated a c-index of 0.787 (95% confidence interval: 0.772–0.802), SEN of 0.71 (95% CI: 0.69–0.73), and SPE of 0.77 (95% CI: 0.74–0.79) in the training dataset for predicting LNM. The LR models incorporated various types of modeling variables, with clinical characteristics being the most prevalent among them. Specifically, the corresponding c-index, SEN, and SPE values for these clinical variables were 0.784 (95% CI: 0.769-0.800), 0.71 (95% CI: 0.69–0.73), and 0.76 (95% CI: 0.74–0.78), respectively. Upon validation, the LR models achieved a c-index of 0.784 (95% CI: 0.765–0.802), SEN of 0.72 (95% CI: 0.70–0.74), and SPE of 0.77 (95% CI: 0.73–0.80). Similarly, a majority of LR models utilized clinical characteristics as the primary modeling variables, yielding a c-index, SEN, and SPE of 0.778 (95% CI: 0.758–0.797), 0.71 (95% CI: 0.68–0.73), and 0.75 (95% CI: 0.72–0.78) respectively.

Prediction of CLNM

The LR models in the training set demonstrated a c-index of 0.774 (95% CI: 0.759–0.789), with corresponding SEN and SPE values of 0.69 (95% CI: 0.66–0.71) and 0.77 (95% CI: 0.74–0.79) in predicting CLNM. These LR models encompassed various types, with the majority incorporating patients’ clinical characteristics as modeling variables. The associated c-index, SEN, and SPE for these models were 0.773 (95% CI: 0.756–0.789), 0.68 (95% CI: 0.66–0.71), and 0.76 (95% CI 0.74–0.78). In the validation set, the LR models achieved a c-index of 0.771 (95% CI: 0.750–0.791), with corresponding SEN and SPE values of 0.68 (95% CI: 0.66–0.71) and 0.76 (95% CI: 0.74–0.78) in predicting CLNM. Similar to the models in the training set, the majority of these models utilized patients’ clinical characteristics as modeling variables. The corresponding c-index, SEN, and SPE were 0.775 (95% CI: 0.754–0.795), 0.70 (95% CI: 0.67–0.73), and 0.76 (95% CI: 0.72–0.79), respectively (Table 2; Fig. 4).

Table 2 The sensitivity and specificity of machine learning models in papillary thyroid carcinoma (PTC) prediction applied to different subgroups and different variables used for model construction
Full size table
Fig. 4
figure 4

Forest map for the c-index of machine learning in detecting CLNM of papillary thyroid cancer

Full size image

Prediction of LLNM

The LR models demonstrated strong predictive performance for LLNM in the training set, achieving a c-index of 0.826 (95% CI: 0.792–0.861). The SEN and SPE of these models were 0.78 (95% CI: 0.75–0.81) and 0.79 (95% CI: 0.70–0.85) respectively. The majority of LR models utilized the patients’ clinical characteristics as variables for modeling. Similarly, the corresponding c-index, SEN, and SPE in these models were 0.818 (95% CI: 0.781–0.855), 0.78 (95% CI: 0.75–0.81), and 0.78 (95% CI: 0.69–0.85) respectively. In the validation set, the LR models also displayed strong predictability for LLNM, with a c-index of 0.848 (95% CI: 0.813–0.883). The sensitivity and specificity of these models were 0.77 (95% CI: 0.72–0.82) and 0.84 (95% CI: 0.77–0.89) respectively. Similar to the training set, the majority of models used the patients’ clinical characteristics as variables for modeling. The corresponding c-index, SEN, and SPE in these models were 0.800 (95% CI: 0.765–0.835), 0.76 (95% CI: 0.69–0.82), and 0.74 (95% CI: 0.66–0.82) respectively (Table 2; Fig. 5).

Fig. 5
figure 5

Forest map for the c-index of machine learning in detecting LLNM of thyroid papillary carcinoma

Full size image

Discussion

Principal findings

This comprehensive review has shown that ML-based prediction models achieve high accuracy in predicting the presence of preoperative LNM in PTC patients. Furthermore, our analysis uncovered that the LR models utilized different combinations of modeling variables, such as clinical characteristics, radiomics characteristics, clinical characteristics combined with genomics characteristics, radiomics characteristics combined with ultrasonics characteristics, ultrasonics characteristics combined with clinical characteristics, and radiomics ultrasonic characteristics combined with clinical characteristics. Among these variables, the patients’ clinical characteristics were the most frequently used. Our clinical practice also reflects this, as the aforementioned clinical characteristics can be easily obtained during the diagnosis and treatment of PTC.

Machine learning has become increasingly popular in the field of medicine in recent years, particularly in the diagnosis and prognosis of malignant tumors. There is significant ongoing research in predicting lymph node metastasis in thyroid cancer. Multiple machine learning models have been developed to predict lymph node metastasis of thyroid cancer, utilizing different modeling variables and algorithms. When choosing a model, it is critical to consider not only its accuracy but also its interpretability. Models with higher interpretability, such as LR models and DT models, tend to have relatively lower accuracy. In contrast, models with poor interpretability, such as XGBoost, SVM, and RF models, generally have higher accuracy levels. In the context of building predictive models based on interpretable clinical features, LR models are recommended. It is possible to create more clinically applicable and intuitive nomograms or scoring tools based on LR. This study incorporated a total of 10 predictive models, with LR models being the most prevalent, accounting for 81.34% of the total. The LR models used in predicting LNM exhibited a c-index of 0.787 (95% CI: 0.772–0.802), SEN of 0.71 (95% CI: 0.69–0.73), and SPE of 0.77 (95% CI: 0.74–0.79) in the training set. In the validation set, the LR models achieved a c-index of 0.784 (95% CI: 0.765–0.802), SEN of 0.72 (95% CI: 0.70–0.74), and SPE of 0.77 (95% CI: 0.73–0.80). When compared with other models, the LR models outperformed the rest. Consequently, it can be inferred that LR models, constructed based on clinical features, not only demonstrate excellent predictive performance but also possess easily accessible modeling variables, rendering them highly applicable in clinical settings.

Ultrasound is commonly utilized as the primary method for detecting and evaluating lymph node metastasis prior to surgery. However, its accuracy is heavily reliant on the experience and expertise of the operators, leading to limited sensitivity [40,41,42,43]. According to a study conducted by M.T. Khokhar et al., the sensitivity of preoperative ultrasound detection was found to be less than 50% for CLNM and ranged from 70 to 80% for LLNM [44, 45]. Although computerized tomography (CT) has advantages in detecting deep cervical lymph nodes, its performance is constrained in identifying lymph nodes with a diameter smaller than 5 mm. Therefore, the development of a predictive model capable of quantifying the risk of LNM in PTC patients would significantly aid in the selection of surgical strategies, particularly for clinically node-negative (cN0) patients [15]. In the treatment of PTC, it is recommended to avoid PLND, especially for cN0 patients. A recent randomized controlled trial focusing on PLND in PTC revealed that patients undergoing PLND had significantly higher rates of permanent hypoparathyroidism [46]. Research by Hu Y et al. [47] suggests that while PLND may reduce the local recurrence rate of PTC, it significantly increases the risk of permanent hypocalcemia and transient parathyroid function decline. Studies by other scholars also indicate that PLND results in approximately 35% of PTC patients experiencing temporary or permanent voice changes, hypocalcemia, etc [48,49,50]. Thus, it is essential to conduct an assessment of risk factors related to metastases, and patients with high-risk factors should be selected for appropriate treatment. Incorporating various variables into a predictive model can effectively stratify patients based on their risk factors for metastases. This approach enables the guidance of individualized and precise treatments, while also preventing complications and injuries caused by unnecessary lymph node dissection.

Despite the diverse array of ML-based models available, LR models remain the most prevalent and are therefore the preferred choice for constructing ML-based prediction models. The nomogram presents a distinct listing for each variable, with each sub-variable quantified by a specific score. LR-based nomograms provide an intuitive visualization of prediction probability and are widely utilized as a practical and convenient tool in clinical practice. Our study involved a substantial sample size, leading us to conclude that LR-based models exhibit satisfactory performance in predicting LNM in PTC patients. It is important to note that the predictive accuracy of the model may be impacted by various modeling variables. The efficacy of prediction models in this study was assessed by examining their modeling variables, including clinical, radiomic, genomic, and ultrasonic characteristics of the patients. The incorporation of clinical, radiomic, and ultrasonic characteristics into LR models showed superior efficacy. Significantly, these findings were consistent with observations from clinical practice.

The evaluation of preoperative LNM in papillary thyroid carcinoma (PTC) patients relies on ultrasonic and clinical characteristics. Various factors, such as age, gender, history of Hashimoto’s thyroiditis, and specific ultrasonic characteristics (tumor diameter, tumor site, capsular infiltration, microcalcification, multifocality, etc.), along with the identification of abnormal lymph nodes through ultrasound imaging, have been recognized as risk factors for LNM in PTC patients [51,52,53]. Radiomics, a novel technology employing computer-aided systems, has emerged as a valuable tool for predicting LNM and staging tumors by conducting high-throughput extraction of medical images and transforming them into accessible data for comprehensive statistical analysis [54, 55]. Extensive research has demonstrated the potential of radiomics in predicting LNM, assessing treatment response, and determining patient survival rates [56,57,58]. Furthermore, machine learning algorithms have gained popularity among researchers for managing various types of cancers. These machine learning-based prediction models possess the ability to self-optimize through iterative processes, leading to improved prediction accuracy and practicality by considering valuable variables used in the model [59]. Machine learning-based prediction models have already found applications in the assessment, diagnosis, and treatment of different cancers, including prostate cancer, lung cancer, and breast cancer. These models are also recommended in clinical practice guidelines as effective tools for establishing individualized anticancer treatments and offering valuable references to enhance the prognosis of cancer patients [60,61,62,63]. A comprehensive systematic review supports the notion that machine learning-based prediction models show promising predictive capabilities for preoperative LNM in PTC patients, thus serving as effective guides when selecting appropriate treatment options for PTC patients.

In the context of predicting preoperative LNM in PTC using ML models, the modeling variables can be classified into structured data and unstructured data. Structured data mainly include clinical features and genomics, with a primary focus on the former. These clinical features encompass age, gender, history of Hashimoto thyroiditis, extracapsular involvement, tumor calcification, tumor location, tumor diameter, and multifocality. Each of these clinical features holds practical significance and has shown relatively good predictive values, especially age, gender, tumor diameter, extracapsular involvement, and multifocality [64,65,66]. On the other hand, unstructured data mainly consist of ultrasonics and radiomics characteristics. Our research indicates that ML prediction models incorporating both radiomics and clinical characteristics demonstrate promising predictive values. However, relying solely on unstructured data presents limitations in clinical application due to the lack of corresponding operating guidelines, which increases the risk of bias. Therefore, future studies should focus on developing corresponding operating guidelines to minimize this bias risk.

The study incorporated various prediction models, utilizing an extensive range of variables to develop these models. These variables encompassed clinical features, hematological tests, imaging tests, and genetic tests. The majority of the modeling variables can be conveniently acquired prior to surgery, resulting in prediction models with a commendable predictive value.

Strengths

The systematic review has several advantages: (1) This systematic review is the first of its kind to include multiple relevant studies and a large sample size in assessing the performance of ML-based models for predicting LNM in PTC. The inclusion of all reported prediction models enables meaningful comparisons among different models. The review also highlights the need for continuous improvement in these models. (2) A comprehensive assessment of multiple ML-based models was conducted, revealing that LR models are the most commonly utilized. Additionally, the choice of variables used in model construction can significantly impact the performance of these models. It is important to note that the LR model offers good interpretability, effectively demonstrating the relationship between modeling variables and the likelihood of the outcome event. This feature makes it suitable for the development of simple and visualized intelligent tools, similar to the nomograms created by the Memorial Sloan Kettering Cancer Center for predicting lymph node metastasis in breast (https://nomograms.mskcc.org/breast) and prostate cancer (https://www.mskcc.org/nomograms/prostate). This valuable insight can serve as a reference for the future advancement of model construction tools.

Limitations

It is important to recognize several limitations. Firstly, despite the presence of various mathematical algorithms, there is a lack of research exploring alternative methods for building models. This makes it difficult to assess the predictive accuracy of these models and restricts their practical application in treating PTC. Secondly, the variation in sample size and distribution across different studies directly affects the performance and general applicability of the models. Thirdly, there is a scarcity of studies on models developed using ultrasonics. Lastly, many of the original studies we included were single-center studies, with models undergoing only internal validation and lacking independent external validation. Therefore, the interpretation of the results may be limited to some extent, making the models less universally applicable.

Future directions

In the future, there will be a focus on conducting additional research to develop efficient and refined prediction tools. Throughout the development phase, it is crucial to consider easily accessible and interpretable clinical features. Moreover, equal attention should be given to molecular genetic features, radiomics features, and ultrasonics features. It is essential to integrate these standard clinical pathological features, along with pertinent imaging and molecular genetic features, in order to create intelligent and effective clinical prediction tools.

Conclusion

This systematic review emphasizes the outstanding performance of ML-based prediction models in accurately forecasting LNM in PTC patients. Considering the increasing incidence of PTC and the vital role of predicting preoperative LNM in customizing tumor-specific therapies, there is an urgent need to create a clinically feasible and universally applicable disease prediction model. By leveraging the current models, such a model would significantly improve clinical decision-making and ultimately lead to enhanced patient outcomes.

Data availability

The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.

Abbreviations

CLNM:

Central lymph node metastasis

LLNM:

Lateral lymph node metastasis

LNM:

Lymph node metastasis

LR:

Logistic Regression

ML:

Machine learning

PTC:

Papillary thyroid carcinoma

SEN:

Sensitivity

SPE:

Specificity

References

  1. Carlson RW. The NCCN 2019 annual conference: improving the quality, effectiveness, and efficiency of cancer care. Journal of the National Comprehensive Cancer Network: JNCCN. 2019;17(5.5):529–30. PMID: 31117029. https://doiorg.publicaciones.saludcastillayleon.es/10.6004/jnccn.2019.5001

  2. Zhai M, Zhang D, Long J, Gong Y, Ye F, Liu S, et al. The global burden of thyroid cancer and its attributable risk factor in 195 countries and territories: a systematic analysis for the global burden of disease study. Cancer Med. 2021;10(13):4542–54. https://doiorg.publicaciones.saludcastillayleon.es/10.1002/cam4.3970. PMID: 34002931.

    Article  PubMed  PubMed Central  Google Scholar 

  3. Sung H, Ferlay J, Siegel RL, Laversanne M, Soerjomataram I, Jemal A, et al. Global cancer statistics 2020: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries. Cancer J Clin. 2021;71(3):209–49. https://doiorg.publicaciones.saludcastillayleon.es/10.3322/caac.21660. PMID: 33538338.

    Article  CAS  Google Scholar 

  4. Siegel RL, Miller KD, Fuchs HE, Jemal A. Cancer statistics, 2021. CA: A Cancer Journal for Clinicians. 2021;71(1):7–33. PMID: 33433946. https://doiorg.publicaciones.saludcastillayleon.es/10.3322/caac.21654

  5. Zheng R, Zhang S, Zeng H, Wang S, Sun K, Chen R et al. Cancer incidence and mortality in China, 2016. J Natl Cancer Cent. 2022 2022/03/01/;2(1):1–9. https://doiorg.publicaciones.saludcastillayleon.es/10.1016/j.jncc.2022.02.002

  6. Callender GG, Carling T, Christison-Lagay E, Udelsman R. Surgery for thyroid cancer. Endocrinol Metab Clin North Am. 2014;43(2):443–58. https://doiorg.publicaciones.saludcastillayleon.es/10.1016/j.ecl.2014.02.011. PMID: 24891171.

    Article  PubMed  Google Scholar 

  7. Mao Y, Xing M. Recent incidences and differential trends of thyroid cancer in the USA. Endocrine-related Cancer. 2016;23(4):313–22. https://doiorg.publicaciones.saludcastillayleon.es/10.1530/erc-15-0445. PMID: 26917552.

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  8. Randle RW, Balentine CJ, Leverson GE, Havlena JA, Sippel RS, Schneider DF, et al. Trends in the presentation, treatment, and survival of patients with medullary thyroid cancer over the past 30 years. Surgery. 2017;161(1):137–46. https://doiorg.publicaciones.saludcastillayleon.es/10.1016/j.surg.2016.04.053. PMID: 27842913.

    Article  PubMed  Google Scholar 

  9. Feng JW, Qin AC, Ye J, Pan H, Jiang Y, Qu Z. Predictive factors for lateral lymph node metastasis and skip metastasis in papillary thyroid carcinoma. Endocr Pathol. 2020;31(1):67–76. https://doiorg.publicaciones.saludcastillayleon.es/10.1007/s12022-019-09599-w. PMID: 31828583.

    Article  PubMed  CAS  Google Scholar 

  10. Lim H, Devesa SS, Sosa JA, Check D, Kitahara CM. Trends in thyroid cancer incidence and mortality in the United States, 1974–2013. Jama. 2017;317(13):1338-48. PMID: 28362912. https://doiorg.publicaciones.saludcastillayleon.es/10.1001/jama.2017.2719

  11. Dong W, Horiuchi K, Tokumitsu H, Sakamoto A, Noguchi E, Ueda Y, et al. Time-varying pattern of mortality and recurrence from papillary thyroid cancer: lessons from a long-term follow-up. Thyroid: Official J Am Thyroid Association. 2019;29(6):802–8. https://doiorg.publicaciones.saludcastillayleon.es/10.1089/thy.2018.0128. PMID: 30931815.

    Article  CAS  Google Scholar 

  12. Ling Y, Zhang L, Li K, Zhao Y, Zhao J, Jia L, et al. Carbon nanoparticle-guided intraoperative lymph node biopsy predicts the status of lymph nodes posterior to right recurrent laryngeal nerve in cN0 papillary thyroid carcinoma. Gland Surg. 2021;10(5):1554–63. https://doiorg.publicaciones.saludcastillayleon.es/10.21037/gs-20-920. PMID: 34164300.

    Article  PubMed  PubMed Central  Google Scholar 

  13. Chow SM, Law SC, Chan JK, Au SK, Yau S, Lau WH. Papillary microcarcinoma of the thyroid-prognostic significance of lymph node metastasis and multifocality. Cancer. 2003;98(1):31–40. https://doiorg.publicaciones.saludcastillayleon.es/10.1002/cncr.11442. PMID: 12833452.

    Article  PubMed  Google Scholar 

  14. Park AY, Kim JA, Son EJ, Youk JH. Shear-wave elastography for papillary thyroid carcinoma can improve prediction of cervical lymph node metastasis. Ann Surg Oncol. 2016;23(Suppl 5):722–9. https://doiorg.publicaciones.saludcastillayleon.es/10.1245/s10434-016-5572-x. PMID: 27654109.

    Article  PubMed  Google Scholar 

  15. Thompson AM, Turner RM, Hayen A, Aniss A, Jalaty S, Learoyd DL, et al. A preoperative nomogram for the prediction of ipsilateral central compartment lymph node metastases in papillary thyroid cancer. Thyroid: Official J Am Thyroid Association. 2014;24(4):675–82. https://doiorg.publicaciones.saludcastillayleon.es/10.1089/thy.2013.0224. PMID: 24083952.

    Article  CAS  Google Scholar 

  16. Shaha AR. Prognostic factors in papillary thyroid carcinoma and implications of large nodal metastasis. Surgery. 2004;135(2):237–9. https://doiorg.publicaciones.saludcastillayleon.es/10.1016/j.surg.2003.08.023. PMID: 14739864.

    Article  PubMed  Google Scholar 

  17. Zaydfudim V, Feurer ID, Griffin MR, Phay JE. The impact of lymph node involvement on survival in patients with papillary and follicular thyroid carcinoma. Surgery. 2008;144(6):1070–7. https://doiorg.publicaciones.saludcastillayleon.es/10.1016/j.surg.2008.08.034. discussion 7–8. PMID: 19041020.

    Article  PubMed  Google Scholar 

  18. Smith VA, Sessions RB, Lentsch EJ. Cervical lymph node metastasis and papillary thyroid carcinoma: does the compartment involved affect survival? Experience from the SEER database. J Surg Oncol. 2012;106(4):357–62. https://doiorg.publicaciones.saludcastillayleon.es/10.1002/jso.23090. PMID: 22392921.

    Article  PubMed  Google Scholar 

  19. Medas F, Canu GL, Cappellacci F, Anedda G, Conzo G, Erdas E, et al. Prophylactic central lymph node dissection improves disease-free survival in patients with intermediate and high risk differentiated thyroid carcinoma: a retrospective analysis on 399 patients. Cancers. 2020;12(6). https://doiorg.publicaciones.saludcastillayleon.es/10.3390/cancers12061658. PMID: 32585797.

  20. Liu Z, Zeng W, Liu C, Wang S, Xiong Y, Guo Y et al. Diagnostic accuracy of ultrasonographic features for lymph node metastasis in papillary thyroid microcarcinoma: a single-center retrospective study. World journal of surgical oncology. 2017;15(1):32. PMID: 28125992. https://doiorg.publicaciones.saludcastillayleon.es/10.1186/s12957-017-1099-2

  21. Shahriarirad R, Meshkati Yazd SM, Zahedi R, Mokhtari Ardekani A, Rekabi MM, Nasiri S. Evaluation of the role of prophylactic bilateral central neck lymph node dissection in patients with papillary thyroid carcinoma: a case controlled study. Updates Surg. 2023;75(3):679–89. https://doiorg.publicaciones.saludcastillayleon.es/10.1007/s13304-022-01440-0. PMID: 36527603.

    Article  PubMed  Google Scholar 

  22. Zhao WJ, Luo H, Zhou YM, Dai WY, Zhu JQ. Evaluating the effectiveness of prophylactic central neck dissection with total thyroidectomy for cN0 papillary thyroid carcinoma: an updated meta-analysis. Eur J Surg Oncol. 2017;43(11):1989–2000. https://doiorg.publicaciones.saludcastillayleon.es/10.1016/j.ejso.2017.07.008. PMID: 28807633.

    Article  PubMed  Google Scholar 

  23. Chen L, Wu YH, Lee CH, Chen HA, Loh EW, Tam KW. Prophylactic central neck dissection for papillary thyroid carcinoma with clinically uninvolved central neck lymph nodes: a systematic review and meta-analysis. World J Surg. 2018;42(9):2846–57. https://doiorg.publicaciones.saludcastillayleon.es/10.1007/s00268-018-4547-4. PMID: 29488066.

    Article  PubMed  Google Scholar 

  24. Yazıcı D, Çolakoğlu B, Sağlam B, Sezer H, Kapran Y, Aydın Ö et al. Effect of prophylactic central neck dissection on the surgical outcomes in papillary thyroid cancer: experience in a single center. Eur Arch Otorhinolaryngol. 2020;277(5):1491-7. PMID: 32052141. https://doiorg.publicaciones.saludcastillayleon.es/10.1007/s00405-020-05830-1

  25. Haugen BR, Alexander EK, Bible KC, Doherty GM, Mandel SJ, Nikiforov YE, et al. 2015 American Thyroid Association management guidelines for adult patients with thyroid nodules and differentiated thyroid cancer: the American Thyroid Association guidelines task force on thyroid nodules and differentiated thyroid cancer. Thyroid: Official J Am Thyroid Association. 2016;26(1):1–133. https://doiorg.publicaciones.saludcastillayleon.es/10.1089/thy.2015.0020. PMID: 26462967.

    Article  Google Scholar 

  26. Sancho JJ, Lennard TW, Paunovic I, Triponez F, Sitges-Serra A. Prophylactic central neck disection in papillary thyroid cancer: a consensus report of the European Society of Endocrine Surgeons (ESES). Langenbecks Arch Surg. 2014;399(2):155–63. https://doiorg.publicaciones.saludcastillayleon.es/10.1007/s00423-013-1152-8. PMID: 24352594.

    Article  PubMed  Google Scholar 

  27. Agcaoglu O, Sengun B, Ozoran E, Bilgic C, Karabay O, Taskin OC et al. Should we perform routine prophylactic central neck dissection in patients with thyroid papillary microcarcinoma? Annali Italiani Di Chirurgia. 2018;89:485–8. PMID: 30665217.

  28. Liu C, Xiao C, Chen J, Li X, Feng Z, Gao Q et al. Risk factor analysis for predicting cervical lymph node metastasis in papillary thyroid carcinoma: a study of 966 patients. BMC cancer. 2019;19(1):622. PMID: 31238891. https://doiorg.publicaciones.saludcastillayleon.es/10.1186/s12885-019-5835-6

  29. Beam AL, Drazen JM, Kohane IS, Leong TY, Manrai AK, Rubin EJ. Artificial intelligence in medicine. N Engl J Med. 2023;388(13):1220–1. https://doiorg.publicaciones.saludcastillayleon.es/10.1056/NEJMe2206291. PMID: 36988598.

    Article  PubMed  Google Scholar 

  30. Barnes BM, Nelson L, Tighe A, Burghel GJ, Lin IH, Desai S et al. Distinct transcriptional programs stratify ovarian cancer cell lines into the five major histological subtypes. Genome medicine. 2021;13(1):140. PMID: 34470661. https://doiorg.publicaciones.saludcastillayleon.es/10.1186/s13073-021-00952-5

  31. Liu J, Liu L, Antwi PA, Luo Y, Liang F. Identification and validation of the diagnostic characteristic genes of ovarian cancer by bioinformatics and machine learning. Frontiers in genetics. 2022;13:858466. PMID: 35719392. https://doiorg.publicaciones.saludcastillayleon.es/10.3389/fgene.2022.858466

  32. Bareche Y, Kelly D, Abbas-Aghababazadeh F, Nakano M, Esfahani PN, Tkachuk D et al. Leveraging big data of immune checkpoint blockade response identifies novel potential targets. Annals of oncology: official journal of the European Society for Medical Oncology. 2022;33(12):1304-17. PMID: 36055464. https://doiorg.publicaciones.saludcastillayleon.es/10.1016/j.annonc.2022.08.084

  33. Kuo YL, Chen WC, Yao WJ, Cheng L, Hsu HP, Lai HW, et al. Validation of Memorial Sloan-Kettering Cancer Center nomogram for prediction of non-sentinel lymph node metastasis in sentinel lymph node positive breast cancer patients an international comparison. Int J Surg (London England). 2013;11(7):538–43. https://doiorg.publicaciones.saludcastillayleon.es/10.1016/j.ijsu.2013.05.005. PMID: 23707626.

    Article  Google Scholar 

  34. Vieni S, Graceffa G, La Mendola R, Latteri S, Cordova A, Latteri MA, et al. Application of a predictive model of axillary lymph node status in patients with sentinel node metastasis from breast cancer. A retrospective cohort study. Int J Surg (London England). 2016;35:58–63. https://doiorg.publicaciones.saludcastillayleon.es/10.1016/j.ijsu.2016.09.015. PMID: 27633450.

    Article  Google Scholar 

  35. Cagiannos I, Karakiewicz P, Eastham JA, Ohori M, Rabbani F, Gerigk C, et al. A preoperative nomogram identifying decreased risk of positive pelvic lymph nodes in patients with prostate cancer. J Urol. 2003;170(5):1798–803. https://doiorg.publicaciones.saludcastillayleon.es/10.1097/01.ju.0000091805.98960.13. PMID: 14532779.

    Article  PubMed  Google Scholar 

  36. Moons KG, de Groot JA, Bouwmeester W, Vergouwe Y, Mallett S, Altman DG et al. Critical appraisal and data extraction for systematic reviews of prediction modelling studies: the CHARMS checklist. PLoS medicine. 2014;11(10):e1001744. PMID: 25314315. https://doiorg.publicaciones.saludcastillayleon.es/10.1371/journal.pmed.1001744

  37. Wynants L, Van Calster B, Collins GS, Riley RD, Heinze G, Schuit E, et al. Prediction models for diagnosis and prognosis of covid-19: systematic review and critical appraisal. BMJ (Clinical Res ed). 2020;369:m1328. https://doiorg.publicaciones.saludcastillayleon.es/10.1136/bmj.m1328. PMID: 32265220.

    Article  Google Scholar 

  38. Debray TP, Damen JA, Riley RD, Snell K, Reitsma JB, Hooft L, et al. A framework for meta-analysis of prediction model studies with binary and time-to-event outcomes. Stat Methods Med Res. 2019;28(9):2768–86. https://doiorg.publicaciones.saludcastillayleon.es/10.1177/0962280218785504. PMID: 30032705.

    Article  PubMed  Google Scholar 

  39. Wolff RF, Moons KGM, Riley RD, Whiting PF, Westwood M, Collins GS, et al. PROBAST: a tool to assess the risk of bias and applicability of prediction model studies. Ann Intern Med. 2019;170(1):51–8. https://doiorg.publicaciones.saludcastillayleon.es/10.7326/m18-1376. PMID: 30596875.

    Article  PubMed  Google Scholar 

  40. Han Z, Lei Z, Li M, Luo D, Ding J. Differential diagnosis value of the ultrasound gray scale ratio for papillary thyroid microcarcinomas and micronodular goiters. Quant Imaging Med Surg. 2018;8(5):507–13. https://doiorg.publicaciones.saludcastillayleon.es/10.21037/qims.2018.06.04. PMID: 30050785.

    Article  PubMed  PubMed Central  Google Scholar 

  41. Lee JY, Na DG, Yoon SJ, Gwon HY, Paik W, Kim T, et al. Ultrasound malignancy risk stratification of thyroid nodules based on the degree of hypoechogenicity and echotexture. Eur Radiol. 2020;30(3):1653–63. https://doiorg.publicaciones.saludcastillayleon.es/10.1007/s00330-019-06527-8. PMID: 31732777.

    Article  PubMed  Google Scholar 

  42. Karkada M, Costa AF, Imran SA, Hart RD, Bullock M, Ilie G et al. Incomplete thyroid ultrasound reports for patients with thyroid nodules: implications regarding risk assessment and management. AJR American journal of roentgenology. 2018;211(6):1348-53. PMID: 30332287. https://doiorg.publicaciones.saludcastillayleon.es/10.2214/ajr.18.20056

  43. Cheng SCH, Ahuja AT, Ying M. Quantification of intranodal vascularity by computer pixel-counting method enhances the accuracy of ultrasound in distinguishing metastatic and tuberculous cervical lymph nodes. Quant Imaging Med Surg. 2019;9(11):1773–80. https://doiorg.publicaciones.saludcastillayleon.es/10.21037/qims.2019.10.02. PMID: 31867231.

    Article  PubMed  PubMed Central  Google Scholar 

  44. Xu SY, Yao JJ, Zhou W, Chen L, Zhan WW. Clinical characteristics and ultrasonographic features for predicting central lymph node metastasis in clinically node-negative papillary thyroid carcinoma without capsule invasion. Head Neck. 2019;41(11):3984–91. https://doiorg.publicaciones.saludcastillayleon.es/10.1002/hed.25941. PMID: 31463972.

    Article  PubMed  Google Scholar 

  45. Khokhar MT, Day KM, Sangal RB, Ahmedli NN, Pisharodi LR, Beland MD, et al. Preoperative high-resolution ultrasound for the assessment of malignant central compartment lymph nodes in papillary thyroid cancer. Thyroid: Official J Am Thyroid Association. 2015;25(12):1351–4. https://doiorg.publicaciones.saludcastillayleon.es/10.1089/thy.2015.0176. PMID: 26431908.

    Article  Google Scholar 

  46. Zhao J, Wang L, Zhang Y, He H, Zhao P, Luo Y. Predictors of metastasis in cervical indeterminate lymph nodes after thyroid cancer ablation by long-term ultrasound follow-up. Int J Hyperthermia. 2023;40(1):2207792. PMID: 37156534. https://doiorg.publicaciones.saludcastillayleon.es/10.1080/02656736.2023.2207792

  47. Hu Y, Cui M, Chang X, Wang O, Chen T, Xiao J et al. Patterns and predictors of cervical lymph node metastasis in parathyroid carcinoma. Cancers. 2022;14(16). PMID: 36010997. https://doiorg.publicaciones.saludcastillayleon.es/10.3390/cancers14164004

  48. Xie X, Deng J, Zheng B, Zhong L, Miao J. The effect of central lymph node dissection on the prognosis of cN0 papillary thyroid microcarcinoma: a mid-term follow-up study. BMC Endocr Disord. 2023;23(1):125. PMID: 37248461. https://doiorg.publicaciones.saludcastillayleon.es/10.1186/s12902-023-01375-6

  49. Wang B, Zhu CR, Fei Y, Liu H, Yao XM, Wu J. Prelaryngeal and/or pretracheal lymph node metastasis could help to identify papillary thyroid carcinoma with intermediate risk from unilateral lobe cT1-2N0 papillary thyroid carcinoma. Frontiers in endocrinology. 2023;14:1156664. PMID: 37124749. https://doiorg.publicaciones.saludcastillayleon.es/10.3389/fendo.2023.1156664

  50. Yan XQ, Zhang ZZ, Yu WJ, Ma ZS, Chen ML, Xie BJ. Prophylactic central neck dissection for cN1b papillary thyroid carcinoma: a systematic review and meta-analysis. Front Oncol. 2021;11:803986. PMID: 35096606. https://doiorg.publicaciones.saludcastillayleon.es/10.3389/fonc.2021.803986

  51. Sahin S, Daglar G, Menekse E, Cavdarli B, Baglan T. The effect of BRAF (V600E) mutation on lymph node involvement in papillary thyroid cancer. Turkish J Surg. 2020;36(3):249–55. https://doiorg.publicaciones.saludcastillayleon.es/10.47717/turkjsurg.2020.4696. PMID: 33778379.

    Article  Google Scholar 

  52. Yu J, Deng Y, Liu T, Zhou J, Jia X, Xiao T et al. Lymph node metastasis prediction of papillary thyroid carcinoma based on transfer learning radiomics. Nature communications. 2020;11(1):4807. PMID: 32968067. https://doiorg.publicaciones.saludcastillayleon.es/10.1038/s41467-020-18497-3

  53. Medas F, Canu GL, Cappellacci F, Boi F, Lai ML, Erdas E et al. Predictive factors of lymph node metastasis in patients with papillary microcarcinoma of the thyroid: retrospective analysis on 293 cases. Frontiers in endocrinology. 2020;11:551. PMID: 32982963. https://doiorg.publicaciones.saludcastillayleon.es/10.3389/fendo.2020.00551

  54. Lubner MG, Smith AD, Sandrasegaran K, Sahani DV, Pickhardt PJ. CT texture analysis: definitions, applications, biologic correlates, and challenges. Radiographics: a review publication of the Radiological Society of North America, Inc. 2017 Sep-Oct;37(5):1483–503. PMID: 28898189. https://doiorg.publicaciones.saludcastillayleon.es/10.1148/rg.2017170056

  55. Ștefan PA, Lupean RA, Mihu CM, Lebovici A, Oancea MD, Hîțu L et al. Ultrasonography in the diagnosis of adnexal lesions: the role of texture analysis. Diagnostics (Basel, Switzerland). 2021;11(5). PMID: 33947150. https://doiorg.publicaciones.saludcastillayleon.es/10.3390/diagnostics11050812

  56. Schob S, Meyer HJ, Dieckow J, Pervinder B, Pazaitis N, Höhn AK, et al. Histogram analysis of diffusion weighted imaging at 3T is useful for prediction of lymphatic metastatic spread, proliferative activity, and cellularity in thyroid cancer. Int J Mol Sci. 2017;18(4). https://doiorg.publicaciones.saludcastillayleon.es/10.3390/ijms18040821. PMID: 28417929.

  57. Cui Y, Yang X, Shi Z, Yang Z, Du X, Zhao Z, et al. Radiomics analysis of multiparametric MRI for prediction of pathological complete response to neoadjuvant chemoradiotherapy in locally advanced rectal cancer. Eur Radiol. 2019;29(3):1211–20. https://doiorg.publicaciones.saludcastillayleon.es/10.1007/s00330-018-5683-9. PMID: 30128616.

    Article  PubMed  Google Scholar 

  58. Wu J, Cao G, Sun X, Lee J, Rubin DL, Napel S, et al. Intratumoral spatial heterogeneity at perfusion MR imaging predicts recurrence-free survival in locally advanced breast cancer treated with neoadjuvant chemotherapy. Radiology. 2018;288(1):26–35. https://doiorg.publicaciones.saludcastillayleon.es/10.1148/radiol.2018172462. PMID: 29714680.

    Article  PubMed  Google Scholar 

  59. Deo RC. Machine learning in medicine. Circulation. 2015;132(20):1920–30. https://doiorg.publicaciones.saludcastillayleon.es/10.1161/circulationaha.115.001593. PMID: 26572668.

    Article  PubMed  PubMed Central  Google Scholar 

  60. Gandaglia G, Martini A, Ploussard G, Fossati N, Stabile A, De Visschere P, et al. External validation of the 2019 Briganti nomogram for the identification of prostate cancer patients who should be considered for an extended pelvic lymph node dissection. Eur Urol. 2020;78(2):138–42. https://doiorg.publicaciones.saludcastillayleon.es/10.1016/j.eururo.2020.03.023. PMID: 32268944.

    Article  PubMed  Google Scholar 

  61. Collins GS, Reitsma JB, Altman DG, Moons KGM. Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): the TRIPOD statement. European urology. 2015;67(6):1142-51. PMID: 25572824. https://doiorg.publicaciones.saludcastillayleon.es/10.1016/j.eururo.2014.11.025

  62. Muller DC, Johansson M, Brennan P, Lung Cancer Risk Prediction Model Incorporating Lung Function. Development and validation in the UK Biobank prospective cohort study. J Clin Oncology: Official J Am Soc Clin Oncol. 2017;35(8):861–9. https://doiorg.publicaciones.saludcastillayleon.es/10.1200/jco.2016.69.2467. PMID: 28095156.

    Article  Google Scholar 

  63. Recht A. Radiation-induced heart disease after breast cancer treatment: how big a problem, and how much can-and should-we try to reduce it? Journal of clinical oncology: official journal of the American Society of Clinical Oncology. 2017;35(11):1146-8. PMID: 28095154. https://doiorg.publicaciones.saludcastillayleon.es/10.1200/jco.2016.71.4113

  64. Abuahmed MY, Rashid R, Aboelwafa WA, Hamza YM. The oncologic outcomes of bilateral central lymph node dissection in unilobar papillary thyroid cancer and its risks: a prospective cohort study. Cureus. 2024;16(7):e65443. https://doiorg.publicaciones.saludcastillayleon.es/10.7759/cureus.65443. PMID: 39184776.

    Article  PubMed  PubMed Central  Google Scholar 

  65. Altiner S, Kozan R, Emral AC, Taneri F, Karamercan A. Effects of patient and tumor characteristics on central lymph node metastasis in papillary thyroid cancer: a guide for selective node dissection. Arch Iran Med. 2022;25(11):730-6. PMID: 37543897. https://doiorg.publicaciones.saludcastillayleon.es/10.34172/aim.2022.115

  66. Feng JW, Ye J, Wu WX, Qu Z, Qin AC, Jiang Y. Management of cN0 papillary thyroid microcarcinoma patients according to risk-scoring model for central lymph node metastasis and predictors of recurrence. J Endocrinol Invest. 2020;43(12):1807-17. PMID: 32557354. https://doiorg.publicaciones.saludcastillayleon.es/10.1007/s40618-020-01326-1

Download references

Acknowledgements

We would like to thank the researchers and study participants for their contributions.

Funding

The authors did not receive support from any organization for the submitted work.

Author information

Author notes

  1. Feng Liu and Fei Han contributed equally to this work and share first authorship.

Authors and Affiliations

  1. Department of Head and Neck Surgery, Shanxi Provincial Cancer Hospital, Shanxi Hospital Cancer Hospital of Chinese Academy of Medical Sciences, Taiyuan, 031000, Shanxi Province, China

    Feng Liu, Fei Han, Lifang Lu, Zhen Guo & Jingchun Yao

  2. Department of General Surgery, The First Affiliated Hospital of Shanxi Medical University, Taiyuan, Shanxi Province, China

    Yizhang Chen

Authors
  1. Feng Liu
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Fei Han
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Lifang Lu
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Yizhang Chen
    View author publications

    You can also search for this author inPubMed Google Scholar

  5. Zhen Guo
    View author publications

    You can also search for this author inPubMed Google Scholar

  6. Jingchun Yao
    View author publications

    You can also search for this author inPubMed Google Scholar

Contributions

We confirm that all listed authors meet the authorship criteria, and all authors are in agreement with the content of the manuscript. LFL, ZG and JCY had the original idea for the study and proposed the study design. FL and FH conducted the literature search, screened and selected the studies initially identified. LFL and ZG read and evaluated the quality of the studies included. FL, FH and YZC conducted the meta-analysis. FL and FH wrote the initial manuscript and serves as guarantor. YZC, ZG and JCY revised the manuscript. All authors contributed to interpreting the study findings and to the final manuscript.

Corresponding author

Correspondence to Jingchun Yao.

Ethics declarations

Ethics approval and consent to participate

All analyses were based on previous published studies, thus no ethical approval and patient consent are required.

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material. You do not have permission under this licence to share adapted material derived from this article or parts of it. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc-nd/4.0/.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Liu, F., Han, F., Lu, L. et al. Meta-analysis of prediction models for predicting lymph node metastasis in thyroid cancer. World J Surg Onc 22, 278 (2024). https://doiorg.publicaciones.saludcastillayleon.es/10.1186/s12957-024-03566-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doiorg.publicaciones.saludcastillayleon.es/10.1186/s12957-024-03566-4

Keywords

World Journal of Surgical Oncology

ISSN: 1477-7819

Contact us