ObjectiveTo evaluate the reliability and validity of Guideline Implementation Success Assessment Tool (A-GIST). MethodsWith the guideline for the prevention and treatment of type 2 diabetes mellitus in China (2020 edition) as the target guideline, health care providers and patients from different hospitals across the country were investigated by questionnaire using A-GIST. Spearman-Brown coefficient and Cronbach's α coefficient were used to evaluate the split-half reliability and internal consistency reliability, while the structural validity, convergent validity and discriminant validity were investigated by confirmatory factor analysis based on structural equation. ResultsThe internal consistency reliability and split half reliability coefficients of the whole tool and each dimension ranged from 0.650 to 0.986. The scale-level content validity index (S-CVI/Ave) of content validity was 0.846. Confirmatory factor analysis showed that, the χ2/df of two sections of the tool were 8.695 and 6.123, respectively. The root mean square residual (RMR), the standard root mean square residual (SRMR) and the root mean square error of approximation (RMSEA) were under or almost under the threshold. Besides, the goodness-of-fit index (GFI) of them were 0.901 and 0.822, the adjusted goodness-of-fit index (AGFI) were 0.836 and 0.787, and the parsimonious normed fit index (PNFI) were 0.545 and 0.788, respectively. ConclusionGuideline Implementation Success Assessment Tool (A-GIST) was proved to be valid and reliable, and it shows that it is necessary to optimize the items under the dimensions of maintenance and evaluation of diagnosis and treatment effect in the future.
ObjectiveTo explore the validity period of oral sterilizing instruments in different packages inside the bio-safety cabinet. MethodsInside the bio-safety cabinet, we brought the autoclaved dental gadgets packed in glassware and paper-plastic into microbial sampling cultivation experiment, and made observation of the bacteria growth at the 4th, 8th, 24th hour after sterilizing, with the sampling of once per 24 hours, lasting 10 days. Samples taken from the gadgets in paper-plastic each time when samples were taken from the gadgets in glassware were as the control. ResultsUnder the condition that the bio-safety cabinet was not contaminated, no colony grew in neither group in 144 hours; 168 hours later, the cultivation results indicated that colonies started to grow（P<0.05）. ConclusionUnder the condition that the bio-safety cabinet is pollution-free, the validity period of gadgets in glassware packaging for dental clinic practices is far longer than the 4-hour limit after the sterilized packages are opened, as is stipulated in the "Operating Specifications for Sterilization Technology on Dental Instruments of Medical Institutions".
Objective To develop a behavioral assessment scale for medication management plans in women of childbearing age with epilepsy and to test its reliability and validity. Methods Based on the Theory of Planned Behavior, a pool of questionnaire items was initially drafted through literature review and focus group discussions. A two-round Delphi expert consultation was conducted with 15 experts to form a test version of the behavioral assessment scale for medication management plans in women of childbearing age with epilepsy (including 27 items and 5 dimensions). Convenience sampling was used to conduct surveys among women of childbearing age with epilepsy in some tertiary hospitals in Chuxiong, Shenzhen and Wuhan from February to May 2024 (the first time) and from June to October 2024 (the second time). ResultsThe effective recovery rates of the two rounds of questionnaires were 95.5% and 94.6%, respectively. The final scale included 24 items and 5 dimensions, with good reliability and validity: the content validity index (S-CVI) was 0.934, Cronbach's α coefficient was 0.876, split-half reliability was 0.819, and test-retest reliability was 0.901; exploratory factor analysis extracted 5 factors (cumulative variance explained rate 73.97%, item load 0.42~0.85), and confirmatory factor analysis showed that the model had good fit (χ2/df=1.849, RMSEA=0.075, CFI, GFI, AGFI, IFI, TLI all>0.85). Conclusion The scale meets the reliability and validity standards and can be used to assess the medication management plans and behaviors of women of childbearing age with epilepsy.
ObjectiveTo compare and evaluate the discrimination, validity, and reliability of different data envelopment analysis (DEA) models for measuring the effectiveness of models by selecting different input and output indicators of the model.MethodsData from health statistical reports and pilot program of diagnosis-related groups of tertiary hospitals in Hubei Province from 2017 to 2018 were used to analyze the discrimination, content and structure validity, and reliability of the models. Six DEA models were established by enriching the details of input and output on the basis of the input and output indicators of the conventional DEA model of hospitals.ResultsFrom the view of discrimination, the results of all models were left-skewed, the cost-efficiency model had the lowest left-skewed degree (skewness coefficient: -0.14) and was the flattest (kurtosis coefficient: -1.02). From the view of structure validity, the results of the cost-efficiency model were positively correlated with total weights, outpatient visits, and inpatient visits (r=0.328, 0.329, 0.315; P<0.05). From the perspective of content validity, the interpretation of model was more consistent with theory of production after revision of input and output indicators. From the view of reliability, the cost efficiency model had the largest correlation coefficient between the data of 2017 and 2018 (r=0.880, P<0.05).ConclusionsAfter refining the input and output indicators of the DEA model, the discrimination, validity, and reliability of the model are higher, and the results are more reasonable. Using indicators such as discrimination, validity, and reliability can measure the effectiveness of the DEA model, and then optimize the model by selecting different input and output indicators.
Patient-reported outcome measures (PROM) measure attribute studies refer to studies conducted by investigators to validate the measurement attributes of PROM. The consensus-based standards for the selection of health measurement instruments (COSMIN), an international consensus standard for the selection of health measurement instruments, divides this attribute into three aspects: reliability, validity and responsiveness, and adds interpretability as an additional important feature for evaluating PROM. The purpose of this paper is to introduce the verification methods and principles of the three major measurement attributes in the COSMIN consensus, as well as the significance and direction of interpretability evaluation, and to provide international methodological experience and reference for the development of high-quality PROM psychometric attribute verification in China.
Evidence synthesis is the process of systematically gathering, analyzing, and integrating available research evidence. The quality of evidence synthesis depends on the quality of the original studies included. Validity assessment, also known as risk of bias assessment, is an essential method for assessing the quality of these original studies. Currently, there are numerous validity assessment tools available, but some of them lack a rigorous development process and evaluation. The application of inappropriate validity assessment tools to assessing the quality of the original studies during the evidence synthesis process may compromise the accuracy of study conclusions and mislead the clinical practice. To address this dilemma, the LATITUDES Network, a one-stop resource website for validity assessment tools, was established in September 2023, led by academics at the University of Bristol, U.K. This Network is dedicated to collecting, sorting and promoting validity assessment tools to improve the accuracy of original study validity assessments and increase the robustness and reliability of the results of evidence synthesis. This study introduces the background of the establishment of the LATITUDES Network, the included validity assessment tools, and the training resources for the use of validity assessment tools, in order to provide a reference for domestic scholars to learn more about the LATITUDES Network, to better use the appropriate validity assessment tools to conduct study quality assessments, and to provide references for the development of validity assessment tools.
Objective To develop an evaluation tool for the screening of high risk population for oral complications in critically ill patients, which can be performed accurately and scientifically. Methods Basing on the related foreign oral assessment scale, combined with the method of brainstorming, expert consultation, method of clinical status and so on, the item pool of the assessment scale was determined. Five nursing experts and two oral experts assessed the content validity and 50 ICU nurses were tested. Then, the screening accuracy of the assessment scale was proved by application in 100 critically ill patients selected randomly. Results The Cronbach’s a coefficient of final version of the High Risk Assessment Scale for Oral Complications in Critically Ill Patients (including seven parts contents of oral health assessment and oral pH value test) was 0.815, the content validity index (Sr-CVI/Ave) was 0.932. The results of 50 nurses to the 91.2% assessment items of the assessment scale were very important and important. For screening related indicators of oral complications in high-risk patients, the sensitivity of the assessment scale was 97.53%, the specificity was 94.11%, the positive predictive value was 98.75%, the negative predictive value was 88.89%, and the crude agreement was 95%. Conclusion There are good reliability, validity and a high accuracy of screening test in the High Risk Assessment Scale for Oral Complications in Critically Ill Patients. It can be used for screening patients at high risk for oral complications in critically ill patients, and help clinical nurses to complete the oral health status of the critically ill patients quickly.
ObjectiveTo formulate the Chinese version of Fear of Progression Questionnaire-Short Form/Caregiver Version (FoP-Q-SF/C) and examine the reliability and validity of the scale.MethodsA questionnaire survey of FoP-Q-SF/C was conducted among the caregivers of melanoma out-patients in West China Hospital of Sichuan University from June 2019 to March 2020. Convenient sampling method was adopted. The validity and reliability of the scale were analyzed.ResultsA total of 247 caregivers of melanoma out-patients were investigated by the FoP-Q-SF/C, and 101 valid questionnaires were finally collected. The Cronbach’s α of the FoP-Q-SF/C scale was 0.919, and the Guttman Split-Half coefficient was 0.906. Using exploratory factor analysis to extract 3 common factors, the cumulative explainable total variation was 73.964%. The model fit was as follows: chi-square/degree of freedom was 1.950, standardized root mean square residual was 0.067, goodness of fit index was 0.859, incremental fit index was 0.939, comparative fit index was 0.938, Tucker-Lewis index or non-normed fit index was 0.918, and the root-mean-square error of approximation was 0.097.ConclusionsThe FoP-Q-SF/C scale formulated in this study is divided into three dimensions, which has good reliability and validity, meanwhile, it is relatively simple and can be used to clinically screen melanoma caregivers’ FoP-Q-SF/C levels. However, the application of this scale in other diseases still needs further testing.
ObjectiveTo investigate the reliability and validity of Short-Form 36 Health Survey Scale (SF-36) for evaluating quality of life (QOL) of thoracic surgery patients in a specific regional medical center,and improve care and nursing plan for these patients. MethodsNinety-five patients who were admitted in Department of Thoracic Surgery of West China Hospital from March to May 2012 were enrolled in this study. Ninety-four patients finished a valid questionnaire study including 68 male and 26 female patients with their average age of 62.0±13.0 years. Preoperative diagnosis was squamous cell lung cancer in 8 patients,lung adenocarcinoma in 6 patients,small cell lung cancer in 1 patient,esophageal cancer in 12 patients and undefined lung mass in 67 patients. Postoperative diagnosis was squamous cell lung cancer in 39 patients, lung adenocarcinoma in 28 patients,small cell lung cancer in 8 patients,esophageal cancer in 12 patients,pulmonary tuberculosis in 3 patients and inflammatory pseudo-tumor in 4 patients. Chinese edition of SF-36 was used to evaluate patients' QOL. Cronbach's coefficients (α) and split-half reliability were used to assess its reliability. Its validity was assessed through factor analysis. ResultsCronbach's coefficients (α) of SF-36 were as followed:Physical Functioning (PF) 0.721,Role-Physical (RP) 0.859,General Health (GH) 0.721,Vitality (VT) 0.899,Social Functioning (SF) 0.852,Role-Emotional (RE) 0.872,and Mental Health (MH) 0.598. Split-half reliability of each part was PF 0.725,RP 0.784,GH 0.758,VT 0.749,SF 0.745,RE 0.740,and MH 0.426. Nine principal components were extracted by factor analysis and generally reflected the 8 dimensions of SF-36,which was correspondent to the SF-36 structure. ConclusionSF-36 scale can be used to measure QOL of thoracic surgery patients with good reliability and validity.
ObjectiveTo evaluate the reliability and validity of the instrument of clinical applicability of guidelines (version 2.0). MethodsThe experts of domestic medical institutions were investigated by questionnaire, and the instrument of clinical applicability of guidelines (version 2.0) were evaluated the guidelines for the diagnosis and treatment of tinea mantis and tinea pedis (revised edition 2017) and the guidelines for the diagnosis and treatment of cerebral hemorrhage in China (2019). Using Cronbach's α coefficient and Spearman-Brown coefficient to evaluate the inherent reliability and split-half reliability. The content validity was evaluated by calculating the content validity index of the item level and the adjusted Kappa value. The correlation coefficient between each item and the dimension and the hypothesis test were used to evaluate the convergent and discriminant validity. The structural validity was evaluated by using structural equation model to evaluate the structural validity of the tool. ResultsThe Cronbach's α coefficient and Spearman-Brown coefficient of the instrument of clinical applicability of guidelines (version 2.0) were both greater than 0.7, the content validity index (S-CVI/Ave) were more than 0.8, the success rates of convergent were 100%, and the success rates of discriminant validity calibration were 100% and 96%. In the second-order confirmatory factor analysis model, the χ2/ df were less than 3, the fitting index (CFI), the goodness of fit index (GFI) and the adjustment goodness of fit index (AGFI) were all greater than 0.9. The root mean square residual (RMR) were all less than 0.05, and approximate error root mean square (RMSEA) were less than 0.09. The P value of RESEA hypothesis test were more than 0.05. ConclusionThe instrument of clinical applicability of guidelines (version 2.0) has good reliability and validity, which can be further verified in practical application in the future.