Development and Psychometric Properties of the Periodontal Diseases Assessment Index in Pregnancy for Midwives

Document Type : Original Research Article

Authors

1 Associate Professor, Health Sciences University, Head of Midwifery Department, Ankara, Turkey

2 Associate Professor, Department of Periodontology, Faculty of Dentistry, Karabuk University, Karabuk, Turkey

Abstract

Background & aim: Periodontal diseases cause adverse obstetric outcomes in pregnancy. However, pregnant women generally avoid going to the dentist. This study was conducted to assess the reliability and validity of Periodontal Diseases Assessment Index (PEDIAP) in Pregnancy for Midwives in Turkish society.
Methods: This is a validation study consisted of 111 pregnant women referred to antenatal clinic of a training hospital in western Turkey in 2017. Construct validity of the index was calculated by exploratory and confirmatory factor analyses (EFA, CFA), reliability by Cronbach's alpha coefficient, inter-applicant agreement with Kendal W coefficient and intraclass correlation coefficient (ICC) and factor structure with principal components analysis and Varimax rotation using SPSS for Windows 20.0 software.
Results: Results showed that the index explained 33.94% of the total variance, and the factor loading values were between 0.47 and 0.68. Based on the CFA, fit indices were found to be χ2/sd = 1.6384, RMSEA = 0.0762, and CFI = 0.9795. The regression coefficients and t values were significant (t > 1.96). The index was found to have a positive correlation with (CPITN) scores (p < .001). The Cronbach alpha coefficient was calculated to be 0.774. It was found that the Kendal W coefficient ranged from 0.550-1 for each item, the ICC coefficient was 0.869, and correlation was significant (p < 0.001). The cut-off value of the index was specified as 3.5
Conclusion: The results of this study show that the PEDIAP is a valid and reliable index for use in research and midwifery care.

Keywords

Main Subjects


Introduction

Periodontal disease is a common progressive disease characterized by inflammatory cell accumulation that affects teeth and the tissues surrounding the teeth (1). The main cause of periodontal diseases is microbial dental plaque. However, cases affecting inflammatory response, such as genetic susceptibility, environmental factors, and systemic diseases, play a significant role in the formation and exacerbation of the disease. One of the conditions in which the inflammatory response change is pregnancy. Therefore, pregnant women are at risk for periodontal diseases (1-2).

Hormonal, immunological, and vascular changes that occur during pregnancy negatively impact the teeth and gums (2-3). Estrogen hormone during pregnancy facilitates plaque adhesion on dental surfaces, disturbs the collagen mechanism, and reduces keratinization of the gums, weakening their epithelial barrier properties. Progesterone hormone increases prostaglandin (PGE1-PGE2) levels with a pro-inflammatory effect in the gingival crevicular fluid. It disrupts the metabolism of fibroblasts in periodontal ligaments and reduces glycosaminoglycan synthesis. And it increases vascular permeability and makes the gingiva prone to bleeding. Moreover, progesterone increases the breakdown of tissue proteins by reducing plasminogen activator inhibitor type 2 (PAI-2) in gingival tissue. Progesterone is also responsible for the reduction of cellular immunity (3-4). Subgingival microflora also changes during pregnancy. In oral flora, the proportion of anaerobic microorganisms to aerobic microorganisms increases. Certain periodontal pathogens such as Prevotella intermedia, Bacteriodes species and Campylobacter rectus increase. Especially, Prevotella species use hormones in the form of steroids as growth factors. These changes in microflora lead potentially to periodontal disease associated with an inflammatory response when accompanied by oral hygiene deficiency (2). Ehler et al. (2013) have investigated the gingival inflammation symptoms and enzyme activity in gingival crevicular fluid. They have reported that pocket depths, enzyme aMMP-8 values, and gingival inflammation increased in pregnant women compared to non-pregnant women, and 80% of the pregnant women and 40% of the non-pregnant women had gingival inflammation (5).

Periodontal diseases affect the fetus directly through the translocation of bacterial products or indirectly through the activation of maternal inflammatory mediators. Periodontal diseases can cause early pregnancy loss, abortion, preterm birth, low birth weight, and preeclampsia (1,6-11).  The research shows that periodontal disease may be an independent risk factor for prematurity and low birth weight (12). More incidence of attachment loss has been reported in the mothers of babies born with low birth weight. In the study of Offenbacher et al. (1998), mothers of infants with low birth weight had severe periodontal disease and higher levels of PGE2 and interleukin-1 beta (IL-1β) in the gingival crevicular fluid. Periodontal disease has been identified as a risk factor for low birth weight (13). Lopez et al. (2005) showed that low birth weight incidence rates decreased in pregnant women treated for periodontal disease (14). Vivares-Builes et al. (2018) have reported a relationship between clinical indicators of periodontal disease such as bleeding during probing, probing depth, loss of attachment, and negative neonatal outcomes (DDA, preterm birth, and preeclampsia) (15).

These results point to the importance of protecting pregnant women from periodontal diseases and treating them in the early stages. However, pregnant women do not present to the dentist for reasons such as fear of harming the baby or thinking that people should not be treated while they are pregnant, even if they have gum problems (16-18). From this point of view, midwives need to evaluate women in terms of periodontal diseases during routine pregnancy checks. When offering pregnancy and maternity services, midwives should be able to assess, identify and appropriately approach all risks. Identification of pregnant women with periodontal problems by midwives and referral of them to dentists will be an important preventive service in the reduction of adverse obstetric consequences stemming from such diseases. This is because the most important component of studies aimed at controlling oral and dental health problems during pregnancy is to provide pregnant women with dentistry services (19). No index in the literature can be used by health professionals other than dentists to determine the presence of periodontal diseases. This study aims to develop a valid and reliable index for midwives to evaluate the presence of periodontal disease in pregnant women and to refer suspected to the dentist.  This study was conducted to develop the “Periodontal Diseases during Pregnancy Assessment Index for Midwives” and to examine its psychometric properties in Turkish society.

Materials and Methods

This validation study was designed to assess the reliability and validity of the “Periodontal Diseases Assessment Index (PEDIAP) in Pregnancy for Midwives-”. This study was carried out in the antenatal clinic of a training and research hospital in western Turkey between March 1 and 30, 2017.In this study, the sample consisted of pregnant women who presented to the hospital due to routine pregnancy follow-ups and volunteered to participate in the study. Exploratory factor analysis was conducted with data obtained from 55 pregnant women. The sample group to which confirmatory factor analysis and other statistical analyses were applied consisted of 111 pregnant women. For factor analysis to be effective, the sample size should be between 3 and 10 times larger than the number of items in the scale. Because the index contained 5 items, a minimum of 100 survey forms had to be filled in. The purposive sampling method was used in this study to reach a sample representing the range of the measured feature in the study group. Pregnant women who smoked and refused to participate in the study were excluded from the study.

The data of this study were collected using the draft form of the “PEDIAP” and the “Community Periodontal Index of Treatment Needs (CPITN).”

Preparation of the Periodontal Diseases Assessment Index in Pregnancy forMidwives-PEDIAP draft Form”: The index has been prepared in Turkish by the researcher. The properties of healthy gums were taken into consideration when forming the index items. In this respect, healthy gums should appear like orange peel, have a light pink color, have a regular gum line that wraps the teeth at the root heads and follows their contours, should not bleed during brushing or flossing, should be firm, and should appear hard (20,21). In line with these properties, the index was planned to consist of one dimension and a total of 5 items. The first item of the index asks the patient if there is bleeding in the gums during brushing. In the other 4 (four) items of the index, the midwife observes the gums of the pregnant woman by lifting the lips with the help of a small spatula and marks the appropriate option for each item based on the result of the observation. Each item of the index is rated 0, 1or 2, and one can score at least “0” points and at most “10” points on the index. After the index items were formed, the index was sent for expert opinion to 3 periodontologist academicians and 2 academics midwives. .  The opinions of the experts were scored as “not appropriate (1 point)”, “somewhat appropriate (2 points)”, appropriate but minor changes required (3 points)” and “very appropriate (4 points)”. After expert opinions, the Content Validity Index of the scale was found to be 0.97. The scale items were finalized in line with the experts’ recommendations for minor changes.

 “Community Periodontal Index of Treatment Needs (CPITN)” was utilized to evaluate the criterion-related validity of the index being developed. CPITN is an assessment system proposed by WHO in 1977 that assesses the need for periodontal treatment in the community. According to CPITN, a periodontologist divides the jaw into 6 sections and examines it by employing a WHO periodontal probe. During the examination, the probe is placed in the gingival pocket with a force not exceeding 20 g. The ball at the tip of the probe is moved around the pocket in such a way that it can detect tartar in the pocket, and the depth and morphology of the pocket. By determining the highest values in each section, the person’s need for treatment is determined. Scoring by CPITN  values is as follows (22).

  • 0 Points: Healthy periodontal tissue (no need for treatment).
  • 1 Points: After touching it with the probe, the presence of bleeding is visible directly to the eye or a mouth mirror (Oral hygiene should be improved).
  • 2 Points: Feeling tartars or the presence of overhanging restoration edges during the examination with the probe. The entire black band of the probe is visible (Scaling and root planning).
  • 3 Points: 4–5 mm periodontal pocket presence is detected. (Scaling and root planning).
  • 4 Points: 6 mm or deeper pocket presence. The black band of the probe is completely invisible (Complex treatment).

In the study, CPITN was used only by the periodontist. The obtained CPITN score was accepted as the gold standard for the developed scale.

A pilot study was carried out to check the clarity of the items before data collection in the study. During the pilot study, two midwives assessed a total of four pregnant women by using the index: two with periodontal disease and two who did not have periodontal disease. After the pilot study, no changes were made to the index items. Data collected during the pilot study were excluded from the scope of the study. Following that, the implementation phase of the study began.

The data of this study were collected by three midwives and one periodontologist. A room was created in the obstetrics clinic of the hospital to collect the data for the study. An oral examination of pregnant women was performed in this room. Women who applied to the hospital for routine pregnancy follow-ups were informed about the purpose and importance of the study and the procedures to be performed on them. Afterward, the pregnant women who accepted to participate in the study were directed to the room where the examination would take place. With adequate lighting and ventilation facilities, this room consisted of two intertwined sections.  The pregnant woman was first taken to the first section and seated on a chair. Then 3 (three) different midwives took turns entering the room in different times and the midwives performed the oral examination of the pregnant women using PEDIAP. Then, the pregnant women were taken to the second part of the room and the periodontist first examined the pregnant women using PEDIAP and then CPITN. During the oral examination of pregnant women, a person who was not from the research team took the pregnant women and midwives into the room, numbered the forms, and stapled the forms together. Thus, the midwives and the periodontist did not see the forms filled by others and were not influenced by each other's examination findings.

The study design was reviewed and approved by the local ethics committee (Number: 2015/09, dated 18/09/2015).  By with the Declaration of Helsinki, oral and written consent about the design, aim, and clinical implication of our study was obtained from all participants. Finally, only the voluntary participants were included in the study. Moreover, all pregnant women were given oral hygiene training by the periodontist, and the ones who were discovered to have periodontal problems were referred for treatment.

 SPSS for Windows 20.0 software (Statistical Package for Social Sciences) was used in data analysis.  Exploratory and confirmatory factor analyses were carried out to test the construct validity of the draft form of “PEDIAP.” Kaiser-Meyer-Olkin (KMO) and Bartlett’s Sphericity values were investigated to test the suitability of the data for factor analysis. The scale’s factor structure was investigated using Principal Components Analysis and Varimax rotation, and explanatory factor analysis. EFA was conducted with a sample of 55 participants. Then, this matter confirmatory factor analysis (CFA) was carried out to assess whether the structure of the index that was being developed was validated. CFA was conducted with a sample of 111 participants.  The correlation of the index with CPITN was examined to assess the criterion-related validity of the index, and whether the answers given to the items could be summed was tested by carrying out a Tukey test. The agreement among the operators was examined by using the Kendal W coefficient and the Intraclass Correlation Coefficient (ICC). Cronbach alpha internal consistency reliability coefficient was calculated to determine the reliability of the index. ROC analysis was carried out to specify a cut-off score for the index. The significance level was set at p < 0.05.

Results

The mean age of the pregnant women who participated in the study was 27.84 ± 8.03 years, and the mean gestational week was 24.5 ± 3.2 weeks. Of the pregnant women, 32.1% were in the first trimester of pregnancy, 15.4% in the third trimester, and 52.5% in the second trimester.

Exploratory and confirmatory factor analyses were carried out in the order given to assess the construct validity of the index. Basic assumptions (outliers, normality, and multicollinearity) in multivariate statistics were tested before exploratory factor analysis (EFA). Kaiser-Meyer-Olkin (KMO) and Bartlett’s Sphericity values were investigated to test the suitability of the data for factor analysis. The KMO value for the data that were obtained from the sample was found to be 0.64. The result of Bartlett’s Test of Sphericity was significant (χ2 = 30.869, p = 0.001). In order to determine the factor structure of the scale, the data obtained from 55 participants were subjected to exploratory factor analysis (EFA) using Principal Component Analysis and Varimax rotation. In this respect, it was determined that the items of the draft form of the index explained 33.94% of the variance, and the factor loading values of the pattern matrix were between 0.47 and 0.68. Then, confirmatory factor analysis (CFA) was carried out to assess whether the structure of the index that was being developed was validated. Figure 1 shows the path diagram. According to the CFA results, χ2 = 4.9152, χ2/sd = 1.6384, Root Mean Square Error of Approximation (RMSEA) = 0.0762, Comparative Fit Index (CFI) = 0.9795. An examination of the coefficients showing the relationship between the observed variables and factors of the model presenting the factorial structure of the index revealed that all coefficients were sufficient.

Table 1 shows the regression values and t values of the items of the index. Based on Table 1, it was determined that the resulting regression coefficients and t values were significant (t > 1.96), and the model was validated.

The correlation between CPITN and the draft form of “PEDIAP” was examined to determine the criterion-related validity of the index. Accordingly, the total scores of the index determined by all midwife operators and the periodontist showed a positive and statistically significant correlation with the total scores of CPITN determined by the periodontist through dental examination (p < 0.001) (Table 2).

In this study, whether the answers given to the index items could be summed was examined through a Tukey test.  It was seen that the answers marked by all practices could be summed (p > 0.05) (Table 3).

The Cronbach alpha coefficient, which is an internal consistency coefficient, was calculated to determine the reliability of the index. The Cronbach alpha value of the index was determined to be 0.774. In this study, because the draft items of the “PEDIAP” were rated by 3 (three) midwives and 1 (one) periodontologist, the agreement among the operators was assessed within the scope of reliability studies. Accordingly, when the Kendal W coefficient for each item was examined, it was seen that it ranged from 0.550 to 1.

Furthermore, when the correlation between all operators was examined, the correlation coefficient between all operators was 0.869 according to the ICC (Intraclass Correlation Coefficient) result, and the resulting correlation was statistically significant (p < 0.001) (Table 4).

CPITN was used as the gold standard to determine the cut-off point of the index. For this, the periodontologist examined all participants using the CPITN. The result of this examination was categorized by the periodontologist in 2 groups: need for treatment/ (CPITN results: 0 and 1) / no need for treatment (CPTN results: 2, 3, and 4).

These two categories have been accepted as the gold standard and then, ROC analysis was performed to determine the cut-off score of the index. Accordingly, the cut-off value of the index was specified as 3.5 Roc analysis was performed twice with the data obtained from the first and last practitioner midwives. It was determined that the same cut-off point was 3.5 in both. . Figures 2 give the ROC curves. When the data obtained from the PEDIAP application of the periodontologist and the data obtained from the first and last practitioner midwives were compared, it was determined that the sensitivity, specificity and PPV/NPV values of the cut-off point of the index were completely compatible for both midwives.

Discussion

This study was carried out to develop a valid and reliable index for midwives to evaluate pregnant women in terms of gum diseases during routine follow-ups. The factor analysis is carried out to evaluate the validity of a measurement instrument. Kaiser-Meyer-Olkin (KMO) and Bartlett’s Sphericity values are examined to assess the suitability of data before conducting factor analysis. KMO value greater than 0.60 and a result of Bartlett’s Test of Sphericity at p < 0.050 indicates that the results are significant (23-24). In this study, the KMO value for the data that were obtained from the sample was found to be 0.64. The result of Bartlett’s Test of Sphericity was significant (χ2 = 30.869, p = 0.001). These results indicate that the data were suitable for factor analysis, that the data had a normal multivariate distribution, and that the sample size was sufficient.

Exploratory factor analysis (EFA) is a method commonly used for specifying the structural characteristics of a scale. As the value of total variance that is explained in EFA increases, the power of factors increases. Based on the EFA results, none of the items was excluded from the questionnaire because none had a factor loading below 0.30 (23-24). Moreover, it has been reported that in scales with a single factor, it is sufficient that at least 30% of the total variance is explained (24-25). The EFA results of the “PEDIAP” in the study show that 33.94% of the total variance was explained, and the factor loading values of the pattern matrix were between 0.47 and 0.68. These results suggest that the index being developed is a good measurement instrument.

In this study, confirmatory factor analysis (CFA) was carried out to determine whether the structure of the “PEDIAP” was validated and to examine the data–model integrity. Fit indices are examined to assess the validity of a model in CFA. The RMSEA was smaller than .08, and χ2/sd ≤ 2, which indicates a good fit of the data. A CFI value within the .90–.95 range indicates that the model is satisfactory (26). In our study, based on the CFA, fit indices were found as χ2/sd = 1.6384, RMSEA = 0.0762, and CFI = 0.9795. ased on the Confirmatory factor analyses show that the structure of the index that was developed is verified and that the data–model integrity has a high level of compliance.  In this study, the regression coefficients and t values were significant, and the model was validated.

The correlation between the “PEDIAP” and the results of the dental examinations the periodontist performed using CPITN was examined to determine the criterion-related validity of the index that was tried to be developed in this study. Accordingly, the index scores of all operators (three midwives and one periodontist) had a positive and statistically significant correlation with the CPITN total scores (p < 0.001). This correlation result indicates that the “PEDIAP” is a suitable tool for evaluating periodontal diseases.

In this study, a Cronbach alpha coefficient was calculated and the agreement among the operators was taken into consideration to determine the reliability of the “PEDIAP.” An increase in Cronbach alpha coefficient equates with an increase in internal consistency, and it has been reported in the literature that a Cronbach alpha coefficient between 0.60 and 0.80 is reliable (23,26-27). The Cronbach alpha value of the PEDIAP was determined to be 0.774. This result suggests that the scale is reliable.

In the study, PEDIAP was applied by both three midwives and one periodontist. The periodontist first applied PEDIAP and then performed the oral examination. The study examined the compatibility among all PEDIAP practitioners. It was determined that the Kendal W coefficient varied between 0.550 and 1, and there was a high level of agreement between the operators for each item. When the correlation among the practitioners was examined, the ICC coefficient was 0.869, and the correlation was found to be statistically significant.

Finally, a cut-off point was determined for the index, which was found to be valid and reliable. The pregnant women were divided into two categories according to the CPITN results, as those who needed treatment and those who did not need treatment, and ROC analysis was carried out. And the cut-off score of the index was determined as 3.5. Midwife 1 had the lowest agreement according to the cut-off score and gold standard, and midwives 2 and 3 had a full agreement.

The strongest aspect of this study is that the data were collected by three different midwives who had not received any training on the subject, the agreement of these midwives with each other and with the periodontology specialist was examined, and there was a good level of agreement between all practitioners.

The most important limitation of this study was that the sample size was small (n = 111) (for five items). It was very difficult to achieve a large sample size when collecting data for the development of the index because each pregnant woman was examined by four different people at each session. A sample size is considered sufficient if it is 10 times larger than the number of items on the scale or arguably five times larger than the number of items on the scale or no less than 100 (28). In this context, it is thought that the sample size is sufficient.

Conclusion

The results of this study show that the “PEDIAP” is a valid and reliable measurement instrument and can be employed in research and midwifery care.    The most important component of studies aimed at controlling oral and dental health problems during pregnancy is the referral of pregnant women with problems to dentists.  The PEDİAP is an index that can be used by midwives to identify pregnant women with periodontal problems and refer them to dentists the cut-off score of the index is 3.5, and pregnant women who score 3.5 or above because of the examination should be referred by midwives to the dentist with suspected periodontal disease.

Acknowledgement

 This work was supported by Research Fund of the Karabuk University. Project Number: KBÜ-KAP-15/2-KP-066. We would like to thank their support and all the participants in this study.

Conflicts of interest

Authors declared no conflicts of interest.

  1. Xiong X, Buekens P, Fraser WD, Beck J, Offenbacher S. Periodontal disease and adverse pregnancy outcomes: a systematic review. An İnternational Journal of Obstetrics and Gyneacology2006; 113(?): 135-143.
  2. Armitage GC. Bi‐directional relationship between pregnancy and periodontal disease. Periodontol. 2013; 61(1): 160-176.
  3. Mascarenhas P, Gapski R, Al-Shammari K, Wang HL. Influence of sex hormones on the periodontium. Journal of Clinical Periodontology  2003; 30(8): 671-681.
  4. Üstün K, Özdemir EÇ, Sezer U, Şenyurt S, Cebesoy FB, Erciyas K. Hamile Bireylerde Periodontal Farkındalık ve Dental Anksiyetenin Sosyo-Demografik Veriler Işığında Değerlendirilmesi. Annals of Health Sciences. Res. 2013; 2(5): 5-14
  5. Ehlers V, Callaway A, Hortig W, Kasaj A, Wıllershausen B. Clinical parameters and aMMP-8-concentrations in gingival crevicular fluid in pregnancy gingivitis. Clinical   2013; 59(5-6): 605-611.
  6. Scannapieco F, Bush R, Paju S. Periodontal disease as a risk factor for adverse pregnancy outcomes. A systematic review. Annals Periodontology . 2003; 8(1): 70-78.
  7. Bulut G, Olukman O, Calkavur S. Is there a relationship between maternal periodontitis and pre-term birth? A prospective hospital- based case-control study. Acta Odontologica   2014; 72(8): 866-873.
  8. Rangel-Rincón LJ, Vivares-Builes AM, Botero JE, Agudelo-Suárez AA. An umbrella review exploring the effect of periodontal treatment in pregnant women on the frequency of adverse obstetric outcomes. Journal Evidence Based Dental Practice. 2018; 18(3): 218–239.
  9. Offenbacher S, Beck JD. Periodontitis: a potential risk factor for spontaneous preterm birth. Compendium of Continiuing Education in Dentistry. 2001; 22(2 Spec No): 17-20.
  10. Offenbacher S, Lieff S, Boggess KA, Murtha AP, Madianos PN, Champagne CME, Herbert WNP. Maternal periodontitis and prematurity. Part I: Obstetric outcome of prematurity and growth restriction. Annals Periodontology  2001; 6(1): 164-174.
  11. Bansal J, Bansal A, Kukreja N, Kukreja U. Periodontal diseases as an emerging potential risk factor for adverse pregnancy outcomes: A review of concepts. Journal of  Turkish Ger Gynecology Association 2011; 12(3): 176-180.
  12. Rashidi Maybodi F, Amirzade Iranaq MH. Association between Poor Oral Health and Fertility Problems: A Narrative Mini-Review. Journal of Midwifery and Reproductive Health. 2017; 5(1): 849-854. DOI: 10.22038/jmrh.2016.7708
  13. Offenbacher S, Jared HL, O’Reilly PG, Wells SR, Salvi GE, Lawrence HP, Socransky SS, Beck JD. Potential pathogenic mechanisms of periodontitisassociated pregnancy complications. Annals Periodontology.  1998; 3(1): 233-250.
  14. Khader Y, Al-shishani L, Obeidat B, Khassawneh M, Burgan S, Amarin ZO, Alomari M, Alkafajei A. Maternal periodontal status and preterm low birth weight delivery: a case-control study. Archive Gynecology and Obstetrics. 2009; 279(2): 165-169.
  15. Vivares-Builes AM, Rangel-Rincón LJ, Botero JE, Agudelo-Suárez AA. Gaps in Knowledge About the Association Between Maternal Periodontitis and Adverse Obstetric Outcomes: An Umbrella Review. Jorunal of Evidence -Based DentalPractice. 2018; 18(1): 1-27.
  16. Kısa S, Zeyneloğlu S. Inpatient postpartum women’s status of oral hygiene habits and visit to the dentist during their most recent pregnancy TAF Prevention  Medicine Bulletin. 2013; 12(1):64-75
  17. Detman LA, Cottrell BH, Denis Luque MF. Exploring dental care misconceptions and barriers in pregnancy. Birth. 2010; 37(4): 318-332.
  18. Ressler-Maerlender J, Krishna R, Robison V. Oral health during pregnancy: Current Research. Jounal of Womens Health. 2005; 14(10): 880-882.
  19. Chung LH, Gregorich SE, Armitage GC, Gonzales-Vargas J, Adams SH Sociodemographic disparities and behavioral factors in clinical oral health status during pregnancy. Community Dentistry of Oral Epidemiology. 2014; 42(2): 151-159.
  20. Mariotti A. Dental plaque‐induced gingival diseases. Annals Periodontology. 1999; 4(1): 7-17.
  21. Denissen H, Kuijkens A, Ozić A. A photographic method to measure the colour characteristics of healthy gingiva. International Journal of  Dental Hygiene. 2007; (1): 22-26.
  22. Lewis JM, Morgan MV, Clive Wright FA. The validity of the CPITN scoring and presentation method for measuring periodontal conditions. Journal of Clinical Periodontology. 1994; 21(1):1-6.
  23. Büyüköztürk Ş. Data Analysis Handbook (In Turkish: Veri Analizi El Kitabı.  24th Edition,  Ankara; Pegem Akademy, 2018.
  24. Kline R B.  Principles and practice of structural equation modeling.  4th Edition. Guilford Press. 2016.
  25. Şencan H.  Reliability and Validity in Social and Behavioral Measurements (In Turkish: Sosyal ve Davranışsal Ölçümlerde Güvenirlik ve Geçerlilik, Ankara; Seçkin Publishing, 2005 .
  26. Schreiber JB, Nora A, Stage FK, Barlow EA, King J. Reporting structural equation modeling and confirmatory factor analysis results: A review. Journal of Educational Research. 2006; 99(6): 323–338.
  27. Erdoğan S, Nahcivan N, Esin N. Research in Nursing Process, Application and Critical (In Turkish: Hemşirelikte Araştırma Süreç, Uygulama ve Kritik). Nobel Medicine Publishing, 2014.  
  28. Hatcher L. A Step-by-Step Approach to Using the SAS System for Factor Analysis and Structural Equation Modeling. SAS Institute, Inc., Cary.