The Satisfaction Assessment Tool for Obstetrics Triage (SATOT): Development and Psychometric Properties

Introduction

Triage is the preliminary evaluation process for delivering high quality, timely, effective services based on the client’s clinical conditions and available resources (1). The obstetrics triage units provide inaccessible medical care in emergencies (2). The obstetrics triage is a multifunctional unit which include pregnant women admission, fetal assessment, acute midwifery emergencies, performance evaluation, and various methods of gynecology and midwifery (3-5). Obstetrics triage is one of the major concerns of health care providers and managers (3). However, proper triage will increase the quality and efficiency of services (6, 7).  

The quality of services is one of the most important health priorities (8). According to the Donabedian model, client’s satisfaction is a consequence of quality of health services (9). Therefore, satisfaction with services is an important indicator of the quality of services (10). Quality is a multidimensional concept, and clients’ satisfaction is an important dimension of the quality of health services (11). Clients’ satisfaction is a set of experiences  about health care system; it is essential to continuously assess satisfaction for planning of health services and facilities (12). In recent years, quality assessment of health services has been increasingly important for the patients, as it is the clients' right to evaluate the services provided (13). Therefore, service providers should regularly assess the clients’ satisfaction and use high reliable and valid tools to meet clients' needs (10, 14). Evaluation of the organizational performance provides a base for improvement and correction of the procedures, increasing productivity and satisfaction. It also identifies efficient and effective activities and challenges (15).

Studies have shown that triage environment and the personnel’s’ behavior affect the clients' satisfaction with obstetrics triage (16-18). There is a limited information on the clients’ satisfaction with obstetrics triage; one reason for the lack of such information may be the lack of a valid and reliable tool for assessment of the satisfaction with obstetrics triage. Therefore, this study was performed aimed to design and evaluate psychometric properties of a tool to assess the clients’ satisfaction with obstetrics triage.

Materials and Methods

This validation study is a part of an exploratory sequential mixed study which was performed in two stages of qualitative (designing the concept of satisfaction with obstetrics triage) and quantitative (examining psychometric properties of the satisfaction assessment tool for obstetrics triage).

Designing the procedure: The four stage approach proposed by Waltz and colleagues (2010) was used to design the preliminary tool (19). 

The First step: In this step the concept and dimensions of satisfaction with obstetrics triage were described. The inclusion criteria were the clients of obstetric triage services in teaching hospitals affiliated to the Shahid Beheshti University of Medical Sciences, obstetric triage personnel and key informants who had tendency to participate in the study.

In this stage, the semi-structured interviews were conducted with 23 participants including clients of obstetrics triage, triage personnel, emergency specialist, gynecologist, the officer in charge of the Department of Maternal Death, emergency department supervisor from different hospitals in various parts of Tehran.

Participants were selected with maximum diversity in terms of age, education, and field of the study. The interviews started with a general question: ‘How was the obstetrics triage services? Then it continued with more specific questions: ‘What do you think about the ways to improve the quality of obstetrics triage services? ‘In your opinion, how is the status of obstetrics triage?’.  The duration of each interview was 15-20 minutes. The interviews were continued until data saturation. Data were analyzed based on the steps described by Graneheim and Lundman (20). The interviews were recorded and transcribed immediately after each interview. All interviews were typed and then entered into the MAXQDA10 software. The transcripts of the interviews were coded by the researcher using an open coding system to extract the codes and categories. For this purpose, the texts of the interviews were first divided into meaning units, and then different codes were extracted and categorized based on their similarities and differences.

Then, using the deductive approach, a comprehensive review was done by the use of the key terms of triage, obstetrics, midwifery triage, satisfaction, questionnaire, scale, index, and checklist in databases of PubMed, Proquest, Science Direct, Ovid, Google Scholar, Magiran, Iranmedex, Irandoc databases. Inclusion criteria were the related Persian or English articles, without time limit. Articles were then reviewed and the items related to the quality of obstetric triage were added to the preliminary item pool SATOT. The initial tool was prepared for evaluating of psychometric properties.

 The second step: In this step the objectives for designing obstetrics triage satisfaction tool were explicated.

 The third step: In third step the initial map was designed.

The fourth step: Finally, a measurement structure including items selection and scoring was created for the "Satisfaction Assessment Tool for Obstetrics Triage".

In the first stage: to describe different aspects of the concept of satisfaction with obstetrics triage, a conceptual model was used using inductive (data obtained from qualitative method) and deductive (data obtained from reviewing literature) approaches. After performing these steps, the designed tool was assessed regarding psychometric properties.

Assessment of psychometric properties: At this stage, face-, content-, and construct validity of SATOT (Satisfaction Assessment Tool for Obstetrics Triage) was assessed. Then, reliability of SATOT was evaluated by assessing the internal consistency and stability.

Face validity assessment: Face validity was assessed by qualitative and quantitative methods. In the qualitative face validity assessment, the items of the questionnaire were evaluated by ten women referring to the obstetrics triage regarding the items’ difficulty level, appropriateness, and ambiguity. For quantitative face validity assessment, the items impact scores were measured. Ten participants were asked to rate the importance of each item by the five-point Likert scale. The item was recognized as suitable for subsequent analysis if the impact score was ≥1.5 (21).

Content validity assessment: Qualitative and quantitative methods were used to determine the content validity of SATOT. In reviewing the qualitative content validity, the researcher asked emergency gynecologists, obstetricians, and midwives to comment on the items’ clarity, simplicity, grammar use, appropriate words, placement and scoring (21). To calculate the content validity ratio (CVR), 15 experts in emergency medicine, gynecology, reproductive health and midwifery were asked to rank each item based on the three-point scale, "necessary”, “useful but not necessary”, and “not necessary” to check the necessity of the item. The items with CVR values ​​above 0.49 were retained based on the Lawshe table for 15 evaluators (22).

For investigating the content validity index, the relationship of the items to the purpose of the research was evaluated. To determine this index, 15 experts (emergency medicine, gynecology, reproductive health, midwifery) were asked to examine the relevance and adequacy of the items based on the four-point Likert scale. In this study, two indices, Item Content Validity Index (I-CVI) and Scale Content Validity Index (S-CVI) were calculated. The I-CVI represents the ratio of agreement on the relevance of each item. It is obtained by dividing the number of experts who rated the relevance or appropriateness of item 3 or 4 divided by the total number of experts. The index validity score above 0.79 is appropriate, 0.7-0.79 is questionable and needs to be revised and a score below 0.7 is unacceptable and should be deleted. The S-CVI acceptance criterion is 0.9 as the highest criterion and 0.8 as the lower content acceptance threshold (19).

Constract validity assessment: Exploratory factor analysis was used to determine the construct validity of the tool. In this study, regarding to 10 items, 200 questionnaires were completed by the individuals referring to the obstetrics triage (23). The convenience method was used for sampling.

Inclusion criteria were all clients referring to the obstetrics triage at Shahid Beheshti University of Medical Sciences teaching hospitals in Tehran.

The questionnaires were completed by those referring to the obstetrics triage of hospitals in north, south, east, and west areas affiliated to Shahid Beheshti University of Medical Sciences (the hospitals of Taleghani, Mahdyieh, Emam Hossein, Loqman, and Shohadaye Tajrish). The Kaiser-Meyer-Olkin (KMO) (24) test was used to determine the adequacy of the sample. The latent factors were extracted using maximum likelihood analysis and Promax rotation assuming the independence of the factors. In this model, two factors with eigenvalues higher than one were extracted. The minimum factor loading required for retaining each item was 0.4.

Reliability Assessment: Reliability of SATOT was evaluated by assessing the internal consistency and stability. The internal consistency was evaluated by measuring Cronbach's alpha coefficient) 30 clients of obstetrics triage services). Alpha coefficient above 0.7 was considered acceptable (25). Stability of the questionnaire was assessed using test retest method and calculating intra-cluster correlation coefficient (ICC). The questionnaire was completed by 15 clients of obstetrics triage in two stages with two weeks’ interval and then the scores obtained in two stages were used to calculate ICC. The optimal stability of the tool was considered ICC ≥0.7 (26).

The interpretability floor and ceiling effects were assessed. A ceiling effect occurs when a high proportion of subjects in a study have maximum scores on the observed variable. This discriminates the subjects among the top end of the scale impossible, and the floor effect arises when a high proportion of subjects select the responses on the lower limit. The index should be less than 20% to include all criteria and show changes over time. In this study, the ceiling and floor effects were calculated for the total score of the questionnaire and the scores of all subscales as a percentage to evaluate the scale discrimination and response distribution (27).

Data were analyzed using SPSS software (version 24). P<0.05 was considered statistically significant.

Ethical approval was obtained from the Ethics Committee, Faculty of Pharmacy and Nursing Midwifery, Shahid Beheshti University (ID:  IR.SBMU.PHARMACY.REC.1399.067). Informed written consent was obtained from all participants. They were assured that their personal information would be kept confidential and they can leave the study whenever they wished.

Results

Primary questionnaire were designed in the following 4 steps of Waltz (19). 

The first step: Using the results of a qualitative study, the concept of satisfaction with obstetrics triage and its dimensions were explained. The initial items extracted from the qualitative interview included: satisfaction with the triage personnel, providing the necessary explanations by the personnel, the triage personnel’s behavior, existing tools and equipment, availability of tools and facilities, and physical structure.

The second Step: The theoretical definition of satisfaction was "Satisfaction with obstetrics triage which is a multidimensional concept and involves satisfaction with structure and process".

The third step: The initial map was composed of the items resulting from a qualitative study. Then, with an extensive review of the literature, the items related to satisfaction with obstetric triage that did not exist in the qualitative stage were added to the primary tool. Thus, the initial map of the tool was designed.

Based on the findings of a qualitative study and review of literature and similar tools, 10 items were extracted. The items were placed as follows: Process of Satisfaction (5 items), Structure of Satisfaction (5 Items). Each item was scored by a five-point Likert scale (very unsatisfied, unsatisfied, neutral, satisfied, and very satisfied). The primary instrument was psychometric Properties.

The fourth step: A 5-point Likert scale (scored 1 to 5) with responses of very unsatisfied, unsatisfied, neutral, satisfied, and very satisfied was formed.  Therefore, the total scores for the ten items ranged 10 to 50. The scores of items were summed up for calculation of total scores and then calculated scores were converted to 0 to 100. The following conversion formula was used to convert the scores of the sub-scales and the entire questionnaire to a score of 0 to 100. Adjusted score = (The raw score obtained-minimum possible score/maximum possible-minimum possible score) ×100.Higher scores shows more clients’ satisfaction with the triage services.

Psychometric Properties

Face validity: An item was modified (grammatically) in the investigation of qualitative face validity according to the opinion of 10 client's referring to the obstetrics triage. In determining quantitative face validity, the maximum impact of the item was obtained as five and the minimum impact score was 3.5, and no item was eliminated at this stage.

Content validity: There were no revisions in the investigation of the qualitative content validity. Quantitative content validity assessment showed CVR above 0.49 for all items. The modified content validity index (K^*) I-CVI for all items ranged 0.86 to 1, and the S-CVI score was calculated as 0.97. 

Structural validity: Adequacy of sample size shown by KMO test was 0.89 that was at the acceptable level, and Bartlett test was statistically significant (p <0.0001).

Construct validity: Using exploratory factor analysis with Promax rotation showed two factors. The eigenvalues ​​of the two extracted factors were 5.30 and 1.03, respectively. The two extracted factors account for about 63.40% of the total variance of the variables of SATOT. Thus, after performing factor analysis once, the items were assigned to the factor that had the highest factor loadings. Also, an item with a factor loading less than 0.4 was eliminated (Table 1).

Figure 1 showed the scree plot diagrams explaining the components of the obstetrics triage satisfaction questionnaire.

Table 2 illustrated the two factors extracted from factor analysis using the Promax rotation and the factor loadings of the items.

Internal consistency: Cronbach's alpha for the whole tool was 0.793, and Cronbach's alpha coefficient in case of omitting items was obtained (Table 2).

 Stability: The intra-cluster correlation coefficients for the whole tool and the two subscales of process and structure were 0.884, 0.823, and 0.846, respectively.

Finally, the first factor with four items and the second factor with five items were determined. Subsequently, each of the factors was named based on their items of process and structure satisfaction, which was consistent with the findings of the qualitative section (Table 3).

The ceiling effect of the tool was 5% and the floor effect was 0.5%. The ceiling effect and floor effect for the whole tool as well as the individual tool areas were listed in Table 4.

To facilitate responsiveness in this study, precise item analysis and factor analysis were performed to prevent tool prolongation. Also, no items were unanswered.

The items of the SATOT scored by the five-point Likert scale of very unsatisfied, unsatisfied

(2), neutral, satisfied, and very satisfied, which were scored from 1 to 5, respectively. The SATOT score converted to percent using the formula  ×100. The higher scores demonstrate more satisfaction.

Discussion

This study presented a reliable and valid tool for assessment of satisfaction with obstetrics triage for the first time in Iran and the world. The questionnaire consists of two subscales of structure and process. Obstetrics triage satisfaction is defined as the clients' understanding from the concept of satisfaction with the structure, space, and activities of the triage services, and how they are performed based on the guidelines. Since satisfaction from services is an important indicator in evaluating the quality of services (9, 18, 28) and obstetrics triage is one of the major concerns of hospitals and the health system, it is essential to develop a valid and reliable obstetrics triage satisfaction tool to identify the causes of clients' dissatisfaction and specify the needs for appropriate planning and therefore improve their satisfaction and the quality of services (18).

Although it is critical to measure triage satisfaction, the main criticism is that many of the tools used to measure patient's satisfaction are not valid and reliable (standard), while the instrument must have sufficient power to determine the results (29).

In the present study, the satisfaction with the process was less than the satisfaction with the structure. This finding indicates that the process of service delivery is more important for the clients than the structure and physical space of service delivery. According to the Donabedian model, the path to assess the quality of care starts with the structure (having the right objects and supplies), then moving on to the process (doing the right things) and eventually ending in the outcome (doing the right thing and satisfaction) (30). Assuming the provision of the minimum appropriate structure, the process is an essential element in quality assurance (9). Continuous attention to the process can achieve the desired results even if the causal relationship between care delivery processes and their end-results is not consistent (31).

In the process satisfaction dimension, the highest factor loading is related to talking with triage personal about the client’s anxiety and fear. The results of studies have shown that two important elements in the patient-centered relationship are drawing out and responding to the patient's real fears and concerns. Removing all patient's concerns in the first referral is as easy as this question: "Is there anything else you're worried about?” Although the pressures of lack of time are real, service providers, even if they have enough time, often ignore paying attention to the fears and worries of clients (32).

In the structure satisfaction dimension, the highest factor loading is on the physical space. The results of studies have shown that physical environment and space are important factors in satisfaction with triage and a proper and appropriate triage system and orderly and comfortable space can enhance satisfaction (33).

Since no studies have been conducted to design and psychometrically evaluate tools for obstetrics triage satisfaction, the findings of this study are compared with those of similar tools. McKinley et al. (34) designed a tool for assessment of patients’ satisfaction with receiving out of hours primary care. The tool consists of 32 items (20 are related to teleconsultation), and 6 subscales (Satisfaction with communication and management, physician attitudes, waiting time, continuity of care, access to out-of-hours care, and telephone counseling).  Its nonresponse was 96.5% with 61% variance. The internal consistency of several subscales is less than 0.7 (34).

Garratt et al. (35) designed a tool to measure patients' experiences for out-of-hours primary care. The tool consists of 24 items (10 are related to teleconsultation), and 4 subscales (medical services, call contact and organization, nursing services, and unanswered questions). Its nonresponse rate was 96.5% with 79% variance. Internal consistency was higher than 0.80 (35). Moll van Charante et al. (36) designed a tool to assess patients' satisfaction with out-of-hours primary health care. The tool consists of 66 items (14 are related to teleconsultation), and 3 subscales. The construct validity was assessed by PCA and the percentage of variance for each factor was 77.72 and 89%, respectively. Its response rate for teleconsultation was 36-57% and internal consistency was above 0.70 for all factors.  Content and face validity information was not mentioned (36).

Compared to existing tools, SATOT tool not only examines satisfaction with the way tasks are handled and the client is treated during the triage process (satisfaction with the process), but also examines satisfaction with the physical structure, space, and equipment of the obstetrics triage, while other tools focus only on the service delivery process. On the other hand, SATOT tool is very short compared to the size of other tools.

The questionnaire used in the present study can be considered as a new questionnaire in Iran and the world. One of the strengths of this questionnaire is that its main constructs have been extracted and identified after defining the concept of satisfaction with obstetrics triage through a qualitative content analysis approach. One of the limitations of the present study is the lack of convergent validity due to the lack of similar tools in this field.

Conclusion

SATOT is a valid and reliable and easy-to-use questionnaire to assess clients’ satisfaction with obstetrics triage. The questionnaire consists of nine items with two dimensions (process and structure of satisfaction) and measures by the five-point Likert scale. This questionnaire may help triage personnel and managers to identify the problems and challenges in the structure and process of obstetrics triage that lead to dissatisfaction and inappropriate outcomes and may help them to take appropriate actions.

Acknowledgements

The authors appreciate the cooperation and assistance of the officials of Taleghani, Mahdyieh, Emam Hossein, Loqman, and Shohadaye Tajrish Educational Hospital. We are greatly thankful to the partipants of the study.

Conflicts of interest

Authors declared no conflicts of interest.