|Year : 2018 | Volume
| Issue : 1 | Page : 40-43
An alternative approach for supportive supervision and skill measurements of health workers for integrated management of neonatal and childhood illnesses program in 10 districts of Haryana
Arun K Aggarwal1, Rakesh Gupta2, Dhritiman Das1, Anar S Dhakar1, Gourav Sharma1, Himani Anand1, Kamalpreet Kaur1, Kiran Sheoran1, Suresh Dalpath2, Jaidev Khatri2, Madhu Gupta1
1 Department of Community Medicine, Postgraduate Institute of Medical Education and Research, Chandigarh, India
2 National Health Mission, Panchkula, Haryana, India
|Date of Submission||28-Dec-2016|
|Date of Acceptance||05-Aug-2017|
|Date of Web Publication||13-Feb-2018|
Dr. Arun K Aggarwal
Department of Community Medicine, School of Public Health, Postgraduate Institute of Medical Education and Research, Chandigarh
Source of Support: None, Conflict of Interest: None
| Abstract|| |
Context: “Integrated Management of Neonatal and Childhood Illnesses” (IMNCI) needs regular supportive supervision (SS). Aims: The aim of this study was to find suitable SS model for implementing IMNCI. Settings and Design: This was a prospective interventional study in 10 high-focus districts of Haryana. Subjects and Methods: Two methods of SS were used: (a) visit to subcenters and home visits (model 1) and (b) organization of IMNCI clinics/camps at primary health center (PHC) and community health center (CHC) (model 2). Skill scores were measured at different time points. Routine IMNCI data from study block and randomly selected control block of each district were retrieved for 4 months before and after the training and supervision. Statistical Analysis Used: Change in percentage mean skill score difference and percentage difference in median number of children were assessed in two areas. Results: Mean skill scores increased significantly from 2.1 (pretest) to 7.0 (posttest 1). Supportive supervisory visits sustained and improved skill scores. While model 2 of SS could positively involve health system officials, model 1 was not well received. Outcome indicator in terms of number of children assessed showed a significant improvement in intervention areas. Conclusions: SS in IMNCI clinics/camps at PHC/CHC level and innovative skill scoring method is a promising approach.
Keywords: Child health, Integrated Management of Neonatal and Childhood Illnesses, supportive supervision, training
|How to cite this article:|
Aggarwal AK, Gupta R, Das D, Dhakar AS, Sharma G, Anand H, Kaur K, Sheoran K, Dalpath S, Khatri J, Gupta M. An alternative approach for supportive supervision and skill measurements of health workers for integrated management of neonatal and childhood illnesses program in 10 districts of Haryana. Indian J Community Med 2018;43:40-3
|How to cite this URL:|
Aggarwal AK, Gupta R, Das D, Dhakar AS, Sharma G, Anand H, Kaur K, Sheoran K, Dalpath S, Khatri J, Gupta M. An alternative approach for supportive supervision and skill measurements of health workers for integrated management of neonatal and childhood illnesses program in 10 districts of Haryana. Indian J Community Med [serial online] 2018 [cited 2020 Nov 23];43:40-3. Available from: https://www.ijcm.org.in/text.asp?2018/43/1/40/225358
| Introduction|| |
Achievement of child survival goal (MDG 4) demands urgent attention. Integrated Management of Neonatal and Childhood Illnesses(IMNCI) trainings can improve the health workers' performance,,, but require effective supervision.,,, Without this, knowledge and skills diminish over time.,,
Various models of IMCI/IMNCI supervision have been tried globally; but it is still unclear which model of supervision is better suited to the health system. There was a need to explore simpler methods for IMNCI skill measurement and supervision that can be incorporated uniformly across the country. Haryana state had implemented IMNCI very early on. However, implementation lacked effective monitoring and supervision. Thus, this project was initiated.
Specific objectives of the study were
- To assess the performance of the health workers using established tools during training and supervision
- To workout strategy to implement supportive supervision (SS) and
- To measure the impact of the training using routine health system data.
| Subjects and Methods|| |
The study was conducted in ten high-priority districts of Haryana.
The study duration was 8 months (December 2013–July 2014)
Longitudinal prospective study design was employed. Three-day IMNCI training was given to health workers of one block in each district. Their skills were scored five times (pretest, posttest 1, posttest 2, SS round 1, and SS round 2). Outcome of the training implementation was measured in terms of the number of children assessed and reported as per IMNCI, on monthly web portal report, before and after the training. These data were captured from routine health management information system, for 4 months preceding intervention, for 4 months after the intervention for both intervention and control blocks of the same district. Control block was chosen randomly from each district.
Skill measurements in classroom setting
Globally, IMCI/IMNCI training is done using video films using the same case studies. We used two of these video case studies to assess the skills of the workers before and after the training. Workers used IMNCI case recording forms, along with IMNCI chart-booklet, and other materials such as photo booklet and module in local Hindi language. They were shown a video case study and were asked to fill the IMNCI case recording form and do the disease classifications accordingly. Posttest 1 was done after the training using the same video case study. Discussions on the case were done after the completion of posttest 1. This was followed by running a second video case study for posttest 2. Participants' performances were scored out of 10. The scores were based on several criteria such as correct identification of condition and correct treatment identification.
Skill measurements in field setting
Health workers were allocated children for assessment, disease classification, and to record findings on the same structured IMNCI case recording forms that were used in the trainings. Project supervisors observed the entire process, and at the end, scored the performance using the same scoring criteria as was used during the trainings and scored out of 10 marks. Additional observations were made to score the communication skills of the workers and were scored out of 5 for 0–2-month children and out of 10 for children 2 months–5 years. Thus, total scores were out of 15 for 0–2 months and 20 for 2 months–5 years. However, these scores were then converted to 10. Mistakes made by the workers were corrected on site through demonstration and redemonstration. Shortcomings in communication and nutritional counseling were also observed and addressed.
Organization of supportive supervision visits
To minimize the gap between trainings and supervision and to involve the district health system officials right from the planning stage, it was envisaged that district officials will prepare a supportive supervisory schedule in consultation with the health workers and project officials during the training itself, so that in the month following the training, supervision can be done.
Following two models for SS were followed that evolved during the implementation:
Model 1: Supervisors (project coordinators) visited health centers at subcenter (SC) level after 2–3 weeks of training with preinformation to the health workers to make due preparations for the event. Cases of <5 children were arranged from the daily outpatient department. Accredited Social Health Activist and Anganwadi worker also fetched children from concerned villages. Participants examined children and wrote prescription in front of supervisors. Gaps in the practices if any were corrected on the site.
Model 2 – Auxiliary nurse midwives from different SCs were called to primary health center (PHC)/community health center (CHC) for supervision. They were allocated cases in the health center for assessment and prescription. Area medical officer also participated. This event was later termed as IMNCI clinic.
The second round of SS was undertaken within approximately 1 month following the completion of the first round.
Measurement of training impact on the health workers performance
Health workers report the data of number of children assessed as per the IMNCI every month. These numbers were taken for 4-month preintervention period and similarly for 4 months in the postintervention period. Data were captured for both the intervention block where IMNCI trainings were imparted and the control block which was randomly selected from the same district with ongoing routine activities.
An excel file was created to compile the skill scores of all participants from all the districts before training (pretraining score), after the training (posttraining score 1 and posttraining score 2), and during supervisions (SS round 1 and SS round 2). The data were assessed for normality using Kolmogorov–Smirnov test 1 and thereafter, pre- and post-comparisons of scores were done using paired students t-test. Missing values of pretest, posttest 1, and posttest 2 were treated by imputing mean values as the number of missing values was few. For making paired comparison with supervisory data, pairs with missing values were not entered into analysis.
From routine Health Management Information System portal of the National Rural Health Mission, Haryana, data on IMNCI case assessment were retrieved block wise from all these 10 districts. Data of intervention block were compared with the control block (where training had not been imparted by us, but routine health services are implementing the program). Trainings were done in the month of December and supervision was done thereafter. We retrieved data for 4 months clearly before the intervention (April–July 2013) and 4 months after the intervention (April–July, 2014). The difference in the number of children assessed (0–2 months and 2 months–5 years) in these 4 months of pre-and post-training/supervision was calculated for both control and intervention areas. The difference was expressed as percentage and median value was calculated out of this difference percentage, and thereafter, compared for control and intervention areas using Mann–Whitney U-test.
| Results|| |
A total of 240 participants were trained. About 216 participants attended the pretraining assessment test, 229 attended 1st posttraining assessment test, and 232 attended the 2nd posttraining assessment test. Two rounds of SS followed this, wherein 204 and 176 trainees participated, respectively.
Mean skill scores in all five-group observations improved significantly after the trainings and supervision. The mean score at pretest was 2.1 (95% confidence interval [CI]: 1.9–2.2, 21% of the total score). The score improved from 21% to 70% at posttest 1 (mean 7.0, 95% CI: 6.7–7.2), and to 81% at posttest 2 (mean 8.1, 95% CI: 7.8–8.2).
About 2–3 weeks after the end of training, there was a downward deflection of 10% score at SS round 1 from posttest 2 (mean 7.0, 95% CI: 6.6–7.2), which thereafter improved to 80% at SS round 2 from SS round 1(mean 8.0,95% CI: 7.7–8.2) [Figure 1]. The increase in scores was tested for statistical significance using paired students t-test [Table 1].
|Figure 1: Group-wise mean skill scores of health workers from pretest to postsupervision|
Click here to view
|Table 1: Grouped comparison of mean differences* in skill scores, using paired t-test, during trainings and supervisions|
Click here to view
Number of children assessed and reported pre- and post-intervention
Difference in the median number of children assessed during two time periods was calculated for each of the intervention and control group and expressed as percentage. For young infants aged 0–2 months, median increase was 0.15% in control areas and 57.7% in intervention areas (P < 0.01). For children between 2 months and 5 years of age, the increase was 30.8% in control and 98.3% in intervention blocks (P < 0.01) [Table 2]. The impact of the training in the number of children assessed was found to be highly significant.
|Table 2: Median difference in number of children assessed per month* during April-July 2013 and April-July 2014 in the intervention and control blocks|
Click here to view
| Discussion|| |
Use of training videos to assess the skills in classroom setting was innovative, useful, and seems to be replicable, as case studies and case recording forms are uniform across the country. The scoring system can be further refined and validated. We tried two supervisory models and found that model of supervision in the camp/clinic setting is more productive and acceptable to the health system officials.
Low pretest scores were startling considering that all these participants had presumably received full IMNCI training in the past and were supposed to be already implementing the same in their areas. The poor scores may be attributed to either inadequate or the absence of IMNCI supervision or trainings and motivation. Our refresher trainings significantly improved the skill scores. Significant decline (~11%, P < 0.001) in scores within 1 month of the training suggests that without SS, the score would have eventually come down to baseline over a period of time. SSs acted as a booster leading to a rise in scores signifying that supervisory rounds were successful in bringing to halt the logic of natural reduction in skills overtime. It has been well documented that there is a proportional relationship between the gap in training and supervision and skill loss.,,,
In the present study, conventionally recommended method of supervision where supervisors visited the health workers' place of posting required more number of visits, in addition to exhaustive planning and immense efforts. Despite the availability of funds to visit these health centers, there was an overall reluctance in the health system to undertake this activity, thereby, resulting in an almost negligible response rate.
Second approach required initiation of monthly IMNCI clinics at PHC or CHC level. It eliminated the need for the medical officer to move out from health center. Trained health workers of his/her area visited the center on the predecided IMNCI camp date and examined under-five children in the presence of project supervisors and the medical officers. This approach was found to be more acceptable thereby increasing the participation of medical officers. However, a long-term follow-up can establish the sustainability of this approach.
Higher number of children assessed in the intervention blocks is encouraging. This is unlikely to be due to any reason other than the intervention itself since both the blocks were in the rural area of same districts, thereby eliminating the possibility of differences arising from socioeconomic milieu, administrative capacity as well as constraints and strengths of the rural blocks.
There are many strengths of this study. Being a grass-root implementation research is a major strength of this study. The need for assessing the impact of SS was identified by the state, which partnered at every stage of the implementation apart from funding and providing administrative push at many stages. The use of an innovative and virgin method of refresher training and skill scoring and the ease with which it can be used by the future program implementers makes this study novel in its own regards. Moreover, the use of two different approaches of supervision to assess the constraints and strengths and the acceptability of the same in the routine health system supervisory structure further strengthen the basis of this study.
Even though the strengths are many, certain possible limitations of the study are also worth noticing. The use of routine data for measuring the impact of the training and the absence of any special data collection may be argued to be a source of unreliability. However, considering that this was an implementation research, use of routine data can be considered as strength as well. Weakness in the routine data would be equally applicable to the control and intervention blocks. Any change over a period in the routine data is worth noticing. A second possible limitation could be that impact data have been taken from the report of complete block although not all the workers of the intervention block were trained so far, signifying that the current impact could possibly be an underestimation. With more number of workers trained, output will be expected to improve further.
| Conclusions|| |
Current refresher training package with method to assess skill scores and IMNCI camp approach for SS can be successfully applied in the health system. However, it still needs to stand the test of time.
Financial support and sponsorship
Conflicts of interest
There are no conflicts of interest.
| References|| |
United Nations' Children Fund. UNICEF Annual Report; 2013.
Nguyen DT, Leung KK, McIntyre L, Ghali WA, Sauve R. Does Integrated Management of Childhood Illness (IMCI) training improve the skills of health workers? A systematic review and meta-analysis. PLoS One 2013;8:e66030.
Kumar D, Aggarwal AK, Kumar R. The effect of interrupted 5-day training on integrated management of neonatal and childhood illness on the knowledge and skills of primary health care workers. Health Policy Plan 2009;24:94-100.
Bhandari N, Mazumder S, Taneja S, Sommerfelt H, Strand TA, IMNCI Evaluation Study Group et al.
Effect of implementation of Integrated Management of Neonatal and Childhood Illness (IMNCI) programme on neonatal and infant mortality: Cluster randomised controlled trial. BMJ 2012;344:e1634.
Mohan P, Kishore B, Singh S, Bahl R, Puri A, Kumar R, et al.
Assessment of implementation of integrated management of neonatal and childhood illness in India. J Health Popul Nutr 2011;29:629-38.
Horwood C, Voce A, Vermaak K, Rollins N, Qazi S. Experiences of training and implementation of Integrated Management of Childhood Illness (IMCI) in South Africa: A qualitative evaluation of the IMCI case management training course. BMC Pediatr 2009;9:62.
Anand K, Patro BK, Paul E, Kapoor SK. Management of sick children by health workers in Ballabgarh: Lessons for implementation of IMCI in India. J Trop Pediatr 2004;50:41-7.
Hoque DM, Arifeen SE, Rahman M, Chowdhury EK, Haque TM, Begum K, et al.
Improving and sustaining quality of child health care through IMCI training and supervision: Experience from rural Bangladesh. Health Policy Plan 2014;29:753-62.
Venkatachalam J, Kumar D, Gupta M, Aggarwal AK. Knowledge and skills of primary health care workers trained on integrated management of neonatal and childhood illness: Follow-up assessment 3 years after the training. Indian J Public Health 2011;55:298-302.
] [Full text]
Chaudhary N, Mohanty PN, Sharma M. Integrated Management of Childhood Illness (IMCI) follow-up of basic health workers. Indian J Pediatr 2005;72:735-9.
Simoes EA, Desta T, Tessema T, Gerbresellassie T, Dagnew M, Gove S, et al.
Performance of health workers after training in integrated management of childhood illness in Gondar, Ethiopia. Bull World Health Organ 1997;75 Suppl 1:43-53.
Shewade HD, Aggarwal AK, Bharti B. Integrated Management of Neonatal and Childhood Illness (IMNCI): Skill assessment of health and Integrated Child Development Scheme (ICDS) workers to classify sick under-five children. Indian J Pediatr 2013;80:448-54.
Bharani S, Parmar T, Kantharia N, Parmar R, Kharod N. A study to assess the implementation and effectiveness of IMNCI program and evaluation of skills of rural anganwadi workers in Vadodara taluka. Natl J Commun Med 2012;3:207-12.
Rowe AK, Osterholt DM, Kouamé J, Piercefield E, Herman KM, Onikpo F, et al.
Trends in health worker performance after implementing the integrated management of childhood illness strategy in Benin. Trop Med Int Health 2012;17:438-46.
Shrivastava SR, Shrivastava PS. Evaluation of trained Accredited Social Health Activist (ASHA) workers regarding their knowledge, attitude and practices about child health. Rural Remote Health 2012;12:2099.
[Table 1], [Table 2]