Abstract
Objectives. To characterize community health worker (CHW) performance using an algorithm for managing common childhood illnesses in Siaya District, Kenya, we conducted CHW evaluations in 1998, 1999, and 2001.
Methods. Randomly selected CHWs were observed managing sick outpatient and inpatient children at a hospital, and their management was compared with that of an expert clinician who used the algorithm.
Results. One hundred, 108, and 114 CHWs participated in the evaluations in 1998, 1999, and 2001, respectively. The proportions of children treated “adequately” (with an antibiotic, antimalarial, oral rehydration solution, or referral, depending on the child's disease classifications) were 57.8%, 35.5%, and 38.9%, respectively, for children with a severe classification and 27.7%, 77.3%, and 74.3%, respectively, for children with a moderate (but not severe) classification. CHWs adequately treated 90.5% of malaria cases (the most commonly encountered classification). CHWs often made mistakes assessing symptoms, classifying illnesses, and prescribing correct doses of medications.
Conclusions. Deficiencies were found in the management of sick children by CHWs, although care was not consistently poor. Key reasons for the deficiencies appear to be guideline complexity and inadequate clinical supervision; other possible causes are discussed.
In rural areas of developing countries, several reasons support the use of community health workers (CHWs) as a complement to health facilities as a source of medical care for children. Compared with health facilities, CHWs are geographically closer and available when health facilities are closed; moreover, CHWs are community members, and therefore cultural and linguistic barriers that may be present at health facilities are overcome.
In addition, CHWs can help ensure that treatment at home is appropriate. For example, studies from several countries documented that when a child had a febrile illness, mothers frequently treated their children with an antimalarial at home rather than seeking care at a health facility.1–3 Although treatment was often begun promptly, the dose administered was incorrect (usually too low) 70% to 88% of the time. Providing care through CHWs could improve the dosing of antimalarials administered at home, an approach consistent with World Health Organization (WHO) recommendations.4
CHWs who use simple diagnostic algorithms based on a small number of clinical signs can detect and treat pneumonia,1,5–8 malaria,2,9 and dehydration secondary to diarrhea.10 Some studies have documented that CHWs trained to manage a single disease can reduce childhood mortality.8–11 The singledisease approach is limited, however, because children frequently have symptoms indicating more than one illness.11–14 Studies of CHWs taught to manage multiple diseases suggest that, 1 to 4 years after initial training, CHWs retain some of their clinical competency, and that children in the populations served by the CHWs are more likely to be treated for the illnesses targeted by the CHW intervention.15,16 However, these studies have important limitations, such as small sample size, nonrepresentative samples of CHWs, and no evaluation of CHW performance in the treatment of actual patients.
In 1995, CARE Kenya addressed the need for more comprehensive care at the community level by initiating the Community Initiatives for Child Survival in Siaya project in approximately 200 villages in Siaya District, Kenya. The mortality rate for children younger than 5 years in Nyanza Province, which includes Siaya District, was estimated to be 199 per 1000 live births in 1998.17 The goals of this project, now implemented in 332 villages covering approximately 140 000 persons, are (1) to improve child survival by training and supporting unpaid CHW volunteers, who are literate and selected by their communities; (2) to provide care for children with acute respiratory infections, diarrhea, and malaria; and (3) to refer severely ill children to health facilities. In addition, CHWs counsel caregivers on the continued care of their sick children at home and on behaviors related to health and disease prevention (e.g., immunizations, family planning, and preventing HIV infection).
CHWs were trained to use a simplified version of WHO's Integrated Management of Childhood Illness (IMCI) guidelines.18 IMCI is a global strategy designed to strengthen health systems, implement clinical guidelines in health facilities, and implement community-based interventions that might include care by CHWs. The algorithm for CARE CHWs (available from N. Kukreja at [email protected]) relies on clinical signs and requires minimal equipment (a watch with a second hand) to guide a CHW to a diagnostic classification and an effective treatment. A representative page of the algorithm is shown in Figure 1. Like IMCI, CARE uses separate algorithms for children younger than 2 months and aged 2 to 59 months.
In 1997, 336 CHWs received 3 weeks of initial training from project staff and job aids (clinical registers and flow sheets); in 1998 and 2000, CHWs received refresher training; and throughout the project, CHWs have been supervised. Refresher training lasted for 1 week, was conducted in groups of 8 to 10 CHWs, and included lectures, roleplaying, reviewing videotaped examination findings, and clinical preceptorship at Siaya District Hospital. Note that some changes were made in the algorithm after the start of the program. For example, in 1998, the criteria for the dehydration classifications were revised. Also, the original algorithm provided treatment guidelines (now excluded) if referral was not considered possible. Treatment and counseling job aids were also modified several times.
During the project, 3 surveys of CHW performance were conducted to identify clinical skills needing reinforcement during refresher training and to evaluate the project as a whole. Because little has been published on the ability of CHWs to follow a clinical algorithm addressing multiple illnesses, we present results of these evaluations. Experience with programs in which CHWs manage multiple illnesses is particularly relevant today as more countries consider including CHW interventions in their community component of IMCI.
The study consisted of 3 cross-sectional, hospital-based evaluations in February 1998 (7–13 months after initial training), November 1999 (8–12 months after the first refresher training), and February and March 2001 (3–5 months after the second refresher training). In each evaluation, 120 active CHWs were selected by systematic sampling. In the second evaluation, CHWs were selected from a list of those who had attended the first refresher training (289 of the 336 CHWs initially trained). In the third evaluation, the sampling frame included all 259 CHWs who were active in February 2001, regardless of their participation in refresher training. Selected CHWs were invited to the Siaya District Hospital, the site of the evaluation, and asked to manage up to 4 ill children. CHWs who were unable to make their initial appointment were rescheduled up to 2 times, and CHWs who never appeared for an evaluation were not replaced. Four methods were used for collecting data: (1) observation of consultations using a checklist; (2) CHW documentation of the assessment findings, classifications, and treatments for each sick child on a standard form (i.e., the CHW encounter form) with the same layout as the CHW clinical register used in their home communities; (3) standardized exit interviews with caregivers (usually the mother) to assess the caregiver's understanding of treatment instructions; and (4) repeat examination of children by a study clinician with demonstrated expertise in the CARE algorithm. Additionally, 15 CHW supervisors were evaluated by the same protocol.
Observer training continued until the agreement of practice results of observers and study investigators was greater than 95% (1–3 days). The same 2 study clinicians participated in all 3 evaluations; one clinician (investigator B. O.) was the CHW training officer and the other was a medical assistant selected for her expertise in using the algorithm. For each evaluation, agreement in their clinical examination findings was greater than 90%. Exit interviewers were nurses who were fluent in the local language (Luo). Observers were CARE staff.
Each CHW was evaluated with 1 or 2 outpatients and 1 or 2 inpatients, depending on the availability of sick children. Because few inpatients were available, the same child frequently was seen by more than one CHW.
Definitions for correct classification, treatment, and referral were based on the CARE algorithm. Because the criteria for some classifications changed between evaluations, for each evaluation, we used definitions that corresponded to the guidelines in use at the time of the evaluation. Danger signs were inability to drink or breastfeed, convulsions, “vomiting everything,” and lethargy or unconsciousness. “Gold standard” classifications and treatments were generated with a computer program that applied the CARE guidelines to the assessment findings from the study clinician's repeat examination.
Treatment was defined as “recommended” (all indicated medications were selected and children with a severe classification were referred) or “adequate” (antibiotic, antimalarial, or oral rehydration solution, as indicated, was selected, plus referral for severe classifications).
Data from questionnaires were double entered into an electronic database and verified with Epi Info.19 Indicators of health care quality, which were usually proportions (e.g., the proportion of children with a severe classification who were referred), were estimated with SAS.20 Precision of proportions was estimated with 95% confidence intervals via SUDAAN,21 which adjusts for the potential correlation of CHW performance among the children managed by any specific CHW. Confidence intervals were estimated for all proportions presented in this report and are available from the authors.
In the first, second, and third evaluations, 100, 108, and 114 CHWs were evaluated and 200, 216, and 414 consultations were observed, respectively. In the first 2 evaluations, each CHW was evaluated with 1 outpatient and 1 inpatient. In the third evaluation, CHWs managed 1 or 2 outpatients and 1 or 2 inpatients. To adjust for undersampling, consultations of the 36 CHWs in the third evaluation who were observed managing only 1 outpatient and consultations of the 6 CHWs who managed only 1 inpatient were assigned a weight of 2; all other consultations were assigned a weight of 1.
CHW performance varied depending on the task (Table 1). For example, the sensitivities for obtaining a history of cough or difficult breathing for the 3 evaluations ranged from 80.0% to 88.2%; by contrast, the sensitivity for detecting a slow skin pinch ranged from 17.1% to 25.0%. Although we could not draw conclusions about trends for most tasks because of small numbers of children, in some cases performance appeared to improve or worsen from the first to the third evaluation. Trends were not consistent over time, however.
By the third evaluation, CHWs missed identifying many children with severe classifications (lowest sensitivity = 0% for severe dehydration; highest sensitivity = 57.3% for very severe disease) and moderate classifications (lowest sensitivity = 24.0% for some dehydration; highest sensitivity = 63.9% for malaria) (Table 2). Similarly, few CHWs provided recommended treatments (Table 3). The proportion of children who were adequately treated was somewhat better, ranging from 0% (severe dehydration) to 58.8% (severe pneumonia) for severe classifications and from 50.0% (pneumonia) to 90.5% (malaria) for moderate classifications.
Recommended and adequate treatments, defined as correct drug selection, did not consider drug dosing. When both correct drug selection and dosing were required for treatment to be considered correct, treatment scores were lower. For example, the proportion of children with malaria who received adequate treatment (90.5%) decreased to 66.7% when correct dosing was required, and the proportion receiving recommended treatment decreased from 60.5% to 29.9%. The proportion of caregivers who were able to recall recommended medication instructions for malaria was still lower (9.5%).
To understand the causes of case management problems, we examined the ability of CHWs to process the information they collected (i.e., assessments) into classifications and then to use their classifications to make treatment and referral decisions. In the third evaluation, CHWs correctly processed their assessment findings into classifications for 87.5% of children whose CHW assessments were consistent with a moderate classification; the proportion for children with at least 1 serious classification was 79.7%. Of children given at least 1 severe classification by the CHW, 75.3% received adequate medication for all severe classifications assigned and 83.0% were referred. Of those given at least 1 moderate (but no severe) classification by the CHW, 96.4% received adequate treatments for all moderate classifications assigned.
Counseling about home care, which is required for all children without severe illness, varied by the type of counseling message. In the third evaluation, 62.2% of caregivers were counseled to increase fluids, feeds, or both, 50.5% to return if the child became sicker, and 13.8% to return the next day for a follow-up visit. Counseling for correct medication dosing varied by drug and classification. For example, no CHWs gave correct instructions on making and administering oral rehydration solution to caregivers whose child had diarrhea but no dehydration; analogous results for administering cotrimoxazole (for pneumonia) and paracetamol (for malaria) were 33.3% and 74.7%, respectively. One reason for the low counseling scores was that CHWs did not select the correct medication in the first place and therefore did not provide information on that medication. Furthermore, failure to select the correct medication may ultimately be the result of 1 or more CHW errors in the treatment, classification, or assessment task step. However, their treatment was usually consistent with the classifications they made; for example, when their classification was pneumonia, CHWs taught the caregiver how to give cotrimoxazole 80.4% of the time.
In general, CHW supervisors did not perform substantially better than the CHWs for the assessment of children with at least 1 danger sign (65.0%), classification of children with at least 1 severe classification (55.6%), classification of children with at least 1 moderate classification (66.0%), recommended treatment for children with at least 1 severe classification (55.6%), and recommended treatment for children with at least 1 moderate and no severe classification (48.9%).
In Siaya District, we found deficiencies in the management of sick children by CHWs, although care was not consistently poor. The deficiencies were a result of errors both in performing assessments and in using assessments to choose classifications and treatments. Considering that these CHWs were trained and supported with job aids, refresher training, and supervision for 4 years, why did these errors occur and how could they be remedied?
The first reason why CHWs may have had difficulty managing sick children was the complexity of the guidelines. The guidelines were 6 pages long, contained ambiguities in the clinical algorithm, and had discrepancies between the drug dosing chart and the algorithm; they required CHWs to perform up to 21 assessment tasks per child and contained 14 classifications (although any single child could have had a maximum of only 5 classifications). Furthermore, the guidelines changed twice during the years covered by this study. Not surprisingly, in focus groups conducted before the third evaluation, CHWs commented that inconsistencies in the guidelines confused them. The reason the guidelines were complex is that CARE staff wanted children in the project area to benefit from IMCI-like care being delivered in their home communities; thus, the guidelines preserved much of the complexity of IMCI. Although simpler guidelines would reduce complexity, mistakes occur even when guidelines are simple. In studies of acute respiratory infection treatment programs (i.e., single-disease programs), CHWs recognized chest in-drawing in 80% of children,6 made correct classifications for 81% to 83% of children,6,22 and chose correct treatments for only 71% to 81% of children.7
A second problem concerned supervision. According to supervisory records, fewer than half of CHWs had received one-on-one clinical supervision at a health facility in the past year (results not shown). We have little information on the nature and frequency of other supervisory contacts. We also found that CHW supervisors did not perform substantially better than CHWs in assessments and treatments. These results underscore the importance of monitoring supervisors' skills and providing better support for supervisors.
Third, in focus groups, CHWs expressed concern about criticism from both caregivers and health facility staff if they misclassified a moderate condition as a severe one and unnecessarily referred a child to a health facility. As a result, when CHWs are in doubt, they may tend to choose a less severe classification or decide not to recommend referral even when they assign a severe classification. Although it may be difficult to overcome the natural tendency to avoid situations that may lead to criticism, support from community leaders and supervisors for the CHW who refers when uncertain may prevent mismanagement of life-threatening illnesses. Similarly, it may help to form an alliance with health facility staff and ask them to reassure CHWs, telling them that if they are in doubt, they should refer.
Fourth, CHWs may have lacked confidence in the guidelines. In particular, the ability of CHWs to obtain an accurate history of convulsions declined in successive evaluations; in focus groups, CHWs said they had difficulty because mothers use the term oriere, the Luo word for “convulsions,” when a child was only shivering or startled. Thus, CHWs might have dismissed caregivers' history of convulsions. This, too, is a difficult problem to solve because the best translation for a word like “convulsions” might have a low positive predictive value. However, defining the term more precisely in the guidelines would be helpful (e.g., “If a caregiver says the child has oriere, verify that she did not mean shivering or startled”).
Finally, CHWs may not be getting enough practice using the guidelines. Some CHWs told us the watch they received during the initial training had broken, making it impossible to practice counting respiratory rates. Also, certain signs, such as chest in-drawing, are uncommon. In Papua New Guinea, researchers documented that only 63% of CHWs maintained clinical competency 1 to 4 years after completing training and attributed this loss to a lack of opportunities to practice.15 The project staff of Community Initiatives for Child Survival in Siaya tried to address this problem by holding the second refresher training at the district hospital to provide CHWs with an opportunity to see sicker children on the inpatient ward; however, the ability to recognize dehydration, fast respiratory rate, and chest in-drawing did not improve throughout 3 evaluations.
This study has several limitations. First, the evaluation process might have introduced a bias: the unfamiliar hospital setting could have made CHWs nervous, leading to errors; in contrast, being observed might have motivated CHWs to work more carefully than usual. In addition, although they were instructed to provide treatment as though they were in their home community, CHWs might not have documented the need for referral, because the evaluation was conducted in the hospital setting. Second, the skills of CHWs in some areas could not be meaningfully evaluated because of the small number of children with certain classifications (e.g., severe dehydration). Finally, when data were missing, CHW performance may have been underestimated because a failure to document was interpreted as an error; this problem was especially common in our analysis of drug dosing.
In addition to bringing health services to communities with high child mortality rates, the CARE project has provided an opportunity to examine the strengths and weaknesses of a well-supported CHW intervention. We found serious deficiencies in CHW clinical skills. Our evaluation suggested a number of possible causes, but we are unable to say which is the most important. CHW performance might have been better if their supervisors had mastered the guidelines. Alternatively, with the same supervision, CHW performance might have been better with simpler guidelines. Because frequent, effective CHW supervision may often be difficult to sustain, it would be helpful to test the effectiveness of CHW guidelines that are substantially simpler than those used by CHWs in the CARE project. Such guidelines, for example, might avoid clinical assessment tasks that are difficult for even more experienced clinicians (e.g., chest in-drawing). Treatment guidelines might be reduced to the following: “If fever, give an antimalarial; if fast breathing, give an antibiotic; if diarrhea, give oral rehydration solution.” Similarly, referral guidelines might be reduced to the following: “If there is any other problem, send the child to a health facility; if the child is not better the next day, send the child to a health facility; if the child looks very sick, send the child to a hospital immediately.”
A better understanding of the determinants of CHW performance would help address deficiencies such as those identified in this study. Once we know why CHWs perform well or poorly, we can more effectively target efforts to improve their performance.
The deficiencies in CHW skills that we found should be put in perspective. CHW interventions in other settings have been shown to improve child survival, even if CHW classification and treatment were not always accurate.6,8,10,13,23 In our study, the adequate treatment by CHWs of malaria, the most common condition affecting child morbidity and mortality in Siaya District, was relatively high, at 90.5%. The gaps we found between the clinical performance of CHWs and the CHW algorithm should be seen as part of the broader challenge of ensuring that in both developing and developed countries, health workers in different levels of preservice training adhere to clinical guidelines. For example, wide gaps between practice and guidelines in the care provided have been documented for health workers (including physicians) at health facilities in Benin,24 for IMCI-trained, facility-based health workers in Kenya,25 and for physicians in the United States.26,27
For a full understanding of the role of multiple-disease CHW interventions in the public health systems of developing countries, important questions remain to be answered. To what extent are CHWs truly providing care that children would not otherwise receive, as opposed to providing care that is more convenient? Of the children who would go to a health facility in the absence of CHW care, how many would receive recommended or adequate treatment by the health workers there? And what is the cost-effectiveness of CHW programs compared with the value of increasing access to and the quality of care at health facilities?
Note. CI = confidence interval. aSensitivity equals the number of children with an assessment finding according to the study clinician for whom the CHW identified the assessment finding (n) divided by the number of children with an assessment finding according to the study clinician (N). bFor the third evaluation, sensitivities of assessment tasks other than danger signs are generated only for sick children who do not have a sign of severe illness present. Note. NA = not applicable because these classifications did not exist in the first evaluation; CI = confidence interval. aSensitivity equals the number of children with the classification according to the study clinician for whom the community health worker identified the classification (n) divided by the number of children with the classification according to the study clinician (N). bVery severe disease = presence of any 1 of 4 danger signs or, for children younger than 2 months, presence of fever; for children aged 2 to 59 months, severe pneumonia = cough and a respiratory rate of more than 50 breaths a minute and chest in-drawing; for children younger than 2 months, severe pneumonia = cough and either a respiratory rate of more than 60 breaths a minute or chest in-drawing; pneumonia = cough and respiratory rate of more than 50 breaths a minute among children aged 2 to 59 months; no pneumonia: cough or cold = cough with none of the above abnormal signs present; dehydration = slow skin pinch or irritability in children younger than 2 months; severe dehydration = presence of at least 2 signs of dehydration (irritable or restless, slow skin pinch, or drinks eagerly); some dehydration = presence of 1 sign of dehydration in children aged 2 to 59 months; no dehydration = skin pinch that does not return slowly in alert children with diarrhea; malaria = fever (by history or the child currently feels hot) with no current measles or history of measles in the last 3 months. cFor the first evaluation, severe classifications are very severe disease, complicated measles or measles, severe pneumonia, persistent diarrhea, bloody diarrhea, and dehydration; moderate classifications are pneumonia and malaria. For the second and third evaluations, severe classifications are very severe disease, complicated measles, severe pneumonia, persistent diarrhea, bloody diarrhea, dehydration, and severe dehydration; moderate classifications are pneumonia, malaria, measles, and some dehydration. Note. CI = confidence interval; ORS = oral rehydration solution; NA = not applicable; SP = sulfadoxine pyramethamine. aAt the time of the first evaluation, vitamin A was not listed on the algorithm as a medication to be given for any classification, and community health workers were taught that SP, but not cotrimoxazole, was to be given to treat malaria. bFor the first evaluation, severe classifications are very severe disease, complicated measles or measles, severe pneumonia, persistent diarrhea, bloody diarrhea, and dehydration; moderate classifications are pneumonia and malaria. For the second and third evaluations, severe classifications are very severe disease, complicated measles, severe pneumonia, persistent diarrhea, bloody diarrhea, dehydration, and severe dehydration; moderate classifications are pneumonia, malaria, measles, and some dehydration. cOnly among children with symptoms of cough, diarrhea, or fever according to the study clinician. FIGURE 1— Section of the community health worker algorithm on the management of cough or difficult breathing for children aged 2 months to 5 years: Siaya District, Kenya.
First Evaluation Second Evaluation Third Evaluation (Weighted) Assessment Finding n/N % (95% CI) n/N % (95% CI) N % (95% CI) Danger signs emsp; emsp;Child unable to drink 6/15 40.0 (13.6, 66.4) 3/3 100 … 3 33.3 (0, 83.6) emsp; emsp;Child vomits everything 6/12 50.0 (24.2, 75.8) 1/5 20.0 (0, 55.1) 9 22.2 (0, 49.4) emsp; emsp;Convulsions during this illness 12/13 92.3 (78.0, 100) 26/35 74.3 (59.8, 88.8) 37 51.3 (34.8, 67.8) Child is lethargic or unconscious 4/9 44.4 (12.0, 76.9) 2/4 50.0 (1.0, 99.0) 2 0 … Other assessmentsb emsp; emsp;Cough or difficulty breathing 157/178 88.2 (84.2, 92.2) 151/175 86.3 (81.6, 90.9) 265 80.0 (75.4, 84.7) emsp; emsp;Fast respiratory rate 68/110 61.8 (53.4, 70.2) 74/112 66.1 (58.2, 74.0) 108 40.6 (30.7, 50.5) emsp; emsp;Chest in-drawing present 30/50 60.0 (45.9, 74.1) 7/37 18.9 (6.0, 31.8) 16 52.9 (28.0, 77.9) emsp; emsp;Child has diarrhea 40/60 66.7 (54.6, 78.7) 47/64 73.4 (62.8, 84.1) 132 83.1 (76.8, 89.4) emsp; emsp;Child has bloody stool 5/9 55.6 (23.1, 88.0) 3/9 33.3 (2.5, 64.1) 11 53.3 (21.6, 85.0) emsp; emsp;Skin pinch is slow 1/5 20.0 (0, 55.1) 2/8 25.0 (0, 53.7) 33 17.1 (4.5, 29.8) emsp; emsp;Child drinks thirstily, eagerly, or poorly 3/6 50.0 (10.0, 90.0) 4/15 26.7 (2.8, 50.5) 19 4.5 (0, 13.4) emsp; emsp;Child is irritable or restless 1/2 50.0 (0, 100) 0/5 0 … 5 0 … emsp; emsp;Child feels hot or has history of fever 153/175 87.4 (83.0, 91.9) 170/192 88.5 (83.8, 93.3) 301 88.7 (84.9, 92.5) Summary measure: CHW recognizes ≥1 sign 27/43 62.8 (49.0, 76.5) 29/43 67.4 (53.6, 81.3) 71 54.7 (43.0, 66.3) emsp; emsp;of severe illness in children with ≥1 sign emsp; emsp;of severe illness (a danger sign or fever emsp; emsp;in children aged <2 months) First Evaluation Second Evaluation Third Evaluation (Weighted) Classificationb n/N % (95% CI) n/N % (95% CI) N % (95% CI) Very severe disease 28/43 65.1 (52.6, 77.6) 27/43 62.8 (48.0, 77.6) 71 57.3 (46.1, 68.6) Severe pneumonia 11/37 29.7 (13.8, 45.6) 5/28 17.8 (3.8, 31.9) 16 52.9 (28.0, 77.9) Pneumonia 17/51 33.3 (21.3, 45.3) 36/66 54.5 (43.9, 65.2) 97 31.5 (21.9, 41.2) No pneumonia: cough or cold 10/54 18.5 (8.1, 29.1) 24/52 46.1 (33.2, 59.0) 152 39.3 (31.2, 47.3) Severe dehydration NA … 0 … 14 0 … Some dehydration NA … 2/8 25.0 (0, 53.7) 20 24.0 (4.6, 43.4) Dehydration 2/5 40.0 (0, 82.9) 0 … 0 … No dehydration 4/39 10.3 (0.6, 19.9) 19/40 47.5 (32.0, 63.0) 75 40.2 (29.5, 50.9) Malaria 55/133 41.3 (33.8, 48.9) 103/152 67.8 (61.2, 74.3) 299 63.9 (58.3, 69.4) Summary measures emsp; emsp;All severe classifications identified among children with ≥1 severe classificationc correctly 41/83 49.4 (39.3, 59.5) 34/76 44.7 (33.4, 56.0) 122 51.1 (42.4, 58.9) All moderate classifications identified among children with ≥1 moderate classificationc correctly 63/139 45.3 (37.8, 52.8) 115/156 73.7 (66.9, 80.5) 308 66.7 (61.3, 72.0) First Evaluation Second Evaluation Third Evaluation (Weighted) Classification and Treatmenta n/N % (95% CI) n/N % (95% CI) N % (95% CI) Very severe disease Recommended treatment (referral and cotrimoxazole and paracetamol if fever and child is aged ≥2 months and ORS if diarrhea and vitamin A if diarrhea and child is aged ≥6 months) 20/43 46.5 (32.5, 60.5) 18/43 41.9 (34.5, 56.4) 71 30.7 (19.9, 41.5) emsp; emsp;Adequate treatment (referral and cotrimoxazole and ORS if diarrhea) 23/43 53.5 (38.3, 68.7) 23/43 53.5 (46.1, 68.0) 71 41.3 (29.8, 52.8) Severe pneumonia emsp; emsp;Recommended treatment (referral and cotrimoxazole and vitamin A if child is aged ≥6 months) 22/37 59.5 (43.8, 75.1) 3/28 10.7 (4.9, 22.1) 16 47.1 (21.8, 72.3) emsp; emsp;Adequate treatment (referral and cotrimoxazole) 22/37 59.5 (43.8, 75.1) 3/28 10.7 (4.9, 22.1) 16 58.8 (34.5, 83.2) Pneumonia (with no severe classification) emsp; emsp;Recommended treatment (cotrimoxazole and vitamin A if child is aged ≥6 months) 28/48 58.3 (45.6, 71.1) 38/66 57.6 (52.3, 67.9) 92 39.4 (29.0, 49.8) emsp; emsp;Adequate treatment (cotrimoxazole) 28/48 58.3 (45.6, 71.1) 43/66 65.1 (59.8, 75.6) 92 50.0 (39.1, 60.8) Some dehydration (with no severe classification) emsp; emsp;Recommended treatment (ORS and vitamin A if child is aged ≥6 months) NA … 5/8 62.5 (43.4, 100) 20 60.0 (37.6, 82.4) emsp; emsp;Adequate treatment (ORS) NA … 6/8 75.0 (60.3, 100) 20 84.0 (68.9, 99.1) Malaria (with no severe classification) emsp; emsp;Recommended treatment (SP or cotrimoxazole and paracetamol if child is aged ≥2 months and vitamin A if child is aged ≥6 months) 38/95 40.0 (29.6, 50.4) 86/124 69.3 (65.5, 76.9) 262 60.5 (54.6, 66.5) emsp; emsp;Adequate treatment (SP or cotrimoxazole) 39/95 41.0 (30.6, 51.5) 115/124 92.7 (90.3, 97.4) 262 90.5 (86.9, 94.1) Summary measures emsp; emsp;All severe classificationsb correctly treated in children with ≥1 severe classification emsp; emsp;Recommended treatment (referral and recommended drugs) 49/90 54.4 (45.0, 63.8) 22/76 28.9 (23.8, 39.0) 122 30.5 (22.4, 38.6) emsp; emsp;Adequate treatment (referral and adequate drugs) 52/90 57.8 (48.1, 67.4) 27/76 35.5 (30.2, 46.0) 122 38.9 (30.3, 47.5) emsp; emsp;Recommended drug(s)c 56/83 67.5 (58.1, 76.8) 46/72 63.9 (58.2, 75.0) 120 52.7 (43.5, 61.9) emsp; emsp;Adequate drug(s)c 60/83 72.3 (63.4, 81.2) 52/72 72.2 (67.0, 82.3) 120 63.6 (54.7, 72.5) emsp; emsp;Referral 63/90 70.0 (60.9, 79.1) 32/76 42.1 (36.5, 53.0) 122 52.7 (44.1, 61.2) emsp; emsp;All moderate classificationsb correctly treated in children with ≥1 moderate classification and no severe classifications emsp; emsp;Recommended drug(s) 28/101 27.7 (18.6, 36.8) 71/128 55.5 (51.4, 63.5) 271 48.4 (42.4, 54.3) emsp; emsp;Adequate drug(s) 28/101 27.7 (18.6, 36.8) 99/128 77.3 (73.8, 84.3) 271 74.3 (68.7, 79.9) 
All authors helped plan the study. J. M. Kelly, B. Osamba, R. M. Garg, M. J. Hamel, J. J. Lewis, and S. Y. Rowe conducted the fieldwork. All authors worked on the analysis, and J. M. Kelly, A. K. Rowe, and M. S. Deming cowrote the paper.
This research was funded by a grant from the R. W. Woodruff Foundation to CARE and the CDC Foundation, as part of the CARE–CDC Health Initiative.
The authors thank the project staff of the Community Initiatives for Child Survival in Siaya and Ms Olewnya Sabenzia for their assistance in these evaluations.
