|Year : 2018 | Volume
| Issue : 2 | Page : 107-112
Use of computer-assisted personal interviewing and information management system in a survey among hiv high-risk groups in India: Strengths, weaknesses, opportunities, and threats analysis
Radhika Brahme1, Sheela Godbole2, Raman Gangakhedkar2, Kuldeep Singh Sachdeva3, Vinita Verma3, Arun Risbud2
1 Department of Epidemiology, National AIDS Research Institute; Department of Health and Biomedical Science, Symbiosis International University, Pune, Maharashtra, India
2 Department of Epidemiology, National AIDS Research Institute, Pune, Maharashtra, India
3 Department of Evaluation and Research, National AIDS Control Organization, New Delhi, India
|Date of Submission||24-Oct-2017|
|Date of Acceptance||10-Apr-2018|
|Date of Web Publication||18-May-2018|
Mrs. Radhika Brahme
National AIDS Research Institute, Post Box: 1895, 73-G Block, MIDC Bhosari, Pune - 411 026, Maharashtra
Source of Support: None, Conflict of Interest: None
| Abstract|| |
Objectives: In India, integrated biological and behavioral surveillance was carried out in 2014–2015 among high-risk key population as a part of second-generation HIV surveillance system. Computer-assisted personal interviewing and integrated information management system were used for the first time in this large national field based survey. We evaluated the strengths and weaknesses of technology use in this survey. Methods: Mixed methods comprising of the key informant's interviews and structured data collected from field interviewers were used to do the strengths, weaknesses, opportunities, and threats analysis with defined attributes. Results: Despite the challenges, the technology use in this survey was a huge success with respect to data coverage, response rates, real-time data, and acceptance by respondents. However, such techniques require more focus on the competency of human resource, training, and concurrent evaluation systems to get better data quality, time adherence, and effective use of technology. Conclusion: The recommendations resulted from this analysis will help for strategic management while designing such systems in field-based community surveys.
Keywords: Computer-assisted personal interviewing, computer-assisted survey research, HIV surveillance, strengths, weaknesses, opportunities, and threats analysis
|How to cite this article:|
Brahme R, Godbole S, Gangakhedkar R, Sachdeva KS, Verma V, Risbud A. Use of computer-assisted personal interviewing and information management system in a survey among hiv high-risk groups in India: Strengths, weaknesses, opportunities, and threats analysis. Indian J Community Med 2018;43:107-12
|How to cite this URL:|
Brahme R, Godbole S, Gangakhedkar R, Sachdeva KS, Verma V, Risbud A. Use of computer-assisted personal interviewing and information management system in a survey among hiv high-risk groups in India: Strengths, weaknesses, opportunities, and threats analysis. Indian J Community Med [serial online] 2018 [cited 2019 May 26];43:107-12. Available from: http://www.ijcm.org.in/text.asp?2018/43/2/107/232767
| Introduction|| |
The HIV surveillance carried out by National AIDS Control Organization (NACO) in India is one of the largest surveillance systems in the world. The HIV epidemic in India is concentrated in high-risk populations, and adequate surveillance of behavioral and biological factors is crucial to control the spread of the epidemic. In India, there are cultural, linguistic, and geographic diversities, and hence, it is extremely challenging to conduct such surveys among hidden high-risk populations. NACO carried out a national bio behavioural survey namely Integrated Biological and Behavioural Surveillance (IBBS) among the five high-risk group subpopulations in 2014–2015. A similar, integrated biological and behavioral assessment (IBBA) was carried out between 2005 and 2010 across six high HIV prevalence states. In IBBA, paper-based questionnaires were used for data collection. This was a labor-intensive process, wrought with challenges with respect to data collection, management, and report generation. In contrast, the IBBS used technology with real-time data entry and management to collate data on all aspects of the survey.
Computer-assisted personal interviewing (CAPI) is a technique used for data collection on a portable device. In the last decade, CAPI has been popularly used in social research because of its cost-effectiveness, ease of use, and immediate availability of data. The use of CAPI in large scale survey avoids missing data, duplication of identifiers, does mathematical calculations, eliminates routing problem, and checks for inadmissible responses. In Cambodia, Vietnam, Sri Lanka, and Kenya, CAPI was used in HIV surveillance system.,,, In Nepal and Pakistan, paper-based systems were used to capture the surveillance data., However, no assessment has been conducted so far in these countries to evaluate the effect of use of devices on data quality. Although the use of CAPI/real-time systems is becoming essential, it is challenging in terms of designing the system and it is associated with many operational/implementation issues such as proper selection of human resource, training, supervision, and support.
We aimed to conduct evaluation of the use of this technology in this large survey using strengths, weaknesses, opportunities, and threats (SWOT) analysis for optimum performance and to improve the data quality.
| Methods|| |
IBBS used a community-based, unlinked, anonymous, and cross-sectional survey design with probability-based sampling. Behavioral data were collected using CAPI devices. The blood samples were collected as dried blood spot for HIV testing. IBBS was carried out in 31 states (271 districts) with a planned sample of 1,38,400 (400 per district/domain) among female sex workers (FSW), men having sex with men (MSM), injecting drug users (IDU), transgender (TG), migrants (KIG), and currently married women.
The CAPI-based questionnaires were designed in 16 languages. This questionnaire/tool was elaborate with >100 structured questions. The CAPI application was android based. All the essential questions were in compulsory format to avoid the missing data. The data captured in CAPI were stored locally on the device and then were synchronized to the main server in encrypted form using internet.
Integrated information management system (IIMS) was a custom-designed web-based comprehensive project management package linked with CAPI. This included questionnaire entry and managed the data on the program elements such as training, human resources, supervision, and community activities with defined roles for access with layers of security.
Methods for current analysis
The current analysis is part of a substudy on “Data Quality Assessment and Process Evaluation” carried out in three states in western India – Maharashtra, Goa, and Gujarat. These included 38 districts/domains. Ten in-depth interviews of the stake holders/key experts worked on various committees of planning/implementation in IBBS were conducted. The interviews continued till data saturation occurred. The areas of expertise and codes of the interviews are provided in [Table 1]. The interviews were taken by the principle investigator of this study who is trained in qualitative data collection. These interviews were taken face to face/telephonically after completion of the survey.
The interviews with semistructured questionnaire were conducted for two field interviewers (FIs) (50% of the four interviewers in each domain) and 580 randomly selected IBBS respondents. The survey was started simultaneously in majority of the domains, and hence, the respondent's interviews were conducted mainly in Maharashtra and Goa.
The IBBS study was approved by the NACO Ethics Committee, and the current substudy was approved by the NARI Ethics committee. The written informed consents were obtained from the study participants. The oral informed consent was obtained for the in-depth interviews from the stake holders.
Agencies involved in the survey
The field activity of IBBS was outsourced to field research agency (FRA). A separate technology partner (TP) was appointed for development of the software (IIMS) and supply of the CAPI devices. FIs were appointed by FRA for data collection. Training was provided to the field staff by FRA, regional institutes (RI), and NACO.
Data collection and analysis
Mixed method analysis was done based on both qualitative and quantitative data. The interviewer guides were prepared for in-depth interviews as per the expertise of the stake holders. This included the questions on planning, methods/systems, training issues, difficulties faced, and recommendations. The qualitative data of interviews were transcribed, and thematic analysis was performed. Based on the findings of qualitative and quantitative data, the defined attributes were used in SWOT analysis. The attributes were simplicity, innovation, effectiveness, responsiveness, quality, and timelines.
| Results|| |
The overall CAPI use for survey was 91% (29774 CAPI entries out of 32654) in 38 domains of Maharashtra, Goa, and Gujarat. The maximum CAPI use was in Solapur district in MSM typology (99%) and minimum was at Yavatmal in FSWs (58%) in Maharashtra. The highest use of CAPI was in Gujarat (93%) followed by Goa (92%) and 88% in Maharashtra. It was highest in IDUs (98%, Gujarat) followed by MSM (95%, Goa and Gujarat), migrant (94%, Gujarat), and 90% in FSWs and TGs of Gujarat. Overall, 428 CAPI failure complaints were reported and eight tablets were lost. The themes emerging from qualitative and quantitative data are represented in [Table 2]. The quotes of the interviews are mentioned below with the interviewee code.
|Table 2: SWOT analysis using qualitative and quantitative data on technology use|
Click here to view
System design (scope and acceptance)
Overall, the technology use in IBBS was appreciated by stake holders because electronic entries saved time and cost for data entry and provided easy and immediate access to the data for monitoring and evaluation.
”It was not only data collection process alone but also human resource, training, management, and travel plan. This entire operation throughout the country across 30 states involving >3000 people moving in and around and the system should not crash”
- IT experts (#KII3)
Less than 10% respondents reported discomfort in communication due to use of CAPI. Similarly, 92% interviewers stated that CAPI had not hampered the data collection.
Data quality and management
Some of the stake holders reported that the field staff focused more on the CAPI device rather than doing rapport building with respondents. Similarly, problems were reported such as there were some data points entered in paper documents as well as on CAPI/computer which resulted redundancy in data collection process (#KII7, #KII8). In IIMS, some duplicate information was getting accepted and eligibility criteria was not linked with the behavioural data.
Human resource and training
The staff hired for the survey by the FRA was not techno savvy, and the time spent on hands-on training of CAPI and IIMS was insufficient.
”I feel it will not be good for the morale of the team as it was technically very challenging, when certain technology is supposed to ease the work but making it more difficult. I think moreover they would have felt discouraged”
- Project Coordinator (#KII-8).
Out of 71 FI, 39 (55%) said that the training of CAPI was useful, 14 (19.7%) ranked it as average, 11 (15.5%) felt it extremely useful, and 6 (8.5%) said that it was not much useful. One had not attended the training.
Information technology infrastructure
The quality of the CAPI was the major problem reported by FRA.
”In using gadgets/tablets, the FRA staff was failed. Interview was supposed to be for 40–45 min. But sometimes, it took 2 h because the gadgets were not working”
- Project Director-SACs (#KII4)
”Instead of having a separate IT agency, FRA could have managed CAPIs because there was blame game that FRA reported about defective CAPIs and TP said that the team members are not handing it properly”
- Senior Research officer (#KII7)
”There is big claim in our country about the connectivity and all. But it is limited only to the big cities. Even at district level, we struggled to download files of 4–5 MBs”
- IT expert (#KII2)
Experience of using CAPI was poor due to network problems, and poor quality of device was mentioned by 48% of FIs (n = 71) and 28% suggested to have alternative offline arrangement for data collection. According to 32 (45%) FIs, it took 2 days to resolve the CAPI issues by the FRA. However, the visit of IT coordinator for CAPI resolution was useful according to 24 (34%) FIs.
| Discussion|| |
For the first time in India, use of technology was leveraged for HIV surveillance. Overall, the use of CAPI for real-time entries was successful to a large extent because over 90% data were captured using these devices. The use of CAPI and IIMS demonstrated “Innovation,” providing benefits over traditional paper-based data collection with availability of real-time detailed data for review. This indicates the ”Simplicity” and ”Responsiveness” for accountability of the many activities involved in the study implementation. The data security and safety issues were also managed well. The method of data collection using CAPI was accepted by the field staff and the study respondents without resistance. Hence, this technique contributed to the “Effectiveness.” Similar observations were present in other studies that using CAPI/computer assisted self-interview (CAPI/CASI) for data collection was able to generate good quality timely data.,
Special feature of GPS used in CAPI device enabled system to identify the data collection from the unidentified study locations. In IIMS, the features such as built-in validation checks, skip patterns, and stepwise entries as per the protocol increased the reliability of the data. However, use of such technology for community-based surveys requires additional preparations in terms of planning, training, and support. The same contributed as weakness in terms of “Timelines” and “data quality.”
Use of gadgets also requires information technology literacy and thorough training. A study in Kenya highlighted how human–computer interaction affects data quality on CAPI and recommended to focus efforts on selecting a device, hiring, training, and monitoring of interviewers. In our study, although hands-on training was provided to the field staff, majority had not used such gadgets before, and retrospectively, it seems that the training provided had not percolated as expected to the field staff.
The attrition of the field staff before and during the survey meant that similar training may not have been provided to the new recruits. In Kenya, KAIS survey 2012, an additional training for the field staff was conducted immediately before survey implementation on troubleshooting basic technology issues and daily quality control checks. Such refresher's training just before the implementation is essential.
Quality and quantity of CAPI devices is another important aspect, and sufficient number of good quality devices must be available for training and field work. In places like Mumbai, the field staff had to travel by trains/buses and this resulted in CAPI device/screen breakage. Hence, it is necessary to have appropriate containers for CAPIs or proper transportation arrangement of the field staff. In addition, adequate technical staff is necessary on sites for troubleshooting.
Quality control reports were generated in IIMS to follow the adherence to the protocol. However, real-time respondent's behavioral data review was not done. Concurrent data reviews help to improve the data quality by providing feedback and training to the field team. A study carried out in Indonesia found that using a quality control feedback module and organizing continuous feedback sessions help to optimize the quality of data collection.
In traditional survey methodology of face-to-face settings, the researcher observes respondent reactions and accordingly probes for the questions. However, with online surveying, the opportunities to identify needs and provide support are limited. In our study, staff were focusing more on CAPI device to ensure the correct data entry which sometimes loses the eye-to-eye contact with the respondents and may not always reflect the true responses. This may affect the accuracy of the sexual behavioral data. Generating unbiased and precise measures of individual behavior patterns requires privacy and rapport with the interviewer. Computer-assisted techniques could be utilized by improving internal consistency and increasing privacy and interviewee control to improve the data validity. Audio-based CAPIs or audio CASI (ACASI) with the presence of field staff may address this issue and is useful in risk behavior assessment of female and male sex workers.
| Conclusion|| |
Data quality is an important aspect in ensuring data accuracy and reliability. Advanced appropriate technologies are useful in providing accurate and rapid information, particularly in overcoming bottlenecks in data processing. The cost-effective and time effective methods will be used for research data collection, and mobile-based or web-based devices are likely to be extensively utilized. However, proper selection of technology and human resource for data collection, appropriate training and support, concurrent data assessment, and feedback system with monitoring and action plans are the key recommendations arising from this analysis.
The survey in the western region was delayed compared to other states, and there were no quality control reports generated for this region; hence, we were unable to show the quantified impact of various factors of technology on the data quality. Similarly, the data of FI presented here are restricted only to the issues related to western region of India as the principle investigator belongs to regional institute of western region.
We acknowledge the support of Dr. Neeraj Dhingra and Dr. Yujwal Raj from NACO for providing permission conduct the sub study with IBBS. The primary author is Ph.D. scholar of Symbiosis International University (SIU), Pune and we acknowledge the support provided by SIU. We thank Dr. Nikhil Gupte from CTU, BJ medical college, Pune and Dr. Amit Lokhande, Mrs Neelam Joglekar, Ms Sucheta Deshpande, Mr Rajesh Yadav who were part of NARI IBBS team for providing the inputs for data collection and analysis. We sincerely thank the stake holders and study respondents for their participation and sharing their feedback.
Financial support and sponsorship
Conflicts of interest
There are no conflicts of interest.
| References|| |
Bachani D, Sogarwal R. National response to HIV/AIDS in India. Indian J Community Med 2010;35:469-72.
] [Full text]
Saidel T, Adhikary R, Mainkar M, Dale J, Loo V, Rahman M,et al
. Baseline integrated behavioural and biological assessment among most at-risk populations in six high-prevalence states of India: Design and implementation challenges. AIDS 2008;22 Suppl 5:S17-34.
Thriemer K, Ley B, Ame SM, Puri MK, Hashim R, Chang NY,et al
. Replacing paper data collection forms with electronic data entry in the field: Findings from a study of community-acquired bloodstream infections in Pemba, Zanzibar. BMC Res Notes 2012;5:113.
Hughes S, Haddaway S, Zhou H. Comparing Smartphones to Tablets for Face-to-Face Interviewing in Kenya. Survey Methods: Insights from the Field; 2016. Available from: http://www.surveyinsights.org/?p=7031
. [Last accessed on 2017 Jul 06].
Ojwang' JK, Lee VC, Waruru A, Ssempijja V, Ng'ang'a JG, Wakhutu BE,et al
. Using information and communications technology in a national population-based survey: The Kenya AIDS indicator survey 2012. J Acquir Immune Defic Syndr 2014;66 Suppl 1:S123-9.
Purba FD, Hunfeld JA, Iskandarsyah A, Fitriana TS, Sadarjoen SS, Passchier J,et al
. Employing quality control and feedback to the EQ-5D-5L valuation protocol to improve the quality of data collection. Qual Life Res 2017;26:1197-208.
Rhodes SD, Bowie DA, Hergenrather KC. Collecting behavioural data using the world wide web: Considerations for researchers. J Epidemiol Community Health 2003;57:68-73.
Fenton KA, Johnson AM, McManus S, Erens B. Measuring sexual behaviour: Methodological challenges in survey research. Sex Transm Infect 2001;77:84-92.
van der Elst EM, Okuku HS, Nakamya P, Muhaari A, Davies A, McClelland RS,et al
. Is audio computer-assisted self-interview (ACASI) useful in risk behaviour assessment of female and male sex workers, Mombasa, Kenya? PLoS One 2009;4:e5340.
Forster D, Behrens RH, Campbell H, Byass P. Evaluation of a computerized field data collection system for health surveys. Bull World Health Organ 1991;69:107-11.
[Table 1], [Table 2]