https://www.cureus.com/articles/82162-ivermectin-prophylaxis-used-for-covid-19-a-citywide-prospective-observational-study-of-223128-subjects-using-propensity-score-matching Ivermectin Prophylaxis Used for COVID-19: A Citywide, Prospective, Observational Study of 223,128 Subjects Using Propensity Score MatchingLucy Kerr, Flavio A. Cadegiani, Fernando Baldi, Raysildo B. Lobo, Washington Luiz O. Assagra, Fernando Carlos Proença, Pierre Kory, Jennifer A. Hibberd, Juan J. Chamie-Quintero
Published: January 15, 2022 (see history) DOI: 10.7759/cureus.21272 Cite this article as: Kerr L, Cadegiani F A, Baldi F, et al. (January 15, 2022) Ivermectin Prophylaxis Used for COVID-19: A Citywide, Prospective, Observational Study of 223,128 Subjects Using Propensity Score Matching. Cureus 14(1): e21272. doi:10.7759/cureus.21272
AbstractBackground: Ivermectin has demonstrated different mechanisms of action that potentially protect from both coronavirus disease 2019 (COVID-19) infection and COVID-19-related comorbidities. Based on the studies suggesting efficacy in prophylaxis combined with the known safety profile of ivermectin, a citywide prevention program using ivermectin for COVID-19 was implemented in Itajaí, a southern city in Brazil in the state of Santa Catarina. The objective of this study was to evaluate the impact of regular ivermectin use on subsequent COVID-19 infection and mortality rates. Materials and methods: We analyzed data from a prospective, observational study of the citywide COVID-19 prevention with ivermectin program, which was conducted between July 2020 and December 2020 in Itajaí, Brazil. Study design, institutional review board approval, and analysis of registry data occurred after completion of the program. The program consisted of inviting the entire population of Itajaí to a medical visit to enroll in the program and to compile baseline, personal, demographic, and medical information. In the absence of contraindications, ivermectin was offered as an optional treatment to be taken for two consecutive days every 15 days at a dose of 0.2 mg/kg/day. In cases where a participating citizen of Itajaí became ill with COVID-19, they were recommended not to use ivermectin or any other medication in early outpatient treatment. Clinical outcomes of infection, hospitalization, and death were automatically reported and entered into the registry in real time. Study analysis consisted of comparing ivermectin users with non-users using cohorts of infected patients propensity score-matched by age, sex, and comorbidities. COVID-19 infection and mortality rates were analyzed with and without the use of propensity score matching (PSM). Results: Of the 223,128 citizens of Itajaí considered for the study, a total of 159,561 subjects were included in the analysis: 113,845 (71.3%) regular ivermectin users and 45,716 (23.3%) non-users. Of these, 4,311 ivermectin users were infected, among which 4,197 were from the city of Itajaí (3.7% infection rate), and 3,034 non-users (from Itajaí) were infected (6.6% infection rate), with a 44% reduction in COVID-19 infection rate (risk ratio [RR], 0.56; 95% confidence interval (95% CI), 0.53-0.58; p < 0.0001). Using PSM, two cohorts of 3,034 subjects suffering from COVID-19 infection were compared. The regular use of ivermectin led to a 68% reduction in COVID-19 mortality (25 [0.8%] versus 79 [2.6%] among ivermectin non-users; RR, 0.32; 95% CI, 0.20-0.49; p < 0.0001). When adjusted for residual variables, reduction in mortality rate was 70% (RR, 0.30; 95% CI, 0.19-0.46; p < 0.0001). There was a 56% reduction in hospitalization rate (44 versus 99 hospitalizations among ivermectin users and non-users, respectively; RR, 0.44; 95% CI, 0.31-0.63; p < 0.0001). After adjustment for residual variables, reduction in hospitalization rate was 67% (RR, 0.33; 95% CI, 023-0.66; p < 0.0001). Conclusion: In this large PSM study, regular use of ivermectin as a prophylactic agent was associated with significantly reduced COVID-19 infection, hospitalization, and mortality rates. IntroductionIvermectin has been demonstrated to have not only extensive anti-parasitic actions [1,2], but also anti-viral, anti-bacterial, and anti-protozoan properties. Ivermectin has been long proposed for use as a repurposed antiviral agent [3-6]. Indeed, antiviral effects of ivermectin have been reported against both RNA and DNA types of viruses, including HIV-1, yellow fever, Japanese encephalitis, tick-borne encephalitis, West Nile, Zika, dengue fever, chikungunya, Venezuelan equine encephalitis, and the pseudorabies virus [3,5,7,8], as well as functioning in regulation of proteins involved in antiviral responses [8]. Additional actions of ivermectin described include agonism activity to the liver X receptor (LXR) and farnesoid X receptor (FXR), with multiple potential metabolic benefits [9,10]; neuronal regeneration [11,12], prevention of muscle hypoxia [13], and actions on specific sites, including interferon (INF) [14], nuclear factor-κB (NF-κB), lipopolysaccharide (LPS) [15], and Janus kinase/signal transducer and activator of transcription (JAK-STAT) and PAI-1 pathway [16,17]; generation of P21 activated kinase 1 (PAK-1) [18,19]; reduction of interleukin-6 (IL-6) levels [15]; allosteric modulation of P2X4 receptor [20]; inhibition of high mobility group box 1 (HMGB1) [21,22]; and suppression of mucus hypersecretion, diminished recruitment of immune cells, and production of cytokines in the lung [23]. Ivermectin is also described to induce T helper 1 cell (Th1)-type immune response against protozoan infections [24], and anti-coagulant action through binding to the S protein of some viruses [25]. The hypothesis that ivermectin could be protective against coronavirus disease 2019 (COVID-19) is substantiated by its multi-pathway, anti-inflammatory effects [15,26], and multi-antiviral mechanisms. COVID-19 pathogenesis is largely understood as an inflammation-mediated hemagglutinating infection disrupting pulmonary, vascular, and endothelial systems, leading to a multi-systemic disease. In vitro and in silico, ivermectin has demonstrated anti-severe acute respiratory syndrome coronavirus 2 activity through more than 20 direct and indirect mechanisms [2,27,28]. Ivermectin has demonstrated preliminary protective effects against severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infection in terms of reducing times to clinical recovery and rates of disease progression and mortality [2,29,30]. However, more robust studies with larger sample sizes are still recommended to confirm the possible beneficial effects of ivermectin in COVID-19. Since the onset of the COVID-19 pandemic, the use of inexpensive options based on a consistently beneficial signal of efficacy, a well-established safety profile, and favorable cost-effectiveness, ivermectin is a highly attractive intervention for the patient-centered medicine practiced by frontline clinicians, with use aligning strongly with the bioethical principles for medical practice outlined in Article 36 of the Declaration of Helsinki [31]. However, despite this favorable risk/benefit profile and absence of therapeutic alternatives, ivermectin is yet to be approved for prophylaxis and treatment of COVID-19 by agencies throughout the world, including FDA (USA), European Medicines Agency (EMA; Europe), and ANVISA (Agência Nacional de Vigilância Sanitária - Brazilian Health Regulatory Agency; Brazil). The ability to prescribe ivermectin or any other off-label drug for COVID-19 has long been at the discretion of frontline physicians once all risks, uncertainties, potential benefits, and patients’ rights are exposed, and informed consent has been obtained. Of particular note, in Brazil, this follows the medical autonomy to determine the best therapeutic strategies for individuals, as per the Medical Code of Ethics of the Brazilian Board of Medical Doctors, the Federal Council of Medicine - Conselho Federal de Medicina (CFM), that determines the obligations and rights of medical doctors in Brazil [32]. Since vaccines for COVID-19 were not available in Brazil until 2021, and because of the lack of prophylactic alternatives in the absence of vaccines, Itajaí, a city in the southern Brazilian state of Santa Catarina, initiated a population-wide government program for COVID-19 prophylaxis. The medical-focused decision parameters established are based on the distribution of ivermectin to whole populations in different countries. To ensure the safety of the population, a well-controlled computer program was developed to compile and maintain all relevant demographic and clinical data (detailed in the Materials & Methods section). The use of ivermectin was optional and based on patients’ preferences, given its benefits as a preventative agent was unproven. This study’s objective is to assess the impact on important clinical outcomes when ivermectin is used as prophylaxis for COVID-19. The prophylaxis program occurred in addition to the standard non-pharmacological strategies of masking and social distancing, as part of a citywide program conducted in outpatient settings. Materials & MethodsStudy populationThis was a prospective, observational study. Although study design, institutional review board (IRB) approval, and data analysis occurred after completion of the voluntary prophylaxis program, all data were collected prospectively in real time with mandated reporting to the registry of all events as they occurred during the citywide governmental COVID-19 prevention with ivermectin program, from July 2020 to December 2020, developed in the city of Itajaí, in the state of Santa Catarina, Brazil. Demographic and clinical data were reported from medical records of patients followed in a large outpatient setting (a provisional outpatient clinic set in the Convention Center of Itajaí) and several secondary outpatient settings, as part of the universal health system (Sistema Único de Saúde [SUS]). The objective was to determine the number of patients affected by COVID-19 (positivity rate of reverse transcription-polymerase chain reaction [RT-PCR] for SARS-CoV-2), risk of death due to COVID-19 (whether infected or not), and COVID-19 mortality rate (risk of death from COVID-19) of those who used and did not use ivermectin prophylactically for COVID-19. These data were stratified by age, sex, presence of comorbidities, and correlated demographic characteristics. The present retrospective analysis of the prospectively collected data was approved by the National Research Ethics Council (CONEP) under the number 4.821.082 with the project number CAAE: 47124221.2.0000.5485. Although study design, IRB approval, and data analysis occurred after completion of the voluntary prophylaxis program, all data were collected prospectively in real-time with mandated reporting to the registry of all events as they occurred during the citywide governmental COVID-19 prevention with ivermectin program, from July 7, 2020, to December 2, 2020, developed in the city of Itajaí, in the state of Santa Catarina, Brazil. Study procedures and data collectionOptional, voluntary prophylactic use of ivermectin was offered to patients during regular medical visits between July 7, 2020, and December 2, 2020, in 35 different sites, including 34 local SUS health centers and a large temporary patient setting 24/7. Doctors working in these sites were free to prescribe ivermectin prophylactically. Subjects that did not use ivermectin either refused or their primary care physicians opted not to offer ivermectin. To avoid underreported data, strict procedure sequencing was followed: (1) registration and recording of patient data, documented by assistants; (2) weighing subjects (subject's weight was essential to calculate the appropriate dose of ivermectin); (3) brief medical evaluation of past medical history, comorbidities, use of medications, and contraindications to drugs; and (4) medical prescription with prophylactic doses of ivermectin (within recommended usual, safe doses of ivermectin), according to medical judgment and following a subject’s informed consent related to potential benefits, risks, and side effects. All details of this citywide program and campaign had been previously agreed upon between the city local department of the National Healthcare System (SUS), city mayor, and local public prosecutors. Regarding drug interactions with ivermectin, the use of warfarin was a contraindication for prophylaxis with ivermectin due to drug interactions. Subjects under chronic use of glucocorticoids, protease inhibitors, and anti-epileptics were recommended to schedule regular medical visits every six to eight weeks. Subjects were recommended to inform medical doctors about the use of ivermectin, in case one or more of the following medications were prescribed: warfarin, azithromycin, dexamethasone, prednisone, or prednisolone (hydrocortisone or cortisone are not commercially available in regular pharmacies in Brazil). The following variables were analyzed: (1) age, (2) sex, (3) previous diseases (myocardial infarction [MI] and stroke), (4) pre-existing comorbidities (type 2 diabetes [T2D], asthma, chronic obstructive pulmonary disease [COPD], hypertension, dyslipidemia, cardiovascular diseases [CVD], cancer [any type], and other pulmonary diseases), and (5) smoking. Variables were adjusted as confounding factors and used as variables for balancing and matching groups for propensity score matching (PSM). Patients who presented signs or the diagnosis of COVID-19 before July 7, 2020, were excluded from the sample. Other exclusion criteria were contraindications to ivermectin and subjects below 18 years of age. The dose and frequency of ivermectin treatment was 0.2 mg/kg/day; i.e., giving one 6 mg tablet for every 30 kg for two consecutive days every 15 days. During the study, subjects who were diagnosed with COVID-19 underwent a specific medical visit to assess COVID-19 clinical manifestations and severity. All subjects were recommended not to use ivermectin, nitazoxanide, hydroxychloroquine, spironolactone, or any other drug claimed to be effective against COVID-19. The city did not provide or support any specific pharmacological outpatient treatment for subjects infected with COVID-19. They were questioned for the presence of common COVID-19 symptoms. These included chills, high-grade fever, cough, myalgia, fatigue, anosmia, ageusia, sore throat, headache, nasal congestion, sneeze, runny nose, hemoptysis, nausea, vomiting, abdominal pain, diarrhea, cutaneous rash, arthralgia, chest pain, eye pain and pinkeye, and presence of alert signs, including shortness of breath, signs of hypoxia, signs of coagulation abnormalities, and an altered level of consciousness. Systolic and diastolic blood pressure, heart rate, respiratory rate, oxygen saturation, and axillary temperature were measured. The same signs and symptoms and vital signs were collected at each following medical visit during COVID-19. Individual data were compiled and reviewed by the researchers. Registry data of all patient records from the city of Itajaí between July 7, 2020, and December 2, 2020, including those who used ivermectin and did not use ivermectin were reviewed. All subjects who tested positive for COVID-19 in the city of Itajaí during the study were considered for this analysis. Of the infected subjects, two groups were considered: subjects who used ivermectin prophylactically (treated group) and subjects who did not use ivermectin prophylactically (untreated group). Missing data from patients were clarified with patients or relatives directly, via phone or in person, by the investigators. Since this is a citywide program, all recorded data must have matched the exact number of COVID-19 cases and deaths of the city. This strict interval avoids differences in terms of periods of exposure. Due to the uncertainty of reinfection with COVID-19, subjects with a history of previous COVID-19 did not participate in the program although they were still permitted to use ivermectin prophylactically. Limiting parameters of the government system allowed the recording of a first episode of COVID-19 infection only. Subjects below 18 years old and subjects with a diagnosis of COVID-19 before July 7, 2020, were excluded from all datasets and analyses. From the registry of the city population (223,128 inhabitants), subjects below 18 years old (61,583 subjects) were removed. Of the 161,545 subjects above 18 years old from the city of Itajaí, we removed the 1,984 COVID-19 cases that occurred before July 7, 2020, and 159,561 subjects remained. Subjects above 18 years old were considered those who were born before June 30, 2002. A total of 147,223 subjects participated in the program of ivermectin prophylaxis used for COVID-19. Of these, 24,304 subjects were below 18 years old. Of the 122,919 ivermectin users above 18 years old, 8,346 were from other cities, and 728 had COVID-19 before July 7, 2020, although they used ivermectin afterward. In total, 113,845 subjects that participated in the program remained in the dataset. The 45,716 non-participants, remaining subjects among the 159,561 subjects, were considered as the ivermectin non-users. Finally, citywide COVID-19 hospitalization and mortality rates of Itajaí were compared between the period before the program (before July 7, 2020) and during the program (between July 7, 2020, and December 2, 2020) aiming to evaluate whether a program of prophylaxis with ivermectin for COVID-19 would cause a positive impact in the overall numbers of the city, despite only partial adoption. Chances of dying of COVID-19 in the overall population, according to use or non-use of ivermectin (irrespective of COVID-19 infection) were only calculated prior to matching. Conversely, the mortality rate among those who were infected by the SARS-CoV-2 was calculated for both pre and post-matched cohorts. Hospitalization and mortality rates before matching groups, the mortality rate in subpopulations before and after PSM, and the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) checklist are presented in the Appendix. Statistical analysisThe full underlying data for the present analysis were analyzed by two independent statisticians, and discrepancies were evaluated by a third statistics expert. In this outpatient study of those who tested positive for SARS-CoV-2, the mortality rate was evaluated according to each parameter that was adjusted against other variables (for multivariate regression analysis) and used for balancing and matching groups, including age intervals, sex, history of smoking, prophylactic ivermectin use, T2D, asthma, COPD, cardiovascular diseases and other pulmonary diseases, hypertension, current cancer (any type), and history of stroke and/or MI. Before matching, a generalized linear mixed model was employed, assuming the binomial distribution for the residues and including the fixed classificatory effects of each of these parameters. Age intervals were adjusted for the evaluation of ivermectin prophylactic use as an independent predictor of death from COVID-19. Unadjusted and multivariate Poisson-adjusted probabilities to survive from COVID-19 (p-value), according to each parameter, were provided. PSM was performed for mortality risk between ivermectin and non-ivermectin users. COVID-19 infection rate and risk of dying were also calculated for variables. After PSM, a second adjustment ("double adjustment") with multivariate linear regression was performed for residual variables [33,34]. There were no missing data since the registry system design mandated that all data variables be filled to be formally included in the registry. Only erroneously entered (illogical) data were found. In such instances, a medical record review was performed to obtain accurate data. The program used for the analysis was the Statistical Analysis Software (SAS/STAT) (SAS Institute Inc., Cary, NC). For transparency reasons, two datasets of the 7,345 COVID-19 cases and the 113,845 participating subjects considered for the present analysis will be made public upon peer-reviewed publication. ResultsA detailed description of the data considered for the present analysis is illustrated in Figure 1. Of the 220,517 citizens of Itajaí without COVID-19 until July 7, 2020, 159,561 were above 18 years old. Of the 159,561 citizens above 18 years old without COVID-19 until July 7, 2020, 113,845 (71.3% of the population above 18 years old) received ivermectin before being infected by COVID-19. A total of 45,716 citizens (28.7%) did not receive or did not want to receive ivermectin during the program, including as a prophylactic or as a treatment after having COVID-19. Of the 113,845 prophylaxed subjects from the city of Itajaí, 4,197 had a positive RT-PCR SARS-CoV-2 (3.7% infection rate), while 3,034 of the 37,027 untreated subjects had positive RT-PCR SARS-CoV-2 (6.6% infection rate), a 44% reduction in COVID-19 infection rate (risk ratio [RR], 0.56; 95% confidence interval (95% CI), 0.53-0.58; p < 0.0001). An addition of 114 subjects who used ivermectin and were infected were originally from other cities but were registered as part of the program, in a total of 4,311 positive cases among ivermectin users. For the present analysis, the 4,311 positive cases among subjects that used ivermectin and 3,034 cases among subjects that did not use ivermectin were considered. After PSM, two cohorts of 3,034 subjects were created. Baseline characteristics of the 7,345 subjects included prior to PSM and the baseline characteristics of the 6,068 subjects in the matched groups are shown in Table 1. Prior to PSM, ivermectin users had a higher percentage of subjects over 50 years old (p < 0.0001), higher prevalence of T2D (p = 0.0004), hypertension (p < 0.0001), and CVD (p = 0.03), and a higher percentage of Caucasians (p = 0.004), than non-users. After PSM, all baseline parameters were similar between groups. Figure 2 summarizes the main findings of this study.
| Pre-matching | Propensity score-matched |
| Overall (n = 7,345) | Ivermectin users (n = 4,311) | Non-ivermectin users (n = 3,034) | p-value | Overall (n = 6,068) | Ivermectin users (n = 3,034) | Non-ivermectin users (n = 3,034) | Age |
|
|
|
|
|
|
| Mean ± SD | 42.0 ± 14.7 | 43.5 ± 14.9 | 39.8 ± 14.2 | <0.0001 | 39.7 ± 14.0 | 3967 ± 13.8 | 39.8 ± 14.2 | <30 years old | 1,730 (23.6%) | 886 (20.5%) | 844 (27.8%) |
| 1,691 (27.9%) | 844 (27.9%) | 847 (27.8%) | 30-50 years old | 3,703 (50.4%) | 2,121 (49.2%) | 1,582 (52.2%) |
| 3,155 (52.0%) | 1,573 (51.9%) | 1,582 (52.1%) | >50 years old | 1,912 (26.0%) | 1,304 (30.3%) | 608 (20.0%) |
| 1,222 (20,1%) | 614 (20.2%) | 608 (20.1%) | Sex |
|
|
| 0.31 |
|
|
| Female | 3,983 (54.2%) | 2,359 (54.7%) | 1,624 (53.5%) |
| 3,231 (53.2%) | 1,607 (53.0%) | 1,624 (53.5%) | Male | 3,362 (45.8%) | 1,952 (45.3%) | 1,410 (46.5%) |
| 2,837 (46.8%) | 1,427 (47.0%) | 1,410 (46.5%) | Race |
|
|
|
|
|
|
| Caucasians | 5,437 (74.0%) | 3,245 (75.3%) | 2,192 (72.2%) | 0.004 | 4,398 (72.5%) | 2,206 (72.7%) | 2,192 (72.3%) | Afro-Brazilians | 209 (2.8%) | 109 (2.5%) | 100 (3.3%) | 0.052 | 193 (3.2%) | 93 (3.1%) | 100 (3.3%) | Mixed | 1,583 (22.6%) | 901 (20.9%) | 682 (22.5%) | 0.10 | 1,364 (22.5%) | 93 (3.1%) | 100 (3.3%) | Asian-Brazilians | 116 (1.6%) | 56 (1.3%) | 60 (2.0%) | 0.023 | 113 (1.9%) | 53 (1.8%) | 60 (2.0%) | Type 2 diabetes |
|
|
| 0.0004 |
|
|
| Yes | 214 (2.9%) | 151 (3.5%) | 63 (2.1%) |
| 141 (2.3%) | 78 (2.6%) | 63 (2.1%) | No | 7,131 (97.1%) | 4,160 (96.5%) | 2,971 (97.9%) |
| 5,927 (97.7%) | 2,956 (97.4%) | 2,971 (97.9%) | Asthma |
|
|
| 0.067 |
|
|
| Yes | 26 (0.3%) | 20 (0.5%) | 6 (0.2%) |
| 21 (0.3%) | 15 (0.5%) | 6 (0.2%) | No | 7,319 (99.7%) | 4,291 (99.5%) | 3,028 (99.8%) |
| 6,047 (99.7%) | 3,019 (99.5%) | 3,028 (99.8%) | COPD |
|
|
| 0.72 |
|
|
| Yes | 13 (0.2%) | 7 (0.2%) | 6 (0.2%) |
| 12 (0.2%) | 6 (0.2%) | 6 (0.2%) | No | 7,332 (99.8%) | 4,304 (99.8%) | 3,028 (99.8%) |
| 6,056 (99.8%) | 3,028 (99.8%) | 3,028 (99.8%) | Hypertension |
|
|
| <0.0001 |
|
|
| Yes | 528 (7.2%) | 362 (8.4%) | 166 (5.5%) |
| 343 (5.6%) | 177 (5.8%) | 166 (5.5%) | No | 6,817 (92.8%) | 3,949 (91.6%) | 2,868 (94.5%) |
| 5,725 (94.4%) | 2,857 (94.2%) | 2,868 (94.5%) | CVD |
|
|
| 0.03 |
|
|
| Yes | 56 (0.8%) | 41 (1.0%) | 15 (0.5%) |
| 32 (0.5%) | 17 (0.6%) | 15 (0.5%) | No | 7,289 (99.2%) | 4,270 (99.0%) | 3,019 (99.5%) |
| 6,036 (99.5%) | 3,017 (99.4%) | 3,019 (99.5%) | Other pulmonary diseases |
|
|
| 0.53 |
|
|
| Yes | 15 (0.2%) | 10 (0.2%) | 5 (0.2%) |
| 9 (0.1%) | 4 (0.1%) | 5 (0.1%) | No | 7,330 (99.8%) | 4,301 (99.8%) | 3,029 (99.8%) |
| 6,059 (99.9%) | 3,030 (99.9%) | 3,029 (99.9%) | Cancer (any type) |
|
|
| 0.66 |
|
|
| Yes | 32 (0.4%) | 20 (0.5%) | 12 (0.4%) |
| 22 (0.4%) | 10 (0.3%) | 12 (0.4%) | No | 7,313 (99.6%) | 4,291 (99.5%) | 3,023 (99.6%) |
| 6,046 (99.6%) | 3,024 (99.7%) | 3,022 (99.6%) | Current smoking |
|
|
| 0.76 |
|
|
| Yes | 110 (1.5%) | 63 (1.5%) | 47 (1.5%) |
| 95 (1.6%) | 48 (1.6%) | 47 (1.6%) | No | 7,235 (98.5%) | 4,248 (98.5%) | 2,987 (98.5%) |
| 5,973 (98.4%) | 2,986 (98.4%) | 2,987 (98.4%) | History of MI |
|
|
| 0.26 |
|
|
| Yes | 15 (0.2%) | 11 (0.3%) | 4 (0.1%) |
| 8 (0.1%) | 4 (0.1%) | 4 (0.1%) | No | 7,330 (99.8%) | 4,300 (99.7%) | 3,030 (99.9%) |
| 6,060 (99.9%) | 3,030 (99.9%) | 3,030 (99.9%) | History of stroke |
|
|
| 0.56 |
|
|
| Yes | 21 (0.3%) | 11 (0.3%) | 10 (0.3%) |
| 21 (0.4%) | 11 (0.4%) | 10 (0.3%) | No | 7,324 (99.7%) | 4,300 (99.7%) | 3,024 (99.7%) |
| 6,047 (99.6%) | 3,023 (99.6%) | 3,024 (99.7%) |
Table 1: Baseline characteristics of subjects enrolled in the study before matching and after propensity score matching.COPD = chronic obstructive pulmonary disease; CVD = cardiovascular disease; MI = myocardial infarction; SD = standard deviation. Hospitalization and mortality rates in ivermectin users and non-users in propensity score-matched analysisAs described in Table 2, after employing PSM, of the 6,068 subjects (3,034 in each group), there were 44 hospitalizations among ivermectin users (1.6% hospitalization rate) and 99 hospitalizations (3.3% hospitalization rate) among ivermectin non-users, a 56% reduction in hospitalization rate (RR, 0.44; 95% CI, 0.31-0.63). When adjustment for variables was employed, the reduction in hospitalization rate was 67% (RR, 0.33; 95% CI, 023-0.66; p < 0.0001).
|
| Overall | IVM users | Non-IVM users | PSM mortality risk ratio (95% CI) and p-value [p] | Adjusted PSM mortality risk ratio (95% CI) and p-value [p] | COVID-19 infection | Infected population (n) | 6,068 | 3,034 | 3,034 | - | - | COVID-19 hospitalization | Hospitalization due to COVID-19 | 143 | 44 | 99 | - | - | Hospitalization rate* (in case of COVID-19) (%) | 2.3% | 1.6% | 3.3% | 0.44 (0.31-0.63) [<0.0001] | 0.33 (0.23-0.46) [<0.0001] | COVID-19 death | COVID-19 deaths (n)** | 104 | 25 | 79 | - | - | Mortality rate (among infected subjects) (%) | 1.7% | 0.8% | 2.6% | 0.32 (0.20-0.49) [<0.0001] | 0.30 (0.19-0.46) [<0.0001] |
Table 2: Propensity score-matched hospitalization and mortality rate among ivermectin users and non-users.IVM = ivermectin; PSM = propensity score matching. * Only subjects hospitalized in public hospitals. ** All deaths, including from public and private hospitals, and in-home. There were 25 deaths among ivermectin users (0.8% mortality rate) and 79 deaths among non-ivermectin users (2.6% mortality rate), a 68% reduction in mortality rate (RR, 0.32; 95% CI, 0.20-0.49). When PSM was adjusted, reduction in mortality rate was 70% (RR, 0.30; 95% CI, 0.19-0.46; p < 0.0001). Determinants of COVID-19 mortality through propensity score-matched analysisTable 3 describes the resulting risk factors for COVID-19 death amongst the overall population through PSM analysis. Risk factors for mortality in COVID-19 included aging (p < 0.0001), male sex (p = 0.015), T2D (p < 0.0001), hypertension (p < 0.0001), asthma (p = 0.011), COPD (p < 0.0001), other pulmonary diseases (p = 0.048), history of MI (p = 0.034), and history of stroke (p < 0.0001). To detect independent risk factors, post-PSM adjustment for variables showed that ivermectin (p < 0.0001; 70% reduction in mortality risk) and female sex (p = 0.022; 38% reduction in mortality risk) were protectors, whereas T2D (p = 0.041; 79% increase in mortality risk), hypertension (p = 0.008; 98% increase in mortality risk), and, marginally, other pulmonary diseases (p = 0.061; 468% increase in mortality risk) and history of stroke (p = 0.054; 97% increase in mortality risk) were identified as independent risk factors.
| Propensity score-matched groups | Variable | Overall (n = 6,068) | Death (%) | Unadjusted COVID-19 mortality risk ratio and p-value [p] | Multivariate adjusted COVID-19 mortality risk ratio and p-value [p] | Ivermectin use - n (%) |
|
| 0.32 (0.20-0.49) [<0.0001] | 0.30 (0.19-0.46) [<0.0001] | Yes | 3,034 | 25 (0.8%) |
|
| No | 3,034 | 79 (2.6%) |
|
| Age - n (%) |
|
| [<0.0001] | [<0.0001] | <30 years old | 1,691 | 1 (0.1%) |
|
| 30-50 years old | 3,155 | 12 (0.4%) |
|
| >50 years old | 1,222 | 91 (7.4%) |
|
| Sex - n (%) |
|
| 0.62 (0.42-0.91) [0.015] | 0.64 (0.44-0.93) [0.022] | Female | 3,231 | 43 (1.3%) |
|
| Male | 2,837 | 61 (2.2%) |
|
| Race - n (%) |
|
| [0.24] | [0.44] | Caucasians | 4,398 | 79 (1.8%) |
|
| Afro-Brazilians | 193 | 6 (3.1%) |
|
| Mixed | 1.364 | 17 (1.3%) |
|
| Asian-Brazilians | 113 | 2 (1.9%) |
|
| Type 2 diabetes - n (%) |
|
| 10.0 (6.32-15.8) [<0.0001] | 1.79 (1.03-3.12) [0.041] | Yes | 141 | 20 (14.2%) |
|
| No | 5,927 | 84 (1.4%) |
|
| Hypertension - n (%) |
|
| 8.83 (5.99-13.0) [< 0.0001] | 1.98 (1.19-3.30) [0.008] | Yes | 343 | 36 (10.5%) |
|
| No | 5,725 | 68 (1.2%) |
|
| Asthma - n (%) |
|
| 5.64 (1.49-21.4) [0.011] | 1.74 (0.52-5.81) [0.36] | Yes | 21 | 2 (9.5%) |
|
| No | 6,047 | 102 (1.7%) |
|
| COPD - n (%) |
|
| 15.0 (5.52-40.7) [<0.0001] | 1.71 (0.68-4.31) [0.25] | Yes | 12 | 3 (25.0%) |
|
| No | 6,056 | 101 (1.7%) |
|
| Cardiovascular diseases - n (%) |
|
| 7.54 (2.96-19.3) [<0.0001] | 1.22 (0.44-3.37) [0.70] | Yes | 32 | 4 (12.5%) |
|
| No | 6,036 | 100 (1.7%) |
|
| Other pulmonary diseases - n (%) |
|
| 6.54 (1.02-41.9) [0.048] | 5.68 (0.92-35.0) [0.061] | Yes | 9 | 1 (11.1%) |
|
| No | 6,059 | 103 (1.7%) |
|
| Cancer (any type) - n (%) |
|
| 2.67 (0.39-18.3) [0.32] | 1.97 (0.30-12.9) [0.48] | Yes | 22 | 1 (4.6%) |
|
| No | 6,046 | 103 (1.7%) |
|
| Current smoking - n (%) |
|
| 1.23 (0.31-4.92) [0.77] | 0.36 (0.08-1.70) [0.20] | Yes | 95 | 2 (2.1%) |
|
| No | 5,973 | 102 (1.7%) |
|
| History of MI - n (%) |
|
| 7.35 (1.16-46.5) [0.034] | 1.91 (0.17-21.6) [0.60] | Yes | 8 | 1 (12.5%) |
|
| No | 6,060 | 103 (1.7%) |
|
| History of stroke - n (%) |
|
| 17.6 (8.72-35.7) [< 0.0001] | 1.97 (0.99-3.92) [0.054] | Yes | 21 | 6 (28.6%) |
|
| No | 6,047 | 98 (1.6%) |
|
|
Table 3: Propensity score-matched COVID-19 mortality rate according to each characteristic in the overall population, ivermectin users, and non-users.COPD = chronic obstructive pulmonary disease; CVD = cardiovascular disease; MI = myocardial infarction. In a comparison of citywide COVID-19 hospitalization rates prior to and during the program, COVID-19 mortality decreased from 6.8% before the program with prophylactic use of ivermectin, to 1.8% after its beginning (RR, 0.27; 95% CI, 0.21-0.33; p < 0.0001), and in COVID-19 mortality rate, from 3.4% to 1.4% (RR, 0.41; 95% CI, 0.31-0.55; p < 0.0001) (Table 4).
| Overall | Until July 30th | After July 30th | Relative risk ratio (95% CI) | p-value | Infected COVID-19 population (n) | 9,956 | 2,663 | 7,293 | - | - | Infected non-hospitalized COVID-19 population (n) | 9,641 | 2,481 | 7,160 | - | - | Hospitalized COVID-19 population (n) | 315 | 182 | 133 | - | - | COVID-19 hospitalization rate COVID-19 (%) | 3.2% | 6.8% | 1.8% | 0.27 (0.21-0.33) | <0.0001 | Overall number of COVID-19 deaths | 192 | 90 | 102 | - | - | Overall mortality rate (%) | 1.9% | 3.4% | 1.4% | 0.41 (0.31-0.55) | <0.0001 |
Table 4: Hospitalization and mortality rates registered in the city of Itajaí, Brazil, before versus after the beginning of the citywide program with ivermectin use as prophylaxis for COVID-19, independent of the ivermectin use status.DiscussionThis prospective, citywide COVID-19 ivermectin prophylaxis program resulted in significant reductions in COVID-19 infections, hospitalizations, and deaths. The ivermectin non-users were two times more likely to die of COVID-19 than ivermectin users in the overall population analysis. Since groups were compared for the exposure during the same period, in a parallel manner, changes in transmission rates would affect ivermectin users and non-users equally. The city of Itajaí, in the state of Santa Catarina, Brazil, started a citywide program of prophylaxis with ivermectin in July 2020 as part of several initiatives to reduce the burden of COVID-19. The use of ivermectin was based on the existing literature at that time and on the virtual absence of risks. The National Health System (SUS) functions as full healthcare support to the entire population allowed the city to establish a non-restricted population program. This program included a support structure consisting of a large outpatient clinic located at the Convention Center of Itajaí. This outpatient clinic became the main locale of assistance for COVID-19 patients, supported by multiple public facilities where general practitioners regularly saw patients. The use of ivermectin was optional unless contraindicated and given upon medical discretion. A structured medical-based program with a medical visit and evaluation of basic demographic characteristics and comorbidities offered ivermectin as optional prophylaxis to those who agreed to participate in this preventive treatment program. Health status was assessed and data were entered prospectively throughout the period of the program, in a fully digitized system provided by the National Health System (SUS). Since the system existed prior to the pandemic, a significant number of the population were already registered with their health information, including past and current diseases, use of medications, and other characteristics. The adaptations made to the SUS for the pandemic preparedness, prior to the initiation of this ivermectin outpatient program, allowed a structured, well-organized collection of the data that monitored any missing values, reinforcing the reliability of the results. An important conservative bias was present. Major risk factors for severe COVID-19 and mortality due to COVID-19, including aging, diabetes, and hypertension, were more present among ivermectin users, which may have underestimated the benefits of ivermectin as it was demonstrated to be particularly effective in subjects above 49 years old in terms of reduction of absolute risk, which corresponds to the group at the highest risk for COVID-19. This allows the understanding that prophylactic use of ivermectin can be particularly impactful in older subjects. In addition, ivermectin seemed to reduce the exceeding risk of hypertension, T2D, and other diseases. In accordance with the literature, subjects with higher age, diabetes, and males were less likely to survive (p < 0.05 for all), and only aging remained as an independent risk factor after PSM (p < 0.0001). However, prophylactic ivermectin use appears to mitigate the additional risk of COVID-19 death due to T2D, hypertension, and cardiovascular diseases. The narrative that using preventive and early treatment therapies will have people relax their caution of remaining socially and physically distanced to allow more COVID-19-related infections is not supported here. These study data demonstrate that the use of preventive ivermectin significantly lowers the infection rate and that benefits outweigh the speculated increased risk of changes in social behaviors. Hence, we can speculate that the prophylactic use of ivermectin could play an important role in the reduction of the pandemic burden. Even after adjustments to measure the most relevant variables that could influence COVID-19-related outcomes, including age, sex, comorbidities, and habits, aiming to avoid overestimation of the effects of ivermectin and to resemble a randomized clinical trial, prophylactic ivermectin proved to be protective for the overall population, with a reduction of 68% in mortality rate and p < 0.0001 after employment of PSM. The protection provided by ivermectin when used prophylactically for COVID-19 may have reflected in the reduction in COVID-19 hospitalization and mortality rates observed at a population level. Compared to before the beginning of the program, COVID-19 hospitalization and mortality rates were reduced by 73% and 59%, respectively (p < 0.0001 for both). These reductions were obtained when the overall population and the number of COVID-19 cases, hospitalizations, and deaths in the city of Itajaí were considered, irrespective of the percentage of patients using ivermectin prophylactically. There were no changes in SARS-CoV-2 variants, infectivity, and pathogenicity before and during the program. When compared to all other major cities in the state of Santa Catarina, differences in COVID-19 mortality rate before July 7, 2020, and between July 7, 2020, and December 21, 2020, Itajaí was ranked number one [35]. These results indicate that medical-based optional prescription and citywide covered ivermectin can have a positive impact on the healthcare system. However, the present results do not provide sufficient support for the hypothesis that ivermectin could be an alternative to COVID-19 vaccines. Due to a large number of participants, this citywide program was unable to supervise whether ivermectin users were using ivermectin regularly, although the accumulated number of ivermectin tablets was strictly controlled. This occurred to be a potential conservative bias since the effects of ivermectin on prophylaxis could be underestimated due to adherence to the recommended frequency of ivermectin use. While ivermectin is a multi-target drug [36], its maximum benefits occur when it is present at a minimum concentration in a wide range of sites to inhibit multiple metabolic and inflammatory pathways. However, although the dose of ivermectin employed in the program was smaller than the minimum to reach the concentration required to act in these multiple sites, the reduction in infection, mortality, and death rates in the infected group that used ivermectin prophylactically was surprisingly lower. Long-term or accumulated ivermectin could also play a critical role in its long-term protection against COVID-19. LimitationsBeing a prospective observational study that allowed subjects to self-select between treatment vs. non-treatment instead of relying on randomization, important confounders may have been differentially present, which could otherwise explain the differences observed. Given that the benefits measured occurred despite negative risk factors being more present in the treatment group, this suggests the benefits are likely accurate and unbiased. Further, studies relying on PSM techniques have been shown to consistently agree with those employing randomization [37,38], again supporting the likelihood that the benefits measured are accurate. The prevailing type of SARS-CoV-2 in the city was unknown due to the lack of genotyping surveillance during the period of the program. Whether the prophylaxis proposed in this program would be as effective in other SARS-CoV-2 variants is unclear. Also, there was no strict control on whether infected subjects used any specific drug in case of COVID-19 infection, and this allows the possibility that the differences may be explained by differences in the use of ivermectin or other medications as treatment. Final discussionIn this citywide ivermectin prophylaxis program, a large, statistically significant decrease in mortality rate was observed after the program began among the entire population of city residents. When comparing subjects that used ivermectin regularly, non-users were two times more likely to die from COVID-19 while ivermectin users were 7% less likely to be infected with SARS-CoV-2 (p = 0.003). Although this study is not a randomized, double-blind, placebo-controlled clinical trial, the data were prospectively collected and resulted in a massive study sample that allowed adjustment for numerous confounding factors, thus strengthening the findings of the present study. Due to the well-established, long-term safety profile of ivermectin, with rare adverse effects, the absence of proven therapeutic options to prevent death caused by COVID-19, and lack of effectiveness of vaccines in real-life all-cause mortality analyses to date, we recommend that ivermectin be considered as a preventive strategy, in particular for those at a higher risk of complications from COVID-19 or at higher risk of contracting the illness, not as a substitute for COVID-19 vaccines, but as an additional tool, particularly during periods of high transmission rates. ConclusionsIn a citywide ivermectin program with prophylactic, optional ivermectin use for COVID-19, ivermectin was associated with significantly reduced COVID-19 infection, hospitalization, and death rates from COVID-19.
References
- Chen IS, Kubo Y: Ivermectin and its target molecules: shared and unique modulation mechanisms of ion channels and receptors by ivermectin. J Physiol. 2018, 596:1833-45. 10.1113/JP275236
- Kaur H, Shekhar N, Sharma S, Sarma P, Prakash A, Medhi B: Ivermectin as a potential drug for treatment of COVID-19: an in-sync review with clinical and computational attributes. Pharmacol Rep. 2021, 73:736-49. 10.1007/s43440-020-00195-y
- Martin RJ, Robertson AP, Choudhary S: Ivermectin: an anthelmintic, an insecticide, and much more. Trends Parasitol. 2021, 37:48-64. 10.1016/j.pt.2020.10.005
- Mastrangelo E, Pezzullo M, De Burghgraeve T, et al.: Ivermectin is a potent inhibitor of flavivirus replication specifically targeting NS3 helicase activity: new prospects for an old drug. J Antimicrob Chemother. 2012, 67:1884-94. 10.1093/jac/dks147
- Wagstaff KM, Sivakumaran H, Heaton SM, Harrich D, Jans DA: Ivermectin is a specific inhibitor of importin α/β-mediated nuclear import able to inhibit replication of HIV-1 and dengue virus. Biochem J. 2012, 443:851-6. 10.1042/BJ20120150
- Crump A: Ivermectin: enigmatic multifaceted 'wonder' drug continues to surprise and exceed expectations. J Antibiot (Tokyo). 2017, 70:495-505. 10.1038/ja.2017.11
- Heidary F, Gharebaghi R: Ivermectin: a systematic review from antiviral effects to COVID-19 complementary regimen. J Antibiot (Tokyo). 2020, 73:593-602. 10.1038/s41429-020-0336-z
- Li N, Zhao L, Zhan X: Quantitative proteomics reveals a broad-spectrum antiviral property of ivermectin, benefiting for COVID-19 treatment. J Cell Physiol. 2021, 236:2959-75. 10.1002/jcp.30055
- Jin L, Feng X, Rong H, et al.: The antiparasitic drug ivermectin is a novel FXR ligand that regulates metabolism. Nat Commun. 2013, 4:1937. 10.1038/ncomms2924
- Yang JS, Qi W, Farias-Pereira R, Choi S, Clark JM, Kim D, Park Y: Permethrin and ivermectin modulate lipid metabolism in steatosis-induced HepG2 hepatocyte. Food Chem Toxicol. 2019, 125:595-604. 10.1016/j.fct.2019.02.005
- Cairns DM, Giordano JE, Conte S, Levin M, Kaplan DL: Ivermectin promotes peripheral nerve regeneration during wound healing. ACS Omega. 2018, 3:12392-402. 10.1021/acsomega.8b01451
- Zheng YY, Ma YT, Zhang JY, Xie X: COVID-19 and the cardiovascular system. Nat Rev Cardiol. 2020, 17:259-60. 10.1038/s41569-020-0360-5
- Nagai H, Satomi T, Abiru A, et al.: Antihypertrophic effects of small molecules that maintain mitochondrial ATP levels under hypoxia. EBioMedicine. 2017, 24:147-58. 10.1016/j.ebiom.2017.09.022
- Park A, Iwasaki A: Type I and type III interferons - induction, signaling, evasion, and application to combat COVID-19. Cell Host Microbe. 2020, 27:870-8. 10.1016/j.chom.2020.05.008
- Zhang X, Song Y, Ci X, et al.: Ivermectin inhibits LPS-induced production of inflammatory cytokines and improves LPS-induced survival in mice. Inflamm Res. 2008, 57:524-9. 10.1007/s00011-008-8007-8
- Okeahialam BN: Serine protease inhibitors could be of benefit in the treatment of COVID-19 disease. Ther Adv Infect Dis. 2021, 8:10.1177/20499361211032048
- Matsuyama T, Kubli SP, Yoshinaga SK, Pfeffer K, Mak TW: An aberrant STAT pathway is central to COVID-19. Cell Death Differ. 2020, 27:3209-25. 10.1038/s41418-020-00633-7
- Wang K, Gao W, Dou Q, Chen H, Li Q, Nice EC, Huang C: Ivermectin induces PAK1-mediated cytostatic autophagy in breast cancer. Autophagy. 2016, 12:2498-9. 10.1080/15548627.2016.1231494
- Dou Q, Chen HN, Wang K, et al.: Ivermectin induces cytostatic autophagy by blocking the PAK1/Akt axis in breast cancer. Cancer Res. 2016, 76:4457-69. 10.1158/0008-5472.CAN-15-2887
- Layhadi JA, Turner J, Crossman D, Fountain SJ: ATP evokes Ca2+ responses and CXCL5 secretion via P2X4 receptor activation in human monocyte-derived macrophages. J Immunol. 2018, 200:1159-68. 10.4049/jimmunol.1700965
- Juarez M, Schcolnik-Cabrera A, Dueñas-Gonzalez A: The multitargeted drug ivermectin: from an antiparasitic agent to a repositioned cancer drug. Am J Cancer Res. 2018, 8:317-31.
- Andersson U, Ottestad W, Tracey KJ: Extracellular HMGB1: a therapeutic target in severe pulmonary inflammation including COVID-19?. Mol Med. 2020, 26:42. 10.1186/s10020-020-00172-4
- Yan S, Ci X, Chen N, et al.: Anti-inflammatory effects of ivermectin in mouse model of allergic asthma. Inflamm Res. 2011, 60:589-96. 10.1007/s00011-011-0307-8
- Reis TA, Oliveira-da-Silva JA, Tavares GS, et al.: Ivermectin presents effective and selective antileishmanial activity in vitro and in vivo against Leishmania infantum and is therapeutic against visceral leishmaniasis. Exp Parasitol. 2021, 221:108059. 10.1016/j.exppara.2020.108059
- Scheim D: Ivermectin for COVID-19 treatment: clinical response at quasi-threshold doses via hypothesized alleviation of CD147-mediated vascular occlusion. [PREPRINT]. SSRN J. 2020, 10.2139/ssrn.3636557
- Ci X, Li H, Yu Q, et al.: Avermectin exerts anti-inflammatory effect by downregulating the nuclear transcription factor kappa-B and mitogen-activated protein kinase activation pathway. Fundam Clin Pharmacol. 2009, 23:449-55. 10.1111/j.1472-8206.2009.00684.x
- Zaidi AK, Dehgani-Mobaraki P: The mechanisms of action of Ivermectin against SARS-CoV-2: an evidence-based clinical review article. [PREPRINT]. J Antibiot (Tokyo). 2021, 10.1038/s41429-021-00430-5
- Kalfas S, Visvanathan K, Chan K, Drago J: The therapeutic potential of ivermectin for COVID- 19: a systematic review of mechanisms and evidence. [PREPRINT]. medRxiv. 2020, 10.1101/2020.11.30.20236570
- Behera P, Patro BK, Singh AK, et al.: Role of ivermectin in the prevention of SARS-CoV-2 infection among healthcare workers in India: a matched case-control study. PLoS One. 2021, 16:e0247163. 10.1371/journal.pone.0247163
- Hellwig MD, Maia A: A COVID-19 prophylaxis? Lower incidence associated with prophylactic administration of ivermectin. Int J Antimicrob Agents. 2021, 57:106248. 10.1016/j.ijantimicag.2020.106248
- World Medical Association. Declaration of Helsinki. (2020). Accessed: December 24, 2021: https://www.wma.net/what-we-do/medical-ethics/declaration-of-helsinki/doh-oct2000/.
- Conselho Federal de Medicina (CFM). Código de Ética Médica. (2010). Accessed: December 24, 2021: https://portal.cfm.org.br/images/stories/biblioteca/codigo%20de%20etica%20medica.pdf.
- Nguyen TL, Collins GS, Spence J, Daurès JP, Devereaux PJ, Landais P, Le Manach Y: Double-adjustment in propensity score matching analysis: choosing a threshold for considering residual imbalance. BMC Med Res Methodol. 2017, 17:78. 10.1186/s12874-017-0338-0
- Zhang Z, Kim HJ, Lonjon G, Zhu Y: Balance diagnostics after propensity score matching. Ann Transl Med. 2019, 7:16. 10.21037/atm.2018.12.10
- Boletim epidemiológico do estado de Santa Catarina, Brasil. (2020). Accessed: December 24, 2021: http://www.dive.sc.gov.br.
- Choudhury A, Das NC, Patra R, Bhattacharya M, Ghosh P, Patra BC, Mukherjee S: Exploring the binding efficacy of ivermectin against the key proteins of SARS-CoV-2 pathogenesis: an in silico approach. Future Virol. 2021, 16:277-91. 10.2217/fvl-2020-0342
- Austin PC: An introduction to propensity score methods for reducing the effects of confounding in observational studies. Multivariate Behav Res. 2011, 46:399-424. 10.1080/00273171.2011.568786
- Propensity score matching in the absence of randomized controlled trials: a case study on the effects of breastfeeding on childhood obesity. (2020). Accessed: December 24, 2021: https://methods.sagepub.com/case/propensity-score-matching-trials-effects-breastfeeding-childhood-obesity.
|