Farmeco 2020;21(Suppl 1).html

Farmeconomia. Health economics and therapeutic pathways 2020; 21(Suppl 1): 3-20

https://doi.org/10.7175/fe.v21i1S.1490

Supplement

Clinical and Economic Rationale for the Early use of SGLT2 Inhibitors in Patients with Type 2 Diabetes

Edoardo Mannucci 1, Pier Paolo Mangia 2, Lorenzo Pradelli 2

1 Diabetology, Careggi Hospital and University of Florence, Florence, Italy

2 AdRes HE&OR, Turin, Italy

Abstract

Type 2 diabetes (T2D) is a chronic disease associated with a high epidemiological and economic burden. It is associated with a high risk of developing both macrovascular and microvascular complications and cardiovascular diseases represent the main cause of mortality and morbidity in T2D patients. The economic impact of diabetes is primarily due to the cost and duration of treatment and secondary complications of diabetes and associated costs. Sodium-glucose co-transporter-2 (SGLT2) inhibitors are an effective therapy for providing a long-term improvement of glucose control, thus contributing to the long-term prevention of diabetic (particularly microvascular) complications. Furthermore, SGLT-2 inhibitors seem to lead to significant reductions in hospital admissions due to heart failure and progression of renal disease, regardless of baseline atherosclerotic risk category or history of heart failure. Evidence from randomized controlled trials, observational and pharmacoeconomic studies suggest that SGLT2 inhibitors should be considered not only in patients with established cardiovascular disease and incipient nephropathy but also in earlier stages of T2D in order to prevent the first onset of cardiovascular and renal complications and contain the cost of illness.

Keywords

SGLT2i, Pharmacoutilization; Cardiovascular protection; Complications, Type 2 diabetes

Corresponding author

Lorenzo Pradelli

l.pradelli@adreshe.com


Received: 4 December 2020

Accepted: 15 December 2020

Published: 23 December 2020

Cardiovascular Protection in Type 2 Diabetes

Prevalence of diabetes in Italy

The data reported by ISTAT 2019 indicate that diabetes affects 5.6% of Italians (5.8% of males and 5.3% of females), equal to over 3 million people [1]. According to the most recent ARNO Observatory data, related to over 11 million subjects, in 2018 the total prevalence rate of diabetes in Italy was 6.2% for a total of about 3,750,000 patients which reaches over 4 million when patients who do not take drugs supplied by the Italian National Health Service and undiagnosed diabetic patients are taken into account [2]. The same ARNO register also revealed a gender difference in the prevalence of diabetes (6.5% in men versus 5.9% in women), which had already emerged since 2000 and remained unchanged over the time [2]. According to the population projections of the International Diabetes Federation, the prevalence of diabetes in Italy could reach 10.4% in the age group 20-79 years in 2030 [3].

Type 2 diabetes and cardiovascular risk

Type 2 diabetes (T2D) is a chronic disease associated with a high epidemiological and economic burden. National epidemiological data suggest about 250,000 new diagnoses of type 2 diabetes and about 25,000 new diagnoses of type 1 diabetes every year [4]. According to data from the 2020 AMD (Associazione Medici Diabetologi) Annals, related to 473,740 patients from 258 Italian diabetes centers, in 2018 in Italy there were 52,111 first admissions of which 6% (n=3,126) were new diagnosis [5].

T2D is associated with a high risk of developing both macrovascular (atherosclerotic cardiovascular disease and heart failure) and microvascular (chronic kidney disease, and eye and nerve damages) complications [6,7]. In particular, cardiovascular diseases represent the main cause of mortality and morbidity in T2D patients, with an increase of mortality of about 30-40% compared to non-diabetic population [8,9]. Furthermore, T2D is a key and independent risk factor for the development of heart failure [10]. Features of diabetes which are associated with the development of heart failure include poor glycemic control, longer duration of T2D, insulin treatment, and the presence of microvascular complications [11,12].

According to the ARNO Observatory Data, in 2018 heart failure was the second cause of hospitalization in diabetic patients and among the top twenty causes of hospitalization, cardiovascular diseases represent about 20% of all diagnosis: heart failure (5,4% of all hospitalization, n = 8.594), other forms of chronic ischemic heart disease (2.8%; n = 4.401), cardiac arrhythmias (2.2%, n = 3.546), acute myocardial infarction (2.2%; n = 3544), occlusion of the cerebral arteries (1.3%, n = 2.091), hypertensive heart disease (1.3%, n = 2.048), other cerebral vasculopathies (1.1%, n = 1813) and atherosclerosis (1.3%, n = 2.093) [2].

According to AMD Annals, in 2018 the proportion of patients with history of myocardial infarction was 7.5%, higher than in 2016 (4%) but lower than in 2015 (9.3%) [5]. Similarly, the proportion of patients with a history of cardiovascular disease (composite outcome which include infarction, stroke, coronary or peripheral revascularization, or coronary or peripheral bypass) increased from 12.8% in 2016 to 14.2% in 2018 (in 2015 it was 18.1%), while the proportion of patients with a history of stroke decrease from 3.5% to 2.6% (in 2015 was 4.6%), respectively [5]. Regarding chronic kidney disease, the AMD Annals showed a slight increase in the prevalence of glomerular filtration rate (GFR) <60 ml/min/1.73m2 from 2016 (26.1%) to 2018 (29.1%), doubling the proportion of those with GFR <30 ml/min/1.73m2 (from 3.69% to 7.1%). Conversely, micro/macroalbuminuria, which is a cardiovascular and renal failure marker, showed a slight reduction (from 41.6% to 36.4%) despite a rather high prevalence [5].

Historically, the therapeutic goal in T2D patients was the attainment and maintenance of good glycemic control, aimed at reducing microvascular complications; however, results from randomized controlled trials showed that it was not sufficient and, despite intensive glycemic control, a residual microvascular and macrovascular (i.e. heart failure) risk still remain [13,14]. In fact, large-scale trials specifically designed for the assessment of the long-term effects of the improvement of glycemic control on diabetic complications showed that the reduction in the incidence of major cardiovascular events determined by the intensification of diabetes therapy is small, although statistically significant [13]. The unexpected results of some cardiovascular outcome trials (CVOTs) requested by regulatory agencies bodies for cardiovascular safety assessment of antidiabetic drugs changed this picture, shifting the focus of the therapeutic approach to T2D from glycemic control to long-term benefit on cardiovascular and microvascular outcomes.

In recent years, several new classes of antidiabetic drugs with different mechanism of action have been introduced in the treatment of type 2 diabetes. Some of those drugs have been shown, beyond their glucose-lowering effects, to prevent the onset and progression of cardiovascular and renal complications of T2DM. Among new glucose-lowering agents, sodium-glucose co-transporter-2 (SGLT2) inhibitors and glucagon-like peptide-1 (GLP1) receptor agonists showed the best cardiovascular outcomes with a reduction of cardiovascular risk and a protective effect on composite endpoint which include cardiovascular deaths, myocardial infarction and stroke. In particular, SGLT2 inhibitors showed to reduce the risk of cardiovascular death and hospitalization for heart failure and, at the same time, to reduce the progression of diabetic nephropathy [15-18]. The network meta-analysis of Fei et al. compared cardiovascular effects among different classes of new antidiabetic drugs, including SGLT2 inhibitors, GLP-1 receptor agonists, and dipeptidyl-peptidase-4 (DPP-4) inhibitors [19]. The results of the meta-analysis, which included 14 trials for a total of 121,047 patients, showed clear superiority of SGLT2 inhibitors in reducing cardiovascular and all-cause deaths compared to both placebo (OR = 0.82; 95% CI = 0.73-0.93 and OR = 0.84, 95% CI = 0.77-0.92) and DPP-4 inhibitors (OR = 0.83; 95% CI = 0.70-0.99 and OR = 0.83, 95% CI = 0.73-0.94), and in reducing hospitalization for heart failure (OR = 0.79; 95% CI: 0.69-0.90) and renal composite outcome (OR = 0.69, 95% CI = 0.59-0.80) compared to GLP-1 receptor agonists.

Due to their beneficial cardiovascular and renal effects, SGLT2 inhibitors have been proposed as a tool for the prevention and treatment of heart failure in patients with T2D, regardless of the presence of established cardiovascular disease. Indeed, while metformin associated with a comprehensive lifestyle intervention represents the first choice of treatment in patients with T2D, most recent guidelines suggest the early use of SGLT2 inhibitors in patients with T2D who present high atherosclerotic cardiovascular Disease (ASCVD) or heart failure risk [20-23].

Rationale for the early use of SGLT2 inhibitors

Mechanism of action

The benefit of SGLT2 inhibitors is linked to the mechanism of action of these drugs that lower blood glucose by preventing the reabsorption of glucose and sodium from the proximal renal tubule of the kidney, resulting in glycosuria and reduction of glycemia [24-26]. Furthermore, glycosuria leads to several metabolic changes (i.e. increased lipolysis, with consequent consumption of adipose tissue, and fat and weight loss) that have been associated with inflammation, fibrosis, and atherosclerosis reduction [26,27]. Moreover, the antihyperglycemic effect of SGLT2 inhibitors is long lasting, allowing a long-term control of microvascular complications, and since the mechanism of action is independent of beta cell function and insulin pathway, SGLT2 inhibitors are associated with a low risk of hypoglycemia [18,28,29]. The reduction of sodium reabsorption leads to natriuretic and diuretic effects which are responsible for the antihypertensive effect [26,27,30] and the reduction of eGFR due to a reduction in blood pressure within the glomerulus, followed by long-term preservation of renal function, is associated with the nephroprotective effect of SGLT2 inhibitors [31,32].

Cardiovascular protection, in particular the improvement of myocardial function and the reduction of the heart failure risk associated with the use of SGLT2 inhibitors, seems to be related to the hemodynamic and metabolic effect on glycemic control, reduction of blood pressure, decrease in preload, afterload and arterial stiffness, prevention of albuminuria progression, and preservation of eGFR [26,27,33]. On the other hand, a direct myocardial effect is unlikely, since SGLT2 is not expressed by myocardiocytes [34]

img_01.jpg

Figure 1. Primary mechanism of action of SGLT2 inhibitors

Clinical trials results

SGLT2 inhibitors, like all other new antidiabetic drugs, have been assessed in cardiovascular outcome trials (CVOT) required by both Food and Drug Administration to exclude and prevent an excessive increase of cardiovascular risk. Individual CVOTs, conducted on both patients with well-established ASCVD and patients without the disease but with cardiovascular risk factor, showed that SGLT2 inhibitors reduce the risk of major cardiovascular events [35-39].

Since no single trial was adequately powered to test the heterogeneity of cardiovascular efficacy by baseline ASCVD risk, Zelniker et al. [15] performed a systematic review and meta-analysis of CVOTs conducted on empagliflozin (EMPA-REG OUTCOME), canagliflozin (CANVAS program) and dapagliflozin (DECLARE-TIMI). The meta-analysis, which included 34,322 patients (60.2% of whom with established ASCVD), showed that SGLT2 inhibitors reduced the risk of cardiovascular death or hospitalization for heart failure by 23% (HR = 0.77; 95% CI = 0.71-0.84; p<0·0001) and the risk of progression of renal disease by 45% (HR = 0.55; 95% CI = 0.48-0.64), p<0.0001). In both cases, the benefit was similar in patients with and without ASCVD. On the other hand, the effect on Major Cardiovascular Events (MACE) was evident only in patients with ASCVD, but not in those without [15].

Also, the meta-analysis of Arnott et al. [16] aimed to define the cardiovascular benefits of SGLT2 inhibitors across patient subsets (i.e. with and without established CVD, reduced kidney function, or heart failure) and included the three CVOTs analyzed by Zelniker et al. (EMPA-REG, CANVAS, and DECLARE-TIMI) plus the CREDENCE trial on canagliflozin, for a total of 38723 patients. The efficacy outcomes included were major adverse cardiac events, cardiovascular death, fatal or nonfatal myocardial infarction, fatal or nonfatal stroke, hospitalization for heart failure, a composite outcome for cardiovascular death or hospitalization for heart failure, and all-cause mortality. The results highlighted an overall benefit on major adverse cardiac events (HR = 0.88; 95% CI = 0.82-0.94; p<0.001). Although no statistically significant difference for any of the efficacy outcomes was detected between patients with or without cardiovascular disease at baseline, there was a trend for a greater benefit on MACE and except for cardiovascular death in patients with cardiovascular disease at baseline. Notably, in the subgroups without established cardiovascular disease, SGLT-2 inibitors significantly reduced hospitalization of heart failure heart failure (HR, 0.63; 95% CI, 0.50–0.80) and the composite outcome of cardiovascular death and heart failure (HR, 0.81; 95% CI, 0.69–0.96). Finally, the presence of a history of heart failure at baseline did not affect the beneficial effect of SGLT2 inhibitors on all efficacy outcomes, including the risk of hospitalization for heart failure.

More recently, the meta-analysis of McGuire et al. [17] assessed the cardiovascular and kidney outcomes of SGLT2 inhibitors including data from the VERTIS CV trial on ertugliflozin. The analysis, which included 46,969 patients, showed an association between the use of SGLT2 inhibitors and the reduction of the risk of major adverse CV events (HR = 0.90; 95% CI = 0.85-0.95), hospitalization for heart failure or cardiovascular death (HR = 0.78; 95% CI = 0.73-0.84), and kidney outcomes (HR = 0.62; 95% CI = 0.56-0.70). In particular, a significant consistency across the trial was observed for the reduction of the risk of hospitalization for heart failure (HR = 0.68; 95% CI = 0.61-0.76). Finally, the differences in outcomes of SGLT2 inhibitor treatment between patients with or without established cardiovascular disease, and with or without a history of heart failure, were not statistically significant.

Real-world outcomes

In addition to clinical trials, several real-world observational studies have been conducted to assess cardiovascular outcomes in large cohorts of patients treated with SGLT2 inhibitors and compare them with those of patients treated with other glucose-lowering drugs. In the CVD-REAL study [40], which enrolled 309,056 patients from US, UK, Denmark, Germany, Norway, and Sweden, the use of empagliflozin, dapagliflozin and canagliflozin was associated with a significantly lower incidence of hospitalization for heart failure (HR = 0.61; 95% CI = 0.51-0.73; p<0.001), death (HR = 0.49; 95% CI = 0.41-0.57; p<0.001), and hospitalization for heart failure or death (HR = 0.54; 95% CI = 0.48-0.60; p<0.001), compared to other glucose-lowering drugs. These findings were consistent across patients with (87%) and without the presence of established cardiovascular disease at baseline. Similarly, in the CVD-REAL 2 study [41], which enrolled 470,128 patients in Asia-Pacific region, Middle East and North America, the use of SGLT2 inhibitors were associated with a lower risk of death (HR = 0.51; 95% CI = 0.37-0.70; p<0.001), hospitalization for heart failure (HR = 0.64; 95% CI = 0.50-0.82; p=0.001), hospitalization for heart failure or death (HR = 0.60; 95% CI = 0.47-0.76; p<0.001), myocardial infarction (HR = 0.81; 95% CI = 0.74-0.88; p < 0.001), and stroke (HR = 0.68; 95% CI = 0.55-0.84; p<0.001), compared to other glucose-lowering drugs, both in patients with (27%) and without established cardiovascular disease at baseline. Finally, the OBSERVE-4D study [42], which analyzed data from four large US administrative databases for a total of 714,582 patients, showed a greater reduction of hospitalization for heart failure with SGLT2 inhibitors compared to other glucose-lowering drugs (HR = 0.43, 95% CI = 0.30-0.62; p = 0.01) regardless of the presence or not of prior cardiovascular disease.

Moreover, the first interim analysis of the Empagliflozin Comparative Effectiveness and Safety (EMPRISE) study [43], which investigated the risk of hospitalization for heart failure among 16,443 T2D patients initiating empagliflozin vs. sitagliptin in routine care, showed that the initiation of empagliflozin was associated with a decreased risk of hospitalization for heart failure, both specific (defined as a heart failure discharge diagnosis in the primary position) and broad (defined as a heart failure discharge diagnosis in any position) of about 50%. The results, which remained consistent among patients with (about 25% of patients) and without (about 75% of patients) baseline cardiovascular disease, were comparable in timing and magnitude to those obtained in the EMPA-REG OUTCOME trial in which empagliflozin showed to reduce the risk of hospitalization for heart failure by 35% when added to standard of care in patients with established cardiovascular disease [35].

Finally, in order to determine the cardiovascular and long-term non-CV safety of SGLT2 inhibitors, Zhang et al. [18] performed a meta-analysis which included 5 RCTs and observational studies comprising 351,476 patients (for CV outcomes) and a median follow-up of 3.1 years, and 9 RCTs comprising 23,035 patients (long-term non-CV outcomes) with a medium follow-up of 2 years. Meta-analyses on CV outcomes showed that SGLT2 inhibitors significantly reduced the risks of major adverse cardiac events (HR = 0.80; 95% CI = 0.69-0.92; p=0.002), all-cause mortality (HR: 0.67; 95% CI, 0.54–0.84;P<0.001), cardiovascular mortality (HR: 0.77; 95% CI, 0.60–0.98;P=0.03), nonfatal myocardial infarction (HR: 0.86; 95% CI, 0.76–0.98;P=0.02), hospitalization for heart failure (HR: 0.62; 95% CI, 0.55–0.69;P<0.001), and progression of albuminuria (HR: 0.68; 95% CI,0.58–0.81;P<0.001). Pooled analysis of non-CV outcomes showed that SGLT2 inhibitors were associated with less serious adverse events OR: 0.90; 95% CI, 0.81–1.00;P=0.05), hypoglycemia (OR: 0.48; 95% CI,0.28–0.82;P=0.008), and acute kidney injury (OR:0.80; 95% CI, 0.67–0.96;P=0.014) than controls. Furthermore, SGLT2 inhibitors showed to significantly reduce HbA1c levels, fasting blood glucose, body weight, systolic and diastolic blood pressure compared with controls.

Observational studies (i.e., the so called “real-world” studies) are by no means a substitute for clinical trials. In fact, since the choice of a drug is affected by the characteristics of patients, and only some of those confounders can be appropriately measured and accounted for in analyses. However, when concordant with the results of clinical trials, as in the case of SGLT2 inhibitors, the results of those observational studies suggest that the benefit of SGLT2 inhibitors on cardiovascular outcomes is not limited to the selected populations enrolled in randomized trials.

Conclusions

SGLT2 inhibitors are an effective therapy for providing a long-term improvement of glucose control, thus contributing to the long-term prevention of diabetic (particularly microvascular) complications. Beyond their effects on blood glucose, SGLT2 inhibitors prevent and reduce the progression of cardiovascular disease and nephropathy. Although the effect on some outcomes (i.e., MACE) appears to be possibly greater in patients with established disease, for many other endpoints (e.g., hospitalization of heart failure and decline of renal function) SGLT2 inhibitors seem to effective independently of the baseline characteristics of patients. In addition, observational studies confirm the results of clinical trials in wider and unselected populations of patients. On the basis of all this evidence, SGLT2 inhibitors should be considered among treatments of choice in patients with established cardiovascular disease and incipient nephropathy. At the same time, they appear to be a very interesting treatment option also in earlier stages of the natural history of type 2 diabetes, as a means of effectively preventing the first onset of cardiovascular and renal complications.

Pharmacoutilization of SGLT2 Inhibitors in Italy

Utilization trend

In Italy SGLT2 inhibitors are available since 2015 and, according to OsMed data, their use in monotherapy grew from 0.1 DDD/1000 inhabitants in 2015 to 1.3 DDD/1000 inhabitants in 2019. Similarly, the association with metformin increased from 0.2 DDD/1000 inhabitants in 2016 to 1.4 in 2019 DDD/1000 inhabitants with a percentage increase from 2018 to 2019 equal to 38.4% for the association and to 27.7% for the monotherapy [44].

AMD Annals also showed that the use of SGLT2 inhibitors grew from 2016 to 2018 (from 4.0% to 9.5%) even if they remain in fifth place after metformin (69.4%), insulin (32.4%), DPP-4 inhibitors (21.1%), and sulfonylureas (16.2%) [5].

Patients (n)

2016

2018

SGLT2 inhibitors

11,661

16,962

Empagliflozin

4,100

8,411

Dapagliflozin

2,681

6,086

Metformin + dapagliflozin

2,265

7,197

Metformin + empagliflozin

1,280

8,661

Metformin + canagliflozin

908

1,227

Canagliflozin

518

748

Table I. Utilization of SGLT2 inhibitors in 2016 and 2018 [2,45]

Finally, the ARNO Observatory Data showed a great increase in the use of SGLT2 inhibitors from 2016 to 2018 (Table I). In 2016, 85% (n = 516,073) of diabetic patients detected by ARNO Observatory were treated with antidiabetic drugs and, among those who were treated with oral and injectable (non-insulin) hypoglycemic drugs (n = 446,320), 11,661 (2.3%) were treated with SGLT2 inhibitors: 7,236 (1.4%) in monotherapy and 4,425 (0.9%) in association with metformin. In particular, 4,100 (0.8%) patients were treated with empagliflozin, 2,681 (0.5%) with dapagliflozin, 2,265 (0.4%) with metformin + dapagliflozin, 1,280 (0.2%) with metformin + empagliflozin, 908 (0.2) with metformin + canagliflozin, and 518 (0,1%) with canagliflozin [45].

In 2018, 88% (n = 619,849) of diabetic patients detected by ARNO Observatory were treated with antidiabetic drugs, and among those who were treated with oral and injectable (non-insulin) hypoglycemic drugs (n = 545,391), 32,071 (5.1%) were treated with SGLT2 inhibitors: 16,962 (2.7%) in association with metformin and 15,109 (2.4%) in monotherapy. In particular, 8,661 (1.4%) were treated with metformin + empagliflozin, 8,411 (1.4%) with empagliflozin, 7,197 (1.2%) with metformin + dapagliflozin, 6,086 (1.0%) with dapagliflozin, 1,227 (0.2%) with metformin + canagliflozin, and 748 (0.1%) with canagliflozin [2].

Pharmaceutical expenditure

Years

Total expenditure on anti-diabetic drugs (€)

2015

867,000,000

2016

897,000,000

2017

904,000,000

2018

945,400,000

2019

1,010,000,000

Table II. Total expenditure on antidiabetic drugs [44]

Years

Total expenditure on SGLT2 inhibitors (€)

Delta

2016

23,700,000

2017

44,100,000

86%

2018

62,800,000

43%

2019

80,100,000

28%

Table III. Total expenditure on SGLT2 inhibitors [44]

Treatment

2016

2018

Empagliflozin

€ 6.319.000

€ 14.305.000

Empagliflozin + Metformin

€ 1.242.000

€ 14.952.000

Canagliflozin

€ 774.388

€ 1.262.576

Canagliflozin + Metformin

€ 1.458.113

€ 2.227.085

Dapagliflozin

€ 4.059.408

€ 9.681.116

Dapagliflozin + Metformin

€ 3.421.380

€ 12.728.565

Table IV. Total expenditure on SGLT2 inhibitors [2]

According to OsMed data, pharmaceutical expenditure on antidiabetic drugs, which has grown steadily in recent years, has likewise reached the extraordinary amount of 1 billion (Table II) [44].

Also, with regards to SGLT2 inhibitors, based on OsMed data, in recent years there has been a considerable increase in expenditure, which is gradually flattening out (Table III).

Likewise, according to a data extraction performed on the ARNO Report, the expenditure increased more than twice between 2016 and 2018, for empagliflozin and dapagliflozin, and about 50% for canagliflozin, including both monotherapy and association with metformin (Table IV). Note that there is a difference between the total expenditure recorded in the OsMed report and the ARNO report, in 2016 [45] and 2018 [2], of about 27% and 13% respectively, likely due to the use of different data sources.

Type 2 Diabetes: Burden of the Disease and Pharmacoeconomic Aspects of SGLT2 Inhibitors

Burden of disease

The burden diabetes is increasing due to both the progressively aging population and the worsening lifestyle. In addition to the increase in prevalence, a concomitant, dramatic rise in the clinical, social and economic burden of diabetes and its complications, especially those of the cardiovascular system, is also expected [46,47]. In fact, the management of its complications, is a major factor driving the total direct costs of diabetes.

Recently, the London School of Economics has carried out an analysis on the trend of diabetic disease in the five major European countries (including Italy) and found that the inpatient costs were steadily higher than outpatient costs because of rising medical costs linked to diabetes complications [48]. In a study of about 300,000 patients with diabetes in Lombardy, followed from 2000 to 2008, it was found that the average cost per patient per year was € 3,315.06. Hospitalizations were the main cost driver, contributing to 54.2% of total costs. Drugs were the second highest cost item (31.5%), while outpatient expenses contributed to 14.3% of total costs. Regarding hospital costs, 35.6% was ascribable to hospital admissions for cardio- or cerebrovascular causes as the leading diagnosis, 6% to other possible complications of diabetes and 4.3% to diabetes per se. Drugs account for 35% of total costs, noteworthy is that one third of which is imputable to cardiovascular reasons [49]. These results are comparable with those reported by Marchesini and colleagues who carried out an analysis based on a database of about 7 million Italian inhabitants: their assessment of the annual direct cost of diabetes was about € 2,600, half of which related to hospitalization, 30% to pharmacological treatment and the rest to the service use [50].

The data are even more impressive considering the total cost of the disease. In Italy, the amount spent by the National Health Fund to treat people with diabetes is about 15 billion euro per year, more than 10% of the total. This sum is calculated using the real costs of hospitalizations and the various specialist services [51]. When applying estimated prevalence rates to the annual costs of a diabetic patient, as calculated by the ARNO project basing on administrative databases analyses, the cost is likewise substantial and amounts to about 10 billion euro per year [52]. To this large amount of public money must be added about 3 billion euros of direct expenses incurred by people and their families and roughly 10-12 billion euro of indirect costs, many of which are borne by the state budget for early retirement and absences from work [53]. The total amounts to 25-30 billion euros.

Pharmacoeconomic analysis of SGLT2 inhibitors

According to some recent findings, SGLT2 inhibitors significantly reduced the risk for the composite of cardiovascular death or hospitalization for heart failure by 23% and hospitalization for heart failure by 31% in patients with or without atherosclerotic cardiovascular disease, and reduced the composite of worsening of renal function, end-stage renal disease, or renal death by 45%. These data suggest that SGLT2 inhibitors should be evaluated in patients with type 2 diabetes regardless of the presence of atherosclerotic cardiovascular disease or a history of heart failure [15]. Furthermore, considering that the probability of hospital admission for heart failure, expressed as OR, is 2.48 for individuals with diabetes versus controls, with an excess of hospitalization equal to 1,650 (expressed as number of hospitalizations per 100,000 persons/year), while the probability of hospital admission for renal complications, expressed as OR, is 2.82 with an excess of hospitalization equal to 1,817 [54], this class of drugs could contribute to a considerable saving of resources consequent to a reduction in hospitalizations.

Three systematic reviews of the pharmacoeconomic analysis of SGLT2i have recently been carried out.

The systematic review and meta-analysis conducted by Yoshida et al. [55] included a total of 24 studies in which SGLT2 inhibitors were compared vs. DPP4 inhibitors, sulfonylureas (SU), GLP1 receptor agonists, SGLT2 inhibitors, other antidiabetic therapies (including thiazolidinediones, alpha-glucosidase inhibitors or insulin), and standard care/metformin and most studies concluded SGLT2 inhibitors was cost-effective compared to comparators.

The review of Rahman et al. [56] included 37 cost-effectiveness studies of SGLT2 inhibitors: 15 analyzed dapagliflozin, 10 canagliflozin and 12 empagliflozin. Canagliflozin and dapagliflozin were both cost-effective compared to insulin or other oral agents thanks to the significant favorable impact of lower hypoglycemia, weight variations and improved efficacy of HbA1c on ICER.

Among the 12 pharmacoeconomic studies, which evaluated cost-effectiveness of empagliflozin in monotherapy, dual or triple therapy, or in comparison to the standard of care (as per the EMPA-REG trial, the standard of care included antidiabetic therapy taken alone or in combination with metformin, sulfonylurea, DPP4 inhibitors, thiazolidinedione, GLP1 receptor agonists and/or insulin with an HbA1c between 7% and 9% at enrolment), five were conducted in the UK, four in other European countries, one in Canada and two in the US. The main endpoints of efficacy assessed in most of the studies were ICER and QALY. In all the studies analyzed, empagliflozin was reported to be a cost-effective treatment option. On a background of metformin, empagliflozin was found to be cost-effective compared to DPP-4 inhibitor therapy; furthermore combination therapy (empagliflozin plus DPP-4 inhibitor) was cost-effective when compared to empagliflozin or DPP-4 inhibitor alone. Results were, however, sensitive to changes in age, gender and race. Analysis from the perspective of payers in several countries, including Italy, showed that treatment with empagliflozin was cost-effective and obtained QALYs compared to the standard of care treatment in individuals with diabetes and cardiovascular disease.

Different results have been documented by some cost-effectiveness studies between empagliflozin and dapagliflozin, probably due to different methods of models simulation. However, when CANVAS and EMPA-REG data were applied in an indirect treatment comparison, empagliflozin was associated with lower amputation (HR 0.58 vs. 1.08), renal injury (HR 0.33 vs. 0.47) and bone fracture (HR 1.12 vs. 1.45). This resulted in empagliflozin being cost-effective when compared to canagliflozin [56,57].

It is important to note, however, that none of these analyses have been conducted at the therapeutic class level that directly incorporate all clinical trial and real-world evidence data. The majority of the health benefits reported in these studies were determined by improved health-related quality of life (HRQoL) in relation to weight loss and low rates of hypoglycemia, with all estimates of the long-term benefit of CV resulting exclusively from changes in risk factors established using published risk equations that do not adequately reflect the cardiovascular event rates and related risk reductions observed in CVOTs [58,59].

The study of McEwan and colleagues [60] sought to evaluate the cost effectiveness of SGLT2i as a class, compared with placebo in addition to current standard of care, or compared with oral glucose-lowering drugs, in people with and without established atherosclerotic CV disease, using clinical trial and real-world evidence.

The model was indeed informed by the most up-to-date results from the CVD-REAL-2 observational study [41] and the meta-analysis of Zelniker et al. [15] of three CVOTs, i.e. CANVAS, EMPA-REG and DECLARE-TIMI. Thanks to the reduction in costs associated with the progression of renal disease, in tests carried out in the UK, SGLT2 inhibitors has been estimated to be cost saving compared to the control arm in all scenarios informed by real-world or experimental evidence. Indeed, despite the longer life expectancy predicted in the SGLT2 inhibitors arm, the lower incidence of complications and the progression of renal disease have led to a shortened time lived with comorbidity, with significant consequences for long-term costs and HRQoL.

With regard to pharmacoeconomic studies on empagliflozin published in Italy, apart from a cost-effectiveness study already included in the systematic review mentioned above, a Budget Impact Model has been developed from the NHS perspective [61]. Over a time horizon of three years, approximately 46,000 patients per year treated with sulfonylureas in monotherapy or association with metformin and/or insulin were considered potentially eligible.

To calculate the impact on the budget, the direct healthcare costs relating to: acquisition of antidiabetic drugs as the main therapy (empagliflozin, insulin, metformin, sulfonylureas) and as a rescue antidiabetic therapy; self-monitoring of blood glucose; management of severe hypoglycemic events; management of major cardiovascular events were taken into account.

Overall, a progressive replacement of sulfonylureas with empagliflozin (at a rate of 10%, 20% and 30%, respectively in the first, second and third year of analysis) does not increase expenditure by the NHS, even reducing the budget by 0.1%. The replacement of sulfonylureas with empagliflozin causes an increase in drug acquisition costs, which is fully offset by the reduction in the costs of self-monitoring of blood glucose, management of hypoglycemic events and cardiovascular events.

Conclusions

The goal of treatment in people with diabetes, with or without pre-existing cardiovascular disease, is to provide effective and affordable healthcare. One of the limitations of this care remains the high cost of medical therapy. The economic impact of diabetes therapy is primarily due to the cost and duration of treatment and secondary complications of diabetes such as renal disease and increased risk of cardiovascular disease and associated costs. SGLT-2 inhibitors may play an important role in providing long-term treatment and improving HbA1c, reducing cardiovascular mortality and all factors causing mortality, and ensuring cost-effective management. In addition, SGLT-2 inhibitors lead to significant reductions in progression of renal disease, regardless of baseline atherosclerotic risk category or history of heart failure. In particular, empagliflozin showed to significantly reduce the risk of hospitalization for heart failure in patients with established cardiovascular disease in the clinical trial EMPA-REG OUTCOME and this finding was confirmed in the real-world study EMPRISE which mainly involved patients without history of cardiovascular disease. These results reinforce the rationale for early use of SGLT2 inhibitors in order to prevent cardiovascular events and they can certainly play a key role in containing the costs of diabetes.

Funding

This supplement was realized with an unconditional contribution by Boehringer Ingelheim Italia S.p.A. and Eli Lilly. Italia

Conflict of interest

The Authors have nothing to disclose regarding the present work.

References

ISTAT. Annuario statistico italiano 2019. Available at: https://www.istat.it/it/files//2019/12/Asi-2019.pdf (last accessed November 2020)

2. Osservatorio ARNO diabete. Il profilo assistenziale della popolazione con diabete. Rapporto 2019

3. Shaw JE, Sicree RA, Zimmet PZ. Global estimates of the prevalence of diabetes for 2010 and 2030. Diabetes Res ClinPract 2010; 87: 4-14; https://doi.org/10.1016/j.diabres.2009.10.007

4. Società Italiana di Diabetologia. Il diabete in Italia. G. Bruno editore. Minerva Medica, Torino, 2012

5. Manicardi V, Aglialoro A, Cavallo M, et al. Diabete di tipo 2. AMD Annali 2020

6. Handelsman Y. Rationale for the Early Use of Sodium-Glucose Cotransporter-2 Inhibitors in Patients with Type 2 Diabetes. Adv Ther 2019; 36:2567-86; https://doi.org/10.1007/s12325-019-01054-w

7. Verma S, McMurray JJV. The Serendipitous Story of SGLT2 Inhibitors in Heart Failure. Circulation 2019; 139: 2537-41; https://doi.org/10.1161/CIRCULATIONAHA.119.040514

8. Bruno G, Merletti F, Boffetta P, et al. Impact of glycaemic control, hypertension and insulin treatment on general and cause-specific mortality: an Italian population-based cohort of type II (non-insulin-dependent) diabetes mellitus. Diabetologia 1999; 42: 297-301; https://doi.org/10.1007/s001250051154

9. Brun E, Nelson RG, Bennett PH, et al; Verona Diabetes Study. Diabetes duration and cause-specific mortality in the Verona Diabetes Study. Diabetes Care 2000; 23: 1119-23; https://doi.org/10.2337/diacare.23.8.1119

10. McMurray JJ, Gerstein HC, Holman RR, et al. Heart failure: a cardiovascular outcome in diabetes that can no longer be ignored. Lancet Diabetes Endocrinol 2014; 2: 843-51; https://doi.org/10.1016/S2213-8587(14)70031-2

11. Seferović PM, Petrie MC, Filippatos GS, et al. Type 2 diabetes mellitus and heart failure: a position statement from the Heart Failure Association of the European Society of Cardiology. Eur J Heart Fail 2018; 20: 853-872; https://doi.org/10.1002/ejhf.1170

12. Verma S, Wanner C, Zwiener I, et al.; EMPA-REG OUTCOME Investigators. Influence of Microvascular Disease on Cardiovascular Events in Type 2 Diabetes. J Am Coll Cardiol 2019; 73: 2780-2; https://doi.org/10.1016/j.jacc.2019.03.002

13. Turnbull FM, Abraira C, Anderson RJ, et al. Intensive glucose control and macrovascular outcomes in type 2 diabetes. Diabetologia 2009; 52: 2288-98; https://doi.org/10.1007/s00125-009-1470-0

14. Zoungas S, Arima H, Gerstein HC, et al.; Collaborators on Trials of Lowering Glucose (CONTROL) group. Effects of intensive glucose control on microvascular outcomes in patients with type 2 diabetes: a meta-analysis of individual participant data from randomised controlled trials. Lancet Diabetes Endocrinol 2017; 5: 431-7; https://doi.org/10.1016/S2213-8587(17)30104-3

15. Zelniker TA, Wiviott SD, Raz I, et al. SGLT2 inhibitors for primary and secondary prevention of cardiovascular and renal outcomes in type 2 diabetes: a systematic review and meta-analysis of cardiovascular outcome trials. Lancet 2019; 393: 31-9; https://doi.org/10.1016/S0140-6736(18)32590-X

16. Arnott C, Li Q, Kang A, et al. Sodium-Glucose Cotransporter 2 Inhibition for the Prevention of Cardiovascular Events in Patients With Type 2 Diabetes Mellitus: A Systematic Review and Meta-Analysis. J Am Heart Assoc 2020; 9: e014908; https://doi.org/10.1161/JAHA.119.014908

17. McGuire DK, Shih WJ, Cosentino F, et al. Association of SGLT2 Inhibitors With Cardiovascular and Kidney Outcomes in Patients With Type 2 Diabetes: A Meta-analysis . JAMA Cardiol 2020; e204511; https://doi.org/10.1001/jamacardio.2020.4511

18. Zhang XL, Zhu QQ, Chen YH, et al. Cardiovascular Safety, Long-Term Noncardiovascular Safety, and Efficacy of Sodium-Glucose Cotransporter 2 Inhibitors in Patients With Type 2 Diabetes Mellitus: A Systemic Review and Meta-Analysis With Trial Sequential Analysis. J Am Heart Assoc 2018; 7: e007165; https://doi.org/10.1161/JAHA.117.007165

19. Fei Y, Tsoi MF, Cheung BMY. Cardiovascular outcomes in trials of new antidiabetic drug classes: a network meta-analysis. Cardiovasc Diabetol 2019; 18: 112; https://doi.org/10.1186/s12933-019-0916-z

20. Arnett DK, Blumenthal RS, Albert MA, et al. 2019 ACC/AHA Guideline on the Primary Prevention of Cardiovascular Disease: A Report of the American College of Cardiology/American Heart Association Task Force on Clinical Practice Guidelines. Circulation 2019; 140: e596-e646; https://doi.org/10.1161/CIR.0000000000000678

21. Cosentino F, Grant PJ, Aboyans V, et al.; ESC Scientific Document Group. 2019 ESC Guidelines on diabetes, pre-diabetes, and cardiovascular diseases developed in collaboration with the EASD. Eur Heart J 2020; 41: 255-323; https://doi.org/10.1093/eurheartj/ehz486

22. Buse JB, Wexler DJ, Tsapas A, et al. 2019 Update to: Management of Hyperglycemia in Type 2 Diabetes, 2018. A Consensus Report by the American Diabetes Association (ADA) and the European Association for the Study of Diabetes (EASD). Diabetes Care 2020; 43: 487-93; https://doi.org/10.2337/dci19-0066

23. American Diabetes Association. Pharmacologic Approaches to Glycemic Treatment: Standards of Medical Care in Diabetes-2020. Diabetes Care 2020; 43(Suppl 1): S98-S110; https://doi.org/10.2337/dc20-S009

24. Candido R, Cucinotta D, Manicardi V. Applicabilità di una serie di studi clinici SGLT-2i CVOT in un ambito di popolazione real-world di pazienti con DM2. Le monografie degli Annali. AMD 2018

25. Ito M, Tanaka T. The anticipated renoprotective effects of sodium-glucose cotransporter 2 inhibitors. Intern Med 2018; 57: 2105-14; https://doi.org/10.2169/internalmedicine.9842-17

26. Heerspink HJ, Perkins BA, Fitchett DH, et al. Sodium glucose cotransporter 2 inhibitors in the treatment of diabetes mellitus: cardiovascular and kidney effects, potential mechanisms, and clinical applications. Circulation 2016; 134: 752-72; https://doi.org/10.1161/CIRCULATIONAHA.116.021887

27. Woo V, Connelly K, Lin P, et al. The role of sodium glucose cotransporter-2 (SGLT-2) inhibitors in heart failure and chronic kidney disease in type 2 diabetes. Curr Med Res Opin 2019; 35: 1283-95; https://doi.org/10.1080/03007995.2019.1576479

28. Thewjitcharoen Y, Yenseung N, Malidaeng A, et al. Effectiveness of long-term treatment with SGLT2 inhibitors: real-world evidence from a specialized diabetes center. Diabetol Metab Syndr 2017; 9: 96; https://doi.org/10.1186/s13098-017-0297-y

29. Li J, Gong Y, Li C, et al. Long-term efficacy and safety of sodium-glucose cotransporter-2 inhibitors as add-on to metformin treatment in the management of type 2 diabetes mellitus: A meta-analysis. Medicine (Baltimore) 2017; 96: e7201; https://doi.org/10.1097/MD.0000000000007201

30. Lytvyn Y, Bjornstad P, Udell JA, et al. Sodium glucose cotransporter-2 inhibition in heart failure: potential mechanisms, clinical applications, and summary of clinical trials. Circulation 2017; 136: 1643-58; https://doi.org/10.1161/CIRCULATIONAHA.117.030012

31. Nespoux J, Vallon V. SGLT2 inhibition and kidney protection. Clin Sci 2018; 132: 1329-39; https://doi.org/10.1042/CS20171298

32. Kohan DE, Fioretto P, Johnsson K, et al. The effect of dapagliflozin on renal function in patients with type 2 diabetes. J Nephrol 2016; 29: 391-400; https://doi.org/10.1007/s40620-016-0261-1

33. Scheen AJ. Cardiovascular Effects of New Oral Glucose-Lowering Agents: DPP-4 and SGLT-2 Inhibitors. Circ Res 2018; 122: 1439-59; https://doi.org/10.1161/CIRCRESAHA.117.311588

34. Di Franco A, Cantini G, Tani A, et al. Sodium-dependent glucose transporters (SGLT) in human ischemic heart: A new potential pharmacological target. Int J Cardiol 2017; 243:86-90; https://doi.org/10.1016/j.ijcard.2017.05.032

35. Zinman B, Wanner C, Lachin JM, et al.; EMPA-REG OUTCOME Investigators. Empagliflozin, Cardiovascular Outcomes, and Mortality in Type 2 Diabetes. N Engl J Med 2015; 373: 2117-28; https://doi.org/10.1056/NEJMoa1504720

36. Wiviott SD, Raz I, Bonaca MP, et al.; DECLARE-TIMI 58 Investigators. Dapagliflozin and Cardiovascular Outcomes in Type 2 Diabetes. N Engl J Med 2019; 380: 347-57; https://doi.org/10.1056/NEJMoa1812389

37. Neal B, Perkovic V, Mahaffey KW, et al.; CANVAS Program Collaborative Group. Canagliflozin and Cardiovascular and Renal Events in Type 2 Diabetes. N Engl J Med 2017; 377: 644-57; https://doi.org/10.1056/NEJMoa1611925

38. Perkovic V, Jardine MJ, Neal B, et al.; CREDENCE Trial Investigators. Canagliflozin and Renal Outcomes in Type 2 Diabetes and Nephropathy. N Engl J Med 2019; 380: 2295-306; https://doi.org/10.1056/NEJMoa1811744

39. Cannon CP, Pratley R, Dagogo-Jack S, et al.; VERTIS CV Investigators. Cardiovascular Outcomes with Ertugliflozin in Type 2 Diabetes. N Engl J Med 2020; 383: 1425-35; https://doi.org/10.1056/NEJMoa2004967

40. Kosiborod M, Cavender MA, Fu AZ, et al.; CVD-REAL Investigators and Study Group*. Lower Risk of Heart Failure and Death in Patients Initiated on Sodium-Glucose Cotransporter-2 Inhibitors Versus Other Glucose-Lowering Drugs: The CVD-REAL Study (Comparative Effectiveness of Cardiovascular Outcomes in New Users of Sodium-Glucose Cotransporter-2 Inhibitors). Circulation 2017; 136: 249-259; https://doi.org/10.1161/CIRCULATIONAHA.117.029190

41. Kosiborod M, Lam CSP, Kohsaka S, et al.; CVD-REAL Investigators and Study Group. Cardiovascular Events Associated With SGLT-2 Inhibitors Versus Other Glucose-Lowering Drugs: The CVD-REAL 2 Study. J Am Coll Cardiol 2018; 71: 2628-39; https://doi.org/10.1016/j.jacc.2018.03.009

42. Ryan PB, Buse JB, Schuemie MJ, et al. Comparative effectiveness of canagliflozin, SGLT2 inhibitors and non-SGLT2 inhibitors on the risk of hospitalization for heart failure and amputation in patients with type 2 diabetes mellitus: A real-world meta-analysis of 4 observational databases (OBSERVE-4D). Diabetes Obes Metab 2018; 20: 2585-97; https://doi.org/10.1111/dom.13424

43. Patorno E, Pawar A, Franklin JM, et al. Empagliflozin and the Risk of Heart Failure Hospitalization in Routine Clinical Care. Circulation 2019; 139: 2822-30; https://doi.org/10.1161/CIRCULATIONAHA.118.039177

44. The Medicines Utilisation Monitoring Centre. National Report on Medicines use in Italy. Year 2019. Rome: Italian Medicines Agency, 2020

45. Osservatorio ARNO diabete. Il profilo assistenziale della popolazione con diabete. Rapporto 2017

46. Zhang P, Zhang X, Brown J, et al. Global healthcare expenditure on diabetes for 2010 and 2030. Diabetes Res Clin Pract 2010; 87: 293-301; https://doi.org/10.1016/j.diabres.2010.01.026

47. Engelgau MM, Geiss LS, et al. The evolving diabetes burden in the United States. Ann Intern Med 2004; 140: 945-50; https://doi.org/10.7326/0003-4819-140-11-200406010-00035

48. Kanavos P, van den Aardweg S, Schurer W. Diabetes expenditure, burden of disease and management in 5 EU countries. LSE Health, London School of Economics, 2012

49. Scalone L, Cesana G, Furneri G, et al. Burden of diabetes mellitus estimated with a longitudinal population-based study using administrative databases. PLoS One 2014; 9: e113741; https://doi.org/10.1371/journal.pone.0113741

50. Marchesini G, Forlani G, Rossi E, et al.; ARNO Working Group. The direct economic cost of pharmacologically-treated diabetes in Italy-2006. The ARNO observatory. Nutr Metab Cardiovasc Dis 2011; 21: 339-46; https://doi.org/10.1016/j.numecd.2009.10.009

51. Bruno G, Picariello R, Petrelli A, et al. Direct costs in diabetic and non-diabetic people: the population-based Turin study, Italy. Nutr Metab Cardiovasc Dis 2012; 22: 684-90; https://doi.org/10.1016/j.numecd.2011.04.007

52. Osservatorio ARNO diabete. Il profilo assistenziale della popolazione con diabete. Rapporto 2015

53. Marcellusi A, Viti R, Mecozzi A, Mennini FS. The direct and indirect cost of diabetes in Italy: a prevalence probabilistic approach. Eur J Health Econ 2016; 17: 139-47; https://doi.org/10.1007/s10198-014-0660-y

54. De Berardis G, D’Ettorre A, Graziano G, et al.; DADA (Diabetes Administrative Data Analysis) Study Group. The burden of hospitalization related to diabetes mellitus: a population-based study. Nutr Metab Cardiovasc Dis 2012; 22: 605-12; https://doi.org/10.1016/j.numecd.2010.10.016

55. Yoshida Y, Cheng X, Shao Het al. A Systematic Review of Cost-Effectiveness of Sodium-Glucose Cotransporter Inhibitors for Type 2 Diabetes. Curr Diab Rep 2020; 20: 12; https://doi.org/10.1007/s11892-020-1292-5

56. Rahman W, Solinsky PJ, Munir KM, et al. Pharmacoeconomic evaluation of sodium-glucose transporter-2 (SGLT2) inhibitors for the treatment of type 2 diabetes. Expert Opin Pharmacother 2019; 20: 151-61; https://doi.org/10.1080/14656566.2018.1543408

57. Hong D, Si L, Jiang M, et al. Cost Effectiveness of Sodium-Glucose Cotransporter-2 (SGLT2) Inhibitors, Glucagon-Like Peptide-1 (GLP-1) Receptor Agonists, and Dipeptidyl Peptidase-4 (DPP-4) Inhibitors: A Systematic Review. Pharmacoeconomics 2019; 37: 777-818; https://doi.org/10.1007/s40273-019-00774-9

58. Clarke PM, Gray AM, Briggs A, et al. A model to estimate the lifetime health outcomes of patients with type 2 diabetes: the United Kingdom prospective diabetes study (UKPDS) outcomes model (UKPDS no. 68). Diabetologia 2004; 47: 1747-59; https://doi.org/10.1007/s00125-004-1527-z

59. Hayes AJ, Leal J, Gray AM, et al. UKPDS outcomes model 2: a new version of a model to simulate lifetime health outcomes of patients with type 2 diabetes mellitus using data from the 30 year United Kingdom prospective diabetes study: UKPDS 82. Diabetologia 2013; 56: 1925-33; https://doi.org/10.1007/s00125-013-2940-y

60. McEwan P, Bennett H, Khunti K, et al. Assessing the cost-effectiveness of sodium-glucose cotransporter-2 inhibitors in type 2 diabetes mellitus: A comprehensive economic evaluation using clinical trial and real-world evidence. Diabetes Obes Metab 2020; 22: 2364-74; https://doi.org/10.1111/dom.14162

61. Iannazzo S, Mannucci E, Mantovani L. Budget impact analysis of empagliflozin for the treatment of type 2 diabetes in Italy. Farmeconomia. Health economics and therapeutic pathways 2016; 17: 19-27; https://doi.org/10.7175/fe.v17i1.1238

Refbacks

  • There are currently no refbacks.