Data

All data were retrieved in the first week of January 2022.

Demographics, health infrastructure, and economic indicators

The indicators in Supplementary Table 1 (Appendix) support the claim that France, Germany, and Italy are similar in terms of demography, health infrastructure, and their economies.

Health data

For all OECD and EU countries, the share of the population who received one dose of a COVID-19 vaccine, two doses of a COVID-19 vaccine, hospital admissions per 1 million, daily ICU patients per 1 million, daily deaths per 1 million, and population estimates have been retrieved from Our World In Data46. For France age-stratified data on hospital admissions, ICU patients, and deaths was retrieved from official government sources (https://www.data.gouv.fr/fr/datasets/synthese-des-indicateurs-de-suivi-de-lepidemie-covid-19/) and deaths outside hospitals from the French Institute for Demographic Studies (INED) (https://dc-covid.site.ined.fr/en/data/france/). For Italy age-stratified data on deaths was also retrieved from INED (https://dc-covid.site.ined.fr/en/data/italy/).

Age-stratified vaccine uptake statistics for France and Italy were both retrieved from the European Centre of Disease Prevention and Control (note that such data is not available for Germany) (https://vaccinetracker.ecdc.europa.eu/public/extensions/COVID-19/vaccine-tracker.html#age-group-tab).

The share of different vaccines used until the end of 2021 in France, Germany, and Italy (made by BioNTech/Pfizer, Moderna, AstraZeneca, Janssen Pharmaceutica NV) have been retrieved from the official government sources (France: https://covidtracker.fr/vaccintracker/, Germany: https://impfdashboard.de/, Italy: https://www.governo.it/it/cscovid19/report-vaccini/).

OECD Weekly Tracker

The OECD Weekly Tracker (short ‘Weekly Tracker’) provides weekly estimates of economic activity based on Google Trends data and performs well across the 46 OECD and G20 countries in forecast simulations. The Tracker’s methodology40 relies on a machine learning algorithm, which extracts signals from search intensities related to approximately 250 categories of search keywords to infer a timely picture of the economy. It is trained on official GDP series to predict weekly GDP from the weekly Google Trends series. It provides estimates of weekly GDP relative to the pre-crisis trend.

The Tracker is based on several Google Trends variables that were hand-picked to cover a wide range of aspects of economic activity. Importantly, for our analysis, the Tracker only uses search behaviour on economic variables and not health variables. Data about search behaviours can be informative about consumption (e.g., related to searches for “vehicles”, “household appliances”), labour markets (e.g., “unemployment benefits”), housing (e.g., “real estate agency”, “mortgage”), business services (e.g., “venture capital”, “bankruptcy”), industrial activity (e.g., “maritime transport”, “agricultural equipment”) and economic sentiment (e.g., “recession”), and poverty (e.g., “food bank”). Signals about multiple facets of the economy can be aggregated to infer a timely picture of the macroeconomy.

The relationship between the search volume indices and GDP, f, is learnt at the quarterly frequency using official quarterly GDP series and quarterly aggregates of the search indices. It is then used to disaggregate GDP growth at the weekly frequency by applying f to the weekly search indices. The relationship between Google Trends variables and GDP growth is fitted using a neural network. It is trained using a dataset comprising the whole panel of observations from 46 countries.

The Tracker measures the percentage difference in GDP relative to a pandemic-free counterfactual, where the counterfactual is taken to be the OECD Economic Outlook projection published in November 201947. Formally, the Tracker is defined as

$$T_w=\fracy_wx_w-1,$$

(1)

where \(y_w\) is weekly GDP in week w, and \(x_w\) is weekly GDP in a no-COVID counterfactual, proxied by a twelfth of quarterly GDP projected by the OECD Economic Outlook prior to the crisis. The Tracker thus measures weekly GDP relative to the pre-crisis trend (Fig. 5b).

Fig. 5: The OECD weekly tracker.
figure 5

a OECD Weekly Tracker for France, Germany, and Italy40. b Schematic depiction of the OECD Weekly Tracker that is providing a proxy of weekly GDP relative to the pre-crisis trend based on Google Trends search data and machine learning.

Since it was released in December 2020, the Tracker has shed light on major policy issues related to the economic impact of lockdowns, infection waves, vaccination campaigns, and economic policy responses. The high-frequency nature of the Tracker makes it a relevant tool to assess the impact of COVID-19 policy responses. The Tracker series for France, Germany, and Italy are shown in Fig. 5a.

The OECD Weekly Tracker’s accuracy was assessed using pseudo-real time forecast simulations40. On average across 46 countries over the period 2008Q1-2020Q4, it has a Root Mean Squared Error (RMSE) that is 17% lower than an autoregressive model that just uses lags of year-on-year GDP growth. The underlying model captures a sizeable share of business cycle variations, including the time around the global financial crisis (when the available data for training the algorithm was much smaller) and the euro area sovereign debt crisis. Its RMSE is on average 8% lower than an autoregressive model in 2008-10 and 41% lower in 2020. The timing of the downturn and subsequent rebound is well captured by the model, although the full magnitude of the negative shock in the second quarter of 2020 is typically under-estimated, given its unprecedented scale. The mean absolute error in predicting year-on-year GDP growth in the first (resp. second) quarter was 2.42 (resp. 3.86) p.p., compared with actual falls in GDP for the median country of 0.12% (resp. 10.4%). Note that the error is larger when the fall is very large, namely of a magnitude unseen before. The tracker thus provides a useful tool for real-time narrative analysis on a weekly basis, although it does not on average outperform models based on more standard variables, once these are eventually released.

Vaccine acceptance

Is taken from the University of Maryland Social Data Science Centre “Global COVID-19 Trends and Impact Survey” in partnership with Facebook (https://jpsm.umd.edu/research/global-covid-19-trends-and-impact-survey%2C-partnership-facebook). The survey is administered to a representative sample of Facebook users daily and includes questions on symptoms, social distancing behaviour, vaccine acceptance, mental health issues, and financial constraints. The acceptance rates used in this paper are built as the quarterly average of the proportion of respondents that said to ‘definitely’ or ‘probably’ choosing to get vaccinated if a COVID-19 vaccine was offered to them over the first quarter of 2021. For the United States, acceptance rates were complemented using data from the Johns Hopkins Centre for Communication Programs. The acceptance rate for Malta was imputed using the median across countries.

Mobility

A mobility index is built from the Google Mobility reports (https://www.google.com/covid19/mobility/), which document mobility per type of destination relative to the pre-crisis levels at a daily frequency. The mobility index used in this paper is the simple average of mobility towards workplaces and places of retail and recreation.

Temperature

Daily temperature series for the 46 OECD and G20 countries across 2020 and 2021 were collected from the National Oceanic and Atmospheric Administration’s National Centres for Environmental Information (https://www.ncei.noaa.gov/). The Global Historical Climatology Network daily (GHCNd) provides daily climate summaries from land surface stations across the globe. Temperature data for each station were averaged at the country level.

Policy interventions

For each country, we consider the date when COVID certificates for day-to-day use were announced, namely, 12 July 2021 for France, 10 August 2021 for Germany, and 22 July 2021 for Italy. See Supplementary Table 2 (Appendix) for details of the corresponding regulations.

OECD and EU countries that announced the use of COVID certificates before 22 September 2021

Austria, Canada, Chile, Colombia, Croatia, Cyprus, Denmark, Estonia, Finland, France, Germany, Greece, Ireland, Israel, Italy, Latvia, Lithuania, Luxembourg, Mexico, Portugal, Romania, Slovakia, Slovenia, Spain, Switzerland, the United States.

Remaining OECD and EU countries

Australia, Belgium, Bulgaria, Costa Rica, Czech Republic, Hungary, Iceland, Japan, Malta, the Netherlands, New Zealand, Norway, Poland, South Korea, Sweden, Turkey, the United Kingdom.

Donor pool countries for synthetic control

Australia, Belgium, Czech Republic, Hungary, Japan, Malta, the Netherlands, New Zealand, Norway, Poland, South Korea, Sweden, Turkey, the United Kingdom. (Costa Rica, and Iceland have been removed from the donor pool used for the synthetic control method due to lack of data for covariates. Bulgaria has been removed from the donor pool due to lack of vaccination coverage data over most of the analysed period.)

See Supplementary Table 3 (Appendix) for information on all OECD and EU countries regarding the implementation of COVID certificates.

Estimation of vaccine uptake

The impact of COVID certificates on vaccination uptake is estimated using innovation diffusion theory and supported by the synthetic control method.

Innovation diffusion theory26,27,28,29,30 attempts to formalise the way in which an innovation is gradually taken up by a population, where early adopters are then joined by followers. Every individual has their own, heterogeneous, threshold at which they decide to adopt the innovation. The model relies on growth models with capacity limits, i.e., logistic curves, positing that thresholds are distributed accordingly. In our context, vaccines are the innovation that every (eligible) person may choose to adopt.

Denote by t0 the date when the vaccine is introduced and by x(t) [0,1] the cumulative fraction of the population who has received at least one dose on day t. Thus, by assumption, x(t) [0,1] for all t, the function x(t) is nondecreasing, and x(t)=0 for all t ≤ t0. The innovation diffusion model depends on three additional parameters: p > 0 is the ‘coefficient of innovation’, i.e., the instantaneous rate at which a non-vaccinated person opts to get vaccinated, independently of how many people are already vaccinated; q > 0 is the ‘coefficient of imitation’, i.e., the rate at which a non-vaccinated person is influenced by the fraction of vaccinated people; and 0 < K ≤ 1 is the capacity, i.e., the fraction of the population that is eventually eligible and willing to get vaccinated.

Mathematically, the innovation diffusion model is described by the ordinary differential equation

$$x^\prime (t)=\left(1-\fracx(t)K\right)(p+qx(t)),\,t\ge t_0\,and\,x^\prime (t)=0\,elsewhere.$$

(2)

The unique solution to the latter differential equation is given by:

$$x(t)=K\frac1-e^-(p+q)(t-t_0)1+\fracqpe^-(p+q)(t-t_0),t\ge t_0\,and\,x\left(t\right)=0\,elsewhere.$$

(3)

Logistic functions model the diffusion of an innovation in the absence of major shocks, including supply shortages or policy interventions. While this is the case over the time period considered (i.e., date of announcement of COVID certificate to 31 December 2021), an extension to 2022 may be less appropriate due to the exogenous shock caused by the less severe Omicron variant becoming dominant in France, Germany, and Italy48. Regarding supply shortages and eligibility constraints, the early stages of the vaccine rollout were significantly impacted by supply constraints, which led most countries to give age-based priority. This effect could be captured by an extension of the original innovation diffusion model49, but requires additional data, which is not available in our case (i.e., the fraction of the population willing to get vaccinated among the not-yet-eligible). Next to the imitation and innovation coefficients, this extended model would then add a ‘word-of-mouth’ coefficient that captures the influence of individuals willing to get vaccinated, not yet eligible, on the eligible population. When the word-of-mouth coefficient is assumed equal to the imitation coefficient, this model boils down to the original Bass model, thus lending further support to our modelling choice.

Innovation diffusion theory assumes constant parameters, which may be seen as a limitation. On the other hand, adding a time-dependent effect would result in a statistical model such as ordinary least squares (OLS), which does not have predictive power.

Parameters \(t_0,p,q,\) and K are estimated using the least-square method to fit the data on vaccine uptake (see Table 1). The fit is computed over the 100 days prior to the announcement of a country’s COVID certificate, when the majority of the adult population was eligible for vaccination, and then extended to the end of the year. For convenience, we define \(t_0\) to be the number of days before or after 100 days prior to the announcement of the COVID certificate. Note that \(t_0\) is not decisive for the estimation, as initial growth of the logistic function is near zero.

Table 1 Estimated parameters of the innovation diffusion model for France, Germany, and Italy.

The fit is robust with respect to slightly longer or shorter fitting windows. On the other hand, when using a much shorter window (e.g., 60 days) estimates become very noisy with respect to the exact window length, likely because the fit does not appropriately pick up the curvature and inflection point of the logistic function. We use the function ‘curve_fit’ from Python’s package ‘scipy.optimize’ over vaccine uptake and synthetic counterfactuals. We use block bootstrap to account for time dependence in the data with 1000 iterations and 30 non-overlapping blocks50. The 95% confidence intervals are shown and reported throughout.

Counterfactual vaccine uptakes

For each country, denote by \(V_t(v)\) for \(v\in \\mathrm0,1,2\\) the proportion of the population having received v doses at time t. We do not consider \(v\ge 3\) doses as individuals who were not vaccinated before the announcement of the COVID certificate were not eligible to receive a booster before early 2022; a counterfactual is thus not needed. (For example, in France, a person getting vaccinated on 12 July –the announcement date– could not get a booster until 2 January due to a 3-week gap between first and second dose, and 5 months gap for the booster).

For the first dose, the counterfactual is denoted by \(\hatV_t(1)\), and is obtained from the estimation described above. Let \(T_0\) denote the date when COVID certificates are announced in each country and \(T_1\) be 31 December 2021. Then, \(\hatV_t(1)\) is equal to \(V_t(1)\) for all \(t\le T_0\) and is equal to the estimate obtained through the innovation diffusion model for all \(t\in [T_0,T_1]\). To obtain a counterfactual for fully vaccinated individuals, \(V_t\left(2\right),\) we assume the same ratio between first and second doses between the counterfactual and realised scenarios three weeks prior, i.e.:

$$\hatV_t\left(2\right)=\hatV_t-21(1)\cdot \fracV_t(2)V_t-21(1).$$

(4)

Assuming the same ratio between second and first dose uptake (with a three-week lag, corresponding to the minimum required gap between first and second dose) for the counterfactual and realised scenarios is well-motivated as this ratio was not affected by the intervention (see Supplementary Fig. 1, Appendix). Finally, \(\hatV_t(0)=1-\hatV_t(1)-\hatV_t\left(2\right)\).

For robustness, we estimated the impact of COVID certificates in Germany starting at the French announcement instead, and observed a slightly higher effect (i.e., 1.1 p.p. more by the end of the year). Assessing the impact of COVID certificates from the date it was announced in the country results in a more conservative estimate, as the potential impact of other countries’ announcements is neglected.

Age-stratified vaccine uptake

To estimate age-stratified vaccine uptake, we consider the population aged 60 and over (60+) and the rest, separately. This is particularly relevant as health outcomes are generally more severe for older people. As age-stratified data are not available for Germany before mid-September 2021 it cannot be included in this analysis. We use the innovation diffusion model to construct a counterfactual for the 60+ group for France and Italy. For all vaccination statuses v, the realised vaccination uptake for 60+ is denoted by \(V_t^60+(v)\), and the counterfactual is denoted by \(\hatV_t^60+(v)\). To ensure consistency with our overall estimates, we set counterfactual vaccine uptake for the 59 years old and below as the difference between the overall and the 60+ estimates.

For the population aged 60 years and older, the fit is computed from the start of 2021 to the date of the announcement of the COVID certificate and then extended to the end of the year; the start date is chosen because by then the majority of the 60+ population was eligible for vaccination (Fig. 6). We use the function “curve_fit” from Python’s package “scipy.optimize” over the vaccine uptake counterfactuals. We use block bootstrap to account for time dependence in the data with 1000 iterations and 30 non-overlapping blocks50. The 95% confidence intervals are shown and reported throughout. The confidence intervals are narrow compared to those obtained in Fig. 1. This is the case as the fitting window is longer than for the whole population (due to eligibility), and, more importantly, it includes the three phases of the logistic curve (initial growth, inflection point, and deceleration). This leads to a precise fit, with narrow confidence intervals, and which is robust to shortening the fitting window by up to April.

Fig. 6: Realised and counterfactual vaccination rates for the population over age 60.
figure 6

For France (a) and Italy (b), cumulative proportion of the population over age 60 who received at least one COVID-19 vaccine dose in the actual intervention deployment (blue) and in the no-intervention counterfactual scenario (red). The red shaded area is the 95% confidence interval centred around the main estimate. The counterfactual scenario is estimated via innovation diffusion theory. The black dashed vertical line is the date of the announcement of the COVID certificate.

Model support via synthetic control

Figure 7a shows the estimated vaccination uptake via synthetic control for France, Germany, and Italy. Its computation is described below. The synthetic control for each country falls within the 95% confidence interval of the counterfactual based on innovation diffusion theory. This gives additional support for the model choice and findings.

Fig. 7: Alternative counterfactual via synthetic control.
figure 7

a For France, Germany, and Italy, cumulative share of the population who received at least one COVID-19 vaccine dose in the actual intervention deployment (blue), in the no-intervention counterfactual scenario via innovation diffusion theory (red) with 95% confidence interval centred around the main estimate (shaded red), and in the no-intervention synthetic control scenario (green, dashed). The red shaded area is the 95% confidence interval of the counterfactual scenario via innovation diffusion theory, which is centred around the main estimate. Note that the synthetic control scenario ends on 22 September 2021, when the method becomes infeasible. The black dashed vertical line is the date of the announcement of the COVID certificate. b Country weights for synthetic control for France, Germany, and Italy. c Placebos for France, Germany, and Italy in blue and in grey for donor pool countries. d Root mean squared prediction errors (RMSPE) ratios for France, Germany, and Italy (red) and the donor pool countries (blue).

Synthetic Control32,51,52 provides a counterfactual based on the evolution of nontreated countries, i.e., countries that did not implement COVID certificates. This counterfactual is computed as a weighted average of the nontreated units. To this end, we define the control group as the OECD and EU countries that did not resort to COVID certificates during this period. This choice is motivated by broad socioeconomic resemblance and sufficient vaccine supply. The weights applied to the nontreated units are chosen to minimise the error of the synthetic control in the pre-treatment period. The impact of COVID certificates on vaccination is thus estimated as the difference between vaccination after the implementation of the policy and the counterfactual –14 countries feature in the donor pool of nontreated countries and include OECD and EU countries that did not implement COVID mandates before 22 September 2021 and have sufficient data availability (see list in Methods A). We posit that after this date the synthetic control method is no longer feasible due to an insufficient donor pool.

For each treated country (France, Germany, Italy), the synthetic first-dose vaccination rate (\(SV_i,t\)) is computed as a weighted average of the vaccination rates in the donor countries:

$$SV_i,t=\mathop\sum \limits_j=1^J\omega _jV_j,t,$$

(5)

where J = 14 is the number of countries in the donor pool, and \(\nu _j\) is the weight associated with \(V_j,t\) and the vaccination rate in country j. The weights are in the interval [0,1] and sum to one to avoid extrapolation32. They are chosen to minimise the error prior to the treatment, which occurs in \(T_0\):

$$min_\omega \mathop\sum \limits_t=1^T_0[(V_i,t-\mathop\sum \limits_j=1^J\omega _jV_j,t)^2+\mathop\sum \limits_k=1^K\lambda _k(X_i,t^k-\mathop\sum \limits_j=1^J\omega _jX_j,t^k)^2].$$

(6)

The weights \(\omega _j\) are chosen to minimise a composite loss function that includes, on the left, the mean squared prediction error of the pre-treatment outcome, and, on the right, the mean squared prediction errors of K covariates, whose respective importance is weighed by the coefficients \(\lambda _k\)for k between 1 and K. The covariates are selected on the basis of their predictive power of vaccination, and include annual GDP per capita, the average fatalities and cases over the pre-treatment period, the share of the population aged over 65, the average Mobility Index over 2020, and average vaccine acceptance over the first quarter of 2021. The covariate weights are assumed to be constant (i.e., \(\lambda _k=\lambda\) for all k), and the weight applied to all the covariates λ is optimised using five-fold cross-validation.

The country weights used for building the synthetic vaccination rates for France, Germany, and Italy are shown in Fig. 7b. The three synthetic vaccination rates are built as averages of the vaccination rates from European countries, except for Japan and South Korea, which each account for less than 10% of the French synthetic vaccination rate. All three heavily feature the Czech Republic and Belgium, thus meriting additional description of the policies in these countries. In Belgium, the federal government announced the adoption of COVID certificates for mass events on 19 July 2021, but the use of COVID certificates for access to day-to-day activities, such as going to cinemas, cafes, or restaurants, was not announced until mid-September, and at a regional level (Supplementary Table 3, Appendix). The situation was similar in the Czech Republic where COVID certificates were required to attend mass events from June 2021, and the announcement of their extension to day-to-day activities occurred on 21 October 2021 (Supplementary Table 3, Appendix). Further, the two control countries started requiring a valid COVID certificate for international travel by early July. The COVID certificate policies in Belgium and the Czech Republic were thus comparable to the policies in France, Germany, and Italy, up to the date of their respective announcements.

The significance of the results is assessed using placebo tests52. A synthetic vaccination rate \(SV_j,t\) is built for each country j in the donor pool (placebos); see Fig. 7c.

Inference is performed by comparing the ratio of the root mean squared prediction errors (RMSPE) of the synthetic vaccination rate for the treated country after and before the treatment to the distribution of these RMSPE ratios over the placebos. The ratio between the post-intervention RMSPE and pre-intervention RMSPE for unit j is

$$r_j=\fracRMSPE_j(post-intervention)RMSPE_j(pre-intervention).$$

(7)

A p-value for the inferential procedure based on the permutation distribution of \(r_j\) is given by the rank of the treated country’s ratio divided by the size of the donor pool. The RMSPE ratios and their permutation distributions for the synthetic vaccination rates in France, Germany, and Italy are shown in Fig. 7d. The causal estimates for France and Italy are statistically significant with a p-value equal to \(0.071(=\!\!1/14)\), while the estimates for Germany are not significant.

Overfitting may be an issue with synthetic control. The RMSPE ratios that are used for inference can be recast as follows in the absence of policy intervention:

$$\fracRMSPE_j(out-of-sample){RMSPE_j(in-sample)}.$$

(8)

For placebos, the ratio of the post-treatment RMSPE and pre-treatment RMSPE is simply a ratio of the RMSPE taken out-of-sample over the RMSPE measured in-sample, given that the optimal weights are computed using the pre-treatment observations. This ratio is classically understood as a measure of overfitting. If the model overfits, its out-of-sample prediction error is large compared to its in-sample prediction error. As a result, overfitting synthetic control models yields spurious results whose validity is rejected by permutation tests. In the present case, overfitting is limited by using several covariates in the fit, as well as by resorting to cross-validation to select the weights attributed to these covariates.

Impact on health outcomes

Vaccine effectiveness

We estimate vaccine effectiveness against hospitalisation, ICU admission, and deaths between one week and six months after inoculation when infected with the Delta variant –the dominant strain of SARS-CoV-2 in France, Germany, and Italy over the considered time period– by taking the weighted average over the different types of vaccines (see Supplementary Table 4, Appendix). We rely on the few available studies from different contexts, thus precluding a statistical meta-study. We therefore gather conservative, lower-bound estimates below.

For mRNA vaccines (BioNTech/Pfizer and Moderna), conservative estimates for the effectiveness are 80%33 after one dose and 93% after two doses33,34,35,36. The second estimate integrates the effect of waning immunity, as the protection against severe outcomes is higher than 95% up to 14 weeks after inoculation and above 90% thereafter36. For AstraZeneca’s vaccine, comparable estimates are 90% after one dose and 85% after two doses36. Here, waning immunity explains the lower effectiveness of two AstraZeneca doses versus one. Finally, for Janssen Pharmaceutica NV, the effectiveness after the single dose is estimated at 85%37. Overall, vaccine effectiveness appears to be similar across age groups, and we have thus opted for a common estimate34. The overall vaccine effectiveness against hospital admissions, ICU admissions, and deaths is approximately the same and is also similar across France, Germany, and Italy, namely 81% protection after one dose and 92% after two doses. We do not include the additional protection provided by boosters, as the calculations we perform are only concerned with individuals who were not fully vaccinated before the COVID certificate; therefore, they were not eligible for a booster shot over the period of study.

Realised health outcomes by vaccine status

Let \(X_t\) denote the realised health outcome (i.e., hospital admissions and patients, ICU admissions and patients, and deaths, for a given country) at time t, and let \(X_t(v)\) denote the same outcome by vaccine status, where \(v\in \{\mathrm0,1,2\}\) denotes the number of vaccine shots received. When the data by vaccine status are not available, we can derive them from Bayes’ rule and the level of protection against the health outcome by vaccine status, \(\beta (v)\), as shown above. The \(X_t(v)\)’s satisfy the following linear system:

$$\fracX_t(v)V_t-d(v)=\beta \left(v\right)\cdot \fracX_t\left(0\right)V_t-d\left(0\right),\;for\,v\in \left\0,1,2\right\,\,and\,X_t\left(0\right)+X_t\left(1\right)+X_t\left(2\right)=X_t.$$

(9)

This system admits a unique solution, given by

$$X_t(v)=X_t\cdot \frac\beta (v)\fracV_t-d(v)V_t-d(0)\sum _v^\prime =0^2\beta (v^\prime )\fracV_t-d(v^\prime )V_t-d(0),\;for\,all\,v\in \0,1,2\.$$

(10)

Note that vaccine uptake has been lagged by d days to account for the lag between infection and the health outcome, \(l_hosp\), \(l_ICU\), and \(l_death\), the lag between vaccination and full effectiveness, \(l_vaccine\), and the duration of the health hazard, \(l_stay\; hosp\) and \(l_stay\; ICU\), which are only relevant for hospital admissions and ICU patients. For example, for a patient who is in an ICU at time t, on average, their admission occurred at time \(t-l_stay\; ICU\), their infection at time \(t-l_stay\; ICU-l_ICU\), and at that time \(N_t-l_stay\; ICU-l_ICU-l_vaccine(2)\), people were fully protected by vaccination.

Counterfactual health outcomes by vaccine status

Similarly, let \(\hatX_t(v)\) denote the counterfactual health outcome (number of hospital admissions, ICU patients, or deaths, for a given country) at time t, with vaccine status \(v\in \{{{{{\mathrm0,1,2}}}}\}\). Then,

$$\hatX_t\left(v\right)=X_t\left(v\right)\cdot \frac\hatV_t-d\left(v\right)V_t-d\left(v\right)\;for\,all\,v,$$

(11)

where d is the lag that was introduced in the previous paragraph. The estimated counterfactual number of a given health outcome at time t for a given country is given by:

$$\hatX_t=\hatX_t\left(0\right)+\hatX_t\left(1\right)+\hatX_t\left(2\right).$$

(12)

Finally, the overall realised and counterfactual of a given health outcome, from the announcement of COVID certificates in the country until the end of 2021, are estimated respectively by

$$X_total=\mathop\sum \limits_t=T_0^T_1X_t\;{{\rmand}}\;\overlineX_total=\mathop\sum \limits_t=T_0^T_1\hatX_t.$$

(13)

The difference \(\hatX_total-X_total\) = \(\sum _t=T_0^T_1(\hatX_t-X_t)\) is attributed to the adoption of COVID certificates.

Age-stratified health outcomes

When the data are available, we analogously compute age-stratified (i.e., 60 years old and above, and the rest of the population) health outcomes, as well as the corresponding counterfactuals. The total numbers are obtained by summing over all age groups.

The lag parameters53

We assume the lag between vaccination and full effectiveness is \(l_vaccine=7\) days, the lag between infection and hospital admission is \(l_hosp=7\) days, the lag between infection and ICU admission is \(l_ICU=10\) days, the total number of days in ICU is 8, so that an ICU patient has been admitted \(l_stay\; ICU=4\) earlier, and the average lag between infection and death is \(l_death=14\) days. Thus, for hospital admissions, the total lag is \(l_vaccine+l_hosp=14\); for ICU patients, the total lag is \(l_vaccine+l_stay\; ICU+l_ICU=21\); and for deaths, the total lag is \(l_vaccine+l_death=21\).

Confidence intervals

For the estimation, as the vaccine effectiveness is assumed fixed, the 95% confidence intervals come only from the uncertainty of our estimation on vaccine uptake.

Impact on the economy

The analysis of the economic impact of COVID certificates is based on an indirect causal model: COVID certificates boost vaccination, and vaccination encourages individuals to resume economic activities, thus increasing GDP growth. The quantification exercise thus follows a two-step approach. First, we estimate the average impact of a marginal increase in vaccination rates on economic activity using two-way fixed-effect regressions based on a large panel of data from 46 countries. Second, the estimate of the average effect of vaccination on economic activity is combined with the estimate of the uplift in vaccination obtained in section B to gauge the effect of COVID certificates on economic activity.

Average effect of vaccine uptake on economic activity

Our estimation is based on data from 46 countries. The identification exploits high-frequency within-country variations in vaccination rates, and assumes a static relationship whereby vaccination at time t-28 impacts economic activity at time t. We use a two-way fixed-effect regression and identify the effect through a difference-in-differences design, which assumes a common trend across countries conditional on several covariates described below (see also the descriptive statistics in Supplementary Table 5, Appendix).

The measure of economic activity used in this paper is the OECD Weekly Tracker, a proxy of weekly GDP relative to the pre-crisis trend, which is available for 46 countries with no publication delay (Methods A). It is regressed on vaccination rates along with controls as well as country and week fixed effects. To estimate the average total effect of vaccination on GDP, we use the following closed-form model:

$$T_i,w=\beta V_i,w-l+\gamma I_i,w-l+\eta X^f_i,w+\iota Z_i,w+\alpha _i+\delta _w+\sigma _i,w.$$

(14)

Weekly GDP is proxied by the Tracker \(T_i,w\)and is regressed on the share of vaccinated people lagged by l weeks (l = 4), \(V_i,w-l\), as well as three vectors of controls, week, and country dummies. The first controls vector \(I_i,w-l\) includes lagged cases, deaths, reproduction rate, and mobility index, which may have impacted both past vaccine uptake decisions and present weekly GDP54. The model also averts confounding effects that could emerge from trade and other spillovers due to the relative synchronicity of vaccination campaigns across countries by controlling for vaccination and deaths in the main trading partners (\(X^f_i,w\)). The vector \(X^f_i,w\) is the weighted average of vaccination rates and deaths in country i’s main 10 trading partners, i.e.,

$$V^f_i,w=\mathop\sum \limits_j=1^10\gamma _i,jV_j,w,$$

(15)

where \(V_j,w\) is the vaccination rate in trading partner j and \(\gamma _i,j\)is the share of exports from country i to trade partner j in total exports from country i. The same formula is used to build the vector of weighted average death rates in trading partners. Last, the model includes the vector of average weekly temperatures \(Z_i,w\), which can influence virus transmission42.

The model is estimated using data from 46 OECD and G20 countries (see Table 2). Denote by *\(p \, < \, 0.1\), **\(p \, < \, 0.05\), and ***\(p \, < \, 0.01\). The average effect of a 1 p.p. increase in the share of vaccinated people after a month is 0.052*** p.p. in weekly GDP. This order of magnitude seems plausible and implies, if the impact was permanent, that 100% vaccination uptake would increase GDP by 5.2 p.p., which broadly corresponds to 85% of the average GDP loss suffered in 2020 by the countries in the sample. This is consistent with the notion that a complete vaccination would not be sufficient to a return to pre-crisis trends due to partial vaccine effectiveness and the waning-out of vaccine-provided immunity. Adding controls for deaths and vaccination in trade partners decreases the main estimate from 0.054*** to 0.052*** by partialling out the confounding effect of trade spillovers. Finally, the third column models the direct effect by controlling for current cases, deaths, and reproduction rates. This indicates that 83% of the total economic effect of vaccination is through the direct effect on individual behaviour, while the remaining 17% is related to the effect through the impact on virus circulation. Note, however, that we do not estimate the indirect effects independently, as we do not estimate a policy response function.

Table 2 Regression results for vaccination-GDP elasticity.

Estimating the impact of vaccination on economic activity needs to take into account the uncertainty relative to the measure of the latter, proxied by the OECD Weekly Tracker. The OECD Weekly Tracker is an algorithm prediction and includes an error term that can impact the accuracy of the estimation of the parameter β in the regression above. Its standard deviation thus needs to be adjusted to account for the uncertainty from both the econometric estimation and the weekly GDP measure. This can be achieved by using the multiple estimates from the OECD Weekly Tracker. The OECD provides 300 bootstrap series for the Tracker. Similar to the context of multiple imputation, we can apply Rubin’s law55 to infer the variance of β from the estimates \(\beta _i\) obtained from each of the 300 replicates and using the following formula:

$$Var\left(\beta \right)=W_M+\left(1+M^-1\right)B_M,$$

(16)

where M is the number of replicate series (M = 300 in our case), and \(W_M\) and \(B_M\) are respectively the within and between components, given by \(W_M=M^-1\mathop\sum \nolimits_i=1^M\widehatVar_i(\beta _i)\) and \(B_M=\left(M-1\right)^-1\mathop\sum \nolimits_i=1^M\left[\beta _i-\beta \right]^2\). The total empirical standard deviation of β is 0.009, which results in 95% confidence intervals of [0.034, 0.070], which are around twice the size of the confidence intervals computed in Table 1.

Results in Table 2 are to be understood as average effects; the regression analysis does not aim at capturing possibly heterogeneous effects across countries or time. The effect of vaccination on economic activity may differ across countries and regions, based on cross-country or cross-region differences in sectoral composition and institutional settings. Moreover, the effect of vaccination may also differ over time. This is the case as the protection granted by the vaccine may have a smaller impact when virus circulation is near zero. The present analysis simply aims at gauging the average impact of vaccination on the economy. Some of the sources of heterogeneity are uncovered in the robustness checks performed below.

Economic impact of COVID certificates

The economic impact of COVID certificates is assessed through the indirect causal model whereby COVID certificates spur vaccination, and which in turn increases economic activity. As a result, the counterfactual for economic activity (i.e., in the absence of COVID certificates) is obtained by plugging the counterfactual for vaccination rates in the regression model. The latter provides an estimate for weekly GDP absent COVID certificates by subtracting the estimation of the impact of COVID certificates on weekly GDP, denoted by \(\delta _i,w\), from the observed weekly GDP:

$$\hatT_i,w=T_i,w-\delta _i,w$$

(17)

$${{\rmwith}}\,\delta _i,w=\beta \left(V_i,w-l-\hatV_i,w-l\right),$$

(18)

where \(\hatT_i,w\) is the counterfactual tracker and \(\hatV_i,w-l\) is the counterfactual vaccine uptake.

Confidence intervals for counterfactual weekly GDP are derived from the fact that the estimate of the causal impact of COVID certificates on weekly GDP is the product of two random variables:

$$Var\left(\delta _i,w\right)= \, Var\left(\beta \right)\cdot Var\left(\hatV_i,w-l\right)+Var\left(\beta \right)\cdot E^2\left(V_{i,w{-}l}-\hatV_i,w-l\right)\\ +Var\left(\hatV_i,w-l\right)\cdot E^2\left(\beta \right).$$

(19)

\(Var\left(\hatV_i,w-l\right)\) is estimated from 1,000 bootstrap runs of the logistic model, and \(Var\left(\beta \right)\) is given by Rubin’s law. Then, supposing that \(\delta _i,w\) follows a normal distribution, its 95% confidence intervals are [\(\delta _i,w\pm 1.96* \sigma _\delta _i,w\)].

Robustness checks

We complemented our analysis with robustness checks regarding the statistical method, the choice of lag, the modelling assumption that the vaccine-GDP relationship did not vary substantially across the considered time period, and the choice of the measure of economic activity.

Two-way fixed effects regressions

A recent literature56,57,58 has shed light on the limitations of two-way fixed effects regressions when the treatment effect is heterogeneous. Alternative estimators have been proposed56, which limit the risk of bias by restricting the comparisons between units and times. This literature is still young, and there are currently no satisfactory options for cases where the treatment is dynamic, with a fuzzy design and in the absence of stayers or quasi-stayers. More specifically, the fuzzy design (continuous treatment, with treatment intensities that vary both in time and across units) precludes the use of the estimators introduced by Callaway and Sant’Anna59 or Sun and Abraham60. In the absence of quasi-stayers, it is impossible to estimate the time fixed effects based on the imputation estimator introduced by Borusyak et al. 61.

We assume that the effect is static, which means that the past treatments do not impact the outcome variable. The reason for this is mostly that the treatment is the cumulative vaccination rate. This implies a nested causal relationship \(V_j,t-n\to V_j,t-n+1\to \ldots \to V_j,t\) so that it is unnecessary to model dynamic effects. Under the alternative assumption that the effect is dynamic, the DIDl estimator62 could be applied but our experiments were inconclusive. This seems to result from the fact that identification is based on the timing of the intervention rather than the intensity of the treatment; yet, in our paper, we focus on the intensity. Further, their DIDl estimator is a weighted average of DID between first-time switchers in \(t-l\) and not-yet-switchers in l. As a result, the estimation for each possible value of l is based on a small number of observations (e.g., 44 for l = 1), and the results are not credible.

To nevertheless validate our findings with a different statistical method, we reproduced the regressions in Table 2 without week dummies. Note that this model is underspecified, so the results can only be seen as indicative. For the preferred specification, estimates of the vaccine effect on GDP are of the same order of magnitude, albeit substantially smaller (0.034***). The model, which includes week dummies, remains more plausible, as it seems critical to control for the very large shocks caused by successive COVID infection waves across the globe.

Choice of lag

The main model regresses weekly activity over the vaccination rate lagged by l weeks, which is equal to 4 in the favourite specification. The coefficients of interest were computed for all values of l between 1 and 10. Figure 8a shows that the main estimates are robust to the choice of lag parameter.

Fig. 8: Robustness checks for varying lags and time-varying impact of vaccination.
figure 8

a Main regression coefficients for various lag values. b Time-varying estimate of the impact of vaccination on economic activity. The blue shaded area is the 95% confidence interval obtained from 100 bootstrap samples and centred around the main estimate (blue). The black dashed line indicates the average effect across time.

Time-varying relationship between vaccination and GDP

The regression analysis used to analyse the economic impact of vaccination yields an average effect which is assumed to be a valid approximation of the economic impact of the increment in vaccination caused by the COVID certificates. To test the robustness of this hypothesis, we use causal machine learning to produce a time-varying estimate of the economic effect of the vaccine. We considered a model with time-varying effects using Double Machine Learning63,64. This approach uses machine learning to capture non-linearities and complex interactions while correcting for the bias caused by penalised loss functions. More specifically, we used the R-learner64, which allows us to include a large number of interaction terms while averting overfitting by applying a penalty term such as the L2-norm. We apply the R-learner to the regression of weekly GDP on vaccine uptake by including as interaction terms the complete set of overlapping period dummies, \(\ P_t^w=I_(t\le w) \_w\in [1,M]\), i.e., the dummy \(P^w\) is equal to 1 at times t prior to week w, and zero otherwise. Using overlapping period dummies rather than week dummies in the context of a penalised regression allows for smoother time-varying estimates, and both the value of the coefficient associated with the period dummy and the difference between two consecutive coefficients are penalised65. This approach yields a time-varying coefficient β (Fig. 8b). Given that the interaction terms are common to all countries, this model estimates a time-varying effect without allowing for cross-country heterogeneity. The estimated impact lies between 0.035 and 0.060, with a 95% confidence interval of approximately 0.02 percentage points on average. We note that the estimates do not vary significantly across time, and our average estimator of 0.052 is consistent with this method.

Measure of economic activity

The main analysis of the impact of vaccination on economic activity is based on the OECD Weekly Tracker, which proxies weekly GDP relative to the pre-crisis trend. Additional regression analyses were performed to ensure that the results are robust to the choice of an economic activity metric. Supplementary Table 6 (Appendix) shows estimation results of regressions on both the Mobility Index and official quarterly GDP. The former is a high-frequency proxy for economic activity, although its relationship with GDP is not straightforward, and the latter is the official low-frequency measure of economic production. The regression on the Google Maps mobility index yields results that are comparable to the main regression (Table 2), although with a smaller direct effect of vaccination. From our preferred specification, an increase of 10 p.p. in vaccination increases mobility by 6.9 p.p. This effect is one third larger than the effect on the Weekly Tracker, which seems consistent with the fact that mobility was only partially correlated with economic activity as by the start of 2021 households and firms had both adapted to the pandemic constraints (through working from home, e-commerce, etc.). As a result, around two thirds of the increase in mobility triggered by vaccination translates into an increase in economic activity. The regression on quarterly GDP year-on-two-year growth rates cannot include regressors lagged by 4 weeks, which implies that the baseline model cannot be estimated. Note that, the use of year-on-two-year growth rates avoids the strong base effects in year-on-year (resp. quarter-on-quarter) series around one year (resp. one quarter) after the first lockdowns. The direct effect model yields an estimate of the impact of vaccination (0.028) on GDP that is close to the one that was obtained from the regression on the Weekly Tracker (0.043), although it is much less precise given that the identification cannot rely on the high-frequency variations in both vaccination and economic activity. The results obtained from regressions on these two alternative measures confirm the robustness of the main effect captured with greater precision from the regression on the OECD Weekly Tracker.

Reporting summary

Further information on research design is available in the Nature Research Reporting Summary linked to this article.

Topics #Accounting #accounting cycle #Biz equation #Jobs #Manager