Results Top

Descriptive statistics and matching procedure

Descriptive statistics of performance indicators by f irm characteristics in pre-funds period (2003-2006) are presented in Table S4 [suppl.]. In terms of regional distribution, most of firms are situated in eastern part of Croatia, which is predominantly more agrarian compared to other regions. However, firms in this region showed the lowest values of total assets, turnover and value added. Central Croatia is the region with highest employment figures in three analyzed industries although average wages seem to be highest in western Croatia. In terms of firm size, majority of them were micro- and small-sized. Most firms are focused only on the domestic market, although there has been a slight increase in firms that are both exporters and importers. Firms that participate in international trade have, on average, more assets and sales, employ more people, and pay those employees more money.

Descriptive statistics before and after matching are presented in Table S5 [suppl.]. The average firm in our sample was 9 years old (on the market) (9.0 control firms and 8.5 treated firms), located in Eastern Croatia (46% of control f irms and 32% of treated firms), micro-sized (77% of control firms and 49% of treated firms), from the agricultural sector (84% of control firms and 73% of treated firms), and mainly focused on the domestic market (71% of control firms and 50% of treated firms). Treated firms were on ave-rage outperforming their potential controls in performance variables, most notably in the value of capital.

We calculated propensity score using a probit model with a dependent dummy variable indicating whether the firm i received the grant in time t, and with all indepen-dent covariates (Table 1) from the pre-treatment period t-1. The estimated model (Table S6 [suppl.]) was found to be statistically significant and the pseudo R2 shows that the model was able to explain 30.9% variance in the de-pendent variable.

Control group to all treated observations is then found using this propensity score. We combined exact matching and nearest neighbor matching. Since our analysis spans over the 2007–2016 period, during which the economic climate in Croatia changed dramatically due to the reces-sion in the period 2008–2015, we wanted to make sure to pair beneficiaries to those non-beneficiaries in very si-milar economic conditions. For this reason, treated and control observations were exactly matched on year of receiving treatment, region of the firm, and NACE 2-di-git sector; and then within each of these combinations of groups we used propensity score to find the nearest neighbor for each treated observation. After matching we found no significant differences in means of all covariates and a significant reduction in standardized bias (Table S5 [suppl.]), implying that both samples were now compara-ble, and that balancing property was satisfied.

 

Table 1. Covariates used for calculation of propensity score

[a] Regions were defined based on the 21 Croatian counties. Details are available on request. [b] All monetary variables were deflated using AMECO implicit price deflators with base in 2010. [c] The purpose of this transformation is to make variables in our analysis as “normal” as possible (making them follow the normal distribution, “bell” curve) so that the statistical analysis results become more valid. In other words, the log transformation reduces or removes the skewness in original data. Since natural logarithm function is a monotonic transformation, it also preserves ordinal property of original dataset. Transformation 1 + [var] is used simply to reduce the loss of available data, given that ln (0) is undefined. So, instead of transforming variables using simply ln [var] and losing all observations with zeros as variables values, we use transformation ln (1 + [var]) which then yields ln + (1 + 0) = 0. This transformation is also quite standard in similar research (e.g. Srhoj et al., 2019, 2021). [d] Total factor productivity (TFP) was estimated using Wooldridge (2009) methodology based on the production function approach using value added as output, labor and capital as inputs, and intermediate inputs to control for unobservables. As technologies used in the production process differ across different industries, TFP was estimated separately for each NACE Rev. 2 two-digit industry.

 

Table 2. Outcome variables used in the analysis

 

 

Average treatment effect on the treated

The ATET estimations are presented in Table 3, en-compassing both firms’ survival and performance out-comes. The results indicate a positive grant premium on the firm survival rate in one year after receiving the treatment. Grants also induced a positive output addi-tionality in assets, turnover, and value added in both the short and the midterm after receiving the grant. Firms’ profit was significantly higher for grant reci-pients only in the short term (in the first year after the grant was obtained). The treatment also recorded a po-sitive significant effect on employment throughout the analyzed period, even though effects on average wage were not significant. Regarding capital inputs, grants induced positive capital input additionality throughout the analyzed period, while growth rates in bank loans for treated firms were higher starting with the fourth year after undergoing treatment. Intermediate inputs showed growth effects in both the short and the mi-dterm. Regarding productivity, grants yielded additio-nality in TFP and labor productivity in both the short and the midterm. Debt analysis revealed that treated firms managed to reduce their debt ratio in the first year following grant receipt, but this effect vanished in the midterm. Treatment effect in Z-score was not shown to be significant.

These findings are corroborated using different robust-ness tests. Empirical distributions of statistically signifi-cant ATETs from original findings for 10,000 replications of placebo test are presented in Fig. S1 [suppl.]. In each histogram green dashed lines divide the top and bottom 10% of the distribution. On the other hand, red line (mar-king our initial ATET estimates) mostly falls within these 10% tails of distribution, indicating only a small proba-bility that estimated ATETs occur by chance, and are ins-tead attributable to the grants. Next, sensitivity analysis utilizing different matching methods (Table S7 [suppl.]) also confirm the robustness of the main results, as does the Rosenbaum bounds test (Table S8 [suppl.]), indicating that most of our significant effects are robust for up to 20% of hidden bias.

We additionally estimated heterogeneous ATET using different firm characteristics: size, region, and tra-de orientation (Tables S9 and S10 [suppl.]). In terms of survival on the market, all the significant effects in the first year after obtaining the grant come from micro- and small-sized firms from the Central Croatia region, focu-sed exclusively on the domestic market. In terms of firm size, the most significant and greatest effects were ob-served in micro-sized firms, particularly in survival on the market (only first year after obtaining the grant) and in output additionality. For the latter, compared to their large competitors, micro-sized firms recorded up to ten times greater growth rates in sales or value added, with significant increases in number of employees and obtai-ned capital. They also managed to increase their TFP in the short and midterm, as well as to reduce their indeb-tedness. Moving on to results based on regional distri-bution of beneficiaries, the allocated grants seem to be the most effective in the Central Croatia region, which is somewhat surprising, given that traditionally the eas-tern parts of Croatia are more reliant on agriculture. Firms situated in Central Croatia showed the greatest grant additionality in turnover, value added, and inter-mediate inputs. Also, these firms managed to boost their labor productivity and their TFP in the short and mid run, and to reduce their indebtedness in the short run. On the other hand, firms in Western Croatia managed to acquire more capital and recorded a significant increase in total assets. In terms of firms’ trade orientation, there does not seem to be a clear pattern in which one group outper-forms the other. Firms concentrating solely on the do-mestic market showed greater survival effects (only in the first year after treatment) and greater increase in tur-nover and capital. On the other hand, firms that are both exporters and importers managed to increase their value added, employ additional workers, increase their average wage, and boost their labor productivity and TFP.

 

Table 3. Treatment effects of SAPARD and IPARD grants on firm performance

[a] “t” denotes the year the firm received the grant. Standard errors (SE) are presented in brackets below estimated ATETs and are based on Abadie & Imbens (2008). * p < 0.1; ** p < 0.05; *** p < 0.01, one-sided p-values.

 

Table 4. Quantification of treatment effects for the SAPARD/IPARD grant scheme

We estimated the effects for the sample of treated firms in our analysis. All monetary variables are expressed in EUR. 1 EUR ~ 7.42 Croatian kuna (HRK). “t” denotes the year the firm received the grant. [a] We report only significant effects

 

Cost-benefit analysis

Focusing on the cost side, the amount of public funds provided for 157 SAPARD/IPARD grants was 64.9 million EUR (2007–2016 period, Table S2 [suppl.]). Switching over to benefits side, awarded grants are as-sociated with average increase in turnover of 0.9 million EUR, 1.3 million EUR and 0.8 million EUR per firm at ti-mes t+1, t+3, and t+5, respectively (Table 4). In terms of total turnover increase (multiplying average values with total number of awarded grants) this amounts to 140.7 million EUR, 202.1 million EUR, and 121.8 million EUR at times t+1, t+3, and t+5, respectively. This implies that the estimated benefits of the grant schemes (in terms of extra turnover) outweigh the grant scheme costs by 2.2, 3.1, and 1.9 times in the short and midterm after the grants were distributed. We can thus speculate that the grants had the greatest effect in the midterm, three years after they were distributed.

 

 

 

Discussion Top

Our results suggest that analyzed grants had different impacts with respect to the period considered, the type of performance indicator, and the group of companies. Overall, positive output additionality in all five years after treatment was found for turnover, value added, and to-tal assets. The beneficiaries of the grants also saw higher growth in employment, TFP, labor productivity, and ca-pital input throughout the five years. The grants affected higher growth in bank loans in a long term (t+4, t+5), and growth in profits in short term (t+1), while decrease in the debt ratio was significant in year t+1. The treatment did not affect the wage growth and financial stability of the companies. Our results are consistent with those of most other studies examining the impact of investment aids or grants from rural development programs on business operations (Dantler et al., 2010; Pagliarino et al., 2014; Hlavsa et al., 2017). The only difference was found for profit, where other authors found no effect or a negative effect (Mezera & Špička, 2013; Ratinger et al., 2013) while we found a positive one. Cost-benefit analysis showed that estimated benefits of the grant schemes outweigh the grant scheme costs in the short and midterm. These benefits are similar to 2.7 times higher value added created by women entrepreneurship policies in Croatia (Srhoj et al., 2019) or by export-oriented policies in Croatia (Srhoj & Walde, 2020). Since the grants also had positive additionality on employment, total social benefits were even higher.

The results obtained largely justify the purpose of the grants, which was the modernization and upgrading of production capacities. Looking at the results by groups of companies, it is interesting that the effect of the grants on survival on the market in year t+1 is concentrated on micro and small enterprises that are focused on the domestic market and located in Central Croatia. Micro-sized enterprises that received grants saw ten times higher growth in sales and value added than large enterprises. For micro-sized enterprises, employment and TFP also increased more, while indebtedness decreased. In terms of regional differences, the grants appear to have had the greatest impact in Central Croatia.

Considering that the results show a much stronger response through the growth of indicators in small enterprises, it is necessary to set rules and criteria in support programs to enable the participation of as many micro and small enterprises as possible. In the current Croatian Rural Development Program, we can see the consequences of criteria set in exactly the opposite way. In the first few calls within the investment measures, small enterprises found it difficult to qualify for an investment grant and the very high amount of the maximum grant per beneficiary (EUR 5 million) led to a drastic decrease in the number of beneficiaries, especially small enterprises.

Despite some circumstances that could lead to opposite conclusions (such as the demanding approval procedure for the grants, relatively modest funding compared to generous national support programs, expensive pre-financing loans, and the unfavorable economic situation), the results of this study suggest that the pre-accession programs in Croatia had a positive impact on the beneficiary’s growth and business performance indicators. Given that our analysis captured just under three-quarters of all firms who obtained SAPARD or IPARD funding, we would argue that positive impact of analyzed grants are quite generalizable.

This paper provides an insight into the net impact of pre-accession grants in Croatia, and thus promotes the application of similar research in other EU candidate countries where the same or similar funds are implemented. The study also demonstrates the applicability of the selected methods in the circumstances of available data at the national level. In addition, we suggest the same approach to be applied for EU programs available to Croatia as an EU member state.

Our research adds a novelty to previous SAPARD/ IPARD evaluation papers in several ways. Firstly, our research uses rich firm-level dataset from population of Croatian enterprises, which enabled us to expand the number of analyzed performance indicators, and to analyze grant effect according to different firm characteristics (region, size, sector, export orientation) as suggested by Dvouletý et al. (2021). Furthermore, dimensionality of our dataset enabled us to select a counterfactual that is as close as possible to the treated firms, increasing the robustness of our results. Secondly, we have also added a temporal analysis of these grant effects in terms of short- and mid-term effects, as our dataset enables us to follow firms up to 2017. Next, our study is based on quasi-experimental research approach by utilizing counterfactual impact analysis, i.e., a combination of DID approach and PSM, to investigate policy impact on firm survival and performance, which has numerous advantages over simple “before-and-after” analysis. Fourthly, our paper presents a novelty to the policy evaluation literature on agri-food firms as our analysis is performed in specific conditions of a long recession period. Finally, this research can be viewed as an important contribution to the successful adoption of impact evaluation methods of agricultural pre-accession programs in present-day EU candidate countries.

Based on the experience in data collection and obtained results, we developed several recommendations for policy makers regarding the impact analysis of support programs in agriculture and rural development. Firstly, it is necessary to insist on the use of counterfactual analysis as part of the evaluation process of publicly funded development programs to obtain an accurate impact assessment of the program itself and not just an overall assessment of its implementation process. Our second recommendation aims at the need to strengthen existing and/or develop new systems to collect the necessary data on all enterprises, as only in this way can a quasi-experimental approach be applied to the whole population of potential beneficiaries. Consideration should be given to introducing an obligation to provide the business data needed to analyze the impact of the aid scheme. This is particularly important for users who do not have business reporting obligations. Our third recommendation, given the observed differences in the intensity of the program's impact on beneficiaries from different groups (e.g., by size and geographical area), is that rules and criteria for granting aid should be designed to maximize their impact across these heterogeneous groups of firms. Based on these policy recommendations, this research may be of interest to the research community, particularly in neighboring countries, which have financial databases of enterprises like Croatia, as to how these databases can be used for counterfactual analyses of the impact of public support programs.

Finally, there are a few limitations to our study, some of which can be addressed in further research. First one is related to the possibility that an essential unobserved covariate was not included in the analysis. Therefore, future research should look for new enterprise characteristics, which could be related to organizational and human resources as well, and which may play an important role, especially in small enterprises. Another standard limitation is that we did not conduct a general equilibrium analysis, but an average treatment effect analysis only. There might be spillovers to other firms (e.g., consultancies or suppliers of equipment), the unintended deadweight effects, or interactions with other policies. The quality of the research could also be raised by including the number of points per application during the tender, as rejected applicants could represent an additional control group. Finally, this study includes enterprises that issue financial statements, i.e., are subject to profit tax, meaning that family farms and similar enterprises, which make up by far the largest number of farmers in Croatia, are not included. Consequently, similar research at a comprehensive level requires finding alternative sources of information about assets, production, sales, and business performance. At the end, despite the superiority of counterfactual analysis in quantitative terms, the overall evaluation of the implementation and/ or impact of the support program on society and the economy needs to incorporate other, i.e., qualitative methods to provide an answer to the question of why the identified effects occurred.