Term
What is the role of the control variables in crosssectional multiple regression designs for OIE? 

Definition
Control variables capture the factors that may affect Y independently from the program and that may be differently distributed between the treated and nontreated units. Once inserted in a crosssectional multiple regression design, control variables enable the analysis to separate the impact on Y due to the program from the impact on Y due to other factors that differentiate the treated units from the nontreated units. 


Term
The control variables should be selected for a multiple regression model using what criteria in OIE? 

Definition
They have to represent factors that may potentially affect the outcome Y of the analysis independently from the program. b) They are distributed differently between the treated and nontreated group 


Term
What is the meaning of the term: “sensitivity of impact estimates to different functional forms of the control variables”? 

Definition
means that the impact estimates changes drastically based on the type of functional form (dummy, continuous, categorical, etc.) used for inserting the control variables in the estimation model. Such high volatility of the impact estimates is a problem because it does not allow to draw meaningful policy recommendations from the results of the analysis. 


Term
What possible solutions can be adopted when results are sensitive to the different functional forms of the control variables? 

Definition
1. Run an extensive sensitivity analysis by replicating the model estimation under all possible combinations of alternative functional form choices for the control variables. 2. Implement the analysis with a Propensity score matching technique (through the balancing property test, propensity score matching does allow the analysis to be implemented with a validated functional form for the control variables) 


Term
Explain what is the role of the Propensity score for OIE. 

Definition
The propensity score (PS) is a parameter (ranging from 0 to 1) capable of summarizing all different control variables adopted in the analysis. For programs targeting units of observation in need of assistance, the PS can be intuitively assimilated to the % degree of initial distress of the units of observations (PS values close to 0 represent low distress, PS values close to 1 represent high distress). PS scores are then used to match treated units with comparable nontreated units through different propensity score matching techniques. 


Term
When should you use CrossSectional Multiple Regression? 

Definition
Treated and nontreated units have not identical characteristics and Panel Data is not available 


Term
What form of bias do multiple regression control variables combat? 

Definition
Inserting such characteristics as “control variables” in a multiple regression model helps in reducing selection bias: 


Term
How do you choose the correct functional form of a variable in multiple regression? 

Definition
Unfortunately there are no certain criteria to choose “correct” functional forms. Possible Solutions: 1) extensive sensitivity analysis to check whether or not results are stable to different functional form options; 2) using PSM (propensity score matching) to run the analysis instead of multiple regression models. 


Term
How does Propensity Score Matching avoid the problems that arise with differing functional forms in regression? 

Definition
This is due to the PSM “balancing property” 


Term
In what way is using Propensity Scores like regression 

Definition
It basically is logit regression.
Besides a couple of technical details, Propensity Scores work very similar to multiple regression. The model is only as good as the control variables. 


Term
How is the output of the propensity score interpreted? 

Definition
It is the output of a logit regression (from 0 to 1)
close to 1 = highly disadvantaged initial conditions, more likely to see intervention (e.g areas with high crime, low income, …) close to 0 = favorable initial conditions unlikely to receive intervention 


Term
How do we put into practice the "balancing property" for Propensity score matching? 

Definition
1) The propensity score  P(T1)  B0 + B1 + B2 + B3 is estimated based on a specific functional form for each variable
2) All units (both the treated and the nontreated) are sorted according to their Propensity Score value 


Term
Under the balancing property of Propensity Score matching, how do you the functional form of your variables is validated? 

Definition
The functional form is validated if: The entire sample can be stratified with contiguous strata containing at least one treated and one nontreated unit (We need to divide the data into different strata, in our class data for example, the first acceptable strata would be 136 since 37 is the first treated unit, 136 are all T=0)
2)Within each strata the mean value for each control variable (B1, B2, B3, etc) has to have no statistically significant differences between the treated and the nontreated units 


Term
How does "Nearest Available" Propensity Score Matching work? 

Definition
You can only use each district once, so you're removing them in pairs of two. The more you use them, the worse the matches will get as there will be less suitable comparisons
1) The treated units (NT) are listed in a separate file and sorted based on their PS value (or in a random order)
2) The first of the treated units (NT) is matched with the nontreated unit having the most similar PS
3) The two matched units are removed from the original lists and placed in a third file. Steps 13 are replicated for each of the NT treated units. 


Term
Impact Estimate for "Nearest Match" Propensity Score 

Definition
1) summing the single ΔY (change in the dependent variable) between each treated and matched units
2) computing the weighted averages of the single ΔY between each treated and matched units with weight Wi 


Term
Impact estimate for "Nearest Match" Propensity Score  formula. 

Definition


Term

Definition
"Nearest Match" Propensity Score Matching 


Term
What are tradeoffs involved with choosing the tolerance in radius (PS) matching estimators? 

Definition
On the one hand, it would be an advantage to enlarge the radius in order to obtain a larger estimation sample (improving the statistical efficiency of the impact estimates). On the other hand, to limit selection bias it would be an advantage to choose a radius as small as possible. 


Term
What are advantages and disadvantages of PS matching with replacement versus nearest available PS matching? 

Definition
Advantages: it reduces the risk of having to match the last treated units with nontreated units with too distant PS. Disadvantages: it is more sensitive to measurement errors in the data affecting PS values of nontreated units (i.e. one nontreated unit with high PS can be matched to a large number of treated units, amplifying the effects of the possible measurement errors on the impact estimates). 


Term
What is the main advantage of PS matching over multiple regression models? 

Definition
PS matching can exploit the “balancing property” to test whether or not a given functional form of the control variables is appropriate. As a consequence, PS matching does not suffer from possible sensitivity of impact estimates to different functional forms of the control variables. 


Term
What is the additional advantage of PS matching versus multiple regression models when Y data have to be retrieved through primary data collection? 

Definition
Using a PS matching procedure (except for kernel matching) reduces the number of units used in the analysis and for which data collection has to take place; Reduced costs for the primary data collection of the Y data (in cases in which Y data are not available from statistical offices) 


Term

Definition
Radius matching allows each treated units to be matched with more than one nontreated unit
A “tolerance radius” is established, so that each nontreated units with a PS within the tolerance is selected to be part of the comparison group for a given treated unit.
Radius matching is usually implemented with replacement: a same nontreated unit can be included in the comparison group of more than one treated unit. 


Term
How to choose the tolerance radius in radius matching PS? 

Definition
Tradeoff to be balanced:
•To obtain a larger estimation sample (improving the statistical efficiency of the impact estimates) radius should be kept not too small
•To limit selection bias issues radius should be chosen as small as possible.
In all cases, once eliminated the units outside the common support, min radius has to be chosen so that each treated unit has a nonzero comparison group. 


Term

Definition


Term

Definition
Most advanced statistical matching procedure in this class
With Kernel matching the outcome (YT) of each treated unit is compared to a weighted average of the outcomes (YNT) of all nontreated units
Weights are inversely proportional to the distance between PS of the given treated unit (i) and the PS of the nontreated units 


Term

Definition


Term
When is Conditional Matching (Kernel Matching) the appropriate matching form? 

Definition
Conditional matching is recommended when it is suspected that relevant unobservable characteristics may be differently distributed between treated and nontreated units that
For example, nontreated units: 1) Are located in different regions or;
II) Operate in different sectors (for analyses with firmlevel outcomes B) or;
III) Have different sizes. 


Term
How do you conduct conditional matching? 

Definition
Treated and nontreated units are separately sorted into the categories defined by location or business sector or size
Separately for each category based on location, business sector or size.one matching procedure is applied
such that, the treated units are matched only with the nontreated units belonging to the same category (and therefore possessing the same unobserved characteristics of the treated unit). 


Term
The Importance of using Panel Data 

Definition
Comparing treated with nontreated units is not useful when inferring the impact of the program because of different initial characteristics 


Term
Difference in Difference Estimator 

Definition
a^=E(Ypost  Ypre  Ti=1)  E(Ypost  YpreTi=0)
DD estimators require the availability of both pre and postintervention data
DD allows the analysis to control for some unobservable differences between the treated and the nontreated units 


Term
What is the major advantage of panel data? 

Definition
Every time we compare outcomes between the treated and nontreated we face a certain danger  namely, the two groups are not really comparable and suffers from selection bias. The treated unit might simply have different characteristics than nontreated units. The major advantage of panel data is that when you don't compare not just the outcomes but also the prepost change between T=1 and T=0, is that any initial characteristic that is potentially different between T=1 and T=0 (as long as it is a fixed effect) does not have to be controlled for with a control variable. At the very least, you don't need AS MANY control variables. 


Term
How does adding additional preintervention data make impact estimates for reliable? 

Definition
the additional data (e.g. 1995) allows to estimate whether or not the preintervention trend of Y was different between the treated and nontreated units. Any difference that is detected between treated and nontreated is incorporated in the analysis as a factor used to adjust the initial estimate of the counterfactual trend. 


Term
Difference in Difference in Difference impact formula 

Definition
a^ = E[(Y2005  Y2000)  (Y2000  Y1995)Ti=1]  E{(Y2005  Y2000)  (Y2000  Y1995)Ti=0] 


Term
With a DDD model, in which way is the counterfactual estimated? 

Definition
The counterfactual is estimated as the prepost intervention change of Y recorded in the nontreated units, corrected by the preintervention differential change of Y between the treated and nontreated units. 


Term
What is the advantage of combining a DD scheme with Multiple regression or PS matching compared to multiple regression or PS matching without a DD scheme? 

Definition
When panel data are available, combining a DD scheme with Multiple regression or PS matching reduces the need to include in the analysis observable control variables. This is because all factors that can be assumed to be fixed effects 


Term
How does conditional Difference in Difference with Propensity Score Matching work? 

Definition
1) Based on an appropriate set of control variables, a PS variable is estimated 2) A nearest available (with or without replacement) PS matching, or a radius matching procedure is implemented 3) The impact estimates are obtained comparing the prepost intervention difference of Y between the treated and the matched nontreated units (i.e. a^=E(Ypost  YpreTi=1)  (Ypost  Ypre Ti=0) 


Term
What is the advantage of Conditional DD with multiple regression models (and of Conditional DD with PS matching models) compared to pure DD models? 

Definition
Compared to pure DD model, in order to obtain unbiased results, CDD with MR models do not require making the hypothesis that the observable control variables X are fixed effects 


Term
What is the advantage of Conditional DD with PS matching models compared to Conditional DD with multiple regression models? 

Definition
Compared to Conditional DD with multiple regression models, CDD with PS matching offers the following advantages: it solves the issue of sensitivity of impact estimates to different functional forms of the control variables it reduces costs for datacollection if Y data has to be collected for the evaluation 


Term
With Conditional DD with Multiple regression models (and Conditional DD with PS matching), in which time do you have to measure the control variables? Why is this the case? 

Definition
Typically you have to measure control variables at the preintervention time. This is to reduce the risk of the control variables becoming endogenous to the treatment (i.e. the control variables becoming affected by the treatment itself) 


Term

Definition
Very often control variables cannot be included if measured during the same times of the program intervention. This is because of “endogeneity” problems
For example, If EZ incentives works very well, they could lower crime rates in the years during the program intervention. Crime rate changes during the program intervention would not be something to control for, but they would be a secondary outcome of the program intervention 

