Package: fastreg
Fast sparse regressions with advanced formula syntax. Good for high-dimensional fixed effects. Installation and usage are described below. Detailed documentation can be foundfurther down.New: generalized linear models and maximum likelihood estimation with JAX.InstallTo install from PyPI with pip:pipinstallfastregTo install directly from GitHub:pipinstallgit+https://github.com/iamlemec/fastregAlternatively, you can clone this repository locally and runpipinstall-e.Optionally, for the maximum likelihood routines, you'll needjax(andjaxlib) as well. Seeherefor detailed instructions.UsageFirst import the necessary functionsimportfastregasfrfromfastregimportI,R,CCreate some testing datadata=fr.dataset(N=100_000,K1=10,K2=100,models=['linear','poisson'])y0yx1x2id1id200.1403.450-0.2600.958E371-0.5520.9550.334-1.046I652-0.6831.5170.067-0.631I10...We can construct formulas to define our specification. To make a realFactoronx1, useR('x1')or more convenientlyR.x1. These can then be combined intoTerms with*and then intoFormulas with+. Regressy0on1,x1, andx2givenpandasDataFramedata:fr.ols(y=R.y0,x=I+R.x1+R.x2,data=data)coeffstderrlow95high95pvalueI0.0990.0030.0930.1050.000x10.3040.0030.2970.3100.000x20.6030.0030.5970.6090.000Regressyon1,x1,x2, categoricalid1, and categoricalid2:fr.ols(y=R.y,x=I+R.x1+R.x2+C.id1+C.id2,data=data)coeffstderrlow95high95pvalueI0.1530.0330.0880.2180.000x10.2950.0030.2890.3020.000x20.5940.0030.5880.6000.000id1=B0.0720.0140.0440.0990.000id1=C0.1680.0140.1400.1950.000...Regressyon1,x1,x2, and all combinations of categoricalsid1andid2(Note that*is analogous to:in R-style syntax):fr.ols(y=R.y,x=I+R.x1+R.x2+C.id1*C.id2,data=data)coeffstderrlow95high95pvalueI0.1580.107-0.0510.3680.138x10.2950.0030.2890.3010.000x20.5930.0030.5870.5990.000id1=A,id2=1-0.0680.144-0.3500.2130.634id1=A,id2=20.0600.155-0.2440.3630.700...Instead of passingyandx, you can also pass an R-style formula string toformula, as in:fr.ols(formula='y ~ 1 + x1 + x2 + C(id1):C(id2)',data=data)There's even a third intermediate option using lists and tuples, which might be more useful when you are defining specifications programmatically:fr.ols(y=R.y,x=[I,R.x1,R.x2,(C.id1,C.id2)],data=data)Right now, categorical coding schemes other than treatment are not supported. You can pass a list of column names toclusterto cluster standard errors on those variables.Categorical codingFor categorical variables, one must avoid collinearity by either not including an intercept term or by dropping one value. The default for categorical variables is to drop the first value in alphabetical/numerical order. You can specify which value to drop by passing that as an argument to the specified variable. For instance, if one wanted to dropBfrom the factorid1, they would writeC.id1('B')or equivalentlyC('id1', 'B'), or more verboselyC('id1', drop='B'). You can also tell it to not drop any values by passingfr.NONEand explicitly tell it to drop the first value withfr.FIRST.In the case of interacted categorical variables, you would typically specify the dropped value for each factor and this will be inherited to the term level. For instance, if one wished to dropid1 = Bandid2 = 3from the interaction of these two terms, they would writeC.id1('B')*C.id2(3). An alternative method would be to write(C.id1*C.id2).drop('B', 3). When creating compound categorical terms, an attempt is made to find the correct drop strategy. In the case of ambiguity or when no information is given, the default is againFIRST. When interacting categorical and real variables, the default isNONE, as this source of collinearity is no longer an issue.High dimensionalPoint estimates are obtained efficiently by using a sparse array representation of categorical variables. However, computing standard errors can be costly due to the need for large, dense matrix inversion. It is possible to make clever use of block diagonal properties to quickly compute standard errors for the case of a single (possibly interacted) categorical variable. In this case, we can recover the individual standard errors, but not the full covariance matrix. To employ this, pass a singleTerm(such asC.id1orC.id1*C.id2) with thehdfeflag, as infr.ols(y='y',x=I+R.x1+R.x2+C.id1,hdfe=C.id2,data=data)You can also pass a term to theabsorbflag to absorb those variables a la Stata'sareg. In this case you do not recover the standard errors for the absorbed categorical, though it may be faster in the case of multiple high-dimensional regressors. This will automatically cluster standard errors on that term as well, as the errors will in fact be correlated, even if the original data was iid.Generalized linear modelsWe can do GLM now too! The syntax and usage is identical to that ofols. For instance, to run a properly specified Poisson regression using our test data:fr.poisson(y=R.p,x=I+R.x1+R.x2+C.id1+C.id2,data=data)coeffstderrlow95high95pvalueI0.3220.0110.3000.3440.000x10.2940.0010.2930.2960.000x20.5970.0010.5960.5990.000id1=B0.0720.0050.0620.0810.000id1=C0.1780.0050.1690.1870.000...You can use thehdfeflag here as well, for instance:fr.poisson(y=R.p,x=I+R.x1+R.x2+C.id1,hdfe=C.id2,data=data)Under the hood, this is all powered by a maximum likelihood estimation routine ingeneral.pycalledmaxlike_panel. Just give this a function that computes the mean log likelihood and it'll take care of the rest, computing standard errors from the inverse of the Fisher information matrix. This is then specialized into a generalized linear model routine calledglm, which accepts a loss function along with data. I've provided implementations forlogit,poisson,negbin,zinf_poisson,zinf_negbin, andgols.Custom factorsThe algebraic system used to define specifications is highly customizable. First, there are the core factorsI(identity),R(real), andC(categorical). Then there are the provided factorsD(demean) andB(binned). You can also create your own custom column types. The simplest way is using thefactorfunction decorator. For instance, we might want to standardize variables:@fr.factordefZ(x):return(x-np.mean(x))/np.std(x)The we can using this in a regression with eitherZ('x1')orZ.x1, as in:fr.ols(y=R.y0,x=I+Z.x1+Z.x2,data=data)We may also want factors that use data from multiple columns. In this case we need to useeval_argsto tell it what expressions to map, as it defaults to only the first argument (0). For example, to implement conditional demean (which is also included by default asfr.D), we would do:@fr.factor(eval_args=(0,1))defCD(x,i):datf=pd.DataFrame({'vals':x,'cond':i})cmean=datf.groupby('cond')['vals'].mean().rename('mean')datf=datf.join(cmean,on='cond')returndatf['vals']-datf['mean']and then use it in a regression, though we can't use the convenience syntax with multiple argumentsfr.ols(y=R.y0,x=I+CD('x1','id1')+CD('x2','id2'),data=data)Thefactordecorator also accepts acategflag that you can set toTruefor categorical variables. Finally, it may be useful to inject functions into the evaluation namespace rather than create a whole new factor type. To do this, you can pass adictto theexternflag and prefix the desired variable or function with@, as in:extern={'logit':lambdax:1/(1+np.exp(-x))}fr.ols(y='y0',x=I+R('@logit(x1)')+R.x2,data=data,extern=extern)DocumentationThe core functionality of this library lies in creating well-structured data matrices (often called "design matrices") from actual data in the for of Pandas DataFrames and a regression specification, either Fastreg-style or R-style. For that, we have the following function defined informula.py. You must always passdataas well as eithery/xorformula.fastreg.design_matrices(y=None,x=None,formula=None,data=None,dropna=True,prune=True,validate=False,flatten=True,extern=None,warn=True)y: specification for the outcome variable, a column name (str) or a singleTerm, which might be the combination of multipleFactorsx: specification for the input variables, aFormulaorlistofTermsformula: an R-style specification string, this will override anyyorxgiven abovedata: a DataFrame with the underlying datasetdropna: drop any rows containing null dataprune: prune categories that have no instancesvalidate: return binary mask specifying which rows were droppedflatten: combine dense and sparsexvariables into one matrixextern: a dictionary of functions for use in specificationwarn: output info on dropped rows or categoriesThis returns (data, name) pairs for bothyandxvariables. In addition, if you only want to deal with thexvariables, you can usedesign_matrix, which has nearly identical syntax but does not accept theyargument. Next is theolsfunction defined inlinear.pythat handles regressions.fastreg.ols(y=None,x=None,formula=None,data=None,cluster=None,absorb=None,hdfe=None,stderr=True,output='table')y: specification for the outcome variable, a column name (str) or a singleTerm, which might be the combination of multipleFactorsx: specification for the input variables, aFormulaorlistofTermsformula: an R-style specification string, this will override anyyorxgiven abovedata: a DataFrame with the underlying datasetcluster: cluster standard errors on the givenTermabsorb: regress on differences within groups specified by givenTermhdfe: use block inversion to speed up standard error calculation for givenTermstderr: standard error type,Truefor basic, andhc0-hc3for robust typesoutput: control output,tablegives DataFrame of estimates,dictgives much more infoOther estimators use syntax very similar to that ofols. This includesglmingeneral.py, which also accepts custom alossfunctions. For instance, the built-inpoissonuses a Poisson likelihood loss function (with an exponential link). Below only the arguments not common toolsare listed.fastreg.glm(y=None,x=None,formula=None,data=None,hdfe=None,loss=None,model=None,extra={},raw={},offset=None,epochs=None,display=True,per=None,stderr=True,output='table')loss: the loss (log likelihood) function to use for optimization, can be one of'logit','poisson','negbin','normal','lognorm','lstsq', or a custom function that accepts(params, data, yhat, y)model: in lieu of a loss function, one can specify a model function mapping from(params, data)to an average log likelihoodextra: adictof extra parameter names mapping to initial values that can be accessed by thelossfunctionraw: adictof extraTermspecifications that are evaluated and passed to thelossfunction as part ofdataoffset: aTermto evaluate and add to the linear predictor (for instance,R('log(t)'))epochs: how many full iterations over the dataset to do during optimizationdisplay: whether to display updates on objective and parameter values during optimizationper: how often to display updates during optimization