U.S. flagAn official website of the United States government
                    Here's how you know **Official websites use .gov**
                            
                            A **.gov** website belongs to an official government organization in the United States.
 **Secure .gov websites use HTTPS**
                            
                            A **lock** (                            
                              A locked padlock                                                      
                            
                            ) or **https://** means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites.


Menu Agricultural Research Service
						
                    U.S. DEPARTMENT OF AGRICULTURE
 Food Animal Environmental Systems Research: Bowling Green, KY
 close



    Research

                    Research Projects
                                    Publications
                                    Collaborations
                                    News
                                    Software
                People

                    Staff Listing
                                    Careers


        


                    Models & Software
                        Sorption Isotherm Spreadsheets
                                    Annual Phosphorus Loss Estimator (APLE) Model
                                    Modified Arcsine Log Calibration Curve (ALCC)Search small
                    Search
            
            ARS Home
                                About ARS
                                Contact Us




ARS Home Midwest Area Bowling Green, Kentucky Food Animal Environmental Systems Research Research Publications at this Location





- ARS National Programs
- Search for a scientific manuscript
- Search for a research project
- ARS Software
- Programs and Projects
- Collaborations
- Subjects of Investigation
- Publications

Title:   Author  item
                                            TELLINGHUISEN, JOEL - VANDERBILT UNIVERSITYitem
                                            Bolster, Carl
  Submitted to: Journal of Physical Chemistry
                                    Publication Type: Peer Reviewed Journal
                                    Publication Acceptance Date: 2/20/2009
                                    Publication  Date: 4/4/2009
                                    Citation: Tellinghuisen, J., Bolster, C.H. 2009. Weighting formulas for the least-squares analysis of binding phenomena data. Journal of Physical Chemistry. 113:6151-6157.Interpretive Summary: The Langmuir equation is of the form of a rectangular hyperbola, y = a b x /(1 + b x ), and is widely used for fitting phosphorus sorption data to obtain model soil sorption constants. The most common approach for fitting this model to data is through the use of least-squares analysis. However, it is commonly the case in most sorption studies that the "independent variable" is actually a directly measured quantity, and the dependent variable is simply a computed function of the independent variable. These circumstances violate one of the fundamental tenets of most least-squares methods — that the independent variable be error-free. We treat this problem by deriving weighting formulas for the least-squares analysis of such data for the rectangular hyperbola and all of its common linearized versions — the double reciprocal, y-reciprocal, and x-reciprocal forms. We verify the correctness of these expressions by computing the nonlinear least squares parameter standard errors for exactly fitting data, and we confirm their utility through Monte Carlo simulations. Our findings will improve the statistical treatment of phosphorus sorption data.
                                Technical Abstract:  The rectangular hyperbola, y = a b x /(1 + b x ), is widely used as a fit model in the analysis of data obtained in studies of complexation, sorption, fluorescence quenching, and enzyme kinetics. Frequently the "independent variable" x is actually a directly measured quantity, and y may be a simply computed function of x, like y = x0 – x. These circumstances violate one of the fundamental tenets of most least squares methods — that the independent variable be error-free — and they lead to fully correlated error in x and y. Using an effective variance approach, we treat this problem to derive weighting formulas for the least-squares analysis of such data by the given equation and by all of its common linearized versions — the double reciprocal, y-reciprocal, and x-reciprocal forms. We verify the correctness of these expressions by computing the nonlinear least squares parameter standard errors for exactly fitting data, and we confirm their utility through Monte Carlo simulations. The latter reveal a problem with the inversion methods when the inverted data are moderately uncertain (~30%), leading to the recommendation that the reciprocal methods not be used for such data. For benchmark tests, results are presented for specific data sets having error in x alone and in both x and x0. The actual estimates of a and b and their standard errors vary somewhat with the choice of fit model, with one important exception: The Deming/Lybanon algorithm treats multiple uncertain variables equivalently and returns a single set of parameters and standard errors independent of the manner in which the fit model is expressed.


Return to top


Connect with ARS Sign up


ARS Home
                            USDA.gov
                            Plain Writing
                            Policies & Links
                            Civil Rights Statements
                            FOIA
                            Accessibility Statement
                            Privacy Policy
                            Non-Discrimination Statement
                            Quality of Information
                            USA.gov
                            WhiteHouse.gov