Package: ebcic
EBCIC: Exact Binomial Confidence Interval Calculator日本語(github.io,github)These programs are mainly for researchers, developers, and designers who calculate Binomial Confidence Intervals.EBCICcalculates binomial intervals exactly, i.e. by implementing Clopper-Pearson interval [CP34] without simplifying mathematical equations that may deteriorate intervals for certain combinations of parameters. EBCIC can also shows graphs for comparing exact intervals with approximated ones.How to useJupyter notebookOpenebcic.ipynbwith Jupyter-notebook-compatible development environment such as Jupyter Notebook, JupyterLab, or Visual Studio Code.Run the following initial cells:# Run this cell, if `ebcic` package has not been installed yet:%pipinstallebcicimportebcicfromebcicimport*Run the cells you want to execute.Command lineInstallationWhen usingPyPI ebcic package:pip install ebcicWhen using githubebcicrepo:git clone https://github.com/KazKobara/ebcic.gitcd ebcicCommand-line helpCheck the version and options:python -m ebcic -hCf. theexamplesbelow.MATLAB (with Python andebcicpackage)Install Python for MATLAB andebcicpackage according tothis page.Open a sample MATLAB code fileebcic_in_matlab.mas a 'live script' as shownthis page.Edit and run the sections you want to execute.NOTE: If you manage the edited file with git, save it as a MATLAB code file (*.m) file to commit (or commit the live code file (*.mlx) to a git LFS (Large File Storage)) since live code files (*.mlx) are not git friendly. If necessary, save it as a *.html file as well to check its look.Command Line ExamplesTo print exact intervals as text.One-sided upper 95% confidence interval for no error among 100 trialspython -m ebcic -k 0 -n 100 -c 95 -uFork=0ork=n, give-coptionone-sidedconfidence percentage.v0.0.4 or newer returns the same value as the above result by setting--rej-perc-lower(-r) option the percentage of the lower rejection area in assuming population as follows:python -m ebcic -k 0 -n 100 --rej-perc-lower 5 -uTwo-sided 95% confidence interval for one error among 100 trialspython -m ebcic -k 1 -n 100 -c 95 -luFor0<k<n, give-coptiontwo-sidedconfidence percentage.v0.0.4 or newer returns the same value as the above result by setting both--rej-perc-lower(-r) and--rej-perc-upper(-s) options equally divided percentages of assuming population as follows:python -m ebcic -k 1 -n 100 -r 2.5 -s 2.5 -luOne-sided upper 95% confidence interval for one error among 100 trialspython -m ebcic -k 1 -n 100 -r 5 -uFor v0.0.4 and newer, set--rej-perc-lower(-r) option the percentage of the lower rejection area in assuming population.For v0.0.3 and older and0<k<n, give-coption2*s-100as follows wheresis the one-sided confidence percentage (in this cases=95and2*s-100=2*95-100=90).python -m ebcic -k 1 -n 100 -c 90 -uGiving-c 90is the same as giving--alpha 0.1(or-a 0.1).python -m ebcic -k 1 -n 100 --alpha 0.1 -uOne-sided lower 95% confidence interval for 99 errors among 100 trialspython -m ebcic -k 99 -n 100 -s 5 -lFor v0.0.4 and newer, set--rej-perc-upper(-s) option the percentage of the upper rejection area in assuming population.For v0.0.3 and older and0<k<n, the equivalent value is obtained in the same way as the previous example using-cor-aoption as follows:python -m ebcic -k 99 -n 100 -c 90 -lpython -m ebcic -k 99 -n 100 -a 0.1 -lPython Interpreter or Jupyter Cell ExamplesPrint exact interval as textEdit the following parameters,k,n, andconfi_perc(orrej_perc_lowerandrej_perc_upper), and then run the cell.print_interval(Params(k=1,# Number of errorsn=501255,# Number of trialsconfi_perc=99.0# Confidence percentage))whereconfi_percis set as follows:Fork=0ork=n:give the percentage ofone-sidedconfidence interval.For0<k<n:give the percentage oftwo-sidedconfidence interval.For0<k<nand to get one-sided confidence interval:give-coption2*s-100wheresis the one-sided confidence percentage.Result:=====Exactintervalofpwith99.0[%]two-sided(or99.5[%]one-sided)confidence=====Upper:1.482295806e-05Lower:9.99998e-09Width:1.481295808e-05For v0.0.4 and newer, instead ofconfi_percoralpha, Params() can set the confidence with
either or both ofrej_perc_lowerandrej_perc_upperin percentage of0 <= x < 50(or either or both ofrej_lowerandrej_upperin ratio of0 <= x < 0.5. Params()'s class functions are also available.Params(k=1,# Number of errorsn=501255,# Number of trials# Rejection area in percentagerej_perc_lower=0.5# Lower rejection area (to get upper interval)rej_perc_upper=0.5# Upper rejection area (to get lower interval)).print_interval()Note that it uses thelowerrejection area to get theupperconfidence interval and vice versa.Result:===== Exact interval of p with rejection area of lower 0.5 [%] and upper 0.5 [%] =====
Upper :  1.482295806e-05
Lower :  9.99998e-09
Width :  1.481295808e-05Depict graphsExact intervals and the line of k/n for k=1This program can show not only the typical 95% and 99% confidence lines but also any confidence percentage lines.Python Interpreter or Jupyter cell to run:interval_graph(GraProps(# Set the range of k with k_*k_start=1,# >= 0k_end=1,# >= k_startk_step=1,# >= 1# Edit the list of confidence percentages to depict, [confi_perc, ...],#   for two-sided of 0<k<n where 0 < confi_perc < 100, or#   for one-sided of k=0 or k=n.# NOTE For one-sided of 0<k<n, set#   confi_perc=(2 * confi_perc_for_one_sided - 100)#   where 50 < confi_perc_for_one_sided < 100#   (though both lower and upper intervals are shown).confi_perc_list=[90,95,99,99.9,99.99],# Lines to depictline_list=['with_exact','with_line_kn',# Line of k/n],# savefig=True,  # uncomment on Python Interpreter# fig_file_name='intervals.png',))Result:If figures or links are not shown appropriately, visitgithub.io pageorgithub page.Exact intervals for k=0 to 5Python Interpreter or Jupyter cell to run:interval_graph(GraProps(k_start=0,# >= 0k_end=5,# >= k_startline_list=['with_exact'],# savefig=True,  # uncomment on Python Interpreter# fig_file_name='intervals.png',))Result:Comparison of exact and approximated intervals for k=0Python Interpreter or Jupyter cell to run:interval_graph(GraProps(k_start=0,# >= 0k_end=0,# >= k_startlog_n_end=3,# max(n) = k_end*10**log_n_endline_list=['with_exact','with_rule_of_la',# rule of -ln(alpha)# available only for k=0 and k=n#'with_normal',     # not available for k=0 and k=n'with_wilson','with_wilson_cc','with_beta_approx',],# savefig=True,  # uncomment on Python Interpreter# fig_file_name='intervals.png',))where interval names to be added in theline_listand their conditions are as follows:Interval name (after 'with_')ExplanationConditionexactImplementation of Clopper-Pearson interval [CP34] without approximation.rule_of_la'Rule of -ln(a)' or 'Rule of -log_e(alpha)'; Generalization of the 'Rule of three' [Lou81,HL83,JL97,Way00,ISO/IEC19795-1] that is fork=0andalpha=0.05(95% confidence percentage), to other confidence percentages than 95% andk=n.k=0ork=nwilsonWilson score interval[Wil27].wilson_ccWilson score interval with continuity correction[New98].beta_approxApproximated interval using beta distribution.normalNormal approximation intervalorWald confidence interval.0<k<nResult:As you can see from the following figure, 'rule of -ln(a)' for largenand 'beta_approx' are good approximations fork=0.Fork=0, interval_graph() of EBCIC v0.0.3 and newer, display only one-sided upper intervals since their lower intervals must be0(though some approximations, such as 'Wilson cc', output wrong values than0).Comparison of exact and approximated intervals fork=1Python Interpreter or Jupyter cell to run:interval_graph(GraProps(k_start=1,# >= 0k_end=1,# >= k_startline_list=['with_line_kn'# 'with_rule_of_la',  # available only for k=0'with_exact','with_normal','with_wilson','with_wilson_cc','with_beta_approx',],# savefig=True,  # uncomment on Python Interpreter# fig_file_name='intervals.png',))Result:As you can see from the following figures and as warned in many papers such as [BLC01], normal-approximation intervals are not good approximations for smallk.The upper intervals of the other approximations look tight.
The approximation using beta distribution looks tight where the confidence interval fork=n=1is one-sided.Comparison of exact and approximated intervals fork=10Python Interpreter or Jupyter cell to run:interval_graph(GraProps(k_start=10,# >= 0k_end=10,# >= k_startlog_n_end=2,# max(n) = k_end*10**log_n_endline_list=['with_exact','with_normal','with_wilson','with_wilson_cc','with_beta_approx',],# savefig=True,  # uncomment on Python Interpreter# fig_file_name='intervals.png',))Result:Fork=10, 'normal' still does not provide a good approximation.Comparison of exact and approximated intervals fork=100Python Interpreter or Jupyter cell to run:interval_graph(GraProps(k_start=100,# >= 0k_end=100,# >= k_startlog_n_end=2,# max(n) = k_end*10**log_n_endline_list=['with_exact','with_normal','with_wilson','with_wilson_cc','with_beta_approx',],# savefig=True,  # uncomment on Python Interpreter# fig_file_name='intervals.png',))Result:At least fork=100and confidence percentage,confi_perc=99.0, all these approximations look tight.API ManualDownloadgit clone https://github.com/KazKobara/ebcic.gitOpen the following file with your browser (after replacing<path to the downloaded ebcic>appropriately):file://<path to the downloaded ebcic>/docs/_build/index.htmlFor WSL Ubuntu-20.04, replace<username>and<path to the downloaded ebcic>appropriately:file://wsl%24/Ubuntu-20.04/home/<username>/<path to the downloaded ebcic>/docs/_build/index.htmlBibliography[CP34] Clopper, C. and Pearson, E.S. "The use of confidence or fiducial limits illustrated in the case of the binomial," Biometrika. 26 (4): pp.404-413, 1934[Lou81] Louis, T.A. "Confidence intervals for a binomial parameter after observing no successes," The American Statistician, 35(3), p.154, 1981[HL83] Hanley, J.A. and Lippman-Hand, A. "If nothing goes wrong, is everything all right? Interpreting zero numerators," Journal of the American Medical Association, 249(13), pp.1743-1745, 1983[JL97] Jovanovic, B.D. and Levy, P.S. "A look at the rule of three," The American Statistician, 51(2), pp.137-139, 1997[Way00] Wayman, J.L. "Technical testing and evaluation of biometric identification devices," Biometrics: Personal identification in networked society, edited by A.K. Jain, et al., Kluwer, pp.345-368, 2000[ISO/IEC19795-1] ISO/IEC 19795-1, "Information technology-Biometric performance testing and reporting-Part 1: Principles and framework"[New98] Newcombe, R.G. "Two-sided confidence intervals for the single proportion: comparison of seven methods," Statistics in Medicine. 17 (8): pp.857-872, 1998[Wil27] Wilson, E.B. "Probable inference, the law of succession, and statistical inference," Journal of the American Statistical Association. 22 (158): pp.209-212, 1927[BLC01] Brown, L.D., Cai, T.T. and DasGupta, A. "Interval Estimation for a Binomial Proportion," Statistical Science. 16 (2): pp. 101-133, 2001ChangelogLicenseMIT LicenseWhen you use or publish the confidence interval obtained with the software, pleaserefer to the software name, version, platform, and so on, so that readers can verify the correctness and reproducibility of the interval with the input parameters.An example of the reference is:The confidence interval is obtained by EBCIC X.X.X on Python 3."where X.X.X is the version of EBCIC.The initial software is based on results obtained from a project, JPNP16007, commissioned by the New Energy and Industrial Technology Development Organization (NEDO).Copyright (c) 2020-2022 National Institute of Advanced Industrial Science and Technology (AIST)https://github.com/KazKobara/https://kazkobara.github.io/