OpenLLM-France/wikimedia huggingface dataset:
# Programming Fundamentals/Sequence Operator

Outlines several uses of the sequence operator within the C++ programming language.

## General Discussion

The sequence (or comma) operator is used to separate items. It has several uses, four of which are listed then demonstrated:
* To separate identifier names when declaring variables or constants
* To separate several parameters being passed into a function
* To separate several initialization items or update items in a for loop
* Separate values during the initialization of an array
This first example is often seen in textbooks, but this method of declaring variables is not preferred. It is difficult to quickly read the identifier names.
The following vertical method of declaring variables or constants is preferred.

### Example 1: Preferred Vertical Method of Defining Variables

The data types and identifier names (known as parameters) are separated from each other. This example is a function prototype.
In the syntax of a for loop you have three parts each separated by a semi-colon. The first is the initialization area which could have more than one initialization. The last is the update area which could have more than one update. Mutiple initializations or updates use the comma to separate them. This example is only the first line of a for loop.
The variable ages is an array of integers. Initial values are assigned using block markers with the values separated from each other using a comma.

## Definitions
### sequence

    An operator used to separate multiple occurrences of an item.
# Programming Fundamentals/Sizeof Operator

An explanation of the sizeof operator with examples as used within the C++ programming language.

## Overview

Every data item, constants and variables, not only have a data type, but the data type determines how many bytes the item will use in the memory of the computer. The size of each data type varies with the complier being used and the computer. This effect is known as being machine dependent. Additionally, there have been some size changes with upgrades to the language. In "C" the int data type was allocated 2 bytes of memory storage on an Intel compatible central processing unit (cpu) machine. In "C++" an int is allocated 4 bytes.
There is an operator named "sizeof (… )" that is a unary operator, that is it has only one operand. The operand is to the right of the operator and is placed within the parentheses if it is a data type. The operand may be any data type (including those created by typedef). If the operand is an identifier name it does not need to go inside a set of parentheses. It works for both variable and memory constant identifier names. This operator is unique in that it performs its calculation at compile time for global scoped items and at run time for local scoped items. Examples:
The compiler would determine the byte size of an integer on the specific machine and in essence replaces the sizeof operator with a value. Integers are usually 4 bytes long, thus the line of code would be changed to:
If you place an identifier name that represents a data storage area (variable or memory constant), it looks at the definition for the identifier name. NOTE: the parentheses are not needed and often not included for an identifier name.

### Example 1: sizeof with a Variable

The compiler would determine the byte size of money by looking at the definition where it indicates that the data type is double. The double data type on the specific machine (usually 8 bytes) would replace the code and it would become:

## Definitions
### sizeof

    An operator that tells you how many bytes a data type occupies in storage.
# Programming Fundamentals/Selection Control Structures
## Overview

In selection control structures, conditional statements perform different computations or actions depending on whether a programmer-specified Boolean condition evaluates to true or false.

## Discussion

The basic attribute of a selection control structure is to be able to select between two or more alternate paths. This is described as either two-way selection or multi-way selection. A question using Boolean concepts usually controls which path is selected. All of the paths from a selection control structure join back up at the end of the control structure, before moving on to the next lines of code in a program.

### If Then Else Control Structure

The if then else control structure is a two-way selection.
| Language | Reserved Words |
| C++ | if, else |
| C# | if, else |
| Java | if, else |
| JavaScript | if, else |
| Python | if, elif, else |
| Swift | if, else |

### Case Control Structure

The case control structure is a multi-way selection. Case control structures compare a given value with specified constants and take action according to the first expression to match.
| Language | Reserved Words |
| C++ | switch, case, break, default |
| C# | switch, case, break, default |
| Java | switch, case, break, default |
| JavaScript | switch, case, break, default |
| Python | N/A |
| Swift | switch, case, break (optional), default |
Python does not support a case control structure. There are workarounds, but they are beyond the scope of this book.

## Key Terms
### if then else

    A two-way selection control structure.

### case

    A multi-way selection control structure.

## References

* cnx.org: Programming Fundamentals – A Modular Structured Approach using C++
* ↑ Wikipedia: Conditional (computer programming)
* ↑ Wikipedia: Conditional (computer programming)
# Python Programming/Metaclasses

| Previous: Reflection | Index | Next: Tips and Tricks |
In Python, classes are themselves objects. Just as other objects are instances of a particular class, classes themselves are instances of a metaclass.

### Python3

The Pep 3115 defines the changes to python 3 metaclasses. In python3 you have a method __prepare__ that is called in the metaclass to create a dictionary or other class to store the class members. Then there is the __new__ method that is called to create new instances of that class.

### The type Metaclass

The metaclass for all standard Python types is the "type" object.
Just like list, int and object, "type" is itself a normal Python object, and is itself an instance of a class. In this case, it is in fact an instance of itself.
It can be instantiated to create new class objects similarly to the class factory example above by passing the name of the new class, the base classes to inherit from, and a dictionary defining the namespace to use.
For instance, the code:
Could also be written as:

### Metaclasses

It is possible to create a class with a different metaclass than type by setting the metaclass keyword argument when defining the class. When this is done, the class, and its subclass will be created using your custom metaclass. For example
This will print
By creating a custom metaclass in this way, it is possible to change how the class is constructed. This allows you to add or remove attributes and methods, register creation of classes and subclasses creation and various other manipulations when the class is created.

### More resources

* Wikipedia article on Aspect Oriented Programming
* Unifying types and classes in Python 2.2
* O'Reilly Article on Python Metaclasses

### References

* ↑ http://www.python.org/dev/peps/pep-3115/
* ↑ http://eli.thegreenplace.net/2011/08/14/python-metaclasses-by-example/
| Previous: Reflection | Index | Next: Tips and Tricks |
# Python Programming/Naming conventions

There are various naming conventions used in Python programs.
PEP 0008 specifies naming conventions for class names (e.g. GenericTree), package and module names (e.g. generictree and generic_tree), function and variable names (storestate or store_state; mixedCase is dispreferred), and more. Google Python Style Guide follows similar naming conventions.
The above stands in contrast to Java naming convention (e.g. storeState for method names) and C# naming convention (e.g. StoreState for method names).
In Python 2, the standard library contains multiple deviations from PEP 0008. For instance, Tkinter module is spelled as Tkinter with capital T; this was renamed to tkinter in Python 3.

## External links

* Naming Conventions in PEP 0008: Style Guide for Python Code, python.org
* Naming in Google Python Style Guide, google.github.io
* pycodestyle - Python style guide checker, pypi.python.org
* Modules to Rename in PEP 3108, python.org
* W:Naming convention (programming)
# Python Programming/Networks

| Previous: Internet | Index | Next: Databases |

## Sockets

Python can also communicate via sockets.

### Connecting to a server

This simple Python program will fetch a 4096 byte HTTP response from Google:

## High-level interfaces

Most Python developers will prefer to use a high-level interface over using sockets, such as Twisted and urllib2.
| Previous: Internet | Index | Next: Databases |
# Python Programming/Loops/Solutions

< Back to Problems
1. Create a program to count by prime numbers. Ask the user to input a number, then print each prime number up to that number.
For this program we will make a list of possibilities up to the user's input, then test each one using Fermat's little theorem, verifying that it is prime, then adding the prime to a list of primes. Finally we print the list of primes.
Comment: All primes fulfill Fermat's little theorem, but some returned results are "pseudoprimes" which are not primes (e.g. 341=11*31, in python: (((2**341)-2)%341 == 0)). More information about finding primes & proving primality
2. Instruct the user to pick an arbitrary number from 1 to 100 and proceed to guess it correctly within seven tries. After each guess, the user must tell whether their number is higher than, lower than, or equal to your guess.
Year 9 work for python goes like this, this is a game for guessing a number and includes most aspects of python.
# Python Programming/Math

For basic math including addition, subtraction, multiplication and the like, see Basic Math and Operators chapters. For quick reference, the built-in Python math operators include addition (+), subtraction (-), multiplication (*), division (/), floor division (//), modulo (%), and exponentiation (**). The built-in Python math functions include rounding (round), absolute value (abs), minimum (min), maximum (max), division with a remainder (divmod), and exponentiation (pow). Sign function can be created as "sign = lambda n: 1 if n > 0 else -1 if n < 0 else 0".

## Math

A range of mathematical functions is available from math module of the standard library:

## Example code using in-built operators

This code was made to replicate the log function in a calculator

## Cmath

The cmath module provides similar functions like the math module but for complex numbers, and then some.

## Random

Pseudo-random generators are available from the random module:

## Decimal

The decimal module enables decimal floating point arithmethic, avoiding certain artifacts of the usual underlying binary representation of floating point numbers that are unintuitive to humans.

## Fractions

The fractions module provides fraction arithmetic via Fraction class. Compared to floating point numbers representing fractions, Fraction fractions do not lose precision.

## Statistics

The statistics module, available since Python 3.4, provides some basic statistical functions. It only provides basics; it does not replace full-fledged 3rd party libraries such as numpy. For Python 2.7, the statistics module can be installed from pypi.

## External links

* 2. Built-in Functions, python.org
* 9. Numeric and Mathematical Modules, python.org
* 9.2. math — Mathematical functions, python.org
* 9.3. cmath — Mathematical functions for complex numbers, python.org
* 9.4. decimal — Decimal fixed point and floating point arithmetic, python.org
* 9.5. fractions — Rational numbers, python.org
* 9.6. random — Generate pseudo-random numbers, python.org
* 9.7. itertools — Functions creating iterators for efficient looping, python.org
* 9.7. statistics — Mathematical statistics functions, python.org
# Python Programming/Natural Language Toolkit

| Previous: pyFormex | Index | Next: HTTP Tools |
NLTK is a module that provides many options for literature and word handling.

## Downloading

The NLTK module does not have anything downloaded. It has a function, download to download things. In order to bring up the download gui, run
You can also put in the specific thing you want to download
| Previous: pyFormex | Index | Next: HTTP Tools |
# Python Programming/Numbers

| Previous: Data types | Index | Next: Strings |
Python 2.x supports 4 built-in numeric types - int, long, float and complex. Of these, the long type has been dropped in Python 3.x - the int type is now of unlimited length by default. You don’t have to specify what type of variable you want; Python does that automatically.
* Int: The basic integer type in python, equivalent to the hardware 'c long' for the platform you are using in Python 2.x, unlimited in length in Python 3.x.
* Long: Integer type with unlimited length. In python 2.2 and later, Ints are automatically turned into long ints when they overflow. Dropped since Python 3.0, use int type instead.
* Float: This is a binary floating point number. Longs and Ints are automatically converted to floats when a float is used in an expression, and with the true-division / operator. In CPython, floats are usually implemented using the C languages double, which often yields 52 bits of significand, 11 bits of exponent, and 1 sign bit, but this is machine dependent.
* Complex: This is a complex number consisting of two floats. Complex literals are written as a + bj where a and b are floating-point numbers denoting the real and imaginary parts respectively.
In general, the number types are automatically 'up cast' in the following order:
Int → Long → Float → Complex. The farther to the right you go, the higher the precedence.
The result of divisions is somewhat confusing. In Python 2.x, using the / operator on two integers will return another integer, using floor division. For example, 5/2 will give you 2. At least one of the operands must be float to get true division, e.g. 5/2. or 5./2 (the dot makes a number a float) will yield 2.5. Starting with Python 2.2 this behavior can be changed to true division by the future division statement from __future__ import division. In Python 3.x, the result of using the / operator is always true division (you can ask for floor division explicitly by using the // operator since Python 2.2).
This illustrates the behavior of the / operator in Python 2.2+:
For operations on numbers, see chapters Basic Math and Math.

## Links

* 5.4. Numeric Types — int, float, long, complex, docs.python.org
| Previous: Data types | Index | Next: Strings |
# Python Programming/MS Word

Microsoft Word documents of the .docx format can be created or changed using python-docx 3rd party module.
The .doc format can be worked with on Windows using PyWin32 via COM interface, provided Word is installed. An example:

## External links

* python-docx, pypi.python.org
* python-docx, readthedocs.io
# R Programming/Count Data Models

R Programming
* R Basics
** Introduction
** Sample Session
** Manage your workspace
** Settings and Installation
** Documentation
** Control Structures
** Working with functions
** Debugging
** Using C or Fortran
** Utilities
** Estimation utilities
** Packages
* Data Management
** Data types
** Working with data frames
** Importing and exporting data
** Text Processing
** Times and Dates
* Graphics
** Grammar of graphics
* Publication quality output
* Descriptive Statistics
* Mathematics
** Optimization
** Probability Distributions
** Random Number Generation
* Statistical Core Methods
** Maximum Likelihood
** Method of Moments
** Bayesian Methods
** Bootstrap
** Multiple Imputation
** Nonparametric Methods
* Regression Models
** Linear Models
** Quantile Regression
** Binomial Models
** Multinomial Models
** Tobit And Selection Models
** Count Data Models
** Duration Analysis
* Time Series
* Factor Analysis
* Classification
** Ordination
** Clustering
* Network Analysis
* High Performance Computing
** Profiling R code
** Parallel computing with R
* Appendix
** Sources
** Index

## The Poisson model
### Fake data simulations

We assume that y follows a poisson distribution with mean exp(1 + 1 * x). We store the data in the "df" dataframe.

### Maximum likelihood

We estimate this simple model using the glm function with family = poisson as option.

### Bayesian estimation

The model can also be estimated using bayesian methods with the MCMCpoisson function which is provided in the MCMCpack.

## Overdispersion test

* dispersiontest (AER package) provides a test for equidispersion.

## Zero inflated model

See the zic package

## Bivariate poisson regression

* bivpois package for bivariate poisson regression.

## References

* See UCLA website for an example
* Zeileis, A., Kleiber, C. and Jackman, S. Regression Models for Count Data in R
* Replication files for Cameron and Trivedi's 1998 book are provided in the AER package. You can simply type ?CameronTrivedi1998 and you will find the source code.
* ↑ Markus Jochmann (2010). zic: Bayesian Inference for Zero-Inflated Count Models. R package version 0.5-3. http://CRAN.R-project.org/package=zic
* ↑ Cameron, A.C. and Trivedi, P.K. (1998). Regression Analysis of Count Data. Cambridge: Cambridge University Press.
* ↑ Christian Kleiber and Achim Zeileis (2008). Applied Econometrics with R. New York: Springer-Verlag. ISBN 978-0-387-77316-2. URL http://CRAN.R-project.org/package=AER
# R Programming/Advanced programming

R Programming
* R Basics
** Introduction
** Sample Session
** Manage your workspace
** Settings and Installation
** Documentation
** Control Structures
** Working with functions
** Debugging
** Using C or Fortran
** Utilities
** Estimation utilities
** Packages
* Data Management
** Data types
** Working with data frames
** Importing and exporting data
** Text Processing
** Times and Dates
* Graphics
** Grammar of graphics
* Publication quality output
* Descriptive Statistics
* Mathematics
** Optimization
** Probability Distributions
** Random Number Generation
* Statistical Core Methods
** Maximum Likelihood
** Method of Moments
** Bayesian Methods
** Bootstrap
** Multiple Imputation
** Nonparametric Methods
* Regression Models
** Linear Models
** Quantile Regression
** Binomial Models
** Multinomial Models
** Tobit And Selection Models
** Count Data Models
** Duration Analysis
* Time Series
* Factor Analysis
* Classification
** Ordination
** Clustering
* Network Analysis
* High Performance Computing
** Profiling R code
** Parallel computing with R
* Appendix
** Sources
** Index

## Conditional execution

* Help for programming :
if accepts a unidimensional condition.
The ifelse command takes as first argument the condition, as second argument the treatment if the condition is true and as third argument the treatment if the condition is false. In that case, the condition can be a vector. For instance we generate a sequence from 1 to 10 and we want to display values which are lower than 5 and greater than 8.

## Loops

R provides three ways to write loops: for, repeat and while. The for statement is excessively simple. You simply have to define index (here k) and a vector (in the example below the vector is 1:5) and you specify the action you want between braces.
When it is not possible to use the for statement, you can also use break or while by specifying a breaking rules. One should be careful with this kind of loops since if the breaking rules is misspecified the loop will never end. In the two examples below the standard normal distribution is drawn in as long as the value is lower than 1. The cat function is used to display the present value on screen.

### Implicit loops

Loops are generally slow and it is better to avoid them when it is possible.
* apply can apply a function to elements of a matrix or an array. This may be the rows of a matrix (1) or the columns (2).
* lapply applies a function to each column of a dataframe and returns a list.
* sapply is similar but the output is simplified. It may be a vector or a matrix depending on the function.
* tapply applies the function for each level of a factor.
* See also aggregate which is similar to tapply but is applied to a dataframe instead of a vector.

### Iterators

* Loops in R are generally slow. iterators may be more efficient than loops. See this entry in the Revolution Computing Blogs
# R Programming/Bayesian Methods

R Programming
* R Basics
** Introduction
** Sample Session
** Manage your workspace
** Settings and Installation
** Documentation
** Control Structures
** Working with functions
** Debugging
** Using C or Fortran
** Utilities
** Estimation utilities
** Packages
* Data Management
** Data types
** Working with data frames
** Importing and exporting data
** Text Processing
** Times and Dates
* Graphics
** Grammar of graphics
* Publication quality output
* Descriptive Statistics
* Mathematics
** Optimization
** Probability Distributions
** Random Number Generation
* Statistical Core Methods
** Maximum Likelihood
** Method of Moments
** Bayesian Methods
** Bootstrap
** Multiple Imputation
** Nonparametric Methods
* Regression Models
** Linear Models
** Quantile Regression
** Binomial Models
** Multinomial Models
** Tobit And Selection Models
** Count Data Models
** Duration Analysis
* Time Series
* Factor Analysis
* Classification
** Ordination
** Clustering
* Network Analysis
* High Performance Computing
** Profiling R code
** Parallel computing with R
* Appendix
** Sources
** Index

## Introduction

R has lots of bayesian estimation procedures, much more than Stata or SAS.
* LearnBayes by Jim Albert
* bayesm by Peter Rossi and Rob McCulloch
* BaM by Jeff Gill
* arm package by Jennifer Hill and Andrew Gelman.
* MCMCpack package.
* mcsm package by Christian Robert and George Casella.
* umacs (link) by Jouni Kerman and Andrew Gelman.

## Interface with WinBugs

* WinBugs/OpenBugs is a popular statistical package for MCMC techniques.
* Andrew Gelman has some instruction to use R and WinBugs on his webpage
* There is also an interface with JAGS

## Resources

* Christian Robert and Jean Michel Marin The Bayesian Core (link including slides and R scripts)
* Jim Albert : Bayesian Computation with R, Use R!, Springer 2007.
* Christian Robert and George Casella Introducing Monte Carlo Methods with R, Use R!, Springer 2009 (link including the mcsm package).
* Peter Rossi, Greg Allenby, Robert McCulloch : Bayesian Statistics and Marketing and "bayesm" package.
* CRAN task view for bayesian statistics (link)