W.I.P - Scikit tutorials 

Work in progress - Merging the scikit-learn tutorials into the main documentation.

Still lots to be done here. This is just to keep it open to feedback and guidance from the community.

Below follows the original discussion from the #472 pull request.

___________________________________________________

**GaelVaroquaux** 
  **WIP: merge in statistical learning tutorial**
--------------------------------------------------------------------
The goal of this pull request is to merge in the statistical learning tutorial into the main documentation.

The pros would be:

   * To make the tutorial more easily maintainable
   * To be able to link from the tutorial to the docs, and vice-versa
   * To have examples and images shared between the docs and the tutorial
   * To make the tutorial more visible.

The tutorial should be clearly advertise as a tutorial, with a specific focus (using the scikit for inference and feature discovery more than simple prediction). It should not replace the docs and be kept short. There should be room for other tutorials with other focuses. For instance, @ogrisel's tutorial should be merged in later on.

Right now this pull request it not ready at all to be merged. I am putting it up so that people can pitch in, for instance @Balu-Varanasi, or @amueller have expressed interested. There is a lot of work. Specifically (in order of priority and difficulty):

   * The index.rst should be added in the tutorial
   * I do not want any images stored in the git tree: as a result I have removed some images necessary to build to avoid polluting the tree. These should all be generated by the Python code in the examples, following the rest of the scikit documentation
   * I have created a examples/tutorial folder to host these examples, but each one should be moved in the right example folder and this folder should be deleted. Also, redundant examples should be deleted or merged.
   * The tutorial needs to be fully doctested. I believe that this is the case currently, but the doctests take way too long to run, partly because of the model selection part (grid search). This model selection part is important, as it is important to teach it to people, but we'll need to figure out way to make it run faster: simpler data on some of these, and delegating the more complexes ones to examples that get ran only once in a while, and using 'literalinclude' directives to show the code in the docs.

What we be great is if we could iterate on the above todo list and update it as we go. For the sake of simplicity, I can give right permissions to me repo to people working on this.

------------------------------------------------------

*Balu-Varanasi commented*

@GaelVaroquaux ... I would like to work on this !

*GaelVaroquaux commented*

Great. Just try plowing your way through it, and ask for any help when you need it. Gael

*Balu-Varanasi commented*

Thank you...

*mblondel commented*

I got feedback that the main documentation is lacking information for beginners, so +1.

*jaquesgrobler commented*

I'm gonna get going on this.. I'm a tad new to Scikit, so I my call on your assistance at some point if I need an experienced opinion :) @mblondel

*ogrisel commented*

@jaquesgrobler I think it would be great if you could directly make some room for the other tutorial currently hosted at: http://scikit-learn.github.com/scikit-learn-tutorial/ / https://github.com/scikit-learn/scikit-learn-tutorial

The goal would be to have a "tutorials/" container with a "index.rst" file that gives an overview of the two tutorials and their target audience: one (Gael's) would be for people with a Scientific Computing background (they already have their own data in numerical form and are mostly aware of their statistical properties) the other for a more general audience with a larger focus on defining what a feature is, how to deal with categorical feature and how to extract features from text.

Both tutorials should be self contained with cross-refences to the narrative doc, reference doc and examples (and soon the FAQ).

I can deal with the refactoring of my part but it would be great to plan the placeholder structure in advance to make it easier to mere later.

*jaquesgrobler commented*

@ogrisel That sounds like a good idea.. I'll have a look at it later today and I'll get also get Gael's inputs on it. I'll be in touch. All the best.J

*GaelVaroquaux commented*

I agree what @ogrisel 's comment. As he said, you should leave a placeholder structure. One option would be to have a section 'Tutorials', and a few sub-sections below it, one of them would be Olivier's tutorial, and the other mine.

*jaquesgrobler commented*

Okay, I'll get going on this. Thanks for the inputs

*jaquesgrobler commented*

@ogrisel @GaelVaroquaux I've had a quick look at the Oliver's scikit tutorial. Just to be sure on the main desire here:
Basically, neither of these two tutorials are in the documentation yet, and we want a main 'Tutorials' section that will refer to them as mentioned above (obviously along the above suggested specifications).
Do I understand this correctly?

Just one other thing - Section 2 in the User Guide, titled 'Getting started: an introduction to machine learning with scikit-learn', is also like a mini-tutorial (it's .rst file is called 'tutorial' as well). Perhaps it would be good to within this chapter, once the 'tutorials' section is complete, also give a link to it there or something of this effect
Seeing as the Section 2 'tutorial' is more like an introduction tot he machine learning vocabulary and a simple learning example, it may be good to then mention that there are tutorials available too, once the basic idea has been discussed in this section (2). Just a thought

*GaelVaroquaux commented*

> @ogrisel @GaelVaroquaux I've had a quick look at the Oliver's scikit tutorial. Just to be sure on the main desire here: > Basically, neither of these two tutorials are in the documentation yet, and we want a main 'Tutorials' section that will refer to them as mentioned above (obviously along the above suggested specifications). > Do I understand this correctly?

Yes.

> Just one other thing - Section 2 in the User Guide, titled 'Getting started: an introduction to machine learning with scikit-learn', is also like a mini-tutorial (it's .rst file is called 'tutorial' as well). Perhaps it would be good to within this chapter, once the 'tutorials' section is complete, also give a link to it there or something of this effect.. or something to that effect. > Seeing as the Section 2 'tutorial' is more like an introduction tot he machine learning vocabulary and a simple learning example, it may be good to then mention that there are tutorials available too. Just a thought

Actually, we could break the section 2 into 3 subsections, the first one being 'A quick introduction', that would include the current content of section 2, and the 2 others being the tutorials that we are discussing. How does that sound? G

*jaquesgrobler commented*

Sounds good to me.. Perhaps, assuming that section 3 will be yours( @GaelVaroquaux ) , one could just start it with a note saying that this subsection is more suited to people with Scientific Computing background, and that people that are new to the field can leave it until later.. Or would that be unnecessary?

*GaelVaroquaux commented*

> Sounds good to me.. Perhaps, assuming that section 3 will be yours( @GaelVaroquaux ) , one could just start it with a note saying that this subsection is more suited to people with Scientific Computing background, and that people that are new to the field can leave it until later.. Or would that be unnecessary?

A note is useful. In general, every section of a documentation should tell the reader why they want to read, or not to read it. The exact content of the note can be discussed. I would say that it should rather say that the tutorial is targeted toward people wanting to do scientific inference, i.e. reach conclusions on their data, using the scikit-learn. G

*jaquesgrobler commented*

noted.. thank you :)

*ogrisel commented*

I agree. Also we should cross link the tutorials in their intro and make it explicit which audience is target there as well (if not already the case).

*GaelVaroquaux commented*

> Also we should cross link the tutorials in their intro and make it > explicit which audience is target there as well (if not already the > case).

Yes, this is very important