snoyberg/conduit — pull request #122 by Davorak (2013-11-04 20:44:42+00:00)
Title: Added facilities in Data.Conduit.Lift to run local monad transformers in isolated section of Conduit

I wanted to open a discussion on weather it was appropriate to add the following functionality to the conduit package or if it was better off in its own package.

One of the great features provided by Pipes package is the ability to utilize monad transformers in isolated sections of a pipeline. Conduit historically has not had this advantage because it did not quite [follow the monad transformer laws.](https://github.com/snoyberg/conduit/wiki/Dealing-with-monad-transformers)

`Data.Conduit.Lift` gives a method to get around this limitation so that Conduit can utilize monad transformers in isolate sections of conduit much like the `Pipes.Lift` module provides for Pipes. It is also a rather mechanical translation of the work I recently provided for `Pipes.Lift`. The technique seems to be quite general and applies to both Pipes, Conduit and probably other streaming systems as well.

Here is an example from the [link above](https://github.com/snoyberg/conduit/wiki/Dealing-with-monad-transformers) demonstrating the problem with running the state transformer in an isolate section of conduit.

``` haskell
{-# LANGUAGE OverloadedStrings #-}
import Data.Conduit
import qualified Data.Conduit.List as CL
import Control.Monad.State
import Control.Monad.Trans.Class

source :: Source IO ()
source = CL.sourceList $ replicate 10 ()

replaceNum :: Conduit () (StateT Int IO) Int
replaceNum = awaitForever $ \() -> do
i <- lift get
lift $ put $ i + 1
yield i

main :: IO ()
main = do
x <- source $$ transPipe (flip evalStateT 1) replaceNum =$ CL.consume
print x

y <- flip evalStateT 1
$ transPipe lift source $$ replaceNum =$ CL.consume
print y

> main
[1,1,1,1,1,1,1,1,1,1]
[1,2,3,4,5,6,7,8,9,10]
```

`Data.Conduit.Lift` provides evalStateC to handle this evaluating the state transformer locally rather then globally along the entire conduit:

``` haskell

test = do
x <- source3 $$ evalStateC 1 replaceNum =$ CL.consume
print x

> test
[1,2,3,4,5,6,7,8,9,10]

```

This has several advantages:

* Conduit sections that need to eval a state transform can be implemented and used with out having to worry about adding an evalStateT outside the conduit
* Monad transformers can be isolated to only where they are needed. If you need state it no longer needs to be global to the entire conduit.
* You can avoid using a mutable varible in `ReaderT`, which is one of the current recomened work arounds.

`Data.Conduit.Lift` implements functionality for the:
* ErrorT
* MaybeT
* ReaderT
* StateT
* WriterT
* rwsT

This technique will work for any monad transformer you can write a [`MFunctor`](http://hackage.haskell.org/package/mmorph-1.0.0/docs/Control-Monad-Morph.html#t:MFunctor) instance for.

You can read a rough outline of how the technique works [here](https://www.fpcomplete.com/user/davorak/nearly-effortless-monad-transformers-for-pipes-conduit-and-more-1). As I mentioned above this is nearly a mechanical translation from recent work on [`Pipes.Lift`](https://github.com/Gabriel439/Haskell-Pipes-Library/blob/master/src/Pipes/Lift.hs) and some insight may be gain there as well.

Since `Conduit` and `Source` use `()` as the base monad value some monad transforms are a little more limited. For example the EtiherT monad transform has the right value(If I remember correctly) forced to `()`. There may be other gotchas that I have not yet caught, I would be interested in hearing what they are if you find them.

I hope this makes it easier to include your favorite monad transformer in your Conduit code or vs versa.