Showing posts with label monad. Show all posts
Showing posts with label monad. Show all posts

Friday, November 30, 2012

Arrows and Kleisli in C++

Control.Arrow shows an odd-but-interesting part of Haskell. Arrows are functions; composable and callable. Arrows and Monads sometimes seem to be referred to as alternatives to each other. Perhaps now would be a good time to relate some category theory.

From A to B.

A category theorist might view functions as a transformation from one type to another. For example,

    std::to_string : X -> std::string

would mean that to_string is a function that maps an X to a string.

In "Generic Function Objects", I talked about type constructors.

/* MakeT T : X -> T<X> */
template< template<class...> class T > struct MakeT {
    template< class ...X, class R = T< typename std::decay<X>::type... > >
    constexpr R operator () ( X&& ...x ) {
        return R( std::forward<X>(x)... );
    }
};

/* pair : X x Y -> std::pair<X,Y> */
constexpr auto pair = MakeT<std::pair>();

I use the notation X x Y to mean that MakeT<pair> takes two arguments. Though, in Haskell, it would actually look like this:

    make_pair :: X -> Y -> (X,Y)

Haskell uses curried notation.

    g : pair<X,A> -> pair<X,B>

Here, g is a function that transforms the second element of the pair to a B, but leaves the first as an X. Although this cannot be inferred from the definition, let's assume that the first value (X) is not transformed in any way. There is a function that represents this non-transformation.

/* id : X -> X */
constexpr struct Id {
    template< class X >
    constexpr X operator () ( X&& x ) {
        return std::forward<X>(x);
    }
} id{};

Arrows provide the tools to take a normal function, f : A -> B, and convert it to a function like g. This is sort of like composition

/* compose : (B -> C) x (A -> B) -> (A -> C) */
template< class F, class G >
struct Composition
{
    F f; G g;

    template< class _F, class _G >
    constexpr Composition( _F&& f, _G&& g ) 
        : f(std::forward<_F>(f)), g(std::forward<_G>(g)) { }

    template< class X >
    constexpr auto operator() ( X&& x) 
        -> decltype( f(g(std::declval<X>())) )
    {
        return f( g( std::forward<X>(x) ) );
    }
};

constexpr auto compose = MakeT<Composition>();

If we look at A -> B as a type itself, say AB, and B -> C as a type, BC, then it is clear that composition is BC x AB -> AC. Functions themselves are values with types and composition creates a new value and type. We can think of them as being on a geometric plain with the points A, B, and C. Functions are connections from point-to-point. Similarly, we can think of AB, BC, and AC as points on a different plain with arrows connecting AB and BC to AC (though AB and BC cannot be connected).

 

Functions as Arrows.

Arrows have several operations. The first is arr, which transforms a function to an arrow; but since functions are arrows, it isn't useful, right off the bat. first and second take functions, A -> B, and lift them to pair-oriented functions. fan takes two functions, one A -> B, another A -> C, and returns a function to pair<B,C>. split takes two functions as well, A -> B and X -> Y, and returns a function pair<A,X> -> pair<B,Y>.

    arr : (A -> B) -> (A -> B)
    first : (A -> B) -> (pair<A,X> -> pair<B,X>)
    second : (A -> B) -> (pair<X,A> -> pair<X,B>)
    split : (A -> B) x (X -> Y) -> (pair<A,X> -> pair<B,Y>)
    fan : (A -> B) x (A -> C) -> (A -> pair<B,C>)

First, we fill in the declarations.

template< class A, class F, class Arr = Arrow< Cat<A> > >
constexpr auto arr( F&& f ) ->  decltype( Arr::arr( std::declval<F>() ) )
{
    return Arr::arr( std::forward<F>(f) );
}

template< class A > struct Arr {
    template< class F >
    constexpr auto operator () ( F&& f ) -> decltype( arr(std::declval<F>()) )
    {
        return arr( std::forward<F>(f) );
    }
};

constexpr struct Split {
    template< class F, class G, class A = Arrow<Cat<F>> >
    constexpr auto operator () ( F&& f, G&& g )
        -> decltype( A::split(std::declval<F>(), std::declval<G>()) )
    {
        return A::split( std::forward<F>(f), std::forward<G>(g) );
    }
} split{};

constexpr struct Fan {
    template< class F, class G, class A = Arrow<Cat<F>> >
    constexpr auto operator () ( F&& f, G&& g )
        -> decltype( A::fan(std::declval<F>(),std::declval<G>()) )
    {
        return A::fan( std::forward<F>(f), std::forward<G>(g) );
    }
} fan{};

constexpr struct First {
    template< class F, class A = Arrow<Cat<F>> >
    constexpr auto operator () ( F&& f ) 
        -> decltype( A::first(std::declval<F>()) ) 
    {
        return A::first( std::forward<F>(f) );
    }
} first{};

constexpr struct Second {
    template< class F, class A = Arrow<Cat<F>> >
    constexpr auto operator () ( F&& f ) -> decltype( A::second(std::declval<F>()) ) {
        return A::second( std::forward<F>(f) );
    }
} second{};

arr will be trivial to implement, but the others are tricky. Luckily, we can define it mostly in terms of split--it looks like this:

/* pairCompose : (A -> B) x (X -> Y) -> (pair<A,X> -> pair<B,Y>) */
template< class F, class G > struct PairComposition {
    F f; G g;

    template< class _F, class _G >
    constexpr PairComposition( _F&& f, _G&& g )
        : f(std::forward<_F>(f)), g(std::forward<_G>(g))
    {
    }

    template< class P/*air*/ >
    constexpr auto operator() ( const P& p ) 
        -> decltype( std::make_pair( f(std::get<0>(p)), g(std::get<1>(p)) ) )
    {
        return std::make_pair( f(std::get<0>(p)), g(std::get<1>(p)) );
    }
};

constexpr auto pairCompose = MakeT<PairComposition>();

pairCompose returns a function expecting a pair and returns a pair, threading the first value through f and the second through g. We can compose PairCompositions.

namespace std {
std::string to_string( const std::string& s );
template< class X, class Y >
std::string to_string( const std::pair<X,Y>& p );

std::string to_string( const std::string& s ) {
    return "\"" + s + "\"";
}

template< class X, class Y >
std::string to_string( const std::pair<X,Y>& p ) {
    return "(" + to_string(p.first) + "," + to_string(p.second) + ")";
}
}

constexpr struct ToString {
    template< class X >
    std::string operator () ( const X& x ) const {
        return std::to_string(x);
    }
} string{};

int main() {
    using std::cin;
    using std::cout;
    using std::endl;

    auto plus2 = []( int x ){ return x+2; };

    std::pair<int,int> p( 1, 1 );

    cout << "((+2) . string, (+4))( 1, 1 ) = " 
         << compose( pairCompose(string,plus2), 
                     pairCompose(plus2, plus2) )(p) << endl;
}

This will output ("3",5). It's easiest to look at this as p.first and p.second being on two separate paths. I have written it such that the individual paths are verticle. The first path starts at 1, and goes to plus2(1), to show(plus2(1)). The second path starts at 2 and ends at plus2(plus2(2)). The odd part is that we're defining both paths at once.

 Observe that first(f) = split(f,id). Proof?

    first : (A -> B) -> (pair<A,X> -> pair<B,X>)
    split : (A -> B) x (X -> Y) -> (pair<A,X> -> pair<B,Y>)  
    id : X -> X
    f : A -> B 
    split(f,id) : pair<A,X> -> pair<B,X>

Since we know first(f) = split(f,id), we can intuit that second(f) = split(id,f) and also, split(f,g) = compose( first(f), second(g) ).

fan represents a fork in the road. One variable gets fed into two functions, the results of which get zipped into a pair. duplicate will do the splitting, but we'll rely on split to implement fan.

constexpr struct Duplicate {
    template< class X, class P = std::pair<X,X> > 
    constexpr P operator() ( const X& x ) {
        return P( x, x );
    }
} duplicate{};


With this, we can say that fan(f,g)(x) = split(f,g)( duplicate(x) ). Since we know what everything looks like, we can define Arrow<F>.

template< class Func > struct Arrow<Func> {
    template< class F >
    static constexpr F arr( F&& f ) { return std::forward<F>(f); }

    /* split(f,g)(x,y) = { f(x), g(y) } */
    template< class F, class G >
    static constexpr auto split( F f, G g ) 
        -> PairComposition<F,G>
    {
        return pairCompose( std::move(f), std::move(g) );
    }

    /*
     * first(f)(x,y)  = { f(x), y }
     * second(f)(x,y) = { x, f(y) }
     */
     
    template< class F >
    static constexpr auto first( F f ) 
        -> decltype( split(std::move(f),id) )
    {
        return split( std::move(f), id );
    }

    template< class F >
    static constexpr auto second( F f ) 
        -> decltype( split(id,std::move(f)) )
    {
        return split( id, std::move(f) );
    }

    /* fan(f,g)(x) = { f(x), g(x) } */
    template< class F, class G >
    static constexpr auto fan( F f, G g ) 
        -> decltype( compose( split(std::move(f),std::move(g)), duplicate ) )
    {
        return compose( split(std::move(f),std::move(g)), duplicate );
    }
};

Now, we can rewrite the above example:

int main() {
    auto plus2 = []( int x ){ return x+2; };
 
     cout << "(+2) *** (+2) >>> string *** (+2) $ 1 = " 
         << compose( split(string, plus2), 
                     fan(  plus2,  plus2) )(1) << endl;
}

One way to think of Arrows is as paths. fan represents a fork in the road, split defines two separate, but parallel, paths at once, and first and second allow progress on one of the paths. For an arbitrary example, consider a branching path that hides some treasure. What sequence of moves are required to recover it?

/* 
 * Get 0 : (X,Y) -> X
 * Get 1 : (X,Y) -> Y
 */
template< size_t N > struct Get {
    template< class P >
    constexpr auto operator () ( P&& p )
        -> decltype( std::get<N>(std::declval<P>()) )
    {
        return std::get<N>( std::forward<P>(p) );
    }
};

int main() {
    constexpr auto fst = Get<0>();
    constexpr auto snd = Get<1>();
    constexpr auto oneHundred = []( int x ){ return 100; };
    
    // Hide the hundred.
    // hidden = pair( pair( 0, pair(100,0) ), 0 )
    auto hidden = fan( fan(id,fan(oneHundred,id)), id )( 0 );
    // Function to find it again.
    auto find = compose( fst, compose(snd,fst) );
    
    cout << "I found " << find(hidden) << "!" << endl;
}


Enter Kleisli.

 This all works great for free functions, but some functions look like this:

    f : X -> M<Y> -- Where M is some Monad

 This f is a member of the Kleisli category. It accepts a normal value and returns a Monadic one. Examples of Kelisli functions (psuedocode):

    Just : X -> unique_ptr<X>
    Just = MakeT<unique_ptr>();

    Seq : X -> std::vector<X>
    Seq(x) = std::vector<X>{x} 

    mreturn<M> : X -> M<X>

unique_ptrs and vectors are monads, so any function that produces one from some x can be considered in the Kleisli category. Though, to the compiler, there's no logic to that, so we define a wrapper type around the function. This is a fairly common pattern, so I define a base class for it.

template< class F > struct Forwarder {
    using function = typename std::decay<F>::type;
    function f = F();

    template< class ...G >
    constexpr Forwarder( G&& ...g ) : f( std::forward<G>(g)... ) { }

    template< class ...X >
    constexpr auto operator() ( X&& ...x )
        -> decltype( f(std::declval<X>()...) )
    {
        return f( std::forward<X>(x)... );
    }

    constexpr operator function () { return f; }
};

The Kleisli itself can be implemented two ways. The Haskell way would be something like this:

/* Kleisli M A B : A -> M<B> */
template< template<class...> class M, class A, class B,
          class F = std::function<M<A>(B)> >
struct Kleisli : Forwarder<F> {
    template< class ...G >
    constexpr Kleisli( G&& ...g ) : Forwarder<F>(std::forward<G>(g)...) { }
};

This is faithful to how Haskell defines it.

    newtype Kleisli m a b = Kleisli { runKleisli :: a -> m b }

But just think about this for a moment. A Kleisli is an Arrow, right? What would Arrow<Kleisli>::first return?

    first : (A -> B) -> (pair<A,X> -> pair<B,X>)
    Kleisli f = A -> M<B>
    first(Kleisli f) : (A -> M B) -> (pair<A,X> -> M pair<B,X>)

What's the type of X? It's truly impossible to know because it depends on what gets passed in, which is what the notation above means.

Is it impossible to define Kleisli in this way? I don't know. I attempted to specialize its composition function for when A or B were pair types, but there are four combinations of whether A or B or both is a pair. I tried assigning X the type of std::placeholders::_1, but none of my attempts to make it really work compiled. (It was horrible.)

But we don't have any of that trouble if we define Kleisli differently.


Kleisli<F>.

/* Kleisli M F : F -- where F : A -> M<B> */ 
template< template<class...> class M, class F = Id >
struct Kleisli : Forwarder<F> {
    template< class ...G >
    constexpr Kleisli( G&& ...g ) : Forwarder<F>(std::forward<G>(g)...) { }
};

An implicit requirement of arrows is that they can be composed, but Kleislies?

    compseKleisli : Kleisli(B -> M<C>) x Kleisli(A -> M<B>) -> (...?)

We can reasonably assume that it should be Kleisli(A -> M<C>), but our naive definition of composition must be specialized. Literally.

/* Composition : Kleisli(B -> M<C>) x Kleisli(A -> M<B>) -> (A -> M<C>) */
template< template<class...> class M, class F, class G >
struct Composition<Kleisli<M,F>,Kleisli<M,G>>
{
    Kleisli<M,F> f; 
    Kleisli<M,G> g;

    template< class _F, class _G >
    constexpr Composition( _F&& f, _G&& g ) 
        : f(std::forward<_F>(f)), g(std::forward<_G>(g)) { }

    template< class X >
    constexpr auto operator() ( X&& x ) 
     -> decltype( g(std::forward<X>(x)) >>= f )
    {
        return g(std::forward<X>(x)) >>= f;
    }
};

/* kcompose : Kleisli(B -> M<C>) x Kleisli(A -> M<B>) -> Kleisli(A -> M<C>) */
constexpr struct KCompose {
    template< template<class...> class M, class F, class G >
    constexpr auto operator () ( Kleisli<M,F> f, Kleisli<M,G> g )
        -> Kleisli< M, Composition<Kleisli<M,F>,Kleisli<M,G> >
    {
        return kleisli<M> ( 
            compose( std::move(f), std::move(g) )
        );
    }
} kcompose{};
 
int main() {
    auto stars = kleisli<std::basic_string> (
        [] (char c) -> std::string { 
            return c == '*' ? "++" :
                   c == '+' ? "* *" : std::string{c};
        }
    );
    
    auto starsSqr = kcompose( stars, stars );
    
    auto starsCube = kcompose( starsSqr, stars );
    
    cout << "stars of '*' : " << stars('*') << endl;
    cout << "stars^2 of '*' : " << starsSqr('*') << endl;
    cout << "stars^3 of '*' : " << starsCube('*') << endl;
}

This outputs  

    stars of '*' : "++"
    stars^2 of '*' : "* ** *"
    stars^3 of '*' : "++ ++++ ++"


Like before, it would be most convenient to define Arrow<Kleisli> in terms of split. split(f,g), given the pair {x,y}, will have to pass x into f and y into g, both of which will return Monadic values. Finally, a pair will have to be constructed from the values extracted from f(x) and g(y).

    split: Kleisli (A -> M B) x Kleisli (X -> M Y) -> Kleisli (pair<A,X> -> M pair<B,Y>)

To extract the values from f(x) and g(y), we need to call mbind on each, which in Haskell might look like this:

    f(x) >>= (\x' -> g(y) >>= (\y' -> return (x,y)))

Or, Control.Monad defines liftM.

    liftM2 (,) f(x) g(y) -- where (,) is equivalent to make_pair

template< class M > struct Return {
    template< class X >
    constexpr auto operator () ( X&& x ) 
        -> decltype( mreturn<M>(std::declval<X>()) )
    {
        return mreturn<M>( std::forward<X>(x) );
    }
}; 
 
/*
 * liftM : (A -> B) x M<A> -> M<B>
 * liftM : (A x B -> C) x M<A> x M<B> -> M<C>
 */
constexpr struct LiftM {
    template< class F, class M, class R = Return<typename std::decay<M>::type> >
    constexpr auto operator () ( F&& f, M&& m )
        -> decltype( std::declval<M>() >>= compose(R(),std::declval<F>()) )
    {
        return std::forward<M>(m) >>= compose( R(), std::forward<F>(f) );
    }

    template< class F, class A, class B >
    constexpr auto operator () ( F&& f, A&& a, B&& b )
        -> decltype( std::declval<A>() >>= compose (
                rcloset( LiftM(), std::declval<B>() ),
                closet(closet,std::declval<F>())
            ) )
    {
        return std::forward<A>(a) >>= compose (
            rcloset( LiftM(), std::forward<B>(b) ),
            closet(closet,std::forward<F>(f))
        );
    }
} liftM{};

It acts basically as a n-ary mbind, though one could also define an n-ary mbind! liftM works even if you don't.

Finally, we have all the pieces in place to implement KleisliSplit.

/* kleisliSplit : Kleisli(A -> M<B>) x Kleisli(X -> M<Y>) -> (piar<A,X> -> M<pair<B,Y>>) */ 
template< template<class...> class M, class F, class G >
struct KleisliSplit {
    F f;
    G g;

    constexpr KleisliSplit( F f, G g ) : f(std::move(f)), g(std::move(g)) { }

    template< class X, class Y >
    constexpr auto operator () ( const std::pair<X,Y>& p )
        -> decltype( liftM(pair,f(std::get<0>(p)),g(std::get<1>(p))) )
    {
        return liftM (
            pair, 
            f( std::get<0>(p) ), 
            g( std::get<1>(p) )
        );
    }
};

The final note before moving on: arr. Kleisli's arr is like a Monad's mreturn.

    arr : (A -> B) -> Kleisli(A -> M<B>)
    arr(f)(x) = mreturn<M>( f(x) )

or:

     arr = liftM

template< template<class...> class M, class F >
struct Arrow< Kleisli<M,F> > {

    template< class G >
    using K = Kleisli<M,G>;

    template< class G >
    static constexpr auto arr( G g ) -> Kleisli< M, Part<LiftM,G> > {
        return kleisli<M>( closet(liftM,std::move(g)) );
    }

    template< class G >
    static constexpr auto first( G g ) 
        -> K<decltype( ::split(std::move(g),arr(id)) )> 
    {
     // id is not a Kleisli. 
     // The call to arr refers to the arr above, not ::arr.
     // arr(id) : Kleisli(X -> M<X>)
        return ::split( std::move(g), arr(id) );
    }

    template< class G >
    static constexpr auto second( G g) 
        -> K<decltype( ::split(arr(id),std::move(g)) )>
    {
        return ::split( arr(id), std::move(g) );
    }

    template< class G >
    static constexpr auto split( Kleisli<M,F> f, Kleisli<M,G> g )
        -> K< KleisliSplit<M,F,G> >
    {
        return KleisliSplit<M,F,G>( std::move(f.f), std::move(g.f) );
    }

    template< class _F, class _G >
    static constexpr auto fan( _F&& f, _G&& g )
        -> decltype( kcompose(split(std::declval<_F>(),std::declval<_G>()),arr(duplicate)) )
    {
        return kcompose( split(std::forward<_F>(f),std::forward<_G>(g)), arr(duplicate) );
    }
};
 
int main() {
    auto stars = kleisli<std::vector> (
        [] (char c) { 
            return c == '*' ? std::vector<char>{'+','+'} :
                   c == '+' ? std::vector<char>{'*',' ','*'} : std::vector<char>{c};
        }
    );
    
    auto hairs = kleisli<std::vector> (
        [] (char c) -> std::vector<char> { 
            return c == '*' ? std::vector<char>{'\'','"','\''} :
                   c == '+' ? std::vector<char>{'"',' ','"'} :
                   c == '"' ? std::vector<char>{'\''} :
                   c == '\'' ? std::vector<char>{} :
                   std::vector<char>{c};
        }
    );

    cout << "hairs of '*' : " << hairs('*') << endl;
    cout << "hairs^2 of '*' : " << compose(hairs,hairs)('*') << endl;
    
    cout << "split(stars,hairs) (*,*) = " << split(stars,hairs)(pair('*','*')) << endl;
    cout << "fan(stars,hairs)    (*)  = " << fan(stars,hairs)('*') << endl;
    cout << "fan(hairs,stars)    (*)  = " << fan(hairs,stars)('*') << endl;
    cout << "split(hairs,stars) . fan(stars,hairs) = " 
      << compose( split(hairs,stars), fan(stars,hairs) )('*') << endl;
}
 
Finally, this outputs

    hairs of '*' : [',",']
    hairs^2 of '*' : [']
    split(stars,hairs) (*,*) = [(+,'),(+,"),(+,'),(+,'),(+,"),(+,')]
    fan(stars,hairs)    (*)  = [(+,'),(+,"),(+,'),(+,'),(+,"),(+,')]
    fan(hairs,stars)    (*)  = [(',+),(',+),(",+),(",+),(',+),(',+)]
    split(hairs,stars) . fan(stars,hairs) = [(",'),( ,'),(",'),(","),( ,"),(","),(",'),( ,'),(",'),(",'),( ,'),(",'),(","),( ,"),(","),(",'),( ,'),(",')]




Extras -- Convenience and Pitfalls.

Perhaps the most important parts of Control.Arrow are here, but there are still a few things that can be added. For example, compose is backwards! compose(f,g) means "do g, then do f." Because of this we have to read it backwards. fomp is useful, if only because we read left-to-right, not the other way around.

/* fcomp : (A -> B) x (B -> C) -> (A -> C) */
constexpr struct FComp {
    template< class G, class F, class C = Composition<F,G> >
    constexpr C operator () ( G g, F f ) {
        return C(std::move(f),std::move(g));
    }
} fcomp{};

Another tool is precompose and postcompose. Perhaps one wants to compose something that is not a Kleisli with something that is.

/* prefcomp : (A -> B) x (Arrow X Y) -> Arrow A Y */ 
constexpr struct PreFComp {
    template< class F, class A >
    constexpr auto operator () ( F&& f, A&& a )
        -> decltype( arr<A>(declval<F>()) > declval<A>() )
    {
        return arr<A>(forward<F>(f)) > forward<A>(a);
    }
} prefcomp{};

/* postfcomp : Arrow A B x (X -> Y) -> Arrow A Y */ 
constexpr struct PostFComp {
    template< class A, class F >
    constexpr auto operator () ( A&& a, F&& f )
        -> decltype( declval<A>() > arr<A>(declval<F>()) )
    {
        return forward<A>(a) > arr<A>(forward<F>(f));
    }
} postfcomp{};

Beyond that, there's ArrowZero and ArrowPlus to work with Monoids, ArrowChoice based on Either, ArrowApply and ArrowLoop.

ArrowZero, defining zeroArrow, shows the downside to implementing Kleisli as K<M,F> instead of K<M,A,B>. It is defined like so:

    zeroArrow = Kleisli (\_ -> mzero)

The type mzero needs to return is M<B>. Its argument, the _, will be of type A. So we have this problem where zeroArrow(k) (for some Kleisli, k) doesn't know what type to return and can't deduce it!

If we had implemented Kleisli with A and B, this would be no problem, but then it would have been impossible(?) to implement Arrow<Kleisli>::first. In Haskell, all functions carry around information about their domain and codomain, so there is no difference between passing around A and B or just F.

The easiest solution is to diverge from Haskell's definition and require zeroArrow to take an explicit result-type parameter.


Conclusions.

Arrows let us do interesting types of composition that one would not normally likely think of. They fall somewhere between Functors and Monads (this article has a graph). My Monadic parser could have also been written with Arrows.

I realize this has been a very long post. Hopefully all the code examples work, there are no inaccuracies, and it is understandable, but with an article this large, something has probably been fudged. Due to the large amount of material, it may be a good idea to split this article up and go more in depth. Please speak up if something seems off.


Source code: https://gist.github.com/4176121
Haskell reference: Control.Arrow

Monday, November 19, 2012

Monadic Parsing in C++

When researching parsing in Haskell, I always find this pdf: eprints.nottingham.ac.uk/223/1/pearl.pdf

I will be referring back to this paper and encourage my readers to at least skim it as well. It may provide more understanding in how it works.

It describes a simple, yet hard to comprehend functional parser. I have attempted to translate this to C++, but first I wrote a more intuitive, less functional one. Just a simple calculator, somewhat close to what's described in the pdf, and it worked like this:

The first stage took the input string and created a list of token type-lexeme pairs. Given the input "1+2*2", it would spit back {(INT,"1"),(OP,"+"),...}. The next stage uses two mutually recursive functions to represent two parse states: that of sums and subtractions and that of numbers, multiplications and divisions. Since addition has the lowest precedence in this mini-language, it's safe to parenthesize the rest of the expression, meaning that if it parsed "1+", it will expect the next number to be a number, and it might want to add one to it, eagerly, but not if fallowed by a "2*2" since multiplication has higher precedence.

These functions convert the vector of tokens to an expression tree that evaluates the arguments."2*2" would parse to a tree with a multiplier at its root and two numbers as its leaves. So, after the first state (sums) had read "1+", it would eagerly construct an addition object with the left argument being 1, and the right argument being the result of the rest of the parse! So it reads "2*2" and builds a tree that becomes the right-side argument for our "1+" tree.

This solution is theoretically consistent with building a language using flex and bison. The source can be found here (gist). But it isn't functional. That's when I stumbled back onto this research paper and decided to give a real go at it.


Functional Parsing.

The parser described in this paper does not act in the same way as my parser does. Rather than lexing, tokenizing, and building an expression tree, it cuts straight to the point. A Parser<int> will be a function that accepts a string and produces ints. But, the job may not be done; there may be more to parse. For every int it parses, it will also store the suffix of the parse. So, if p is some parser of type Parse<int>, p("1abc") will return a vector holding one match: the value, 1, and the suffix, "abc".

What does this class look like?

/* A Parser is a function taking a string and returning a vector of matches. */
template< class X > struct Parser {
    // The value is the target of the parse. (For example "1" may parse to int(1).
    using value_type    = X;

    // A match consists of a value and the rest of the input to process.
    using parse_state   = std::pair<X,std::string>;

    // A parse results in a list of matches.
    using result_type   = std::vector< std::pair<X,std::string> >;

    // A parser is a function that produces matches.
    using function_type = std::function< result_type( const std::string& ) >;

    function_type f;

    Parser( function_type f ) : f(std::move(f)) { }

    Parser( const Parser<X>& p ) : f(p.f) { }
    Parser() { }

    result_type operator () ( const std::string& s ) const {
        return f( s );
    }
};

template< class X, class F > Parser<X> parser( F f ) {
    using G = typename Parser<X>::function_type;
    return Parser<X>( G(std::move(f)) );
}

I have mentioned in previous articles that one should avoid std::function for efficiency reasons, but it vastly simplifies things here. As you can see, Parser merely wraps itself around std::function. I would encourage the reader to think of it as a type alias--not deserving of being called a new type, but more than a typedef.

This is consistent with how the paper defines this type:

    newtype Parser a = Parser (String -> [(a,String)])

If nothing can be parsed, and empty list will be returned. If many things are parsed, a list of each match will be returned.


The Parser Monad.

As a reminder, the basic monadic operations are these:

    a >> b = b -- Do a, then b.
    a >>= f = b -- For each x from a, construct b with f(x).
    mreturn x = a -- Construct a monad from a value.

How does this relate to parsing? A parser is basically just a function, so if p and q are both parsers, p >> q must return a new parser, a new function. The simple explanation (do p, then q) is correct. First, p parses and let's say it returns a match, (x,rest). rest is sent to q for parsing and the x is thrown away. It may sound odd to just throw away a value, but it will become more obvious soon.

If p had failed to parse a value, then q would not have been run.

The bind operation, p >>= f, extracts the parsed value, x, from p and creates a new parser from f(x). mreturn x creates a new parser that returns x as its value. It accepts any string, even an empty one. Ideally, x came from the output of another parser.

    p >> q -- Run parser p, then q.
    p >>= f -- Construct a parser with p's matches.
    mreturn x -- Construct a parser that returns x.

We can define it like so:

template< class X > struct Monad< Parser<X> > {
    using Pair = typename Parser<X>::parse_state;
    using R    = typename Parser<X>::result_type;

    /* 
     * mreturn x = parser(s){ vector( pair(x,s) ) }
     * Return a parsed value. Forwards rest of input to the next parser.
     */
    template< class M >
    static M mreturn( X x ) {
        return parser<typename M::value_type> (
            [x]( const std::string& s ) { 
                return R{ Pair(std::move(x),s) }; 
            }
        );
    }
    
    /* a >> b = b */
    template< class Y, class Z >
    template< class Y, class Z >
    static Parser<Y> mdo( Parser<Z> a, Parser<Y> b ) {
        return a >>= [b]( const Z& z ) { return b; };
    }

    /* Continue parsing from p into f. */
    template< class F, class Y = typename std::result_of<F(X)>::type >
    static Y mbind( F f, Parser<X> p ) 
    {
        using Z = typename Y::value_type;
        return parser<Z> (
                [f,p]( const std::string& s ) {
                    // First, extract p's matches.
                    return p(s) >>= [&]( Pair p ) {
                        // Then construct the new parser from the p's output.
                        // Continue parsing with the remaining input with the new parser.
                        return f( std::move(p.first) )( std::move(p.second) );
                    };
                }
        );
    }
};

Do not worry if this source is difficult to understand. It is more important to understand how it is used (which is perhaps common with monads). Note that mdo is defined such that for every successful parse of a, b is parsed. If a fails to parse anything, b fails, too.

These monadic operations are the core building blocks from which we can build more complex system, but the paper also discusses MonadZero and MonadPlus. They are both type classes, like Monad, but extend it to do a few interesting things. In C++, one can concatenate two string by using simple addition: s1 + s2 = s12. MonadPlus is generalization of this. MonadZero completes this generalization by supplying the additive identity. For example, we know that zero + x = x. Thus, "" + s = s.

In parsing terms, zero would refer to a parser that matches nothing and adding two parsers, p+q, will produce a third parser that accepts either p's or q's. For example, itentifier+number would create a function that parses either identifiers or numbers.

We can define MonadPlus and MonadZero in the same way we would define Monad.

template< class ... > struct MonadZero;
template< class ... > struct MonadPlus;

template< class M, class Mo = MonadZero< Cat<M> > >
auto mzero() -> decltype( Mo::template mzero<M>() )
{
    return Mo::template mzero<M>();
}

template< class A, class B, class Mo = MonadPlus<Cat<A>> >
auto mplus( A&& a, B&& b ) -> decltype( Mo::mplus(std::declval<A>(),std::declval<B>()) )
{
    return Mo::mplus( std::forward<A>(a), std::forward<B>(b) );
}

template< class X, class Y >
auto operator + ( X&& x, Y&& y ) -> decltype( mplus(std::declval<X>(),std::declval<Y>()) )
{
    return mplus( std::forward<X>(x), std::forward<Y>(y) );
}

First, we define these for sequences.

template<> struct MonadZero< sequence_tag > {
    /* An empty sequence. */
    template< class S >
    S mzero() { return S{}; }
};

/* mplus( xs, ys ) = "append xs with ys" */
template<> struct MonadPlus< sequence_tag > {
    template< class A, class B >
    static A mplus( A a, const B& b ) {
        std::copy( b.begin(), b.end(), std::back_inserter(a) );
        return a;
    }
};

And then for Parsers.

/* mzero: a parser that matches nothing, no matter the input. */
template< class X > struct MonadZero< Parser<X> > {
    template< class P >
    static P mzero() { 
        return parser<X> (
            []( const std::string& s ){
                return std::vector<std::pair<X,std::string>>(); 
            }
        );
    }
};

/* mplus( pa, pb ) = "append the results of pa with the results of pb" */
template< class X > struct MonadPlus< Parser<X> > {
    using P = Parser<X>;
    static P mplus( P a, P b ) {
        return parser<X> (
            [=]( const std::string& s ) { return a(s) + b(s); }
        );
    }
};

Since we usually only want the first successful parse, the paper define an operator, +++, that does this.

template< class X >
Parser<X> mplus_first( const Parser<X>& a, const Parser<X>& b ) {
    return parser<X> (
        [=]( const std::string s ) {
            using V = std::vector< std::pair< X, std::string > >;
            V c = (a + b)( s );
            return c.size() ?  V{ c[0] } : V{};
        }
    );
}

This completes the required building blocks. A parser of significant complexity could be made using only the above functions and types. The paper describes building a notably simple parser, so let's do that instead!


Basic Monadic Parsers.

The simplest parser is item, which accepts any char.

std::string tail( const std::string& s ) {
    return std::string( std::next(s.begin()), s.end() );
}

/* 
 * Unconditionally match a char if the string is not empty.
 * Ex: item("abc") = {('a',"bc")}
 */
auto item = parser<char> (
    []( const std::string& s ) {
        using Pair = Parser<char>::parse_state;
        using R = Parser<char>::result_type;
        return s.size() ? R{ Pair( s[0], tail(s) ) } : R();
    }
);

To demonstrate its usage, the paper defines a simple parser that takes a string of length three or more and returns the first and third values.

auto p = item >>= []( char c ) {
    return item >> item >>= [c]( char d ) {
        return mreturn<Parser>( std::make_pair(c,d) );
    };
};

p first runs item to extract c, then runs item again but throws away the value. It runs item a third time to extract d and finally returns as its value (c,d). p("abcd") would return {(('a','c'),"d")}.

The next function creates a parser that is a little more helpful:

/* sat( pred ) = "item, if pred" */
template< class P >
Parser<char> sat( P p ) {
    return item >>= [p]( char c ) { 
        return p(c) ? mreturn<Parser>(c) : mzero<Parser<char>>();
    };
}

Given some function that operates on chars, this extracts an item, but then checks the condition without consuming any additional input. If p(c) returns true, then it returns a parser with the value c, otherwise zero, a failed parse. Using this, we can define a parser to accept only a specific char.

Parser<char> pchar( char c ) {
    return sat( [=](char d){ return c == d; } );
}

And then, a parser that accepts only a specific string.

Parser<std::string> string( const std::string& s ) {
    if( s.size() == 0 )
        return mreturn<Parser>( s );

    Parser<char> p = pchar( s[0] );
    for( auto it=std::next(s.begin()); it != s.end(); it++ )
        p = p >> pchar( *it );

    return p >> mreturn<Parser>(s);
}

Note: There is no name conflict with std::string because the source does not contain "using namespace std;".

This function does something very odd. For every char in s, it chains together char parsers. For example, string("abc") would return a parser equivalent to pchar('a') >> pchar('b') >> pchar('c') >> mreturn<Parser>("abc"). If any of the char parsers fail down the line, mreturn<Parser>(s) will fail. Since we already know the values of the successful parses, their values are thrown away.

Though faithful to the paper, this may be less efficient than desirable. One could implement string in this way, too:

Parser<std::string> string( const std::string& str ) {
    return parser<std::string> (
        [str]( const std::string& s ) {
            using R = typename Parser<std::string>::result_type;
            if( std::equal(str.begin(),str.end(),s.begin()) ) {
                return R {
                    { str, s.substr( str.size() ) }
                };
            } else {
                return R();
            }
        }
    );
}

It can at times be simpler to write out these functions instead of composing them, however, that can be thought of as an optimization.

The next function creates a new parser from a parser, p, that accepts one or zero p's.

template< class X >
Parser<std::vector<X>> some( Parser<X> p ) {
    using V = std::vector<X>;
    using Pair = std::pair<V,std::string>;
    using R = std::vector< Pair >;
    using P = Parser<V>;
    return mplus_first( 
        parser<V> (
            [=]( const std::string& s ) {
                auto matches = p(s);

                return matches.size() == 0 ? R{}
                    : R{ 
                        Pair( 
                            V{std::move(matches[0].first)}, 
                            std::move( matches[0].second )
                        ) 
                    };
            }
        ),
        mreturn<Parser>( V{} )
    );
}

It was unfortunately difficult to translate, but does work. (The gist also contains a version called many, which accepts zero or many p's.) some converts a parser of type X to one that produces a vector or X's. It always succeeds, even if it does not successfully parse anything.

To create a parser that consumes whitespace is now trivial.

template< class X >
using ManyParser = Parser< std::vector<X> >;

ManyParser<char> space = some( sat([](char c){return std::isspace(c);}) );

We require one more function to parse alternating sequences. Like what? A sum, like "1+2+3" is a sort of alternating sequence of numbers and +'s. A product is an alternating sequence of numbers and *'s.

Here's the weird part: what does a parser that accepts a "+" return? What about a parser that accepts a "-"? The value of such a parser, as it turns out, is the binary function that it represents! In the case of the implementation below, this is a function pointer of type int(*)(int,int).

/* 
 * chain(p,op): Parse with p infinitely (until there is no match) folding with op.
 *
 * p and op are both parsers, but op returns a binary function, given some
 * input, and p returns the inputs to that function. For example, if:
 *      input: "4"
 *      p returns: 4
 * No operator is read, no operation is performed. But:
 *      input: "4+4"
 *      p returns: 4
 * op is then parsed with the function, rest:
 *      input: "+4"
 *      op returns: do_add
 *      input: "4"
 *      p returns: 4
 *      rest returns: 8
 * rest applies the operation parsed by op. It alternates between parsing p and
 * op until there are no more matches. 
 */
constexpr struct Chainl1 {
    template< class X, class F >
    static Parser<X> rest( const Parser<X>& p, const Parser<F>& op, const X& a ) {
        // Alternate between op and p until input is consumed or a parse fails.
        auto r = op >>= [=]( const F& f ) {
                return p >>= [&]( const X& b ) {
                    return rest( p, op, f(a,b) );
                };
        };

        // Return the first successful parse, or a if none.
        return mplus_first( r, mreturn<Parser>(a) );
    }

    template< class X, class F >
    Parser<X> operator () ( Parser<X> p, Parser<F> op ) const {
        return p >>= closet( rest<X,F>, std::move(p), std::move(op) );
    }
} chainl1{};


Adding the final touches.

The paper describes a few generally useful functions for constructing parsers based off the above function. space (defined above) consumes whitespace. token consumes any trailing whitespace after parsing p.

constexpr struct Token {
    template< class X >
    Parser<X> operator () ( Parser<X> p ) const {
        return p >>= []( X x ) {
            return space >> mreturn<Parser>(std::move(x));
        };
    }
} token{};

symb converts a string to a token.

auto symb = compose( token, string ); // symb(s) = token( string(s) )

apply consumes any leading whitespace.

constexpr struct Apply {
    template< class X >
    Parser<X> operator () ( Parser<X> p ) const {
        return space >> std::move(p);
    }
} apply{};

The big idea is that we never want to manually write a function that returns a list of successful parses. It's hard! It's much easier to compose such  functions from smaller, more comprehensible ones and use those to build reasonable complex, but more simple to reason about, parsers.

The parser itself.

Now, using all of the tools provided, we can create a parser much more trivially than otherwise--though that is perhaps true of any time one has a new set of tools. First, we define a parser that accepts digits.

constexpr bool is_num( char c ) {
    return c >= '0' and c <= '9';
}

/* Parse one digit. */
Parser<int> digit = token( sat(is_num) ) >>= []( char i ) { 
    return mreturn<Parser>(i-'0'); 
};

Now, digit("2") returns 2, but (digit >> digit)("22") returns 2 as well! Why? Because the first run of digit extracts the first 2, but throws that value away and the second run of digit extracts the second 2. To parse a two-digit number, we need something like this:

Parser<int> twoDigit = digit >>= []( int x ) {
    return digit >>= [x]( int y ) {
        return mreturn<Parser>( x*10 + y );
    };
};

It extracts the first then the second digit and returns the original number, converted from a string to an int! To parse arbitrarily long numbers (diverging from the paper's version), we can define a chain operation!

int consDigit( int accum, int digit ) { return accum*10 + digit; }
Parser<int> num = chainl1( digit, mreturn<Parser>(consDigit) );

For every two digits parse, num calls consDigit to fold the values together. As mentioned earlier, chainl1 works by alternating between its two parsers. Since the second argument is a parser which consumes no input, num only accepts digits.

Next, we can define the parsers for binary operations.

// Binary operations of type int(*)(int,int).
int do_add(  int x, int y ) { return x + y; }
int do_sub(  int x, int y ) { return x - y; }
int do_mult( int x, int y ) { return x * y; }
int do_div(  int x, int y ) { return x / y; }

auto addop = mplus (
    pchar('+') >> mreturn<Parser>(do_add),
    pchar('-') >> mreturn<Parser>(do_sub)
);

auto mulop = mplus (
    pchar('*') >> mreturn<Parser>(do_mult),
    pchar('/') >> mreturn<Parser>(do_div)
);

addop parses either a "+" or "-" and returns either do_add or do_sub, respectively. Because the parsers must return functions of the same types, std::plus and std::minus could not be used.

With this, we can define a term as an alternating sequence of numbers and multiplications and divisions; and an expr(ession) as an alternating sequence of terms, +'s and -'s.

/* 
 * Parse terms: series of numbers, multiplications and divisions.
 * Ex: "1*3*2" -> (3,"*2") -> 6
 */
Parser<int> term = chainl1( num, mulop );

/*
 * Parse expressions: series of terms, additions and subtractions.
 * Ex: "1+7*9-1" -> (1."+7*9-1") -> (63,"-1") -> 62
 */
Parser<int> expr = chainl1( term, addop );

And we're done! We have just built a calculator! It can evaluate any expression of additions, subtractions, multiplications, divisions, and it is whitespace agnostic. While implementing Parser itself took a considerable amount of work, using it does not.

int main() {
    using std::cin;
    using std::cout;
    using std::endl;

    cout << "Welcome to the calculator!\n" 
         << "Press ctrl+d or ctrl+c to exit.\n"
         << "Type in an equation and press enter to solve it!\n" << endl;

    std::string input;
    while( cout << "Solve : " and std::getline(std::cin,input) ) {
        auto ans = apply(expr)(input);
        if( ans.size() and ans[0].second.size() == 0 )
            cout << " = " << ans[0].first;
        else
            cout << "No answer.";
        cout << endl;
    }
}

I highly encourage anyone reading this to attempt to compile and modify the source code.

See the gist at github for the source in full: https://gist.github.com/4112114
And for the original parser I wrote: https://gist.github.com/4112114#file_trivial_parser.cpp

Friday, November 9, 2012

Understanding Monads

Monads can be a stumbling block, even for Haskell programmers. While I have gotten little feedback that people need help in understanding, I thought it unlikely they didn't. I thought I would suppose that a lot of people didn't get it and post some resources to help out.

The easiest way to learn is probably to pick up Haskell. It's a weird language, but if you can read LISP, you'll recognize is as syntactically similar, but without everything being in polish prefix notation. Learn You a Haskell helped me get a good start earlier this year.

Learning a new language to grasp monads isn't quite necessary. The difficulty is that very little has been written in this direction in C++. There's FC++ which has been around for at least over a decade. Also, FACT!. And finally, my own library (from which most of these articles come). Unfortunately, there does not seem to have been much interest in functional programming in general in C++. fpcomplete has several articles definitely worth mentioning, such as the one on the continuation monad and a three-part video on the functor pattern.

The easiest way to learn monads is exposure, practice, and experience, which builds an intuition about how and why they work. This is difficult in a language where so few examples exist. In other languages, we have the wonderfully written introduction in Javascript (I don't know Javascript, but I found it very easy to understand.) and You Could Have Invented Monads.

It's also a really good idea to study the foundations. Category theory (wiki) is not about Haskell or monads in particular; it's a highly programming-relevant field of mathematics. Monads (wiki) are mathematical concept within category theory. (See also Category Theory for Computer Scientists (pdf))

I do not expect one to read through each link since that might take weeks! But different people learn different ways, so one source may be more helpful than another. Good luck!

Thursday, November 1, 2012

Monadic IO in C++

Previously, I covered Functors (fmap), and Monads (mbind, mreturn), and today, IO. Whether or not monadic IO is desirable in C++, I cannot say. But it represents a significantly more difficult monad to conceptualize and implement, and many of Haskell's monads are similar (StateT, Reader, etc.), so if nothing else, this is good practice. It's also good practice for implementing contiuations.

I neglected (by mistake) to discuss Monads in full, mainly do and fail. In Haskell, fail takes a string, some error message, and returns a failure which, for a pointer, might be null, for a sequence, an empty one. More useful today will be do. Since this should have been in the Monads article, I will explain that first, and then how it's relevant to IO.

do simply takes two monads and returns the second. It's symbolically noted >> like bind's >>=. If integers were monads, 1 >> 2 would equal 2. For pointers p and q, p >> q would equal q, but only if p. By that, I mean if p = null, then p >> q = null. If q is null, then p >> q is null regardless. For sequences, Haskell defines s >> t as t appended to itself for every element of s.

    [a,b] >> [x,y] = [x,y,x,y]
    [a] >> [x,y] = [x,y]
    [] >> [x,y] = []

It should be fairly intuitive how to write this, so I will continue to IO.

IO<X>

So far, we have always made a few implicit assumption about monads. They hold a value and exactly the type in the angle-brackets. In a functional sense, input is a mechanism that somehow produces a value, but it comes out of thin air from some unseen external force. Output is a device that makes seen values disappear. An IO is either a value-producing machine, or a value-taking one. So an IO monad is a function! A function with IO decoration, that is. 

What does an IO<int> contain? It must contain a function that produces an int! IO<void>? A function that outputs some value. But since it's actually a function, wouldn't it be more like IO<int(*)()>? Or IO<MyFuncType>? Well, the IO monad can be implemented with std::function, and that allows us to keep our assertions about monadic types, as you will soon see.

So Monads aren't just containers. One might have heard they are also computations. That's just an obfuscated way of saying they're functions.

Hello World

Let's first define IO<X> as an object holding an std::function.

template< class X > struct IO {
    using function = std::function<X()>;
    function f;

    template< class F >
    IO( F&& f ) : f(std::forward<F>(f)) { }

    X operator () () const {
        return f();
    }
};

template< class F, class R = typename std::result_of<F()>::type >
IO<R> io( F f ) {
    return IO<R>( std::move(f) );
}

And now, a few examples:

IO<int> readInt = io( []{ int x; std::cin >> x; return x; } );
IO<std::string> readStr = io( []{ std::string s; std::cin >> s; return s; } );
IO<int> constant = io( []{ return 5; } );

And that's input. We'll have to specialize IO for output.

template< > struct IO<void> {
    using function = std::function<void()>;
    function f;

    IO( function f ) : f(std::move(f)) { }

    void operator () () const {
        f();
    }
};

IO<void> hello = io( []{ std::cout << "Hello world." << std::endl; } );
IO<void> doNothing = io( []{} );

Now, let's try and imagine some operation on IO.

IO<void> hello2 = hello >> hello

do is a lot like bind, except that it does not pass the result of the previous expression to the next. When we do two IO monads, we get a third back. No IO has yet been executed. hello2 is the function that actually executes the IO.

I wanted to go over IO<X> because it is easier to reason about than IO<F>, and much more Haskell-like. However, C++ is about zero-overhead abstractions and this ends up being less efficient because of the tricks std::function uses in order to work. To see the std::function implementation, check out the gist.

IO<F>

This version is initially much simpler to implement.

template< class F > struct IO {
    using function = F;
    F f;

    constexpr IO( function f ) : f(std::move(f)) { }

    constexpr decltype(f()) operator () () {
        return f();
    }
};

template< class F, class _F = typename std::decay<F>::type >
constexpr IO<_F> io( F&& f ) {
    return std::forward<F>(f);
}

But now what should the type of hello2 be? Well, it'll basically be executing hello, twice, so it's a sort of composition, but with two void functions. Let's define a type to represent this composition: Incidence.

template< class F, class G > struct Incidence {
    F a;
    G b;

    constexpr Incidence( F a, G b )
        : a(std::move(a)), b(std::move(b))
    {
    }&;

    // Execute a and b coincidentally.
    auto operator () () const 
        -> typename std::result_of<G()>::type
    {
        a();
        return b();
    }
};

template< class F, class G, class I = Incidence<F,G> >
constexpr I incidence( F f, G g ) {
    return I( std::move(f), std::move(g) );
}

What will mreturn do? An IO is just a function, so it'll take some x and make a function that just returns x. To do this, we'll define another type, Identity.

template< class X > struct Identity {
    using value_type = X;
    value_type x;

    template< class Y >
    Identity( Y&& y ) : x( std::forward<Y>(y) ) { }

    constexpr value_type operator () () { return x; }
};

template< class X, class D = typename std::decay<X>::type >
Identity<D> identity( X&& x ) {
    return Identity<D>( std::forward<X>(x) );
}

It may make more sense to have Identity return a reference or const reference to x, but since we are dealing strictly with values, this would complicate things. Any decltype or result_of involving an Identity would have to remove the reference.

Now we can write things like:

    auto getFive = mreturn<IO>( 5 );
    int five = getFive();

and all is good. Next is mbind.

    auto someIO = readInt >>= []( int x ) { return mreturn<IO>(x + 5); }

Here, we have our IO called readInt which gets an int from std::cin. It passes the int to our lambda, which adds 5 and returns a new IO. No integer has been read from std::cin yet, nor the five added; we have merely composed readInt with our continuation to create a new function, a new IO.

This part might be a little mind boggling. Let's say we call someIO; what happens? We know it'll return an int with 5 added to it. It starts by calling readInt. Then the number gets passed to our lambda. It adds five and returns a new IO<Identity<int>>. If we stopped here, we wouldn't have our value, so we then execute the Identity IO. We'll implement DoubleCall to express the pattern of constructing the IO with the first call and running it with the second. We'll also expand Incidence to handle the case where it needs to pass the value (from readInt) to the continuation.

template< class T, class R >
using EVoid = typename std::enable_if< std::is_void<T>::value, R >::type;
template< class T, class R >
using XVoid = typename std::enable_if< !std::is_void<T>::value, R >::type;

// Incidental.
template< class F, class G, class R = typename std::result_of<F()>::type >
constexpr auto incidentallyDo( F&& f, G&& g ) 
    -> XVoid <
        R,
        decltype( std::declval<G>()( std::declval<F>()() ) )
    >
{
    return std::forward<G>(g)( std::forward<F>(f)() );
}

// Co-incidental.
template< class F, class G, class R = typename std::result_of<F()>::type >
constexpr auto incidentallyDo( F&& f, G&& g ) 
    -> EVoid <
        R,
        decltype( std::declval<G>()() )
    >
{
    std::forward<F>(f)();
    return std::forward<G>(g)();
}

// Either an incidence or coincidence.
template< class F, class G > struct Incidence {
    F a;
    G b;

    constexpr Incidence( F a, G b )
        : a(std::move(a)), b(std::move(b))
    {
    }

    using R = decltype( incidentallyDo(a,b) );

    constexpr R operator () () {
        return incidentallyDo( a, b );
    }
};

template< class F, class G, class I = Incidence<F,G> >
constexpr I incidence( F f, G g ) {
    return I( std::move(f), std::move(g) );
}

template< class F > struct DoubleCall {
    F f;

    using F2 = typename std::result_of<F()>::type;
    using result_type = typename std::result_of<F2()>::type;

    constexpr result_type operator () () {
        return f()();
    }
};

template< class F >
constexpr DoubleCall<F> doubleCall( F f ) {
    return { std::move(f) };
}

We may now implement Functor<IO> and Monad<IO> totally in terms of IdentityIncidence, and DoubleCall.


template< class _ > struct Functor< IO<_> > {
    template< class F, class G, class I = Incidence<F,G> >
    constexpr static IO<I> fmap( F f, IO<G> r ) {
        return I( std::move(f), std::move(r.f) );
    }
};

template< class _ > struct Monad< IO<_> > {
    template< class F, class G, 
        class R = typename std::result_of<G()>::type >
    static constexpr auto mbind( F f, IO<G> m ) 
        -> IO< DoubleCall< Incidence<G,F> > >
    {
        // f returns a new IO, so doubleCall the incidence to execute it!
        return doubleCall (
            incidence( std::move(m.f), std::move(f) )
        );
    }

    template< class F, class G >
    static constexpr IO<Incidence<F,G>> mdo( IO<F> f, IO<G> g ) {
        return incidence( std::move(f.f), std::move(g.f) );
    }

    template< class __, class X, class D = typename std::decay<X>::type >
    static constexpr IO<Identity<D>> mreturn( X&& x ) {
        return Identity<D>( std::forward<X>(x) );
    }

    template< class _IO >
    static _IO mfail() {
        return _IO( []{ } );
    }
};

Now, we can string along arbitrary operations in whatever way we'd like, but a sane person might realize that if one lambda reads in an int and another prints it, then we've really just obfuscated this process because we could otherwise we could just do that ourselves. Like any other useful thing, IO requires other useful things to make it useful.

The tools

First, input. We could write a function that read an int and returned it and it would look something like this:

    int readInt() { int x; std::cin >> x; return x; }

Or, we can template read to work on any type. Even better, we can make read a type!

template< class T >
struct ReadT {
    T operator () () const {
        T x;
        std::cin >> x;
        return x;
    }
};

template< class T >
constexpr IO< ReadT<T> > readT() {
    return ReadT<T>();
}

We can read; can we write? Since an IO takes no arguments, anything we want to write has to be held within the IO object. First, we just need some generic functions.

constexpr struct Print {
    template< class X >
    void operator () ( const X& x ) const {
        std::cout << x;
    }
} print{};

template< class X >
static std::string show( const X& x ) {
    static std::ostringstream oss;
    oss.str( "" );
    oss << x;
    return oss.str();
}

static std::string show( std::string str ) {
    return str;
}

static constexpr const char* show( const char* str ) {
    return str;
}

template< class X, class Y, class ...Z >
static std::string show( const X& x, const Y& y, const Z& ...z )
{
    return show(x) + show(y,z...);
}

Now, we want to turn a show/Print combo into an IO. I'll be borrowing my code from "Partial Application in C++" for this.

constexpr struct Echo {
    using F = PartialApplication< Print, std::string >;
    using result_type = IO<F>;

    template< class ...X >
    result_type operator () ( const X& ...x ) const {
        // We don't know if x will still be around when the IO executes, so
        // convert to a string right away!
        return closet( print, show(x...) );
    }
} echo{};

auto newline = io( []{ std::cout << std::endl; } );

We can now write rudimentary IO programs like

    echo("Give me an int!\n") >> (readT<int>() >>= echo) >> newline

Which will, if you've been following along, echo "Give me an int!" to the screen, read one from std::cin, and echo the int back out.

Complications

Remember addM?

template< class M >
M addM( const M& a, const M& b ) {
    return mbind (
        [&]( int x ) {
            return mbind ( 
                [=]( int y ) { return mreturn<M>(x+y); },
                b
            );
        }, a

    );
}

For IO, this should read in two ints and add them, but it won't work since IO<F> >>= []{...} =/= IO<F>! We could wrap the function in a decltype, but not while using lambdas. The challenge is to rewrite this function without lambdas.Looking at the lambda, it takes only one argument, but also captures b. We can replace it with a function that takes f, b, and x. x has to be the last argument because we're going to partially apply f and b, making it a function of x.

template< class M > struct _AddM {
    constexpr auto operator () ( int x, int y ) -> decltype( mreturn<M>(1) )
    {
        return mreturn<M>( x + y );
    }
};

constexpr struct BindCloset {
    template< class F, class X, class M >
    constexpr auto operator () ( F&& f, M&& m, X&& x )
        -> decltype( std::declval<M>() >>=
                     closet(std::declval<F>(),std::declval<X>()) )
    {
        return std::forward<M>(m) >>=
            closet( std::forward<F>(f), std::forward<X>(x) );
    }
} bindCloset{};

template< class M >
constexpr auto addM( const M& a, const M& b )
    -> decltype (
        a >>= closure( bindCloset, _AddM<M>(), b )
              
    )
{
    return a >>= closure( bindCloset, _AddM<M>(), b );
}

When we bind a to the closure, x gets extracted as the last argument. The closure then partially applies x to f and extracts y from b.

The same technique can be used to implement liftM, the two-argument version, or any other function with embedded lambdas.

Conclusions

How powerful is this? We can write almost a whole program in it! Here's the main I used to test IO (using some code from the Monads article).

int main() {
    std::unique_ptr<int> p( new int(5) );
    auto f = []( int x ) { return Just(-x); };
    std::unique_ptr<int> q = mbind( f, p );

    std::vector<int> v={1,2,3}, w={3,4};

    auto readInt = readT<int>();

    auto program = echo( "Unique pairs of [1,2,3]:\n\t" )
        >> echo( uniquePairs(v) ) >> newline

        >> echo("Unique pairs of Just 5:\n\t")
        >> echo( uniquePairs(p) ) >> newline

        >> echo( "Please enter two numbers, x and y: " )
        >> (
            addM( readInt, readInt ) >>= []( int x ) {
                return echo( "x+y = ", x ) >> newline;
            }
        )

        >> echo("The quadratic root of (1,3,-4) = ")
            >> echo( qroot(1,3,-4) ) >> newline
        >> echo("The quadratic root of (1,0,4) = ")
            >> echo( qroot(1,0,4) ) >> newline;

    program();
}

The variable program is a complex data structure that contains many Incidences made of PartialApplications of Print and std::string. GCC does a very good job of optimizing it, however, if we had written this out normally (no monads), GCC would just insert each item into std::cout. Instead we're creating a structure with Prints and std::strings. GCC optimizes away the Print object and inlines much of the code, which it could not do if it were a regular function (since it would have to maintain a pointer), but it wouldn't ordinarily construct this many std::strings. There is room for optimization--this is not hopelessly inefficient, but be aware of how your objects are constructed and passed along.

Would one ever want to actually use this over vanilla IO in C++? I certainly wouldn't argue it should be prefered. Still, if one was in the situation of needing to pass along a function, this offers a simple way of composing it from simpler objects. Composition itself is a powerful tool.

At first, we may have thought of a monad or functor as a container, but IO disproves this. So monads can be containers or functions? IO is more than just a function, it's a program encoded as data, constructed at run-time. It's hard to point to one sentence that well-describes all monads, but we know they aren't one exact thing over another. It's almost anything!


As always, the source code: https://gist.github.com/3994038
IO implemented with std::functionhttps://gist.github.com/3994038#file_io_monad.cpp

Saturday, October 27, 2012

Monads in C++

In my last two articles, I discussed fmap in C++. At the end, I implemented it with a solution of tag dispatch and a type class, calling it type-class dispatch. Today I want to talk about the next step: monads. Familiarity with fmap is required, but not monads or Haskell. I will be using the same type-class dispatch code here as in the last post, but without explanation.

struct sequence_tag {};
struct pointer_tag {};

template< class X >
X category( ... );

template< class S >
auto category( const S& s ) -> decltype( std::begin(s), sequence_tag() );

template< class Ptr >
auto category( const Ptr& p ) -> decltype( *p, p==nullptr, pointer_tag() );

Monads are scary. Or at least they seem scary. People talk about them like they are. In reality, they are not much more complicated than Functors, being very similar. Previously, the problem was that we have a function f and a Functor, F(x). fmap simply allowed us to apply f to the data inside the Functor. Monads do the same thing, except that f is monad-aware and returns a monad of the correct type. For example, with fmap we might write

std::unique_ptr<int> p( new int(5) );
auto f = []( int x ) { return -x; };
std::unique_ptr<int> q = fmap( f, p );

and we know that (*q) = -(*p). What if f knew that we wanted to have a unique_ptr returned? Well, then we could use the monad's version of fmap, which I'll refer to as mbind (monad bind).

std::unique_ptr<int> p( new int(5) );
auto f = []( int x ) { 
  return std::unique_ptr<int>( new int(-x) ); 
};
std::unique_ptr<int> q = mbind( f, p );

So what is a Monad? Almost the same as what a Functor is!

    If fmap(f,F(x)) = F(f(x)),
        then mbind(f,M(x)) = f(x), or something like that.

Monads can be std::vectors or std::unique_ptrs, std::pairs; the limit is your imagination.

Monads have one more ability: to construct a type, M(x), given an x, with a function called return. But return means two different things in Haskell and C++, so I'll use the term mreturn. This is a pretty simple concept--we can rewrite the above example like so:

std::unique_ptr<int> p( new int(5) );
auto f = []( int x ) { return mreturn(-x); };
std::unique_ptr<int> q = mbind( f, p );

In this example, mreturn is a function that takes an int and returns an std::unique_ptr<int>.

And so we have two basic operations:

    auto m = mreturn<M>(x); // creates an M<X>
    mbind( f, m ); // Applies f to x.

And we know

    auto p = mreturn<unique_ptr>(3); // will create a unique_ptr<int>.
    mbind( f, p ); // is equivalent to f(*p)

And we'll consider, for the moment, that a Monad is a type for which this operation is defined.

I'll implement this much the same way I did fmap. We start with a free function, mbind, which maps to the static member function Monad::mbind, and mreturn which maps to Monad::mreturn.

template< class ... > struct Monad;

template< class F, class M, class Mo=Monad<Cat<M>> >
auto mbind( F&& f, M&& m )
    -> decltype( Mo::mbind(std::declval<F>(),std::declval<M>()) )
{
    return Mo::mbind( std::forward<F>(f), std::forward<M>(m) );
}

// The first template argument must be explicit!
template< class M, class X, class Mo = Monad<Cat<M>> >
M mreturn( X&& x ) {
    // We have to forward the monad type, too.
    return Mo::template mreturn<M>( std::forward<X>(x) );
}

One might notice that the above example using mreturn and this definition don't match. Instead of calling mreturn(-x), it should call mreturn<std::unique_ptr<int>>(-x). However, the int part is redundant, so let's overload mreturn using a template template parameter so we only have to supply std::unique_ptr.

template< template<class...>class M, class X, 
          class Mo = Monad<Cat<M<X>>> >
M<X> mreturn( const X& x ) {
    return Mo::template mreturn<M<X>>( x );
}

Now, we can write that example like so:

std::unique_ptr<int> p( new int(5) );
auto f = []( int x ) { 
  return mreturn<std::unique_ptr>(-x); 
};
std::unique_ptr<int> q = mbind( f, p );

The pointer monad.

template< > struct Monad< pointer_tag > {
    template< class F, template<class...>class Ptr, class X,
              class R = typename std::result_of<F(X)>::type >
    static R mbind( F&& f, const Ptr<X>& p ) {
        // Just like fmap, but without needing to explicitly return the correct type.
        return p ? std::forward<F>(f)( *p ) : nullptr;
    }


    template< class M, class X >
    static M mreturn( X&& x ) {
        // All smart pointers define element_type.
        using Y = typename M::element_type; 
        return M( new Y(std::forward<X>(x)) );
    }
};

This may not be the most exciting code, but we can use it to translate a small Haskell function into C++.

    -- Haskell
    addM a b = do
        x <- a -- Extract x from a
        y <- b -- and y from b.
        return (x+y) -- Return a new monad with the value (x+y)

If we supplied two unique_ptrs, we'd get one back holding the value x+y. The first line, x <- a, syntactically means "what fallows is a function of x." This is addM with do notation; another way to write it:

    addM a b = a >>= (\x -> b >>= (\y -> return (x+y)) )

Here, >>= denotes a bind and (\x->...) denotes a lambda that takes x. The inner-most function, (\y -> return (x+y)) returns the actual value as a monad. It gets called when we extract the value from b with (\x -> b >>= ... ). The x came from a >>= (\x -> ... ). So it extracts x from a, then y from b, and constructs a new monad with the value x+y.

// C++
template< class M >
M addM( const M& a, const M& b ) {
    return mbind (
        [&]( int x ) {
            return mbind ( 
                [=]( int y ) { return mreturn<M>(x+y); },
                b
            );
        }, a

    );
}

Yuck! This is a literal translation, but Haskell handles scope automatically with do notation and it implicitly returns the last statement, while we write return mreturn<M>.  We can rewrite this to use fmap([=](int y){return x+y;},b) and that solves the return problem, but not the scoping one. We can alleviate that by defining an operator overload for mbind, and why not use the very same operator as in Haskell?

template< class M, class F >
auto operator >>= ( M&& m, F&& f )
    -> decltype( mbind(std::declval<F>(),std::declval<M>()) )
{
    return mbind( std::forward<F>(f), std::forward<M>(m) );
}

template< class M >
M addM( const M& a, const M& b ) {
    return a >>= []( int x ) {
        return fmap( [=]( int y ){ return x+y }, b );
    };
}

It's hard to justify the use of operator overloads in C++, but this one rarely gets any use. It won't change the behavior of basic types; given some int x, x >>= 2, this still means you with to shift the bits by two. If this gives one an uncomfortable feeling, it can be put in its own namespace so that in order to make use of the operator overload, the user would have to write using namespace monad; or whatever before writing >>=.

Monadic sequences.

Remember, fmap(f,seq) took a regular function and made a new sequence by applying f to seq. What will mbind(f,seq) do? This time, f is monad-aware, so it already returns a sequence. Does mbind return a sequence of sequences? That would be very confusing. It actually returns the concatenation of every sequence produced by f(x). So, if f(x)={-x,x}, then mbind(f,{1,2}) = {-1,1,-2,2}.

template< > struct Monad< sequence_tag > {
    template< class F, template<class...>class S, class X,
              class R = typename std::result_of<F(X)>::type >
    static R mbind( F&& f, const S<X>& xs ) {
        R r;
        for( const X& x : xs ) {
            auto ys = std::forward<F>(f)( x );
            std::move( std::begin(ys), std::end(ys), std::back_inserter(r) );
        }
        return r;
    }

    template< class S, class X >
    static S mreturn( X&& x ) {
        return S{ std::forward<X>(x) }; // Construct an S of one element, x.
    }
};
std::move from <algorithm>

What implications does this have on our addM function? If v={1,2} and w={3,4}, what does addM(v,w) return? Try it!

int main() {
    std::vector<int> v={1,2}, w={3,4};
    auto vw = addM(v,w);

    std::cout << "v+w = { ";
    std::copy (
        std::begin(vw), std::end(vw),
        std::ostream_iterator<int>(std::cout, " ")
    );
    std::cout << '}' << std::endl;
}

Just in case you didn't actually run the code, it prints { 4 5 5 6 }. Does this sequence seem odd? It's { 1+3 1+4 2+3 2+4 }. Basically, it applied the addition function on every pair of elements from v and w. That means add(v[0],w[0]) then add(v[0],w[1]) then add(v[1],w[0]) then add(v[1],w[1]).

This is the magic of monads. The functionality of addM changed appropriately to how its arguments changed. It did so without us even thinking about how it might. And now, every type that can hold an int that one specialized mbind for works with addM, too!


In conclusion:

Monads are often talked about as mysterious, tricky, and hard to understand. They are none of these. It is of little importance to know concretely what a monad is. mbind is a simple function that applies some function, f, to some object M(x), where f returns M(y). mreturn is a simple function that constructs an object of type, M(x), given an x.

Note that Haskell also has a Monad function, >>, or mdo as I call it (though I can't remember why). mdo is not always as obvious as mbind, however I did implement it in the gist (see below).

In full, the monadic operations are:

    a >> b ; //  see the gist
    a >>= f ; // Apply the value(s) in a to f.
    mreturn<M>(x) ; // Create an M<X>.

There are a few helpful properties of this:

    mreturn<M>(x) >>= f == f(x)
    m >>= mreturn<M> == m 
    m >>= (\x -> k x >>= h) == (m >>= k) >>= h


Here's the code I wrote for this article: https://gist.github.com/3965514 (It contains a few extra examples.)
Monads in Haskel: http://www.haskell.org/ghc/docs/latest/html/libraries/base/Control-Monad.html#t:Monad