Archive | Applicative Functor RSS for this section

Aeson Revisited

As many of you know, the documentation situation in Haskell leaves something to be desired. Sure, if you are enlightened, and can read the types, you’re supposedly good. Personally, I prefer a little more documentation than “clearly this type is a monoid in the category of endofunctors”, but them’s the breaks.

Long ago, I wrote about some tricks I found out about using Aeson, and I found myself using Aeson again today, and I’d like to revisit one of my suggestions.

Types With Multiple Constructors

Last time we were here, I wrote about parsing JSON objects into Haskell types with multiple constructors. I proposed a solution that works fine for plain enumerations, but not types with fields.

Today I had parse some JSON into the following type:

data Term b a = App b [Term b a] | Var VarId | UVar a

I thought “I’ve done something like this before!” and pulled up my notes. They weren’t terribly helpful. So I delved into the haddocs for Aeson, and noticed that Aeson's Result type is an instance of MonadPlus. Could I use mplus to try all three different constructors, and take whichever one works?

instance (FromJSON b, FromJSON a) => FromJSON (Term b a) where parseJSON (Object v) = parseVar `mplus` parseUVar `mplus` parseApp where parseApp = do ident <- v .: "id" terms <- v .: "terms" return $ App ident terms parseVar = Var <$> v .: "var" parseUVar = UVar <$> v .: "uvar" parseJSON _ = mzero

It turns out that I can.

An Intro To Parsec

Taking a break from the Photo Booth, I’ve been working on a re-write of dmp_helper. If you’ve been following me for a while, you may remember that as a quick, dirty hack I did in perl one day to avoid manually writing out HTML snippets. While it mostly works, it’s objectively terrible. I guess if my goal is to never get a job doing Perl (probably not the worst goal to have) then it might serve me well.

But I digress. DmpHelper, the successor to dmp_helper, is going to be written in Haskell, and make extensive use of the Parsec parser library. The old dmp_helper makes extensive use of regular expressions to convert my custom markup to HTML, but I’d prefer to avoid them here. I find regular expressions to be hard to use and hard to read.

So I set off to work. And not 3 lines into main :: IO (), I came across a need to parse something! This is something that has been parsed many times before, but it’s good practice so it’s a wheel I’ll be re-inventing. I’m talking of course about parsing the ArgV!

Some Choices To Make

But before I get to that, I need to make some choices about how my arguments will look. I’ve decided to use the “double dash” method. Therefore, all flags will take the form of --X, where X is the flag. Any flag can take an optional argument, such as --i foo, all text following a flag is considered to belong the the preceeding flag until another flag is encountered. Some flags are optional, and some are mandatory. The order flags appear is not important.

So far, I have 3 flags implemented: --i [INPUT_FILE], the file to be converted, --o [OUTPUT_FILE], the location to save the resulting HTML, and --v, which toggles on verbose mode. Other flags may be added in as development progresses.

Data ArgV

Next, I’ll create a type for my ArgV.

data ArgV = ArgV {inputFile :: FilePath, outputFile :: FilePath, isVerbose :: Bool} deriving (Show)

As you can see, my type has three fields, which line up with my requirements above.

Introducing Parsec

Parsec is a fairly straight-forward library to use. It has a lot of operators and functions, whcih require some thought. However they all amount to basic functions that do a specific parsing task. Your job is to tell these functions what to do. So let’s get right down to it.

parseArgV :: [String] -> Either ParseError ArgV parseArgV i = parse pArgV "" $ foldArgV i foldArgV :: [String] -> String foldArgV i = foldl' (++) [] i

Here we have two functions to get us started. The function foldArgV collapses the list of strings into a single string, to be parsed. Because of the way that arguments are parsed by the operating system, the argument string --i foo --o bar --v will be collapsed to --ifoo--obar--v. This is good for us because this means we don’t have to worry about parsing white space.

The second function, parseArgV is our entry point into Parsec. All of Parsec’s combinator functions return a Parser monad. The function parse parses the data in the parser monad and returns either an error, or a whatever. In our case, it’s parsing to an ArgV.

Parse takes 3 arguments: a parser, a “user state”, and a string to parse. The first and third are pretty self explanatory, but the second is a mystery. Unfortunately I don’t have an answer for you as to it’s nature. I spent a good hour or two researching it yesterday, and the best I could come up with is “don’t worry about it”. None of the tutorials use it, and the documentation basically just says “this is state. Feel free to pass an empty string, it’ll be fine”. I’ve decided I won’t worry my little head about it for the time being.

pArgV :: CharParser st ArgV pArgV = do permute $ ArgV <$$> pInputFile <||> pOutputFile <|?> (False, pIsVerbose)

This function is the true heart of my parser. I’d like to draw your attention to the invocation of permute. This function implements the requirement that the order of arguments shouldn’t matter. Ordinarily, parsec will linearly go through some input and test it. But what do you do if you need to parse something that doesn’t have a set order? The library Text.Parsec.Perm solves this problem for us.

The function permute has a difficult type signature to follow, so a plain-english explanation is in order. Permute is used in a manner that resembles an applicative functor. Permute takes a function, and then using it’s operators, it takes functions that return a parser for the type of each field in the function in a pseudo-applicative style. These functions don’t need to come in the order that they’re parsed in, but in the order that the initial function expects them to be. Permute will magically parse them in the correct order and return a parser for your type. Let’s break this down line-by-line:

... permute $ ArgV <$$> pInputFile ...

Here we call permute on the constructor for ArgV. We feed it the first function, pInputFile using the <$$> operator. This operator is superficially, and logically similar to the applicative functor’s <$> operator. It creates a new permutation parser for the type on the left using the function on the right.

... <||> pOutputFile ...

Here we feed the second parser function, pOutputFile to the permutation parser using the <||> operator. This operator is superficially, and logically similar to applicative’s <*> operator.

... <|?> (False, pIsVerbose) ...

Finally, we feed it a parser for isVerbose. The operator <|?> works the same as the <||> operator, except that this operator is allowed to fail. If <||> fails, the whole parser fails. If <|?> fails, then the default value (False, in this case) is used. This allows us to have optional parameters.

Moving along from here, here are some extra parsing functions we need.

pInputFile :: CharParser st FilePath pInputFile = do try $ string "--i" manyTill anyChar pEndOfArg

This parser parses the --i parameter. The first line of the do expression parses the input to see if it is --i. Because it is wrapped in a call to try, it will only consume input if it succeeds. Since we don’t actually need this text, we don’t bind it to a name. If the parser had failed, it would exit the function and not evaluated the second line. On the second line, we take all the text until pEndOfArg succeeds. This text is what is returned from the function.

pEndOfArg :: CharParser st () pEndOfArg = nextArg <|> eof where nextArg = do lookAhead $ try $ string "--" return ()

This function detects if the parser is at the end of an argument. Notice the <|> operator. This is the choice operator. Basically, if the parser on the left fails, it runs the parser on the right. You can think of it like a boolean OR.

The parser eof is provided by parsec, and succeeds if there is no more input. The function lookAhead is the inverse of try. It only consumes input if the parser fails. By wrapping a try inside of a lookAhead, we create a parser that never consumes input.

The parser functions pOutputFile and pIsVerbose are almost identical to pInputFile, so I’m not going to bother typeing them out here.

Additional Reading

That’s about all there is to my ArgV parser. You can find some additional reading on the subject in Real World Haskell, whcih has an entire chapter dedicated to this subject. Also, check out the Parsec haddoc page on hackage.

Yet Another Aeson Tutorial

Lately I’ve been trying to make sense of Aeson, a prominent JSON parsing library for Haskell. If you do some googling you’ll see two competing notions: 1) Aeson is a powerful library, but it’s documentation is terrible and 2) about 10,000 Aeson tutorials. Even the Aeson hackage page has a huge “How To Use This Library” section with several examples. These examples usually take the form of:

“Make a type for your JSON data!”

data Contrived = Contrived { field1 :: String , field2 :: String } deriving (Show)

“Write a FromJSON instance!”

instance FromJSON Contrived where parseJSON (Object v) = Contrived <$> v .: "field1" <*> v .: "field2"


I’ll spare you the long form, other people have done it already and I’m sure you’ve seen it already. What I quickly noticed was that this contrived example wasn’t quite cutting it. I’ve had some challenges that I’ve had to overcome. I’d like to share some of the wisdom I’ve accumulated on this subject.

Nested JSON

The first Problem I’ve run into is the nested JSON. Let’s take a look at an example:

{ "error": { "message": "Message describing the error", "type": "OAuthException", "code": 190 , "error_subcode": 460 } }

That is an example of an exception that can get returned by any Facebook Graph API call. You’ll notice that the exception data is actually contained in a nested JSON object. If passed to a parseJSON function, the only field retrievable by operator .: is “error”, which returns the JSON object. We could define two types and two instances for this like:

data EXTopLevel = EXTopLevel { getEx :: FBException } deriving (Show) data FBException = FBException { exMsg :: String , exType :: String , exCode :: Int , exSubCode :: Maybe Int } deriving (Show) instance FromJSON EXTopLevel where parseJSON (Object v) = EXTopLevel <$> v .: "error" instance FromJSON FBException where parseJSON (Object v) = FBException <$> v .: "message" <*> v .: "type" <*> v .: "code" <*> v .:? "error_subcode"

In this case, you could decode to a EXTopLevel, and call getEx to get the actual exception. However, it doesn’t take a doctor of computer science to see that this is silly. Nobody needs the top-level object, and this is a silly amount of boilerplate. The solution? We can use our friend the bind operator. Aeson Objects are instances of Monad, and it turns out that it’s bind function allows us to drill down into objects. We can re-implement that mess above simply as:

data FBException = FBException { exMsg :: String , exType :: String , exCode :: Int , exSubCode :: Maybe Int } deriving (Show) instance FromJSON FBException where parseJSON (Object v) = FBException <$> (e >>= (.: "message")) <*> (e >>= (.: "type")) <*> (e >>= (.: "code")) <*> (e >>= (.:? "error_subcode")) where e = (v .: "error")

Much better right? I thought so too.

Types With Multiple Constructors

The next problem I’d like to talk about is using types with multiple constructors. Let’s take a look at an example:

{ "value": "EVERYONE" }

When creating a new album of facebook, the API needs you to set a privacy on the album. This setting is set using this JSON object. This would seem to be a trivial case for Aeson:

data Privacy = Privacy { value :: String } deriving (Show) instance FromJSON Privacy where parseJSON (Object v) = Privacy <$> v .: "value" <*>

Unfortunately, the following is not a valid privacy setting:

"{ "value": "NOT_THE_NSA" }

However, our Privacy type would allow that. In reality, this should be an enumeration:

data Privacy = Everyone | AllFriends | FriendsOfFriends | Self

But how would you write a FromJSON instance for that? The method we’ve been using doesn’t work, and parseJSON takes and returns magical internal types that you can’t really do anything with. I was at a loss for a while, and even considered using the method I posted above. Finally, the answer hit me. Like many things in Haskell, the answer was stupidly simple; just define a function to create the privacy object, and use that in the parseJSON function instead of the type constructor! My solution looks like this:

instance FromJSON Privacy where parseJSON (Object v) = createPrivacy v .: "value" createPrivacy :: String -> Privacy createPrivacy "EVERYONE" = Everyone createPrivacy "ALL_FRIENDS" = AllFriends createPrivacy "FRIENDS_OF_FRIENDS" = FriendsOfFriends createPrivacy "SELF" = Self createPrivacy _ = error "Invalid privacy setting!"

If the parseJSON needs a singular function to create a type, and you have more than one type constructor for your type, you can wrap your type constructors in one function that picks and uses the correct one. Your type function doesn’t need to know about Aeson; Aeson magically turns it’s parser into whatever type your function calls for.

A Trip To The Magical Land Of Monads

Monads can be a tricky topic. On the surface, they’re not hard at all. Taking Maybe for instance:

Prelude> Just 1 Just 1 Prelude> Nothing Nothing

That couldn’t possibly be easier. Something is either Just [something], or it is Nothing. However Monad world is the magical world of gotchas, unenforceable rules, and magical syntax. I had been planning on writing a post on Monads, much like I did with Applicatives and Functors. While researching this topic, I’ve determined that I’m not qualified to speak on this subject yet.

I am qualified to discuss the usage of Monads. I feel that say you are going to “learn how to do Monads” is similar to saying you are going to “learn how to do data structures”. Data structures are similar to each other in that they serve a common purpose: to contain data. However, a Vector is nothing like a Linked List, which is nothing like a Hash Map.

Similarly, a Maybe is nothing like an IO, which is nothing like a Writer. While they are all Monads, they serve different purposes. Today, I’d like to lay some groundwork on the topic and talk about binding functions.

On Magical Syntax

Much like Functor and Applicative, Monad brings functions that allow you to use normal values with monadic values. Monad brings the >>= operator. This operator is called the bind operator. (this is important for monads in general, but I won’t get into this today. Just know that the operator’s name is bind) The signature for >>= is:

(>>=) :: m a -> (a -> m b) -> m b

As you can see, it takes a Monad that contains an a, a function that takes an a and returns a Monad b, and returns a Monad b. Basically, it calls the function on the value contained in the monad, and does whatever additional action is appropriate for the monad, and returns a monad containing the result. In short: it is fmap for Monads. Let’s take a look at a quick example for Maybe:

Prelude> Just 1 >>= (\a -> Just $ a + 5) Just 6

As you can see, we bind Just 1 to a function that takes a value, adds it to 5, and wraps the result in a Just, which results in Just 6. Bind will correctly handle Nothing as well:

Prelude> Nothing >>= (\a -> Just $ a + 5) Nothing

Still with me? Good, because things are about to get magical.

Do Notation

Monads are so special, they have their own magical syntax! When working with monads, you may use do notation. What does do notation look like? Let’s take a look at a sample function:

justAdd :: (Num a) => a -> a -> Maybe a justAdd a b = Just $ a + b

This function takes 2 numbers, adds them, and wraps them up in a Maybe. Nothing earth shattering here. Let’s take a look at how to work with these using Bind:

justAddDemoBind = justAdd 2 2 >>= (justAdd 4) justAddDemoBindNothing = Nothing >>= (justAdd 4)

I’ve defined 2 simple functions here. The first calls justAdd 2 2, which returns Just 4. The function (justAdd 4) is then applied to it using the bind operator, which will return Just 8. The second attempts to apply the function (justAdd 4) to Nothing. Since the bind operator is smart enough to handle this, the final result of this function is Nothing. Simple, really. Now, let’s see do notation in action:

justAddDemoDo = do first <- justAdd 2 2 justAdd 4 first justAddDemoDoNothing = do first <- Nothing justAdd 4 first

Looks like a completely different language, right? In fact, these two functions do the exact same thing as the previous two. The purpose of do notation is to make working with Monads easier. In practice, if your logic is non-trivial, you end up with hugely nested statements. To see what’s going on, let’s break justAddDemoDo down:

justAddDemoDo = do

In the first line, we open our do block.

first <- justAdd 2 2

In the second line, we call justAdd 2 2, and assign the result to the name first. Notice that <- operator? That works basically the same as it does in list comprehensions, it does the assigning.

justAdd 4 first

Finally, we add 4 to first, resulting in Just 8, which is the value returned from the function. It seems like we’re treating our monad contained in first as a regular value. This is part of the magic of do notation. In fact, if this line were written as:justAdd first 4 it would have worked.

Another very important thing to note is that the last line of a do block must be an expression that returns a Monad! GHCI will throw a fit if it’s not.

The Pervasiveness Of Do

As you can see, under the hood, do notation just uses the >>= operator. You can also see that it is much simpler and cleaner looking than using the bind operator. However, that doesn’t mean that do is better. Like many things, it comes down to personal choice.

When reading literature on Haskell, you should be prepared to interpret do blocks, and usage of the bind operator. Like any tool, do and bind are not always the correct one. Picking the correct tool for the job is part of being a programmer. Hopefully this tutorial gave you enough familiarity to be able to use both.

The Point Of Applicative Functors

A few weeks ago, we talked about Functors. In case you’ve forgotten, you might want to click that link and read the post, but long story short: Functors allow you to call functions that take “bare” values on “dressed up” values.

Functors had one big weakness though: you can only call a function that takes one argument. I justified this using function currying, but there is another way. Today, we’ll be talking about that other way: Applicative functors.

Let’s Recap

Functors use a function called fmap to call a function on a functor. This function is called on the value within the functor, and a new functor containing the result is returned. This looks like this:

Prelude> fmap show $ Just 1 Just "1"

I fmap‘d show over Just 1, which returns Just "1". Let’s try another one:

Prelude> fmap (+ 1) $ Just 1 Just 2

Here, we’ve used function currying to fmap (+ 2) over Just 1, which returns Just 2. This is all fine and good in a contrived situation such as adding 1 to 1, but what about the real world? As you probably know, you often don’t have some nice constants ready to use in your function. There must be a better way…

Applicatives To The Rescue

To put it in layman’s terms, Applicative Functors are the middle ground. Let’s take a look at another example:

Prelude> let example = fmap (+) $ Just 1 Prelude> :t example example :: Maybe (Integer -> Integer)

If we fmap (+) over Just 1, the result is a function that takes an Integer and returns an Integer, wrapped in a Maybe. To do anything useful with this, you need to find a way to call this function on something else. That’s what Applicative Functors do for us. Let’s take a look at part of the class definition of Applicative:

class (Functor f) => Applicative f where pure :: a -> f a (<*>) :: f (a -> b) -> f a -> f b

We have two functions here: pure, which takes a value and wraps it in a minimal Functor, and (<*>), which takes a functor that contains a function, and calls it on another functor which contains a value, returning a value. In a nutshell, this allows us to call multi-argument functions with functors. There’s one more function exported by Control.Applicative that bears mentioning: (<$>). (<$>) is basically an alias for fmap that allows us to use fmap as infix:

Prelude> fmap (+1) $ Just 2 Just 3 Prelude> (+1) <$> Just 2 Just 3

As you can see, they function identically. Feel free to use (<$>), just know that it requires you to import Control.Applicative. Anyways, with that out of the way, here’s how you use an applicative. For this example, I will be adding 2 to 2. But 2 will be wrapped in a Maybe requiring extra steps:

Prelude> (+) <$> Just 2 <*> pure 2 Just 4

Let’s talk about what just happened. First, I fmap‘d (+) over Just 2 using the (<$>) operator. This returns a function that takes an Integer and returns an integer wrapped in a Maybe. Next I used the (<*>) operator to call that function on pure 2, which returns Just 4.

You may be wondering why I used pure 2 instead of Just 2. Sure, Just 2 would have worked, but pure 2 would have worked for any type of Applicative. Let’s take a look at another example:

Prelude> (+) <$> [1,2,99,100] <*> pure 2 [3,4,101,102]

As you can see, this is the exact same expression, except I used a List instead of a Maybe. This is where pure comes in handy. It allows you to write more generic code. pure works for any type of Applicative, therefore you needn’t concern yourself with what kind of Applicative is being passed into your function. You can rest easy knowing that it will work regardless.

Hopefully That All Made Sense

Applicatives are a pretty abstract concept, and it can be difficult to visualize how they can be useful. This is a topic that I don’t feel is documented very well. It’s a topic I had trouble with for a while.

Since I suspect that most readers will find this post with a google search term like “what is the point of applicative functors”, hopefully I’ve shed some light on the topic for you.

%d bloggers like this: