tag:blogger.com,1999:blog-86384740634644896512024-02-20T04:28:19.952-08:00Multi-paradigmThis blog is interested in imperative, functional, procedural, logic-based, and all sorts of ways of thinking about programming. I write mostly about C++, my bread-and-butter.
Recent articles have focussed around functional programming in C++; this is one paradigm C++ programmers often neglect. Many believe that it is not possible or efficient to do. I challenge this assertion by example! Splinter of Chaoshttp://www.blogger.com/profile/14715348728512729776noreply@blogger.comBlogger38125tag:blogger.com,1999:blog-8638474063464489651.post-13748986905690135272023-03-06T10:12:00.001-08:002023-03-06T10:12:07.493-08:00 Nippon Ichi And the Love of Grind<p><span style="font-family: Arial; font-size: 11pt; white-space: pre-wrap;">The RPG-ification of AAA titles has been going on for so long that it’s almost strange to see the rare game where you aren’t collecting some resource to get stronger and improve one attribute or another. The motivation is obvious: It gives a sense of progression, expression, replayability, and allows players to trade skill for time. It seems hard for AAA studios to release games that are difficult because challenge could dissuade players looking for a cheap power fantasy, but don’t have the time to invest in mastering the games they play. Publishers want the widest possible audiences so they want to know that players can continually make progress and enjoy the game, not get hung up on sudden difficulty spikes. Can’t get past a boss or this mission is too hard? A short little grind should suffice.</span></p><span id="docs-internal-guid-cb71327f-7fff-7770-a558-b4df682ff426"><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">Time, being a player’s most valuable resource, should not be taken for granted, which I think most developers are aware of. This is why enemy difficulty often scales with player level rather than becoming DPS checkpoints, and why games like The Division try to fill the map full with various EXP-awarding activities and new guns, and games like God of War carefully control the player’s power level at every step of the journey. “Grinding is just bad game design,” you might read on forums and social media. “Good” games use their RPG mechanics to give you as much power as you need, when you need it.</span></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">This does not appear to be how Nippon Ichi sees the world, but I have to take a slight detour for those unfamiliar with their works. Readers familiar with them may skip the following section.</span></p><h1 dir="ltr" style="line-height: 1.38; margin-bottom: 6pt; margin-top: 20pt;"><span style="font-family: Arial; font-size: 20pt; font-variant-east-asian: normal; font-variant-numeric: normal; font-weight: 400; vertical-align: baseline; white-space: pre-wrap;">Incredible Power Levels</span></h1><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">Disgaea 6 had this trailer that introduced a new level cap. RPG’s rarely go above 100, but this cap was 99,999,999. If you look at the experience calculations for a Disgaea game, you’ll find that the first 100 levels use a standard exponentially-increasing EXP requirement for each level, but then they switch to a different expression and then another later on. Most RPG’s </span><span style="font-family: Arial; font-size: 11pt; font-style: italic; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">have</span><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;"> to cap EXP so low because it would become exuberantly expensive to continue leveling up. Even if there is no cap that players might realistically reach, the </span><span style="font-family: Arial; font-size: 11pt; font-style: italic; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">content</span><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;"> must end before this happens.</span></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">But Disgaea doesn’t stop there, it has reincarnation. Much of the end-game of a Disgaea entry is spent reaching max level. Then you reincarnate. This reduces your unit’s level back to one, but retains some of the stats from its previous life. You max its level out again and then reincarnate again. </span></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">This is all, by the way, required for post-game content. The final, final boss is always Baal, an incredibly difficult encounter where only the strongest teams can even make it past the first turn of combat. And it takes, as you’ve probably guessed, a lot of grinding. Grinding to get to max level, reincarnating, and grinding some more.</span></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">Reincarnation and grinding are essential features in most of the Nippon Ichi games I’ve played, including: Disgaea, Makai Kingdom, Labyrinth of Refrain and its sequel, etc.. It’s just a company that loves the grind.</span></p><h1 dir="ltr" style="line-height: 1.38; margin-bottom: 6pt; margin-top: 20pt;"><span style="font-family: Arial; font-size: 20pt; font-variant-east-asian: normal; font-variant-numeric: normal; font-weight: 400; vertical-align: baseline; white-space: pre-wrap;">Grind Is Good</span></h1><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">On its surface, Nippon Ichi’s philosophy seems a step in the wrong direction, but the popularity of their franchises speak differently. Players of most RPGs will talk about the story, characters, and world, perhaps viewing combat as a way to space out the more interesting elements of the game, but players of Nippon Ichi games will talk about the gameplay and achievable power levels, saying “the game doesn’t </span><span style="font-family: Arial; font-size: 11pt; font-style: italic; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">really</span><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;"> start until the post-game.” They’ll talk about ways of gaining thousands of levels worth of experience very quickly after a reincarnation, how to create OP builds, how to use the game’s built-in cheat functions to gain money and experience quickly, etc..</span></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">Grind in a Nippon Ichi game isn’t the activity done between story beats, it </span><span style="font-family: Arial; font-size: 11pt; font-style: italic; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">is</span><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;"> the game. Their stories aren’t generally that bad and are full of humor, but they’re not the main attraction.</span></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">They accomplish this feat in two ways:</span></p><ol style="margin-bottom: 0; margin-top: 0; padding-inline-start: 48px;"><li aria-level="1" dir="ltr" style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; list-style-type: decimal; vertical-align: baseline; white-space: pre;"><p dir="ltr" role="presentation" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">Unparalleled depth.</span></p></li><li aria-level="1" dir="ltr" style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; list-style-type: decimal; vertical-align: baseline; white-space: pre;"><p dir="ltr" role="presentation" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">Seemingly endless content.</span></p></li></ol><h1 dir="ltr" style="line-height: 1.38; margin-bottom: 6pt; margin-top: 20pt;"><span style="font-family: Arial; font-size: 20pt; font-variant-east-asian: normal; font-variant-numeric: normal; font-weight: 400; vertical-align: baseline; white-space: pre-wrap;">Depth: It’s Prinnies All The Way Down.</span></h1><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">One of the first concepts players get introduced to in Disgaea is throwing your units around. A unit can only move a number of tiles according to their movement stat, but a unit can pick another up, throw it, and then the thrown unit can move much further. But prinnies are special. Each unit has an “evility” that might trigger at specific moments or add stats, and the prinny evility is to explode when thrown, dealing damage proportionate to health.</span></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">Nippon Ichi has a knack for introducing concepts and immediately turning them on their heads. In Labrynth of Refrain, you learn that to unlock doors, you need keys. Simple enough. Also you can break many, often most, of the walls in the dungeon, subverting the key/door dynamic. In Phantom Brave, you can pick rocks, bushes, and all sorts of stuff off the ground and use them to defeat enemies. Also you can carry them back to base and materialize new units from them. In Disgaea, you’ll want to constantly be equipping better gear, but also each item has a whole “item world” to explore and the item’s stats improve the deeper you go.</span></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">This depth allows Nippon Ichi games to sometimes be puzzles where you have to find out of the box solutions to strategy problems and sometimes be joyous grindfests where you figure out the most efficient way to increase in power to get beyond a challenge. Most of the time, the player gets to decide which way they want to handle the game. When I first played Labyrinth of Refrain, I had to think very carefully about my team compositions, but in playing its sequel I better understood how to use the stockpile system to keep overleveled throughout much of my run.</span></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">A particular moment in Disgaea 5 I keep coming back to in my mind: I was on this map with a bridge over a poisoned water river. Each turn in the river, units will take damage so obviously the bridge is the way to go, but the enemies on the bridge were a bit too powerful for my team. Then it occurred to me I could pick them up and throw them off the bridge to have them take poison damage while my team waited out the battle to clean up the remainders. It’s the kind of thing I could see players going onto forums and saying “cheese discovered in level, trivializes challenge, please fix,” but at the same time the feeling of discovery in that moment was strong enough that I remember it so many years later and the game wouldn’t feel so special to me were it not for the cheese.</span></p><h2 dir="ltr" style="line-height: 1.38; margin-bottom: 6pt; margin-top: 18pt;"><span style="font-family: Arial; font-size: 16pt; font-variant-east-asian: normal; font-variant-numeric: normal; font-weight: 400; vertical-align: baseline; white-space: pre-wrap;">More and More Content</span></h2><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">Disgaea 5 was the first Nippon Ichi game I actually bothered to finish. I, like many RPG players, often lose motivation towards the end and the massive amounts of time these games can take can make completing them difficult. It can mean not playing new releases from other studios or simply require more free time than we have.</span></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">So I saw the ending credits and jumped right back in to see this post-game that people were always talking about. Imagine my surprise to find it wasn’t just bonus challenge levels, but that the story itself kept going, new mechanics were still being introduced. Less content and longer between story intervals and far more grindy, but where the post-game of most games can feel like clean-up work, getting the last few collectibles and viewing scenes you might have missed, the post-game of a Nippon Ichi game is still just the game.</span></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">To many, that may seem daunting. RPGs are already hard to invest into because of the time commitment and the idea that one might not be able to finish makes starting seem pointless. It’s an understandable perspective, but these games really push the “destination is the journey” angle.</span></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">I have heard game writers in GDC talks lament that only a fraction of the people who play a game they work on will see the story conclude and how they write with this in mind, but as a designer the idea of players enjoying a game I worked on for the pure joy of its mechanical interaction or the aesthetic pleasure of interfacing with the controls is what motivates me. I made a toy box and it’s there whenever you want to pick it up and I hope that it never stops offering good experiences.</span></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">Perhaps that’s one of the reasons devs are enjoying making roguelites so much. They offer potentially endless experiences without the need for enticing players with intricate stories or designing finite galleries of content, they can randomly generate content infinitely and players can keep coming back to the same game and frequently find the experience at least a little different from before.</span></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">Well, before roguelites were much a thing, there were endless dungeons and Nippon Ichi found a good formula for it.</span></p><br /><h2 dir="ltr" style="line-height: 1.38; margin-bottom: 6pt; margin-top: 18pt;"><span style="font-family: Arial; font-size: 16pt; font-variant-east-asian: normal; font-variant-numeric: normal; font-weight: 400; vertical-align: baseline; white-space: pre-wrap;">Conclusions</span></h2><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">Nippon Ichi’s philosophy has been a personal inspiration and I adore their games, I love the grind. Players are often right to criticize games for being overly-grind focused when they trap players into repetitive and unfulfilling loops, becoming more work-like than play-like. Well, the criticism probably still stands against Nippon Ichi’s titles, but the difference is in how Ichi’s titles are built to continue to reward players for the act of playing itself. When the core thrust of player motivation is on a steady IV drip of story content then how unfulfilling the core gameplay is only becomes more apparent when the story slows down. But when the gameplay itself is what one draws pleasure from, none of that matters.</span></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">Embrace the grind. Love the grind.</span></p><div><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span></div></span>Splinter of Chaoshttp://www.blogger.com/profile/14715348728512729776noreply@blogger.com0tag:blogger.com,1999:blog-8638474063464489651.post-19594897019800635312022-07-27T22:00:00.001-07:002022-07-28T01:26:49.330-07:00 How Vampire Survivors Made Me Rethink The Concept of the "Core Gameplay Loop"<p><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">I first saw Vampire Survivor as demoed on Northernlion’s youtube channel and it struck me as “just another game”. The gameplay looked simplistic, the art looked rough, and the concept didn’t seem like a very big innovation. I didn’t </span><span style="font-family: Arial; font-size: 11pt; font-style: italic; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">really</span><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;"> look at it with any seriousness until I saw “20 Minutes Till Dawn” and “Spirit Hunters: Infinite Horde”. That’s when it occurred to me: this is kinda a big deal. Vampire Survivors only hit the internet in late 2021 and already it has many clones, often by relatively new studios, sometimes even veteran studios. These studios either decided that this should be a trend and would be so big that it was worth the months of development to try and ship a competitor, or were themselves so infatuated with Survivors’ concept that they needed to try their own ideas.</span></p><span id="docs-internal-guid-1f3b5dce-7fff-fc02-80e9-26bcd1bafd00"><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">Point is, this game is a pretty big deal and I don’t think I’m alone in having initially thought this game didn’t look like much. Its core gameplay loop seems hilariously simplistic: kill enemies to get gems to gain experience, level up, and kill more enemies. One could probably describe very well-respected roguelites like Slay the Spire and Rogue Legacy the same way, but I think many would also point out the boss battles, the lore, and how skill-based those games feel. I want to put a pin in the skill part for now and focus on bosses and lore, which are not necessarily a part of the </span><span style="font-family: Arial; font-size: 11pt; font-style: italic; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">core </span><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">loop, but I think are key to understanding Vampire Survivors.</span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"></p><div class="separator" style="clear: both; text-align: center;"><a href="https://cdn.discordapp.com/attachments/902270181374591016/989194116464267384/CIRCLE.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="749" data-original-width="800" height="414" src="https://cdn.discordapp.com/attachments/902270181374591016/989194116464267384/CIRCLE.png" width="442" /></a></div><br /><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><p></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">It took me quite a while to understand Vampire Survivors on a level deeper than “some people find ‘bullet heaven’ fun” and that led me to question the utility of thinking of core gameplay loops as simple loops.</span></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">I usually hear this discussed in terms of micro and macro; micro being what you’re doing now and macro being what will happen soon. For Vampire Survivors, the micro would be the loop I described above, but the macro is a bit more interesting. When you go into a stage, you might have a goal such as “explore a specific part of the map,” or “collect gold,” the meta-progression resource. You might be completing a challenge which will unlock some new content upon completion, some of which are to “evolve” a weapon (upgrade a weapon to the max level while having a specific item in your inventory). To accomplish this goal, you might have a specific build in mind. But just having goals or higher level objectives doesn’t seem like enough for a game to be </span><span style="font-family: Arial; font-size: 11pt; font-style: italic; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">special</span><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">. What’s the secret? It takes a moment to explain…</span></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">At the start of the game, you have just one weapon and every time you level up, you might gain an additional weapon which changes your build, get an item which enhances the build, or upgrade a weapon which scales your damage. As time goes on, waves get stronger and you need these upgrades in order to match your damage output to keep up with the enemy spawn rate which is also increasing over time, or at least be able to outrun them. On a good run, at some point, you will have outpaced the spawn rate and just mow enemies down–what is often referred to as “bullet heaven”, and from then on the player is unlikely to lose until the timer expires and the reaper comes out. Even from a very simple mechanical base, Vampire Survivors is able to evoke a number of different emotions in the player at different stages. It creates different feelings across each stage which makes them feel more compelling, less monotonous.</span></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">This is what I really missed when I looked at Vampire Survivors before, and what shows me the flaw of viewing a core gameplay loop as just a loop. You could look at it as a spiral, too. As the game continues, the loop of kill -> collect -> level -> kill changes and recontextualizes over time. At first, you are weak, but enemies are infrequent, collecting EXP is slow. As you gain another weapon, your ability to chase down enemies increases and you can gain EXP a little faster. Eventually the enemies start to come in faster than the player can kill them and there’s this push and pull as you need to run away to not get killed, but you also need to circle around to collect the EXP they drop. After a few level ups, you get a weapon evolved and become overpowered and nothing can touch you. Leveling up goes from just trying to gather a few extra weapons so you don’t get outpaced to trying to optimize the damage output as quickly as possible and trying to set up evolutions, to maximizing your weapons, to finalizing the build, and finally just cleaning up whatever upgrades are left to collect. Every part of the loop changes as you progress through the game and so after going a few cycles through it, you find yourself not actually in the same place as where you started.</span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"></p><div class="separator" style="clear: both; text-align: center;"><a href="https://cdn.discordapp.com/attachments/902270181374591016/989194359335424040/spiral.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="749" data-original-width="800" height="502" src="https://cdn.discordapp.com/attachments/902270181374591016/989194359335424040/spiral.png" width="537" /></a></div><br /><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><p></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">I’ll refer to this as the “core gameplay spiral”; the way that iterations around a core gameplay loop alters the dynamics as the game changes over time or just in general progresses. By breaking the ideas of the core loop, progression systems, and time apart, it becomes harder to understand how they relate to each other and especially treating the core gameplay loop as static discourages understanding how it evolves over time. The core actions you’re doing never change, but how they feel does change, as do the dynamics they create. I further posit that games will probably, generally, be more interesting when their core loop is </span><span style="font-family: Arial; font-size: 11pt; font-style: italic; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">not</span><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;"> static, and this extends to meta-progression, too. </span></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">The one backtrack I have to make is that the “core gameplay loop” is typically only a mechanics-deep analysis that does not consider extraneous elements like dynamics or progression. So maybe saying “core spiral” isn’t a great name because it suggests that the spiral is just the core loop unraveled. It is not, they are separate concepts for different layers of abstraction.</span></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">Earlier, I did briefly mention that I think one aspect of Vampire Survivors that might lead some to disregard it is that it appears to have a low skill floor; that is the minimum amount of skill required to be proficient at a game. Even the skill ceiling doesn't seem particularly high. But I hope it’s clear that this doesn’t actually matter very much since the progression of the core gameplay loop more than makes up for this. Both “Spirit Hunters: Infinite Horde” and “20 Minutes Till Dawn” add much more skill-based elements with their faster gameplay and “20 Minutes” even requiring that you manually fire your gun, yet I feel they just miss the point. Their core loops are static in comparison to Vampire Survivors. In “Spirit Hunters”, the stats of weapons can change, but the weapons themselves change very little, they don’t evolve, and only occasional pets will give global bufs to your hero. Every round has largely the same goal: collect gems (the meta-progression currency) to unlock more on the web (a very large skill tree, essentially). “20 Minutes” has more characters and weapons to unlock and you can freely mix and match them, but there is less to unlock, it all unlocks fairly quickly; and since you manually control the weapon and it always fires bullets from the same point, there are just less dramatic changes to the core loop throughout each run. They are both good games and some may prefer one or both over Vampire Survivors, but in my opinion they aren’t quite at the same level.</span></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">The insight I have gained from this analysis is that when I’m thinking about my own designs, I want to think primarily about this core gameplay spiral. One with a simple loop at its base that is expanded as the player spends time in the game and creates different dynamics at different points along the spiral. And what I want to avoid is thinking of the gameplay loop, micros and macros, and progression systems as parallel entities. As a practical example, while working on a deckbuilder roguelite, I asked myself “why does picking the second card feel different from picking the first?” and after doing so, I’d realized it did not. I had the progression systems, the loops, and all the features I thought made a good game, but I didn’t have decisions properly compounding on each other and thinking about how the loop should change over time pushed me to make some significant changes to my mechanics, hopefully for the better. And so I hope that this analysis might similarly help others see their own projects in a different way, ask the right questions, and find better solutions.</span></p><div><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span></div></span>Splinter of Chaoshttp://www.blogger.com/profile/14715348728512729776noreply@blogger.com0tag:blogger.com,1999:blog-8638474063464489651.post-56573660267666492742022-05-02T21:17:00.002-07:002022-05-02T21:17:38.069-07:00Game Design as a Dialogue<p><span style="font-family: Arial; font-size: 11pt; white-space: pre-wrap;">Media criticism is one of my favorite uses of Youtube and somehow, the platform unearthed a really great set of critics to choose from, for movies. There are Lindsay ellis , Folding Ideas, Filmento, CJ the X, Filmento, etc., but in games we have far fewer. GMTK is the current gold standard, then there are Adam Millard, Design Doc, Dunkey, Noah Cadwell, Psych of Play, and a few others strewn here and there. Furthermore, comparing channels focussing on game design or analysis vs movies and books, I always felt something was missing.</span></p><span id="docs-internal-guid-b4e5ce3a-7fff-e4bb-79fc-656407f3ef94"><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">For a case study of good media criticism, I want to look at Lindsay Ellis’s video essay, “</span><a href="https://www.youtube.com/watch?v=vpUx9DnQUkA" style="text-decoration-line: none;"><span style="color: #1155cc; font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; text-decoration-line: underline; text-decoration-skip-ink: none; vertical-align: baseline; white-space: pre-wrap;">That Time Disney Remade Beauty and the Beast</span></a><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">”. The essay discusses the history of Beauty and the Beast, differences in adaptations, social context, political context, how it fits within the Disney brand and how this impacted the motivations of the film, thus creative choices in its development. Then, with all this context, when we arrive at what is the core issues that Ellis takes with the plot, we not only gain a greater understanding of how the creative decisions affected the piece as a whole. but how those decisions form part of a dialogue that extends beyond the movie itself, This grants us new insights on the film and the culture that produced it.</span></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">It’s this dialogue that I feel is most absent from discussion of video game criticism and analysis. Many channels discussing design of a specific topic in games, like jumping physics, will happily discuss all the different ways jump physics may be implemented, and how that affects the way in which the different games play, much more rare is a discussion of the dialogue between different designers and between game makers and players of different games and how and why they chose different jump physics. The kind of analysis seen most often serves as more of a collection of useful ideas and it’s left up to the watcher to pick among the options they like. While this kind of content is unambiguously good and we could use more of it, we also deserve deeper analysis that engages with the larger dialogue, not just describes its existence. The problem is that without engaging in this dialogue, the viewer does not gain insights they could not have otherwise gained by just being aware of the games being discussed.</span></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">For an example, I’d like to use Design Doc’s “</span><a href="https://www.youtube.com/watch?v=ktogjiX3eI4" style="text-decoration-line: none;"><span style="color: #1155cc; font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; text-decoration-line: underline; text-decoration-skip-ink: none; vertical-align: baseline; white-space: pre-wrap;">How Do You Improve Turn Based Combat?</span></a><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">”. This video serves as a good 101 course in understanding what kinds of turn based systems have been experimented with and various ideas in it. DD does a good job of talking about how the systems work and how they affect the player, and what components go into turn based combat design. Again, this is unambiguously good content, but it doesn’t contain entirely novel ideas that add to the discourse or do much to explain why one game might have chosen one system over another.</span></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">For contrast, Hbomberguy’s (Harris Michael Brewis) more focussed video, “</span><a href="https://www.youtube.com/watch?v=AC3OuLU5XCw" style="text-decoration-line: none;"><span style="color: #1155cc; font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; text-decoration-line: underline; text-decoration-skip-ink: none; vertical-align: baseline; white-space: pre-wrap;">Bloodborne Is Genius, And Here's Why</span></a><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">” is one if the most influential discussions of game design, for me, personally. The central bit that always stuck out was his story of a friend of a friend who didn’t so much get the Dark Souls games, but perhaps learned to play them differently by Bloodborne conditioning them to approach them in a new way. This led to them having more fun. Through this story, we gain some insight that the way we play can affect our enjoyment of a game, and the how a game is structured can impact how we play. To get to this point, Brewis discusses the history of Dark Souls, how it relates to other games, personal motivations and ideas about games, references other works and discussions of game design. Importantly, one point I really don’t want to overlook is how Brewis includes the player in the dialogue by referencing how other players respond to Bloodborne and relate to it rather than speaking entirely from their singular perspective.</span></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">To add my own critique to the pool, I want to talk a bit about dodge/parry canceling and why it’s a very important, but frequently omitted feature of action games and Dark Souls (DS) in particular. DS is so synonymous with “difficult” that it is often considered the defining feature and almost any difficult game is inevitably compared to it. Hidetaka Miyazaki allegedly didn’t set out to make a hard game, telling </span><a href="https://metro.co.uk/2012/08/29/dark-souls-interview-hard-master-556118/" style="text-decoration-line: none;"><span style="color: #1155cc; font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; text-decoration-line: underline; text-decoration-skip-ink: none; vertical-align: baseline; white-space: pre-wrap;">metro.co.uk in an interview</span></a><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">, however</span></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">“I personally want my games to be described as satisfying rather than difficult. As a matter of fact, I am aiming at giving players sense of accomplishment in the use of difficulty.”</span></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">As Noah Cadwell points out in their video, “</span><a href="https://www.youtube.com/watch?v=O_KVCFxnpj4" style="text-decoration-line: none;"><span style="color: #1155cc; font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; text-decoration-line: underline; text-decoration-skip-ink: none; vertical-align: baseline; white-space: pre-wrap;">I Beat the Dark Souls Trilogy and All I Made Was This Lousy Video Essay</span></a><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">”, the series is more inviting to players of all skill ranges than the “git gud” crowd seems to think. </span></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">As a game synonymous with “difficult”, difficult games seem to have taken some of its design ideas from DS. The one I find most puzzling is the omission of dodge and parry canceling or even the omission of parrying altogether, not because DS doesn’t have parrying, but because it’s underutilized. </span></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">In DS, attacking involves patience, planning, spacing, and strategy because starting an attack locks you into its animation which can only be canceled after its active frames have completed. This gives DS a slower, more deliberate pace in contrast to the twitch reflex feel of more action-oriented games.</span></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">For twitch-based action games, dodge canceling encourages more aggressive behavior from the player since they can always hit the “get out of danger” button at any time. To incentivise more skilled, less risk-averse play, games like Beyonetta reward the player specifically for dodging at just the right time with some slow-mo where the player can deal significant damage before the enemy can respond. YS VIII gives slow-mo for perfect dodges and temporary invincibility for perfect parries and allows players to potentially have both buffs at once. A well-timed dodge feels good any day of the week, but many designers seem to have wanted to embellish this feeling with extra mechanical satisfaction.</span></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">When FromSoftware made Sekiro after DS3, dodge and parry canceling were indeed features, but the game had a different aesthetic. DS featured a dark, sinister, and cruel world where punishing the player for impatient and poorly-timed attacks added to this dark aesthetic. You are a nobody; a simple, clumsy soldier facing foes far above your weight class. While Sekiro features a similarly dark world, you play as a named protagonist with a refined fighting style, use stealth when it suits you, picking when and how to attack. DS wants you to feel powerless, Sekiro wants you to feel powerful and so it gives you a dodge cancel, even giving a high-damage, cinematic counter attack for a perfect parry.</span></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">Yet, many contemporary action games such as Death’s Door and Sifu are missing this. Why? Death’s Door in particular has combo strings where every attack in the string uses the same animation and the string is of an arbitrary length determined by the weapon’s upgrade level and cannot be canceled out of. The most I ever heard about why the combat system was designed the way was just a snippet from a noclip documentary where one of the team said for this kind of game, you want faster, instant attacks compared to some </span><span style="font-family: Arial; font-size: 11pt; font-style: italic; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">other</span><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;"> games. Was dodge cancellation discussed and omitted for specific reasons? Would the game be better or worse with it? Is this a holdover from Dark Souls’ legacy as part of some anti-dodge canceling movement in game design or an isolated decision?</span></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">Now, I hope it doesn’t sound like I’m negging on Death’s Door’s omission of dodge canceling for no reason, but I do think it’s an interesting example of a lacking in the discourse surrounding game design. When I hear people discuss the game, people will often reference the story, funny writing, art, music, creative bosses, and presentation while spending little time discussing the combat despite players spending a large portion of the game immersed in battles. And that’s kind of interesting in its own right. Perhaps the combat is as good as it needs to be, but not really a central focus of the player’s enjoyment, or perhaps the other elements of the game are just that good. That in itself gives us some ideas about what might be most important to players and how they view games. But designers spend a lot of time deliberating over small decisions like what features to include, what are must-haves and what might be stretch goals, and trying to figure out if they matter–if so, why?</span></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">Personally, I think the absence of dodge rolling in Death’s Door is interesting because it follows a trend in contemporary action games, but also in that it might suggest its roots in 2D Zelda games. Sifu references older beat ‘em up while feeling very modern (despite also feeling like PS2’s God Hand). Both games fit into modern gaming as having retro inspiration with modern twists, asking us to look back at gaming’s past to see if there’s anything we’ve missed before we continue forward.</span></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">This brings me to why I decided to write this piece. A dialogue exists between designers and between designers and players, but it is all too often not spoken. Someone makes a game, another developer sees this and is inspired to make their own, borrowing or stealing ideas, adding new ones, and over time the craft matures. Players develop an aesthetic sense of pleasurable and unpleasurable experiences, they discuss them on social media, they buy games or they don’t. This dialogue permeates all of gaming culture and the games industry, influencing what game makers make and what gamers play. Yet, when we discuss individual game design decisions, we often discuss them in a vacuum rather than as a part of a complex web of desires, motivations, aspirations, and ideas that play off each other. Every decision a games studio makes, from the color of a health bar to the nuances of the mechanics, are a part of a story that extends back to pong and further, and will continue being written as long as people make games.</span></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">So, when looking at a game and appreciating that it is fun, or has generally high production values, or just is interesting in some way, I would encourage the reader to think about this game as part of this greater dialogue. What does it say about how games should work? What does it add to the discussion? What may be left out? What were the intentions of the designer and what kind of experience did they want players to have?</span></p></span>Splinter of Chaoshttp://www.blogger.com/profile/14715348728512729776noreply@blogger.com0tag:blogger.com,1999:blog-8638474063464489651.post-66561234985930953882020-08-07T13:32:00.001-07:002020-08-07T13:32:09.491-07:00OpenGL tutorials can do better than raw pointer math.<p>
If you've ever followed an OpenGL tutorial, you probably saw code like
this:
</p>
<!--HTML generated using hilite.me-->
<div style="background: rgb(248, 248, 248); border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;">
<pre style="line-height: 125%; margin: 0px;"> <span style="color: #204a87; font-weight: bold;">float</span> <span style="color: black;">verts</span><span style="color: black; font-weight: bold;">[]</span> <span style="color: #ce5c00; font-weight: bold;">=</span> <span style="color: black; font-weight: bold;">{</span>
<span style="color: #8f5902; font-style: italic;">// Position TexCoords</span>
<span style="color: #ce5c00; font-weight: bold;">-</span><span style="color: black;">X</span><span style="color: black; font-weight: bold;">,</span> <span style="color: #ce5c00; font-weight: bold;">-</span><span style="color: black;">Y</span><span style="color: black; font-weight: bold;">,</span> <span style="color: black;">Z</span><span style="color: black; font-weight: bold;">,</span> <span style="color: #0000cf; font-weight: bold;">0.0f</span><span style="color: black; font-weight: bold;">,</span> <span style="color: #0000cf; font-weight: bold;">1.0f</span><span style="color: black; font-weight: bold;">,</span>
<span style="color: black;">X</span><span style="color: black; font-weight: bold;">,</span> <span style="color: #ce5c00; font-weight: bold;">-</span><span style="color: black;">Y</span><span style="color: black; font-weight: bold;">,</span> <span style="color: black;">Z</span><span style="color: black; font-weight: bold;">,</span> <span style="color: #0000cf; font-weight: bold;">1.0f</span><span style="color: black; font-weight: bold;">,</span> <span style="color: #0000cf; font-weight: bold;">1.0f</span><span style="color: black; font-weight: bold;">,</span>
<span style="color: black;">X</span><span style="color: black; font-weight: bold;">,</span> <span style="color: black;">Y</span><span style="color: black; font-weight: bold;">,</span> <span style="color: black;">Z</span><span style="color: black; font-weight: bold;">,</span> <span style="color: #0000cf; font-weight: bold;">1.0f</span><span style="color: black; font-weight: bold;">,</span> <span style="color: #0000cf; font-weight: bold;">0.0f</span><span style="color: black; font-weight: bold;">,</span>
<span style="color: #ce5c00; font-weight: bold;">-</span><span style="color: black;">X</span><span style="color: black; font-weight: bold;">,</span> <span style="color: black;">Y</span><span style="color: black; font-weight: bold;">,</span> <span style="color: black;">Z</span><span style="color: black; font-weight: bold;">,</span> <span style="color: #0000cf; font-weight: bold;">0.0f</span><span style="color: black; font-weight: bold;">,</span> <span style="color: #0000cf; font-weight: bold;">0.0f</span>
<span style="color: black; font-weight: bold;">};</span>
<span style="color: black;">GLuint</span> <span style="color: black;">vbo</span> <span style="color: #ce5c00; font-weight: bold;">=</span> <span style="color: black;">glGenBuffer</span><span style="color: black; font-weight: bold;">();</span>
<span style="color: black;">glBindBuffer</span><span style="color: black; font-weight: bold;">(</span><span style="color: black;">GL_ARRAY_BUFFER</span><span style="color: black; font-weight: bold;">,</span> <span style="color: black;">vbo</span><span style="color: black; font-weight: bold;">);</span>
<span style="color: black;">glBufferData</span><span style="color: black; font-weight: bold;">(</span><span style="color: black;">GL_ARRAY_BUFFER</span><span style="color: black; font-weight: bold;">,</span> <span style="color: #204a87; font-weight: bold;">sizeof</span><span style="color: black; font-weight: bold;">(</span><span style="color: black;">verts</span><span style="color: black; font-weight: bold;">),</span> <span style="color: black;">verts</span><span style="color: black; font-weight: bold;">,</span> <span style="color: black;">GL_STATIC_DRAW</span><span style="color: black; font-weight: bold;">);</span>
</pre>
</div>
<p>
So far not... terrible. The <i>verts</i> array may contain additional color
information or random other things, but it'll basically look like this. Later,
we have to tell the graphics card to actually draw this data, informing our
GLSL shaders how to read it:
</p>
<!--HTML generated using hilite.me-->
<div style="background: rgb(248, 248, 248); border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;">
<pre style="line-height: 125%; margin: 0px;"> <span style="color: black;">GLint</span> <span style="color: black;">vertex_pos_attrib</span> <span style="color: #ce5c00; font-weight: bold;">=</span> <span style="color: black;">glGetUniformLocation</span><span style="color: black; font-weight: bold;">(</span><span style="color: #4e9a06;">"vertex_pos"</span><span style="color: black; font-weight: bold;">);</span>
<span style="color: black;">glEnableVertexAttribArray</span><span style="color: black; font-weight: bold;">(</span>vertex_pos_attrib);</pre><pre style="line-height: 125%; margin: 0px;"> <span style="color: black;">glVertexAttribPointer</span><span style="color: black; font-weight: bold;">(</span><span style="color: black;">vertex_pos_attrib</span><span style="color: black; font-weight: bold;">,</span> <span style="color: #0000cf; font-weight: bold;">3</span><span style="color: black; font-weight: bold;">,</span> <span style="color: black;">GL_FLOAT</span><span style="color: black; font-weight: bold;">,</span>
<span style="color: black;">GL_FALSE</span><span style="color: black; font-weight: bold;">,</span> <span style="color: #0000cf; font-weight: bold;">5</span> <span style="color: #ce5c00; font-weight: bold;">*</span> <span style="color: #204a87; font-weight: bold;">sizeof</span><span style="font-weight: bold;">(</span><span style="color: #204a87; font-weight: bold;">float</span><span style="font-weight: bold;">)</span><span style="font-weight: bold;">,</span> <span style="font-weight: bold;">(</span><span style="color: #204a87; font-weight: bold;">void</span><span style="color: #ce5c00; font-weight: bold;">*</span><span style="font-weight: bold;">)</span><span style="color: #0000cf; font-weight: bold;">0</span><span style="font-weight: bold;">); // A</span></pre><pre style="line-height: 125%; margin: 0px;"> <span style="color: black;">GLint</span> <span style="color: black;">tex_coord_attrib</span> <span style="color: #ce5c00; font-weight: bold;">=</span> <span style="color: black;">glGetUniformLocation</span><span style="color: black; font-weight: bold;">(</span><span style="color: #4e9a06;">"tex_coord"</span><span style="color: black; font-weight: bold;">);</span>
<span style="color: black;">glEnableVertexAttribArray</span><span style="color: black; font-weight: bold;">(</span><span style="color: black;">tex_coord_attrib</span><span style="color: black; font-weight: bold;">);</span>
<span style="color: black;">glVertexAttribPointer</span><span style="color: black; font-weight: bold;">(</span><span style="color: black;">tex_coord_attrib</span><span style="color: black; font-weight: bold;">,</span> <span style="color: #0000cf; font-weight: bold;">2</span><span style="color: black; font-weight: bold;">,</span> <span style="color: black;">GL_FLOAT</span><span style="color: black; font-weight: bold;">,</span>
<span style="color: black;">GL_FALSE</span><span style="color: black; font-weight: bold;">,</span> <span style="color: #0000cf; font-weight: bold;">5</span> <span style="color: #ce5c00; font-weight: bold;">*</span> <span style="color: #204a87; font-weight: bold;">sizeof</span><span style="color: black; font-weight: bold;">(</span><span style="color: #204a87; font-weight: bold;">float</span><span style="color: black; font-weight: bold;">),</span> <span style="color: black; font-weight: bold;">(</span><span style="color: #204a87; font-weight: bold;">void</span><span style="color: #ce5c00; font-weight: bold;">*</span><span style="color: black; font-weight: bold;">)(</span><span style="color: #0000cf; font-weight: bold;">3</span> <span style="color: #ce5c00; font-weight: bold;">*</span> <span style="color: #204a87; font-weight: bold;">sizeof</span><span style="color: black; font-weight: bold;">(</span><span style="color: #204a87; font-weight: bold;">float</span><span style="color: black; font-weight: bold;">)));</span>
</pre>
</div>
<p>
and this is where these tutorials seem to be dropping the ball. The call I
marked with A is sending information to the graphics card that the GLSL
variable, vertex_pos, should be filled with two floats, or <span style="color: #0000cf; font-weight: bold;">3</span>
elements of data, a stride of <span style="color: #0000cf; font-weight: bold;">5</span> <span style="color: #ce5c00; font-weight: bold;">*</span> <span style="color: #204a87; font-weight: bold;">sizeof</span><span style="font-weight: bold;">(</span><span style="color: #204a87; font-weight: bold;">float</span><span style="font-weight: bold;">)</span> bytes between vertices, and <span style="color: #0000cf; font-weight: bold;">0</span> bytes offset
from the beginning of the vertices array buffer. The next call passes in
nearly identical information, but <span style="color: #0000cf; font-weight: 700;">2</span> elements and <span style="color: #0000cf; font-weight: bold;">3</span> <span style="color: #ce5c00; font-weight: bold;">*</span>
<span style="color: #204a87; font-weight: bold;">sizeof</span><span style="font-weight: bold;">(</span><span style="color: #204a87; font-weight: bold;">float</span><span style="font-weight: bold;">)</span> bytes from the beginning. The extra <span style="font-weight: bold;">(</span><span style="color: #204a87; font-weight: bold;">void</span><span style="color: #ce5c00; font-weight: bold;">*</span><span style="font-weight: bold;">)</span> cast is just converting the argument to the expected type.</p>
<p>This code is brittle for just too many reasons.</p>
<p></p>
<ol style="text-align: left;">
<li>
If vertex information is added or removed, all the pointer offset math has
to change.
</li>
<li>
The same goes for if the datatype changes, like from an int32 to int64.
</li>
<li>
If one forgets the <span style="color: #ce5c00; font-weight: bold;">*</span>
<span style="color: #204a87; font-weight: bold;">sizeof</span><span style="font-weight: bold;">(</span><span style="color: #204a87; font-weight: bold;">float</span><span style="font-weight: bold;">)</span> part, there could be trouble
when moving to other data types as one might guess the size in bytes wrong.
</li>
<li>The math gets more complicated if types of varying sizes are used.</li>
</ol>
<h2 style="text-align: left;">Just use a struct</h2>
<div>
<!--HTML generated using hilite.me-->
<div style="background: rgb(248, 248, 248); border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;">
<pre style="line-height: 125%; margin: 0px;"> <span style="color: #204a87; font-weight: bold;">struct</span> <span style="color: black;">Vertex</span> <span style="color: black; font-weight: bold;">{</span>
<span style="color: black;">GLfloat</span> <span style="color: black;">pos</span><span style="color: black; font-weight: bold;">[</span><span style="color: #0000cf; font-weight: bold;">3</span><span style="color: black; font-weight: bold;">];</span>
<span style="color: black;">GLfloat</span> <span style="color: black;">tex_coords</span><span style="color: black; font-weight: bold;">[</span><span style="color: #0000cf; font-weight: bold;">2</span><span style="color: black; font-weight: bold;">];</span>
<span style="color: black; font-weight: bold;">};</span>
<span style="color: #204a87; font-weight: bold;">float</span> <span style="color: black;">verts</span><span style="color: black; font-weight: bold;">[]</span> <span style="color: #ce5c00; font-weight: bold;">=</span> <span style="color: black; font-weight: bold;">{</span>
<span style="color: #8f5902; font-style: italic;">// Position TexCoords</span>
<span style="color: black; font-weight: bold;">{{</span><span style="color: #ce5c00; font-weight: bold;">-</span><span style="color: black;">X</span><span style="color: black; font-weight: bold;">,</span> <span style="color: #ce5c00; font-weight: bold;">-</span><span style="color: black;">Y</span><span style="color: black; font-weight: bold;">,</span> <span style="color: black;">Z</span><span style="color: black; font-weight: bold;">},</span> <span style="color: black; font-weight: bold;">{</span><span style="color: #0000cf; font-weight: bold;">0.0f</span><span style="color: black; font-weight: bold;">,</span> <span style="color: #0000cf; font-weight: bold;">1.0f</span><span style="color: black; font-weight: bold;">}},</span>
<span style="color: black; font-weight: bold;">{{</span> <span style="color: black;">X</span><span style="color: black; font-weight: bold;">,</span> <span style="color: #ce5c00; font-weight: bold;">-</span><span style="color: black;">Y</span><span style="color: black; font-weight: bold;">,</span> <span style="color: black;">Z</span><span style="color: black; font-weight: bold;">},</span> <span style="color: black; font-weight: bold;">{</span><span style="color: #0000cf; font-weight: bold;">1.0f</span><span style="color: black; font-weight: bold;">,</span> <span style="color: #0000cf; font-weight: bold;">1.0f</span><span style="color: black; font-weight: bold;">}},</span>
<span style="color: black; font-weight: bold;">{{</span> <span style="color: black;">X</span><span style="color: black; font-weight: bold;">,</span> <span style="color: black;">Y</span><span style="color: black; font-weight: bold;">,</span> <span style="color: black;">Z</span><span style="color: black; font-weight: bold;">},</span> <span style="color: black; font-weight: bold;">{</span><span style="color: #0000cf; font-weight: bold;">1.0f</span><span style="color: black; font-weight: bold;">,</span> <span style="color: #0000cf; font-weight: bold;">0.0f</span><span style="color: black; font-weight: bold;">}},</span>
<span style="color: black; font-weight: bold;">{{</span><span style="color: #ce5c00; font-weight: bold;">-</span><span style="color: black;">X</span><span style="color: black; font-weight: bold;">,</span> <span style="color: black;">Y</span><span style="color: black; font-weight: bold;">,</span> <span style="color: black;">Z</span><span style="color: black; font-weight: bold;">},</span> <span style="color: black; font-weight: bold;">{</span><span style="color: #0000cf; font-weight: bold;">0.0f</span><span style="color: black; font-weight: bold;">,</span> <span style="color: #0000cf; font-weight: bold;">0.0f</span><span style="color: black; font-weight: bold;">}}</span>
<span style="color: black; font-weight: bold;">};</span>
</pre>
</div>
</div>
<div><br /></div>
<div>
Now our data is more organized. We can share information about the binary
layout of our code with other parts of our program, write functions for
generating points, constructors, and all sorts of good stuff. We can
even use other, less basic data types, in theory. My code uses
<i><a href="https://glm.g-truc.net/0.9.9/index.html">glm::vec3</a>s </i>here,
for example.
</div>
<div><br /></div>
<div>
We need the corresponding <i>glVertexAttribPointer</i> calls to change a
bit, too. I've seen a number of attempts at this, sometimes with the idea that
one can take a member pointer and deference it at <i>null</i> in order to
get the offset, then convert it back into a pointer. Or one could just
use <a href="https://en.cppreference.com/w/cpp/types/offsetof"><i>offsetof()</i></a>. Finally, we can achieve something like this:
</div>
<div><br /></div>
<div>
<!--HTML generated using hilite.me-->
<div style="background: rgb(248, 248, 248); border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;">
<pre style="line-height: 125%; margin: 0px;"><span style="color: black;">glVertexAttribPointer</span><span style="color: black; font-weight: bold;">(</span><span style="color: black;">tex_coord_attrib</span><span style="color: black; font-weight: bold;">,</span> </pre><pre style="line-height: 125%; margin: 0px;"><span style="color: #204a87; font-weight: bold;"> sizeof</span><span style="color: black; font-weight: bold;">(</span><span style="color: black;">Vertex</span><span style="color: #ce5c00; font-weight: bold;">::</span><span style="color: black;">tex_coords</span><span style="color: black; font-weight: bold;">) / </span><span style="color: #204a87; font-weight: bold;">sizeof</span><span style="font-weight: bold;">(</span>GLfloat)<span style="font-weight: bold;">,</span> <span>GL_FLOAT</span><span style="font-weight: bold;">,</span> </pre><pre style="line-height: 125%; margin: 0px;"> <span style="color: black;">GL_FALSE</span><span style="color: black; font-weight: bold;">,</span></pre><pre style="line-height: 125%; margin: 0px;"><span style="color: #204a87; font-weight: bold;"> sizeof</span><span style="color: black; font-weight: bold;">(</span><span style="color: black;">Vertex</span><span style="color: black; font-weight: bold;">),</span></pre><pre style="line-height: 125%; margin: 0px;"><span style="color: black; font-weight: bold;"> (</span><span style="color: #204a87; font-weight: bold;">void</span><span style="color: #ce5c00; font-weight: bold;">*</span><span style="color: black; font-weight: bold;">)</span><span style="color: black;">offsetof</span><span style="color: black; font-weight: bold;">(</span><span style="color: black;">Vertex</span><span style="color: black; font-weight: bold;">,</span> <span style="color: black;">tex_coords</span><span style="color: black; font-weight: bold;">));</span>
</pre>
</div>
</div>
<div><br /></div>
<div>and we're pretty close to something approaching ideal. It's unfortunate that we have to <i>know</i> that <i>Vertex::tex_coords</i> is made up of <i>GLfloat</i>s, though. It could be advisable to define a global, <i>constexpr unsigned int D = 3</i>, which is used both instead of hardcoding in the <i>Vertex</i> definition and here instead of <i>sizeof()</i>, but the gains are marginal since we still have to pass in <i>GL_FLOAT</i> as the next parameter</div><div><br /></div><div>Still, to improve further...</div>
<h2 style="text-align: left;">Generalize</h2>
<div>This is the actual code I use:</div>
<div><br /></div>
<div>
<!--HTML generated using hilite.me-->
<div style="background: rgb(248, 248, 248); border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;">
<pre style="line-height: 125%; margin: 0px;"><span style="color: #204a87; font-weight: bold;">template</span><span style="color: #ce5c00; font-weight: bold;"><></span>
<span style="color: #204a87; font-weight: bold;">struct</span> <span style="color: black;">GlTraits</span><span style="color: #ce5c00; font-weight: bold;"><</span><span style="color: black;">GLfloat</span><span style="color: #ce5c00; font-weight: bold;">></span> <span style="color: black; font-weight: bold;">{</span>
<span style="color: #204a87; font-weight: bold;">static</span> <span style="color: black;">constexpr</span> <span style="color: #204a87; font-weight: bold;">auto</span> <span style="color: black;">glType</span> <span style="color: #ce5c00; font-weight: bold;">=</span> <span style="color: black;">GL_FLOAT</span><span style="color: black; font-weight: bold;">;</span>
<span style="color: black; font-weight: bold;">};</span>
<span style="color: #204a87; font-weight: bold;">template</span><span style="color: #ce5c00; font-weight: bold;"><</span><span style="color: #204a87; font-weight: bold;">typename</span> <span style="color: black;">RealType</span><span style="color: black; font-weight: bold;">,</span> <span style="color: #204a87; font-weight: bold;">typename</span> <span style="color: black;">T</span><span style="color: black; font-weight: bold;">,</span> <span style="color: #204a87; font-weight: bold;">typename</span> <span style="color: black;">Mem</span><span style="color: #ce5c00; font-weight: bold;">></span>
<span style="color: #204a87; font-weight: bold;">inline</span> <span style="color: #204a87; font-weight: bold;">void</span> <span style="color: black;">vertexAttribPointer</span><span style="color: black; font-weight: bold;">(</span><span style="color: black;">GLuint</span> <span style="color: black;">index</span><span style="color: black; font-weight: bold;">,</span> <span style="color: black;">GLboolean</span> <span style="color: black;">normalized</span><span style="color: black; font-weight: bold;">,</span>
<span style="color: #204a87; font-weight: bold;">const</span> <span style="color: black;">Mem</span> <span style="color: black;">T</span><span style="color: #ce5c00; font-weight: bold;">::*</span><span style="color: black;">mem</span><span style="color: black; font-weight: bold;">)</span> <span style="color: black; font-weight: bold;">{</span>
<span style="color: #8f5902; font-style: italic;">// This is basically C's offsetof() macro generalized to member pointers.</span>
<span style="color: black;">RealType</span><span style="color: #ce5c00; font-weight: bold;">*</span> <span style="color: black;">pointer</span> <span style="color: #ce5c00; font-weight: bold;">=</span> <span style="color: black; font-weight: bold;">(</span><span style="color: black;">RealType</span><span style="color: #ce5c00; font-weight: bold;">*</span><span style="color: black; font-weight: bold;">)</span><span style="color: #ce5c00; font-weight: bold;">&</span><span style="color: black; font-weight: bold;">(((</span><span style="color: black;">T</span><span style="color: #ce5c00; font-weight: bold;">*</span><span style="color: black; font-weight: bold;">)</span><span style="color: black;">nullptr</span><span style="color: black; font-weight: bold;">)</span><span style="color: #ce5c00; font-weight: bold;">->*</span><span style="color: black;">mem</span><span style="color: black; font-weight: bold;">);</span>
<span style="color: black;">glVertexAttribPointer</span><span style="color: black; font-weight: bold;">(</span><span style="color: black;">index</span><span style="color: black; font-weight: bold;">,</span> <span style="color: #204a87; font-weight: bold;">sizeof</span><span style="color: black; font-weight: bold;">(</span><span style="color: black;">Mem</span><span style="color: black; font-weight: bold;">)</span> <span style="color: #ce5c00; font-weight: bold;">/</span> <span style="color: #204a87; font-weight: bold;">sizeof</span><span style="color: black; font-weight: bold;">(</span><span style="color: black;">RealType</span><span style="color: black; font-weight: bold;">),</span>
<span style="color: black;">GlTraits</span><span style="color: #ce5c00; font-weight: bold;"><</span><span style="color: black;">RealType</span><span style="color: #ce5c00; font-weight: bold;">>::</span><span style="color: black;">glType</span><span style="color: black; font-weight: bold;">,</span> <span style="color: black;">normalized</span><span style="color: black; font-weight: bold;">,</span> <span style="color: #204a87; font-weight: bold;">sizeof</span><span style="color: black; font-weight: bold;">(</span><span style="color: black;">T</span><span style="color: black; font-weight: bold;">),</span>
<span style="color: black;">pointer</span><span style="color: black; font-weight: bold;">);</span>
<span style="color: black; font-weight: bold;">}</span>
</pre>
</div>
</div>
<div><br /></div>
<div>
<i>Mem</i> here is often going to be a <i>glm::vec3</i> or something
similar, <i>T</i> being my vertex class. Unfortunately,
<i>RealType</i> needs to be passed in as an explicit type since neither
<i>Mem</i> nor <i>T</i> suggest what the OpenGL type actually is.
Theoretically, if this library knew what a <i>glm::vec3</i> was, it could
also deduce the real type from the parameter, or one could specialize
<i>GlTraits</i> for it and include a
<i>typedef</i> or<i> using</i> statement that gave the "real"
type.
</div>
<div><br /></div>
<div>I use the function like this:</div>
<div><br /></div>
<div>
<!--HTML generated using hilite.me-->
<div style="background: rgb(248, 248, 248); border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;">
<pre style="line-height: 125%; margin: 0px;"> <span style="color: black;">vertexAttribPointer</span><span style="color: #ce5c00; font-weight: bold;"><</span><span style="color: #204a87; font-weight: bold;">float</span><span style="color: #ce5c00; font-weight: bold;">></span><span style="color: black; font-weight: bold;">(</span><span style="color: black;">tex_coord_attrib</span><span style="color: black; font-weight: bold;">,</span> <span style="color: black;">GL_FALSE</span><span style="color: black; font-weight: bold;">,</span> <span style="color: #ce5c00; font-weight: bold;">&</span><span style="color: black;">Vertex</span><span style="color: #ce5c00; font-weight: bold;">::</span><span style="color: black;">tex_coord</span><span style="color: black; font-weight: bold;">);</span>
<span style="color: black;">vertexAttribPointer</span><span style="color: #ce5c00; font-weight: bold;"><</span><span style="color: #204a87; font-weight: bold;">float</span><span style="color: #ce5c00; font-weight: bold;">></span><span style="color: black; font-weight: bold;">(</span><span style="color: black;">vertex_pos_attrib</span><span style="color: black; font-weight: bold;">,</span> <span style="color: black;">GL_FALSE</span><span style="color: black; font-weight: bold;">,</span> <span style="color: #ce5c00; font-weight: bold;">&</span><span style="color: black;">Vertex</span><span style="color: #ce5c00; font-weight: bold;">::</span><span style="color: black;">pos</span><span style="color: black; font-weight: bold;">);</span>
</pre>
</div>
</div>
<div><br /></div>
<div>Simple!</div><h2 style="text-align: left;">Conclusion</h2><div>Many OpenGL tutorials are suited for younger, less experienced programmers and so--giving their authors some credit--writers may not want to confuse readers with language features their readers hadn't seen before, like <i>offsetof()</i>, but I think this does a disservice to beginners as these tutorials are also authoritative on how good, well-written OpenGL code looks. They may not want to go into the template madness that I enjoy, but even tutorials targeting C or C++ can be significantly improved just by using a data structure for vertex information. Beginners should be taught to write resilient programs that resists the urge to hardcode sizes, types, and offsets based on their specifications at the time of writing and instead rely on the compiler more.</div><div><br /></div><div>Brittle code can punish beginners when they try to expand their knowledge as the slightest changes can cause their screen to stop rendering, or render odd artifacts and this can be difficult to debug. At worst, it will cause segfaults.</div>
<p></p>
Splinter of Chaoshttp://www.blogger.com/profile/14715348728512729776noreply@blogger.com2tag:blogger.com,1999:blog-8638474063464489651.post-88814803597724019422020-07-29T11:46:00.003-07:002020-07-31T13:18:49.404-07:00The Entity Component System<div>
Note: code highlighting thanks to <a href="http://hilite.me/">http://hilite.me/</a>
</div>
<div><br /></div>
<div>
Note2: This blog is primarily on template meta-programming--when the
opportunity presents--using it to solve practical problems. This post assumes
knowledge of <a href="https://en.cppreference.com/w/cpp/language/fold">C++17 fold expressions</a>. If the reader is unfamiliar, this may not be the best learning material,
but if the reader gets the idea and wants a practical example, please read on.
</div>
<div><br /></div>
<div>
Update July 30 2020: I realized a bit late a bug where non-const values
weren't always respect in the final solution. I haven't tracked it down yet so
just be warned.
</div>
<div><br /></div>
<div>Update July31: Fixed a bug that caused an infinite loop.</div>
<div><br /></div>
When I started programming, I made a simple game called <a href="https://github.com/splinterofchaos/Gravity-Battle">Orbital Chaos</a> and the code was organized very simply.
<div><br /></div>
<!--HTML generated using hilite.me-->
<div style="background: rgb(248, 248, 248); border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;">
<pre style="line-height: 125%; margin: 0px;"><span style="color: #204a87; font-weight: bold;">class</span> <span style="color: black;">Actor</span> <span style="color: black; font-weight: bold;">{</span>
<span style="color: #8f5902; font-style: italic;">// ...</span>
<span style="color: black; font-weight: bold;">};</span>
<span style="color: #204a87; font-weight: bold;">class</span> <span style="color: black;">Player</span> <span style="color: #ce5c00; font-weight: bold;">:</span> <span style="color: #204a87; font-weight: bold;">public</span> <span style="color: black;">Actor</span> <span style="color: black; font-weight: bold;">{</span>
<span style="color: #8f5902; font-style: italic;">// ...</span>
<span style="color: black; font-weight: bold;">};</span>
<span style="color: #204a87; font-weight: bold;">class</span> <span style="color: black;">Enemy</span> <span style="color: #ce5c00; font-weight: bold;">:</span> <span style="color: #204a87; font-weight: bold;">public</span> <span style="color: black;">Actor</span> <span style="color: black; font-weight: bold;">{</span>
<span style="color: #8f5902; font-style: italic;">// ...</span>
<span style="color: black; font-weight: bold;">};</span>
</pre>
<pre style="line-height: 125%; margin: 0px;"><span style="color: black; font-weight: bold;"><br /></span></pre>
</div>
<div><br /></div>
<div>
So each thing that an actor could do was represented in the base class and the
derived classes overrode that behavior in classic OOP style. Those were
simpler days.
</div>
<div><br /></div>
<div>
I decided to rewrite my code to what I now understand is a fairly
industry-standard idea: The entity component system or ECS for short. From
what I gather, an ECS is a simple system where entities are merely ID's with a
set of components which define all relevant information important to that
entity. For example, its <i><X,Y></i> coordinates in the 2D space
of a 2d game, the type of an entity (<i>{TREE, ITEM, PLAYER, etc...}</i>),
perhaps information on how to draw it... basically anything that would have
gone in <i>Actor</i> or <i>Player</i>.
</div>
<div><br /></div>
<div>
Proponents of ECS systems will argue for being a better alternative for OOP,
not requiring the base class to consider every possible action or attribute in
the game and cache efficiency--storing like information close together so that
iterations over all entities hopefully involve less cache misses. Sometimes
it's also argued that using ID's to relate objects to each other is much safer
than direct pointers. The alternative is
often <i>std::shared_ptr</i> with locking. Lastly, entities are
easier to extend, mix, match, and define than objects, not having to worry
about hierarchies, mix-ins, and multiple inheritance to get the desired
behavior.
</div>
<div><br /></div>
<div>
Personally, I think writing a base class that considers every possible action
is not good OOP, but doing it the <i>"right"</i> way is probably less
appealing than the data-oriented ECS style. OOP works great for abstract
objects with variations on behavior, but game entities tend to vary more on
configuration. I think this simplicity sometimes comes at the cost of having
to reinvent mix-ins and hierarchies, but one can still put pointers and
virtual in an ECS by storing pointers.
</div>
<div><br /></div>
<div>
Information on ECS's on the internet varies in quality as do their
implementations. I won't try to argue that I will present an objectively
better implementation than any others, these are just my ideas and the story
of my process to write one.
</div>
<div><br /></div>
<h3 style="text-align: left;">Ideas I didn't like</h3>
<div>
<!--HTML generated using hilite.me-->
<div style="background: rgb(248, 248, 248); border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;">
<pre style="line-height: 125%; margin: 0px;"><span style="color: #204a87; font-weight: bold;">class</span> <span style="color: black;">Entity</span> <span style="color: black; font-weight: bold;">{</span>
<span style="color: black;">GraphicsComponent</span> <span style="color: black;">gfx_component</span><span style="color: black; font-weight: bold;">;</span>
<span style="color: black;">PhysicsComponent</span> <span style="color: black;">phys_component</span><span style="color: black; font-weight: bold;">;</span>
<span style="color: black;">EtcComponent</span> <span style="color: black;">etc</span><span style="color: black; font-weight: bold;">;</span>
<span style="color: #8f5902; font-style: italic;">// ...</span>
<span style="color: black; font-weight: bold;">};</span>
</pre>
</div>
</div>
<div><br /></div>
<div>
I think this is closer to the <i>"right"</i> OOP solution as it
encapsulates the complexity into smaller pieces which can then be as simple or
intricate as necessary. In practice, the segmentation of code logic and how
systems relate to each other may become unclear. In a grid-based game like
mine, entities will have an integer <i><X,Y></i> position on
the grid which maps to a very different position when rendering to the screen
which may even by offset slightly by animations. How do components refer to
each other? It encourages using <i>virtual</i> functions to customize
behavior, which is very tidy, but also involve a lot of boilerplate code and
isn't incredibly efficient. Lastly, it doesn't store like data together.
</div>
<div><br /></div>
<div>
There's also the idea of having the <i>Entity</i> store a
<i>vector</i> of component objects and it's one of the more common
implementations I see, but this has the additional issue that one class needs
to consider all the complexity of the game. For example, if each component
object has a <i>init()</i>, <i>update()</i>, <i>etc()</i> functions,
they have to understand how to update themselves and interact with the world,
but the world needs to understand how to contain them leading to two mutually
referential system. That can be fine, but in my previous attempt, I found this
really complicated the definition of more interesting components.
</div>
<div><br /></div>
<div>
<!--HTML generated using hilite.me-->
<div style="background: rgb(248, 248, 248); border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;">
<pre style="line-height: 125%; margin: 0px;"><span style="color: #204a87; font-weight: bold;">using</span> <span style="color: black;">ComponentId</span> <span style="color: #ce5c00; font-weight: bold;">=</span> <span style="color: #204a87; font-weight: bold;">unsigned</span> <span style="color: #204a87; font-weight: bold;">int</span><span style="color: black; font-weight: bold;">;</span></pre>
<pre style="line-height: 125%; margin: 0px;"><br /></pre>
<pre style="line-height: 125%; margin: 0px;"><pre style="line-height: 16.25px; margin-bottom: 0px; margin-top: 0px;"><span style="color: #204a87; font-weight: bold;">template</span><span style="color: #ce5c00; font-weight: bold;"><</span><span style="color: #204a87; font-weight: bold;">typename</span> <span>T</span><span style="color: #ce5c00; font-weight: bold;">></span>
<span style="color: #204a87; font-weight: bold;">struct</span> <span>ComponentData</span> <span style="font-weight: bold;">{</span>
<span>EntityId</span> <span>id</span><span style="font-weight: bold;">;</span>
<span>T</span> <span>data</span><span style="font-weight: bold;">;</span>
<span>ComponentData</span><span style="font-weight: bold;">(</span><span>EntityId</span> <span>id</span><span style="font-weight: bold;">,</span> <span>T</span> <span>data</span><span style="font-weight: bold;">)</span>
<span style="color: #ce5c00; font-weight: bold;">:</span> <span>id</span><span style="font-weight: bold;">(</span><span>id</span><span style="font-weight: bold;">),</span> <span>data</span><span style="font-weight: bold;">(</span><span>std</span><span style="color: #ce5c00; font-weight: bold;">::</span><span>move</span><span style="font-weight: bold;">(</span><span>data</span><span style="font-weight: bold;">))</span> <span style="font-weight: bold;">{</span> <span style="font-weight: bold;">}</span>
<span style="font-weight: bold;">};</span></pre>
<span style="color: #204a87; font-weight: bold;">class</span> <span style="color: black;">Buffer</span> <span style="color: black; font-weight: bold;">{</span>
<span style="color: #204a87; font-weight: bold;">char</span><span style="color: #ce5c00; font-weight: bold;">*</span> <span style="color: black;">data</span> <span style="color: #ce5c00; font-weight: bold;">=</span> <span style="color: black;">nullptr</span><span style="color: black; font-weight: bold;">;</span>
<span style="color: #8f5902; font-style: italic;">// ...</span>
<span style="color: black; font-weight: bold;">};</span>
<span style="color: #204a87; font-weight: bold;">class</span> <span style="color: black;">EntityComponentSystem</span> <span style="color: black; font-weight: bold;">{</span>
<span style="color: black;">std</span><span style="color: #ce5c00; font-weight: bold;">::</span><span style="color: black;">unordered_map</span><span style="color: #ce5c00; font-weight: bold;"><</span><span style="color: black;">ComponentId</span><span style="color: black; font-weight: bold;">,</span> <span style="color: black;">Buffer</span><span style="color: #ce5c00; font-weight: bold;">></span> <span style="color: black;">components</span><span style="color: black; font-weight: bold;">;</span>
<span style="color: black; font-weight: bold;">};</span>
</pre>
</div>
<div><br /></div>
<div>
I think this idea gets closer. Each component type has a unique
<i>ComponentId</i> which is determined by the type system through
template magic and each <i>Buffer</i> holds an array of these
components that is manually managed by itself since
<i>std::</i> classes like <i>vector</i> need to know what type it
contains and how large it is at compile time. This
<i>std::unordered_map</i> could be replaced by an
<i>std::array</i> if the number of different component types are
known in advance.
</div>
<div><br /></div>
<div>
Actually, if the component types themselves are known and I happen to love
tuples, we could just do this:
</div>
<div><br /></div>
<div>
<!--HTML generated using hilite.me-->
<div style="background: rgb(248, 248, 248); border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;">
<pre style="line-height: 125%; margin: 0px;"><span style="color: #204a87; font-weight: bold;">template</span><span style="color: #ce5c00; font-weight: bold;"><</span><span style="color: #204a87; font-weight: bold;">typename</span> <span style="color: black;">T</span><span style="color: #ce5c00; font-weight: bold;">></span>
<span style="color: #204a87; font-weight: bold;">struct</span> <span style="color: black;">ComponentData</span> <span style="color: black; font-weight: bold;">{</span>
<span style="color: black;">EntityId</span> <span style="color: black;">id</span><span style="color: black; font-weight: bold;">;</span>
<span style="color: black;">T</span> <span style="color: black;">data</span><span style="color: black; font-weight: bold;">;</span>
<span style="color: black;">ComponentData</span><span style="color: black; font-weight: bold;">(</span><span style="color: black;">EntityId</span> <span style="color: black;">id</span><span style="color: black; font-weight: bold;">,</span> <span style="color: black;">T</span> <span style="color: black;">data</span><span style="color: black; font-weight: bold;">)</span>
<span style="color: #ce5c00; font-weight: bold;">:</span> <span style="color: black;">id</span><span style="color: black; font-weight: bold;">(</span><span style="color: black;">id</span><span style="color: black; font-weight: bold;">),</span> <span style="color: black;">data</span><span style="color: black; font-weight: bold;">(</span><span style="color: black;">std</span><span style="color: #ce5c00; font-weight: bold;">::</span><span style="color: black;">move</span><span style="color: black; font-weight: bold;">(</span><span style="color: black;">data</span><span style="color: black; font-weight: bold;">))</span> <span style="color: black; font-weight: bold;">{</span> <span style="color: black; font-weight: bold;">}</span>
<span style="color: black; font-weight: bold;">};</span>
<span style="color: #204a87; font-weight: bold;">using</span> <span style="color: black;">EntityId</span> <span style="color: #ce5c00; font-weight: bold;">=</span> <span style="color: #204a87; font-weight: bold;">unsigned</span> <span style="color: #204a87; font-weight: bold;">int</span><span style="color: black; font-weight: bold;">;</span>
<span style="color: #204a87; font-weight: bold;">template</span><span style="color: #ce5c00; font-weight: bold;"><</span><span style="color: #204a87; font-weight: bold;">typename</span><span style="color: black; font-weight: bold;">...</span><span style="color: black;">Components</span><span style="color: #ce5c00; font-weight: bold;">></span>
<span style="color: #204a87; font-weight: bold;">class</span> <span style="color: black;">EntityComponentSystem</span> <span style="color: black; font-weight: bold;">{</span>
<span style="color: black;">std</span><span style="color: #ce5c00; font-weight: bold;">::</span><span style="color: black;">tuple</span><span style="color: #ce5c00; font-weight: bold;"><</span><span style="color: black;">std</span><span style="color: #ce5c00; font-weight: bold;">::</span><span style="color: black;">vector</span><span style="color: #ce5c00; font-weight: bold;"><</span><span style="color: black;">ComponentData</span><span style="color: #ce5c00; font-weight: bold;"><</span><span style="color: black;">Components</span><span style="color: #ce5c00; font-weight: bold;">>></span><span style="color: black; font-weight: bold;">...</span><span style="color: #ce5c00; font-weight: bold;">></span> <span style="color: black;">components</span><span style="color: black; font-weight: bold;">;</span>
<span style="color: black; font-weight: bold;">};</span>
</pre>
</div>
</div>
<div><br /></div>
<div>
and that's what I ended up with in my first iteration (<a href="https://github.com/splinterofchaos/srpg/blob/fb66e25ce6f420c29d6a90cc676631a8a9eb4ce6/ecs.h">link</a>). One big and notable downside is that now, if I have a grid position, for
which I use my custom <i>Vec<int></i> class, and a graphical
position, which uses a <i>Vec<int></i> as well, they'll be stored
as the same component. It's tedious, but I implemented
<i>GraphicalPosition</i> and <i>GridPosition</i> classes to
differentiate.
</div>
<div><br /></div>
<div>
It also means that the declaration of the ECS type needs to contain every
component used in the game, but I don't find that
<i>too</i> problematic.
</div>
<div><br /></div>
<div>
From the above idea, we could construct a simple interface like this:
</div>
<div><br /></div>
<div>
<!--HTML generated using hilite.me-->
<div style="background: rgb(248, 248, 248); border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;">
<pre style="line-height: 125%; margin: 0px;"> <span style="color: black;">EntityComponentSystem</span><span style="color: #ce5c00; font-weight: bold;"><</span><span style="color: black;">Pos</span><span style="color: black; font-weight: bold;">,</span> <span style="color: black;">TextureRef</span><span style="color: #ce5c00; font-weight: bold;">></span> <span style="color: black;">ecs</span><span style="color: black; font-weight: bold;">;</span>
<span style="color: #204a87; font-weight: bold;">auto</span> <span style="color: black;">e1</span> <span style="color: #ce5c00; font-weight: bold;">=</span> <span style="color: black;">iu</span><span style="color: black; font-weight: bold;">.</span><span style="color: black;">write_new_entity</span><span style="color: black; font-weight: bold;">(</span><span style="color: black;">Pos</span><span style="color: black; font-weight: bold;">(</span><span style="color: #0000cf; font-weight: bold;">1</span><span style="color: black; font-weight: bold;">,</span> <span style="color: #0000cf; font-weight: bold;">2</span><span style="color: black; font-weight: bold;">),</span> <span style="color: black;">TextureRef</span><span style="color: black; font-weight: bold;">(</span><span style="color: black;">tex1</span><span style="color: black; font-weight: bold;">));</span>
<span style="color: #204a87; font-weight: bold;">auto</span> <span style="color: black;">e2</span> <span style="color: #ce5c00; font-weight: bold;">=</span> <span style="color: black;">iu</span><span style="color: black; font-weight: bold;">.</span><span style="color: black;">write_new_entity</span><span style="color: black; font-weight: bold;">(</span><span style="color: black;">Pos</span><span style="color: black; font-weight: bold;">(</span><span style="color: #0000cf; font-weight: bold;">3</span><span style="color: black; font-weight: bold;">,</span> <span style="color: #0000cf; font-weight: bold;">4</span><span style="color: black; font-weight: bold;">),</span> <span style="color: black;">TextureRef</span><span style="color: black; font-weight: bold;">(</span><span style="color: black;">tex2</span><span style="color: black; font-weight: bold;">));</span>
<span style="color: #204a87; font-weight: bold;">for</span> <span style="color: black; font-weight: bold;">(</span><span style="color: #204a87; font-weight: bold;">auto</span> <span style="color: black;">id</span> <span style="color: #ce5c00; font-weight: bold;">:</span> <span style="color: black;">ecs</span><span style="color: black; font-weight: bold;">.</span><span style="color: black;">ids</span><span style="color: black; font-weight: bold;">())</span> <span style="color: black; font-weight: bold;">{</span>
<span style="color: black;">Pos</span> <span style="color: black;">p</span><span style="color: black; font-weight: bold;">;</span>
<span style="color: black;">TextureRef</span> <span style="color: black;">tref</span><span style="color: black; font-weight: bold;">;</span>
if (<span style="color: black;">ecs</span><span style="color: black; font-weight: bold;">.</span><span style="color: black;">read</span><span style="color: black; font-weight: bold;">(</span>id, p, <span style="color: black;">tref</span><span style="color: black; font-weight: bold;">))</span>
<span style="color: black;">draw</span><span style="color: black; font-weight: bold;">(</span><span style="color: black;">p</span><span style="color: black; font-weight: bold;">,</span> <span style="color: black;">tref</span><span style="color: black; font-weight: bold;">);</span>
<span style="color: black; font-weight: bold;">}</span>
</pre>
</div>
</div>
<div><br /></div>
<div>
In the end, I use a much more efficient interface, but this gives an idea
about how it might be used. <i>e1</i> and <i>e2</i> are entity
ID's that have a <i>Pos</i> and <i>TextureRef</i> component which
can be read back out in order to draw them onto the screen if the entity has
both components.
</div>
<div><br /></div>
<div>
One note, before we move on: Why an <i>std::vector</i> and not a hash
table keyed by the entity ID? Many entity component systems do it this way
and it's great for <i>O(1)</i> lookups, but the code I have planned out
has little need for fast random access and hash tables don't make good use
of CPU caches. If I run into performance issues, I can see if a hash fixes
it, but for now I'll stick to this. I think the larger concern is that a
particle system might require more efficient random deletion.
</div>
<h3 style="text-align: left;">If only life was so simple...</h3>
<div>
Since my entity ID's monotonically increase (omitting
<i>int32 </i>overflow), keeping the component vectors sorted is
generally simple enough. I used <i>std::lower_bound()</i> to figure out
where to insert new elements, just in case, but generally entities don't
gain or lose components over time so they're almost always inserted at the
end. The component arrays are essentially parallel where
<i>std::get<std::vector<ComponentData<X>>>[i]</i> would resolve to the <i>i</i>'th <i>X</i> component of probably
the <i>i</i>'th entity, and ditto for <i>Y</i>. However, not all entities
have all components so some checking might be needed.
</div>
<div><br /></div>
<div>Say I want an algorithm that does this:</div>
<div><br /></div>
</div>
<blockquote style="border: none; margin: 0px 0px 0px 40px; padding: 0px; text-align: left;">
<div>
<div>
<i>for (grid_pos, graphical_pos) in ecs.read_grid_and_graphical_pos() {</i>
</div>
</div>
<div>
<div><i> graphical_pos = grid_pos * TILE_SIZE</i></div>
</div>
<div>
<div><i>}</i></div>
</div>
</blockquote>
<div>
<div><br /></div>
<div>
Well, if I can iterate through one, I can do two, right? Start at the
beginning, make sure they're pointing at the same element, and in the
increment step, increment one of them and then the other until it has the
same <i>EntityId </i>or higher and keep going back and forth until either
hits the end or they both have the same ID again.
</div>
<div><br /></div>
<div>
Great! Simple! Problem: I have potentially N iterators and they're all of
different types. I want this syntax:
</div>
<div><br /></div>
<div>
<!--HTML generated using hilite.me-->
<div style="background: rgb(248, 248, 248); border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;">
<pre style="line-height: 125%; margin: 0px;"><span style="color: #204a87; font-weight: bold;">int</span> <span style="color: black;">sum</span> <span style="color: #ce5c00; font-weight: bold;">=</span> <span style="color: #0000cf; font-weight: bold;">0</span><span style="color: black; font-weight: bold;">;</span>
<span style="color: #204a87; font-weight: bold;">for</span> <span style="color: black; font-weight: bold;">(</span><span style="color: #204a87; font-weight: bold;">auto</span> <span style="color: black; font-weight: bold;">[</span><span style="color: black;">id</span><span style="color: black; font-weight: bold;">,</span> <span style="color: black;">i</span><span style="color: black; font-weight: bold;">,</span> <span style="color: black;">u</span><span style="color: black; font-weight: bold;">]</span> <span style="color: #ce5c00; font-weight: bold;">:</span> <span style="color: black;">ecs</span><span style="color: black; font-weight: bold;">.</span><span style="color: black;">read_all</span><span style="color: #ce5c00; font-weight: bold;"><</span><span style="color: #204a87; font-weight: bold;">int</span><span style="color: black; font-weight: bold;">,</span> <span style="color: #204a87; font-weight: bold;">unsigned</span><span style="color: #ce5c00; font-weight: bold;">></span><span style="color: black; font-weight: bold;">())</span> <span style="color: black;">sum</span> <span style="color: #ce5c00; font-weight: bold;">+=</span> <span style="color: black;">i</span> <span style="color: #ce5c00; font-weight: bold;">+</span> <span style="color: black;">u</span><span style="color: black; font-weight: bold;">,</span>
</pre>
</div>
</div>
<div><br /></div>
<div>
Obviously, this is non-trivial with tuples since we can't store iterators to
each container in a vector, we have to store them in yet another tuple.
</div>
<div><br /></div>
<div>
My first idea was to have a range class parameterized by the containers
(<i>std::vector<ComponentData<T>>...</i>) and then for it to
define nested iterator classes parameterized by the iterator types of that
container. The basic issue with this is that the container could be const or
non-const, but I had to be able to differentiate since in one case, the
iterator class would store <i>std::vector<T>::iterator</i> and in
the other, <i>::const_iterator </i>so I ended up writing it twice. But what
I landed with is so much better.
</div>
<div><br /></div>
<div>
<!--HTML generated using hilite.me-->
<div style="background: rgb(248, 248, 248); border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;">
<pre style="line-height: 125%; margin: 0px;"><span style="color: #204a87; font-weight: bold;">template</span><span style="color: #ce5c00; font-weight: bold;"><</span><span style="color: #204a87; font-weight: bold;">typename</span> <span style="color: black;">F</span><span style="color: black; font-weight: bold;">,</span> <span style="color: #204a87; font-weight: bold;">typename</span><span style="color: black; font-weight: bold;">...</span><span style="color: black;">T</span><span style="color: #ce5c00; font-weight: bold;">></span>
<span style="color: black;">constexpr</span> <span style="color: #204a87; font-weight: bold;">auto</span> <span style="color: black;">tuple_map</span><span style="color: black; font-weight: bold;">(</span><span style="color: black;">F</span><span style="color: #ce5c00; font-weight: bold;">&&</span> <span style="color: black;">f</span><span style="color: black; font-weight: bold;">,</span> <span style="color: #204a87; font-weight: bold;">const</span> <span style="color: black;">std</span><span style="color: #ce5c00; font-weight: bold;">::</span><span style="color: black;">tuple</span><span style="color: #ce5c00; font-weight: bold;"><</span><span style="color: black;">T</span><span style="color: black; font-weight: bold;">...</span><span style="color: #ce5c00; font-weight: bold;">>&</span> <span style="color: black;">t</span><span style="color: black; font-weight: bold;">)</span> <span style="color: black; font-weight: bold;">{</span>
<span style="color: #204a87; font-weight: bold;">return</span> <span style="color: black;">std</span><span style="color: #ce5c00; font-weight: bold;">::</span><span style="color: black;">tuple</span><span style="color: black; font-weight: bold;">(</span><span style="color: black;">f</span><span style="color: black; font-weight: bold;">(</span><span style="color: black;">std</span><span style="color: #ce5c00; font-weight: bold;">::</span><span style="color: black;">get</span><span style="color: #ce5c00; font-weight: bold;"><</span><span style="color: black;">T</span><span style="color: #ce5c00; font-weight: bold;">></span><span style="color: black; font-weight: bold;">(</span><span style="color: black;">t</span><span style="color: black; font-weight: bold;">))...);</span>
<span style="color: black; font-weight: bold;">}</span>
<span style="color: #204a87; font-weight: bold;">template</span><span style="color: #ce5c00; font-weight: bold;"><</span><span style="color: #204a87; font-weight: bold;">typename</span> <span style="color: black;">F</span><span style="color: black; font-weight: bold;">,</span> <span style="color: #204a87; font-weight: bold;">typename</span><span style="color: black; font-weight: bold;">...</span><span style="color: black;">T</span><span style="color: #ce5c00; font-weight: bold;">></span>
<span style="color: #204a87; font-weight: bold;">auto</span> <span style="color: black;">tuple_map</span><span style="color: black; font-weight: bold;">(</span><span style="color: black;">F</span><span style="color: #ce5c00; font-weight: bold;">&&</span> <span style="color: black;">f</span><span style="color: black; font-weight: bold;">,</span> <span style="color: black;">std</span><span style="color: #ce5c00; font-weight: bold;">::</span><span style="color: black;">tuple</span><span style="color: #ce5c00; font-weight: bold;"><</span><span style="color: black;">T</span><span style="color: black; font-weight: bold;">...</span><span style="color: #ce5c00; font-weight: bold;">>&</span> <span style="color: black;">t</span><span style="color: black; font-weight: bold;">)</span> <span style="color: black; font-weight: bold;">{</span>
<span style="color: #204a87; font-weight: bold;">return</span> <span style="color: black;">std</span><span style="color: #ce5c00; font-weight: bold;">::</span><span style="color: black;">tuple</span><span style="color: black; font-weight: bold;">(</span><span style="color: black;">f</span><span style="color: black; font-weight: bold;">(</span><span style="color: black;">std</span><span style="color: #ce5c00; font-weight: bold;">::</span><span style="color: black;">get</span><span style="color: #ce5c00; font-weight: bold;"><</span><span style="color: black;">T</span><span style="color: #ce5c00; font-weight: bold;">></span><span style="color: black; font-weight: bold;">(</span><span style="color: black;">t</span><span style="color: black; font-weight: bold;">))...);</span>
<span style="color: black; font-weight: bold;">}</span>
<span style="color: #8f5902; font-style: italic;">// A range abstraction that allows multiple component data series to be iterated</span>
<span style="color: #8f5902; font-style: italic;">// lazily over in a range-based for loop.</span>
<span style="color: #204a87; font-weight: bold;">template</span><span style="color: #ce5c00; font-weight: bold;"><</span><span style="color: #204a87; font-weight: bold;">typename</span> <span style="color: black;">StoreTuple</span><span style="color: #ce5c00; font-weight: bold;">></span>
<span style="color: #204a87; font-weight: bold;">class</span> <span style="color: black;">ComponentRange</span> <span style="color: black; font-weight: bold;">{</span></pre>
<pre style="line-height: 125%; margin: 0px;"><pre style="line-height: 16.25px; margin-bottom: 0px; margin-top: 0px;"><span style="color: #8f5902; font-style: italic;"> // A tuple of references to component storage.</span></pre> <span style="color: black;">StoreTuple</span> <span style="color: black;">stores_</span><span style="color: black; font-weight: bold;">;</span>
<span style="color: #f57900;">protected:</span>
<span style="color: #204a87; font-weight: bold;">template</span><span style="color: #ce5c00; font-weight: bold;"><</span><span style="color: #204a87; font-weight: bold;">typename</span> <span style="color: black;">IteratorTuple</span><span style="color: #ce5c00; font-weight: bold;">></span>
<span style="color: #204a87; font-weight: bold;">struct</span> <span style="color: black;">Iterator</span> <span style="color: black; font-weight: bold;">{</span>
<span style="color: #8f5902; font-style: italic;">// ,,,</span>
<span style="color: black;">IteratorTuple</span> <span style="color: black;">its</span><span style="color: black; font-weight: bold;">;</span>
<span style="color: black;">IteratorTuple</span> <span style="color: black;">ends</span><span style="color: black; font-weight: bold;">;</span>
<span style="color: #204a87; font-weight: bold;">bool</span> <span style="color: black;">sentinel</span> <span style="color: #ce5c00; font-weight: bold;">=</span> <span style="color: #204a87;">false</span><span style="color: black; font-weight: bold;">;</span>
<span style="color: #8f5902; font-style: italic;">// To know if all iterators are pointing to the same entity, we'll neet</span>
<span style="color: #8f5902; font-style: italic;">// to remember what entity that is.</span>
<span style="color: black;">EntityId</span> <span style="color: black;">max_id</span><span style="color: black; font-weight: bold;">;</span>
<span style="color: #8f5902; font-style: italic;">// ...</span>
<span style="color: black;">Iterator</span><span style="color: black; font-weight: bold;">(</span><span style="color: black;">IteratorTuple</span> <span style="color: black;">its</span><span style="color: black; font-weight: bold;">,</span> <span style="color: black;">IteratorTuple</span> <span style="color: black;">ends</span><span style="color: black; font-weight: bold;">)</span>
<span style="color: #ce5c00; font-weight: bold;">:</span> <span style="color: black;">its</span><span style="color: black; font-weight: bold;">(</span><span style="color: black;">std</span><span style="color: #ce5c00; font-weight: bold;">::</span><span style="color: black;">move</span><span style="color: black; font-weight: bold;">(</span><span style="color: black;">its</span><span style="color: black; font-weight: bold;">)),</span> <span style="color: black;">ends</span><span style="color: black; font-weight: bold;">(</span><span style="color: black;">std</span><span style="color: #ce5c00; font-weight: bold;">::</span><span style="color: black;">move</span><span style="color: black; font-weight: bold;">(</span><span style="color: black;">ends</span><span style="color: black; font-weight: bold;">))</span> <span style="color: black; font-weight: bold;">{</span>
<span style="color: #8f5902; font-style: italic;">// ...</span>
<span style="color: black; font-weight: bold;">}</span>
<span style="color: black;">Iterator</span><span style="color: black; font-weight: bold;">()</span> <span style="color: black; font-weight: bold;">{</span> <span style="color: black;">sentinel</span> <span style="color: #ce5c00; font-weight: bold;">=</span> <span style="color: #204a87;">true</span><span style="color: black; font-weight: bold;">;</span> <span style="color: black; font-weight: bold;">}</span>
<span style="color: black; font-weight: bold;">};</span>
<span style="color: #f57900;">public:</span>
<span style="color: #204a87; font-weight: bold;">explicit</span> <span style="color: black;">ComponentRange</span><span style="color: black; font-weight: bold;">(</span><span style="color: black;">StoreTuple</span> <span style="color: black;">stores</span><span style="color: black; font-weight: bold;">)</span>
<span style="color: #ce5c00; font-weight: bold;">:</span> <span style="color: black;">stores_</span><span style="color: black; font-weight: bold;">(</span><span style="color: black;">stores</span><span style="color: black; font-weight: bold;">)</span> <span style="color: black; font-weight: bold;">{</span> <span style="color: black; font-weight: bold;">}</span>
<span style="color: #204a87; font-weight: bold;">auto</span> <span style="color: black;">begin</span><span style="color: black; font-weight: bold;">()</span> <span style="color: #204a87; font-weight: bold;">const</span> <span style="color: black; font-weight: bold;">{</span>
<span style="color: #204a87; font-weight: bold;">auto</span> <span style="color: black;">b</span> <span style="color: #ce5c00; font-weight: bold;">=</span> <span style="color: black; font-weight: bold;">[](</span><span style="color: #204a87; font-weight: bold;">auto</span><span style="color: #ce5c00; font-weight: bold;">&&</span> <span style="color: black;">store</span><span style="color: black; font-weight: bold;">)</span> <span style="color: black; font-weight: bold;">{</span> <span style="color: #204a87; font-weight: bold;">return</span> <span style="color: black;">store</span><span style="color: black; font-weight: bold;">.</span><span style="color: black;">begin</span><span style="color: black; font-weight: bold;">();</span> <span style="color: black; font-weight: bold;">};</span>
<span style="color: #204a87; font-weight: bold;">auto</span> <span style="color: black;">e</span> <span style="color: #ce5c00; font-weight: bold;">=</span> <span style="color: black; font-weight: bold;">[](</span><span style="color: #204a87; font-weight: bold;">auto</span><span style="color: #ce5c00; font-weight: bold;">&&</span> <span style="color: black;">store</span><span style="color: black; font-weight: bold;">)</span> <span style="color: black; font-weight: bold;">{</span> <span style="color: #204a87; font-weight: bold;">return</span> <span style="color: black;">store</span><span style="color: black; font-weight: bold;">.</span><span style="color: black;">end</span><span style="color: black; font-weight: bold;">();</span> <span style="color: black; font-weight: bold;">};</span>
<span style="color: #204a87; font-weight: bold;">return</span> <span style="color: black;">Iterator</span><span style="color: black; font-weight: bold;">(</span><span style="color: black;">tuple_map</span><span style="color: black; font-weight: bold;">(</span><span style="color: black;">b</span><span style="color: black; font-weight: bold;">,</span> <span style="color: black;">stores_</span><span style="color: black; font-weight: bold;">),</span> <span style="color: black;">tuple_map</span><span style="color: black; font-weight: bold;">(</span><span style="color: black;">e</span><span style="color: black; font-weight: bold;">,</span> <span style="color: black;">stores_</span><span style="color: black; font-weight: bold;">));</span>
<span style="color: black; font-weight: bold;">}</span>
<span style="color: #204a87; font-weight: bold;">auto</span> <span style="color: black;">end</span><span style="color: black; font-weight: bold;">()</span> <span style="color: #204a87; font-weight: bold;">const</span> <span style="color: black; font-weight: bold;">{</span>
<span style="color: #204a87; font-weight: bold;">return</span> <span style="color: black;">decltype</span><span style="color: black; font-weight: bold;">(</span><span style="color: black;">begin</span><span style="color: black; font-weight: bold;">())();</span>
<span style="color: black; font-weight: bold;">}</span>
<span style="color: black; font-weight: bold;">};</span>
</pre>
</div>
</div>
<div><br /></div>
<div>
The general idea is to be as unconstraining of types as possible so where
ever we can get away with not specifying a type, we do.
<i>StoreTuple</i> will generally resolve to something ugly like...
</div>
<div><br /></div>
</div>
<blockquote style="border: none; margin: 0px 0px 0px 40px; padding: 0px;">
<div>
<div style="text-align: left;">
<i>std::tuple<std::vector<ComponentData<X>>&,
std::vector<ComponentData<Y>>&, ...></i>
</div>
</div>
</blockquote>
<div><br /></div>
<div>
<i>tuple_map</i> and <i>tuple_foreach </i>(the same function, it just
doesn't return a new tuple) are going to be the bread and butter of this
class. For reference about how difficult these functions were to write in
C++11/14, I implemented them way back in <a href="https://yapb-soc.blogspot.com/2012/12/fun-with-tuples.html">"Fun With Tuples"</a>.
</div>
<div><br /></div>
<div>
If any of the <i>its</i> is at the end, the whole tuples is, which made
comparing to the end a bit tricky, thus a sentinel is used. I think a
variation on this idea could have been to consider two <i>Iterator</i>s equal
if <i>any</i> of their <i>its</i> are equal, though.
</div>
<div><br /></div>
<div>
We can't iterate through the tuples easily so it's best we work on them
uniformly, but we don't want any advancing past the <i>max_id</i>, which is
going to be the highest ID of an entity of all iterators. Just iterating once
is easy.
</div>
<div><br /></div>
<div>
<!--HTML generated using hilite.me-->
<div style="background: rgb(248, 248, 248); border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;">
<pre style="line-height: 125%; margin: 0px;"> <span style="color: #204a87; font-weight: bold;">template</span><span style="color: #ce5c00; font-weight: bold;"><</span><span style="color: #204a87; font-weight: bold;">typename</span> <span style="color: black;">It</span><span style="color: #ce5c00; font-weight: bold;">></span>
<span style="color: #204a87; font-weight: bold;">bool</span> <span style="color: black;">at_end</span><span style="color: black; font-weight: bold;">(</span><span style="color: black;">It</span> <span style="color: black;">it</span><span style="color: black; font-weight: bold;">)</span> <span style="color: #204a87; font-weight: bold;">const</span> <span style="color: black; font-weight: bold;">{</span>
<span style="color: #204a87; font-weight: bold;">return</span> <span style="color: black;">it</span> <span style="color: #ce5c00; font-weight: bold;">==</span> <span style="color: black;">std</span><span style="color: #ce5c00; font-weight: bold;">::</span><span style="color: black;">get</span><span style="color: #ce5c00; font-weight: bold;"><</span><span style="color: black;">It</span><span style="color: #ce5c00; font-weight: bold;">></span><span style="color: black; font-weight: bold;">(</span><span style="color: black;">ends</span><span style="color: black; font-weight: bold;">);</span>
<span style="color: black; font-weight: bold;">}</span>
<span style="color: #8f5902; font-style: italic;">// Note that the alternative to the above is</span>
<span style="color: #8f5902; font-style: italic;">// std::get<std::decay_t<decltype(it)>>(ends) if auto-typed (e.g. lambda).</span>
<span style="color: #204a87; font-weight: bold;">template</span><span style="color: #ce5c00; font-weight: bold;"><</span><span style="color: #204a87; font-weight: bold;">typename</span> <span style="color: black;">Iter</span><span style="color: #ce5c00; font-weight: bold;">></span>
<span style="color: #204a87; font-weight: bold;">void</span> <span style="color: black;">increment_iter</span><span style="color: black; font-weight: bold;">(</span><span style="color: black;">Iter</span><span style="color: #ce5c00; font-weight: bold;">&</span> <span style="color: black;">it</span><span style="color: black; font-weight: bold;">)</span> <span style="color: black; font-weight: bold;">{</span>
<span style="color: #204a87; font-weight: bold;">if</span> <span style="color: black; font-weight: bold;">(</span><span style="color: #ce5c00; font-weight: bold;">!</span><span style="color: black;">at_end</span><span style="color: black; font-weight: bold;">(</span><span style="color: black;">it</span><span style="color: black; font-weight: bold;">))</span> <span style="color: #ce5c00; font-weight: bold;">++</span><span style="color: black;">it</span><span style="color: black; font-weight: bold;">;</span>
<span style="color: #204a87; font-weight: bold;">if</span> <span style="color: black; font-weight: bold;">(</span><span style="color: #ce5c00; font-weight: bold;">!</span><span style="color: black;">at_end</span><span style="color: black; font-weight: bold;">(</span><span style="color: black;">it</span><span style="color: black; font-weight: bold;">))</span> <span style="color: black;">max_id</span> <span style="color: #ce5c00; font-weight: bold;">=</span> <span style="color: black;">std</span><span style="color: #ce5c00; font-weight: bold;">::</span><span style="color: black;">max</span><span style="color: black; font-weight: bold;">(</span><span style="color: black;">it</span><span style="color: #ce5c00; font-weight: bold;">-></span><span style="color: black;">id</span><span style="color: black; font-weight: bold;">,</span> <span style="color: black;">max_id</span><span style="color: black; font-weight: bold;">);</span>
<span style="color: black; font-weight: bold;">}</span>
</pre>
</div>
</div>
<div><br /></div>
<div>
This handles the case where an iterator was under <i>max_id</i>, but in an
attempt to catch up it went over and set a new maximum.
</div>
<div><br /></div>
<div>
Next, we have to handle incrementing all the iterators until they all line up
on one ID.
</div>
<div><br /></div>
<div>
<!--HTML generated using hilite.me-->
<div style="background: rgb(248, 248, 248); border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;">
<pre style="line-height: 16.25px; margin-bottom: 0px; margin-top: 0px;"> <span style="color: #8f5902; font-style: italic;">// A helper to catch up iterators that are behind.</span>
<span style="color: #204a87; font-weight: bold;">void</span> <span>increment_if_lower_than_max_id</span><span style="font-weight: bold;">(</span><span style="font-weight: bold;">)</span> <span style="font-weight: bold;">{</span>
<span style="color: #204a87; font-weight: bold;">auto</span> <span>impl</span> <span style="color: #ce5c00; font-weight: bold;">=</span> <span style="font-weight: bold;">[</span><span style="color: #ce5c00; font-weight: bold;">&</span><span style="font-weight: bold;">](</span><span style="color: #204a87; font-weight: bold;">auto</span><span style="color: #ce5c00; font-weight: bold;">&</span> <span>it</span><span style="font-weight: bold;">)</span> <span style="font-weight: bold;">{</span>
<span style="color: #204a87; font-weight: bold;">if</span> <span style="font-weight: bold;">(</span><span style="color: #ce5c00; font-weight: bold;">!</span><span>at_end</span><span style="font-weight: bold;">(</span><span>it</span><span style="font-weight: bold;">)</span> <span style="color: #ce5c00; font-weight: bold;">&&</span> <span>it</span><span style="color: #ce5c00; font-weight: bold;">-></span><span>id</span> <span style="color: #ce5c00; font-weight: bold;"><</span> <span>max_id</span><span style="font-weight: bold;">)</span> <span>increment_iter</span><span style="font-weight: bold;">(</span><span>it</span><span style="font-weight: bold;">);</span>
<span style="font-weight: bold;">};</span>
<span>tuple_foreach</span><span style="font-weight: bold;">(</span><span>impl</span><span style="font-weight: bold;">,</span> <span>its</span><span style="font-weight: bold;">);</span>
</pre>
<pre style="line-height: 125%; margin: 0px;"> <span style="font-weight: bold;">}</span></pre>
<pre style="line-height: 125%; margin: 0px;"><br /></pre>
<pre style="line-height: 125%; margin: 0px;"> <span style="color: #204a87; font-weight: bold;">bool</span> <span style="color: black;">any_at_end</span><span style="color: black; font-weight: bold;">()</span> <span style="color: #204a87; font-weight: bold;">const</span> <span style="color: black; font-weight: bold;">{</span>
<span style="color: #204a87; font-weight: bold;">auto</span> <span style="color: black;">any</span> <span style="color: #ce5c00; font-weight: bold;">=</span> <span style="color: black; font-weight: bold;">[](</span><span style="color: #204a87; font-weight: bold;">auto</span><span style="color: black; font-weight: bold;">...</span><span style="color: black;">args</span><span style="color: black; font-weight: bold;">)</span> <span style="color: black; font-weight: bold;">{</span> <span style="color: #204a87; font-weight: bold;">return</span> <span style="color: black; font-weight: bold;">(</span><span style="color: black;">args</span> <span style="color: #ce5c00; font-weight: bold;">||</span> <span style="color: black; font-weight: bold;">...);</span> <span style="color: black; font-weight: bold;">};</span>
<span style="color: #204a87; font-weight: bold;">auto</span> <span style="color: black;">pred</span> <span style="color: #ce5c00; font-weight: bold;">=</span> <span style="color: black; font-weight: bold;">[</span><span style="color: #204a87; font-weight: bold;">this</span><span style="color: black; font-weight: bold;">](</span><span style="color: #204a87; font-weight: bold;">const</span> <span style="color: #204a87; font-weight: bold;">auto</span><span style="color: #ce5c00; font-weight: bold;">&</span> <span style="color: black;">it</span><span style="color: black; font-weight: bold;">)</span> <span style="color: black; font-weight: bold;">{</span> <span style="color: #204a87; font-weight: bold;">return</span> <span style="color: black;">at_end</span><span style="color: black; font-weight: bold;">(</span><span style="color: black;">it</span><span style="color: black; font-weight: bold;">);</span> <span style="color: black; font-weight: bold;">};</span>
<span style="color: #204a87; font-weight: bold;">return</span> <span style="color: black;">std</span><span style="color: #ce5c00; font-weight: bold;">::</span><span style="color: black;">apply</span><span style="color: black; font-weight: bold;">(</span><span style="color: black;">any</span><span style="color: black; font-weight: bold;">,</span> <span style="color: black;">tuple_map</span><span style="color: black; font-weight: bold;">(</span><span style="color: black;">pred</span><span style="color: black; font-weight: bold;">,</span> <span style="color: black;">its</span><span style="color: black; font-weight: bold;">));</span>
<span style="color: black; font-weight: bold;">}</span>
<span style="color: #8f5902; font-style: italic;">// True if all iterators point to the same ID.</span>
<span style="color: #204a87; font-weight: bold;">bool</span> <span style="color: black;">all_same</span><span style="color: black; font-weight: bold;">()</span> <span style="color: #204a87; font-weight: bold;">const</span> <span style="color: black; font-weight: bold;">{</span>
<span style="color: #204a87; font-weight: bold;">auto</span> <span style="color: black;">all</span> <span style="color: #ce5c00; font-weight: bold;">=</span> <span style="color: black; font-weight: bold;">[](</span><span style="color: #204a87; font-weight: bold;">auto</span><span style="color: black; font-weight: bold;">...</span><span style="color: black;">args</span><span style="color: black; font-weight: bold;">)</span> <span style="color: black; font-weight: bold;">{</span> <span style="color: #204a87; font-weight: bold;">return</span> <span style="color: black; font-weight: bold;">(</span><span style="color: black;">args</span> <span style="color: #ce5c00; font-weight: bold;">&&</span> <span style="color: black; font-weight: bold;">...);</span> <span style="color: black; font-weight: bold;">};</span>
<span style="color: #204a87; font-weight: bold;">auto</span> <span style="color: black;">equals_max_id</span> <span style="color: #ce5c00; font-weight: bold;">=</span>
<span style="color: black; font-weight: bold;">[</span><span style="color: #204a87; font-weight: bold;">this</span><span style="color: black; font-weight: bold;">](</span><span style="color: #204a87; font-weight: bold;">const</span> <span style="color: #204a87; font-weight: bold;">auto</span><span style="color: #ce5c00; font-weight: bold;">&</span> <span style="color: black;">it</span><span style="color: black; font-weight: bold;">)</span> <span style="color: black; font-weight: bold;">{</span> <span style="color: #204a87; font-weight: bold;">return</span> <span style="color: black;">it</span><span style="color: #ce5c00; font-weight: bold;">-></span><span style="color: black;">id</span> <span style="color: #ce5c00; font-weight: bold;">==</span> <span style="color: black;">max_id</span><span style="color: black; font-weight: bold;">;</span> <span style="color: black; font-weight: bold;">};</span>
<span style="color: #204a87; font-weight: bold;">return</span> <span style="color: black;">std</span><span style="color: #ce5c00; font-weight: bold;">::</span><span style="color: black;">apply</span><span style="color: black; font-weight: bold;">(</span><span style="color: black;">all</span><span style="color: black; font-weight: bold;">,</span> <span style="color: black;">tuple_map</span><span style="color: black; font-weight: bold;">(</span><span style="color: black;">equals_max_id</span><span style="color: black; font-weight: bold;">,</span> <span style="color: black;">its</span><span style="color: black; font-weight: bold;">));</span>
<span style="color: black; font-weight: bold;">}</span>
<span style="color: #8f5902; font-style: italic;">// After iteration, or on initialization, increment any iterators that</span>
<span style="color: #8f5902; font-style: italic;">// are behind.</span>
<span style="color: #204a87; font-weight: bold;">void</span> <span style="color: black;">catch_up</span><span style="color: black; font-weight: bold;">()</span> <span style="color: black; font-weight: bold;">{</span>
<span style="color: #204a87; font-weight: bold;">while</span> <span style="color: black; font-weight: bold;">(</span><span style="color: #ce5c00; font-weight: bold;">!</span><span style="color: black;">any_at_end</span><span style="color: black; font-weight: bold;">()</span> <span style="color: #ce5c00; font-weight: bold;">&&</span> <span style="color: #ce5c00; font-weight: bold;">!</span><span style="color: black;">all_same</span><span style="color: black; font-weight: bold;">())</span> increment_if_lower_than_max_id(<span style="font-weight: bold;">);</span><br /> <span style="color: #204a87; font-weight: bold;">if</span> <span style="color: black; font-weight: bold;">(</span><span style="color: black;">any_at_end</span><span style="color: black; font-weight: bold;">())</span> <span style="color: black;">its</span> <span style="color: #ce5c00; font-weight: bold;">=</span> <span style="color: black;">ends</span><span style="color: black; font-weight: bold;">;</span>
<span style="color: black; font-weight: bold;">}</span>
<span style="color: black;">Iterator</span><span style="color: #ce5c00; font-weight: bold;">&</span> <span style="color: #204a87; font-weight: bold;">operator</span><span style="color: #ce5c00; font-weight: bold;">++</span><span style="color: black; font-weight: bold;">()</span> <span style="color: black; font-weight: bold;">{</span>
<span style="color: #8f5902; font-style: italic;">// Increment at least once.</span>
<span style="color: black;">tuple_foreach</span><span style="color: black; font-weight: bold;">([</span><span style="color: #204a87; font-weight: bold;">this</span><span style="color: black; font-weight: bold;">](</span><span style="color: #204a87; font-weight: bold;">auto</span><span style="color: #ce5c00; font-weight: bold;">&</span> <span style="color: black;">it</span><span style="color: black; font-weight: bold;">)</span> <span style="color: black; font-weight: bold;">{</span> <span style="color: black;">increment_iter</span><span style="color: black; font-weight: bold;">(</span><span style="color: black;">it</span><span style="color: black; font-weight: bold;">);</span> <span style="color: black; font-weight: bold;">},</span> <span style="color: black;">its</span><span style="color: black; font-weight: bold;">);</span>
<span style="color: black;">catch_up</span><span style="color: black; font-weight: bold;">();</span>
<span style="color: #204a87; font-weight: bold;">return</span> <span style="color: #ce5c00; font-weight: bold;">*</span><span style="color: #204a87; font-weight: bold;">this</span><span style="color: black; font-weight: bold;">;</span>
<span style="color: black; font-weight: bold;">}</span>
<span style="color: black;">Iterator</span> <span style="color: #204a87; font-weight: bold;">operator</span><span style="color: #ce5c00; font-weight: bold;">++</span><span style="color: black; font-weight: bold;">(</span><span style="color: #204a87; font-weight: bold;">int</span><span style="color: black; font-weight: bold;">)</span> <span style="color: black; font-weight: bold;">{</span>
<span style="color: black;">Iterator</span> <span style="color: black;">old</span> <span style="color: #ce5c00; font-weight: bold;">=</span> <span style="color: #ce5c00; font-weight: bold;">*</span><span style="color: #204a87; font-weight: bold;">this</span><span style="color: black; font-weight: bold;">;</span>
<span style="color: #ce5c00; font-weight: bold;">++</span><span style="color: black; font-weight: bold;">(</span><span style="color: #ce5c00; font-weight: bold;">*</span><span style="color: #204a87; font-weight: bold;">this</span><span style="color: black; font-weight: bold;">);</span>
<span style="color: #204a87; font-weight: bold;">return</span> <span style="color: black;">old</span><span style="color: black; font-weight: bold;">;</span>
<span style="color: black; font-weight: bold;">}</span>
</pre>
</div>
</div>
<div><br /></div>
<div>And finally, comparisons.</div>
<div><br /></div>
<div>
<!--HTML generated using hilite.me-->
<div style="background: rgb(248, 248, 248); border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;">
<pre style="line-height: 125%; margin: 0px;"> <span style="color: #204a87; font-weight: bold;">bool</span> <span style="color: #204a87; font-weight: bold;">operator</span><span style="color: #ce5c00; font-weight: bold;">==</span><span style="color: black; font-weight: bold;">(</span><span style="color: #204a87; font-weight: bold;">const</span> <span style="color: black;">Iterator</span><span style="color: #ce5c00; font-weight: bold;">&</span> <span style="color: black;">other</span><span style="color: black; font-weight: bold;">)</span> <span style="color: #204a87; font-weight: bold;">const</span> <span style="color: black; font-weight: bold;">{</span>
<span style="color: #204a87; font-weight: bold;">return</span> <span style="color: black;">sentinel</span> <span style="color: #ce5c00; font-weight: bold;">?</span> <span style="color: black;">other</span> <span style="color: #ce5c00; font-weight: bold;">==</span> <span style="color: #ce5c00; font-weight: bold;">*</span><span style="color: #204a87; font-weight: bold;">this</span> <span style="color: #ce5c00; font-weight: bold;">:</span> <span style="color: black;">any_at_end</span><span style="color: black; font-weight: bold;">();</span>
<span style="color: black; font-weight: bold;">}</span>
<span style="color: #204a87; font-weight: bold;">bool</span> <span style="color: #204a87; font-weight: bold;">operator</span><span style="color: #ce5c00; font-weight: bold;">!=</span><span style="color: black; font-weight: bold;">(</span><span style="color: #204a87; font-weight: bold;">const</span> <span style="color: black;">Iterator</span><span style="color: #ce5c00; font-weight: bold;">&</span> <span style="color: black;">other</span><span style="color: black; font-weight: bold;">)</span> <span style="color: #204a87; font-weight: bold;">const</span> <span style="color: black; font-weight: bold;">{</span>
<span style="color: #204a87; font-weight: bold;">return</span> <span style="color: #ce5c00; font-weight: bold;">!</span><span style="color: black; font-weight: bold;">(</span><span style="color: #ce5c00; font-weight: bold;">*</span><span style="color: #204a87; font-weight: bold;">this</span> <span style="color: #ce5c00; font-weight: bold;">==</span> <span style="color: black;">other</span><span style="color: black; font-weight: bold;">);</span>
<span style="color: black; font-weight: bold;">}</span>
<span style="color: #204a87; font-weight: bold;">auto</span> <span style="color: #204a87; font-weight: bold;">operator</span><span style="color: #ce5c00; font-weight: bold;">*</span><span style="color: black; font-weight: bold;">()</span> <span style="color: #204a87; font-weight: bold;">const</span> <span style="color: black; font-weight: bold;">{</span>
<span style="color: #204a87; font-weight: bold;">auto</span> <span style="color: black;">data</span> <span style="color: #ce5c00; font-weight: bold;">=</span> <span style="color: black; font-weight: bold;">[](</span><span style="color: #204a87; font-weight: bold;">const</span> <span style="color: #204a87; font-weight: bold;">auto</span><span style="color: #ce5c00; font-weight: bold;">&</span> <span style="color: black;">it</span><span style="color: black; font-weight: bold;">)</span> <span style="color: #ce5c00; font-weight: bold;">-></span> <span style="color: #204a87; font-weight: bold;">auto</span><span style="color: #ce5c00; font-weight: bold;">&</span> <span style="color: black; font-weight: bold;">{</span> <span style="color: #204a87; font-weight: bold;">return</span> <span style="color: black;">it</span><span style="color: #ce5c00; font-weight: bold;">-></span><span style="color: black;">data</span><span style="color: black; font-weight: bold;">;</span> <span style="color: black; font-weight: bold;">};</span>
<span style="color: #204a87; font-weight: bold;">return</span> <span style="color: black;">std</span><span style="color: #ce5c00; font-weight: bold;">::</span><span style="color: black;">tuple_cat</span><span style="color: black; font-weight: bold;">(</span><span style="color: black;">std</span><span style="color: #ce5c00; font-weight: bold;">::</span><span style="color: black;">tuple</span><span style="color: black; font-weight: bold;">(</span><span style="color: black;">max_id</span><span style="color: black; font-weight: bold;">),</span> <span style="color: black;">tuple_map</span><span style="color: black; font-weight: bold;">(</span><span style="color: black;">data</span><span style="color: black; font-weight: bold;">,</span> <span style="color: black;">its</span><span style="color: black; font-weight: bold;">));</span>
<span style="color: black; font-weight: bold;">}</span>
</pre>
</div>
</div>
<div><br /></div>
<div>
The final gotcha is that on construction, not all the iterators might be
pointing to the same ID, so we have to call
<i>catch_up()</i> immediately. The full code for the constructor looks
like this:
</div>
<div><br /></div>
<div>
<!--HTML generated using hilite.me-->
<div style="background: rgb(248, 248, 248); border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;">
<pre style="line-height: 125%; margin: 0px;"> <span style="color: black;">Iterator</span><span style="color: black; font-weight: bold;">(</span><span style="color: black;">IteratorTuple</span> <span style="color: black;">its</span><span style="color: black; font-weight: bold;">,</span> <span style="color: black;">IteratorTuple</span> <span style="color: black;">ends</span><span style="color: black; font-weight: bold;">)</span>
<span style="color: #ce5c00; font-weight: bold;">:</span> <span style="color: black;">its</span><span style="color: black; font-weight: bold;">(</span><span style="color: black;">std</span><span style="color: #ce5c00; font-weight: bold;">::</span><span style="color: black;">move</span><span style="color: black; font-weight: bold;">(</span><span style="color: black;">its</span><span style="color: black; font-weight: bold;">)),</span> <span style="color: black;">ends</span><span style="color: black; font-weight: bold;">(</span><span style="color: black;">std</span><span style="color: #ce5c00; font-weight: bold;">::</span><span style="color: black;">move</span><span style="color: black; font-weight: bold;">(</span><span style="color: black;">ends</span><span style="color: black; font-weight: bold;">))</span> <span style="color: black; font-weight: bold;">{</span>
<span style="color: black;">max_id</span> <span style="color: #ce5c00; font-weight: bold;">=</span> EntityId{0}<span style="color: black; font-weight: bold;">;</span>
<span style="color: black;">tuple_foreach</span><span style="color: black; font-weight: bold;">(</span>
<span style="color: black; font-weight: bold;">[</span><span style="color: #ce5c00; font-weight: bold;">&</span><span style="color: black; font-weight: bold;">](</span><span style="color: #204a87; font-weight: bold;">auto</span> <span style="color: black;">it</span><span style="color: black; font-weight: bold;">)</span> <span style="color: black; font-weight: bold;">{</span>
<span style="color: #204a87; font-weight: bold;">if</span> <span style="color: black; font-weight: bold;">(</span><span style="color: #ce5c00; font-weight: bold;">!</span><span style="color: black;">at_end</span><span style="color: black; font-weight: bold;">(</span><span style="color: black;">it</span><span style="color: black; font-weight: bold;">))</span> <span style="color: black;">max_id</span> <span style="color: #ce5c00; font-weight: bold;">=</span> <span style="color: black;">std</span><span style="color: #ce5c00; font-weight: bold;">::</span><span style="color: black;">max</span><span style="color: black; font-weight: bold;">(</span><span style="color: black;">max_id</span><span style="color: black; font-weight: bold;">,</span> <span style="color: black;">it</span><span style="color: #ce5c00; font-weight: bold;">-></span><span style="color: black;">id</span><span style="color: black; font-weight: bold;">);</span>
<span style="color: black; font-weight: bold;">},</span> <span style="color: black;">its</span><span style="color: black; font-weight: bold;">);</span>
<span style="color: black;">catch_up</span><span style="color: black; font-weight: bold;">();</span> <span style="color: #8f5902; font-style: italic;">// Ensure a valid initial starting position.</span>
<span style="color: black; font-weight: bold;">}</span>
<span style="color: black;">Iterator</span><span style="color: black; font-weight: bold;">()</span> <span style="color: black; font-weight: bold;">{</span> <span style="color: black;">sentinel</span> <span style="color: #ce5c00; font-weight: bold;">=</span> <span style="color: #204a87;">true</span><span style="color: black; font-weight: bold;">;</span> <span style="color: black; font-weight: bold;">}</span>
</pre>
</div>
</div>
<div><br /></div>
<h3 style="text-align: left;">Conclusions</h3>
<div>
The full code can be found here: <a href="https://gist.github.com/splinterofchaos/a67426bb95ea97b3f33fbc44406bc791">https://gist.github.com/splinterofchaos/a67426bb95ea97b3f33fbc44406bc791</a>
</div>
<div><br /></div>
<div>
The implementation is incomplete at the time of this writing and some of the
interface is inconsistent or unnecessary, but it's a start.
</div>
<div><br /></div>
<div>
And for the project in which I'm using it: <a href="https://github.com/splinterofchaos/py-srpg">https://github.com/splinterofchaos/py-srpg</a> (Originally written in Python; the C++ conversion is on the
<i>cpp</i> branch at the time of this writing.)
</div>
<div><br /></div>
<div>
Writing code like this used to be tedious, difficult, and involve heaps of
boilerplate, but at this point, fold expressions over a variable number of
types and arguments feels easier than a <i>for</i> loop from
<i>i=0</i> to <i>N</i>.
</div>
<div><br /></div>
<div>
The biggest frustrations for functional programming in C++ remain being things
like how in order to take a template function and convert it to a callable
function object, one has to wrap in a lambda and there exists weak partial
application. For example, in our <i>begin()</i> function.
</div>
<div><br /></div>
<div>
<!--HTML generated using hilite.me-->
<div style="background: rgb(248, 248, 248); border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;">
<pre style="line-height: 125%; margin: 0px;"> <span style="color: #204a87; font-weight: bold;">auto</span> <span style="color: black;">begin</span><span style="color: black; font-weight: bold;">()</span> <span style="color: #204a87; font-weight: bold;">const</span> <span style="color: black; font-weight: bold;">{</span>
<span style="color: #204a87; font-weight: bold;">auto</span> <span style="color: black;">b</span> <span style="color: #ce5c00; font-weight: bold;">=</span> <span style="color: black; font-weight: bold;">[](</span><span style="color: #204a87; font-weight: bold;">auto</span><span style="color: #ce5c00; font-weight: bold;">&&</span> <span style="color: black;">store</span><span style="color: black; font-weight: bold;">)</span> <span style="color: black; font-weight: bold;">{</span> <span style="color: #204a87; font-weight: bold;">return</span> <span style="color: black;">store</span><span style="color: black; font-weight: bold;">.</span><span style="color: black;">begin</span><span style="color: black; font-weight: bold;">();</span> <span style="color: black; font-weight: bold;">};</span>
<span style="color: #204a87; font-weight: bold;">auto</span> <span style="color: black;">e</span> <span style="color: #ce5c00; font-weight: bold;">=</span> <span style="color: black; font-weight: bold;">[](</span><span style="color: #204a87; font-weight: bold;">auto</span><span style="color: #ce5c00; font-weight: bold;">&&</span> <span style="color: black;">store</span><span style="color: black; font-weight: bold;">)</span> <span style="color: black; font-weight: bold;">{</span> <span style="color: #204a87; font-weight: bold;">return</span> <span style="color: black;">store</span><span style="color: black; font-weight: bold;">.</span><span style="color: black;">end</span><span style="color: black; font-weight: bold;">();</span> <span style="color: black; font-weight: bold;">};</span>
<span style="color: #204a87; font-weight: bold;">return</span> <span style="color: black;">Iterator</span><span style="color: black; font-weight: bold;">(</span><span style="color: black;">tuple_map</span><span style="color: black; font-weight: bold;">(</span><span style="color: black;">b</span><span style="color: black; font-weight: bold;">,</span> <span style="color: black;">stores_</span><span style="color: black; font-weight: bold;">),</span> <span style="color: black;">tuple_map</span><span style="color: black; font-weight: bold;">(</span><span style="color: black;">e</span><span style="color: black; font-weight: bold;">,</span> <span style="color: black;">stores_</span><span style="color: black; font-weight: bold;">));</span>
<span style="color: black; font-weight: bold;">}</span>
</pre>
</div>
</div>
<div><br /></div>
<div>
But things like fold expressions and class template argument deduction make
things absolutely much easier.
</div>
<div><br /></div>
<div>
As for the iterator class, the logic is much less trivial than I'd hope an
iterator class to be, but I think the performance should be decent. If my
program ends up spending more time incrementing iterators than using the
values obtained, I think my bigger issue will be that my game's not doing
much. Most of the iterations through the ECS should be of very similar
entities and so in the general case, it should increment once and exit.
</div>
<div><br /></div>
<div>
As for the entity class itself, I'll have to test it out a bunch. Obviously, I
worry that adding or removing large numbers of entities could be problematic,
but I imagine it'll be good enough for my purposes and there's plenty of room
for optimization, like maintaining a free list of deleted entity IDs and
component data, for one example. Still, when I'd used the same in Python, it
worked well so it should be better here.
</div>
Splinter of Chaoshttp://www.blogger.com/profile/14715348728512729776noreply@blogger.com0tag:blogger.com,1999:blog-8638474063464489651.post-41324895876626972942015-02-19T09:09:00.002-08:002015-02-19T18:46:19.702-08:00SFINAE std::result_of? Yeah, right!Return type deduction pre-<i>decltype</i> and <i>auto</i> could really make one mad. Back then, if you wanted a function object, you had to make it "adaptable", meaning it had to inherit from <i>std::unary_</i> or <i>binary_function</i> and define its <i>first_argument_type</i>, <i>second_argument_type</i>, and <i>result_type</i>. There had been no concept of "<a href="http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2012/n3421">transparent functors</a>" allowing us to pass polymorphic functions to higher order functions (like <i>std::plus<></i> to <i>std::accumulate</i>). For an example of programming in these dark ages, check out the <a href="http://people.cs.umass.edu/~yannis/fc++/tutorial.html">FC++ FAQ</a>.<br />
<br />
Just to recap, SFINAE stands for Substitution Failure Is Not An Error. It's the rule that lets you define two overloads of a function such that and one returns <i>decltype(f(x))</i> and the other <i>decltype(g(x))</i>, if <i>f(x)</i> can't be evaluated, that's not an error. If <i>g(x)</i> can't. either, that's OK too, as long one one of the two can be for any given <i>x</i>.<br />
<br />
And so came <i>decltype</i> and it let us do never-before seen, amazing things, right? Well, yes, we could write things like this:<br />
<pre style="background: #f6f8ff; color: #000020;"><html><body style="background: #f6f8ff; color: #000020;"><pre><span style="color: #200080; font-weight: bold;">template</span><span style="color: #406080;"><</span><span style="color: #200080; font-weight: bold;">class</span> X<span style="color: #406080;">></span>
constexpr <span style="color: #200080; font-weight: bold;">auto</span> append_blah<span style="color: #308080;">(</span>X<span style="color: #308080;">&</span> x<span style="color: #308080;">)</span> <span style="color: #308080;">-</span><span style="color: #308080;">></span> decltype<span style="color: #308080;">(</span>x <span style="color: #308080;">+</span> <span style="color: maroon;">"</span><span style="color: #1060b6;">blah</span><span style="color: maroon;">"</span><span style="color: #308080;">)</span>
<span style="color: #406080;">{</span>
<span style="color: #200080; font-weight: bold;">return</span> x <span style="color: #308080;">+</span> <span style="color: maroon;">"</span><span style="color: #1060b6;">blah</span><span style="color: maroon;">"</span><span style="color: #406080;">;</span>
<span style="color: #406080;">}</span>
<span style="color: #200080; font-weight: bold;">int</span> <span style="color: #400000;">main</span><span style="color: #308080;">(</span><span style="color: #308080;">)</span> <span style="color: #406080;">{</span>
append_blah<span style="color: #308080;">(</span><span style="color: maroon;">"</span><span style="color: #1060b6;">nah</span><span style="color: maroon;">"</span><span style="color: #308080;">)</span><span style="color: #406080;">;</span>
<span style="color: #406080;">}</span>
</pre>
</body></html></pre>
<div>
Obviously, since <i>"nah" </i><i>+= "blah"</i> isn't valid, this will fail to compile, but we'll get a nice, easy to read error message, right? Nah!</div>
<div>
<pre style="background: #f6f8ff; color: #000020;"><html><body style="background: #f6f8ff; color: #000020;"><pre><span style="color: #200080; font-weight: bold;">auto</span><span style="color: #308080;">.</span>cpp<span style="color: #406080;">:</span> In function `<span style="color: #200080; font-weight: bold;">int</span> <span style="color: #400000;">main</span><span style="color: #308080;">(</span><span style="color: #308080;">)</span>`<span style="color: #406080;">:</span>
<span style="color: #200080; font-weight: bold;">auto</span><span style="color: #308080;">.</span>cpp<span style="color: #406080;">:</span><span style="color: #008c00;">11</span><span style="color: #406080;">:</span><span style="color: #008c00;">20</span><span style="color: #406080;">:</span> error<span style="color: #406080;">:</span> no matching function <span style="color: #200080; font-weight: bold;">for</span> call to `append_blah<span style="color: #308080;">(</span><span style="color: #200080; font-weight: bold;">const</span> <span style="color: #200080; font-weight: bold;">char</span> <span style="color: #308080;">[</span><span style="color: #008c00;">4</span><span style="color: #308080;">]</span><span style="color: #308080;">)</span>`
append_blah<span style="color: #308080;">(</span><span style="color: maroon;">"</span><span style="color: #1060b6;">nah</span><span style="color: maroon;">"</span><span style="color: #308080;">)</span><span style="color: #406080;">;</span>
<span style="color: #308080;">^</span>
<span style="color: #200080; font-weight: bold;">auto</span><span style="color: #308080;">.</span>cpp<span style="color: #406080;">:</span><span style="color: #008c00;">11</span><span style="color: #406080;">:</span><span style="color: #008c00;">20</span><span style="color: #406080;">:</span> note<span style="color: #406080;">:</span> candidate is<span style="color: #406080;">:</span>
<span style="color: #200080; font-weight: bold;">auto</span><span style="color: #308080;">.</span>cpp<span style="color: #406080;">:</span><span style="color: #008c00;">5</span><span style="color: #406080;">:</span><span style="color: #008c00;">16</span><span style="color: #406080;">:</span> note<span style="color: #406080;">:</span> <span style="color: #200080; font-weight: bold;">template</span><span style="color: #406080;"><</span><span style="color: #200080; font-weight: bold;">class</span> X<span style="color: #406080;">></span> constexpr decltype <span style="color: #308080;">(</span><span style="color: #308080;">(</span>x <span style="color: #308080;">+</span> <span style="color: maroon;">"</span><span style="color: #1060b6;">blah</span><span style="color: maroon;">"</span><span style="color: #308080;">)</span><span style="color: #308080;">)</span> append_blah<span style="color: #308080;">(</span>X<span style="color: #308080;">)</span>
constexpr <span style="color: #200080; font-weight: bold;">auto</span> append_blah<span style="color: #308080;">(</span>X x<span style="color: #308080;">)</span> <span style="color: #308080;">-</span><span style="color: #308080;">></span> decltype<span style="color: #308080;">(</span>x <span style="color: #308080;">+</span> <span style="color: maroon;">"</span><span style="color: #1060b6;">blah</span><span style="color: maroon;">"</span><span style="color: #308080;">)</span>
<span style="color: #308080;">^</span>
<span style="color: #200080; font-weight: bold;">auto</span><span style="color: #308080;">.</span>cpp<span style="color: #406080;">:</span><span style="color: #008c00;">5</span><span style="color: #406080;">:</span><span style="color: #008c00;">16</span><span style="color: #406080;">:</span> note<span style="color: #406080;">:</span> <span style="color: #200080; font-weight: bold;">template</span> argument deduction<span style="color: #308080;">/</span>substitution failed<span style="color: #406080;">:</span>
<span style="color: #200080; font-weight: bold;">auto</span><span style="color: #308080;">.</span>cpp<span style="color: #406080;">:</span> In substitution of `<span style="color: #200080; font-weight: bold;">template</span><span style="color: #406080;"><</span><span style="color: #200080; font-weight: bold;">class</span> X<span style="color: #406080;">></span> constexpr decltype <span style="color: #308080;">(</span><span style="color: #308080;">(</span>x <span style="color: #308080;">+</span> <span style="color: maroon;">"</span><span style="color: #1060b6;">blah</span><span style="color: maroon;">"</span><span style="color: #308080;">)</span><span style="color: #308080;">)</span> append_blah<span style="color: #308080;">(</span>X<span style="color: #308080;">)</span> <span style="color: #308080;">[</span>with X <span style="color: #308080;">=</span> <span style="color: #200080; font-weight: bold;">const</span> <span style="color: #200080; font-weight: bold;">char</span><span style="color: #308080;">*</span><span style="color: #308080;">]</span>`<span style="color: #406080;">:</span>
<span style="color: #200080; font-weight: bold;">auto</span><span style="color: #308080;">.</span>cpp<span style="color: #406080;">:</span><span style="color: #008c00;">11</span><span style="color: #406080;">:</span><span style="color: #008c00;">20</span><span style="color: #406080;">:</span> required from here
<span style="color: #200080; font-weight: bold;">auto</span><span style="color: #308080;">.</span>cpp<span style="color: #406080;">:</span><span style="color: #008c00;">5</span><span style="color: #406080;">:</span><span style="color: #008c00;">47</span><span style="color: #406080;">:</span> <b>error<span style="color: #406080;">:</span> invalid operands of types `<span style="color: #200080; font-weight: bold;">const</span> <span style="color: #200080; font-weight: bold;">char</span><span style="color: #308080;">*</span>` <span style="color: #200080; font-weight: bold;">and</span> `<span style="color: #200080; font-weight: bold;">const</span> <span style="color: #200080; font-weight: bold;">char</span> <span style="color: #308080;">[</span><span style="color: #008c00;">5</span><span style="color: #308080;">]</span>` to binary `<span style="color: #200080; font-weight: bold;">operator</span><span style="color: #308080;">+</span></b></pre>
</body></html></pre>
</div>
That's GCC's output--more error than code! Even with the most relevant section bolded, it's a mess. Clang does a better job, though.<br />
<div>
<pre style="background: #f6f8ff; color: #000020;"><html><body style="background: #f6f8ff; color: #000020;"><pre><span style="color: #200080; font-weight: bold;">auto</span><span style="color: #308080;">.</span>cpp<span style="color: #406080;">:</span><span style="color: #008c00;">12</span><span style="color: #406080;">:</span><span style="color: #008c00;">3</span><span style="color: #406080;">:</span> error<span style="color: #406080;">:</span> no matching function <span style="color: #200080; font-weight: bold;">for</span> call to <span style="background: #dd9999; color: white; font-style: italic; font-weight: bold;">'append_blah'</span>
append_blah<span style="color: #308080;">(</span><span style="color: maroon;">"</span><span style="color: #1060b6;">nah</span><span style="color: maroon;">"</span><span style="color: #308080;">)</span><span style="color: #406080;">;</span>
<span style="color: #308080;">^</span><span style="color: #308080;">~</span><span style="color: #308080;">~</span><span style="color: #308080;">~</span><span style="color: #308080;">~</span><span style="color: #308080;">~</span><span style="color: #308080;">~</span><span style="color: #308080;">~</span><span style="color: #308080;">~</span><span style="color: #308080;">~</span><span style="color: #308080;">~</span>
<span style="color: #200080; font-weight: bold;">auto</span><span style="color: #308080;">.</span>cpp<span style="color: #406080;">:</span><span style="color: #008c00;">6</span><span style="color: #406080;">:</span><span style="color: #008c00;">16</span><span style="color: #406080;">:</span> note<span style="color: #406080;">:</span> candidate <span style="color: #200080; font-weight: bold;">template</span> ignored<span style="color: #406080;">:</span> substitution <span style="color: #003060;">failure</span> <span style="color: #308080;">[</span>with X <span style="color: #308080;">=</span> <span style="color: #200080; font-weight: bold;">char</span> <span style="color: #200080; font-weight: bold;">const</span><span style="color: #308080;">[</span><span style="color: #008c00;">4</span><span style="color: #308080;">]</span><span style="color: #308080;">]</span><span style="color: #406080;">:</span> invalid operands to binary expression <span style="color: #308080;">(</span><span style="background: #dd9999; color: white; font-style: italic; font-weight: bold;">'char const[4]'</span> <span style="color: #200080; font-weight: bold;">and</span> <span style="background: #dd9999; color: white; font-style: italic; font-weight: bold;">'const char *'</span><span style="color: #308080;">)</span>
constexpr <span style="color: #200080; font-weight: bold;">auto</span> append_blah<span style="color: #308080;">(</span>X<span style="color: #308080;">&</span> x<span style="color: #308080;">)</span> <span style="color: #308080;">-</span><span style="color: #308080;">></span> decltype<span style="color: #308080;">(</span>x <span style="color: #308080;">+</span> <span style="color: maroon;">"</span><span style="color: #1060b6;">blah</span><span style="color: maroon;">"</span><span style="color: #308080;">)</span>
<span style="color: #308080;">^</span> <span style="color: #308080;">~</span><span style="color: #308080;">~</span>
</pre>
</body></html></pre>
</div>
<div>
So now, thanks to <a href="http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2012/n3436.html">N3436</a>, that the standard dictates that <i>std::result_of</i> be SFINAE, too. Has the situation for GCC improved? First, the new source:</div>
<pre style="background: #f6f8ff; color: #000020;"><html><body style="background: #f6f8ff; color: #000020;"><pre><span style="color: #200080; font-weight: bold;">template</span><span style="color: #406080;"><</span><span style="color: #200080; font-weight: bold;">class</span> X<span style="color: #406080;">></span>
constexpr <b><span style="color: #0066ee;">std</span><span style="color: #406080;">::</span>result_of_t<span style="color: #406080;"><</span><span style="color: #0066ee;">std</span><span style="color: #406080;">::</span><span style="color: #003060;">plus</span><span style="color: #406080;"><</span><span style="color: #406080;">></span><span style="color: #308080;">(</span>X<span style="color: #308080;">,</span> <span style="color: #200080; font-weight: bold;">const</span> <span style="color: #200080; font-weight: bold;">char</span><span style="color: #308080;">*</span><span style="color: #308080;">)</span><span style="color: #406080;">></span></b>
append_blah<span style="color: #308080;">(</span>X x<span style="color: #308080;">)</span> <span style="color: #406080;">{</span>
<span style="color: #200080; font-weight: bold;">return</span> <span style="color: #0066ee;">std</span><span style="color: #406080;">::</span><span style="color: #003060;">plus</span><span style="color: #406080;"><</span><span style="color: #406080;">></span><span style="color: #406080;">{</span><span style="color: #406080;">}</span><span style="color: #308080;">(</span>x<span style="color: #308080;">,</span> <span style="color: maroon;">"</span><span style="color: #1060b6;">blah</span><span style="color: maroon;">"</span><span style="color: #308080;">)</span><span style="color: #406080;">;</span>
<span style="color: #406080;">}</span>
<span style="color: #200080; font-weight: bold;">int</span> <span style="color: #400000;">main</span><span style="color: #308080;">(</span><span style="color: #308080;">)</span> <span style="color: #406080;">{</span>
append_blah<span style="color: #308080;">(</span><span style="color: maroon;">"</span><span style="color: #1060b6;">nah</span><span style="color: maroon;">"</span><span style="color: #308080;">)</span><span style="color: #406080;">;</span>
<span style="color: #406080;">}</span>
</pre>
</body></html></pre>
<div>
And GCC's output:</div>
<pre style="background: #f6f8ff; color: #000020;"><html><body style="background: #f6f8ff; color: #000020;"><pre><span style="color: #200080; font-weight: bold;">auto</span><span style="color: #308080;">.</span>cpp<span style="color: #406080;">:</span> In function `<span style="color: #200080; font-weight: bold;">int</span> <span style="color: #400000;">main</span><span style="color: #308080;">(</span><span style="color: #308080;">)</span>`<span style="color: #406080;">:</span>
<span style="color: #200080; font-weight: bold;">auto</span><span style="color: #308080;">.</span>cpp<span style="color: #406080;">:</span><span style="color: #008c00;">12</span><span style="color: #406080;">:</span><span style="color: #008c00;">20</span><span style="color: #406080;">:</span> error<span style="color: #406080;">:</span> no matching function <span style="color: #200080; font-weight: bold;">for</span> call to `append_blah<span style="color: #308080;">(</span><span style="color: #200080; font-weight: bold;">const</span> <span style="color: #200080; font-weight: bold;">char</span> <span style="color: #308080;">[</span><span style="color: #008c00;">4</span><span style="color: #308080;">]</span><span style="color: #308080;">)</span>`
append_blah<span style="color: #308080;">(</span><span style="color: maroon;">"</span><span style="color: #1060b6;">nah</span><span style="color: maroon;">"</span><span style="color: #308080;">)</span><span style="color: #406080;">;</span>
<span style="color: #308080;">^</span>
<span style="color: #200080; font-weight: bold;">auto</span><span style="color: #308080;">.</span>cpp<span style="color: #406080;">:</span><span style="color: #008c00;">12</span><span style="color: #406080;">:</span><span style="color: #008c00;">20</span><span style="color: #406080;">:</span> note<span style="color: #406080;">:</span> candidate is<span style="color: #406080;">:</span>
<span style="color: #200080; font-weight: bold;">auto</span><span style="color: #308080;">.</span>cpp<span style="color: #406080;">:</span><span style="color: #008c00;">5</span><span style="color: #406080;">:</span><span style="color: #008c00;">16</span><span style="color: #406080;">:</span> note<span style="color: #406080;">:</span> <span style="color: #200080; font-weight: bold;">template</span><span style="color: #406080;"><</span><span style="color: #200080; font-weight: bold;">class</span> X<span style="color: #406080;">></span> constexpr <span style="color: #0066ee;">std</span><span style="color: #406080;">::</span>result_of_t<span style="color: #406080;"><</span><span style="color: #0066ee;">std</span><span style="color: #406080;">::</span><span style="color: #003060;">plus</span><span style="color: #406080;"><</span><span style="color: #200080; font-weight: bold;">void</span><span style="color: #406080;">></span><span style="color: #308080;">(</span>X<span style="color: #308080;">,</span> <span style="color: #200080; font-weight: bold;">const</span> <span style="color: #200080; font-weight: bold;">char</span><span style="color: #308080;">*</span><span style="color: #308080;">)</span><span style="color: #406080;">></span> append_blah<span style="color: #308080;">(</span>X<span style="color: #308080;">)</span>
constexpr <span style="color: #200080; font-weight: bold;">auto</span> append_blah<span style="color: #308080;">(</span>X x<span style="color: #308080;">)</span>
<span style="color: #308080;">^</span>
<span style="color: #200080; font-weight: bold;">auto</span><span style="color: #308080;">.</span>cpp<span style="color: #406080;">:</span><span style="color: #008c00;">5</span><span style="color: #406080;">:</span><span style="color: #008c00;">16</span><span style="color: #406080;">:</span> note<span style="color: #406080;">:</span> <span style="color: #200080; font-weight: bold;">template</span> argument deduction<span style="color: #308080;">/</span>substitution failed<span style="color: #406080;">:</span>
</pre>
</body></html></pre>
<div>
While the error message certainly gotten shorter, GCC no longer tells us <b>why</b> the function failed to compile, only that it was a candidate and "<i>template argument deduction/substitution failed</i>"--it doesn't even give us the type of <i>X</i>! SFINAE doesn't mean better diagnostics, it's just an overloading technique. Still, if it decreases the readability of error messages, then having many overloads, all failing, just compounds the problem.<br />
<br />
But clang does better, right?</div>
<pre style="background: #f6f8ff; color: #000020;"><html><body style="background: #f6f8ff; color: #000020;"><pre><span style="color: #200080; font-weight: bold;">auto</span><span style="color: #308080;">.</span>cpp<span style="color: #406080;">:</span><span style="color: #008c00;">12</span><span style="color: #406080;">:</span><span style="color: #008c00;">3</span><span style="color: #406080;">:</span> error<span style="color: #406080;">:</span> no matching function <span style="color: #200080; font-weight: bold;">for</span> call to <span style="background: #dd9999; color: white; font-style: italic; font-weight: bold;">'append_blah'</span>
append_blah<span style="color: #308080;">(</span><span style="color: maroon;">"</span><span style="color: #1060b6;">nah</span><span style="color: maroon;">"</span><span style="color: #308080;">)</span><span style="color: #406080;">;</span>
<span style="color: #308080;">^</span><span style="color: #308080;">~</span><span style="color: #308080;">~</span><span style="color: #308080;">~</span><span style="color: #308080;">~</span><span style="color: #308080;">~</span><span style="color: #308080;">~</span><span style="color: #308080;">~</span><span style="color: #308080;">~</span><span style="color: #308080;">~</span><span style="color: #308080;">~</span>
<span style="color: #200080; font-weight: bold;">auto</span><span style="color: #308080;">.</span>cpp<span style="color: #406080;">:</span><span style="color: #008c00;">5</span><span style="color: #406080;">:</span><span style="color: #008c00;">16</span><span style="color: #406080;">:</span> note<span style="color: #406080;">:</span> candidate <span style="color: #200080; font-weight: bold;">template</span> ignored<span style="color: #406080;">:</span>
substitution <span style="color: #003060;">failure</span> <span style="color: #308080;">[</span>with X <span style="color: #308080;">=</span> <span style="color: #200080; font-weight: bold;">const</span> <span style="color: #200080; font-weight: bold;">char</span> <span style="color: #308080;">*</span><span style="color: #308080;">]</span><span style="color: #406080;">:</span>
<b>no type named <span style="color: #1060b6;">'type'</span> in `<span style="color: #0066ee;">std</span><span style="color: #406080;">::</span>result_of<span style="color: #406080;"><</span><span style="color: #0066ee;">std</span><span style="color: #406080;">::</span><span style="color: #003060;">plus</span><span style="color: #406080;"><</span><span style="color: #200080; font-weight: bold;">void</span><span style="color: #406080;">></span> <span style="color: #308080;">(</span><span style="color: #200080; font-weight: bold;">const</span> <span style="color: #200080; font-weight: bold;">char</span> <span style="color: #308080;">*</span><span style="color: #308080;">,</span> <span style="color: #200080; font-weight: bold;">const</span> <span style="color: #200080; font-weight: bold;">char</span> <span style="color: #308080;">*</span><span style="color: #308080;">)</span><span style="color: #406080;">></span>`</b>
constexpr <span style="color: #200080; font-weight: bold;">auto</span> append_blah<span style="color: #308080;">(</span>X x<span style="color: #308080;">)</span>
<span style="color: #308080;">^</span>
<span style="color: #008c00;">1</span> error generated<span style="color: #308080;">.</span>
</pre>
</body></html></pre>
<div>
Better, yes, but not great. We still don't know why it failed. If you have deeply nested function calls, but your top-level function guarantees a return of <i>std::result_of_t<invalid-expression></i>, you'll never know why it failed to compile because <i>result_of</i> doesn't actually evaluate the semantics; it just checks the signatures and spits out that <i>result_of </i>has no <i>type</i>.</div>
<div>
<br /></div>
<div>
There simply <b>must</b> be a way of writing this such that GCC and clang will give a minimal, correct diagnosis of the problem, right? Well, let's try using <i>auto</i> as our return type.</div>
<pre style="background: #f6f8ff; color: #000020;"><html><body style="background: #f6f8ff; color: #000020;"><pre><span style="color: #200080; font-weight: bold;">template</span><span style="color: #406080;"><</span><span style="color: #200080; font-weight: bold;">class</span> X<span style="color: #406080;">></span>
constexpr <span style="color: #200080; font-weight: bold;">auto</span> append_blah<span style="color: #308080;">(</span>X x<span style="color: #308080;">)</span> <span style="color: #406080;">{</span>
<span style="color: #200080; font-weight: bold;">return</span> x <span style="color: #308080;">+</span> <span style="color: maroon;">"</span><span style="color: #1060b6;">blah</span><span style="color: maroon;">"</span><span style="color: #406080;">;</span>
<span style="color: #406080;">}</span>
<span style="color: #200080; font-weight: bold;">int</span> <span style="color: #400000;">main</span><span style="color: #308080;">(</span><span style="color: #308080;">)</span> <span style="color: #406080;">{</span>
append_blah<span style="color: #308080;">(</span><span style="color: maroon;">"</span><span style="color: #1060b6;">nah</span><span style="color: maroon;">"</span><span style="color: #308080;">)</span><span style="color: #406080;">;</span>
<span style="color: #406080;">}</span>
</pre>
</body></html></pre>
<div>
What does GCC have to say?</div>
<pre style="background: #f6f8ff; color: #000020;"><html><body style="background: #f6f8ff; color: #000020;"><pre><span style="color: #200080; font-weight: bold;">auto</span><span style="color: #308080;">.</span>cpp<span style="color: #406080;">:</span> In instantiation of `constexpr <span style="color: #200080; font-weight: bold;">auto</span> append_blah<span style="color: #308080;">(</span>X<span style="color: #308080;">)</span> <span style="color: #308080;">[</span>with X <span style="color: #308080;">=</span> <span style="color: #200080; font-weight: bold;">const</span> <span style="color: #200080; font-weight: bold;">char</span><span style="color: #308080;">*</span><span style="color: #308080;">]</span>`<span style="color: #406080;">:</span>
<span style="color: #200080; font-weight: bold;">auto</span><span style="color: #308080;">.</span>cpp<span style="color: #406080;">:</span><span style="color: #008c00;">11</span><span style="color: #406080;">:</span><span style="color: #008c00;">20</span><span style="color: #406080;">:</span> required from here
<span style="color: #200080; font-weight: bold;">auto</span><span style="color: #308080;">.</span>cpp<span style="color: #406080;">:</span><span style="color: #008c00;">7</span><span style="color: #406080;">:</span><span style="color: #008c00;">12</span><span style="color: #406080;">:</span> error<span style="color: #406080;">:</span> invalid operands of types `<span style="color: #200080; font-weight: bold;">const</span> <span style="color: #200080; font-weight: bold;">char</span><span style="color: #308080;">*</span>` <span style="color: #200080; font-weight: bold;">and</span> `<span style="color: #200080; font-weight: bold;">const</span> <span style="color: #200080; font-weight: bold;">char</span> <span style="color: #308080;">[</span><span style="color: #008c00;">5</span><span style="color: #308080;">]</span>` to binary `<span style="color: #200080; font-weight: bold;">operator</span><span style="color: #308080;">+</span>`
<span style="color: #200080; font-weight: bold;">return</span> x <span style="color: #308080;">+</span> <span style="color: maroon;">"</span><span style="color: #1060b6;">blah</span><span style="color: maroon;">"</span><span style="color: #406080;">;</span>
<span style="color: #308080;">^</span>
</pre>
</body></html></pre>
<div>
Wow, straight to the point and totally accurate! We get a nice, minimal trail of what GCC had to instantiate to find the error, but not a hard-to-follow mess of output. How does clang handle it?</div>
<pre style="background: #f6f8ff; color: #000020;"><html><body style="background: #f6f8ff; color: #000020;"><pre><span style="color: #200080; font-weight: bold;">auto</span><span style="color: #308080;">.</span>cpp<span style="color: #406080;">:</span><span style="color: #008c00;">7</span><span style="color: #406080;">:</span><span style="color: #008c00;">12</span><span style="color: #406080;">:</span> error<span style="color: #406080;">:</span> invalid operands to binary expression <span style="color: #308080;">(</span>`<span style="color: #200080; font-weight: bold;">const</span> <span style="color: #200080; font-weight: bold;">char</span> <span style="color: #308080;">*</span>` <span style="color: #200080; font-weight: bold;">and</span> `<span style="color: #200080; font-weight: bold;">const</span> <span style="color: #200080; font-weight: bold;">char</span> <span style="color: #308080;">*</span>`<span style="color: #308080;">)</span>
<span style="color: #200080; font-weight: bold;">return</span> x <span style="color: #308080;">+</span> <span style="color: maroon;">"</span><span style="color: #1060b6;">blah</span><span style="color: maroon;">"</span><span style="color: #406080;">;</span>
<span style="color: #308080;">~</span> <span style="color: #308080;">^</span> <span style="color: #308080;">~</span><span style="color: #308080;">~</span><span style="color: #308080;">~</span><span style="color: #308080;">~</span><span style="color: #308080;">~</span><span style="color: #308080;">~</span>
<span style="color: #200080; font-weight: bold;">auto</span><span style="color: #308080;">.</span>cpp<span style="color: #406080;">:</span><span style="color: #008c00;">11</span><span style="color: #406080;">:</span><span style="color: #008c00;">3</span><span style="color: #406080;">:</span> note<span style="color: #406080;">:</span> in instantiation of function <span style="color: #200080; font-weight: bold;">template</span> specialization `append_blah<span style="color: #406080;"><</span><span style="color: #200080; font-weight: bold;">const</span> <span style="color: #200080; font-weight: bold;">char</span> <span style="color: #308080;">*</span><span style="color: #406080;">></span>` requested here
append_blah<span style="color: #308080;">(</span><span style="color: maroon;">"</span><span style="color: #1060b6;">nah</span><span style="color: maroon;">"</span><span style="color: #308080;">)</span><span style="color: #406080;">;</span>
<span style="color: #308080;">^</span>
<span style="color: #008c00;">1</span> error generated<span style="color: #308080;">.</span>
</pre>
</body></html></pre>
<div>
Again, exactly what I wanted!</div>
<div>
<br /></div>
<div>
So in terms of getting good and useful error messages, <i>auto</i> or <i>decltype(auto)</i> gives the best, followed by <i>decltype</i>, and <i>result_of</i> gives the absolute worst. The problem is that <i>auto</i> can't participate in SFINAE, so you can't overload a set of functions on <i>auto</i> and that leaves you with <i>std::enable_if</i>, <i>std::result_of,</i> or <i>decltype</i>. <br />
<br />
Before concluding, let's look at a slightly more obfuscated example.</div>
<pre style="background: #f6f8ff; color: #000020;"><html><body style="background: #f6f8ff; color: #000020;"><pre><span style="color: #004a43;">#</span><span style="color: #004a43;">include </span><span style="color: maroon;"><</span><span style="color: #40015a;">functional</span><span style="color: maroon;">></span>
constexpr <span style="color: #200080; font-weight: bold;">struct</span> plus_f <span style="color: #406080;">{</span>
<span style="color: #200080; font-weight: bold;">template</span><span style="color: #406080;"><</span><span style="color: #200080; font-weight: bold;">class</span> X<span style="color: #308080;">,</span> <span style="color: #200080; font-weight: bold;">class</span> Y<span style="color: #406080;">></span>
constexpr <span style="color: #200080; font-weight: bold;">auto</span> <span style="color: #200080; font-weight: bold;">operator</span><span style="color: #308080;">(</span><span style="color: #308080;">)</span> <span style="color: #308080;">(</span>X<span style="color: #308080;">&</span><span style="color: #308080;">&</span> x<span style="color: #308080;">,</span> Y<span style="color: #308080;">&</span><span style="color: #308080;">&</span> y<span style="color: #308080;">)</span> <span style="color: #200080; font-weight: bold;">const</span>
<span style="color: #308080;">-</span><span style="color: #308080;">></span> decltype<span style="color: #308080;">(</span><span style="color: #0066ee;">std</span><span style="color: #406080;">::</span>declval<span style="color: #406080;"><</span>X<span style="color: #406080;">></span><span style="color: #308080;">(</span><span style="color: #308080;">)</span> <span style="color: #308080;">+</span> <span style="color: #0066ee;">std</span><span style="color: #406080;">::</span>declval<span style="color: #406080;"><</span>Y<span style="color: #406080;">></span><span style="color: #308080;">(</span><span style="color: #308080;">)</span><span style="color: #308080;">)</span>
<span style="color: #406080;">{</span>
<span style="color: #200080; font-weight: bold;">return</span> <span style="color: #0066ee;">std</span><span style="color: #406080;">::</span>forward<span style="color: #406080;"><</span>X<span style="color: #406080;">></span><span style="color: #308080;">(</span>x<span style="color: #308080;">)</span> <span style="color: #308080;">+</span> <span style="color: #0066ee;">std</span><span style="color: #406080;">::</span>forward<span style="color: #406080;"><</span>Y<span style="color: #406080;">></span><span style="color: #308080;">(</span>y<span style="color: #308080;">)</span><span style="color: #406080;">;</span>
<span style="color: #406080;">}</span>
<span style="color: #406080;">}</span> <span style="color: #003060;">plus</span><span style="color: #406080;">{</span><span style="color: #406080;">}</span><span style="color: #406080;">;</span>
<span style="color: #200080; font-weight: bold;">template</span><span style="color: #406080;"><</span><span style="color: #200080; font-weight: bold;">class</span> X<span style="color: #406080;">></span>
constexpr <span style="color: #200080; font-weight: bold;">auto</span> append_blah<span style="color: #308080;">(</span>X x<span style="color: #308080;">)</span> <span style="color: #308080;">-</span><span style="color: #308080;">></span> decltype<span style="color: #308080;">(</span><span style="color: #003060;">plus</span><span style="color: #308080;">(</span>x<span style="color: #308080;">,</span> <span style="color: maroon;">"</span><span style="color: #1060b6;">blah</span><span style="color: maroon;">"</span><span style="color: #308080;">)</span><span style="color: #308080;">)</span> <span style="color: #406080;">{</span>
<span style="color: #200080; font-weight: bold;">return</span> <span style="color: #003060;">plus</span><span style="color: #308080;">(</span>x<span style="color: #308080;">,</span> <span style="color: maroon;">"</span><span style="color: #1060b6;">blah</span><span style="color: maroon;">"</span><span style="color: #308080;">)</span><span style="color: #406080;">;</span>
<span style="color: #406080;">}</span>
<span style="color: #200080; font-weight: bold;">int</span> <span style="color: #400000;">main</span><span style="color: #308080;">(</span><span style="color: #308080;">)</span> <span style="color: #406080;">{</span>
append_blah<span style="color: #308080;">(</span><span style="color: maroon;">"</span><span style="color: #1060b6;">nah</span><span style="color: maroon;">"</span><span style="color: #308080;">)</span><span style="color: #406080;">;</span>
<span style="color: #406080;">}</span>
</pre>
</body></html></pre>
<div>
Now, in order to find the error, the compiler has to go just two levels deep, through <i>append_blah</i>, then to <i>plus_f</i>, and evaluage <i>x + "blah"</i>. But will the error message go that deep? We know that if we used <i>std::result_of</i>, it would just say that <i>result_of<plus_f(X, const char*)></i> has no <i>'type'</i>. But this is <i>decltype</i>!<br />
<br />
First, GCC:</div>
<pre style="background: #f6f8ff; color: #000020;"><html><body style="background: #f6f8ff; color: #000020;"><pre><span style="color: #200080; font-weight: bold;">auto</span><span style="color: #308080;">.</span>cpp<span style="color: #406080;">:</span> In function `<span style="color: #200080; font-weight: bold;">int</span> <span style="color: #400000;">main</span><span style="color: #308080;">(</span><span style="color: #308080;">)</span>`<span style="color: #406080;">:</span>
<span style="color: #200080; font-weight: bold;">auto</span><span style="color: #308080;">.</span>cpp<span style="color: #406080;">:</span><span style="color: #008c00;">19</span><span style="color: #406080;">:</span><span style="color: #008c00;">20</span><span style="color: #406080;">:</span> error<span style="color: #406080;">:</span> no matching function <span style="color: #200080; font-weight: bold;">for</span> call to `append_blah<span style="color: #308080;">(</span><span style="color: #200080; font-weight: bold;">const</span> <span style="color: #200080; font-weight: bold;">char</span> <span style="color: #308080;">[</span><span style="color: #008c00;">4</span><span style="color: #308080;">]</span><span style="color: #308080;">)</span>`
append_blah<span style="color: #308080;">(</span><span style="color: maroon;">"</span><span style="color: #1060b6;">nah</span><span style="color: maroon;">"</span><span style="color: #308080;">)</span><span style="color: #406080;">;</span>
<span style="color: #308080;">^</span>
<span style="color: #200080; font-weight: bold;">auto</span><span style="color: #308080;">.</span>cpp<span style="color: #406080;">:</span><span style="color: #008c00;">19</span><span style="color: #406080;">:</span><span style="color: #008c00;">20</span><span style="color: #406080;">:</span> note<span style="color: #406080;">:</span> candidate is<span style="color: #406080;">:</span>
<span style="color: #200080; font-weight: bold;">auto</span><span style="color: #308080;">.</span>cpp<span style="color: #406080;">:</span><span style="color: #008c00;">14</span><span style="color: #406080;">:</span><span style="color: #008c00;">16</span><span style="color: #406080;">:</span> note<span style="color: #406080;">:</span> <span style="color: #200080; font-weight: bold;">template</span><span style="color: #406080;"><</span><span style="color: #200080; font-weight: bold;">class</span> X<span style="color: #406080;">></span> constexpr decltype <span style="color: #308080;">(</span><span style="color: #003060;">plus</span><span style="color: #308080;">(</span>x<span style="color: #308080;">,</span> <span style="color: maroon;">"</span><span style="color: #1060b6;">blah</span><span style="color: maroon;">"</span><span style="color: #308080;">)</span><span style="color: #308080;">)</span> append_blah<span style="color: #308080;">(</span>X<span style="color: #308080;">)</span>
constexpr <span style="color: #200080; font-weight: bold;">auto</span> append_blah<span style="color: #308080;">(</span>X x<span style="color: #308080;">)</span> <span style="color: #308080;">-</span><span style="color: #308080;">></span> decltype<span style="color: #308080;">(</span><span style="color: #003060;">plus</span><span style="color: #308080;">(</span>x<span style="color: #308080;">,</span> <span style="color: maroon;">"</span><span style="color: #1060b6;">blah</span><span style="color: maroon;">"</span><span style="color: #308080;">)</span><span style="color: #308080;">)</span> <span style="color: #406080;">{</span>
<span style="color: #308080;">^</span>
<span style="color: #200080; font-weight: bold;">auto</span><span style="color: #308080;">.</span>cpp<span style="color: #406080;">:</span><span style="color: #008c00;">14</span><span style="color: #406080;">:</span><span style="color: #008c00;">16</span><span style="color: #406080;">:</span> note<span style="color: #406080;">:</span> <span style="color: #200080; font-weight: bold;">template</span> argument deduction<span style="color: #308080;">/</span>substitution failed<span style="color: #406080;">:</span>
<span style="color: #200080; font-weight: bold;">auto</span><span style="color: #308080;">.</span>cpp<span style="color: #406080;">:</span> In substitution of `<span style="color: #200080; font-weight: bold;">template</span><span style="color: #406080;"><</span><span style="color: #200080; font-weight: bold;">class</span> X<span style="color: #406080;">></span> constexpr decltype <span style="color: #308080;">(</span><span style="color: #003060;">plus</span><span style="color: #308080;">(</span>x<span style="color: #308080;">,</span> <span style="color: maroon;">"</span><span style="color: #1060b6;">blah</span><span style="color: maroon;">"</span><span style="color: #308080;">)</span><span style="color: #308080;">)</span> append_blah<span style="color: #308080;">(</span>X<span style="color: #308080;">)</span> <span style="color: #308080;">[</span>with X <span style="color: #308080;">=</span> <span style="color: #200080; font-weight: bold;">const</span> <span style="color: #200080; font-weight: bold;">char</span><span style="color: #308080;">*</span><span style="color: #308080;">]</span>`<span style="color: #406080;">:</span>
<span style="color: #200080; font-weight: bold;">auto</span><span style="color: #308080;">.</span>cpp<span style="color: #406080;">:</span><span style="color: #008c00;">19</span><span style="color: #406080;">:</span><span style="color: #008c00;">20</span><span style="color: #406080;">:</span> required from here
<span style="color: #200080; font-weight: bold;">auto</span><span style="color: #308080;">.</span>cpp<span style="color: #406080;">:</span><span style="color: #008c00;">14</span><span style="color: #406080;">:</span><span style="color: #008c00;">59</span><span style="color: #406080;">:</span> error<span style="color: #406080;">:</span> no match <span style="color: #200080; font-weight: bold;">for</span> call to `<span style="color: #308080;">(</span><span style="color: #200080; font-weight: bold;">const</span> plus_f<span style="color: #308080;">)</span> <span style="color: #308080;">(</span><span style="color: #200080; font-weight: bold;">const</span> <span style="color: #200080; font-weight: bold;">char</span><span style="color: #308080;">*</span><span style="color: #308080;">&</span><span style="color: #308080;">,</span> <span style="color: #200080; font-weight: bold;">const</span> <span style="color: #200080; font-weight: bold;">char</span> <span style="color: #308080;">[</span><span style="color: #008c00;">5</span><span style="color: #308080;">]</span><span style="color: #308080;">)</span>`
constexpr <span style="color: #200080; font-weight: bold;">auto</span> append_blah<span style="color: #308080;">(</span>X x<span style="color: #308080;">)</span> <span style="color: #308080;">-</span><span style="color: #308080;">></span> decltype<span style="color: #308080;">(</span><span style="color: #003060;">plus</span><span style="color: #308080;">(</span>x<span style="color: #308080;">,</span> <span style="color: maroon;">"</span><span style="color: #1060b6;">blah</span><span style="color: maroon;">"</span><span style="color: #308080;">)</span><span style="color: #308080;">)</span> <span style="color: #406080;">{</span>
<span style="color: #308080;">^</span>
<span style="color: #200080; font-weight: bold;">auto</span><span style="color: #308080;">.</span>cpp<span style="color: #406080;">:</span><span style="color: #008c00;">4</span><span style="color: #406080;">:</span><span style="color: #008c00;">18</span><span style="color: #406080;">:</span> note<span style="color: #406080;">:</span> candidate is<span style="color: #406080;">:</span>
constexpr <span style="color: #200080; font-weight: bold;">struct</span> plus_f <span style="color: #406080;">{</span>
<span style="color: #308080;">^</span>
<span style="color: #200080; font-weight: bold;">auto</span><span style="color: #308080;">.</span>cpp<span style="color: #406080;">:</span><span style="color: #008c00;">6</span><span style="color: #406080;">:</span><span style="color: #008c00;">18</span><span style="color: #406080;">:</span> note<span style="color: #406080;">:</span> <span style="color: #200080; font-weight: bold;">template</span><span style="color: #406080;"><</span><span style="color: #200080; font-weight: bold;">class</span> X<span style="color: #308080;">,</span> <span style="color: #200080; font-weight: bold;">class</span> Y<span style="color: #406080;">></span> constexpr decltype <span style="color: #308080;">(</span><span style="color: #308080;">(</span>declval<span style="color: #406080;"><</span>X<span style="color: #406080;">></span><span style="color: #308080;">(</span><span style="color: #308080;">)</span> <span style="color: #308080;">+</span> declval<span style="color: #406080;"><</span>Y<span style="color: #406080;">></span><span style="color: #308080;">(</span><span style="color: #308080;">)</span><span style="color: #308080;">)</span><span style="color: #308080;">)</span> plus_f<span style="color: #406080;">::</span><span style="color: #200080; font-weight: bold;">operator</span><span style="color: #308080;">(</span><span style="color: #308080;">)</span><span style="color: #308080;">(</span>X<span style="color: #308080;">&</span><span style="color: #308080;">&</span><span style="color: #308080;">,</span> Y<span style="color: #308080;">&</span><span style="color: #308080;">&</span><span style="color: #308080;">)</span> <span style="color: #200080; font-weight: bold;">const</span>
constexpr <span style="color: #200080; font-weight: bold;">auto</span> <span style="color: #200080; font-weight: bold;">operator</span><span style="color: #308080;">(</span><span style="color: #308080;">)</span> <span style="color: #308080;">(</span>X<span style="color: #308080;">&</span><span style="color: #308080;">&</span> x<span style="color: #308080;">,</span> Y<span style="color: #308080;">&</span><span style="color: #308080;">&</span> y<span style="color: #308080;">)</span> <span style="color: #200080; font-weight: bold;">const</span>
<span style="color: #308080;">^</span>
<span style="color: #200080; font-weight: bold;">auto</span><span style="color: #308080;">.</span>cpp<span style="color: #406080;">:</span><span style="color: #008c00;">6</span><span style="color: #406080;">:</span><span style="color: #008c00;">18</span><span style="color: #406080;">:</span> note<span style="color: #406080;">:</span> <span style="color: #200080; font-weight: bold;">template</span> argument deduction<span style="color: #308080;">/</span>substitution failed<span style="color: #406080;">:</span>
<span style="color: #200080; font-weight: bold;">auto</span><span style="color: #308080;">.</span>cpp<span style="color: #406080;">:</span> In substitution of `<span style="color: #200080; font-weight: bold;">template</span><span style="color: #406080;"><</span><span style="color: #200080; font-weight: bold;">class</span> X<span style="color: #308080;">,</span> <span style="color: #200080; font-weight: bold;">class</span> Y<span style="color: #406080;">></span> constexpr decltype <span style="color: #308080;">(</span><span style="color: #308080;">(</span>declval<span style="color: #406080;"><</span>X<span style="color: #406080;">></span><span style="color: #308080;">(</span><span style="color: #308080;">)</span> <span style="color: #308080;">+</span> declval<span style="color: #406080;"><</span>Y<span style="color: #406080;">></span><span style="color: #308080;">(</span><span style="color: #308080;">)</span><span style="color: #308080;">)</span><span style="color: #308080;">)</span> plus_f<span style="color: #406080;">::</span><span style="color: #200080; font-weight: bold;">operator</span><span style="color: #308080;">(</span><span style="color: #308080;">)</span><span style="color: #308080;">(</span>X<span style="color: #308080;">&</span><span style="color: #308080;">&</span><span style="color: #308080;">,</span> Y<span style="color: #308080;">&</span><span style="color: #308080;">&</span><span style="color: #308080;">)</span> <span style="color: #200080; font-weight: bold;">const</span> <span style="color: #308080;">[</span>with X <span style="color: #308080;">=</span> <span style="color: #200080; font-weight: bold;">const</span> <span style="color: #200080; font-weight: bold;">char</span><span style="color: #308080;">*</span><span style="color: #308080;">&</span><span style="color: #406080;">;</span> Y <span style="color: #308080;">=</span> <span style="color: #200080; font-weight: bold;">const</span> <span style="color: #200080; font-weight: bold;">char</span> <span style="color: #308080;">(</span><span style="color: #308080;">&</span><span style="color: #308080;">)</span><span style="color: #308080;">[</span><span style="color: #008c00;">5</span><span style="color: #308080;">]</span><span style="color: #308080;">]</span>`<span style="color: #406080;">:</span>
<span style="color: #200080; font-weight: bold;">auto</span><span style="color: #308080;">.</span>cpp<span style="color: #406080;">:</span><span style="color: #008c00;">14</span><span style="color: #406080;">:</span><span style="color: #008c00;">59</span><span style="color: #406080;">:</span> required by substitution of `<span style="color: #200080; font-weight: bold;">template</span><span style="color: #406080;"><</span><span style="color: #200080; font-weight: bold;">class</span> X<span style="color: #406080;">></span> constexpr decltype <span style="color: #308080;">(</span><span style="color: #003060;">plus</span><span style="color: #308080;">(</span>x<span style="color: #308080;">,</span> <span style="color: maroon;">"</span><span style="color: #1060b6;">blah</span><span style="color: maroon;">"</span><span style="color: #308080;">)</span><span style="color: #308080;">)</span> append_blah<span style="color: #308080;">(</span>X<span style="color: #308080;">)</span> <span style="color: #308080;">[</span>with X <span style="color: #308080;">=</span> <span style="color: #200080; font-weight: bold;">const</span> <span style="color: #200080; font-weight: bold;">char</span><span style="color: #308080;">*</span><span style="color: #308080;">]</span>`
<span style="color: #200080; font-weight: bold;">auto</span><span style="color: #308080;">.</span>cpp<span style="color: #406080;">:</span><span style="color: #008c00;">19</span><span style="color: #406080;">:</span><span style="color: #008c00;">20</span><span style="color: #406080;">:</span> required from here
<span style="color: #200080; font-weight: bold;">auto</span><span style="color: #308080;">.</span>cpp<span style="color: #406080;">:</span><span style="color: #008c00;">7</span><span style="color: #406080;">:</span><span style="color: #008c00;">35</span><span style="color: #406080;">:</span> error<span style="color: #406080;">:</span> invalid operands of types `<span style="color: #200080; font-weight: bold;">const</span> <span style="color: #200080; font-weight: bold;">char</span><span style="color: #308080;">*</span>` <span style="color: #200080; font-weight: bold;">and</span> `<span style="color: #200080; font-weight: bold;">const</span> <span style="color: #200080; font-weight: bold;">char</span> <span style="color: #308080;">[</span><span style="color: #008c00;">5</span><span style="color: #308080;">]</span>` to binary `<span style="color: #200080; font-weight: bold;">operator</span><span style="color: #308080;">+</span>`
<span style="color: #308080;">-</span><span style="color: #308080;">></span> decltype<span style="color: #308080;">(</span><span style="color: #0066ee;">std</span><span style="color: #406080;">::</span>declval<span style="color: #406080;"><</span>X<span style="color: #406080;">></span><span style="color: #308080;">(</span><span style="color: #308080;">)</span> <span style="color: #308080;">+</span> <span style="color: #0066ee;">std</span><span style="color: #406080;">::</span>declval<span style="color: #406080;"><</span>Y<span style="color: #406080;">></span><span style="color: #308080;">(</span><span style="color: #308080;">)</span><span style="color: #308080;">)</span>
<span style="color: #308080;">^</span>
</pre>
</body></html></pre>
<div>
GCC feels obligated to tell us <b>why</b>, so in terms of having an accurate, if not in an overly-verbose, error messages, it does its job. It'll take some digging, but a user will eventually be able to find out why it failed. But curiously, clang does no better than <i>std::result_of</i>.</div>
<pre style="background: #f6f8ff; color: #000020;"><html><body style="background: #f6f8ff; color: #000020;"><pre><span style="color: #200080; font-weight: bold;">auto</span><span style="color: #308080;">.</span>cpp<span style="color: #406080;">:</span><span style="color: #008c00;">19</span><span style="color: #406080;">:</span><span style="color: #008c00;">3</span><span style="color: #406080;">:</span> error<span style="color: #406080;">:</span> no matching function <span style="color: #200080; font-weight: bold;">for</span> call to `append_blah`
append_blah<span style="color: #308080;">(</span><span style="color: maroon;">"</span><span style="color: #1060b6;">nah</span><span style="color: maroon;">"</span><span style="color: #308080;">)</span><span style="color: #406080;">;</span>
<span style="color: #308080;">^</span><span style="color: #308080;">~</span><span style="color: #308080;">~</span><span style="color: #308080;">~</span><span style="color: #308080;">~</span><span style="color: #308080;">~</span><span style="color: #308080;">~</span><span style="color: #308080;">~</span><span style="color: #308080;">~</span><span style="color: #308080;">~</span><span style="color: #308080;">~</span>
<span style="color: #200080; font-weight: bold;">auto</span><span style="color: #308080;">.</span>cpp<span style="color: #406080;">:</span><span style="color: #008c00;">14</span><span style="color: #406080;">:</span><span style="color: #008c00;">16</span><span style="color: #406080;">:</span> note<span style="color: #406080;">:</span> candidate <span style="color: #200080; font-weight: bold;">template</span> ignored<span style="color: #406080;">:</span>
substitution <span style="color: #003060;">failure</span> <span style="color: #308080;">[</span>with X <span style="color: #308080;">=</span> <span style="color: #200080; font-weight: bold;">const</span> <span style="color: #200080; font-weight: bold;">char</span> <span style="color: #308080;">*</span><span style="color: #308080;">]</span><span style="color: #406080;">:</span>
no matching function <span style="color: #200080; font-weight: bold;">for</span> call to object of type `<span style="color: #200080; font-weight: bold;">const</span> <span style="color: #200080; font-weight: bold;">struct</span> plus_f`
constexpr <span style="color: #200080; font-weight: bold;">auto</span> append_blah<span style="color: #308080;">(</span>X x<span style="color: #308080;">)</span> <span style="color: #308080;">-</span><span style="color: #308080;">></span> decltype<span style="color: #308080;">(</span><span style="color: #003060;">plus</span><span style="color: #308080;">(</span>x<span style="color: #308080;">,</span> <span style="color: maroon;">"</span><span style="color: #1060b6;">blah</span><span style="color: maroon;">"</span><span style="color: #308080;">)</span><span style="color: #308080;">)</span> <span style="color: #406080;">{</span>
<span style="color: #308080;">^</span> <span style="color: #308080;">~</span><span style="color: #308080;">~</span><span style="color: #308080;">~</span><span style="color: #308080;">~</span>
<span style="color: #008c00;">1</span> error generated<span style="color: #308080;">.</span>
</pre>
</body></html></pre>
<div>
This doesn't even tell us the types of the arguments to <i>plus_f</i> that caused the failure! If <i>plus_f</i> were a more complicated function with a slight semantic error, we'd be left clueless.<br />
<br />
Just for comparison, here's the output with GCC, using <i>auto</i>:<br />
<pre style="background: #f6f8ff; color: #000020;"><html><body style="background: #f6f8ff; color: #000020;"><pre><span style="color: #200080; font-weight: bold;">auto</span><span style="color: #308080;">.</span>cpp<span style="color: #406080;">:</span> In instantiation of `constexpr <span style="color: #200080; font-weight: bold;">auto</span> plus_f<span style="color: #406080;">::</span><span style="color: #200080; font-weight: bold;">operator</span><span style="color: #308080;">(</span><span style="color: #308080;">)</span><span style="color: #308080;">(</span>X<span style="color: #308080;">&</span><span style="color: #308080;">&</span><span style="color: #308080;">,</span> Y<span style="color: #308080;">&</span><span style="color: #308080;">&</span><span style="color: #308080;">)</span> <span style="color: #200080; font-weight: bold;">const</span> <span style="color: #308080;">[</span>with X <span style="color: #308080;">=</span> <span style="color: #200080; font-weight: bold;">const</span> <span style="color: #200080; font-weight: bold;">char</span><span style="color: #308080;">*</span><span style="color: #308080;">&</span><span style="color: #406080;">;</span> Y <span style="color: #308080;">=</span> <span style="color: #200080; font-weight: bold;">const</span> <span style="color: #200080; font-weight: bold;">char</span> <span style="color: #308080;">(</span><span style="color: #308080;">&</span><span style="color: #308080;">)</span><span style="color: #308080;">[</span><span style="color: #008c00;">5</span><span style="color: #308080;">]</span><span style="color: #308080;">]</span>`<span style="color: #406080;">:</span>
<span style="color: #200080; font-weight: bold;">auto</span><span style="color: #308080;">.</span>cpp<span style="color: #406080;">:</span><span style="color: #008c00;">14</span><span style="color: #406080;">:</span><span style="color: #008c00;">24</span><span style="color: #406080;">:</span> required from `constexpr <span style="color: #200080; font-weight: bold;">auto</span> append_blah<span style="color: #308080;">(</span>X<span style="color: #308080;">)</span> <span style="color: #308080;">[</span>with X <span style="color: #308080;">=</span> <span style="color: #200080; font-weight: bold;">const</span> <span style="color: #200080; font-weight: bold;">char</span><span style="color: #308080;">*</span><span style="color: #308080;">]</span>`
<span style="color: #200080; font-weight: bold;">auto</span><span style="color: #308080;">.</span>cpp<span style="color: #406080;">:</span><span style="color: #008c00;">18</span><span style="color: #406080;">:</span><span style="color: #008c00;">20</span><span style="color: #406080;">:</span> required from here
<span style="color: #200080; font-weight: bold;">auto</span><span style="color: #308080;">.</span>cpp<span style="color: #406080;">:</span><span style="color: #008c00;">8</span><span style="color: #406080;">:</span><span style="color: #008c00;">31</span><span style="color: #406080;">:</span> error<span style="color: #406080;">:</span> invalid operands of types `<span style="color: #200080; font-weight: bold;">const</span> <span style="color: #200080; font-weight: bold;">char</span><span style="color: #308080;">*</span>` <span style="color: #200080; font-weight: bold;">and</span> `<span style="color: #200080; font-weight: bold;">const</span> <span style="color: #200080; font-weight: bold;">char</span> <span style="color: #308080;">[</span><span style="color: #008c00;">5</span><span style="color: #308080;">]</span>` to binary `<span style="color: #200080; font-weight: bold;">operator</span><span style="color: #308080;">+</span>`
<span style="color: #200080; font-weight: bold;">return</span> <span style="color: #0066ee;">std</span><span style="color: #406080;">::</span>forward<span style="color: #406080;"><</span>X<span style="color: #406080;">></span><span style="color: #308080;">(</span>x<span style="color: #308080;">)</span> <span style="color: #308080;">+</span> <span style="color: #0066ee;">std</span><span style="color: #406080;">::</span>forward<span style="color: #406080;"><</span>Y<span style="color: #406080;">></span><span style="color: #308080;">(</span>y<span style="color: #308080;">)</span><span style="color: #406080;">;</span>
<span style="color: #308080;">^</span>
</pre>
</body></html></pre>
So, when it comes to <i>decltype</i> vs. <i>std::result_of</i>, neither really produced ideal diagnostics and both can be rather unhelpful. <i>decltype</i> at least gives us a little more information, but only with GCC.</div>
<div>
<br /></div>
<div>
I discovered this while working on <a href="https://github.com/splinterofchaos/fu">FU</a>. One of my basic functions that I based the library off of had a very simple bug. Not realizing this, I wrote a small function that called another that called another that eventually landed in that one. When I tried to compile, I got several pages of errors, and they kept suggesting that the problem was in one of the mid-level functions. Assuming I might have gotten the <i>result_of</i> expression wrong, I decided to just use <i>auto</i> and let the compiler figure it out. Then, GCC and clang both told me that the next function in the call tree was the culprit. I continued to mark function by function as returning <i>auto</i> until I found the problem. I had tried to pass an rvalue reference to a function taking a non-const reference. After making it pass by const reference, everything else worked.</div>
<div>
<br /></div>
<div>
Does this mean that we should always prefer <i>auto</i> to <i>result_of</i> or <i>decltype</i> when we don't need SFINAE? I think it should be evaluated on a case-to-case basis. In terms of documentation, <i>auto</i> gives the user a pretty muddy perception on what the function will return. For something like <i>std::plus<></i>, that's not a problem because addition generally doesn't change the type of its arguments too much. For something simple like <i><a href="http://en.cppreference.com/w/cpp/utility/functional/invoke">std::invoke</a></i> (C++17), <i>std::result_of_t<F&&(X&&...)></i> gives us the clearest documentation about how it works--it perfect forwards <i>F</i> and the arguments, <i>X...</i>--but this gives us the worst diagnosis if <i>F</i> can't be called with the arguments. <i>decltype</i> would give the best error message (with clang when the error is shallow, otherwise only with GCC), but <i>decltype(std::forward<F>(f)(std::forward<X>(x)...))</i> doesn't read as nicely as <i>result_of<F&&(X&&...)></i>. </div>
<div>
<br /></div>
<div>
Whether to use <i>auto</i>, <i>result_of</i>, or <i>decltype</i> is a matter of priorities, weighing the importance of documentation (<i>result_of</i>; maybe <i>enable_if</i>) against semantics (<i>decltype</i>) and diagnostics (<i>auto</i>). For FU, a library that depends heavily on dependent typing, I will probably over time switch entirely to using <i>auto</i> because determining the cause of errors is of great importance for making it usable. Users will likely be confused with messages like "<i>result_of<multary_n_f<1,lassoc_f>(UserFunc, TypeA, TypeB, TypeC)> has no member, `type`</i>". At the same time, this feels like poor style--an unfortunate dilemma. With any hope, GCC and clang will improve how they propagate errors over time.<br />
<br />
For information on how to improve the diagnostics of functions using <i>decltype</i>, check out pfultz2's article, "<a href="http://pfultz2.com/blog/2015/01/31/improving-error-messages/">Improving error messages in C++ by transporting substitution failures</a>".</div>
Splinter of Chaoshttp://www.blogger.com/profile/14715348728512729776noreply@blogger.com0tag:blogger.com,1999:blog-8638474063464489651.post-12180887034945063962015-02-17T10:45:00.000-08:002015-02-17T11:47:23.820-08:00Common algorithm patterns.Of the STL, <i><algorithm></i> may be the most well-used non-container library, but often requires a level of verbosity that requires just as much typing as the hand-written loop, making it not always feel so convenient. It benefits code that uses it with increased clarity and mathematical soundness, so reducing the syntactic overhead should be a goal of those using it. Today I will talk about these problems and demonstrate ways of making it more terse and sound.<br />
<br />
Usually, I try to put all the code in my articles in one gist, but this time I will base it off my most recent work--a library called <a href="https://github.com/splinterofchaos/fu">FU</a> (Functional Utilities)--for sake of simplicity. Still, it applies as well to <a href="https://github.com/pfultz2/Fit/">fit</a>, <a href="https://github.com/beark/ftl">FTL</a>, and to some degree, <a href="https://github.com/ericniebler/range-v3">range-v3</a>, if you use any of those.<br />
<br />
<h4>
Projection</h4>
<div>
<br />
We often store large data structures for use in many areas, but a specific algorithm may only need one or two members. Say I have a list of people and I want to sort them in order of their <i>year_of_birth</i>. I'm sure anyone with much experience with C++ has seen code like this:</div>
<div>
</div>
<pre style="background: #f6f8ff; color: #000020;"><html><body style="background: #f6f8ff; color: #000020;"><pre><span style="color: #0066ee;">std</span><span style="color: #406080;">::</span><span style="color: #003060;">sort</span><span style="color: #308080;">(</span><span style="color: #0066ee;">std</span><span style="color: #406080;">::</span>begin<span style="color: #308080;">(</span>people<span style="color: #308080;">)</span><span style="color: #308080;">,</span> <span style="color: #0066ee;">std</span><span style="color: #406080;">::</span>end<span style="color: #308080;">(</span>people<span style="color: #308080;">)</span><span style="color: #308080;">,</span>
<span style="color: #308080;">[</span><span style="color: #308080;">]</span><span style="color: #308080;">(</span><span style="color: #200080; font-weight: bold;">const</span> Person<span style="color: #308080;">&</span> a<span style="color: #308080;">,</span> <span style="color: #200080; font-weight: bold;">const</span> Person<span style="color: #308080;">&</span> b<span style="color: #308080;">)</span> <span style="background: #dd9999; color: white; font-style: italic; font-weight: bold;">{</span>
<span style="color: #200080; font-weight: bold;">return</span> a<span style="color: #308080;">.</span>year_of_birth <span style="color: #308080;"><</span> b<span style="color: #308080;">.</span>year_of_birth<span style="color: #406080;">;</span>
<span style="background: #dd9999; color: white; font-style: italic; font-weight: bold;">}</span><span style="color: #308080;">)</span><span style="color: #406080;">;</span>
</pre>
</body></html></pre>
<div>
This pattern occurs so frequently than <b>range-v3</b> accepts "projection" functions for algorithms such as this.</div>
<pre style="background: #f6f8ff; color: #000020;"><html><body style="background: #f6f8ff; color: #000020;"><pre>range<span style="color: #406080;">::</span>v3<span style="color: #406080;">::</span><span style="color: #003060;">sort</span><span style="color: #308080;">(</span>people<span style="color: #308080;">,</span> <span style="color: #0066ee;">std</span><span style="color: #406080;">::</span><span style="color: #003060;">less</span><span style="color: #406080;"><</span><span style="color: #406080;">></span><span style="background: #dd9999; color: white; font-style: italic; font-weight: bold;">{</span><span style="background: #dd9999; color: white; font-style: italic; font-weight: bold;">}</span><span style="color: #308080;">,</span> <span style="color: #308080;">&</span>Person<span style="color: #406080;">::</span>year_of_birth<span style="color: #308080;">)</span><span style="color: #406080;">;</span>
</pre>
</body></html></pre>
<div>
(see: <a href="https://github.com/ericniebler/range-v3/blob/master/include/range/v3/algorithm/sort.hpp">https://github.com/ericniebler/range-v3/blob/master/include/range/v3/algorithm/sort.hpp</a>)<br />
<br /></div>
<div>
</div>
<div>
The projection converts its argument to make it suitable for the comparison function, <i>std::less</i> in this example. Internally, it ends up calculating the same thing as the lambda, extracting <i>year_of_birth</i> from both arguments and calling <i>less</i> on them.<br />
<br /></div>
<div>
</div>
<div>
One might ask, "but <i>&Person::year_of_birth</i> is not a function, so how can we pass it as one?" Internally, range-v3 and FU use a more generic concept of "invoking", which does not limit them to regular functions. See <a href="http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2013/n3727.html">n3727</a>, or <i><a href="http://en.cppreference.com/w/cpp/utility/functional/invoke">std::invoke</a> </i>(C++17) for more.<br />
<br /></div>
<div>
</div>
<div>
This issue is not limited to <i><algorithm></i> or <i>std::sort</i>, it is a generic problem, so I wrote <i>fu::proj</i> for use with the current STL and elsewhere.<br />
<br /></div>
<div>
<pre style="background: #f6f8ff; color: #000020;"><html><body style="background: #f6f8ff; color: #000020;"><pre><span style="color: #0066ee;">std</span><span style="color: #406080;">::</span><span style="color: #003060;">sort</span><span style="color: #308080;">(</span><span style="color: #0066ee;">std</span><span style="color: #406080;">::</span>begin<span style="color: #308080;">(</span>people<span style="color: #308080;">)</span><span style="color: #308080;">,</span> <span style="color: #0066ee;">std</span><span style="color: #406080;">::</span>end<span style="color: #308080;">(</span>people<span style="color: #308080;">)</span><span style="color: #308080;">,</span>
fu<span style="color: #406080;">::</span>proj<span style="color: #308080;">(</span><span style="color: #0066ee;">std</span><span style="color: #406080;">::</span><span style="color: #003060;">less</span><span style="color: #406080;"><</span><span style="color: #406080;">></span><span style="background: #dd9999; color: white; font-style: italic; font-weight: bold;">{</span><span style="background: #dd9999; color: white; font-style: italic; font-weight: bold;">}</span><span style="color: #308080;">,</span> <span style="color: #308080;">&</span>Person<span style="color: #406080;">::</span>year_of_birth<span style="color: #308080;">)</span><span style="color: #308080;">)</span><span style="color: #406080;">;</span>
</pre>
</body></html></pre>
</div>
<div>
But since requiring a projection over <i>less</i> is so very common, <i>fu::proj_less </i>can be more convenient.</div>
<div>
<pre style="background: #f6f8ff; color: #000020;"><html><body style="background: #f6f8ff; color: #000020;"><pre><span style="color: #0066ee;">std</span><span style="color: #406080;">::</span><span style="color: #003060;">sort</span><span style="color: #308080;">(</span><span style="color: #0066ee;">std</span><span style="color: #406080;">::</span>begin<span style="color: #308080;">(</span>people<span style="color: #308080;">)</span><span style="color: #308080;">,</span> <span style="color: #0066ee;">std</span><span style="color: #406080;">::</span>end<span style="color: #308080;">(</span>people<span style="color: #308080;">)</span><span style="color: #308080;">,</span>
fu<span style="color: #406080;">::</span>proj_less<span style="color: #308080;">(</span><span style="color: #308080;">&</span>Person<span style="color: #406080;">::</span>year_of_birth<span style="color: #308080;">)</span><span style="color: #308080;">)</span><span style="color: #406080;">;</span>
</pre>
</body></html></pre>
</div>
<div>
Fit also provides a similar utility, <i>by</i>, although the syntax is a bit different.<br />
<br /></div>
<div>
<pre style="background: #f6f8ff; color: #000020;"><html><body style="background: #f6f8ff; color: #000020;"><pre><span style="color: #0066ee;">std</span><span style="color: #406080;">::</span><span style="color: #003060;">sort</span><span style="color: #308080;">(</span><span style="color: #0066ee;">std</span><span style="color: #406080;">::</span>begin<span style="color: #308080;">(</span>people<span style="color: #308080;">)</span><span style="color: #308080;">,</span> <span style="color: #0066ee;">std</span><span style="color: #406080;">::</span>end<span style="color: #308080;">(</span>people<span style="color: #308080;">)</span><span style="color: #308080;">,</span>
fit<span style="color: #406080;">::</span>by<span style="color: #308080;">(</span><span style="color: #0066ee;">std</span><span style="color: #406080;">::</span>mem_fn<span style="color: #308080;">(</span><span style="color: #308080;">&</span>Person<span style="color: #406080;">::</span>year_of_birth<span style="color: #308080;">)</span><span style="color: #308080;">,</span> _ <span style="color: #308080;"><</span> _<span style="color: #308080;">)</span><span style="color: #308080;">)</span><span style="color: #406080;">;</span>
</pre>
</body></html></pre>
</div>
<div>
I argue that each of the four versions above would be more clear and less error prone than the first, using a lambda. Rather than specifying exactly how to apply the arguments, we need only specify <b>what</b> to apply.<br />
<br /></div>
<div>
</div>
<div>
So why not use <i>std::bind</i> for examples like this?</div>
<div>
<pre style="background: #f6f8ff; color: #000020;"><html><body style="background: #f6f8ff; color: #000020;"><pre><span style="color: #200080; font-weight: bold;">using</span> <span style="color: #200080; font-weight: bold;">namespace</span> <span style="color: #0066ee;">std</span><span style="color: #406080;">::</span>placeholders<span style="color: #406080;">;</span>
<span style="color: #0066ee;">std</span><span style="color: #406080;">::</span><span style="color: #003060;">sort</span><span style="color: #308080;">(</span><span style="color: #0066ee;">std</span><span style="color: #406080;">::</span>begin<span style="color: #308080;">(</span>people<span style="color: #308080;">)</span><span style="color: #308080;">,</span> <span style="color: #0066ee;">std</span><span style="color: #406080;">::</span>end<span style="color: #308080;">(</span>people<span style="color: #308080;">)</span><span style="color: #308080;">,</span>
<span style="color: #0066ee;">std</span><span style="color: #406080;">::</span><span style="color: #400000;">bind</span><span style="color: #308080;">(</span><span style="color: #0066ee;">std</span><span style="color: #406080;">::</span><span style="color: #003060;">less</span><span style="color: #406080;"><</span><span style="color: #406080;">></span><span style="background: #dd9999; color: white; font-style: italic; font-weight: bold;">{</span><span style="background: #dd9999; color: white; font-style: italic; font-weight: bold;">}</span><span style="color: #308080;">,</span>
<span style="color: #0066ee;">std</span><span style="color: #406080;">::</span><span style="color: #400000;">bind</span><span style="color: #308080;">(</span><span style="color: #308080;">&</span>Person<span style="color: #406080;">::</span>year_of_birth<span style="color: #308080;">,</span> _1<span style="color: #308080;">)</span><span style="color: #308080;">,</span>
<span style="color: #0066ee;">std</span><span style="color: #406080;">::</span><span style="color: #400000;">bind</span><span style="color: #308080;">(</span><span style="color: #308080;">&</span>Person<span style="color: #406080;">::</span>year_of_birth<span style="color: #308080;">,</span> _2<span style="color: #308080;">)</span><span style="color: #308080;">)</span><span style="color: #308080;">)</span><span style="color: #406080;">;</span>
</pre>
</body></html></pre>
</div>
<div>
With examples like that, it's no wonder people prefer lambdas! It actually becomes less readable with the overly-verbose and overly-general <i>std::bind</i>. </div>
<div>
<i><br />
</i></div>
<div>
Projection works well enough for comparisons, but then you have a function like <i>std::accumulate</i> and you only want to project the right-hand argument. Consider calculating the average age of the list of people.</div>
<div>
<pre style="background: #f6f8ff; color: #000020;"><html><body style="background: #f6f8ff; color: #000020;"><pre><span style="color: #200080; font-weight: bold;">auto</span> acc <span style="color: #308080;">=</span> <span style="color: #308080;">[</span><span style="color: #308080;">]</span><span style="color: #308080;">(</span><span style="color: #200080; font-weight: bold;">int</span> accum<span style="color: #308080;">,</span> <span style="color: #200080; font-weight: bold;">const</span> Person<span style="color: #308080;">&</span> p<span style="color: #308080;">)</span> <span style="color: #406080;">{</span> <span style="color: #200080; font-weight: bold;">return</span> accum <span style="color: #308080;">+</span> p<span style="color: #308080;">.</span>age<span style="color: #308080;">(</span><span style="color: #308080;">)</span><span style="color: #406080;">;</span> <span style="color: #406080;">}</span>
<span style="color: #200080; font-weight: bold;">int</span> sum <span style="color: #308080;">=</span>
<span style="color: #0066ee;">std</span><span style="color: #406080;">::</span>accumulate<span style="color: #308080;">(</span><span style="color: #0066ee;">std</span><span style="color: #406080;">::</span>begin<span style="color: #308080;">(</span>people<span style="color: #308080;">)</span><span style="color: #308080;">,</span> <span style="color: #0066ee;">std</span><span style="color: #406080;">::</span>end<span style="color: #308080;">(</span>people<span style="color: #308080;">)</span><span style="color: #308080;">,</span> <span style="color: #008c00;">0</span><span style="color: #308080;">,</span> acc<span style="color: #308080;">)</span><span style="color: #406080;">;</span>
<span style="color: #200080; font-weight: bold;">float</span> ave <span style="color: #308080;">=</span> <span style="color: #200080; font-weight: bold;">float</span><span style="color: #308080;">(</span>sum<span style="color: #308080;">)</span> <span style="color: #308080;">/</span> people<span style="color: #308080;">.</span>size<span style="color: #308080;">(</span><span style="color: #308080;">)</span><span style="color: #406080;">;</span>
</pre>
</body></html></pre>
<div>
Again, range-v3 allows projection here:</div>
<pre style="background: #f6f8ff; color: #000020;"><html><body style="background: #f6f8ff; color: #000020;"><pre><span style="color: #200080; font-weight: bold;">int</span> sum <span style="color: #308080;">=</span> range<span style="color: #406080;">::</span>v3<span style="color: #406080;">::</span>accumulate<span style="color: #308080;">(</span>people<span style="color: #308080;">,</span> <span style="color: #0066ee;">std</span><span style="color: #406080;">::</span><span style="color: #003060;">plus</span><span style="color: #406080;"><</span><span style="color: #406080;">></span><span style="background: #dd9999; color: white; font-style: italic; font-weight: bold;">{</span><span style="background: #dd9999; color: white; font-style: italic; font-weight: bold;">}</span><span style="color: #308080;">,</span> <span style="color: #308080;">&</span>Person<span style="color: #406080;">::</span>age<span style="color: #308080;">)</span><span style="color: #406080;">;</span>
<span style="color: #200080; font-weight: bold;">float</span> ave <span style="color: #308080;">=</span> <span style="color: #200080; font-weight: bold;">float</span><span style="color: #308080;">(</span>sum<span style="color: #308080;">)</span> <span style="color: #308080;">/</span> people<span style="color: #308080;">.</span>size<span style="color: #308080;">(</span><span style="color: #308080;">)</span><span style="color: #406080;">;</span>
</pre>
</body></html></pre>
</div>
<div>
One can use <i>fu::rproj</i> (right-projection) with the existing STL.</div>
<div>
<pre style="background: #f6f8ff; color: #000020;"><html><body style="background: #f6f8ff; color: #000020;"><pre><span style="color: #200080; font-weight: bold;">int</span> sum <span style="color: #308080;">=</span>
<span style="color: #0066ee;">std</span><span style="color: #406080;">::</span>accumulate<span style="color: #308080;">(</span><span style="color: #0066ee;">std</span><span style="color: #406080;">::</span>begin<span style="color: #308080;">(</span>people<span style="color: #308080;">)</span><span style="color: #308080;">,</span> <span style="color: #0066ee;">std</span><span style="color: #406080;">::</span>end<span style="color: #308080;">(</span>people<span style="color: #308080;">)</span><span style="color: #308080;">,</span>
<span style="color: #008c00;">0</span><span style="color: #308080;">,</span> fu<span style="color: #406080;">::</span>rproj<span style="color: #308080;">(</span><span style="color: #0066ee;">std</span><span style="color: #406080;">::</span><span style="color: #003060;">plus</span><span style="color: #406080;"><</span><span style="color: #406080;">></span><span style="background: #dd9999; color: white; font-style: italic; font-weight: bold;">{</span><span style="background: #dd9999; color: white; font-style: italic; font-weight: bold;">}</span><span style="color: #308080;">,</span> <span style="color: #308080;">&</span>Person<span style="color: #406080;">::</span>age<span style="color: #308080;">)</span><span style="color: #308080;">)</span><span style="color: #406080;">;</span>
<span style="color: #200080; font-weight: bold;">float</span> ave <span style="color: #308080;">=</span> <span style="color: #200080; font-weight: bold;">float</span><span style="color: #308080;">(</span>sum<span style="color: #308080;">)</span> <span style="color: #308080;">/</span> people<span style="color: #308080;">.</span>size<span style="color: #308080;">(</span><span style="color: #308080;">)</span><span style="color: #406080;">;</span>
</pre>
</body></html></pre>
</div>
<div>
While <i>proj</i> and <i>rproj</i> will be sufficient for most <i><algorithm></i> functions, we do have one special case: <i>std::transform</i> or <i>std::equal</i>. Here, the right-hand and left-hand arguments of our function may be different types, but we still might want to use something like <i>std::plus</i> or <i>std::equal_to</i> to combine them.<br />
<br /></div>
<div>
</div>
<div>
For lack of a better example, consider that my struct, <i>Person</i>, contains a member, <i>year_of_death</i>. I want a sanity check that for any person, <i>year_of_birth < year_of_death</i>.</div>
<div>
<pre style="background: #f6f8ff; color: #000020;"><html><body style="background: #f6f8ff; color: #000020;"><pre><span style="color: #200080; font-weight: bold;">bool</span> ok <span style="color: #308080;">=</span> <span style="color: #0066ee;">std</span><span style="color: #406080;">::</span><span style="color: #003060;">equal</span><span style="color: #308080;">(</span><span style="color: #0066ee;">std</span><span style="color: #406080;">::</span>begin<span style="color: #308080;">(</span>people<span style="color: #308080;">)</span><span style="color: #308080;">,</span> <span style="color: #0066ee;">std</span><span style="color: #406080;">::</span>end<span style="color: #308080;">(</span>people<span style="color: #308080;">)</span><span style="color: #308080;">,</span>
<span style="color: #0066ee;">std</span><span style="color: #406080;">::</span>begin<span style="color: #308080;">(</span>people<span style="color: #308080;">)</span><span style="color: #308080;">,</span>
<span style="color: #308080;">[</span><span style="color: #308080;">]</span><span style="color: #308080;">(</span><span style="color: #200080; font-weight: bold;">const</span> Person<span style="color: #308080;">&</span> a<span style="color: #308080;">,</span> <span style="color: #200080; font-weight: bold;">const</span> Person<span style="color: #308080;">&</span> b<span style="color: #308080;">)</span> <span style="background: #dd9999; color: white; font-style: italic; font-weight: bold;">{</span>
<span style="color: #200080; font-weight: bold;">return</span> a<span style="color: #308080;">.</span>time_of_birth <span style="color: #308080;"><</span> b<span style="color: #308080;">.</span>time_of_death<span style="color: #406080;">;</span>
<span style="background: #dd9999; color: white; font-style: italic; font-weight: bold;">}</span><span style="color: #308080;">)</span><span style="color: #406080;">;</span>
</pre>
</body></html></pre>
</div>
<div>
With range-v3:</div>
<pre style="background: #f6f8ff; color: #000020;"><html><body style="background: #f6f8ff; color: #000020;"><pre><span style="color: #200080; font-weight: bold;">bool</span> ok <span style="color: #308080;">=</span> range<span style="color: #406080;">::</span>v3<span style="color: #406080;">::</span><span style="color: #003060;">equal</span><span style="color: #308080;">(</span>people<span style="color: #308080;">,</span> people<span style="color: #308080;">,</span> <span style="color: #0066ee;">std</span><span style="color: #406080;">::</span><span style="color: #003060;">less</span><span style="color: #406080;"><</span><span style="color: #406080;">></span><span style="background: #dd9999; color: white; font-style: italic; font-weight: bold;">{</span><span style="background: #dd9999; color: white; font-style: italic; font-weight: bold;">}</span><span style="color: #308080;">,</span>
<span style="color: #308080;">&</span>Person<span style="color: #406080;">::</span>year_of_birth<span style="color: #308080;">,</span> <span style="color: #308080;">&</span>Person<span style="color: #406080;">::</span>year_of_death<span style="color: #308080;">)</span>
</pre>
</body></html></pre>
<div>
For <i>fu</i>, I decided to call this function where the right- and left-hand arguments require different projections <i>join</i> instead of <i>binary_proj</i> or <i>bproj</i>.</div>
<div>
<pre style="background: #f6f8ff; color: #000020;"><html><body style="background: #f6f8ff; color: #000020;"><pre><span style="color: #200080; font-weight: bold;">auto</span> pred <span style="color: #308080;">=</span> fu<span style="color: #406080;">::</span>join<span style="color: #308080;">(</span><span style="color: #0066ee;">std</span><span style="color: #406080;">::</span><span style="color: #003060;">less</span><span style="color: #406080;"><</span><span style="color: #406080;">></span><span style="background: #dd9999; color: white; font-style: italic; font-weight: bold;">{</span><span style="background: #dd9999; color: white; font-style: italic; font-weight: bold;">}</span><span style="color: #308080;">,</span> <span style="color: #308080;">&</span>Person<span style="color: #406080;">::</span>year_of_birth<span style="color: #308080;">,</span>
<span style="color: #308080;">&</span>Person<span style="color: #406080;">::</span>year_of_death<span style="color: #308080;">)</span><span style="color: #406080;">;</span>
<span style="color: #200080; font-weight: bold;">bool</span> ok <span style="color: #308080;">=</span> <span style="color: #0066ee;">std</span><span style="color: #406080;">::</span><span style="color: #003060;">equal</span><span style="color: #308080;">(</span><span style="color: #0066ee;">std</span><span style="color: #406080;">::</span>begin<span style="color: #308080;">(</span>people<span style="color: #308080;">)</span><span style="color: #308080;">,</span> <span style="color: #0066ee;">std</span><span style="color: #406080;">::</span>end<span style="color: #308080;">(</span>people<span style="color: #308080;">)</span><span style="color: #308080;">,</span>
<span style="color: #0066ee;">std</span><span style="color: #406080;">::</span>begin<span style="color: #308080;">(</span>people<span style="color: #308080;">)</span><span style="color: #308080;">,</span> pred<span style="color: #308080;">)</span><span style="color: #406080;">;</span>
</pre>
</body></html></pre>
</div>
<div>
<br />
Although, a more obvious solution might be to use <i>std::all_of</i>, using <i>fu::split(f, left, right)</i>, which produces a function, <i>s</i>, such that <i>s(x) == f(left(x), right(x))</i>.<br />
<div>
<pre style="background: #f6f8ff; color: #000020;"><html><body style="background: #f6f8ff; color: #000020;"><pre><span style="color: #200080; font-weight: bold;">bool</span> ok <span style="color: #308080;">=</span> <span style="color: #0066ee;">std</span><span style="color: #406080;">::</span>all_of<span style="color: #308080;">(</span><span style="color: #0066ee;">std</span><span style="color: #406080;">::</span>begin<span style="color: #308080;">(</span>people<span style="color: #308080;">)</span><span style="color: #308080;">,</span> <span style="color: #0066ee;">std</span><span style="color: #406080;">::</span>end<span style="color: #308080;">(</span>people<span style="color: #308080;">)</span><span style="color: #308080;">,</span>
fu<span style="color: #406080;">::</span>split<span style="color: #308080;">(</span><span style="color: #0066ee;">std</span><span style="color: #406080;">::</span><span style="color: #003060;">less</span><span style="color: #406080;"><</span><span style="color: #406080;">></span><span style="background: #dd9999; color: white; font-style: italic; font-weight: bold;">{</span><span style="background: #dd9999; color: white; font-style: italic; font-weight: bold;">}</span><span style="color: #308080;">,</span> <span style="color: #308080;">&</span>Person<span style="color: #406080;">::</span>year_of_birth<span style="color: #308080;">,</span>
<span style="color: #308080;">&</span>Person<span style="color: #406080;">::</span>year_of_death<span style="color: #308080;">)</span><span style="color: #308080;">)</span><span style="color: #406080;">;</span>
</pre>
</body></html></pre>
</div>
So, in this instance, the lambda might actually win out on terseness, but projection does one nice thing for us: it asserts mathematical soundness. It might be very tempting to write an equality operator between a <i>Person</i> and an <i>std::string</i> to compare the person's name, especially for use with <i><algorithm></i>s, but it makes no real sense. I also store the person's birthplace as a string, so should <i>person == "place"</i> be valid, too? No, it makes much more sense to define a predicate based on a function with well-known mathematical properties, like <i>std::equal_to</i>, and use projection to obtain the items for which equality has already been defined.<br />
<br /></div>
<div>
</div>
<div>
Haskell programmers like to say "if it compiles, it works!" Some of that has to do with the idea of "<a href="http://en.wikipedia.org/wiki/Referential_transparency_%28computer_science%29">referential transparency</a>", but another large part of it relates to using mathematically sound operations to assert correctness. If each component, or function, is obviously correct, and the algorithm is made of obviously correct operations, then most likely the algorithm is obviously correct.<br />
<br /></div>
<div>
</div>
<div>
Let's use this concept to prove the correctness of our first example, sorting by the projection of <i>std::less</i> on the projection of <i>&Person::year_of_birth</i>. It may seem unnecessary, but the rigors of mathematics require that we can formally prove these things, not just intuit or know them. </div>
<div>
</div>
<div>
<ul>
<li>We can assume the correctness of <i>std::sort</i> by definition, if the comparison function implements a <a href="http://en.wikipedia.org/wiki/Weak_ordering#Strict_weak_orderings">strict weak order</a>. Strict weak order for any comparison, <i>comp</i>, means that it must be irreflexive (<i>!comp(x,x)</i>), weakly ordered (!<i>comp(a, b) && !comp(b, a) </i>implies<i> a == b</i>), and transitive (<i>comp(a,b) && comp(b,c) => comp(a,c)</i>). </li>
<li>We know by definition that <i>std::less</i> meets this requirement thus <i>proj_less(f)</i> meets this requirement if <i>f(a) < f(b)</i> preserves this property (also by definition). </li>
<li>Finally, we know that <i>f(a) == a.year_of_birth</i>, stored as an integer, and that a set of integers sorted by <i><</i> are strict-weak ordered. </li>
</ul>
</div>
<div>
Thus, we can call that example "obviously correct." In fact, we can further state that any ordering over <i>proj_less(f)</i> will be correct with very few, if any, exceptions.<br />
<br /></div>
<div>
</div>
<div>
Mathematical soundness and obvious correctness make it easy to reason about code, but projection alone does not give us the required tools to accomplish it. We need other fundamental tools, as I will show in the next section.<br />
<br /></div>
<div>
</div>
<h4>
The building blocks.</h4>
<div>
<br />
Say I want to construct a list of people born in Germany. Like always, we can start with the generic lambda.</div>
<div>
<pre style="background: #f6f8ff; color: #000020;"><html><body style="background: #f6f8ff; color: #000020;"><pre><span style="color: #0066ee;">std</span><span style="color: #406080;">::</span><span style="color: #003060;">vector</span><span style="color: #406080;"><</span>Person<span style="color: #406080;">></span> germans<span style="color: #406080;">;</span>
<span style="color: #0066ee;">std</span><span style="color: #406080;">::</span>copy_if<span style="color: #308080;">(</span><span style="color: #0066ee;">std</span><span style="color: #406080;">::</span>begin<span style="color: #308080;">(</span>people<span style="color: #308080;">)</span><span style="color: #308080;">,</span> <span style="color: #0066ee;">std</span><span style="color: #406080;">::</span>end<span style="color: #308080;">(</span>people<span style="color: #308080;">)</span><span style="color: #308080;">,</span>
<span style="color: #0066ee;">std</span><span style="color: #406080;">::</span>back_inserter<span style="color: #308080;">(</span>germans<span style="color: #308080;">)</span><span style="color: #308080;">,</span>
<span style="color: #308080;">[</span><span style="color: #308080;">]</span><span style="color: #308080;">(</span><span style="color: #200080; font-weight: bold;">const</span> Person<span style="color: #308080;">&</span> p<span style="color: #308080;">)</span> <span style="background: #dd9999; color: white; font-style: italic; font-weight: bold;">{</span> <span style="color: #200080; font-weight: bold;">return</span> p<span style="color: #308080;">.</span>birth_place <span style="color: #308080;">=</span><span style="color: #308080;">=</span> <span style="color: maroon;">"</span><span style="color: #1060b6;">Germany</span><span style="color: maroon;">"</span><span style="color: #406080;">;</span> <span style="background: #dd9999; color: white; font-style: italic; font-weight: bold;">}</span><span style="color: #308080;">)</span><span style="color: #406080;">;</span>
</pre>
</body></html></pre>
<div>
Range-v3 does allow a projection for <i>copy_if</i>, but unfortunately, that won't much help us.</div>
<div>
<pre style="background: #f6f8ff; color: #000020;"><html><body style="background: #f6f8ff; color: #000020;"><pre>range<span style="color: #406080;">::</span>v3<span style="color: #406080;">::</span>copy_if<span style="color: #308080;">(</span>people<span style="color: #308080;">,</span> <span style="color: #0066ee;">std</span><span style="color: #406080;">::</span>back_inserter<span style="color: #308080;">(</span>germans<span style="color: #308080;">)</span><span style="color: #308080;">,</span>
<span style="color: #308080;">[</span><span style="color: #308080;">]</span><span style="color: #308080;">(</span><span style="color: #200080; font-weight: bold;">const</span> <span style="color: #0066ee;">std</span><span style="color: #406080;">::</span><span style="color: #003060;">string</span><span style="color: #308080;">&</span> place<span style="color: #308080;">)</span> <span style="background: #dd9999; color: white; font-style: italic; font-weight: bold;">{</span> <span style="color: #200080; font-weight: bold;">return</span> place <span style="color: #308080;">=</span><span style="color: #308080;">=</span> <span style="color: maroon;">"</span><span style="color: #1060b6;">Germany</span><span style="color: maroon;">"</span><span style="color: #406080;">;</span> <span style="background: #dd9999; color: white; font-style: italic; font-weight: bold;">}</span><span style="color: #308080;">,</span>
<span style="color: #308080;">&</span>Person<span style="color: #406080;">::</span>birth_place<span style="color: #308080;">)</span><span style="color: #406080;">;</span>
</pre>
</body></html></pre>
</div>
<div>
It would have been less typing without the projection, although we at least know that the lambda and projection both are obviously correct and thus is the expression. To really get the desired level of terseness, we need a <a href="https://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&cd=1&cad=rja&uact=8&ved=0CB4QFjAA&url=http%3A%2F%2Fen.wikipedia.org%2Fwiki%2FPartial_application&ei=Fk7iVL3gHomrggTYs4L4DQ&usg=AFQjCNFEP2tp6pCOWchvFx7jVgS34MIfgg&sig2=oolPE6_5yk76Kk71tT-pQQ&bvm=bv.85970519,d.eXY">partial application</a> of equality.<br />
<br /></div>
<div>
</div>
<div>
FU supplies a function, <i>part</i>, so that we could write "<i>auto pred = fu::part(std::equal_to<>{}, "Germany")</i>", and then one could write "<i>pred("Hungary")</i>", which would return <i>false</i>, or "<i>pred("Germany")</i>", <i>true</i>, but it's just not terse. Since one often needs to pass binary operators or partially-applied operators to functions like <i>copy_if</i>, FU also supplies a number of such functions. For this example, we need <i>fu::eq</i>, which acts similarly to <i>std::equal_to</i> except that <i>fu::eq(x)</i> returns the partial application. (But <i>fu::eq(x,y)</i> does compute <i>x == y</i>, like one would expect.) So with that, we can return to our example:</div>
<pre style="background: #f6f8ff; color: #000020;"><html><body style="background: #f6f8ff; color: #000020;"><pre>range<span style="color: #406080;">::</span>v3<span style="color: #406080;">::</span>copy_if<span style="color: #308080;">(</span>people<span style="color: #308080;">,</span> <span style="color: #0066ee;">std</span><span style="color: #406080;">::</span>back_inserter<span style="color: #308080;">(</span>germans<span style="color: #308080;">)</span><span style="color: #308080;">,</span>
fu<span style="color: #406080;">::</span>eq<span style="color: #308080;">(</span><span style="color: maroon;">"</span><span style="color: #1060b6;">Germany</span><span style="color: maroon;">"</span><span style="color: #308080;">)</span><span style="color: #308080;">,</span> <span style="color: #308080;">&</span>Person<span style="color: #406080;">::</span>birth_place<span style="color: #308080;">)</span><span style="color: #406080;">;</span>
</pre>
</body></html></pre>
<div>
<a href="http://pfultz2.github.io/Fit/doc/html/placeholders/">Fit's placeholders utilities</a> also recognize the importance of partial applications like this.</div>
<div>
<pre style="background: #f6f8ff; color: #000020;"><html><body style="background: #f6f8ff; color: #000020;"><pre>range<span style="color: #406080;">::</span>v3<span style="color: #406080;">::</span>copy_if<span style="color: #308080;">(</span>people<span style="color: #308080;">,</span> <span style="color: #0066ee;">std</span><span style="color: #406080;">::</span>back_inserter<span style="color: #308080;">(</span>germans<span style="color: #308080;">)</span><span style="color: #308080;">,</span>
_ <span style="color: #308080;">=</span><span style="color: #308080;">=</span> <span style="color: maroon;">"</span><span style="color: #1060b6;">Germany</span><span style="color: maroon;">"</span><span style="color: #308080;">,</span> <span style="color: #308080;">&</span>Person<span style="color: #406080;">::</span>birth_place<span style="color: #308080;">)</span><span style="color: #406080;">;</span>
</pre>
</body></html></pre>
</div>
<div>
And with the STL using FU,</div>
<div>
<pre style="background: #f6f8ff; color: #000020;"><html><body style="background: #f6f8ff; color: #000020;"><pre><span style="color: #0066ee;">std</span><span style="color: #406080;">::</span>copy_if<span style="color: #308080;">(</span><span style="color: #0066ee;">std</span><span style="color: #406080;">::</span>begin<span style="color: #308080;">(</span>people<span style="color: #308080;">)</span><span style="color: #308080;">,</span> <span style="color: #0066ee;">std</span><span style="color: #406080;">::</span>end<span style="color: #308080;">(</span>people<span style="color: #308080;">)</span><span style="color: #308080;">,</span>
<span style="color: #0066ee;">std</span><span style="color: #406080;">::</span>back_inserter<span style="color: #308080;">(</span>germans<span style="color: #308080;">)</span><span style="color: #308080;">,</span>
fu<span style="color: #406080;">::</span>proj<span style="color: #308080;">(</span>fu<span style="color: #406080;">::</span>eq<span style="color: #308080;">(</span><span style="color: maroon;">"</span><span style="color: #1060b6;">Germany</span><span style="color: maroon;">"</span><span style="color: #308080;">)</span><span style="color: #308080;">,</span> <span style="color: #308080;">&</span>Person<span style="color: #406080;">::</span>birth_place<span style="color: #308080;">)</span><span style="color: #308080;">)</span><span style="color: #406080;">;</span>
</pre>
</body></html></pre>
</div>
<div>
<i>(Note: fu::ucompose, short for "unary composition", might be more appropriate here, but proj(f,g)(x) is roughly equivalent to ucompose(f,g)(x).)</i></div>
<div>
<i><br />
</i></div>
<div>
Let's say I need all the names stored in <i>people</i> in a contiguous string, separated by commas. I could just call <i>std::accumulate</i>, starting with an empty <i>std::string</i> and append each one using <i>rproj(std::plus<>{}, &Person::name)</i>, but I want to minimize the number of required allocations so I have to sum up their sizes first. It'll look something like this, with range-v3:</div>
<div>
<pre style="background: #f6f8ff; color: #000020;"><html><body style="background: #f6f8ff; color: #000020;"><pre><span style="color: #200080; font-weight: bold;">auto</span> name_len <span style="color: #308080;">=</span> <span style="color: #308080;">[</span><span style="color: #308080;">]</span><span style="color: #308080;">(</span><span style="color: #200080; font-weight: bold;">const</span> Person<span style="color: #308080;">&</span> p<span style="color: #308080;">)</span> <span style="color: #406080;">{</span> <span style="color: #200080; font-weight: bold;">return</span> p<span style="color: #308080;">.</span>name<span style="color: #308080;">.</span>size<span style="color: #308080;">(</span><span style="color: #308080;">)</span><span style="color: #406080;">;</span> <span style="color: #406080;">}</span><span style="color: #406080;">;</span>
<span style="color: #003060;">size_t</span> size <span style="color: #308080;">=</span> range<span style="color: #406080;">::</span>v3<span style="color: #406080;">::</span>accumulate<span style="color: #308080;">(</span>people<span style="color: #308080;">,</span> <span style="color: #008c00;">0</span><span style="color: #308080;">,</span> <span style="color: #0066ee;">std</span><span style="color: #406080;">::</span><span style="color: #003060;">plus</span><span style="color: #406080;"><</span><span style="color: #406080;">></span><span style="background: #dd9999; color: white; font-style: italic; font-weight: bold;">{</span><span style="background: #dd9999; color: white; font-style: italic; font-weight: bold;">}</span><span style="color: #308080;">,</span> name_len<span style="color: #308080;">)</span><span style="color: #406080;">;</span>
<span style="color: #0066ee;">std</span><span style="color: #406080;">::</span><span style="color: #003060;">string</span> names<span style="color: #406080;">;</span>
names<span style="color: #308080;">.</span>reserve<span style="color: #308080;">(</span>size <span style="color: #308080;">+</span> people<span style="color: #308080;">.</span>size<span style="color: #308080;">(</span><span style="color: #308080;">)</span> <span style="color: #308080;">*</span> <span style="color: #008c00;">2</span><span style="color: #308080;">)</span><span style="color: #406080;">;</span>
<span style="color: #200080; font-weight: bold;">auto</span> append <span style="color: #308080;">=</span> <span style="color: #308080;">[</span><span style="color: #308080;">&</span><span style="color: #308080;">]</span><span style="color: #308080;">(</span><span style="color: #200080; font-weight: bold;">const</span> <span style="color: #0066ee;">std</span><span style="color: #406080;">::</span><span style="color: #003060;">string</span><span style="color: #308080;">&</span> str<span style="color: #308080;">)</span> <span style="color: #406080;">{</span> names<span style="color: #308080;">.</span>append<span style="color: #308080;">(</span>str<span style="color: #308080;">)</span><span style="color: #406080;">;</span>
names<span style="color: #308080;">.</span>append<span style="color: #308080;">(</span><span style="color: maroon;">"</span><span style="color: #1060b6;">, </span><span style="color: maroon;">"</span><span style="color: #308080;">)</span><span style="color: #406080;">;</span> <span style="color: #406080;">}</span><span style="color: #406080;">;</span>
range<span style="color: #406080;">::</span>v3<span style="color: #406080;">::</span><span style="color: #003060;">for_each</span><span style="color: #308080;">(</span>people<span style="color: #308080;">,</span> append<span style="color: #308080;">,</span> <span style="color: #308080;">&</span>Person<span style="color: #406080;">::</span>name<span style="color: #308080;">)</span><span style="color: #406080;">;</span>
</pre>
</body></html></pre>
</div>
<div>
<i>(For simplicity, I'm ignoring that this algorithm will append an extra ", " at the end.)</i></div>
<div>
<i><br />
</i></div>
<div>
Again, we could not represent the mini-functions <i>name_len</i> and <i>append</i> as simple projections and had to resort to writing out the lambdas. However, can we represent them using <i>fu::proj</i>? Well, for <i>name_len</i>, yes.</div>
<pre style="background: #f6f8ff; color: #000020;"><html><body style="background: #f6f8ff; color: #000020;"><pre><span style="color: #200080; font-weight: bold;">auto</span> name_len <span style="color: #308080;">=</span> fu<span style="color: #406080;">::</span>proj<span style="color: #308080;">(</span><span style="color: #308080;">&</span><span style="color: #0066ee;">std</span><span style="color: #406080;">::</span><span style="color: #003060;">string</span><span style="color: #406080;">::</span>size<span style="color: #308080;">,</span> <span style="color: #308080;">&</span>Person<span style="color: #406080;">::</span>name<span style="color: #308080;">)</span><span style="color: #406080;">;</span>
</pre>
</body></html></pre>
<div>
Invoking <i>&Person::name</i> returns an <i>std::string</i>, and <i>&std::string::size</i> gives us the desired result--it's obviously correct. <i>append</i>, however... no, and for two reasons. Firstly, <i>std::string::append</i> is an overloaded function so we can't just take its address; we'd need to wrap it in a lambda, anyway. Secondly, we also need to append <i>", "</i>, and while one could write a series of generic functional utilities and define <i>append</i> as accordingly, we'd most likely lose our "obvious correctness" with no real gains. For completeness, here is the version for the STL:</div>
<div>
<pre style="background: #f6f8ff; color: #000020;"><html><body style="background: #f6f8ff; color: #000020;"><pre><span style="color: #200080; font-weight: bold;">auto</span> name_len <span style="color: #308080;">=</span> fu<span style="color: #406080;">::</span>proj<span style="color: #308080;">(</span><span style="color: #308080;">&</span><span style="color: #0066ee;">std</span><span style="color: #406080;">::</span><span style="color: #003060;">string</span><span style="color: #406080;">::</span>size<span style="color: #308080;">,</span> <span style="color: #308080;">&</span>Person<span style="color: #406080;">::</span>name<span style="color: #308080;">)</span><span style="color: #406080;">;</span>
<span style="color: #003060;">size_t</span> size <span style="color: #308080;">=</span> <span style="color: #0066ee;">std</span><span style="color: #406080;">::</span>accumulate<span style="color: #308080;">(</span><span style="color: #0066ee;">std</span><span style="color: #406080;">::</span>begin<span style="color: #308080;">(</span>people<span style="color: #308080;">)</span><span style="color: #308080;">,</span> <span style="color: #0066ee;">std</span><span style="color: #406080;">::</span>end<span style="color: #308080;">(</span>people<span style="color: #308080;">)</span><span style="color: #308080;">,</span>
<span style="color: #008c00;">0</span><span style="color: #308080;">,</span> fu<span style="color: #406080;">::</span>rproj<span style="color: #308080;">(</span><span style="color: #0066ee;">std</span><span style="color: #406080;">::</span><span style="color: #003060;">plus</span><span style="color: #406080;"><</span><span style="color: #406080;">></span><span style="background: #dd9999; color: white; font-style: italic; font-weight: bold;">{</span><span style="background: #dd9999; color: white; font-style: italic; font-weight: bold;">}</span><span style="color: #308080;">,</span> name_len<span style="color: #308080;">)</span><span style="color: #308080;">)</span><span style="color: #406080;">;</span>
<span style="color: #0066ee;">std</span><span style="color: #406080;">::</span><span style="color: #003060;">string</span> names<span style="color: #406080;">;</span>
names<span style="color: #308080;">.</span>reserve<span style="color: #308080;">(</span>size <span style="color: #308080;">+</span> people<span style="color: #308080;">.</span>size<span style="color: #308080;">(</span><span style="color: #308080;">)</span> <span style="color: #308080;">*</span> <span style="color: #008c00;">2</span><span style="color: #308080;">)</span><span style="color: #406080;">;</span>
<span style="color: #200080; font-weight: bold;">auto</span> append <span style="color: #308080;">=</span> <span style="color: #308080;">[</span><span style="color: #308080;">&</span><span style="color: #308080;">]</span><span style="color: #308080;">(</span><span style="color: #200080; font-weight: bold;">const</span> <span style="color: #0066ee;">std</span><span style="color: #406080;">::</span><span style="color: #003060;">string</span><span style="color: #308080;">&</span> str<span style="color: #308080;">)</span> <span style="color: #406080;">{</span> names<span style="color: #308080;">.</span>append<span style="color: #308080;">(</span>str<span style="color: #308080;">)</span><span style="color: #406080;">;</span>
names<span style="color: #308080;">.</span>append<span style="color: #308080;">(</span><span style="color: maroon;">"</span><span style="color: #1060b6;">, </span><span style="color: maroon;">"</span><span style="color: #308080;">)</span><span style="color: #406080;">;</span> <span style="color: #406080;">}</span><span style="color: #406080;">;</span>
<span style="color: #0066ee;">std</span><span style="color: #406080;">::</span><span style="color: #003060;">for_each</span><span style="color: #308080;">(</span><span style="color: #0066ee;">std</span><span style="color: #406080;">::</span>begin<span style="color: #308080;">(</span>people<span style="color: #308080;">)</span><span style="color: #308080;">,</span> <span style="color: #0066ee;">std</span><span style="color: #406080;">::</span>end<span style="color: #308080;">(</span>people<span style="color: #308080;">)</span><span style="color: #308080;">,</span>
fu<span style="color: #406080;">::</span>proj<span style="color: #308080;">(</span>append<span style="color: #308080;">,</span> <span style="color: #308080;">&</span>Person<span style="color: #406080;">::</span>name<span style="color: #308080;">)</span><span style="color: #308080;">)</span><span style="color: #406080;">;</span>
</pre>
</body></html></pre>
</div>
<div>
</div>
<div>
</div>
<h4>
</h4>
<h4>
Conclusions</h4>
<br />
One might wonder why, in the last example, I prefer to make <i>append</i> operator on just an <i>std::string</i> and insist on projecting over <i>&Person::name</i>. I can trivially prove the correctness of <i>append(str)</i>; it appends <i>str</i> followed by <i>", "</i> to <i>names</i>, and it might even by useful in another part of the program. <i>proj(append, &Person::name)</i> is obviously correct for appending a person's name. Each unit, <i>append</i> and <i>&Person::name</i> operates on one level of abstraction; <i>proj</i> links them together. If the way a <i>Person</i> holds its name changes, I only need to change <i>&Person::name</i> and <i>append</i> remains valid. If I need to change <i>append</i>, the projection still remains valid.<br />
<br />
Keeping our functions small and on one layer of abstraction makes the program more robust, aside from maintaining soundness of the algorithm. FU's <i>proj</i>, <i>rproj</i>, <i>part</i>, and <i>split</i> can be used to construct most simple predicates, relational functions, and accumulators, and can also be combined in simple, granular expressions to make more complex ones.<br />
<br />
<h4>
Links</h4>
</div>
<div>
range-v3: <a href="https://github.com/ericniebler/range-v3">https://github.com/ericniebler/range-v3</a></div>
<div>
Fit: <a href="https://github.com/pfultz2/Fit/">https://github.com/pfultz2/Fit/</a></div>
<div>
FU: <a href="https://github.com/splinterofchaos/fu">https://github.com/splinterofchaos/fu</a></div>
<div>
Gist (requires FU): <a href="https://gist.github.com/splinterofchaos/e867c65e3ed5fe1b41da">https://gist.github.com/splinterofchaos/e867c65e3ed5fe1b41da</a><br />
<br />
<h4>
Notes:</h4>
</div>
<div>
<i>fu::proj_less</i>: <i>proj_less</i> has a very simple definition: <i>proj(std::less<>{})</i>, although it actually uses a helper so that it may be <i>constexpr</i>. Most <i>fu</i> functions are objects that can be implicitly partially applied. In fact, the definition of <i>proj</i> is actually closer to <i>[](auto f, auto pf, auto x, auto y) { return f(pf(x), pf(y)); }</i>, but perfect forwarding. Thus, <i>proj(std::less<>{}, &Person::name)</i> is also a partial application.</div>
<div>
<br /></div>
<div>
<i>fu::rproj(std::plus<>)</i>: I considered adding a function to FU called <i>accumulator = rproj(std::plus<>{})</i>, but it didn't seem very intuitive. Please comment below if you have an opinion on that.</div>
<div>
<br /></div>
<div>
<i>range::v3::copy_if</i>: It may be preferable to use <i>range::v3::view::remove_if(rng, std::not_fn(pred))</i> for the example given. One might want to use <i>range::v3::view::filter</i>, but it has been deprecated.</div>
<div>
<br /></div>
<div>
<i>fit::by</i>: We require <i>std::mem_fn</i> for Fit because it does not use a generalized invoke function.</div>
Splinter of Chaoshttp://www.blogger.com/profile/14715348728512729776noreply@blogger.com1tag:blogger.com,1999:blog-8638474063464489651.post-80164424036979378192015-01-08T05:33:00.000-08:002015-01-08T05:33:42.562-08:00Rvalue references and function overloading.No matter how many articles get written about rvalue references, I still get the impression that many people don't understand them fully. Some belief that rvalues work differently as template parameters than they do in non-template contexts, which requires special rules in overload resolution. I intend to show how rvalues would consistently with the rest of the language. In the coarse of writing this article, I even straightened out my own misconceptions of rvalues.<br />
<br />
I assume that the reader has some level of familiarity with rvalue references; understanding perfect forwarding helps. For either an introduction or refresher, I recommend "<a href="http://thbecker.net/articles/rvalue_references/section_01.html">C++ Rvalue References Explained</a>", a very thorough introduction and a well indexed reference.<br />
<br />
Today, I want to break out of my normal pedagogical style into a Q&A. I will ask that the reader thinks for a moment about what they expect a code example to do, and why.<br />
<br />
<h4>
Non-template rvalue overloading.</h4>
<div>
Typically, discussion of rvalues is limited to template functions and move constructors, but consider the following code example:</div>
<span style="font-family: monospace;"><span style="white-space: pre;">void f(int) { }
void f(int&) { }
void f(int&&) { }
int main() {
f(0); // (1)
int x;
f(x); // (2)
} </span></span><div>
What overload(s) of <i>f</i> get chosen at <i>(1)</i> and <i>(2)</i>? Please take a moment to compare <i>(1)</i> and <i>(2)</i> to each overload before continuing.<br />
<br />
<i>(1)</i> is ambiguous because it can refer to <i>f(int)</i> and <i>f(int&&)</i>, while <i>(2)</i> is ambiguous because it can be <i>f(int)</i> or <i>f(int&)</i>.<br />
<br />
The C++ standard describes a sort of "best fit" algorithm for overload resolution by first determining the viable overloads, then ranking them based on the conversions required of the arguments being passed. The highest rank, considered an "exact match," includes identity (or: "no conversion required") and considers reference binding a member of that category. Because of that, <i>f(int), </i>has the same rank as<i> f(int&) </i>and <i>f(int&&)</i> in overloading.<br />
<span style="font-family: monospace;"><span style="white-space: pre;">struct T {
T() { }
T(const T&) { }
T(T&&) { }
};
</span></span><span style="font-family: monospace; white-space: pre;">void f(T&) { }
</span><span style="font-family: monospace; white-space: pre;">void f(const T&) { }
</span><span style="font-family: monospace;"><span style="white-space: pre;">void f(T&&) { }
int main() {
T t;
T t2(t); // (1)
T t3(T{}); // (2)
f(t); // (3)
f(T{}); // (4)
}</span></span></div>
<div>
Does this example suffer from the same ambiguity? Why or why not?<br />
<br />
Being familiar with copy and move constructors, we know that <i>(1)</i> will invoke <i>T(const T&)</i> and that <i>(2)</i> will invoke <i>T(T&&)</i>, but shouldn't there be an ambiguity because we can bind temporaries to <i>const</i> references? Every overload has the "exact match" rank, but the standard also defines a sub-ranking for "implicit conversion sequences", which covers reference parameters (including rvalue references) and "qualification adjustment" such as converting an <i>int& </i>to <i>const int&</i>. Given two overloads on reference parameters, the rvalue reference overload is prefered to the lvalue one, thus <i>T(T&&)</i> is chosen for <i>(1)</i> and <i>f(T&&)</i> for <i>(4)</i>. <i>(3)</i> has a split between <i>f(T&)</i> and <i>f(const T&)</i>, but the former wins because the argument, <i>t</i>, is not <i>const</i> and would require adjustment.<br />
<br />
Note that <i>f</i> would be ambiguous if we had defined an <i>f(T)</i> overload because<i> </i>every other overload would have the same relative ranking. While <i>f(T&)</i> matches more precisely than <i>f(const T&)</i>, they both have the same rank compared to <i>f(T). </i><br />
<br />
Before considering the impact of rvalues on template functions, let's ponder what we already know about perfect forwarding.<br />
<span style="font-family: monospace;"><span style="white-space: pre;">template<typename T>
void f(T&&) { }
</span></span></div>
<div>
We know that this function can be called on any value and might refer to it as a "universal reference", although it has more recently been redubbed "forwarding reference". Let's try to reproduce this behaviour without templates.</div>
<div>
<span style="font-family: monospace;"><span style="white-space: pre;">void f(int&&) { }
void f(int& &&) { }
void f(const int& &&) { }
</span>
</span></div>
<div>
This code is not valid. I usually like clang's diagnostics the best, but here it complains, rather mundanely, "type name declared as a reference to a reference". Gcc gives us a very nice hint about this code: "cannot declare reference to ‘int&’, which is not a typedef or a template type argument".</div>
<span style="font-family: monospace;"><span style="white-space: pre;">using ref = int&;
using cref = const int&;
void f(int&&) { }
void f(ref&&) { }
void f(cref&&) { }
int main() {
int x = 0;
const int y = 0;
f(0); // calls f(int&&)
f(x); // calls f(ref&&)
f(y); // calls f(cref&&)
}</span></span><div>
Neither clang nor gcc has any difficulty with this version. The above set of overloads looks fairly similar to the familiar perfect<i>-</i>forwarding pattern.</div>
<span style="font-family: monospace;"><span style="white-space: pre;">using rval = int&&;
using rrval = rval&&; // (1)
using rrrval = rrval&&; // (2)
void f(rval) { }
void f(rrval) { } // (3)
void f(rrrval) { } // (4)
</span></span><div>
For <i>(1)</i> and <i>(2)</i>, are they valid? For <i>(3)</i> and <i>(4)</i>, what values would these overloads accept?</div>
<div>
<br /></div>
<div>
You may be surprised, but <i>(1)</i> and <i>(2)</i> are completely valid, however <i>(3)</i> and <i>(4)</i> are not. Again, the diagnostics of gcc and clang give us some good insight. They complain that <i>f(rrval)</i> redefines <i>f(rval)</i>, and ditto for <i>f(rrrval)</i>. That is to say: <i>rval</i>, <i>rrval</i>, and <i>rrrval</i> all refer to the same type! The same works for normal references--<i>(int&) &</i> is the same as <i>int&</i>.</div>
<div>
<br /></div>
<div>
While <i>int</i> and <i>int&&</i> are distinct types, the standard states that there "shall be no references to references," which includes rvalue references to references. This is one context where non-rvalues always take precedence.</div>
<span style="font-family: monospace;"><span style="white-space: pre;">using ref = int&;
using rval = int&&;
using T1 = ref&&; // T1 = int&
using T2 = rval&; // T2 = int& </span></span><div>
This explains why perfect forwarding works the way it does. You see <i>T&&</i>, but it actually decays to the exact type of the parameter.</div>
<div>
<span style="font-family: monospace;"><span style="white-space: pre;"><br /></span></span><h4>
Rvalues in template functions:</h4>
<div>
One might consider rvalue references in the context of template functions as forwarding references, previously called universal references, because they seem to apply to every situation.</div>
<span style="font-family: monospace;"><span style="white-space: pre;">template<typename T>
void f(T) { }
template<typename T>
void f(T&) { }
template<typename T>
void f(T&&) { }
int main() {
int x = 0;
f(0); // (1)
f(x); // (2)
} </span></span><div>
<div>
Which overload of <i>f</i> gets called at <i>(1)</i> and <i>(2)</i>? </div>
<div>
<br /></div>
<div>
As a rule of thumb, one might assume that <i>f(T&&)</i> gets chosen for both because it can be, (with <i>T = int</i> and <i>T = int&</i>, respectively) but actually the very same rules apply as the non-template version. <i>(1)</i> is ambiguous between <i>f(T)</i> or <i>f(T&&)</i> and <i>(2)</i> is ambiguous between all three overloads (again, because they all fit into the "exact match" rank).</div>
</div>
<div>
<span style="font-family: monospace;"><span style="white-space: pre;">template<typename T>
void f(T&&) { }
</span></span></div>
<div>
<br /></div>
<div>
<span style="font-family: monospace;"><span style="white-space: pre;">void f(const int&) { }
int main() {
int x = 0;
f(x);
}</span></span><div>
Which overload of <i>f</i> gets called? </div>
</div>
<div>
<br /></div>
<div>
This time, <i>f(T&&)</i> gets instantiated with <i>T = int&</i> and <i>f(const int&)</i> requires requires a qualification adjustment from <i>int&</i> to <i>const int&</i>, so <i>f(T&&)</i> gets chosen.</div>
<div>
<span style="font-family: monospace;"><span style="white-space: pre;">template<typename T>
struct X { };
template<typename T>
void f(T&&) { }
template<typename T>
void f(X<T&&>&) { }
int main() {
X<int> x;
X<int&> xref;
f(x); // (1)
f(xref); // (2)
}</span></span></div>
<div>
<div>
Which overload of <i>f</i> gets chosen at <i>(1) </i>and <i>(2)</i>?</div>
</div>
<div>
<br /></div>
<div>
It's <i>f(T&&)</i> for both. Sure, <i>T&&</i> can bind to anything, but the <i>T</i> in <i>X<T&&></i> can only bind to actual rvalue references. They compiler does not try to fit <i>T&&</i> with <i>int</i> or <i>int&</i>, but looks at the whole type, <i>X<T&&> </i>vs<i> X<int> </i>and <i>X<int&></i> and cannot deduce <i>T.</i> Ironically, if we wanted a signature that would bind to <i>X</i> of anything, it would have to be <i>X<T></i>.</div>
<h4>
Conclusions</h4>
<div>
Many seem to believe that template rvalue references are inconsistent with normal overload resolution in C++ and find referring to rvalues in this context as "universal" or "forwarding" references as helpful in order to reason about why they work differently. I hope that through the above examples, the reader has learned something about rvalue references and the consistency of the rules surrounding them.</div>
</div>
Splinter of Chaoshttp://www.blogger.com/profile/14715348728512729776noreply@blogger.com0tag:blogger.com,1999:blog-8638474063464489651.post-56126423186049313772014-11-26T13:19:00.001-08:002014-11-27T15:40:48.405-08:00The Python API and C++Recently, for a job interview task, I was asked to write a Python module with a C or C++ implementation to solve an otherwise simple task. Obviously, I chose C++. While I had never used the Python API before, I found that the existing information on extending Python with C quite sufficient. What surprised me, however, is how little information existed for using C++. A few libraries exist, like <a href="http://www.boost.org/doc/libs/1_39_0/libs/python/doc/index.html">Boost.Python</a>, <a href="http://sourceforge.net/projects/cxx/">PyCXX</a>, and some utilities that parse C++ to create Python bindings, but I didn't find much in the way of actual information without examining the sources of these libraries.<br />
<br />
I will not discuss much why someone would want to implement a Python module in another language (efficiency? better library support for certain tasks? preference?), but why C++? The Python API has basically no type safety--everying is a <i>PyObject *</i>, whether it represents a string, number, or tuple. It requires a considerable amount of boiler-plating--something we can reduce by using the C++ type system. It presents some interesting technical challenges, which are what I will focus on. I will assume some knowledge of the Python API.<br />
<br />
<i>Note: I will be basing this off Python 2.7. Yes, Python 3 is newer, but due to incompatibilities, not a replacement, and also not my system default. Also, I have little experience with the Python API, so do not take this article as authoritative. It represents a journal of my experiments.</i><br />
<i><br />
</i> I have started working on a little utility library (<a href="https://github.com/splinterofchaos/py-cxx">https://github.com/splinterofchaos/py-cxx</a>) for personal use, but for a listing of the code for this article, see the gist: <a href="https://gist.github.com/splinterofchaos/b099149a701edfa5948f">https://gist.github.com/splinterofchaos/b099149a701edfa5948f</a><br />
<br />
<br />
<h4>Writing a Python Module: The Basics</h4><div>First, we will want to create a Python module, which alone is rather uninteresting. For a more in-depth study, one should refer to <a href="https://docs.python.org/2/extending/extending.html">https://docs.python.org/2/extending/extending.html</a>.</div><div><br />
</div><div>Every module requires an <i>init</i> function which communicates to the interpreter what functions, types, and objects this module offers. For now, let's consider a module that counts how many time a certain function gets called.</div><pre class="brush: cpp">#include <Python.h>
PyObject *count(PyObject *self, PyObject *args)
{
static int i = 0;
PySys_WriteStdout("%i\n", ++i); // Just like printf.
return PyInt_FromLong(i);
}
static PyMethodDef countMethods[] = {
{"count", count, METH_VARARGS, "Returns the number of times called."},
{NULL, NULL, 0, NULL}
};
PyMODINIT_FUNC initcount()
{
PyObject *m = Py_InitModule("count", countMethods);
}</pre><div><i>See <a href="https://gist.github.com/splinterofchaos/b099149a701edfa5948f#file-setup-py">setup.py</a> for building this example.</i><br />
<br />
Here, <i>countMethods</i> contains the defined functions in a <i>{name, c-function, function-type, __doc__-string}</i> structure. <i>count</i> must be a <i><a href="https://docs.python.org/2/c-api/structures.html#c.PyCFunction">PyCFunction</a></i>, a function taking <i>self</i> (probably null) and <i>args</i> (argument tuple) parameters and returning an object. <i><a href="https://docs.python.org/2/c-api/structures.html#METH_VARARGS">METH_VARARGS</a></i> lets the interpreter know this is a regular function--other types of functions do exist, but more on that later.<br />
<br />
The <i>PyMODINIT_FUNC</i> macro tells Python that, obviously, this function initializes the module. Note that even <i><a href="https://docs.python.org/2/c-api/allocation.html?highlight=py_initmodule#c.Py_InitModule">Py_InitModule()</a></i> returns a regular Python object!<br />
<br />
There are several improvements we can make. First, we could write an overloaded function, <i>to_py_int()</i>, that could dispatch between <i><a href="https://docs.python.org/2/c-api/int.html#c.PyInt_FromLong">PyInt_FromLong()</a></i>, <i><a href="https://docs.python.org/2/c-api/int.html#c.PyInt_FromSsize_t">PyInt_FromSsize_t()</a></i>, <a href="https://docs.python.org/2/c-api/int.html">and friends</a>, but that's rather mundane so I'll be skipping it. More interesting: we can write a function to define methods.<br />
<br />
Aside from <i>METH_VARARGS</i>, we can have a function be a <i><a href="https://docs.python.org/2/c-api/structures.html#METH_KEYWORDS">METH_KEYWORDS</a></i> which takes an additional parameter, a dictionary, and thus not be a <i>PyCFunction</i>, <i><a href="https://docs.python.org/2/c-api/structures.html#METH_NOARGS">METH_NOARGS</a></i>, which must still be a <i>PyCFunction</i>, and may receive a <i>self</i> argument, but always <i>NULL</i> for <i>args</i>, or <i><a href="https://docs.python.org/2/c-api/structures.html#METH_O">METH_O</a></i>, which has an object as <i>self</i>. It may be convenient to write a function that takes a pointer to a specific type instead of the generic <i>PyObject</i>, but by casting we lose certain safety guarantees and it can be easy to do something stupid, like writing a function will the wrong number of arguments or the wrong <i>METH_*</i> variant.<br />
<pre class="brush: cpp">#include <type_traits>
template<typename R, typename...X>
constexpr int arity(R(*)(X...)) {
return sizeof...(X);
}
template<typename R, typename...X>
constexpr bool returns_PyObject(R(*)(X...)) {
// Result is either a PyObject, or a subclass of one.
return std::is_convertible<R, PyObject *>::value;
}
template<typename R, typename...X>
constexpr bool is_PyCFunction(R(*)(X...)) {
return false;
}
template<>
constexpr bool is_PyCFunction(PyCFunction) {
return true;
}
template<typename F>
constexpr int method_type(F f) {
return arity(f) == 3 ? METH_KEYWORDS
: is_PyCFunction(f) ? METH_VARARGS
: METH_O;
}
template<typename F>
constexpr PyMethodDef method_def(const char *name, const char *doc,
int type, F f)
{
static_assert(arity(F()) == 2 || arity(F()) == 3,
"Methods must have an arity of 2 or 3");
static_assert(returns_PyObject(F()), "Methods must return a PyObject *.");
return {name, (PyCFunction)f, type, doc};
}
template<typename F>
constexpr PyMethodDef method_def(const char *name, const char *doc, F f)
{
return method_def(name, doc, method_type(f), f);
}
static PyMethodDef countMethods[] = {
method_def("count", "Returns the number of times called.", count),
{NULL, NULL, 0, NULL}
};</pre><div>Note that in order to use <i>static_assert</i>s, we construct an <i>F</i> instead of passing <i>f</i> because <i>f</i>, as a parameter, may not be a <i>constexpr</i>.<br />
<br />
Now, we can declare methods in a type-safe manor without having to specify <i>METH_*</i> or lose any safety. While it may be a little limiting (for example, we can't use a lambda to define the method), one can always revert to not using <i>method_def</i> as well.<br />
<br />
<i>Note: It may be safe to define a function that takes no arguments and cast it to a PyCFunction, however I don't know that this would be true across all architectures and ABI calling conventions. </i></div></div><div><br />
One thing lacking from this example is actually using the <i>args</i> parameter. For that, we will need to use <i><a href="https://docs.python.org/2/c-api/arg.html?highlight=pyarg_parsetuple#c.PyArg_ParseTuple">PyArg_ParseTuple()</a></i>.<br />
<br />
<br />
<h4>A Type-Safe <a href="https://docs.python.org/2/c-api/arg.html?highlight=pyarg_parsetuple#c.PyArg_ParseTuple">PyArg_ParseTuple()</a>.</h4>Let's use the example of finding the cross product of two vectors.<br />
<pre class="brush: cpp">#include <Python.h>
#include "Py.h" // includes MethodDef()
PyObject *cross(PyObject *self, PyObject *args)
{
float a, b, c;
float x, y, z;
if (!PyArg_ParseTuple(args, "(fff)(fff)", &a, &b, &c, &x, &y, &z))
return nullptr;
float i = b*z - c*y;
float j = c*x - a*z;
float k = a*y - b*x;
return Py_BuildValue("fff", i, j, k);
}
static PyMethodDef vecMethods[] = {
MethodDef("cross", "Returns the cross product of two 3D vectors.", cross),
{NULL, NULL, 0, NULL}
};
PyMODINIT_FUNC initvec()
{
PyObject *m = Py_InitModule("vec", vecMethods);
}</pre>This lets us write, in Python, <i>cross((a,b,c), (x,y,z))</i>. Even simple functions like this benefit from being written in statically typed languages since, in Python, when one wants to do many operations on some variables, their types must be checked every time, lest you try to add a string to an integer. Here, we do nine operations, but only check the types of the initial six arguments.<br />
<i><br />
</i> <i>PyArg_ParseTuple()</i> is really quite simple; you pass in <i>args</i> and a format string (in this case, using <i>f</i> for <i>float</i>), and pointers to the variables you want to fill. If the tuple doesn't fit the expected format, it sets an error so we can just return <i>NULL</i>. We do our calculation and call <i>Py_BuildValue()</i>, which creates a tuple when given more than one value. Unfortunately, it's very verbose and not type-safe. We can fix that, but first, we must build a format string, preferably at compile time, to pass in.<br />
<br />
First, we can use, for convenience, a <i>typedef</i> of <i>std::integer_sequence</i> to build a list of <i>char</i>s.<br />
<pre class="brush: cpp">template<char...cs>
using CharList = std::integer_sequence<char, cs...>;</pre>Then, define mappings for <i>PyArg_ParseTuple</i>.<br />
<pre class="brush: cpp">template<typename...T>
struct CharListConcat;
template<typename T>
struct CharListConcat<T> {
using type = T;
};
template<typename...U, char...cs, char...cs2>
struct CharListConcat<CharList<cs...>, CharList<cs2...>, U...> {
using type = typename CharListConcat<CharList<cs..., cs2...>, U...>::type;
};
template<typename...T>
using CharListConcat_t = typename CharListConcat<T...>::type;
template<> struct PTCharListOf<float> {
using type = CharList<'f'>;
};
template<typename...Ts>
struct PTCharListOf<std::tuple<Ts...>> {
using type = CharListConcat_t<CharList<'('>,
typename PTCharListOf<std::decay_t<Ts>>::type...,
CharList<')'>>;
};
template<typename T>
using PTCharListOf_t = typename PTCharListOf<T>::type;
</pre>Unfortunately, this strategy is a but limited--we couldn't pass in an <i>std::vector</i> to get the desired affect because we wouldn't know how many elements go into it. A better option would be to add a <i>PyObject *</i> specialization for <i>PTCharListOf</i> and manually check that the result is a list.<br />
<br />
<pre class="brush: cpp">template<> struct PTCharListOf<PyObject *> {
using type = CharList<'O'>;
};</pre>Next, we define a type to build the format:<br />
<br />
<pre class="brush: cpp">template<typename...Ts>
struct ParseTupleBuilder { };
template<typename CL, typename T, typename...Ts>
struct ParseTupleBuilder<CL, T, Ts...> {
using type = ParseTupleBuilder<CharListConcat_t<CL, PTCharListOf_t<T>>,
Ts...>;
constexpr static const char *fmt = type::fmt;
};
template<char...cs>
struct ParseTupleBuilder<CharList<cs...>> {
using type = CharList<cs...>;
static const char fmt[sizeof...(cs) + 1];
};
template<char...cs>
const char ParseTupleBuilder<CharList<cs...>>::fmt[] = { cs..., '\0' };
template<typename...Ts>
constexpr const char *ParseTupleFormat(Ts...) {
return ParseTupleBuilder<CharList<>, std::decay_t<Ts>...>::fmt;
}
</pre>One interesting thing: When I defined <i>fmt</i> inside <i>ParseTupleBuilder</i>, I got an error from inside Python on typing <i>"import vec" </i>claiming that <i>fmt</i>'s constructor had not been defined. The Python docs warn that static global variables with constructors may not be used if Python was built with a C compiler, but defining <i>fmt</i> outside the <i>struct</i> seems to fix this.<br />
<br />
Finally, we can start defining <i>ParseTuple()</i>. The strategy I chose was to build an <i>std::tuple</i> of arguments to send to <i>PyArg_ParseTuple()</i> and examine each argument in a helper function. This will require two helpers, defined below, <i>apply_tuple()</i> and <i>map_tuple()</i>.<br />
<pre class="brush: cpp">template<typename F, typename T, size_t...Is>
decltype(auto) apply_tuple(F&& f, T&& t, std::index_sequence<Is...>) {
return std::forward<F>(f)(std::get<Is>(std::forward<T>(t))...);
}
template<typename F, typename T, size_t...Is>
decltype(auto) map_tuple(F&& f, T&& t, std::index_sequence<Is...>) {
return std::make_tuple(std::forward<F>(f)(std::get<Is>(std::forward<T>(t)))...);
}
template<typename F, typename...Ts,
typename Is = std::make_index_sequence<sizeof...(Ts)>>
decltype(auto) map_tuple(F&& f, std::tuple<Ts...> &t) {
return map_tuple(std::forward<F>(f), t, Is());
}
template<typename...Bound,
typename Indicies = std::make_index_sequence<sizeof...(Bound)>>
bool ParseTuple_impl(std::tuple<Bound...> &&bound) {
return apply_tuple(PyArg_ParseTuple, bound, Indicies());
}
template<typename...Bound, typename Arg, typename...Args>
bool ParseTuple_impl(std::tuple<Bound...> &&bound, Arg &a, Args &...as) {
return ParseTuple_impl(std::tuple_cat(std::move(bound), std::make_tuple(&a)),
as...);
}
template<typename...Bound, typename...Args>
bool ParseTuple_impl(std::tuple<Bound...> &&bound, Optional, Args &...as) {
return ParseTuple_impl(std::move(bound), as...);
}
template<typename...Bound, typename...Ts, typename...Args>
bool ParseTuple_impl(std::tuple<Bound...> &&bound, std::tuple<Ts &...> &t,
Args &...as) {
auto &&mapped = map_tuple([](auto &x) { return &x; }, t);
return ParseTuple_impl(std::tuple_cat(bound, std::move(mapped)),
as...);
}
template<typename...Args>
bool ParseTuple(PyObject *args, Args &&...as) {
return ParseTuple_impl(std::make_tuple(args, ParseTupleFormat(as...)),
as...);
}</pre>Before getting back to our cross product function, we will also want a <i>BuildTuple()</i> function. Please excuse the repetitive nature of this code.<br />
<pre class="brush: cpp">template<typename...Bound,
typename Indicies = std::make_index_sequence<sizeof...(Bound)>>
PyObject *BuildValue_impl(std::tuple<Bound...> &&bound) {
return apply_tuple(Py_BuildValue, bound, Indicies());
}
template<typename...Bound, typename Arg, typename...Args>
PyObject *BuildValue_impl(std::tuple<Bound...> &&bound, Arg a, Args ...as) {
return BuildValue_impl(std::tuple_cat(std::move(bound), std::make_tuple(a)),
as...);
}
template<typename...Bound, typename...Args>
PyObject *BuildValue_impl(std::tuple<Bound...> &&bound, Optional, Args &...as) {
return BuildValue_impl(std::move(bound), as...);
}
template<typename...Bound, typename...Ts, typename...Args>
PyObject *BuildValue_impl(std::tuple<Bound...> &&bound, std::tuple<Ts...> &t,
Args &...as) {
return BuildValue_impl(std::tuple_cat(bound, std::move(t)), as...);
}
template<typename...Args>
PyObject *BuildValue(Args &...as) {
return BuildValue_impl(std::make_tuple(ParseTupleFormat(as...)),
as...);
}</pre>And finally, getting back to our cross product...<br />
<pre class="brush: cpp">PyObject *cross(PyObject *self, PyObject *args)
{
float a, b, c;
float x, y, z;
if (!ParseTuple(args, std::tie(a,b,c), std::tie(x,y,z)))
return nullptr;
float i = b*z - c*y;
float j = c*x - a*z;
float k = a*y - b*x;
return BuildValue(i, j, k);
}</pre>That sure was a lot of work, but it created a simple interface that's hard to use improperly.<br />
<br />
</div><h4>Extending Python Types</h4><div>Probably the cornerstone of extending Python itself would be to define new types that interact well with the existing Python infrastructure. For efficiency's sake, the more variables we can statically type and hold in a <i>struct</i>, the better. The Python docs suggest extending a type this way:<br />
<pre class="brush: cpp">typedef struct {
PyObject_HEAD
...
} MyType;</pre>The macro, <i><a href="https://docs.python.org/2/c-api/structures.html?highlight=pyobject_head#c.PyObject_HEAD">PyObject_HEAD</a></i>, contains fields common to any Python Object to ensure that a casting <i>MyType</i> pointer to a <i>PyObject</i> is valid. This is a common technique for representing inheritance in C, however, we can get the same affect in C++ by using inheritance.<br />
<br />
Also, every Python type requires an accompanying <i><a href="https://docs.python.org/2/c-api/typeobj.html">PyTypeObject</a></i>, which is also a <i>PyObject</i>. The <i>PyTypeObject</i> stores lots of runtime information about a type including what function to use for allocation, to convert to a string, its methods, its base class, how to deallocate it, and more.<br />
<br />
We can use a constructor for our extension type, but it may be wisest not to. One of the fields of the type object, <i><a href="https://docs.python.org/2/c-api/typeobj.html#c.PyTypeObject.tp_alloc">tp_alloc</a></i>, defines how to allocate memory for this type, including setting the reference count to one, specifying the <i>ob_type</i> field (a member of <i>PyObject</i>), and a few other things. For example, it must work, even for classes that inherit from our custom type. It relates enough to Python's internals that I think it best be left alone, and can be left as <i>NULL</i> in the <i>PyTypeObject</i> without trouble.<br />
<br />
More interesting would be <i><a href="https://docs.python.org/2/c-api/typeobj.html#c.PyTypeObject.tp_new">tp_new</a></i>, which must point to a function that calls <i>tp_alloc</i> and initializes the memory, and <b>must</b> be defined in order to create instances of our new type. We can define <i>tp_new</i> to use a placement new for objects in our type that require construction.<br />
<br />
We can generalize that an extension of <i>PyObject</i> will look like this:<br />
<pre class="brush: cpp">template<typename T>
struct Extention : PyObject
{
static PyTypeObject type;
T ext;
T &get() & { return this->ext; }
const T &get() const & { return this->ext; }
T *ptr() & { return &this->ext; }
const T *ptr() const & { return &this->ext; }
};</pre>We can define a default <i>tp_new </i>and <i>tp_dealloc </i>and initialize <i>type</i> like so:<br />
<pre class="brush: cpp">template<typename T,
typename = std::enable_if_t<std::is_default_constructible<T>::value>>
newfunc default_new()
{
return [](PyTypeObject *type, PyObject *args, PyObject *kwds)
{
using Self = Extention<T>;
Self *self = (Self *) type->tp_alloc(type, 0);
if (self)
new (self->ptr()) T();
return (PyObject *) self;
};
}
template<typename T,
typename = std::enable_if_t<!std::is_default_constructible<T>::value>>
auto default_new() {
return [](PyTypeObject *type, PyObject *args, PyObject *kwds)
{
return type->tp_alloc(type, 0);
};
}
template<typename T>
PyTypeObject Extention<T>::type = {
PyObject_HEAD_INIT(NULL)
0, // ob_size
0, // tp_name
sizeof(Extention<T>), // tp_basicsize
0, // tp_itemsize
destructor([](PyObject *self) {
((Extention *) self)->get().T::~T();
self->ob_type->tp_free(self);
}),
0, // tp_print
0, // tp_getattr
0, // tp_setattr
0, // tp_compare
0, // tp_repr
0, // tp_as_number
0, // tp_as_sequence
0, // tp_as_mapping
0, // tp_hash
0, // tp_call
0, // tp_str
0, // tp_getattro
0, // tp_setattro
0, // tp_as_buffer
Py_TPFLAGS_DEFAULT, // tp_flags
0, // tp_doc
0, // tp_traverse
0, // tp_clear
0, // tp_richcompare
0, // tp_weaklistoffset
0, // tp_iter
0, // tp_iternext
0, // tp_methods
0, // tp_members
0, // tp_getset
0, // tp_base
0, // tp_dict
0, // tp_descr_get
0, // tp_descr_set
0, // tp_dictoffset
0, // tp_init
0, // tp_alloc
default_new<T>(), // tp_new
};</pre><i>PyTypeObject</i> does have a few more fields, but the compiler sets them to <i>0</i> for us. We do, however, have to set <i>tp_basicsize</i> in order for the right amount of memory to be allocated. Since a type in C++ may not be default-constructible, <i>default_new()</i> may return a function that does not construct the object; this must be done in <i>tp_init</i>.<br />
<br />
Now, returning to the cross product example, consider this:<br />
<pre class="brush: cpp">struct Vec {
float x, y, z;
};
using PyVec = Extention<Vec>;
int init_vec(PyVec *self, PyObject *args, PyObject *)
{
Vec &v = self->get();
if (!ParseTuple(args, v.x, v.y, v.z))
return -1;
return 0;
}
PyObject *vec_str(PyVec *self)
{
return PyString_FromString(("<" + std::to_string(self->get().x) +
", " + std::to_string(self->get().y) +
", " + std::to_string(self->get().z) +
">").c_str());
}
PyMODINIT_FUNC initvec()
{
PyVec::type.tp_name = "vec.Vec";
PyVec::type.tp_init = (initproc) init_vec;
PyVec::type.tp_repr = PyVec::type.tp_str = (reprfunc) vec_str;
if (PyType_Ready(&PyVec::type) < 0)
return;
PyObject *m = Py_InitModule("vec", vecMethods);
if (!m)
return;
Py_INCREF(&PyVec::type);
PyModule_AddObject(m, "Vec", (PyObject *) &PyVec::type);
}</pre>Note that <i>tp_repr</i> is used to display the result of evaluating an expression, and <i>tp_str</i> is used for printing. <i>tp_init</i> is used to construct our value and relates to <i>Vec.__init__()</i> in Python. <i>PyType_Ready()</i> finalizes the type and fills in some of the missing <i>tp_*</i> fields. We add the type to the module as a global object and increment its reference count so Python doesn't try to destruct it. For brevity, I decided not to include functions to check the type safety of the <i>initproc</i> and <i>reprfunc</i> casts.<br />
<br />
Since <i>Vec</i> is default constructible, we only need to worry about assigning the members in the <i>init</i> function.<br />
<br />
And now, <i>cross</i> looks like this:<br />
<pre class="brush: cpp">PyObject *cross(PyObject *self, PyObject *args)
{
PyObject *o1, *o2;
if (!ParseTuple(args, o1, o2))
return nullptr;
// Ensure o1 and 2 are the right types.
if (!PyType_IsSubtype(o1->ob_type, &PyVec::type) ||
!PyType_IsSubtype(o2->ob_type, &PyVec::type))
return nullptr;
Vec &v = ((PyVec *) o1)->get(), &w = ((PyVec *) o2)->get();
float i = v.y*w.z - v.z*w.y;
float j = v.z*w.x - v.x*w.z;
float k = v.x*w.y - v.y*w.x;
PyObject *ret = PyVec::type.tp_new(&PyVec::type, nullptr, nullptr);
PyObject *val = BuildValue(i, j, k);
init_vec((PyVec *) ret, val, nullptr);
Py_DECREF(val);
return ret;
}</pre><br />
<br />
<h4>Conclusions</h4></div><div>Despite this being quite a long article, it has only touched the surface of how the Python API can be extended. There are many restrictions and it certainly puts a cramp on C++'s style, but the moral of this story is that just because you need to work with a C API doesn't mean you can't use modern C++ techniques.</div><div><br />
</div><div>The python docs: <a href="https://docs.python.org/2/c-api/">https://docs.python.org/2/c-api/</a></div><div>The gist: <a href="https://gist.github.com/splinterofchaos/b099149a701edfa5948f">https://gist.github.com/splinterofchaos/b099149a701edfa5948f</a></div><div>My library: <a href="https://github.com/splinterofchaos/py-cxx">https://github.com/splinterofchaos/py-cxx</a></div>Splinter of Chaoshttp://www.blogger.com/profile/14715348728512729776noreply@blogger.com1tag:blogger.com,1999:blog-8638474063464489651.post-79095346454681581022014-02-04T16:52:00.001-08:002014-08-21T13:51:10.429-07:00Clang 3.4 and C++14With each new release, gcc and clang add on more C++11 and C++14 features. While clang has been behind on some features, though ahead on others, they now claim to have <a href="http://llvm.org/releases/3.4/tools/clang/docs/ReleaseNotes.html#id1">C++1y all worked out</a>.<br />
<br />
This article is not comprehensive.<br />
Clang's 3.4 C++ release notes: <a href="http://llvm.org/releases/3.4/tools/clang/docs/ReleaseNotes.html#id1">http://llvm.org/releases/3.4/tools/clang/docs/ReleaseNotes.html#id1</a><br />
libc++'s C++1y status: <a href="http://libcxx.llvm.org/cxx1y_status.html">http://libcxx.llvm.org/cxx1y_status.html</a><br />
<br />
<i>Note: To compile these examples requires the flags, "-stdlib=libc++" and "-std=c++1y".</i><br />
<h4>
</h4>
<h4>
</h4>
<h4>
Variable templates. </h4>
<br />
This feature, from <a href="http://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&cd=1&cad=rja&ved=0CCYQFjAA&url=http%3A%2F%2Fisocpp.org%2Ffiles%2Fpapers%2FN3651.pdf&ei=lyfvUvBK6IzIAcvqgdAJ&usg=AFQjCNHOwaj3WD33nIpo0eMrc29jjI605w&sig2=n11eXRRQRAeBcSNdRQTsdw&bvm=bv.60444564,d.aWc">N3651</a>, took me most be surprise, but it also seems quite obvious. In the simplest example, let <i>def<T></i> be a variable that represents the default-constructed value of any type, <i>T</i>.<br />
<br />
<pre class="brush: cpp">template<typename T>
constexpr T def = T();
auto x = def<int>; // x = int()
auto y = def<char>; // y = char() </pre>
<br />
The proposal uses the example of <i>pi</i>, where it may be more useful to store it as a <i>float</i> or <i>double</i>, or even <i>long double</i>. By defining it as a template, one can have precision when needed and faster, but less precise, operations otherwise.<br />
<br />
For another example, consider storing a few prime numbers, but not specifying the type of their container.<br />
<br />
<pre class="brush: cpp">template<template<typename...> class Seq>
Seq<int> primes = { 1, 2, 3, 5, 7, 11, 13, 17, 19 };
auto vec = primes<std::vector>;
auto list = primes<std::list>;</pre>
<i><a href="https://gist.github.com/splinterofchaos/8810949#file-main-cpp-L59">(gist)</a></i><br />
<br />
Also, the standard library contains many template meta-functions, some with a static <i>value</i> member. Variable templates help there, too.<br />
<br />
<pre class="brush: cpp">template<typename T, typename U>
constexpr bool is_same = std::is_same<T,U>::value;
bool t = is_same<int,int>; // true
bool f = is_same<int,float>; // false</pre>
<i><a href="http://en.cppreference.com/w/cpp/types/is_same">(std::is_same)</a></i><br />
<br />
But since variable templates can be specialized just like template functions, it makes as much sense to define it this way:<br />
<br />
<pre class="brush: cpp">template<typename T, typename U>
constexpr bool is_same = false;
template<typename T>
constexpr bool is_same<T,T> = true;</pre>
<i><a href="https://gist.github.com/splinterofchaos/8810949#file-main-cpp-L186">(gist)</a></i> <br />
<br />
Except for when one requires that <i>is_same</i> refers to an <i>integral_constant</i>.<br />
<br />
One thing worries me about this feature: How do we tell the difference between template meta-functions, template functions, template function objects, and variable templates? What naming conventions will be invented? Consider the above definition of <i>is_same</i> and the fallowing:<br />
<br />
<pre class="brush: cpp">// A template lambda that looks like a template function.
template<typename T>
auto f = [](T t){ ... };
// A template meta-function that might be better as a variable template.
template<typename T>
struct Func : { using value = ...; };</pre>
<br />
They each has subtly different syntaxes. For example, <a href="http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2013/n3545.pdf">N3545</a> adds an <i>operator()</i> overload to <i>std::integral_constant</i> which enables syntax like this: <i>bool b = std::is_same<T,U>()</i>, while <a href="http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2013/n3655.pdf">N3655</a> adds <i>std::is_same_t<T,U></i> as a synonym for <i>std::is_same<T,U>::value</i>. <i>(Note: libc++ is missing std::is_same_t.) </i>Even without variable templates, we have now three ways to refer to the same thing.<br />
<br />
Finally, one problem I did have with it is I wrote a function like so:<br />
<br />
<pre class="brush: cpp">template<typename T>
auto area( T r ) {
return pi<T> * r * r;
};</pre>
<br />
and found that clang thought <i>pi<T></i> was undefined at link time and clang's diagnostics did little to point that out.<br />
<br />
<pre class="brush: cpp">/tmp/main-3487e1.o: In function `auto $_1::operator()<Circle<double> >(Circle<double>) const':
main.cpp:(.text+0x5e3d): undefined reference to `_ZL2piIdE'
clang: error: linker command failed with exit code 1 (use -v to see invocation</pre>
<br />
I solved this by explicitly instantiating <i>pi</i> for the types I needed by adding this to <i>main</i>:<br />
<br />
<pre class="brush: cpp">pi<float>;
pi<double>;
</pre>
<br />
Why in <i>main</i> and not in global scope? When I tried it right below the definition of <i>pi</i>, clang thought I wanted to specialize the type. Finally, attempting <i>template<> pi<float>; </i>left the value uninitialized. This is a <a href="http://llvm.org/bugs/show_bug.cgi?id=17848">bug in clang</a>, and has been fixed. Until the next release, variable templates work as long as only non-template functions refer to them.<br />
<h4>
</h4>
<h4>
</h4>
<h4>
Generic lambdas and generalized capture.</h4>
<br />
Hey, didn't I already do <a href="http://yapb-soc.blogspot.com/2012/12/clang-and-generic-polymorphic-lambdas.html">an article</a> about this? Well, that one covers Faisal Vali's <a href="https://github.com/faisalv/clang">fork of clang</a> based off of the <a href="http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2012/n3418.pdf">N3418</a>, which has many more features than this iteration based off the more conservative <a href="http://www.open-std.org/JTC1/SC22/WG21/docs/papers/2013/n3559.pdf">N3559</a>. Unfortunately it lacks the terseness and explicit template syntax (i.e.<i> []<class T>(T t) f(t)</i>), but it maintains <i>auto</i>matic types for parameters (<i>[](auto t){return f(t);}</i>). <br />
<br />
Defining lambdas as variable templates helps, but variable templates lack the abilities of functions, like implicit template parameters. For the situations where that may be helpful, it's there.<br />
<br />
<pre class="brush: cpp">template<typename T>
auto convert = [](const auto& x) {
return T(x);
};</pre>
<i><a href="https://gist.github.com/splinterofchaos/8810949#file-main-cpp-L181">(gist)</a></i><br />
<br />
Also, previously, clang couldn't capture values by move or forward into lambdas, which prohibited capturing move-only types by anything other than a reference. Transitively, that meant many perfect forwarding functions couldn't return lambdas.<br />
<br />
Now, initialization is "general", to some degree.<br />
<br />
<pre class="brush: cpp">std::unique_ptr<int> p = std::make_unique<int>(5);
auto add_p = [p=std::move(p)](int x){ return x + *p; };
std::cout << "5 + 5 = " << add_p(5) << std::endl;
</pre>
<i>(See also: <a href="http://en.cppreference.com/w/cpp/memory/unique_ptr/make_unique">std::make_unique</a>)</i><br />
<br />
Values can also be copied into a lambda using this syntax, but check out Scott Meyer's article for why <i>[x]</i> or <i>[=]</i> does not mean the same thing as <i>[x=x]</i> for <i>mutable</i> lambdas. (<a href="http://scottmeyers.blogspot.de/2014/02/capture-quirk-in-c14.html">http://scottmeyers.blogspot.de/2014/02/capture-quirk-in-c14.html</a>)<br />
<br />
Values can also be defined and initialized in the capture expression.<br />
<br />
<pre class="brush: cpp">std::vector<int> nums{ 5, 6, 7, 2, 9, 1 };
auto count = [i=0]( auto seq ) mutable {
for( const auto& e : seq )
i++; // Would error without "mutable".
return i;
};
</pre>
<br />
gcc has had at least partial support for this since 4.5, but should fully support it in 4.9.<br />
<h4>
</h4>
<h4>
</h4>
<h4>
Auto function return types.</h4>
<br />
This is also a feature gcc has had since 4.8 (<a href="http://yapb-soc.blogspot.com/2012/12/gcc-48-has-automatic-return-type.html">and I wrote about, as well</a>), but that was based off of <a href="http://www.open-std.org/JTC1/SC22/WG21/docs/papers/2012/n3386.html">N3386</a>, whereas gcc 4.9 and clang 3.4 base off of <a href="http://isocpp.org/files/papers/N3638.html">N3638</a>. I will not say much here because this is not an entirely new feature, not much has changed, and it's easy to groc.<br />
<br />
Most notably, the syntax, <i>decltype(auto)</i>, has been added to overcome some of the shortcomings of <i>auto</i>. For example, if we try to write a function that returns a reference with <i>auto</i>, a value is returned. But if we write it...<br />
<br />
<pre class="brush: cpp">decltype(auto) ref(int& x) {
return x;
}
decltype(auto) copy(int x) {
return x;
} </pre>
<a href="https://gist.github.com/splinterofchaos/8810949#file-main-cpp-L199">(gist)</a><br />
<br />
Then a reference is returned when a is given, and a copy when a value is given. (Alternatively, the return type of <i>ref</i> could be <i>auto&</i>.)<br />
<h4>
</h4>
<h4>
</h4>
<h4>
More generalized constexprs. </h4>
<br />
The requirement that <i>constexpr</i>s be single return statements worked well enough, but simple functions that required more than one line could not be <i>constexpr</i>. It sometimes forced inefficient implementations in order to have at least some of its results generated at compile-time, but not always all. The factorial function serves as a good example.<br />
<br />
<pre class="brush: cpp">constexpr unsigned long long fact( unsigned long long x ) {
return x <= 1 ? 1ull : x * fact(x-1);
}</pre>
<br />
but now we can write...<br />
<br />
<pre class="brush: cpp">constexpr auto fact2( unsigned long long x ) {
auto product = x;
while( --x ) // Branching.
product *= x; // Variable mutation.
return product;
}</pre>
<a href="https://gist.github.com/splinterofchaos/8810949#file-main-cpp-L21">(gist)</a><br />
<br />
This version may be more efficient, both at compile time and run time.<br />
<br />
The accompanying release of libc++ now labels many standard functions as constexpr thanks to <a href="http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2012/n3469.html">N3469 (chrono)</a>, <a href="http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2012/n3470.html">3470 (containers)</a>, <a href="http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2012/n3471.html">3471 (utility)</a>, <a href="http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2011/n3302.html">3302 (std::complex)</a>, and <a href="http://wiki.edg.com/twiki/pub/Wg21chicago2013/FormalMotions/N3789.txt">3789 (functional)</a>.<br />
<br />
<i>Note: gcc 4.9 does not yet implement branching and mutation in constexprs, but does include some of the library enhancements.</i> <br />
<br />
<h4>
</h4>
<h4>
<i> </i></h4>
<h4>
<a href="http://en.cppreference.com/w/cpp/utility/integer_sequence"><i>std::integer_sequence</i></a> for working with <a href="http://en.cppreference.com/w/cpp/utility/tuple">tuple</a>s.</h4>
<br />
Although this library addition may not be of use to everyone, anyone who has attempted to unpack a tuple into a function (like <a href="http://stackoverflow.com/questions/7858817/unpacking-a-tuple-to-call-a-matching-function-pointer">this guy</a> or <a href="http://stackoverflow.com/questions/687490/how-do-i-expand-a-tuple-into-variadic-template-functions-arguments">that guy</a> or <a href="https://groups.google.com/forum/#!topic/comp.lang.c++.moderated/dQ-jgVzarEU">this one</a> or ...) will appreciate <a href="http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2013/n3658.html">N3658</a> for "compile-time integer sequences". Thus far, no standard solution has existed. N3658 adds one template class, <i>std::integer_sequence<T,t0,t1,...,tn></i>, and <i>std::index_sequence<t0,...,tn>, </i>which is an <i>integer_sequence</i> with <i>T=size_t</i>. This lets us write an <i>apply_tuple</i> function like so:<br />
<br />
<br />
<pre class="brush: cpp">template<typename F, typename Tup, size_t ...I>
auto apply_tuple( F&& f, Tup&& t, std::index_sequence<I...> ) {
return std::forward<F>(f) (
std::get<I>( std::forward<Tup>(t) )...
);
}
</pre>
<i>(See also: <a href="http://en.cppreference.com/w/cpp/utility/tuple/get">std::get</a>)</i><br />
<br />
For those who have not seen a function like this, the point of this function is just to capture the indexes from the <i>index_sequence</i> and call <i>std::get</i> variadically. It requires another function to create the <i>index_sequence</i>.<br />
<br />
N3658 also supplies <i>std::make_integer_sequence<T,N></i>, which expands to <i>std::integer_sequence<T,0,1,...,N-1></i>, <i>std::make_index_sequence<N></i>, and <i>std::index_sequence_for<T...></i>, which expands to <i>std::make_index_sequence<sizeof...(T)></i>.<br />
<br />
<br />
<pre class="brush: cpp">// The auto return type especially helps here.
template<typename F, typename Tup >
auto apply_tuple( F&& f, Tup&& t ) {
using T = std::decay_t<Tup>; // Thanks, N3655, for decay_t.
constexpr auto size = std::tuple_size<T>(); // N3545 for the use of operator().
using indicies = std::make_index_sequence<size>;
return apply_tuple( std::forward<F>(f), std::forward<Tup>(t), indicies() );
}
</pre>
<i>(See also: <a href="http://en.cppreference.com/w/cpp/types/decay">std::decay</a>, <a href="http://en.cppreference.com/w/cpp/utility/tuple/tuple_size">std::tuple_size</a>, <a href="https://gist.github.com/splinterofchaos/8810949#file-main-cpp-L88">gist</a>) </i><br />
<br />
Unfortunately, even though the proposal uses a similar function as an example, there still exists no standard <i>apply_tuple</i> function, nor a standard way to extract an <i>index_sequence</i> from a <i>tuple</i>. Still, there may exist several conventions for applying <i>tuple</i>s. For example, the function may be the first element or an outside component. The tuple may have an incomplete argument set and require additional arguments for <i>apply_tuple</i> to work.<br />
<br />
<b>Update:<i> </i></b>Two library proposals in the works address this issue: <a href="http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2013/n3802.pdf">N3802 (apply)</a>, and <a href="http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2013/n3728.html">N3416 (language extension: parameter packs)</a>.<br />
<br />
<h4>
</h4>
<h4>
<i> </i></h4>
<h4>
<i>experimental::optional</i>.</h4>
<br />
While not accepted into C++14, libc++ has an implementation of <a href="http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2013/n3672.html">N3672</a>'s <i>optional</i> hidden away in the <i>experimental </i>folder. Boost fans may think of it as the standard's answer to <a href="http://www.boost.org/doc/libs/1_55_0/libs/optional/doc/html/index.html"><i>boost::optional</i></a> as functional programers may think of it as like <a href="http://www.haskell.org/haskellwiki/Maybe">Haskell's <i>Maybe</i></a>.<br />
<br />
Basically, some operations may not have a value to return. For example<i>, </i>a square root cannot be taken from a negative number, so one might want to write a "safe" square root function that returned a value only when <i>x>0<u>.</u></i><br />
<br />
<br />
<pre class="brush: cpp">#include <experimental/optional>
template<typename T>
using optional = std::experimental::optional<T>;
optional<float> square_root( float x ) {
return x > 0 ? std::sqrt(x) : optional<float>();
}
</pre>
<a href="https://gist.github.com/splinterofchaos/8810949#file-main-cpp-L106">(gist)</a><br />
<br />
Using an <i>optional</i> is simple because they implicitly convert to bools and act like a pointer, but with value semantics (which is incidentally how libc++ implements it). Without <i>optional</i>, one might use a <i>unique_ptr</i>, but value semantics on initialization and assignment make <i>optional</i> more convenient.<br />
<br />
<br />
<pre class="brush: cpp">auto qroot( float a, float b, float c )
-> optional< std::tuple<float,float> >
{
// Optionals implicitly convert to bools.
if( auto root = square_root(b*b - 4*a*c) ) {
float x1 = -b + *root / (2*a);
float x2 = -b - *root / (2*a);
return {{ x1, x2 }}; // Like optional{tuple{}}.
}
return {}; // An empty optional.
} </pre>
<i><a href="https://gist.github.com/splinterofchaos/8810949#file-main-cpp-L115">(gist)</a></i><br />
<br />
<h4>
</h4>
<h4>
Misc. improvements.</h4>
<br />
This version of libc++ allows one to retrieve a <i>tuple</i>'s elements by type using <i>std::get<T></i>.<br />
<br />
<br />
<pre class="brush: cpp">std::tuple<int,char> t1{1,'a'};
std::tuple<int,int> t2{1,2};
int x = std::get<int>(t1); // Fine.
int y = std::get<int>(t2); // Error, t2 contains two ints.</pre>
<br />
<br />
Clang now allows the use of single-quotes (') to separate digits. <i>1'000'000</i> becomes 1000000, and <i>1'0'0</i> becomes 100. <a href="https://gist.github.com/splinterofchaos/8810949#file-main-cpp-L267">(example)</a> (It doesn't require that the separations make sense, but one cannot write <i>1''0</i> or <i>'10</i>.)<br />
<br />
libc++ implements <a href="http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2013/n3655.pdf">N3655</a>, which adds several template aliases in the form of <i>std::*_t = std::*::type</i> to <i><type_traits></i>, such as <i>std::result_of_t</i>, <i>std::is_integral_t</i>, and many more. Unfortunately, while N3655 also adds <i>std::is_same_t</i> (see the top of the 7th page), libc++ does not define it. I do not know, but I believe this may be an oversight that will be fixed soon as it requires only one line.<br />
<br />
<a href="http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2012/n3421.htm">N3421</a> adds specializations to the members of <i><functional></i>. If one wanted to send an addition function into another functions, one might write <i>f(std::plus<int>(),args...)</i>, but we new longer need to specify the type and can instead write <i>std::plus<>()</i>. This instantiates a function object that can accept two values of any type to add them. Similarly, <i>std::greater<></i>, <i>std::less<></i>, <i>std::equal_to<></i>, etc...<br />
<h4>
</h4>
<h4>
</h4>
<h4>
Conclusions.</h4>
<br />
This may not be the <i>most</i> ground-breaking release, but C++14 expands on the concepts from C++11, improves the library, and adds a few missing features, and I find it impressive that the clang team has achieved this so preemptively. I selected to talk about the features I thought were most interesting, but I did not talk about, for example, <a href="http://wiki.edg.com/twiki/pub/Wg21chicago2013/FormalMotions/n3778.html">sized deallocation</a>, <a href="http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2013/n3662.html"><i>std::dynarray</i></a> (<i><experimental/dynarry></i>), some <a href="http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2013/n3671.html">additional overloads in <i><algorithm></i></a>, or <a href="http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2013/n3644.pdf">Null Forward Iterators</a>, to name a few. See the bottom for links to the full lists.<br />
<br />
The GNU team still needs to do more work to catch up to clang. If one wanted to write code for both gcc 4.9 and clang 3.4, they could use generic lambdas, <i>auto</i> for return types, but not variable templates or generalized <i>constexpr</i>s. For the library, gcc 4.9 includes <i>std::make_unique</i> (as did 4.8), the N3412 specializations in <i><functional></i>, integer sequences, <i>constexpr</i> library improvements, even <i>experimental::optional</i> (though I'm not sure where), and much more. It may be worth noting it does not seem to include the <i><type_traits> </i>template aliases, like <i>result_of_t</i>.<br />
<br />
See clang's full release notes related to C++14 here: <a href="http://llvm.org/releases/3.4/tools/clang/docs/ReleaseNotes.html#id1">http://llvm.org/releases/3.4/tools/clang/docs/ReleaseNotes.html#id1</a><br />
For libc++'s improvements, see: <a href="http://libcxx.llvm.org/cxx1y_status.html">http://libcxx.llvm.org/cxx1y_status.html</a><br />
gcc 4.9's C++14 features: <a href="http://gcc.gnu.org/projects/cxx1y.html">http://gcc.gnu.org/projects/cxx1y.html</a><br />
And gcc's libstdc++ improvements: <a href="http://gcc.gnu.org/onlinedocs/libstdc++/manual/status.html#status.iso.2014">http://gcc.gnu.org/onlinedocs/libstdc++/manual/status.html#status.iso.2014</a><br />
<br />
The code I wrote to test these features: <a href="https://gist.github.com/splinterofchaos/8810949">https://gist.github.com/splinterofchaos/8810949</a><br />
<br />Splinter of Chaoshttp://www.blogger.com/profile/14715348728512729776noreply@blogger.com5tag:blogger.com,1999:blog-8638474063464489651.post-83584466381469403972012-12-28T11:11:00.000-08:002012-12-29T15:58:04.593-08:00Clang and Generic (Polymorphic) Lambdas.Recently a Faisal Vali put forth <a href="http://faisalv.github.com/clang-glambda/">an implementation</a> of <a href="http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2012/n3418.pdf">n3418</a>, which he co-authored with Herb Stutter and Dave Abraham, allowing generic lambdas using a fork of clang. It also includes auto type deduction, which I wrote about being <a href="http://yapb-soc.blogspot.com/2012/12/gcc-48-has-automatic-return-type.html">implemented in gcc 4.8</a>. There are a few caveats before continuing: This has not been merged into the mainline. It has a few bugs, but Vali is quick to fix them if you point one out. The implementation itself is a proof of concept (similar to automatic type deduction) and so it isn't unreasonable to assume some things might change. Section <b>2.4</b> of the proposal (named lambdas) has not yet been implemented. And while this doesn't allow us to do many things that were previously impossible, the possible used to be so verbose that no one would want to do it!<br />
<br />
Generic lambdas are profound and may have a great impact on the style of code written in C++. Consider this a light (and lacking) overview of what is possible.<br />
<br />
Before continuing, I want to note that I found evidence that some GCC developers had begun working on generic lambdas (from the mailing list: <a href="http://gcc.gnu.org/ml/gcc/2009-08/msg00174.html">Latest experimental polymorphic lambda patches</a>), however, I cannot find anything more recent than 2009 discussing this, and code using <i>auto</i> and lambdas does not compile.<br />
<br />
<br />
<h4>
Being terse. </h4>
This patch allows the writing of incredibly terse polymorphic functions, such as these:<br />
<br />
<pre class="brush: cpp">auto add = [](auto x, auto y) x + y;
auto times = [](auto x, auto y) x * y;
auto print_int = [](int x) void(std::cout << x);</pre>
<br />
No <i>{}, </i>no <i>return</i>, auto-deduced types, and <i>void</i> can even be used to throw away the value of state-full operations. <i>x</i> and <i>y</i> can be anything and the return type is entirely dependent on them. Why is this interesting? Say you want to find the product of a <i>vector</i>.<br />
<br />
<pre class="brush: cpp">auto prod = std::accumulate (
v.begin(), v.end(), 1,
[]( int x, int y ){ return x * y; }
);</pre>
<br />
Bit of a mouthful, right? <i>v</i> might store <i>int</i>s today, but tomorrow, maybe it will store <i>long long int</i>s to avoid overflow or just <i>unsigned int</i>s to avoid negatives. When the <i>vector</i>'s declaration changes, the lambda's argument types need to change, which is a maintenance problem. I currently solve this by writing polymorphic function objects.<br />
<br />
<pre class="brush: cpp">constexpr struct {
template< class X, class Y >
auto operator () ( X x, Y y )
-> decltype(x*y)
{
return x * y;
}
} timesObj{}; </pre>
<br />
But the above and <i>times</i> are mostly equivalent. (A lambda can be thought of as an automatic function object. It even has a unique type. (see below: <b>Overloading</b>))<br />
<br />
<pre class="brush: cpp">auto prod = std::accumulate (
v.begin(), v.end(), 1,
times
);</pre>
<br />
This never needs to change unless the underlying operation (multiplication) changes.<br />
<br />
Sometimes, an underlying operation is consistent across types. Using <i>zip_tuple</i> as an example from my article "<a href="http://yapb-soc.blogspot.com/2012/12/zipping-and-mapping-tuples.html">Zipping and Mapping Tuples</a>", one could write:<br />
<br />
<pre class="brush: cpp">std::tuple<int,std::string> a{1,"hello "},
b{2,"world!"};
// r = {3,"hello world"}
auto r = zipTuple( ([](auto x, auto y) x + y), a, b );</pre>
<br />
Because of the comma operator, we must put the lambda in parentheses to make clear where it ends. <br />
<br />
Up until now, things like <a href="http://en.wikipedia.org/wiki/Function_composition">composition</a> could not be lambdas.<br />
<br />
<pre class="brush: cpp">template< class F, class G > struct Composition {
F f;
G g;
template< class ...X >
auto operator () ( X&& ...x )
-> decltype( f(g(std::forward<X>(x)...)) )
{
return f( g( std::forward<X>(x)... ) );
}
};
template< class F, class G, class C = Composition<F,G> >
C compose( const F& f, const G& g ) {
return C{f,g};
}
int main () {
auto add1 = []( auto x ) x + 1;
auto Char = []( auto c ) char(c);
// Prints "a + 1 = b"
std::cout << "a + 1 = " << compose(Char,add1)('a') << std::endl;
}</pre>
<br />
<i>compose</i> is generic and it returns a generic function object. Generic lambdas make the same possible by returning another generic lambda that captures <i>f</i> and <i>g</i>.<br />
<br />
<pre class="brush: cpp">auto compose = [](auto f, auto g)
[=](auto x) f(g(x));</pre>
<br />
However, this version of <i>compose</i> only allows one argument. Luckily, generic lambdas can be fully templated, variadic, and perfect forwarding.<br />
<br />
<pre class="brush: cpp">auto compose =
[]( auto f, auto g )
[=]<class ...X>( X&& ...x )
f( g(std::forward<X>(x)...) );</pre>
<br />
However, the syntax for these lambdas is so convenient, one might as well drop the functional programming theory and write <br />
<br />
<pre class="brush: cpp">auto f = [](char c) char(c+1);</pre>
<br />
For an example of the power of nested lambdas, consider currying:<br />
<br />
<pre class="brush: cpp">auto curry3 =
[]( auto f )
[=](auto x) [=](auto y) [=](auto z)
f(x,y,z);
auto sum3 = [](auto x, auto y, auto z) x + y + z;
auto ten = curry3(sum3)(2)(3)(5)</pre>
<br />
Nested lambdas especially aid in the use of <a href="http://en.wikipedia.org/wiki/Monad_%28functional_programming%29">monads</a>, as I have written about previously ("<a href="http://yapb-soc.blogspot.com/2012/10/monads-in-c.html">Monads in C++</a>").<br />
<br />
<pre class="brush: cpp">auto justThree = Just(1) >>= [](auto x)
Just(2) >>= [](auto y)
mreturn<std::unique_ptr>( x + y ); // Or Just(x+y).</pre>
<br />
This also takes care of the <i>return mreturn</i> problem I discussed in that article.<br />
<br />
<h4>
Overloading</h4>
Overloading functions is obviously useful, but impossible with lambdas alone. To fully take advantage of their brevity, we must have a way to overload them. In the proposal, Mathius Gaunard is attributed with the following:<br />
<br />
<pre class="brush: cpp">template<class F1, class F2> struct overloaded : F1, F2
{
overloaded(F1 x1, F2 x2) : F1(x1), F2(x2) {}
using F1::operator();
using F2::operator();
};
template<class F1, class F2>
overloaded<F1, F2> overload(F1 f1, F2 f2)
{ return overloaded<F1, F2>(f1, f2); } </pre>
<br />
(See also: "<a href="http://cpp-next.com/archive/2012/09/unifying-generic-functions-and-function-objects/">Unifying Generic Functions and Function Objects</a>")<br />
<br />
This works because lambdas are function objects with a unique type, and can therefore act as the base class for <i>overloaded. </i>This is an unlikely solution because this fact is so rarely taken advantage of, however there is much advantage to take! <br />
<br />
Unfortunately, one cannot inherit from function pointers, so, in order to overload lambdas and regular functions together requires a little more work. First, we must define a base type that can handle both function pointers and function objects. It's job is to just forward the arguments to its function.<br />
<br />
<pre class="brush: cpp">template< class F > struct Forwarder : F {
constexpr Forwarder( const F& f ) : F(f) { }
};
template< class R, class ...X > struct Forwarder<R(*)(X...)> {
using type = R(*)(X...);
type f;
constexpr Forwarder( type f ) : f(f) { }
constexpr R operator () ( X... x ) {
return f(x...);
}
};
template< class R, class ...X >
struct Forwarder<R(X...)> : Forwarder<R(*)(X...)>
{
using type = R(*)(X...);
constexpr Forwarder( type f )
: Forwarder<R(*)(X...)>(f)
{
}
};
</pre>
<br />
Function pointers get two specializations because <i>decltype(f)=R(X)</i> and <i>decltype(&f)=R(*)(X)</i>. It makes the most sense to specialize for pointers, but only doing so would require we take the address of our functions when we call <i>overload</i>. <br />
<br />
Next, <i>Overloaded</i> inherits from two <i>Forwarder</i>s.<br />
<br />
<pre class="brush: cpp">template< class F, class G >
struct Overloaded : Forwarder<F>, Forwarder<G> {
constexpr Overloaded( const F& f, const G& g )
: Forwarder<F>(f), Forwarder<G>(g)
{
}
};
template< class F > F overload( F&& f ) {
return std::forward<F>(f);
}
template< class F, class G, class ...H,
class O1 = Overloaded<F,G> >
auto overload( const F& f, const G& g, const H& ...h )
-> decltype( overload(O1(f,g),h...) )
{
return overload( O1(f,g), h... );
}</pre>
<br />
Overloads can be of arity and domain (or argument type). The simplest example, for demonstration purposes, is a printing function.<br />
<br />
<pre class="brush: cpp">auto prnt = overload (
// Anything cout is already defined for.
[]( const auto& x )
-> decltype( void(std::cout << x) )
{ std::cout << x; },
// Any STL sequence.
[]<class Sequence>( const Sequence& s )
-> decltype( void(std::begin(s)) )
{
std::cout << "[ ";
for( const auto& x : s )
std::cout << x << ' ';
std::cout << ']';
},
// These are both sequences for which cout is defined.
// Specializing disambiguates this.
[]( const char* const s ) { std::cout << s; },
[]( const std::string& s ) { std::cout << s; }
);
// ...
prnt("abc"); // Prints abc.
prnt(std::vector<int>{1,2,3}); // Prints [ 1 2 3 ]. </pre>
<br />
Although defining all overloads in a single statement is an annoyance, they are grouped together, they don't require a <i>template<...></i> line, and the visual clutter is overall less than if <i>prnt</i> were defined as the equivalent (explicit) function object.<br />
<br />
Perhaps a function must be overloaded, but <i>decltype</i> or <i>std::enable_if </i>is too accepting and specializing for each type is redundant. For example, one might be annoyed by the last two string specializations of <i>prnt</i>. One solution is to define yet another overload type.<br />
<br />
<pre class="brush: cpp">template< class X, class F >
struct UnaryOverload {
F f;
UnaryOverload( const F& f ) : f(f) { }
using result_type = typename std::result_of< F(X) >::type;
result_type operator () ( X x ) const {
return f(x);
}
};
template< class ...X, class F >
auto unary_overload_set( const F& f )
-> decltype( overload(UnaryOverload<X,F>(f)...) )
{
return overload( UnaryOverload<X,F>(f)... );
}
auto prnt = overload (
// ...
unary_overload_set<const char* const,
const std::string&>(
[]( auto&& s ) { std::cout << s; }
)
);</pre>
<br />
One might write an overloading class to specialize for specific types, or a category of types, or more generally, a class might be written to do type conversion before calling the inner function, to prepare the output, or whatever one's needs may be. An overloading type might even select one of two functions based on an <i>enable_if</i>.<br />
<br />
<i> // Where pred is a templated type defining pred<X>::value. </i><br />
<i> auto h = enable_when<pred>( f, g );</i> <br />
<br />
The downsides of overloading function objects include that each overload must be defined all at once and none can be added. That isn't <i>too</i> bad since the point of lambdas is to be brief, but one <i>should</i> be mindful of extensibility when writing generic functions. (In other words, if an overload must be added, is it OK to modify the function object, or must the user be able to add overloads later?)<br />
<br />
<br />
<h4>
Recursion.</h4>
Without generic lambdas, recursion is possible.<br />
<br />
<pre class="brush: cpp">std::function<int(int)> fact = [&]( int x ) x * fact(x-1);</pre>
<br />
Or, with function pointers, which a lambda may implicitly convert to.<br />
<br />
<pre class="brush: cpp">// In global scope:
using IntToInt = int(*)(int);
IntToInt fact = []( auto x ) not x ? 1 : x * fact(x-1);</pre>
<br />
With generic lambdas, we could write it like this:<br />
<br />
<pre class="brush: cpp">auto fact1 = []( auto f, int n ) -> int
not n ? 1 : f(f,n-1) * n;
auto fact = []( int x ) -> int
fact1( fact1, x );</pre>
<br />
One might notice that the Fibonacci sequence could be implemented in a similar fashion. In researching recursive lambdas, I came across the <a href="http://en.wikipedia.org/wiki/Fixed-point_combinator#Y_combinator">fixed-point combinator</a>. Haskell has <a href="http://en.wikibooks.org/wiki/Haskell/Fix_and_recursion">fix</a>, which can be implemented like this:<br />
<br />
<pre class="brush: cpp">auto fix = []( auto f )
[=]( auto x ) -> decltype(x) f( f, x );
auto fact = fix (
[]( auto f, int n ) -> int
not n ? 1 : f(f,n-1) * n
);
auto fib = fix (
[]( int f, int n ) -> int
n == 0 ? 0 :
n == 1 ? 1 :
f(f,n-1) + f(f,n-2)
); </pre>
<br />
<i>fix</i> is a generalization of a certain type of recursion. For an idea of how one would implement <i>fix</i> without lambdas, see <a href="http://codereview.stackexchange.com/questions/3111/my-implementation-of-fixed-point-combinator-in-c1x-compiled-under-vistual-stu">this Stack Overflow post</a>.<br />
<br />
Making <i>prnt</i> above variadic requires a different kind of recursion.<br />
<br />
<pre class="brush: cpp">// Variadic void unary.
auto vvu_impl = overload (
[] (auto,auto) {},
[]<class X, class ...Y>( const auto& self, const auto& u,
X&& x, Y&& ...y )
{
u( std::forward<X>(x) );
self( self, u, std::forward<Y>(y)... );
}
);
auto vvu = []( const auto& u )
[&]<class ...x>( const x& ...x )
vvu_impl( vvu_impl, u, x... );
// Variadic print.
// vprint(x,y...) = prnt(x); prnt(y)...
auto vprint = vvu( prnt );
auto print_line = []<class ...X>( const X& ...x )
vprint( x..., '\n' );
print_line( "abc ", 123, std::vector<int>{1} ); // prints "abc 123 [1]\n" </pre>
<br />
We can generalize left-associativity as well.<br />
<br />
<pre class="brush: cpp">auto chainl_impl = overload (
[]( auto self, auto b, auto r ) { return r; },
[]<class ...Z>( auto self, auto b, auto x, auto y, Z ...z )
self( self, b, b(x,y), z... )
);
auto chainl = []( auto b )
[=]<class ...X>( const X& ...x )
chainl_impl( chainl_impl, b, x... );
auto sum = chainl(add);
auto three = sum(1,1,1);
// Variadic compose.
auto vcompose = chainl( compose );
auto inc = []( auto x ) ++x;
auto addThree = vcompose( inc, inc, inc );
</pre>
<br />
A good exercise might be to <b>(a)</b> write a variadic version of <i>fix</i> and <b>(b)</b> use that version to reimplement <i>chainl </i>and <i>vprint</i>.<br />
<br />
There are, of course, many types of recursion. Implementing recursive lambdas is more complicated than for regular functions, not by <i>too</i> much. <br />
<br />
<br />
<h4>
Conclusions.</h4>
Polymorphic (generic) lambdas are very powerful indeed, but it may take a while before GCC, MSVC, and others catch up, much yet before Faisal Vali's branch is merged back into Clang. Still they may have a strong impact on code written in C++ in the future. Some thought that templates relieved a sort of functional language in C++, and others thought the same of constexpr. Generic lambdas reveal another, but more flexible one.<br />
<br />
Lambdas cannot be marked constexpr. In terms of efficiency, I do not think this matters. They are implicitly inlined, so the compiler may still take advantage of any compile-time information it can gather. However, the result of a lambda expression could never be used in a template parameter, for example, which means they don't replace generic constexpr function objects.<br />
<br />
Also, explicitly specifying a type is <i>more</i> verbose because the rules are the same as for template member functions, so lambdas can't replace template functions that require explicit parameters.<br />
<br />
<i>auto f = []<class X>() { return x(); }; </i> <br />
f.operator()<int>(); // bad<br />
<br />
The proposal to add polymorphic lambdas to C++ is not finalized and a few things are up in the air. For example, can we elide <i>auto</i> and just write <i>[](x) f(x)</i>? Should we be allowed to elid the enclosing braces and <i>return</i>? Are the implemented parts of this proposal useful? Remember that the standardization process is open to the public and that we <i>can</i> make our voices heard about the features that impact us.<br />
<br />
Personally, I like the proposal as implemented currently. (Requiring <i>auto</i>, but eliding <i>{ return ... }</i>.) I would go a step further and say that <i>auto</i> should be extended to allow variadic parameters. (i.e. <i>[](auto ...x) f(x...)</i>) And named lambdas (section <b>2.4</b>) will be a very nice addition.<br />
<br />
What are <b>your</b> thoughts?<br />
<br />
<br />
<br />
<br />
<b>Source for this article</b>:<b> </b><a href="https://gist.github.com/4347130">https://gist.github.com/4347130</a> and <a href="https://gist.github.com/4381376">https://gist.github.com/4381376</a><br />
<b>Another (google translated) article on generic lambdas:</b><a href="http://translate.google.com/translate?hl=en&sl=ja&u=http://d.hatena.ne.jp/osyo-manga/20121225/1356368196&prev=/search%3Fq%3Dgeneric%2Blambda%2Bc%252B%252B%2Bclang%26start%3D10%26hl%3Den%26safe%3Doff%26client%3Dubuntu%26sa%3DN%26tbo%3Dd%26channel%3Dfs%26biw%3D1535%26bih%3D870&sa=X&ei=v-LbUOG5BOmQ2gXw7ICABg&ved=0CFsQ7gEwBjgK"> http://translate.google.com/translate?hl=en&sl=ja&u=http://d.hatena.ne.jp/osyo-manga/20121225/1356368196&prev=/search%3Fq%3Dgeneric%2Blambda%2Bc%252B%252B%2Bclang%26start%3D10%26hl%3Den%26safe%3Doff%26client%3Dubuntu%26sa%3DN%26tbo%3Dd%26channel%3Dfs%26biw%3D1535%26bih%3D870&sa=X&ei=v-LbUOG5BOmQ2gXw7ICABg&ved=0CFsQ7gEwBjgK</a><br />
<b>A long and thoughou article on the fixed-point combinator:</b> <a href="http://mvanier.livejournal.com/2897.html">http://mvanier.livejournal.com/2897.html</a>Splinter of Chaoshttp://www.blogger.com/profile/14715348728512729776noreply@blogger.com2tag:blogger.com,1999:blog-8638474063464489651.post-27349277692776223422012-12-14T17:12:00.000-08:002012-12-24T16:57:59.255-08:00Quick and Easy -- Manipulating C++ Containers Functionally.<b>Update:<i> </i></b>Added examples for <i>dup</i> and <i>foldMap</i>. <br />
<br />
Probably the most useful parts of the standard C++ library would be <a href="http://en.cppreference.com/w/cpp/container">container</a> and <a href="http://en.cppreference.com/w/cpp/algorithm">algorithms</a> support. Who has worked in C++ for any non-trivial amount of time without using <i>std::vector</i>, <i>list</i>, <i>map</i>, or any of the others? <i><algorithm></i>, on the other hand, is more something everyone <i>should</i> know. It solves many of the problems that C++ developers encounter on a daily basis.<br />
<br />
<i>"How do I test if there exists an element, x, where p(x) is true?" : std::any_of</i><br />
<i>"How do I copy each element, x, where p(x)?" : std::copy_if</i><br />
<i> "How do I removed each element, x, where p(x)?" : std::remove_if </i><br />
<i> "How do I move elements from one container to another?" : std::move, <algorithm> version.</i><br />
<i> "How do I find a subsequence?" : std::search</i><br />
<i> "How do I sort an array?" std::sort</i><br />
<i> "How do I find the sum of an array?" : std::accumulate</i><br />
<br />
Any programmer worth half their salt could write any of these functions in their sleep--they're basic--and the thing is that these algorithms <b>do</b> get written, over and over and over again. Either because one does not realize a specific <i><algorithm></i> function exists, or because one is thinking on a low level and unable to see the higher level abstractions.<br />
<br />
What I like most about the STL is that the only requirements for adapting any data type to a sequence are <b>(1)</b> define an iterator, and <b>(2)</b> define <i>begin()</i> and <i>end()</i>. After that, all (if not most) of <i><algorithm></i> becomes instantly usable with that type. (As well as the range-based for loop.) This makes it incredibly generic and useful.<br />
<br />
What I dislike is its verbosity. For example:<br />
<br />
<i>std::transform( xs.begin(), xs.end(), xs.begin(), f );</i><br />
<br />
Wouldn't this be more clear if written...<br />
<br />
<i> </i><br />
<i> xs = std::transform(xs,f);</i><br />
<br />
And this allows us to compose functions.<br />
<br />
<i>std::transform( xs.begin(), xs.end(), xs.begin(), f );</i><br />
<i>std::transform( xs.begin(), xs.end(), xs.begin(), g );</i><br />
<br />
<i> // vs</i><br />
<i> xs = std::transform( std::transform(xs,f), g );</i><br />
<br />
<i> // Or, using actual composition:</i><br />
<i> xs = std::transform( xs, </i><i><i>compose(g,f)</i> ); </i><br />
<br />
That's what this article will be about. An abstraction over the STL that lends itself to writing more terse, concise code without losing any clarity. This abstraction is less general, by design, because it works on entire containers, not iterators. I am <b>not</b> writing about a replacement for any <i><algorithm></i> functions, but an alternative inspired by functional programming.<br />
<br />
However, I do go over many <i><algorithm></i> functions, so this can also be thought of as a review.<br />
<br />
<br />
<h4>
Filtering, Taking, and Dropping: Collecting data.</h4>
I've always found the <a href="http://en.wikipedia.org/wiki/Erase-remove_idiom">erase-remove idiom</a> an unintuitive solution to such a common problem. I certainly would not have figured it out on my own without the help of the C++ community to point it out. Requiring containers to define a predicated <i>erase</i> wouldn't be generic, and <i><algorithm></i> knows only of iterators, not containers, so the standard library can't offer anything simpler. <i>filter</i> fills this gap by combining its knowledge of containers and iterators.<br />
<br />
<pre class="brush: cpp">template< class P, class S >
S filter( const P& p, S s ) {
using F = std::function< bool(typename S::value_type) >;
s.erase (
std::remove_if( std::begin(s), std::end(s), std::not1(F(p)) ),
std::end(s)
);
return s;
}
// ...
std::vector<int> naturals = {1,2,3,4,5,6,7,8,9/*,...*/};
auto evens = filter( [](int x){ return x%2==0; }, naturals );</pre>
<br />
<i>See also: <a href="http://en.cppreference.com/w/cpp/utility/functional/not1">std::not1</a>.</i> <br />
<br />
It does two things: First, it inverses the predicate meaning we can use positive logic (defining what we want to keep, rather than throw away) and second, it abstracts the erase-remove idiom.<br />
<br />
Using <i>filter</i>, we can write a rather quick-and-dirty qsort.<br />
<br />
<pre class="brush: cpp">// For each x in s, returns pair( p(x), not p(x) ).
template< class P, class S >
std::pair<S,S> partition( const P& p, S s ) {
using F = std::function< bool(typename S::value_type) >;
// There does exist std::partition_copy,
// however this function demonstrates a use of filter.
return std::make_pair (
filter( p, s ),
filter( std::not1(F(p)), s )
);
}
// Fake Quick-Sort: A non-in-place, qsort-inspired function.
template< class S >
S fake_qsort( S s ) {
using X = typename S::value_type;
if( s.size() < 2 )
return s;
X pivot = s.back();
s.pop_back();
S low, high;
std::tie(low,high) = partition (
[&]( const X& x ) { return x <= pivot; },
std::move(s)
);
low = fake_qsort( std::move(low) );
low.push_back( pivot );
// Append the sorted high to the sorted low.
high = fake_qsort( std::move(high) );
std::move( std::begin(high), std::end(high),
std::back_inserter(low) );
return low;
}
</pre>
<i>See also: <a href="http://en.cppreference.com/w/cpp/algorithm/partition">std::partition</a>, <a href="http://en.cppreference.com/w/cpp/algorithm/partition_copy">std::partition_copy</a>, and <a href="http://en.cppreference.com/w/cpp/algorithm/sort">std::sort</a>.</i><br />
<br />
<i>take</i> is a function that may seem entirely trivial, at least at first.<br />
<br />
<pre class="brush: cpp">template< class S, class _X = decltype( *std::begin(std::declval<S>()) ),
class X = typename std::decay<_X>::type >
std::vector<X> take( size_t n, const S& s ) {
std::vector<X> r;
std::copy_n( std::begin(s), n, std::back_inserter(r) );
return r;
}
template< class P, class S,
class _X = decltype( *std::begin(std::declval<S>()) ),
class X = typename std::decay<_X>::type >
std::vector<X> takeWhile( const P& p, const S& s ) {
std::vector<X> r;
std::copy( std::begin(s),
std::find_if( std::begin(s), std::end(s), p ),
std::back_inserter(r) );
return r;
}</pre>
<br />
It also breaks the convention of returning <i>s</i>'s type. There's a reason for that. Infinite lists. Consider this Haskell code:<br />
<br />
<i>take 10 [1..] == [1,2,3,4,5,6,7,8,9,10] </i><br />
<br />
<i>[1...]</i> is an infinite list, starting at one. Obviously, it doesn't actually exist in memory. <i>take</i> returns a finite list that does.<br />
<br />
The concept of iterators that represent infinite ranges in C++ isn't new, but neither is it common. <a href="http://en.cppreference.com/w/cpp/iterator/insert_iterator"><i>std::insert_iterator</i></a> could insert a theoretically infinite number of elements into a container. <a href="http://en.cppreference.com/w/cpp/iterator/istream_iterator"><i>std::istream_</i></a> and <a href="http://en.cppreference.com/w/cpp/iterator/ostream_iterator"><i>ostream_iterator</i></a> may read from or write to a file infinitely.<br />
<br />
We can create pseudo-containers to represent infinite ranges and plug them into <i>take</i>.<br />
<br />
<pre class="brush: cpp">template< class X > struct Reader {
using iterator = std::istream_iterator<X>;
iterator b;
iterator e;
Reader( iterator b = iterator( std::cin ),
iterator e = iterator() )
: b(b), e(e)
{
}
iterator begin() const { return b; }
iterator end() const { return e; }
};
// Probably ill-advised,
// but this is /one/ way of doing IO before main().
std::vector<int> three = take( 3, Reader<int>() );</pre>
<br />
Sometimes we want to take the contents of an entire container, so <i>dup</i> may be helpful.<br />
<br />
<pre class="brush: cpp">std::vector<X> dup( const S& s ) {
std::vector<X> r;
std::copy( std::begin(s),
std::end(s),
std::back_inserter(r) );
return r;
}
std::ifstream in( "in.txt" );
// Reader's constructor implicitly converts in to an iterator.
// Some may consider this bad style and require the constructor be "explicit".
std::vector<int> contents = dup( Reader<int>(in) );</pre>
<br />
The counterpart to <i>take</i> is <i>drop</i>, but it does not have <i>take</i>'s quirks.<br />
<br />
<pre class="brush: cpp">template< class S >
S drop( size_t n, const S& s ) {
return S (
std::next( std::begin(s), n ),
std::end(s)
);
}
// Predicate version
template< class P, class S >
S dropWhile( const P& p, const S& s ) {
return S (
std::find_if_not( std::begin(s), std::end(s), p ),
std::end(s)
);
}
Reader<int> r = drop( 2, Reader<int>() );</pre>
<br />
<i>drop</i> makes no promises about infinite lists, but unlike most container- or range-based algorithms, it can work <i>on</i> them. In the above example, two integers are read from <i>std::cin</i>, and their values lost.<br />
<br />
For another example of the use of pseudo-containers, consider this solution the the <a href="http://projecteuler.net/problem=1">first Euler Project problem</a> using <a href="http://www.boost.org/doc/libs/1_46_1/libs/range/doc/html/range/reference/ranges/irange.html">boost::irange</a>.<br />
<br />
<pre class="brush: cpp">#include <boost/range/irange.hpp>
void euler1() {
// multiples of...
auto three = boost::irange( 3, 1001, 3 );
auto five = boost::irange( 5, 1001, 5 );
// Ensure that the final sum has no duplicates.
std::vector<int> all;
std::set_union( std::begin(three), std::end(three),
std::begin(five), std::end(five),
std::back_inserter(all) );
std::cout << "The sum of every multiple of 3 or 5 bellow 1000 :"
<< std::accumulate( std::begin(all), std::end(all), 0 )
<< std::endl;
}</pre>
<br />
<br />
<h4>
Folding: Reducing a list from many to one. (<i>std::accumulate</i>)</h4>
Accumulating is the "imperative" description of folding. Historically, you'd call the variable you update with the results of each calculation the accumulator. To accumulate, then, is to iterate through a sequence, updating the accumulator with each iteration.<br />
<br />
Folding is another way to think of it. A fold is a transformation from a list of values to just one value. Haskell defines <i>foldl</i> and <i>foldr</i>, meaning "fold left" and "right". <br />
<br />
<pre class="brush: cpp">template< class F, class X, class S >
constexpr X foldl( F&& f, X x, const S& s ) {
return std::accumulate (
std::begin(s), std::end(s),
std::move(x), std::forward<F>(f)
);
}
int main() {
std::vector<int> v = { 5, 3, 2 };
std::cout << "((10 - 5) - 3) - 2) = " << foldl( std::minus<int>(), 10, v ) << std::endl;
}</pre>
<br />
<i>foldl</i> is really just another name for <i>accumulate</i>.<i> </i>The accumulation function (here, <i>std::minus</i>) expects the accumulator as the left argument and value to accumulate as its right. <i>foldr</i> is reversed: Not only does it iterate in reverse, but expects the accumulator in the right-hand argument.<br />
<br />
<pre class="brush: cpp">// A function wrapper that flips the argument order.
template< class F > struct Flip {
F f = F();
constexpr Flip( F f ) : f(std::move(f)) { }
template< class X, class Y >
constexpr auto operator () ( X&& x, Y&& y )
-> typename std::result_of< F(Y,X) >::type
{
return f( std::forward<Y>(y), std::forward<X>(x) );
}
};
template< class F, class X, class S >
constexpr X foldr( F&& f, X x, const S& s ) {
using It = decltype(std::begin(s));
using RIt = std::reverse_iterator<It>;
return std::accumulate (
// Just foldl in reverse.
RIt(std::end(s)), RIt(std::begin(s)),
std::move(x),
Flip<F>(std::forward<F>(f))
);
}
int main() {
std::vector<int> v = { 5, 3, 2 };
std::cout << "(2 - (3 - (5-10))) = " << foldr( std::minus<int>(), 10, v ) << std::endl;
}</pre>
<br />
Folding is great for <a href="http://en.wikipedia.org/wiki/Monoid">monoids</a>; types that have a binary operation with, often, the the signature "<i>X(const X&, const X&)</i>". <br />
<br />
<pre class="brush: cpp">std::vector<std::string> strs = { "st", "ri", "ng" };
// std::string associated with (+) is a monoid.
std::cout << "'st' + 'ri' + 'ng' = " <<
foldl( std::plus<std::string>(), std::string(), strs ) << std::endl;
using Func = std::function< int(int) >;
auto comp = []( Func f, Func g ) {
return [f,g]( int x ){ return f(g(x)); };
};
auto inc = []( int x ) { return x+1; };
auto id = []( int x ) { return x; };
std::vector<Func> incs = { inc, inc, inc };
// Functions can be monoids under composition.
std::cout << "(inc . inc . inc)(1) = " <<
foldl( comp, Func(id), incs )(1) << std::endl;</pre>
<br />
Functional programmers also like to build lists using <i>fold</i>. They build lists starting at the tail, so they typically prefer <i>foldr</i> to <i>foldl</i>. <i>std::forward_list</i> works like <i>[]</i> in Haskell and linked lists in other functional languages. This snippet simply copies the values from the <i>std::vector</i>, <i>v</i>.<br />
<br />
<pre class="brush: cpp">using List = std::forward_list<int>;
auto cons = []( List l, int x ) {
l.push_front( x );
return std::move(l);
};
List l = foldr( cons, List(), v );</pre>
<br />
Note: This one is not an example of a monoid. <br />
<br />
<br />
<h4>
Zip and Map: many to many. (<i>std::transform</i>)</h4>
To zip two sequences together by some function is the same as calling <i>std::transform</i>. Transform implies modifying each member by some function. Zip implies the same, but with the visual metaphor of combining two lists into one, starting at one end and working up.<br />
<br />
<pre class="brush: cpp">template< class F, template<class...>class S, class X, class Y,
class Res = typename std::result_of< F(X,Y) >::type >
S<Res> zip( F&& f, const S<X>& v, const S<Y>& w ) {
S<Res> r;
std::transform( std::begin(v), std::end(v),
std::begin(w),
std::back_inserter(r),
std::forward<F>(f) );
return r;
}
int main() {
std::vector<int> v = { 5, 3, 2 };
auto doubleV = zip( std::plus<int>(), v, v );
}</pre>
<br />
Note: The only way I have discovered to write <i>zip</i> variadically is with tuples. Since this article is not on tuples, refer to the definition of <i>transform</i> in "<a href="http://yapb-soc.blogspot.com/2012/12/zipping-and-mapping-tuples.html">Zipping and Mapping Tuples</a>".<br />
<br />
Note2: An in-place version of this function is possible, but showing both general and optimized versions of each function would be redundant, and the topic of optimization is worth discussing on its own. <br />
<br />
Mapping is similar to zipping--in fact the two-argument forms of <i>zip(f,xs)</i> and <i>map(f,xs)</i> should be equivalent. The three argument form, like <i>map(f,xs,ys)</i>, applies <i>f</i> to every combination of <i>x</i> and <i>y</i>.<br />
<br />
<i>map(f,{x,y},{a,b}) == { f(x,a), f(x,b), f(y,a), f(y,b) }</i><br />
<br />
If <i>xs</i> is size <i>N</i> and <i>ys</i> is of size <i>M</i>, then <i>map(f,xs,ys)</i> returns a sequence of size <i>N x M</i>.<br />
<br />
<pre class="brush: cpp">template< class F, template<class...>class S, class X,
class Res = typename std::result_of< F(X) >::type >
S<Res> map( const F& f, const S<X>& s ) {
S<Res> r;
std::transform( std::begin(s), std::end(s),
std::back_inserter(r),
std::forward<F>(f) );
return r;
}
template< class F, template<class...>class S, class X, class Y,
class Res = typename std::result_of< F(X,Y) >::type >
S<Res> map( const F& f, const S<X>& xs, const S<Y>& ys ) {
S<Res> r;
for( const X& x : xs )
for( const Y& y : ys )
r.emplace_back( f(x,y) );
return r;
}
int main() {
std::vector<int> v = { 5, 3, 2 };
std::vector<int> w = { 9, 8, 7 };
auto sums = map( std::plus<int>(), v, w );
}</pre>
<br />
<i>map</i> is a bread and butter function in functional programming.<br />
<br />
<i>// Convert a sequence from one type to another:</i><br />
<i> auto ints = map( toInt, floats ); </i><br />
<br />
<i>// In a game loop:</i><br />
<i> actors = map( update, actors ); </i><br />
<br />
<i> // A deck of cards (four suites with twelve values). </i><br />
<i> auto deck = map( make_card, suites, value ); </i><br />
<i><br />
</i> <i> // Making variants of the same thing from simpler data.</i><br />
<i> auto inits = { 1, 2, 3, 4 };</i><br />
<i> auto cs = map (</i><br />
<i> []( int i ) { return std::complex<float>(i,0.1); },</i><br />
<i> inits</i><br />
<i> );</i><br />
<br />
<i>// Checking for collisions:</i><br />
<i> ColisionObject collisions = map( make_collision, actors, actors );</i><br />
<br />
<i> // AI:</i><br />
<i> states = map( successor, actions, states ); </i><br />
<br />
One downfall of <i>map</i> is that it may create redundancies, which makes <i>filter</i> useful in conjunction.<br />
<br />
<i>states = filter (</i><br />
<i> state_is_valid,</i><br />
<i> map( successor, actions, states )</i><br />
<i> );</i><br />
<br />
While this may turn an algorithm from one-pass (update and add if valid) to two-pass (update all states, then filter), it also makes simpler algorithms that can be optimized more easily by the compiler at times. For example,<br />
<br />
<i>for( auto x : xs ) {</i><br />
<i>for( auto y : ys ) {</i><br />
<i> z = x * y;</i><br />
<i> if( pred(z) ) r.push_back(z);</i><br />
<i> }</i><br />
<i> }</i><br />
<br />
<i> // or:</i><br />
<i> auto r = filter( pred, map(std::multiplies<int>(),xs,ys) );</i><br />
<i> </i><br />
While only profiling can tell in any given instance, the second example may be faster under some circumstances. The compiler may be able to vectorize the call to <i>map</i>, but have difficulties applying the same optimization to the first because it cannot evaluate both the multiplication and predicate in one vectorized step.<br />
<br />
Sometimes, the goal is to calculate something given the data, rather than <i>map</i> it. Naively, one might write something like<br />
<br />
<i>auto r = fold( f, map(g,xs) );</i><br />
<br />
But isn't creating the new container inefficient? What if an in-place version of <i>map</i> were implemented, wouldn't transforming <i>xs</i> before folding still be inefficient? Thus, <i>foldMap</i> is useful.<br />
<i> </i> <i> </i><br />
<pre class="brush: cpp">template< class Fold, class Map, class X, class S >
X foldMap( const Fold& f, const Map& m, X x, const S& s ) {
for( auto& y : s )
x = f( std::move(x), m(y) );
return x;
}
#include <cctype>
int main() {
const char* names[] = { "jonh", "mary", "cary" };
auto appendNames = []( std::string x, std::string y ) {
return x + " " + y;
};
auto capitolizeName = []( std::string name ) {
name[0] = std::toupper( name[0] );
return name;
};
std::cout << "Names : "
<< foldMap (
appendNames,
capitolizeName,
std::string(),
names
) << std::endl;
}</pre>
<br />
<br />
<br />
<h4>
Conclusions.</h4>
Haskell's <a href="http://www.haskell.org/ghc/docs/latest/html/libraries/base/Data-List.html">Data.List</a> is actually a lot like <i><algorithm></i>, though on a higher level of abstraction. There are some things that can only be done with iterators, but many that can also only be done with whole containers. Data.List gives some good inspiration for helpful algorithms, even in C++.<br />
<br />
But unlike in C++, Haskell uses simple linked lists by default and all of Data.List's function work only on linked lists. This gives both Haskell and functional programming a bad name when people compare Haskell code using linked lists to C++ code using <i>std::vector</i>. (See <a href="http://www.baptiste-wicht.com/2012/12/cpp-benchmark-vector-list-deque/">"C++ Benchmark -- std::vector vs. std::list vs. std::deque"</a>) When libraries are written to optimize inefficiencies in the linked list, like <a href="http://hackage.haskell.org/packages/archive/text/0.5/doc/html/Data-Text.html">Data.Text</a>, they re-implement Data.List's interface and often achieve equivalent efficiency to well-optimized C, but not without plenty of redundancy.<br />
<br />
In C++, we can write one static interface that is both generic and efficient. Writing functional code does not mean writing slow code. The mathematical nature of these operations can even help the compiler optimize. The high-level interface of Data.List fits snugly atop of the low-level interface of iterators.<br />
<br />
<br />
<b>Source for this article:</b> <a href="https://gist.github.com/4290166">https://gist.github.com/4290166</a>Splinter of Chaoshttp://www.blogger.com/profile/14715348728512729776noreply@blogger.com11tag:blogger.com,1999:blog-8638474063464489651.post-49508187480606486312012-12-12T08:00:00.000-08:002012-12-16T05:36:29.832-08:00Zipping and Mapping tuples.<a href="http://yapb-soc.blogspot.com/2012/12/fun-with-tuples.html">Previously</a>, I discussed some basic things that can be done with tuples. I showed how a tuple can be applied to a function, however I did not show how member-wise transformations could be done.<br />
<br />
The <a href="https://gist.github.com/4268029">code of this article</a> builds on the <a href="https://gist.github.com/4256092">code of the prior</a>. <br />
<br />
<br />
<h4>
Zipping. </h4>
If we have several tuples, what if we want to apply a function to the nth element of each one?<br />
<br />
<pre class="brush: cpp">template< template<size_t> class Fi = Get, size_t i,
class F, class ...T >
constexpr auto zipRow( const F& f, T&& ...t )
-> decltype( f(Fi<i>()(std::forward<T>(t))...) )
{
return f( Fi<i>()( std::forward<T>(t) )... );
}
</pre>
<br />
Not hard at all! It basically squishes that row (thinking of <i>t</i> as a column and <i>ti...</i> as a row), using <i>f</i>, into one value. Now, let's say we want to zip together <i>t...</i> into one tuple.<br />
<br />
<pre class="brush: cpp">template< template<size_t> class Fi = Get, size_t ...i,
class Ret, class F, class ...T >
constexpr auto zipIndexList( IndexList<i...>,
const Ret& r, const F& f, T&& ...t )
-> decltype( r(zipRow<Fi,i>(f,std::forward<T>(t)...)...) )
{
return r( zipRow< Fi, i >( f, std::forward<T>(t)... )... );
}
template< template<size_t> class Fi = Get,
class Ret, class F, class T, class ...U,
class _T = typename std::decay<T>::type,
class IL = typename IListFrom<_T>::type >
constexpr auto zipTupleTo( const Ret& r, const F& f, T&& t, U&& ...u )
-> decltype( zipIndexList<Fi>(IL(),r,f,std::forward<T>(t),std::forward<U>(u)...) )
{
return zipIndexList<Fi>( IL(), r, f, std::forward<T>(t), std::forward<U>(u)... );
}
int main() {
auto zipped = zipTupleTo( tuple, std::plus<int>(), tuple(1,10),
tuple(2,20) );
std::cout << " 1 + 2 = " << std::get<0>(zipped) << std::endl;
std::cout << "10 + 20 = " << std::get<1>(zipped) << std::endl;
}</pre>
<br />
In <i>zipIndexList</i>, <i>r</i> represents the function defining how the output is returned. <a href="https://gist.github.com/4268029#file-zip-map-tuple-cpp-L95"><i>tuple</i> (gist)</a>, from the previous article, is just a function object form of <i>std::make_tuple</i> that can be passed to higher order functions. By supplying it as our <i>r</i>, we're saying "just make it a tuple again."<br />
<br />
Since most often, we want to zip back into a tuple, it makes sense to define <i>zipTuple</i> like so:<br />
<br />
<pre class="brush: cpp">template< template<size_t> class Fi = Get,
class F, class ...T >
constexpr auto zipTuple( const F& f, T&& ...t )
-> decltype( zipTupleTo<Fi>(tuple,f,std::forward<T>(t)...) )
{
return zipTupleTo<Fi>( tuple, f, std::forward<T>(t)... );
}</pre>
<br />
<i>zipTuple</i> is to tuples what <i>std::transform</i> is to sequences. The drawback of <i>std::transform</i> is that it only allows for a unary transformation or binary. Let's write a version that accepts any number of arguments.<br />
<br />
<pre class="brush: cpp">// We require these polymorphic function objects.
constexpr struct Inc {
template< class X >
constexpr X operator () ( X x ) { return ++x; }
} inc{};
constexpr struct Eq {
template< class X >
constexpr bool operator () ( const X& a, const X& b )
{ return a == b; }
} eq{};
struct Or {
template< class X >
constexpr bool operator () ( const X& a, const X& b )
{ return a || b; }
};
// Wrapper to dereference arguments before applying.
// indirect : (a -> b) -> (a* -> b)
template< class F > struct Indirect {
F f = F();
constexpr Indirect( F f ) : f(std::move(f)) { }
template< class ...It >
constexpr auto operator () ( It ...it )
-> decltype( f(*it...) )
{
return f( *it... );
}
};
template< class F, class I = Indirect<F> >
constexpr I indirect( F f ) {
return I( std::move(f) );
}
#include <vector>
#include <algorithm>
template< class F, class ...X,
class Result = typename std::result_of<F(X...)>::type,
class Ret = std::vector<Result> >
Ret transform( const F& f, const std::vector<X>& ...vs )
{
Ret r;
const auto ends = tuple( vs.end()... );
// Iterate through each vector in parallel.
for( auto its = tuple( vs.begin()... );
// This unrolls to: not (it0==end0 || it1==end1 || ...)
not foldl( Or(), zipTuple(eq,its,ends) );
// Increment each iterator.
its = zipTuple( inc, its ) )
{
r.emplace_back (
applyTuple( indirect(f), its )
);
}
return r;
}
int main() {
std::vector<int> v = {1,10,100},
w = {2,20,200},
x = {3,30,300};
auto vw = transform (
[](int x, int y, int z){ return x+y+z; },
v, w, x
);
std::cout << " 1 + 2 + 3 = " << vw[0] << std::endl;
std::cout << " 10 + 20 + 30 = " << vw[1] << std::endl;
std::cout << "100 + 200 + 300 = " << vw[2] << std::endl;
}
</pre>
<br />
<i>Note: <a href="https://gist.github.com/4268029#file-zip-map-tuple-cpp-L195">foldl (gist)</a>.</i><br />
<br />
<h4>
Mapping.</h4>
Suppose we want to know the results of adding every combination of {1,2,3} with {9,8,7}. We could write a function that cross-applied every variable from each tuple, but slightly more generally, we can start by taking the <a href="http://en.wikipedia.org/wiki/Cartesian_product">Cartesian product</a>.<br />
<br />
<pre class="brush: cpp">// product : {x,y} x {a,b} -> {{x,a},{x,b},{y,a},{y,b}}
constexpr struct Product {
// {...xi...} x {...aj...} -> {xi,aj}
template< size_t i, size_t j, class T, class U >
static constexpr auto zip( const T& t, const U& u )
-> decltype( tuple(std::get<i>(t),std::get<j>(u)) )
{
return tuple( std::get<i>(t), std::get<j>(u) );
}
// {...xi...} x {a0,a1,a2...} -> { {xi,a0}, {xi,a1}, ... }
template< size_t i, size_t ...j, class T, class U >
static constexpr auto withI( IndexList<j...>, const T& t, const U& u )
-> decltype( tuple(zip<i,j>(t,u)...) )
{
return tuple( zip<i,j>(t,u)... );
}
// {x...} x {a...} -> { {x,a}... }
template< size_t ...i, size_t ...j, class T, class U >
static constexpr auto withIndexes( IndexList<i...>, IndexList<j...> js,
const T& t, const U& u )
-> decltype( std::tuple_cat(withI<i>(js,t,u)...) )
{
return std::tuple_cat( withI<i>(js,t,u)... );
}
template< class T, class U,
class IL = typename IListFrom<T>::type,
class IL2 = typename IListFrom<U>::type >
constexpr auto operator () ( const T& t, const U& u )
-> decltype( withIndexes(IL(),IL2(),t,u) )
{
return withIndexes( IL(), IL2(), t, u );
}
} product{};
</pre>
We can now define a map operation to apply the product. <br />
<br />
<pre class="brush: cpp">template< class F > struct ApplyF {
F f = F();
constexpr ApplyF( F f ) : f(std::move(f)) { }
template< class T >
constexpr auto operator () ( T&& t )
-> decltype( applyTuple(f,std::forward<T>(t)) )
{
return applyTuple( f, std::forward<T>(t) );
}
};
template< class F >
constexpr ApplyF<F> applyF( F f ) {
return ApplyF<F>(std::move(f));
}
constexpr struct MapTuple {
template< class F, class T, class U >
constexpr auto operator () ( const F& f, const T& t, const U& u )
-> decltype( zipTuple(applyF(f),product(t,u)) )
{
return zipTuple( applyF(f), product(t,u) );
}
} mapTuple{};
int main() {
auto sums = mapTuple( std::plus<int>(), tuple(1,2,3), tuple(7,8,9) );
std::cout << "map (+) (1,2,3) (7,8,9) = ";
forEach( printItem, sums );
std::cout << std::endl;
}</pre>
<br />
This prints out:<br />
<br />
<pre class="brush: cpp">map (+) (1,2,3) (7,8,9) = 8 9 10 9 10 11 10 11 12 </pre>
<br />
Zipping applies elements across from each other. Mapping applies everything to everything. (<i>Note: a unary definition of map would be equivalent to a unary definition of zip.</i>) <br />
<br />
<br />
<h4>
Tuples as function environments.</h4>
This might seem a little off topic, but Haskell has this neat function, <i>id</i>. It works like this:<br />
<br />
<i>id x = x</i><br />
<br />
Simple, right?<i> </i><br />
<br />
<i>(id f) x y = f x y = id f x y</i><br />
<br />
<i> id</i> has this neat property that if applied multiple arguments, it applies the tail arguments to the first. This is an artifact of Haskell's <a href="http://en.wikipedia.org/wiki/Currying">curried notation</a>, but we can emulate this behaviour:<br />
<br />
<pre class="brush: cpp">constexpr struct Id {
template< class X >
constexpr X operator () ( X&& x ) {
return std::forward<X>(x);
}
template< class F, class X, class ...Y >
constexpr auto operator () ( const F& f, X&& x, Y&& ...y )
-> typename std::result_of< F(X,Y...) >::type
{
return f( std::forward<X>(x), std::forward<Y>(y)... );
}
} id{};
</pre>
And now tuples take on a new role: Contained function environments. Consider:<br />
<br />
<i>applyTuple( id, tuple(std::plus<int>(),1,2) );</i><br />
<br />
What does this output? How about<br />
<i> </i><br />
<i> mapTuple( id, tuple(inc,dec), tuple(5,9) );</i><br />
<br />
<i> auto pm = </i><i><i>tuple(std::plus<int>(),std::minus<int>());</i> </i><br />
<i> zipTuple( id, pm, tuple(10,5), tuple(10,5) );</i><br />
<br />
<i> </i>Or:<br />
<br />
<i>mapTuple( id, pm, tuple(1,2), tuple(3,4);</i><br />
<br />
I leave implementing the three-tuple version of <i>mapTuple </i>as an exercise, but here's a hint: <i>cross( cross({f},{x}), {y}) = {{{f,x},{y}}}</i>, but you need to take it from <i>that</i> to <i>{{f,x,y}}</i>. (Another good exercise might be to write <i>zipTuple</i> in terms of <a href="http://en.wikipedia.org/wiki/Transpose">transposition (wiki)</a>.)<br />
<br />
<br />
<h4>
Conclusions.</h4>
This flushes out some basic applications of tuples to functions. <i>applyTuple</i> unpacks a tuple and applies it to a function. <i>foldl</i> and <i>foldr</i> let one apply binary functions to nary tuples, or even <a href="http://en.wikipedia.org/wiki/Singleton_%28mathematics%29">singletons</a> (maths concept, not design pattern). <i>zipTuple</i> transforms multiples tuples by a functions, member-wise. <i>mapTuple</i> performs a function for every combination of arguments.<br />
<br />
Tuples have unusual mathematical properties compared to other data structures due to the profundity of what they generalize. They can help us shorthand functions to operate in parallel (<i>zip</i>), be passed around as partial or complete function environments, control variadic template parameters, and much, much more that I have either not discussed or yet realized.<br />
<br />
One use I haven't discussed, for example, is as a <a href="http://en.wikipedia.org/wiki/Tuple#Relational_model">relation</a>, but for an example of this use of tuples, look no further than <i><a href="http://en.cppreference.com/w/cpp/container/map">std::map</a>.</i><br />
<br />
I hope this post has been interesting. Happy coding!<br />
<br />
<br />
<b>Source for this article:</b> <a href="https://gist.github.com/4268029">https://gist.github.com/4268029</a>Splinter of Chaoshttp://www.blogger.com/profile/14715348728512729776noreply@blogger.com3tag:blogger.com,1999:blog-8638474063464489651.post-60020701822836874842012-12-10T21:37:00.000-08:002012-12-16T05:37:22.753-08:00Fun with tuples.<i>std::tuple</i> is an odd-but-fun part of the new standard. A lot can be done with them. The members of a tuple can be applied to functions, both in groups and individually. The tuple itself can be treated as a function environment. They can be reorganized, appended, and truncated. The list goes on.<br />
<br />
The thing is, what can tuples be used for? Any POD struct or class can be a tuple instead, although this may or may not be desirable. Still, we can write generic algorithms with tuples, whereas we cannot with structs. If we wrote a function that printed any tuple, then any POD we turned into a tuple would suddenly become printable. <br />
<br />
<br />
<h4>
Indexing.</h4>
Tuples are indexed, which makes accessing them odd.<br />
<br />
<pre class="brush: cpp">// A normal struct
struct Vec { int x, y; };
Vec a = { 1, 2 };
a.x = 2; // Normal access.
std::tuple<int,int> b(1,2);
std::get<0>(b) = 2; // Weird access. </pre>
<br />
One might think "I'll just write an accessor function!", and that certainly would work. It becomes easier if <i>std::get</i> is made into a function object.<br />
<br />
<pre class="brush: cpp">// std::tuple_element<i,T> does not perfect forward.
template< size_t i, class T >
using Elem = decltype( std::get<i>(std::declval<T>()) );
template< size_t i > struct Get {
template< class T >
constexpr auto operator () ( T&& t )
-> Elem< i, T >
{
return std::get<i>( std::forward<T>(t) );
}
};
constexpr auto getx = Get<0>();
constexpr auto gety = Get<1>();
getx(b) = 2; // Less weird.</pre>
<br />
I define <i>Elem</i> because using <i>std::tuple_element</i> returns what the tuple actually holds. <i>std::get<0>(b)</i> would return an <i>int&</i>, but <i>tuple_element<0,decltype(b)>::type</i> would be <i>int</i>.<br />
<br />
One might find it useful to index the tuple backwards, so let's define a function, <i>rget</i>.<br />
<br />
<pre class="brush: cpp"> template< size_t i, class T,
class _T = typename std::decay<T>::type,
size_t N = std::tuple_size<_T>::value - 1, // Highest index
size_t j = N - i >
constexpr auto rget( T&& t )
-> Elem< j, T >
{
return std::get<j>( std::forward<T>(t) );
}</pre>
<br />
Now we can also define functions to get the first and last elements of any tuple.<br />
<br />
<pre class="brush: cpp">constexpr auto head = Get<0>();
constexpr auto last = RGet<0>(); // RGet is defined similarly to Get.
int main() {
constexpr auto t = std::make_tuple( 1, 'a', "str" );
std::cout << "head = " << head(t) << std::endl; // prints 1
std::cout << "last = " << last(t) << std::endl; // prints str
}</pre>
<br />
Just for fun, let's also write a function that, if the index is too high, wraps around to the begining.<br />
<br />
<pre class="brush: cpp">template< size_t i, class T,
class _T = typename std::decay<T>::type,
size_t N = std::tuple_size<_T>::value,
size_t j = i % N >
constexpr auto mod_get( T&& t )
-> Elem< j, T >
{
return std::get<j>( std::forward<T>(t) );
} </pre>
<br />
Now, let's say we want to call a function for every member of a tuple. We need a way of indexing it and applying some function for each index. It starts with the definition of a type to "hold" a list of indexes.<br />
<br />
<pre class="brush: cpp">template< size_t ...i > struct IndexList {};</pre>
<br />
Now, if given a tuple of size 3, we want an <i>IndexList<0,1,2></i> to represent each index. There are many solutions for how to do this, but they all have in common being obtuse or difficult to understand. The following solution is designed first and foremost to be obvious and intuitive.<br />
<br />
<pre class="brush: cpp">template< size_t ... > struct EnumBuilder;
// Increment cur until cur == end.
template< size_t end, size_t cur, size_t ...i >
struct EnumBuilder< end, cur, i... >
// Recurse, adding cur to i...
: EnumBuilder< end, cur+1, i..., cur >
{
};
// cur == end; the list has been built.
template< size_t end, size_t ...i >
struct EnumBuilder< end, end, i... > {
using type = IndexList< i... >;
};
template< size_t b, size_t e >
struct Enumerate {
using type = typename EnumBuilder< e, b >::type;
};
template< class > struct IListFrom;
template< class ...X >
struct IListFrom< std::tuple<X...> > {
static constexpr size_t N = sizeof ...(X);
using type = typename Enumerate< 0, N >::type;
};</pre>
<br />
<br />
Now, a function that applies each index, and one that prints a tuple's elements for testing.<br />
<br />
<pre class="brush: cpp">template< size_t i, size_t ...j, class F, class T >
void forEachIndex( IndexList<i,j...>, const F& f, const T& t ) {
f( std::get<i>(t) );
// Recurs, removing the first index.
forEachIndex( IndexList<j...>(), f, t );
}
template< class F, class T >
void forEachIndex( IndexList<>, const F& f, const T& t ) {
// No more indexes.
}
template< class F, class T >
void forEach( const F& f, const T& t ) {
constexpr size_t N = std::tuple_size<T>::value;
using IL = typename Enumerate<0,N>::type;
forEachIndex( IL(), f, t );
}
constexpr struct PrintItem {
template< class X >
void operator () ( const X& x ) const {
std::cout << x << ' ';
}
} printItem{};
int main() {
constexpr auto t = std::make_tuple( 1, "and", 2 );
std::cout << "t = ";
forEach( printItem, t ); // Prints "1 and 2"
std::cout << std::endl;
}</pre>
<br />
<h4>
Applying a tuple to a function.</h4>
This is probably one of the most pondered questions about tuples. "How do I apply one to a function?" This is easy since we already have <i>IndexList</i> defined.<br />
<br />
<pre class="brush: cpp">template< size_t ...i, class F, class T >
constexpr auto applyIndexList( IndexList<i...>, F f, T&& t )
-> typename std::result_of< F( Elem<i,T>... ) >::type
{
return f( std::get<i>(std::forward<T>(t))... );
}
template< class F, class T,
class _T = typename std::decay<T>::type;
class IL = typename IListFrom<_T>::type >
constexpr auto applyTuple( const F& f, T&& t )
-> decltype( applyIndexList(IL(),f,std::forward<T>(t)) )
{
return applyIndexList( IL(), f, std::forward<T>(t) );
}
// Functional programmers may recognize this as cons.
constexpr struct PushFront {
template< class ...X, class Y >
constexpr auto operator () ( std::tuple<X...> t, Y y )
-> std::tuple< Y, X... >
{
return std::tuple_cat( tuple(std::move(y)), std::move(t) );
}
} pushFront{};
// Chain Left.
constexpr struct ChainL {
template< class F, class X >
constexpr X operator () ( const F&, X x ) {
return x;
}
template< class F, class X, class Y, class ...Z >
constexpr auto operator () ( const F& b, const X& x, const Y& y, const Z& ...z)
-> decltype( (*this)(b, b(x,y), z... ) )
{
return (*this)(b, b(x,y), z... );
}
} chainl{};
// Fold Left.
constexpr struct FoldL {
// Given f and {x,y,z}, returns f( f(x,y), z ).
template< class F, class T >
constexpr auto operator () ( const F& f, const T& t )
-> decltype( applyTuple(chainl,pushFront(t,f)) )
{
return applyTuple( chainl, pushFront(t,f) );
}
} foldl{};
auto ten = foldl( std::plus<int>(), std::make_tuple(1,2,3,4) );
</pre>
<br />
We can call <i>applyIndexList</i> with different index lists to get interesting results.<br />
<br />
<pre class="brush: cpp">// Because std::make_tuple can't be passed
// to higher order functions.
constexpr struct MakeTuple {
template< class ...X >
constexpr std::tuple<X...> operator () ( X ...x ) {
return std::tuple<X...>( std::move(x)... );
}
} tuple{}; // function tuple that construct std::tuples.
// Returns the initial elements. (All but the last.)
// init( {1,2,3} ) = {1,2}
template< class T,
size_t N = std::tuple_size<T>::value,
class IL = typename Enumerate< 0, N-1 >::type >
constexpr auto init( const T& t )
-> decltype( applyIndexList(IL(),tuple,t) )
{
// Construct a new tuple from the initial indexes.
return applyIndexList( IL(), tuple, t );
}
// Returns a new tuple with every value from t except the first.
// tail( {1,2,3} ) = {2,3}
template< class T,
size_t N = std::tuple_size<T>::value,
class IL = typename Enumerate< 1, N >::type >
constexpr auto tail( const T& t )
-> decltype( applyIndexList(IL(),tuple,t) )
{
return applyIndexList( IL(), tuple, t );
} </pre>
<br />
Remember <i>Get</i> and <i>RGet</i> from above? They're templated function objects based on an index. We can write a more generic <i>applyIndexList</i> that allows specifying such a function and without losing the default behaviour.<br />
<br />
<pre class="brush: cpp">template< template<size_t> class Fi = Get, size_t ...i, class F, class T >
constexpr auto applyIndexList( IndexList<i...>, const F& f, T&& t )
-> typename std::result_of< F(
typename std::result_of< Fi<i>(T) >::type...
) >::type
{
return f( Fi<i>()(std::forward<T>(t))... );
}
template< template<size_t> class Fi = Get, class F, class T,
class _T = typename std::decay<T>::type,
class IL = typename IListFrom<_T>::type >
constexpr auto applyTuple( const F& f, T&& t )
-> decltype( applyIndexList<Fi>(IL(),f,std::forward<T>(t)) )
{
return applyIndexList<Fi>( IL(), f, std::forward<T>(t) );
}
// Reconstruct t in reverse.
template< class T >
constexpr auto reverse( const T& t )
-> decltype( applyTuple<RGet>(tuple,t) )
{
return applyTuple< RGet >( tuple, t );
}
// Fold Right.
constexpr struct FoldR {
// Given f and {x,y,z}, returns f( f(z,y), x ).
template< class F, class T >
constexpr auto operator () ( const F& f, const T& t )
-> decltype( foldl(f,reverse(t)) )
{
return foldl( f, reverse(t) );
}
} foldr{};</pre>
<br />
<br />
This leaves us with two ways of transforming tuples: modifying the index list and defining an alternative <i>get</i> function. <i>foldr</i> and <i>foldl</i> give us a way to fold a tuple into one value.<br />
<br />
<br />
<h4>
Tuples <i>and</i> functions.</h4>
Perhaps we have a function that takes a tuple, but its arguments are not tuples. Haskell defined a function, <i>curry</i>, to transform the function from a pair-taking one to a two-argument function. Haskell does not have the same expressiveness with variadic types, so they can't write this more general C++ version.<br />
<br />
<pre class="brush: cpp">// curry : ( (a,b...) -> c ) x a x b... -> c
template< class F, class ...X >
constexpr auto curry( const F& f, X&& ...x )
-> decltype( f( std::forward_as_tuple(std::forward<X>(x)...) ) )
{
return f( std::forward_as_tuple(std::forward<X>(x)...) );
}
// Pair Sum.
// psum : (int,int) -> int
unsigned int psum( std::tuple<int,int> p ) {
return head(p) + last(p);
}
auto five = curry( psum, 3, 2 );
</pre>
Haskell also defines <i>uncurry</i>, which is the same as <i>applyTuple</i>. The most important distinction between Haskell's <i>curry</i> and <i>uncurry</i> and this is that Haskell's <i>curry</i> is a unary higher order function, whereas this is binary. However, the two are the same if one considers the unary version a <a href="http://en.wikipedia.org/wiki/Partial_application">partial application</a> of the binary one.<br />
<br />
Tuples can be used as a sort of partial application on their own. One might store some values in a tuple, and add more arguments later to be applied to some function. For a somewhat contrived example, consider the following:<br />
<br />
<pre class="brush: cpp">constexpr struct PushBack {
template< class ...X, class Y >
constexpr auto operator () ( std::tuple<X...> t, Y y )
-> std::tuple< X..., Y >
{
return std::tuple_cat( std::move(t), tuple(std::move(y)) );
}
} pushBack{};
#include <cmath>
// Quadratic root.
constexpr struct QRoot {
using result = std::tuple<float,float>;
result operator () ( float a, float b, float c ) const {
float root = std::sqrt( b*b - 4*a*c );
float den = 2 * a;
return result( (-b+root)/den, (-b-root)/den );
}
} qroot{};
std::ostream& operator << ( std::ostream& os, const QRoot::result r ) {
return os << std::get<0>(r) << " or " << std::get<1>(r);
}
int main() {
auto ab = std::make_tuple( 1, 3 );
auto qroot_ab = [&] ( float c ) {
return applyTuple( qroot, pushBack(ab,c) );
};
std::cout << "qroot(1,3,-4) = " << qroot_ab(-4) << std::endl;
std::cout << "qroot(1,3,-5) = " << qroot_ab(-5) << std::endl;
auto bc = std::make_tuple( 3, -4 );
auto qroot_bc = [&] ( float a ) {
return applyTuple( qroot, pushFront(bc,a) );
};
std::cout << "qroot(1,3,-4) = " << qroot_bc(1) << std::endl;
std::cout << "qroot(1,3,-5) = " << qroot_bc(2) << std::endl;
}</pre>
<br />
One persuasive use-case of tuples is being able to freely manipulate variadic parameters.<br />
<br />
<pre class="brush: cpp">template< class ...X >
constexpr auto third_arg( X&& ...x )
-> Elem< 2, std::tuple<X...> >
{
return std::get<2>( std::forward_as_tuple(std::forward<X>(x)...) );
}</pre>
<br />
Variadic parameters can be forwarded into a tuple, meaning anything we can do with tuples, we can do with variadic parameters. Arguments can be compacted into tuples, modified, reordered, and re-expanded into functions. One annoyance with the ... is that it eats up everything to the right. The following is ill-formed.<br />
<br />
<pre class="brush: cpp">template< class ...X, class F >
constexpr auto backwards( X&& ...x, const F& f )
-> typename std::result_of< F(X...) >::type
{
return f( std::forward<X>(x)... )
}</pre>
<br />
I leave the solution as an exercise for the reader.<br />
<br />
<br />
<h4>
Conclusions.</h4>
This article is introductory, at best. I must admit I have much less practice with them than other C++11 features, but this seems to be true of GCC, too! Attempting to compile the following will cause an internal error since at least 4.7.<br />
<br />
<pre class="brush: cpp">#include <tuple>
template< class X > struct Hold {
X x;
constexpr Hold( X x ) : x(std::move(x)) { }
};
constexpr auto a = Hold<int>( 1 ); //ok
auto b = Hold<std::tuple<char>>( std::tuple<char>('c') ); // not
constexpr auto c = Hold<std::tuple<char>>( std::tuple<char>('c') ); // not
int main() {} </pre>
<br />
I believe it's quite possible that whole programs could be written with tuples and basic data types, whether or not it should be preferred. We could look at them as a general utility class, but I think it would be appropriate to see them as lightweight, generalized, composable, structs. I plan on writing a follow-up to cover applying multiple tuples, transforming tuples by functions, and a few other things. If anyone has any interesting use-cases or neat tricks for tuples, I'd love to hear about it in the comments. <b> </b><br />
<br />
The next article covers element-wise application, applying multiple tuples, and more: "<a href="http://yapb-soc.blogspot.com/2012/12/zipping-and-mapping-tuples.html">Zipping and Mapping Tuples</a>".<br />
<br />
<br />
<b>Source code: </b><a href="https://gist.github.com/4256092">https://gist.github.com/4256092</a> <b> </b><br />
<b>Tuple reference:</b> <a href="http://en.cppreference.com/w/cpp/utility/tuple">http://en.cppreference.com/w/cpp/utility/tuple</a>Splinter of Chaoshttp://www.blogger.com/profile/14715348728512729776noreply@blogger.com5tag:blogger.com,1999:blog-8638474063464489651.post-27360131230627805192012-12-09T08:55:00.000-08:002012-12-14T18:00:49.070-08:00GCC 4.8 Has Automatic Return Type Deduction. <i>Note: GCC 4.8 is still in development; this article is based on Ubuntu's snapshot package of 4.8. I do not know about availability on other platforms. I say "has" because it does work and code can be written using it right now, even if it's in testing. </i><br />
<br />
<i><b>Update:</b> It turns out this feature has been implemented to test <a href="http://www.open-std.org/JTC1/SC22/WG21/docs/papers/2012/n3386.html">n3386</a>. You can read the discussion and even see the patch on the mailing list: <a href="http://gcc.gnu.org/ml/gcc-patches/2012-03/msg01599.html">http://gcc.gnu.org/ml/gcc-patches/2012-03/msg01599.html</a></i><br />
<br />
Are your two favourite C++11 features <i>decltype</i> and <i>declval</i>? I have mixed feelings. On one hand, it lets me write code like this<br />
<br />
<pre class="brush: cpp">template< class X, class Y >
constexpr auto add( X&& x, Y&& y )
-> decltype( std::declval<X>() + std::declval<Y>() )
{
return std::forward<X>(x) + std::forward<Y>(y);
}</pre>
<br />
and know that it will work on any type and do the optimal thing if <i>x</i> or <i>y</i> should be moved or copied (like if <i>X=std::string</i>). On the other hand, it's tedious. "forward" and "declval" are both seven letter words that have to be typed out every time for every function, per variable. Then there's the <i>std::</i> prefix and <i><X>(x)</i> suffix. The only benefit of using <i>declval</i> over <i>forward</i> is a savings of one letter not typed.<br />
<br />
But <i>someone</i> must have realized there's a better way. If the function is only one statement, and the return type is the <i>declval</i> of that statement, couldn't the compiler just figure out what I mean when I say this?<br />
<br />
<pre class="brush: cpp">template< class X, class Y >
constexpr auto add( X&& x, Y&& y ) {
return std::forward<X>(x) + std::forward<Y>(y);
}</pre>
<br />
March of this year, one Jason Merrill proposed just this (<a href="http://www.open-std.org/JTC1/SC22/WG21/docs/papers/2012/n3386.html">n3386</a>) and GCC has already implemented it in 4.8 (<a href="http://gcc.gnu.org/gcc-4.8/changes.html">change log</a>), though it requires compiling with <i>-std=c++1y</i>. One can play with 4.8 on Ubuntu with <a href="https://launchpad.net/ubuntu/+source/gcc-snapshot">gcc-snapshot</a>. (Note that it doesn't modify your existing gcc install(s) and puts it in <i>/usr/lib/gcc-snapshot/bin/g++</i>.<i> Also, I have been unable to install any but the third-to-most recent package.</i>) I hope it is not too much trouble to install on other distros/platforms.<br />
<br />
So if your favourite c++11 feature is <i>decltype</i> and <i>declval</i>, prepare to never use them again. The compiler can deduce the type for you implicitly, and better, and it works even if the function is longer than one line. Take for example, reasonably complex template functions like the liftM function I wrote for "<a href="http://yapb-soc.blogspot.com/2012/11/arrows-and-kleisli-in-c.html">Arrows and Keisli</a>":<br />
<br />
<pre class="brush: cpp">constexpr struct LiftM {
template< class F, class M, class R = Return<typename std::decay<M>::type> >
constexpr auto operator () ( F&& f, M&& m )
-> decltype( std::declval<M>() >>= compose(R(),std::declval<F>()) )
{
return std::forward<M>(m) >>= compose( R(), std::forward<F>(f) );
}
template< class F, class A, class B >
constexpr auto operator () ( F&& f, A&& a, B&& b )
-> decltype( std::declval<A>() >>= compose (
rcloset( LiftM(), std::declval<B>() ),
closet(closet,std::declval<F>())
) )
{
return std::forward<A>(a) >>= compose (
rcloset( LiftM(), std::forward<B>(b) ),
closet(closet,std::forward<F>(f))
);
}
} liftM{};</pre>
<br />
Could be written instead:<br />
<br />
<pre class="brush: cpp">constexpr struct LiftM {
template< class F, class M >
constexpr auto operator () ( F&& f, M&& m ) {
using R = Return< typename std::decay<M>::type >;
return std::forward<M>(m) >>= compose( R(), std::forward<F>(f) );
}
template< class F, class A, class B >
constexpr auto operator () ( F&& f, A&& a, B&& b ) {
return std::forward<A>(a) >>= compose (
rcloset( LiftM(), std::forward<B>(b) ),
closet(closet,std::forward<F>(f))
);
}
} liftM{};</pre>
<br />
Automatic type deduction didn't exactly make this function more obvious or simple, but it did remove the visual cruft and duplication of the definition. Now, if I improve this function to make it more clear, I won't have a <i>decltype</i> expression to have to also edit.<br />
<br />
To be fair, this doesn't entirely replace <i>decltype</i>. <a href="http://gcc.gnu.org/ml/gcc-patches/2012-03/msg01799.html"><i>auto</i> doesn't perfect forward</a>. But it seems to work as expected, most of the time.<br />
<br />
For another example of the use-case of auto return type deduction, consider this program:<br />
<br />
<pre class="brush: cpp">#include <tuple>
int main() {
auto x = std::get<0>( std::tuple<>() );
}</pre>
<br />
This, small, simple, and obviously wrong program generates an error message 95 lines long. Why? GCC has to check make sure this isn't valid for the <i>std::pair</i> and <i>std::array</i> versions of <i>get</i>, and when it checks the <i>tuple</i> version, it has to instantiate <i>std::tuple_element</i> recursively to find the type of the element. It actually checks for the <i>pair</i> version first, so one has to search the message for the obviously correct version and figure out why it failed. A simple one-off bug in your program could cause a massive and difficult to parse error message. We can improve this simply.<br />
<br />
<pre class="brush: cpp">#include <tuple>
template< unsigned int i, class T >
auto get( T&& t ) {
using Tup = typename std::decay<T>::type;
static_assert( i < std::tuple_size<Tup>::value,
"get: Index too high!" );
return std::get<i>( std::forward<T>(t) );
}
int main() {
int x = get<0>( std::tuple<>() );
}</pre>
<br />
How much does this shrink the error by? Actually, it grew to 112 lines, but right at the top is<br />
<br />
<pre class="brush: cpp">auto.cpp: In instantiation of 'auto get(T&&) [with unsigned int i = 0u; T = std::tuple<>]':
auto.cpp:13:36: required from here
auto.cpp:7:5: error: static assertion failed: get: Index too high!</pre>
<br />
The error message might be a little bigger, but it tells you right off the bat what the problem is, meaning one has less to parse.<br />
<br />
Similar to this <i>static_assert</i> example, typedefs done as default template arguments can be moved to the function body in many cases.<br />
<br />
<pre class="brush: cpp">template< class X, class Y = A<X>, class Z = B<Y> >
Z f( X x ) {
Z z;
...
return z;
}</pre>
<br />
can now be written <br />
<br />
<pre class="brush: cpp">template< class X > // Simpler signature.
auto f( X x ) {
using Y = A<X>; // Type instantiation logic
using Z = B<Y>; // done in-function.
Z z;
...
return z;
}</pre>
<br />
<h4>
Looking forward.</h4>
This release of GCC also implements inheriting constructors, <i>alignas</i>, and attribute syntax. It also may have introduced a few bugs; for example, my library, which compiles with 4.7, does not with 4.8, producing many undefined references.<br />
<br />
The other features of this release might not be quite so compelling, but automatic type deduction alone is one powerful enough to change the way coding is done in C++--again. Heavily templated code will become a breeze to write and maintain as figuring out the return type is often the hardest part. I find it encouraging that this has been implemented so quickly. Of coarse, it's not standard, at least not yet.<br />
<br />
On a final note, I wonder how this will interact with the static if. It would be nice, were the following well-formed:<br />
<br />
<pre class="brush: cpp">template< class X >
auto f( X x ) {
if( std::is_same<int,X>::value )
return "Hi"; // Int? Return string.
else
return x / 2; // Other? halve.
}</pre>
<br />Splinter of Chaoshttp://www.blogger.com/profile/14715348728512729776noreply@blogger.com6tag:blogger.com,1999:blog-8638474063464489651.post-8400354959288579902012-12-01T06:26:00.000-08:002012-12-14T18:01:21.875-08:00std::move and lambda? It's just partial application!So, "<a href="http://marcoarena.wordpress.com/2012/11/01/learn-how-to-capture-by-move/">Learn how to capture by move</a>" was posted <a href="http://isocpp.org/blog/2012/11/learn-how-to-capture-by-move">on isocpp.org</a> and then this morning, "<a href="http://jrb-programming.blogspot.com/2012/11/another-alternative-to-lambda-move.html">Another Alternative to lambda move capture</a>" was on reddit. These are both impressive solutions and I look forward to seeing what other creative answers the C++ community will come up with. I also want to go over how this problem can be solved with partial application.<br />
<br />
First, here's the definition.<br />
<br />
<pre class="brush: cpp">template< class F, class X >
struct Part {
F f;
X x;
template< class _F, class _X >
constexpr Part( _F&& f, _X&& x )
: f(forward<_F>(f)), x(forward<_X>(x))
{
}
template< class ... Xs >
auto operator () ( Xs&& ...xs )
-> decltype( f(x,declval<Xs>()...) )
{
return f( x, forward<Xs>(xs)... );
}
};
template< class F, class X >
constexpr Part<F,X> closet( F f, X x ) {
return Part<F,X>( std::move(f), std::move(x) );
}</pre>
<br />
So, there's this thing called the <a href="http://en.wikipedia.org/wiki/Funarg_problem#Upwards_funarg_problem">upward funarg problem</a>. Let's say we wrote a function like this:<br />
<br />
<pre class="brush: cpp">std::function<std::vector<int>()> bad_originate() {
std::vector<int> v = { 1, 2, 3, 4, 5 };
return [&]{ return v; };
}
auto bad = bad_originate();
bad(); // Runtime error!</pre>
<br />
This fails because the reference to <i>v</i> becomes invalidated when it goes out of scope. We could copy <i>v</i> into the lambda, but our goal is a move.<br />
<br />
So can we fix this with partial application? Let's add a bit to the example by including a function to partially apply.<br />
<br />
<pre class="brush: cpp">std::vector<int> add_all( std::vector<int>& v, int x ) {
// Arbitrarily move v.
auto w = std::move(v);
std::transform( w.begin(), w.end(), w.begin(),
closet(std::plus<int>(),x) );
return w;
}
using Origin = Part<decltype(add_all)*,std::vector<int>>;
Origin originate() {
std::vector<int> v = { 1, 2, 3, 4, 5 };
return closet( add_all, std::move(v) );
}</pre>
<br />
And it ain't hard at all.<br />
<br />
<pre class="brush: cpp">Origin f = originate();
print_vec( "original : ", f.x );
print_vec( "added 10 : ", f(10) );
print_vec( "original : ", f.x );</pre>
<br />
will output<br />
<br />
<i> original : = 1 2 3 4 5 <br />
added 10 : = 11 12 13 14 15 <br />
original : = </i><br />
<br />
Maybe one would prefer that <i>add_all</i> take its argument by value; now, composition becomes useful.<br />
<br />
<pre class="brush: cpp">template< class F, class G > struct Composition {
F f = F();
G g = G();
Composition() { }
Composition( F f, G g)
: f(move(f)), g(move(g)) { }
template< class X, class ...Y >
auto operator () ( X&& x, Y&& ...y )
-> decltype( f(g(declval<X>()), declval<Y>()...) )
{
return f( g( forward<X>(x) ), forward<Y>(y)... );
}
};
template< class F, class G >
Composition<F,G> comp( F f, G g ) {
return Composition<F,G>( std::move(f), std::move(g) );
}
constexpr struct Mover {
template< class X >
X&& operator () ( X& x ) const {
return std::move(x);
}
} mover{};
std::vector<int> add_all2( std::vector<int> w, int x ) {
std::transform( w.begin(), w.end(), w.begin(),
closet(std::plus<int>(),x) );
return w;
}
using MoveAddAll = decltype( comp(add_all2,mover) );
using Origin2 = Part< MoveAddAll, std::vector<int> >;
Origin2 originate2() {
std::vector<int> v = { 1, 2, 3, 4, 5 };
return Origin2( comp(add_all2,mover), std::move(v) );
}</pre>
<br />
and<br />
<br />
<pre class="brush: cpp">Origin2 g = originate2();
print_vec( "original : ", g.x );
print_vec( "added 10 : ", g(10) );
print_vec( "original : ", g.x );</pre>
<br />
will output the same as above.<br />
<br />
The code to this mini-article can be found here: <a href="https://gist.github.com/4182570">https://gist.github.com/4182570 </a>Splinter of Chaoshttp://www.blogger.com/profile/14715348728512729776noreply@blogger.com0tag:blogger.com,1999:blog-8638474063464489651.post-30205619769571066312012-11-30T06:34:00.002-08:002012-12-14T18:02:12.221-08:00Arrows and Kleisli in C++<a href="http://hackage.haskell.org/packages/archive/base/4.6.0.0/doc/html/Control-Arrow.html">Control.Arrow</a> shows an odd-but-interesting part of Haskell. Arrows are functions; composable and callable. Arrows and Monads sometimes seem to be referred to as alternatives to each other. Perhaps now would be a good time to relate some category theory.<br />
<br />
<h4>
From A to B. </h4>
A category theorist might view functions as a transformation from one type to another. For example,<br />
<br />
<i>std::to_string : X -> std::string</i><br />
<br />
would mean that <i>to_string</i> is a function that maps an <i>X</i> to a <i>string</i>.<br />
<br />
In "<a href="http://yapb-soc.blogspot.com/2012/11/generic-function-objects-or.html">Generic Function Objects</a>", I talked about type constructors.<br />
<br />
<pre class="brush: cpp">/* MakeT T : X -> T<X> */
template< template<class...> class T > struct MakeT {
template< class ...X, class R = T< typename std::decay<X>::type... > >
constexpr R operator () ( X&& ...x ) {
return R( std::forward<X>(x)... );
}
};
/* pair : X x Y -> std::pair<X,Y> */
constexpr auto pair = MakeT<std::pair>();</pre>
<br />
I use the notation <i>X x Y</i> to mean that <i>MakeT<pair></i> takes two arguments. Though, in Haskell, it would actually look like this:<br />
<i> </i><br />
<i> make_pair :: X -> Y -> (X,Y)</i><br />
<br />
Haskell uses <a href="http://en.wikipedia.org/wiki/Currying">curried notation</a>. <br />
<i> </i><br />
<i> g : pair<X,A> -> pair<X,B></i><br />
<br />
Here,<i> g</i> is a function that transforms the second element of the <i>pair</i> to a <i>B</i>, but leaves the first as an <i>X</i>. Although this cannot be inferred from the definition, let's assume that the first value (<i>X</i>) is not transformed in any way. There is a function that represents this non-transformation.<br />
<br />
<pre class="brush: cpp">/* id : X -> X */
constexpr struct Id {
template< class X >
constexpr X operator () ( X&& x ) {
return std::forward<X>(x);
}
} id{};</pre>
<br />
Arrows provide the tools to take a normal function, <i>f : A -> B</i>, and convert it to a function like <i>g</i>. This is sort of like composition <br />
<br />
<pre class="brush: cpp">/* compose : (B -> C) x (A -> B) -> (A -> C) */
template< class F, class G >
struct Composition
{
F f; G g;
template< class _F, class _G >
constexpr Composition( _F&& f, _G&& g )
: f(std::forward<_F>(f)), g(std::forward<_G>(g)) { }
template< class X >
constexpr auto operator() ( X&& x)
-> decltype( f(g(std::declval<X>())) )
{
return f( g( std::forward<X>(x) ) );
}
};
constexpr auto compose = MakeT<Composition>();</pre>
<br />
If we look at <i>A -> B</i> as a type itself, say <i>AB</i>, and <i>B -> C</i> as a type, <i>BC</i>, then it is clear that composition is <i>BC x AB -> AC</i>. Functions themselves are values with types and composition creates a new value and type. We can think of them as being on a geometric plain with the points <i>A</i>, <i>B</i>, and <i>C</i>. Functions are connections from point-to-point. Similarly, we can think of <i>AB</i>, <i>BC</i>, and <i>AC</i> as points on a different plain with arrows connecting <i>AB</i> and <i>BC</i> to <i>AC</i> (though <i>AB</i> and <i>BC</i> cannot be connected).<br />
<h4>
</h4>
<h4>
Functions as Arrows.</h4>
Arrows have several operations. The first is <i>arr</i>, which transforms a function to an arrow; but since functions <b>are</b> arrows, it isn't useful, right off the bat. <i>first</i> and <i>second</i> take functions, <i>A -> B</i>, and lift them to pair-oriented functions. <i>fan</i> takes two functions, one <i>A -> B</i>, another <i>A -> C</i>, and returns a function to <i>pair<B,C></i>. <i>split</i> takes two functions as well, <i>A -> B</i> and <i>X -> Y</i>, and returns a function <i>pair<A,X> -> pair<B,Y></i>.<br />
<br />
<i> arr : (A -> B) -> (A -> B)</i><br />
<i> first : (A -> B) -> (pair<A,X> -> pair<B,X>)</i><br />
<i> second : (A -> B) -> (pair<X,A> -> pair<X,B>)</i><br />
<i> split : (A -> B) x (X -> Y) -> (pair<A,X> -> pair<B,Y>)</i><br />
<i> fan : (A -> B) x (A -> C) -> (A -> pair<B,C>)</i><br />
<br />
First, we fill in the declarations.<br />
<br />
<pre class="brush: cpp">template< class A, class F, class Arr = Arrow< Cat<A> > >
constexpr auto arr( F&& f ) -> decltype( Arr::arr( std::declval<F>() ) )
{
return Arr::arr( std::forward<F>(f) );
}
template< class A > struct Arr {
template< class F >
constexpr auto operator () ( F&& f ) -> decltype( arr(std::declval<F>()) )
{
return arr( std::forward<F>(f) );
}
};
constexpr struct Split {
template< class F, class G, class A = Arrow<Cat<F>> >
constexpr auto operator () ( F&& f, G&& g )
-> decltype( A::split(std::declval<F>(), std::declval<G>()) )
{
return A::split( std::forward<F>(f), std::forward<G>(g) );
}
} split{};
constexpr struct Fan {
template< class F, class G, class A = Arrow<Cat<F>> >
constexpr auto operator () ( F&& f, G&& g )
-> decltype( A::fan(std::declval<F>(),std::declval<G>()) )
{
return A::fan( std::forward<F>(f), std::forward<G>(g) );
}
} fan{};
constexpr struct First {
template< class F, class A = Arrow<Cat<F>> >
constexpr auto operator () ( F&& f )
-> decltype( A::first(std::declval<F>()) )
{
return A::first( std::forward<F>(f) );
}
} first{};
constexpr struct Second {
template< class F, class A = Arrow<Cat<F>> >
constexpr auto operator () ( F&& f ) -> decltype( A::second(std::declval<F>()) ) {
return A::second( std::forward<F>(f) );
}
} second{};</pre>
<br />
<i>arr</i> will be trivial to implement, but the others are tricky. Luckily, we can define it mostly in terms of <i>split</i>--it looks like this:<br />
<br />
<pre class="brush: cpp">/* pairCompose : (A -> B) x (X -> Y) -> (pair<A,X> -> pair<B,Y>) */
template< class F, class G > struct PairComposition {
F f; G g;
template< class _F, class _G >
constexpr PairComposition( _F&& f, _G&& g )
: f(std::forward<_F>(f)), g(std::forward<_G>(g))
{
}
template< class P/*air*/ >
constexpr auto operator() ( const P& p )
-> decltype( std::make_pair( f(std::get<0>(p)), g(std::get<1>(p)) ) )
{
return std::make_pair( f(std::get<0>(p)), g(std::get<1>(p)) );
}
};
constexpr auto pairCompose = MakeT<PairComposition>();</pre>
<br />
<i>pairCompose</i> returns a function expecting a <i>pair</i> and returns a <i>pair</i>, threading the first value through <i>f</i> and the second through <i>g</i>. We can compose <i>PairCompositions</i>.<br />
<br />
<pre class="brush: cpp">namespace std {
std::string to_string( const std::string& s );
template< class X, class Y >
std::string to_string( const std::pair<X,Y>& p );
std::string to_string( const std::string& s ) {
return "\"" + s + "\"";
}
template< class X, class Y >
std::string to_string( const std::pair<X,Y>& p ) {
return "(" + to_string(p.first) + "," + to_string(p.second) + ")";
}
}
constexpr struct ToString {
template< class X >
std::string operator () ( const X& x ) const {
return std::to_string(x);
}
} string{};
int main() {
using std::cin;
using std::cout;
using std::endl;
auto plus2 = []( int x ){ return x+2; };
std::pair<int,int> p( 1, 1 );
cout << "((+2) . string, (+4))( 1, 1 ) = "
<< compose( pairCompose(string,plus2),
pairCompose(plus2, plus2) )(p) << endl;
}</pre>
<br />
This will output<i> ("3",5)</i>. It's easiest to look at this as <i>p.first</i> and <i>p.second</i> being on two separate paths. I have written it such that the individual paths are verticle. The first path starts at <i>1</i>, and goes to <i>plus2(1)</i>, to <i>show(plus2(1))</i>. The second path starts at <i>2</i> and ends at <i>plus2(plus2(2))</i>. The odd part is that we're defining both paths at once.<br />
<br />
Observe that <i>first(f) = split(f,id)</i>. Proof?<br />
<br />
<i> first : (A -> B) -> (pair<A,X> -> pair<B,X>)</i><br />
<i> split : (A -> B) x (X -> Y) -> (pair<A,X> -> pair<B,Y>) </i><br />
<i>id : X -> X</i><br />
<i> f : A -> B </i><br />
<i> split(f,id) : pair<A,X> -> pair<B,X> </i><br />
<br />
Since we know <i>first(f) = split(f,id)</i>, we can intuit that <i>second(f) = split(id,f)</i> and also, <i>split(f,g) = compose( first(f), second(g) )</i>.<br />
<br />
<i>fan</i> represents a fork in the road. One variable gets fed into two functions, the results of which get zipped into a pair. <i>duplicate</i> will do the splitting, but we'll rely on <i>split</i> to implement <i>fan</i>.<br />
<br />
<pre class="brush: cpp">constexpr struct Duplicate {
template< class X, class P = std::pair<X,X> >
constexpr P operator() ( const X& x ) {
return P( x, x );
}
} duplicate{};</pre>
<i></i> <br />
<br />
With this, we can say that <i>fan(f,g)(x) = split(f,g)( duplicate(x) )</i>. Since we know what everything looks like, we can define <i>Arrow<F></i>.<br />
<br />
<pre class="brush: cpp">template< class Func > struct Arrow<Func> {
template< class F >
static constexpr F arr( F&& f ) { return std::forward<F>(f); }
/* split(f,g)(x,y) = { f(x), g(y) } */
template< class F, class G >
static constexpr auto split( F f, G g )
-> PairComposition<F,G>
{
return pairCompose( std::move(f), std::move(g) );
}
/*
* first(f)(x,y) = { f(x), y }
* second(f)(x,y) = { x, f(y) }
*/
template< class F >
static constexpr auto first( F f )
-> decltype( split(std::move(f),id) )
{
return split( std::move(f), id );
}
template< class F >
static constexpr auto second( F f )
-> decltype( split(id,std::move(f)) )
{
return split( id, std::move(f) );
}
/* fan(f,g)(x) = { f(x), g(x) } */
template< class F, class G >
static constexpr auto fan( F f, G g )
-> decltype( compose( split(std::move(f),std::move(g)), duplicate ) )
{
return compose( split(std::move(f),std::move(g)), duplicate );
}
};</pre>
<br />
Now, we can rewrite the above example:<br />
<br />
<pre class="brush: cpp">int main() {
auto plus2 = []( int x ){ return x+2; };
cout << "(+2) *** (+2) >>> string *** (+2) $ 1 = "
<< compose( split(string, plus2),
fan( plus2, plus2) )(1) << endl;
}</pre>
<br />
One way to think of Arrows is as paths. <i>fan</i> represents a fork in the road, <i>split</i> defines two separate, but parallel, paths at once, and <i>first</i> and <i>second</i> allow progress on one of the paths. For an arbitrary example, consider a branching path that hides some treasure. What sequence of moves are required to recover it?<br />
<br />
<pre class="brush: cpp">/*
* Get 0 : (X,Y) -> X
* Get 1 : (X,Y) -> Y
*/
template< size_t N > struct Get {
template< class P >
constexpr auto operator () ( P&& p )
-> decltype( std::get<N>(std::declval<P>()) )
{
return std::get<N>( std::forward<P>(p) );
}
};
int main() {
constexpr auto fst = Get<0>();
constexpr auto snd = Get<1>();
constexpr auto oneHundred = []( int x ){ return 100; };
// Hide the hundred.
// hidden = pair( pair( 0, pair(100,0) ), 0 )
auto hidden = fan( fan(id,fan(oneHundred,id)), id )( 0 );
// Function to find it again.
auto find = compose( fst, compose(snd,fst) );
cout << "I found " << find(hidden) << "!" << endl;
}</pre>
<br />
<br />
<h4>
Enter Kleisli.</h4>
This all works great for free functions, but some functions look like this:<br />
<br />
<i>f : X -> M<Y> -- Where M is some Monad</i><br />
<br />
<i> </i>This <i>f</i> is a member of the <a href="http://en.wikipedia.org/wiki/Kleisli_category">Kleisli category</a>. It accepts a normal value and returns a Monadic one. Examples of Kelisli functions (psuedocode):<br />
<br />
<i>Just : X -> unique_ptr<X></i><br />
<i> Just = MakeT<unique_ptr>();</i><br />
<br />
<i> Seq : X -> std::vector<X></i><br />
<i> Seq(x) = std::vector<X>{x} </i><br />
<br />
<i> mreturn<M> : X -> M<X> </i><br />
<br />
<i>unique_ptr</i>s and <i>vector</i>s are monads, so any function that produces one from some <i>x</i> can be considered in the Kleisli category. Though, to the compiler, there's no logic to that, so we define a wrapper type around the function. This is a fairly common pattern, so I define a base class for it.<br />
<br />
<pre class="brush: cpp">template< class F > struct Forwarder {
using function = typename std::decay<F>::type;
function f = F();
template< class ...G >
constexpr Forwarder( G&& ...g ) : f( std::forward<G>(g)... ) { }
template< class ...X >
constexpr auto operator() ( X&& ...x )
-> decltype( f(std::declval<X>()...) )
{
return f( std::forward<X>(x)... );
}
constexpr operator function () { return f; }
};
</pre>
<br />
The Kleisli itself can be implemented two ways. The Haskell way would be something like this:<br />
<br />
<pre class="brush: cpp">/* Kleisli M A B : A -> M<B> */
template< template<class...> class M, class A, class B,
class F = std::function<M<A>(B)> >
struct Kleisli : Forwarder<F> {
template< class ...G >
constexpr Kleisli( G&& ...g ) : Forwarder<F>(std::forward<G>(g)...) { }
};
</pre>
<br />
This is faithful to how Haskell defines it.<br />
<br />
<i>newtype Kleisli m a b = Kleisli { runKleisli :: a -> m b }</i><br />
<br />
But just think about this for a moment. A Kleisli is an Arrow, right? What would <i>Arrow<Kleisli>::first</i> return?<br />
<br />
<i> first : (A -> B) -> (pair<A,X> -> pair<B,X>)</i><br />
<i> Kleisli f = A -> M<B> </i><br />
<i> first(Kleisli f) : (A -> M B) -> (pair<A,X> -> M pair<B,X>)</i><br />
<br />
What's the type of <i>X</i>? It's truly impossible to know because it depends on what gets passed in, which is what the notation above means.<br />
<br />
Is it impossible to define Kleisli in this way? I don't know. I attempted to specialize its composition function for when <i>A</i> or <i>B</i> were pair types<i>, </i>but there are four combinations of whether <i>A</i> or <i>B</i> or both is a pair. I tried assigning <i>X</i> the type of <i>std::placeholders::_1</i>, but none of my attempts to make it really work compiled. (<a href="https://github.com/splinterofchaos/Pure/blob/kleisli-S-T/Arrow.h#L332">It was horrible</a>.)<br />
<br />
But we don't have <b>any<i> </i></b>of that trouble if we define Kleisli differently.<br />
<br />
<br />
<h4>
Kleisli<F>.</h4>
<pre class="brush: cpp">/* Kleisli M F : F -- where F : A -> M<B> */
template< template<class...> class M, class F = Id >
struct Kleisli : Forwarder<F> {
template< class ...G >
constexpr Kleisli( G&& ...g ) : Forwarder<F>(std::forward<G>(g)...) { }
};
</pre>
<br />
An implicit requirement of arrows is that they can be composed, but Kleislies?<br />
<br />
<i> compseKleisli : Kleisli(B -> M<C>) x </i><i><i>Kleisli</i>(A -> M<B>) -> (...?)</i><br />
<br />
We can reasonably assume that it should be <i>Kleisli(A -> M<C></i>), but our naive definition of composition must be specialized. Literally.<br />
<br />
<pre class="brush: cpp">/* Composition : Kleisli(B -> M<C>) x Kleisli(A -> M<B>) -> (A -> M<C>) */
template< template<class...> class M, class F, class G >
struct Composition<Kleisli<M,F>,Kleisli<M,G>>
{
Kleisli<M,F> f;
Kleisli<M,G> g;
template< class _F, class _G >
constexpr Composition( _F&& f, _G&& g )
: f(std::forward<_F>(f)), g(std::forward<_G>(g)) { }
template< class X >
constexpr auto operator() ( X&& x )
-> decltype( g(std::forward<X>(x)) >>= f )
{
return g(std::forward<X>(x)) >>= f;
}
};
/* kcompose : Kleisli(B -> M<C>) x Kleisli(A -> M<B>) -> Kleisli(A -> M<C>) */
constexpr struct KCompose {
template< template<class...> class M, class F, class G >
constexpr auto operator () ( Kleisli<M,F> f, Kleisli<M,G> g )
-> Kleisli< M, Composition<Kleisli<M,F>,Kleisli<M,G> >
{
return kleisli<M> (
compose( std::move(f), std::move(g) )
);
}
} kcompose{};
int main() {
auto stars = kleisli<std::basic_string> (
[] (char c) -> std::string {
return c == '*' ? "++" :
c == '+' ? "* *" : std::string{c};
}
);
auto starsSqr = kcompose( stars, stars );
auto starsCube = kcompose( starsSqr, stars );
cout << "stars of '*' : " << stars('*') << endl;
cout << "stars^2 of '*' : " << starsSqr('*') << endl;
cout << "stars^3 of '*' : " << starsCube('*') << endl;
}</pre>
<br />
This outputs <i> </i><br />
<br />
<i> stars of '*' : "++"<br />
stars^2 of '*' : "* ** *"<br />
stars^3 of '*' : "++ ++++ ++"</i><br />
<br />
Like before, it would be most convenient to define <i>Arrow<Kleisli></i> in terms of <i>split</i>. <i>split(f,g)</i>, given the pair <i>{x,y}</i>, will have to pass <i>x</i> into <i>f</i> and <i>y </i>into <i>g</i>, both of which will return Monadic values. Finally, a <i>pair</i> will have to be constructed from the values extracted from <i>f(x)</i> and <i>g(y)</i>.<br />
<br />
<i> split: Kleisli (A -> M B) x Kleisli (X -> M Y) -> Kleisli (pair<A,X> -> M pair<B,Y>)</i><br />
<br />
To extract the values from <i>f(x)</i> and <i>g(y)</i>, we need to call <i>mbind</i> on each, which in Haskell might look like this:<br />
<br />
<i>f(x) >>= (\x' -> g(y) >>= (\y' -> return (x,y)))</i><br />
<br />
Or, <a href="http://www.haskell.org/ghc/docs/latest/html/libraries/base/Control-Monad.html"><i>Control.Monad</i></a> defines <a href="http://www.haskell.org/ghc/docs/latest/html/libraries/base/src/Control-Monad.html#liftM"><i>liftM</i></a>.<br />
<br />
<i>liftM2 (,) f(x) g(y) -- where (,) is equivalent to make_pair</i><br />
<br />
<pre class="brush: cpp">template< class M > struct Return {
template< class X >
constexpr auto operator () ( X&& x )
-> decltype( mreturn<M>(std::declval<X>()) )
{
return mreturn<M>( std::forward<X>(x) );
}
};
/*
* liftM : (A -> B) x M<A> -> M<B>
* liftM : (A x B -> C) x M<A> x M<B> -> M<C>
*/
constexpr struct LiftM {
template< class F, class M, class R = Return<typename std::decay<M>::type> >
constexpr auto operator () ( F&& f, M&& m )
-> decltype( std::declval<M>() >>= compose(R(),std::declval<F>()) )
{
return std::forward<M>(m) >>= compose( R(), std::forward<F>(f) );
}
template< class F, class A, class B >
constexpr auto operator () ( F&& f, A&& a, B&& b )
-> decltype( std::declval<A>() >>= compose (
rcloset( LiftM(), std::declval<B>() ),
closet(closet,std::declval<F>())
) )
{
return std::forward<A>(a) >>= compose (
rcloset( LiftM(), std::forward<B>(b) ),
closet(closet,std::forward<F>(f))
);
}
} liftM{};</pre>
<br />
It acts basically as a n-ary <i>mbind</i>, though one could also define an n-ary <i>mbind</i>! <i>liftM</i> works even if you don't.<br />
<br />
Finally, we have all the pieces in place to implement <i>KleisliSplit</i>.<br />
<br />
<pre class="brush: cpp">/* kleisliSplit : Kleisli(A -> M<B>) x Kleisli(X -> M<Y>) -> (piar<A,X> -> M<pair<B,Y>>) */
template< template<class...> class M, class F, class G >
struct KleisliSplit {
F f;
G g;
constexpr KleisliSplit( F f, G g ) : f(std::move(f)), g(std::move(g)) { }
template< class X, class Y >
constexpr auto operator () ( const std::pair<X,Y>& p )
-> decltype( liftM(pair,f(std::get<0>(p)),g(std::get<1>(p))) )
{
return liftM (
pair,
f( std::get<0>(p) ),
g( std::get<1>(p) )
);
}
};</pre>
<br />
The final note before moving on: <i>arr</i>. Kleisli's <i>arr</i> is like a Monad's <i>mreturn</i>.<br />
<br />
<i>arr : (A -> B) -> Kleisli(A -> M<B>)</i><br />
<i> arr(f)(x) = mreturn<M>( f(x) )</i><br />
<br />
or:<br />
<i> </i><br />
<i> arr = liftM</i><br />
<br />
<pre class="brush: cpp">template< template<class...> class M, class F >
struct Arrow< Kleisli<M,F> > {
template< class G >
using K = Kleisli<M,G>;
template< class G >
static constexpr auto arr( G g ) -> Kleisli< M, Part<LiftM,G> > {
return kleisli<M>( closet(liftM,std::move(g)) );
}
template< class G >
static constexpr auto first( G g )
-> K<decltype( ::split(std::move(g),arr(id)) )>
{
// id is not a Kleisli.
// The call to arr refers to the arr above, not ::arr.
// arr(id) : Kleisli(X -> M<X>)
return ::split( std::move(g), arr(id) );
}
template< class G >
static constexpr auto second( G g)
-> K<decltype( ::split(arr(id),std::move(g)) )>
{
return ::split( arr(id), std::move(g) );
}
template< class G >
static constexpr auto split( Kleisli<M,F> f, Kleisli<M,G> g )
-> K< KleisliSplit<M,F,G> >
{
return KleisliSplit<M,F,G>( std::move(f.f), std::move(g.f) );
}
template< class _F, class _G >
static constexpr auto fan( _F&& f, _G&& g )
-> decltype( kcompose(split(std::declval<_F>(),std::declval<_G>()),arr(duplicate)) )
{
return kcompose( split(std::forward<_F>(f),std::forward<_G>(g)), arr(duplicate) );
}
};
int main() {
auto stars = kleisli<std::vector> (
[] (char c) {
return c == '*' ? std::vector<char>{'+','+'} :
c == '+' ? std::vector<char>{'*',' ','*'} : std::vector<char>{c};
}
);
auto hairs = kleisli<std::vector> (
[] (char c) -> std::vector<char> {
return c == '*' ? std::vector<char>{'\'','"','\''} :
c == '+' ? std::vector<char>{'"',' ','"'} :
c == '"' ? std::vector<char>{'\''} :
c == '\'' ? std::vector<char>{} :
std::vector<char>{c};
}
);
cout << "hairs of '*' : " << hairs('*') << endl;
cout << "hairs^2 of '*' : " << compose(hairs,hairs)('*') << endl;
cout << "split(stars,hairs) (*,*) = " << split(stars,hairs)(pair('*','*')) << endl;
cout << "fan(stars,hairs) (*) = " << fan(stars,hairs)('*') << endl;
cout << "fan(hairs,stars) (*) = " << fan(hairs,stars)('*') << endl;
cout << "split(hairs,stars) . fan(stars,hairs) = "
<< compose( split(hairs,stars), fan(stars,hairs) )('*') << endl;
}
</pre>
Finally, this outputs<br />
<i><br />
</i> <i> hairs of '*' : [',",']<br />
hairs^2 of '*' : [']<br />
split(stars,hairs) (*,*) = [(+,'),(+,"),(+,'),(+,'),(+,"),(+,')]<br />
fan(stars,hairs) (*) = [(+,'),(+,"),(+,'),(+,'),(+,"),(+,')]<br />
fan(hairs,stars) (*) = [(',+),(',+),(",+),(",+),(',+),(',+)]<br />
split(hairs,stars) . fan(stars,hairs) = [(",'),( ,'),(",'),(","),( ,"),(","),(",'),( ,'),(",'),(",'),( ,'),(",'),(","),( ,"),(","),(",'),( ,'),(",')]</i><br />
<br />
<i><br />
</i> <br />
<h4>
Extras -- Convenience and Pitfalls.</h4>
Perhaps the most important parts of <i>Control.Arrow</i> are here, but there are still a few things that can be added. For example, <i>compose</i> is backwards! <i>compose(f,g)</i> means <i>"do g, then do f."</i> Because of this we have to read it backwards. <i>fomp</i> is useful, if only because we read left-to-right, not the other way around.<br />
<br />
<pre class="brush: cpp">/* fcomp : (A -> B) x (B -> C) -> (A -> C) */
constexpr struct FComp {
template< class G, class F, class C = Composition<F,G> >
constexpr C operator () ( G g, F f ) {
return C(std::move(f),std::move(g));
}
} fcomp{};
</pre>
<br />
Another tool is <i>precompose </i>and <i>postcompose</i>. Perhaps one wants to compose something that is not a Kleisli with something that is.<br />
<br />
<pre class="brush: cpp">/* prefcomp : (A -> B) x (Arrow X Y) -> Arrow A Y */
constexpr struct PreFComp {
template< class F, class A >
constexpr auto operator () ( F&& f, A&& a )
-> decltype( arr<A>(declval<F>()) > declval<A>() )
{
return arr<A>(forward<F>(f)) > forward<A>(a);
}
} prefcomp{};
/* postfcomp : Arrow A B x (X -> Y) -> Arrow A Y */
constexpr struct PostFComp {
template< class A, class F >
constexpr auto operator () ( A&& a, F&& f )
-> decltype( declval<A>() > arr<A>(declval<F>()) )
{
return forward<A>(a) > arr<A>(forward<F>(f));
}
} postfcomp{};
</pre>
<br />
Beyond that, there's <i>ArrowZero</i> and <i>ArrowPlus</i> to work with Monoids, <i>ArrowChoice</i> based on <a href="http://www.haskell.org/ghc/docs/6.12.2/html/libraries/base-4.2.0.1/Data-Either.html"><i>Either</i></a>, <i>ArrowApply</i> and <i>ArrowLoop</i>.<br />
<br />
<i>ArrowZero</i>, defining <i>zeroArrow</i>, shows the downside to implementing Kleisli as <i>K<M,F></i> instead of <i>K<M,A,B></i>. It is defined like so:<br />
<i><br />
</i> <i> zeroArrow = Kleisli (\_ -> mzero)</i><br />
<br />
The type <i>mzero</i> needs to return is <i>M<B></i>. Its argument, the <i>_</i>, will be of type <i>A</i>. So we have this problem where <i>zeroArrow(k)</i> (for some Kleisli, <i>k</i>) doesn't know what type to return and can't deduce it!<br />
<i> </i><br />
If we had implemented Kleisli with <i>A</i> and <i>B</i>, this would be no problem, but then it would have been impossible(?) to implement <i>Arrow<Kleisli>::first</i>. In Haskell, all functions carry around information about their domain and codomain, so there is no difference between passing around <i>A</i> and <i>B</i> or just <i>F</i>.<br />
<br />
The easiest solution is to diverge from Haskell's definition and require <i>zeroArrow</i> to take an explicit result-type parameter.<br />
<br />
<br />
<h4>
Conclusions.</h4>
Arrows let us do interesting types of composition that one would not normally likely think of. They fall somewhere between Functors and Monads (<a href="http://just-bottom.blogspot.com/2010/04/programming-with-effects-story-so-far.html">this article has a graph</a>). My <a href="http://yapb-soc.blogspot.com/2012/11/monadic-parsing-in-c.html">Monadic parser</a> could have also been <a href="http://www.haskell.org/haskellwiki/Arrow#Parser">written with Arrows</a>.<br />
<br />
I realize this has been a <i>very</i> long post. Hopefully all the code examples work, there are no inaccuracies, and it is understandable, but with an article this large, <i>some</i>thing has probably been fudged. Due to the large amount of material, it may be a good idea to split this article up and go more in depth. Please speak up if something seems off.<br />
<br />
<br />
<b>Source code:</b> <a href="https://gist.github.com/4176121">https://gist.github.com/4176121</a><br />
<b>Haskell reference:</b> <a href="http://hackage.haskell.org/packages/archive/base/4.6.0.0/doc/html/Control-Arrow.html">Control.Arrow</a><br />
<b></b>Splinter of Chaoshttp://www.blogger.com/profile/14715348728512729776noreply@blogger.com0tag:blogger.com,1999:blog-8638474063464489651.post-52115046766366309932012-11-26T17:12:00.001-08:002012-12-14T18:09:11.064-08:00The Importance of Function ObjectsFunction objects allow for freedoms that free functions do not. They are more compatible with higher order functions, composition, and generic programming. They are easier to use; extensible and convenient. They are often even more efficient. <br />
<br />
<br />
<h4>
Higher Order Functions.</h4>
<br />
Write a function that adds two arguments of any given type.<br />
<br />
<pre class="brush: cpp">template< class X, class Y >
constexpr auto add( X&& x, Y&& y )
-> decltype( std::declval<x>() + std::declval<y>() )
{
return std::forward<x>(x) + std::forward<y>(y);
}</y></x></y></x></pre>
<br />
<br />
Easy, right? Now pass that in to <i>std::accumulate</i>.<br />
<br />
<pre class="brush: cpp">int main() {
std::vector<int> v = { 0, 1, 2, 3, 4, 5 };
int sum = std::accumulate( v.begin(), v.end(), 0, add );
std::cout << "sum = " << sum << std::endl;
}</pre>
<br />
And compile!<br />
<br />
<pre class="brush: cpp">fo.cpp:20:68: error: no matching function for call to 'accumulate(std::vector<int>::iterator, std::vector<int>::iterator, int, <unresolved overloaded function type>)'
</pre>
<br />
Huh, GCC can't deduce the type of <i>add</i>. Really, the lexical name, <i>add</i>, refers to a whole set of possible template overloads. It could pick one from the set if we supplied some arguments, but not by the name alone. We can fix this by specifying the type arguments.<br />
<br />
<i>int sum = std::accumulate( v.begin(), v.end(), 0, add<int&,int&> );</i><br />
<br />
Why do we specify <i>int&</i> instead of <i>int</i>? If we specify <i>int</i>, then because of the quirks in perfect forwarding, signature is<br />
<br />
<i> int add( int&&, int&& )</i><br />
<br />
Since its inputs will be <i>lvalues</i>, this will fail to compile. Instead, it will look like this:<br />
<br />
<i>int add( int& &&, int& && )</i> <br />
<br />
and the arguments will be forwarded to the addition as <i>int&</i>'s.<br />
<br />
At this point, falling back on <i>std::plus<int>()</i> would make the most sense, but what if we don't want to have to specify the type? The proposal <a href="http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2012/n3421.htm">n3421</a> expresses this concern, and would allow for the syntax, <i>std::plus<>()</i>, which would be a function object version of <i>add</i>. Finally, that line would compile, but n3421 isn't standard yet. Why not do this?<br />
<br />
<pre class="brush: cpp">struct Add {
template< class X, class Y >
constexpr auto operator () ( X&& x, Y&& y )
-> decltype( std::declval<Y>() + std::declval<X>() )
{
return std::forward<X>(x) + std::forward<Y>(y);
}
};
constexpr auto add = Add();
...
int sum = std::accumulate( v.begin(), v.end(), 0, add ); </pre>
<br />
Here, <i>add</i> would be essentially no different from an instance of <i>std::plus<></i>, but because it is already instantiated, we can avoid much of the syntactic cruft. <br />
<br />
<br />
<h4>
Programming Compositionally. </h4>
<br />
Let's say we have a two-dimensional array<br />
<br />
<pre class="brush: cpp">std::vector<int> v = { 1, 2, 3, 4, 5 };
std::vector<std::vector<int>> vv = { v, v, v };</pre>
<br />
and we want to find the sum of all its element. We might first write a small function to find the sum of a one-dimensional array.<br />
<br />
<pre class="brush: cpp">template< class S >
int sum( const S& s ) {
return std::accumulate( s.begin(), s.end(), 0, add );
}</pre>
<br />
With this, we could write a simple <i>for</i> loop to iterate over each element, adding the sums of each vector, but that's the whole point of <i>accumulate</i>! We could use <i>std::transform</i> to build a list of sums and accumulate that, but what's the point? The easiest thing to do is use function composition.<br />
<br />
Previously, I have talked about <a href="http://yapb-soc.blogspot.com/2012/09/function-composition-in-c.html">function composition</a>. I defined a type, <i>Composition<F,G></i>, which passed its first argument to <i>G</i> and the result of that along with any following arguments to <i>F</i>. <br />
<br />
<i>std::accumulate</i> expects a function where the first value is the accumulator (the sum) and the second value comes from the sequence (in this case: another <i>std::vector</i>). We want the <i>sum</i> of the vector, and then to <i>add</i> the results together. One could fairly trivially write a function that does this explicitly, but that will have to be done for every similar use-case. Patterns of composition can be abstracted away.<br />
<br />
<pre class="brush: cpp">template< class F, class G > struct AccumRight {
F f = F();
G g = G();
constexpr AccumRight() { }
constexpr AccumRight( F f, G g)
: f(f), g(g) { }
template< class X, class Y >
constexpr auto operator () ( X&& x, Y&& y )
-> decltype( f(std::declval<X>(), g(std::declval<Y>()) ) )
{
return f( std::forward<X>(x), g(std::forward<Y>(y)) );
}
};
template< class F, class G, class A = AccumRight<F,G> >
constexpr A rAccumulation( F f, G g ) {
return A( f, g );
}</pre>
<br />
Finally, we can write that function. How does it look if we use free functions?<br />
<br />
<pre class="brush: cpp">sum = std::accumulate( vv.begin(), vv.end(), 0,
rAccumulation(add<int&,int>,sum<std::vector<int>>) );</pre>
<br />
Notice how, since the right-hand argument of <i>add</i> will be an <i>rvalue</i> (the result of <i>sum</i>), we can just tell it <i>int</i>. Now, how about function objects?<br />
<br />
<pre class="brush: cpp">sum = std::accumulate( vv.begin(), vv.end(), 0,
rAccumulation(add,sum) );</pre>
<br />
Much nicer, no? Imagine more complex examples involving passing higher-order functions to higher-order functions. Without using such techniques, higher order functional programming in C++ would be daunting. Manually deducing types is difficult and error prone--sometime impossible due to implementation details, but the compiler can do it instead, if given the chance. Because of this, techniques in C++ that would otherwise be thought of as impossible become easy.<br />
<br />
<br />
<h4>
Finer Control.<i> </i></h4>
<br />
<i>std::get<N></i> is a nice function for working with <i>pair</i>s and <i>tuple</i>s, but doesn't work on normal containers. Lets define a function object for that!<br />
<br />
<pre class="brush: cpp">struct Selector {
int i;
Selector( int j=0 ) : i(j) { }
template< class S >
typename const S::value_type& operator () ( const S& s ) const {
return *std::next( s.begin(), i );
}
};
Selector first(0), third(2);
int main() {
std::vector<int> v = { 1, 2, 3, 4, 5 };
std::cout << "first = " << first(v) << std::endl;
std::cout << "third = " << third(v) << std::endl;
std::cout << "fourth = " << Selector(third(v))(v) << std::endl;
}</pre>
<br />
This will output<br />
<br />
<i>first =1</i><br />
<i>third = 3</i><br />
<i> fourth = 4 </i><br />
<br />
We typically think of functions as static, unchanging things, but <i>Selector</i> is dynamic. It could be used, for example, as a function returning the current player in a game by keeping its member, <i>i</i>, always up-to-date. <br />
<br />
For another example, consider a function object, <i>Printer</i>, that prints its arguments to some file. We might write that like this:<br />
<pre class="brush: cpp"> </pre>
<pre class="brush: cpp">namespace std {
template< class X >
string to_string( const std::vector<X>& v ) {
string str = "[ ";
for( const X& x : v ) str += to_string(x) + " ";
str += "]";
return str;
}
// Because the standard doesn't define this.
string to_string( const std::string& s ) {
return s;
}
template< class T >
string to_string( const T& s ) {
ostringstream ss;
ss << s;
return ss.str();
}
} // namespace std
struct Printer {
std::string destination;
Printer( std::string filename )
: destination( std::move(filename) )
{
}
void redirect( std::string filename ) {
destination = std::move( filename );
}
template< class X >
void operator () ( const X& x ) {
std::ofstream s( destination, std::ios_base::app );
s << std::to_string(x);
}
};
int main() {
std::vector<int> v = { 1, 2, 3, 4, 5 };
std::vector<std::vector<int>> vv = { v, v, v };
Printer prnt( "file1" );
prnt( v );
prnt.redirect( "file2" );
prnt( vv );
}</pre>
<br />
Here, the dynamics come from being able to change the destination of our output at runtime. It outputs:<br />
<i><br />
</i> <i> file1: [ 1 2 3 4 5 ]</i><br />
<i> file2: [ [ 1 2 3 4 5 ] [ 1 2 3 4 5 ] [ 1 2 3 4 5 ] ] </i><br />
<br />
<br />
<h4>
Extensibility.</h4>
I have talked before about <a href="http://yapb-soc.blogspot.com/2012/11/generic-function-objects-or.html">generalizing associativity, transitivity</a>, and other such things in function objects. Inheritance adds a new dimension to functions. Derivatives can benefit from functionality offered in base classes and base classes can benefit from their derivations via the <a href="http://en.wikipedia.org/wiki/Curiously_recurring_template_pattern">curiously recurring template pattern</a>. <br />
<br />
For a different example, let's have the <i>Printer</i> class time-stamp its output.<br />
<br />
<pre class="brush: cpp">#include <ctime>
struct StampedPrinter : Printer {
StampedPrinter( std::string filename )
: Printer( std::move(filename) )
{
}
template< class X >
void operator () ( const X& x ) {
std::time_t t = std::time( nullptr );
std::tm tm = *std::localtime( &t );
std::string time = std::to_string(tm.tm_min) + ":" +
std::to_string(tm.tm_sec) + " -- ";
Printer::operator()( time + std::to_string(x) + "\n" );
}
};</pre>
<br />
By simply changing the type of <i>prnt</i> in the previous example to <i>StampedPrinter</i>, it will output:<br />
<br />
<i>file1:<br />
6:46 -- [ 1 2 3 4 5 ]<br />
<br />
file2:<br />
6:46 -- [ [ 1 2 3 4 5 ] [ 1 2 3 4 5 ] [ 1 2 3 4 5 ] ]</i><br />
<br />
And if run again,<br />
<br />
<i>file1:</i><br />
6:46 -- [ 1 2 3 4 5 ]<br />
13:13 -- [ 1 2 3 4 5 ]<br />
<br />
file2: <br />
6:46 -- [ [ 1 2 3 4 5 ] [ 1 2 3 4 5 ] [ 1 2 3 4 5 ] ]<br />
13:13 -- [ [ 1 2 3 4 5 ] [ 1 2 3 4 5 ] [ 1 2 3 4 5 ] ]<br />
<br />
<br />
<h4>
In conclusion,</h4>
Function objects can do anything that free functions can do, but they cooperate better. They are more cohesive and orthogonal. I hope that these simple demonstrations have been thought provoking.<br />
<br />
<b>The code for this post:</b> <a href="https://gist.github.com/4151880">https://gist.github.com/4151880</a><br />
<b>See also: </b>An interesting article that talks about a <i>trampoline</i> function that runs a recursive function object for tail-call optimization: <a href="http://fendrich.se/blog/2012/11/22/compile-time-loops-in-c-plus-plus-11-with-trampolines-and-exponential-recursion/">http://fendrich.se/blog/2012/11/22/compile-time-loops-in-c-plus-plus-11-with-trampolines-and-exponential-recursion/ </a> Splinter of Chaoshttp://www.blogger.com/profile/14715348728512729776noreply@blogger.com2tag:blogger.com,1999:blog-8638474063464489651.post-25600095217607868372012-11-21T01:17:00.000-08:002012-11-21T01:17:49.057-08:00Some Observations on Human BehaviorAre not the ones who praise a drug-free space also the most tempted to bring drugs to it? Are the ones who cry in empathy for the poor not actually rich? Is it not the case that those who despise intolerance are the most intolerant? <br />
<br />
Don't those who scream "Revolution!" fear change? Are the people who demonize sex not the ones who crave it deeply, and they that oppose pornography not the same as those who suffer from addiction? <br />
<br />
Are the polluters not the biggest deniers of global warming? Are the people opposing gay rights not in the closet? Are those who commit voter fraud not condemning it? Are those who speak of freedom not dreaming of fascism? Don't those with the most to give also complain the most about "hand-outs"? <br />
<br />
Are the knowledgeable not the quietest and the ignorant not the loudest or do those who speak know, and those who don't not? Do those who boast not actually suck? <br />
<br />
Are not those who cast suspicion the least trustworthy? Are those who push blame not riddled with guilt? Does a thief not secure her own goods?<br />
<br />
Does laughter imply absence of sorrow? <br />
<br />
Is hypocrisy human nature?Splinter of Chaoshttp://www.blogger.com/profile/14715348728512729776noreply@blogger.com0tag:blogger.com,1999:blog-8638474063464489651.post-60615756215150828242012-11-19T09:58:00.001-08:002012-12-14T18:03:45.529-08:00Monadic Parsing in C++When researching parsing in Haskell, I always find this pdf: <a href="http://eprints.nottingham.ac.uk/223/1/pearl.pdf">eprints.nottingham.ac.uk/223/1/pearl.pdf</a><br />
<br />
I will be referring back to this paper and encourage my readers to at <i>least</i> skim it as well. It may provide more understanding in how it works.<br />
<br />
It describes a simple, yet hard to comprehend functional parser. I have attempted to translate this to C++, but first I wrote a more intuitive, less functional one. Just a simple calculator, somewhat close to what's described in the pdf, and it worked like this:<br />
<br />
The first stage took the input string and created a list of token type-lexeme pairs. Given the input <i>"1+2*2"</i>, it would spit back <i>{(INT,"1"),(OP,"+"),...}</i>. The next stage uses two mutually recursive functions to represent two parse states: that of sums and subtractions and that of numbers, multiplications and divisions. Since addition has the lowest precedence in this mini-language, it's safe to parenthesize the rest of the expression, meaning that if it parsed <i>"1+"</i>, it will expect the next number to be a number, and it might want to add one to it, eagerly, but not if fallowed by a <i>"2*2"</i> since multiplication has higher precedence.<br />
<br />
These functions convert the vector of tokens to an expression tree that evaluates the arguments.<i>"2*2"</i> would parse to a tree with a multiplier at its root and two numbers as its leaves. So, after the first state (sums) had read <i>"1+"</i>, it would eagerly construct an addition object with the left argument being <i>1</i>, and the right argument being the result of the rest of the parse! So it reads <i>"2*2"</i> and builds a tree that becomes the right-side argument for our <i>"1+"</i> tree.<br />
<br />
This solution is theoretically consistent with building a language using flex and bison. The source can be found <a href="https://gist.github.com/4112114#file_trivial_parser.cpp">here (gist)</a>. But it isn't functional. That's when I stumbled back onto this research paper and decided to give a real go at it.<br />
<br />
<br />
<h4>
Functional Parsing.</h4>
The parser described in this paper does not act in the same way as my parser does. Rather than lexing, tokenizing, and building an expression tree, it cuts straight to the point. A <i>Parser<int></i> will be a function that accepts a <i>string</i> and produces <i>int</i>s. But, the job may not be done; there may be more to parse. For every <i>int</i> it parses, it will also store the suffix of the parse. So, if <i>p</i> is some parser of type <i>Parse<int></i>, <i>p("1abc")</i> will return a vector holding one match: the value, <i>1</i>, and the suffix, <i>"abc"</i>.<br />
<br />
What does this class look like?<br />
<br />
<pre class="brush: cpp">/* A Parser is a function taking a string and returning a vector of matches. */
template< class X > struct Parser {
// The value is the target of the parse. (For example "1" may parse to int(1).
using value_type = X;
// A match consists of a value and the rest of the input to process.
using parse_state = std::pair<X,std::string>;
// A parse results in a list of matches.
using result_type = std::vector< std::pair<X,std::string> >;
// A parser is a function that produces matches.
using function_type = std::function< result_type( const std::string& ) >;
function_type f;
Parser( function_type f ) : f(std::move(f)) { }
Parser( const Parser<X>& p ) : f(p.f) { }
Parser() { }
result_type operator () ( const std::string& s ) const {
return f( s );
}
};
template< class X, class F > Parser<X> parser( F f ) {
using G = typename Parser<X>::function_type;
return Parser<X>( G(std::move(f)) );
}</pre>
<br />
I have mentioned in previous articles that one should avoid <i>std::function</i> for efficiency reasons, but it vastly simplifies things here. As you can see, <i>Parser</i> merely wraps itself around <i>std::function</i>. I would encourage the reader to think of it as a type alias--not deserving of being called a new type, but more than a typedef.<br />
<br />
This is consistent with how the paper defines this type:<br />
<br />
<i>newtype Parser a = Parser (String -> [(a,String)])</i><br />
<br />
If nothing can be parsed, and empty list will be returned. If many things are parsed, a list of each match will be returned.<br />
<br />
<br />
<h4>
The Parser Monad.</h4>
As a reminder, the basic monadic operations are these:<br />
<br />
<i>a >> b = b -- Do a, then b.</i><br />
<i> a >>= f = b -- For each x from a, construct b with f(x).</i><br />
<i> mreturn x = a -- Construct a monad from a value.</i><br />
<br />
How does this relate to parsing? A parser is basically just a function, so if <i>p</i> and <i>q</i> are both parsers, <i>p >> q</i> must return a new parser, a new function. The simple explanation (do <i>p</i>, then <i>q</i>) is correct. First, <i>p</i> parses and let's say it returns a match, <i>(x,rest)</i>. <i>rest</i> is sent to <i>q</i> for parsing and the <i>x</i> is thrown away. It may sound odd to just throw away a value, but it will become more obvious soon.<br />
<br />
If <i>p</i> had failed to parse a value, then <i>q</i> would not have been run. <br />
<br />
The bind operation, <i>p >>= f</i>, extracts the parsed value, <i>x</i>, from <i>p</i> and creates a new parser from <i>f(x)</i>. <i>mreturn x</i> creates a new parser that returns <i>x</i> as its value. It accepts any string, even an empty one. Ideally, <i>x</i> came from the output of another parser.<br />
<br />
<i>p >> q -- Run parser p, then q.</i><br />
<i> p >>= f -- Construct a parser with p's matches.</i><br />
<i> mreturn x -- Construct a parser that returns x.</i><br />
<br />
We can define it like so:<br />
<br />
<pre class="brush: cpp">template< class X > struct Monad< Parser<X> > {
using Pair = typename Parser<X>::parse_state;
using R = typename Parser<X>::result_type;
/*
* mreturn x = parser(s){ vector( pair(x,s) ) }
* Return a parsed value. Forwards rest of input to the next parser.
*/
template< class M >
static M mreturn( X x ) {
return parser<typename M::value_type> (
[x]( const std::string& s ) {
return R{ Pair(std::move(x),s) };
}
);
}
/* a >> b = b */
template< class Y, class Z >
template< class Y, class Z >
static Parser<Y> mdo( Parser<Z> a, Parser<Y> b ) {
return a >>= [b]( const Z& z ) { return b; };
}
/* Continue parsing from p into f. */
template< class F, class Y = typename std::result_of<F(X)>::type >
static Y mbind( F f, Parser<X> p )
{
using Z = typename Y::value_type;
return parser<Z> (
[f,p]( const std::string& s ) {
// First, extract p's matches.
return p(s) >>= [&]( Pair p ) {
// Then construct the new parser from the p's output.
// Continue parsing with the remaining input with the new parser.
return f( std::move(p.first) )( std::move(p.second) );
};
}
);
}
};</pre>
<br />
Do not worry if this source is difficult to understand. It is more important to understand how it is used (which is perhaps common with monads). Note that <i>mdo</i> is defined such that for every successful parse of <i>a</i>, <i>b</i> is parsed. If <i>a</i> fails to parse anything, <i>b</i> fails, too.<br />
<br />
These monadic operations are the core building blocks from which we can build more complex system, but the paper also discusses <i>MonadZero</i> and <i>MonadPlus</i>. They are both type classes, like <i>Monad</i>, but extend it to do a few interesting things. In C++, one can concatenate two string by using simple addition: <i>s1 + s2 = s12</i>. <i>MonadPlus</i> is generalization of this. <i>MonadZero</i> completes this generalization by supplying the additive identity. For example, we know that <i>zero + x = x</i>. Thus, <i>"" + s = s</i>.<br />
<br />
In parsing terms, <i>zero</i> would refer to a parser that matches nothing and adding two parsers, <i>p+q</i>, will produce a third parser that accepts either<i> p</i>'s or <i>q</i>'s. For example, <i>itentifier+number</i> would create a function that parses either identifiers or numbers.<br />
<br />
We can define <i>MonadPlus</i> and <i>MonadZero</i> in the same way we would define <i>Monad</i>.<br />
<br />
<pre class="brush: cpp">template< class ... > struct MonadZero;
template< class ... > struct MonadPlus;
template< class M, class Mo = MonadZero< Cat<M> > >
auto mzero() -> decltype( Mo::template mzero<M>() )
{
return Mo::template mzero<M>();
}
template< class A, class B, class Mo = MonadPlus<Cat<A>> >
auto mplus( A&& a, B&& b ) -> decltype( Mo::mplus(std::declval<A>(),std::declval<B>()) )
{
return Mo::mplus( std::forward<A>(a), std::forward<B>(b) );
}
template< class X, class Y >
auto operator + ( X&& x, Y&& y ) -> decltype( mplus(std::declval<X>(),std::declval<Y>()) )
{
return mplus( std::forward<X>(x), std::forward<Y>(y) );
}</pre>
<br />
First, we define these for sequences.<br />
<br />
<pre class="brush: cpp">template<> struct MonadZero< sequence_tag > {
/* An empty sequence. */
template< class S >
S mzero() { return S{}; }
};
/* mplus( xs, ys ) = "append xs with ys" */
template<> struct MonadPlus< sequence_tag > {
template< class A, class B >
static A mplus( A a, const B& b ) {
std::copy( b.begin(), b.end(), std::back_inserter(a) );
return a;
}
};</pre>
<br />
And then for Parsers.<br />
<br />
<pre class="brush: cpp">/* mzero: a parser that matches nothing, no matter the input. */
template< class X > struct MonadZero< Parser<X> > {
template< class P >
static P mzero() {
return parser<X> (
[]( const std::string& s ){
return std::vector<std::pair<X,std::string>>();
}
);
}
};
/* mplus( pa, pb ) = "append the results of pa with the results of pb" */
template< class X > struct MonadPlus< Parser<X> > {
using P = Parser<X>;
static P mplus( P a, P b ) {
return parser<X> (
[=]( const std::string& s ) { return a(s) + b(s); }
);
}
};</pre>
<br />
Since we usually only want the first successful parse, the paper define an operator, <i>+++</i>, that does this.<br />
<br />
<pre class="brush: cpp">template< class X >
Parser<X> mplus_first( const Parser<X>& a, const Parser<X>& b ) {
return parser<X> (
[=]( const std::string s ) {
using V = std::vector< std::pair< X, std::string > >;
V c = (a + b)( s );
return c.size() ? V{ c[0] } : V{};
}
);
}</pre>
<br />
This completes the required building blocks. A parser of significant complexity could be made using only the above functions and types. The paper describes building a notably simple parser, so let's do that instead!<br />
<br />
<br />
<h4>
Basic Monadic Parsers.</h4>
The simplest parser is <i>item</i>, which accepts any char.<br />
<br />
<pre class="brush: cpp">std::string tail( const std::string& s ) {
return std::string( std::next(s.begin()), s.end() );
}
/*
* Unconditionally match a char if the string is not empty.
* Ex: item("abc") = {('a',"bc")}
*/
auto item = parser<char> (
[]( const std::string& s ) {
using Pair = Parser<char>::parse_state;
using R = Parser<char>::result_type;
return s.size() ? R{ Pair( s[0], tail(s) ) } : R();
}
);</pre>
<br />
To demonstrate its usage, the paper defines a simple parser that takes a string of length three or more and returns the first and third values.<br />
<br />
<pre class="brush: cpp">auto p = item >>= []( char c ) {
return item >> item >>= [c]( char d ) {
return mreturn<Parser>( std::make_pair(c,d) );
};
};</pre>
<br />
<i>p</i> first runs <i>item</i> to extract <i>c</i>, then runs <i>item</i> again but throws away the value. It runs <i>item</i> a third time to extract <i>d</i> and finally returns as its value <i>(c,d)</i>. <i>p("abcd")</i> would return <i>{(('a','c'),"d")}</i>.<br />
<br />
The next function creates a parser that is a little more helpful:<br />
<br />
<pre class="brush: cpp">/* sat( pred ) = "item, if pred" */
template< class P >
Parser<char> sat( P p ) {
return item >>= [p]( char c ) {
return p(c) ? mreturn<Parser>(c) : mzero<Parser<char>>();
};
}</pre>
<br />
Given some function that operates on chars, this extracts an <i>item</i>, but then checks the condition without consuming any additional input. If <i>p(c)</i> returns true, then it returns a parser with the value <i>c</i>, otherwise <i>zero</i>, a failed parse. Using this, we can define a parser to accept only a specific char.<br />
<br />
<pre class="brush: cpp">Parser<char> pchar( char c ) {
return sat( [=](char d){ return c == d; } );
}</pre>
<br />
And then, a parser that accepts only a specific string.<br />
<br />
<pre class="brush: cpp">Parser<std::string> string( const std::string& s ) {
if( s.size() == 0 )
return mreturn<Parser>( s );
Parser<char> p = pchar( s[0] );
for( auto it=std::next(s.begin()); it != s.end(); it++ )
p = p >> pchar( *it );
return p >> mreturn<Parser>(s);
}</pre>
<br />
<i>Note: There is no name conflict with std::string because the source does not contain "using namespace std;".</i><br />
<br />
This function does something very odd. For every <i>char</i> in <i>s</i>, it chains together char parsers. For example, <i>string("abc")</i> would return a parser equivalent to <i>pchar('a') >> pchar('b') >> pchar('c')</i> >> <i>mreturn<Parser>("abc")</i>. If any of the char parsers fail down the line, <i>mreturn<Parser>(s)</i> will fail. Since we already know the values of the successful parses, their values are thrown away.<br />
<br />
Though faithful to the paper, this may be less efficient than desirable. One could implement <i>string</i> in this way, too:<br />
<br />
<pre class="brush: cpp">Parser<std::string> string( const std::string& str ) {
return parser<std::string> (
[str]( const std::string& s ) {
using R = typename Parser<std::string>::result_type;
if( std::equal(str.begin(),str.end(),s.begin()) ) {
return R {
{ str, s.substr( str.size() ) }
};
} else {
return R();
}
}
);
}</pre>
<br />
It can at times be simpler to write out these functions instead of composing them, however, that can be thought of as an optimization.<br />
<br />
The next function creates a new parser from a parser, <i>p</i>, that accepts one or zero <i>p</i>'s.<br />
<br />
<pre class="brush: cpp">template< class X >
Parser<std::vector<X>> some( Parser<X> p ) {
using V = std::vector<X>;
using Pair = std::pair<V,std::string>;
using R = std::vector< Pair >;
using P = Parser<V>;
return mplus_first(
parser<V> (
[=]( const std::string& s ) {
auto matches = p(s);
return matches.size() == 0 ? R{}
: R{
Pair(
V{std::move(matches[0].first)},
std::move( matches[0].second )
)
};
}
),
mreturn<Parser>( V{} )
);
}</pre>
<br />
It was unfortunately difficult to translate, but does work. (The gist also contains a version called <i>many</i>, which accepts zero or many <i>p</i>'s.)<i> some</i> converts a parser of type <i>X</i> to one that produces a <i>vector</i> or <i>X'</i>s. It always succeeds, even if it does not successfully parse anything.<br />
<br />
To create a parser that consumes whitespace is now trivial.<br />
<br />
<pre class="brush: cpp">template< class X >
using ManyParser = Parser< std::vector<X> >;
ManyParser<char> space = some( sat([](char c){return std::isspace(c);}) );</pre>
<br />
We require one more function to parse alternating sequences. Like what? A sum, like "1+2+3" is a sort of alternating sequence of numbers and +'s. A product is an alternating sequence of numbers and *'s.<br />
<br />
Here's the weird part: what does a parser that accepts a "+" return? What about a parser that accepts a "-"? The value of such a parser, as it turns out, is the binary function that it represents! In the case of the implementation below, this is a function pointer of type <i>int(*)(int,int)</i>.<br />
<br />
<pre class="brush: cpp">/*
* chain(p,op): Parse with p infinitely (until there is no match) folding with op.
*
* p and op are both parsers, but op returns a binary function, given some
* input, and p returns the inputs to that function. For example, if:
* input: "4"
* p returns: 4
* No operator is read, no operation is performed. But:
* input: "4+4"
* p returns: 4
* op is then parsed with the function, rest:
* input: "+4"
* op returns: do_add
* input: "4"
* p returns: 4
* rest returns: 8
* rest applies the operation parsed by op. It alternates between parsing p and
* op until there are no more matches.
*/
constexpr struct Chainl1 {
template< class X, class F >
static Parser<X> rest( const Parser<X>& p, const Parser<F>& op, const X& a ) {
// Alternate between op and p until input is consumed or a parse fails.
auto r = op >>= [=]( const F& f ) {
return p >>= [&]( const X& b ) {
return rest( p, op, f(a,b) );
};
};
// Return the first successful parse, or a if none.
return mplus_first( r, mreturn<Parser>(a) );
}
template< class X, class F >
Parser<X> operator () ( Parser<X> p, Parser<F> op ) const {
return p >>= closet( rest<X,F>, std::move(p), std::move(op) );
}
} chainl1{};</pre>
<br />
<br />
<h4>
Adding the final touches.</h4>
The paper describes a few generally useful functions for constructing parsers based off the above function. <i>space</i> (defined above) consumes whitespace. <i>token</i> consumes any trailing whitespace after parsing <i>p</i>.<br />
<br />
<pre class="brush: cpp">constexpr struct Token {
template< class X >
Parser<X> operator () ( Parser<X> p ) const {
return p >>= []( X x ) {
return space >> mreturn<Parser>(std::move(x));
};
}
} token{};</pre>
<br />
<i>symb</i> converts a string to a token.<br />
<br />
<pre class="brush: cpp">auto symb = compose( token, string ); // symb(s) = token( string(s) )</pre>
<br />
<i>apply</i> consumes any leading whitespace.<br />
<br />
<pre class="brush: cpp">constexpr struct Apply {
template< class X >
Parser<X> operator () ( Parser<X> p ) const {
return space >> std::move(p);
}
} apply{};</pre>
<br />
The big idea is that we never want to manually write a function that returns a list of successful parses. It's hard! It's much easier to compose such functions from smaller, more comprehensible ones and use those to build reasonable complex, but more simple to reason about, parsers.<br />
<br />
<h4>
The parser itself.</h4>
Now, using all of the tools provided, we can create a parser much more trivially than otherwise--though that is perhaps true of any time one has a new set of tools. First, we define a parser that accepts digits.<br />
<br />
<pre class="brush: cpp">constexpr bool is_num( char c ) {
return c >= '0' and c <= '9';
}
/* Parse one digit. */
Parser<int> digit = token( sat(is_num) ) >>= []( char i ) {
return mreturn<Parser>(i-'0');
};</pre>
<br />
Now, <i>digit("2")</i> returns <i>2</i>, but <i>(digit >> digit)("22")</i> returns <i>2</i> as well! Why? Because the first run of <i>digit</i> extracts the first <i>2</i>, but throws that value away and the second run of <i>digit</i> extracts the second <i>2</i>. To parse a two-digit number, we need something like this:<br />
<br />
<pre class="brush: cpp">Parser<int> twoDigit = digit >>= []( int x ) {
return digit >>= [x]( int y ) {
return mreturn<Parser>( x*10 + y );
};
};</pre>
<br />
It extracts the first then the second digit and returns the original number, converted from a string to an int! To parse arbitrarily long numbers (diverging from the paper's version), we can define a chain operation!<br />
<br />
<pre class="brush: cpp">int consDigit( int accum, int digit ) { return accum*10 + digit; }
Parser<int> num = chainl1( digit, mreturn<Parser>(consDigit) );</pre>
<br />
For every two digits parse, <i>num</i> calls <i>consDigit</i> to fold the values together. As mentioned earlier, <i>chainl1</i> works by alternating between its two parsers. Since the second argument is a parser which consumes no input, <i>num</i> only accepts <i>digit</i>s.<br />
<br />
Next, we can define the parsers for binary operations.<br />
<br />
<pre class="brush: cpp">// Binary operations of type int(*)(int,int).
int do_add( int x, int y ) { return x + y; }
int do_sub( int x, int y ) { return x - y; }
int do_mult( int x, int y ) { return x * y; }
int do_div( int x, int y ) { return x / y; }
auto addop = mplus (
pchar('+') >> mreturn<Parser>(do_add),
pchar('-') >> mreturn<Parser>(do_sub)
);
auto mulop = mplus (
pchar('*') >> mreturn<Parser>(do_mult),
pchar('/') >> mreturn<Parser>(do_div)
);</pre>
<br />
<i>addop</i> parses either a "+" or "-" and returns either <i>do_add</i> or <i>do_sub</i>, respectively. Because the parsers must return functions of the same types, <i>std::plus</i> and <i>std::minus</i> could not be used.<br />
<br />
With this, we can define a <i>term</i> as an alternating sequence of numbers and multiplications and divisions; and an <i>expr</i>(ession) as an alternating sequence of <i>term</i>s, +'s and -'s.<br />
<br />
<pre class="brush: cpp">/*
* Parse terms: series of numbers, multiplications and divisions.
* Ex: "1*3*2" -> (3,"*2") -> 6
*/
Parser<int> term = chainl1( num, mulop );
/*
* Parse expressions: series of terms, additions and subtractions.
* Ex: "1+7*9-1" -> (1."+7*9-1") -> (63,"-1") -> 62
*/
Parser<int> expr = chainl1( term, addop );</pre>
<br />
And we're done! We have just built a calculator! It can evaluate any expression of additions, subtractions, multiplications, divisions, and it is whitespace agnostic. While implementing <i>Parser</i> itself took a considerable amount of work, using it does not. <br />
<br />
<pre class="brush: cpp">int main() {
using std::cin;
using std::cout;
using std::endl;
cout << "Welcome to the calculator!\n"
<< "Press ctrl+d or ctrl+c to exit.\n"
<< "Type in an equation and press enter to solve it!\n" << endl;
std::string input;
while( cout << "Solve : " and std::getline(std::cin,input) ) {
auto ans = apply(expr)(input);
if( ans.size() and ans[0].second.size() == 0 )
cout << " = " << ans[0].first;
else
cout << "No answer.";
cout << endl;
}
}
</pre>
<br />
I highly encourage anyone reading this to attempt to compile and modify the source code.<br />
<br />
<b>See the gist at github for the source in full: </b><a href="https://gist.github.com/4112114">https://gist.github.com/4112114</a><br />
<b>And for the original parser I wrote: </b><a href="https://gist.github.com/4112114#file_trivial_parser.cpp">https://gist.github.com/4112114#file_trivial_parser.cpp</a>Splinter of Chaoshttp://www.blogger.com/profile/14715348728512729776noreply@blogger.com7tag:blogger.com,1999:blog-8638474063464489651.post-25314084047179672752012-11-15T12:33:00.000-08:002012-11-17T05:03:07.508-08:00A public letter to Microsoft about the CTP.Dear Microsoft,<br />
<br />
I see that you have decided to release C++11 in the form of a <a href="http://www.microsoft.com/en-us/download/details.aspx?id=35515">CTP</a>. Congratulations on catching up! For a long time, it seemed like you just didn't give a shit, but thankfully that's not true.<br />
<br />
I see you now offer variadic templates. Great! You know when GCC 4.3 came out offering back in 2008, and Clang 2.9 the April of last year, it was really amazing, so I'm glad to see you're offering this too now. It's really something that developers <strike>chained</strike> faithful to MSVC will finally be compiling code written with GCC years ago!<br />
<br />
Initializer lists, too!? Wow! How nostalgic. Like when GCC 4.4 came out in '09. Oh, but you <a href="http://stackoverflow.com/questions/12654394/initializer-list-not-working-with-vector-in-visual-studio-2012">can't use it for <i>std::vector</i></a>? How silly. Well, I'm sure it's just one of those things you'll get around to in the same way you finally got around to C++11. There are probably some major complexities that I am not aware of for integrating this feature into the standard library.<br />
<br />
You also now offer delegated constructors. It took GCC a long time to implement that! It only happened earlier this year. Clang didn't even have it until December 2011. Being only a year behind must feel like a major accomplishment. The whole team who implemented this should be promoted for their unusual efficiency.<br />
<br />
My, my, default template argument for functions was a huge oversight for the standardizing committee. It's a good thing they corrected themselves, allowing GCC to include it in their 4.3 release series. It's good to see you have also corrected yourselves by including it in your CTP. Developers of MSVC will finally be able to write<br />
<br />
<i>template< class X, class Y = type-exp ></i><br />
<i> Y f( X x ) {return Y(...); }</i><br />
<br />
instead of what<i> </i>we used to have to write:<br />
<br />
<i>template< class X ></i><br />
<i> type-exp(...) f( X x ) { return type-exp(...); }</i><br />
<br />
Remember how it took GCC until 4.5 (April 2010) to get explicit conversion operators in, and Clang until 3.0 in December 2011? Well now MSVC developers too can take advantage of the fact that allowing smart pointers to explicitly convert to <i>bools</i> won't allow them to then convert to <i>ints</i>!<br />
<br />
<i>std::unique_ptr<int> p( new int(6); );</i><br />
<i> if( p ) { } // Explicit conversion!</i><br />
<i> p + p; // Not allowed!</i><br />
<br />
Have I forgotten a feature to applaud you for? Oh yes, raw string literals! Good job. Not the first here, either, but hey, it's not like you're competing for market share, right? After all, you already got it. Developers are locked in, aren't they? They <i>have</i> do use MSVC to develop professionally for Windows, right? Isn't that the idea?<br />
<br />
Now, it would be absolutely horrible if you released all these features that the rest of the C++ community has been using for years and none of your customers knew any of it. So how wonderful it is that you have <a href="http://channel9.msdn.com/">channel 9</a> where you have been publishing videos like Scott Meyer's influential <a href="http://channel9.msdn.com/Shows/Going+Deep/Cpp-and-Beyond-2012-Scott-Meyers-Universal-References-in-Cpp11">Universal References</a> talk and Herb Sutter's (who works for you) talk on <a href="http://channel9.msdn.com/Events/Build/2012/2-005">the future of our langauge</a> and <a href="http://channel9.msdn.com/Series/C9-Lectures-Stephan-T-Lavavej-Core-C-/STLCCSeries6">Lavavej's core C++ course</a>. Your parade of C++ intellectuals surely adds much to the discussion. I wonder: did they use your CTP to research those talks, GCC, or Clang?<br />
<br />
So thank you, Microsoft. Thanks for holding the C++ community back several years. Thanks for offering educational content on these features, now that you finally decided to implement them. Thanks for parading all the C++ intellectuals in our faces to get us excited about these features, now that you offer them. Thank you for showing that you really care about advancements in C++, in your products. Thank you for demonstrating that you care about the advancement of the C++ community at-large, when they're using MSVC. Thank you for dishing out pieces of C++11 like chocolate rations. And thank you for hiring Herb Sutter back in 2002, which obviously made you hurry much faster in releasing C++11 features.<br />
<br />
Thank you for being you. Without you, we'd all be programming in C++11! That would be insanity.Splinter of Chaoshttp://www.blogger.com/profile/14715348728512729776noreply@blogger.com14tag:blogger.com,1999:blog-8638474063464489651.post-5225219842417271402012-11-15T11:05:00.001-08:002012-12-14T18:09:27.412-08:00Generalized Function Objects Explained (For the C++11 unfluent.)It occurred to me that some of those who found my "<a href="http://yapb-soc.blogspot.com/2012/11/generic-function-objects-or.html">Generic Function Objects</a>" article were not all that familiar with C++11 syntax, and in particular, <a href="http://publib.boulder.ibm.com/infocenter/lnxpcomp/v8v101/index.jsp?topic=%2Fcom.ibm.xlcpp8l.doc%2Flanguage%2Fref%2Ftemplate_template_arguments.htm">template template parameters</a> and <a href="http://www.generic-programming.org/~dgregor/cpp/variadic-templates.html">variadic templates</a>. Learning these concepts is one thing--just takes some googling, reading, and a little practice. Understanding examples like in my article may be more difficult, so I will explain how it works line-by-line. I will <b>not</b> be explaining these features, or how and why they work as that is too much for one blog post to accomplish, but I will be explaining how they work in context.<br />
<br />
Note that if one understood the article up to the definition of <i>ConstructT</i>, then feel free to skip to that section.<br />
<br />
<b>Update:</b> I have attempted to improve the original article with some of this one, so it may be a little redundant. <br />
<br />
<br />
<h4>
Binary</h4>
Binary defines member functions for <a href="http://en.wikipedia.org/wiki/Partial_application">partially applying</a> the first and last arguments. I <a href="http://yapb-soc.blogspot.com/2012/09/partial-application-in-c.html">wrote about this</a> back in September.<br />
<br />
<pre class="brush: cpp">template< class F, class X >
struct Part {
F f;
X x;
template< class _F, class _X >
constexpr Part( _F&& f, _X&& x )
: f(forward<_F>(f)), x(forward<_X>(x))
{
}
template< class ... Xs >
constexpr auto operator () ( Xs&& ...xs )
-> decltype( f(x,declval<Xs>()...) )
{
return f( x, forward<Xs>(xs)... );
}
};
template< class F, class X > struct RPart {
F f;
X x;
template< class _F, class _X >
constexpr RPart( _F&& f, _X&& x )
: f( forward<_F>(f) ), x( forward<_X>(x) ) { }
template< class ...Y >
constexpr decltype( f(declval<Y>()..., x) )
operator() ( Y&& ...y ) {
return f( forward<Y>(y)..., x );
}
};
template< class F > struct Binary {
template< class X >
constexpr Part<F,X> operator () ( X x ) {
return Part<F,X>( F(), move(x) );
}
template< class X >
constexpr RPart<F,X> with( X x ) {
return RPart<F,X>( F(), move(x) );
}
};</pre>
<br />
So, given some type, <i>F</i>, inherited from <i>Binary<F></i>, <i>F</i> is defined as taking one argument (partial application), and calling the member function <i>with</i> (reverse-partial application). <i>F</i> must be a type that defines a two-argument <i>operator()</i> overload. <br />
<br />
We define a derivation like so:<br />
<br />
<pre class="brush: cpp">constexpr struct Add : public Binary<Add> {
using Binary<Add>::operator();
template< class X, class Y >
constexpr auto operator () ( X&& x, Y&& y )
-> decltype( std::declval<X>() + std::declval<Y>() )
{
return std::forward<X>(x) + std::forward<Y>(y);
}
} add{}; </pre>
<br />
<br />
<i>Add</i>, is not a function in the C sense, it's a function type. <i>add</i> is an instance of <i>Add</i> and is a function object. <i>Add</i> does not inherit <i>Binary's operator()</i> overload by default, so we explicitly tell it to by saying "<i>using Binary<Add>::operator()</i>". <i>Binary's</i> <i>with</i> requires no work to inherit.<br />
<br />
<i>auto inc = add(1); // Calls Binary<Add>::operator(); returns Part<Add,int>.</i><br />
<i> auto dec = add.with(-1); // Calls Binary<Add>::with; returns RPart<Add,int>. </i><br />
<i> int two = inc(1); // Calls Part<Add,int>::operator(); returns add(1,1).</i><br />
<i> int one = dec(two); // returns add(2,-1).</i><br />
<i> add(1,2) -- Calls Add::operator(); returns int(3).</i><br />
<br />
<br />
<h4>
<i>Chainable</i>.</h4>
<br />
<pre class="brush: cpp">template< class F > struct Chainable : Binary<F> {
using Binary<F>::operator();
template< class X, class Y >
using R = typename std::result_of< F(X,Y) >::type;
// Three arguments: unroll.
template< class X, class Y, class Z >
constexpr auto operator () ( X&& x, Y&& y, Z&& z )
-> R< R<X,Y>, Z >
{
return F()(
F()( std::forward<X>(x), std::forward<Y>(y) ),
std::forward<Z>(z)
);
}
template< class X, class Y, class ...Z >
using Unroll = typename std::result_of <
Chainable<F>( typename std::result_of<F(X,Y)>, Z... )
>::type;
// Any more? recurse.
template< class X, class Y, class Z, class H, class ...J >
constexpr auto operator () ( X&& x, Y&& y, Z&& z, H&& h, J&& ...j )
-> Unroll<X,Y,Z,H,J...>
{
// Notice how (*this) always gets applied at LEAST three arguments.
return (*this)(
F()( std::forward<X>(x), std::forward<Y>(y) ),
std::forward<Z>(z), std::forward<H>(h), std::forward<J>(j)...
);
}
}; </pre>
<br />
<i>Chainable</i> works in much the same way as <i>Binary</i> does, except that it extends <i>F</i> to take any arbitrary number of arguments (except for zero, but that'd be pretty useless anyway). Redefining <i>Add</i> to be <i>Chainable</i> is not hard.<br />
<br />
<pre class="brush: cpp">constexpr struct Add : public Chainable<Add> {
using Chainable<Add>::operator();
template< class X, class Y >
constexpr auto operator () ( X&& x, Y&& y )
-> decltype( std::declval<X>() + std::declval<Y>() )
{
return std::forward<X>(x) + std::forward<Y>(y);
}
} add{}; </pre>
<br />
Only two lines have changed, however <i>add</i>'s behaviour has changed dramatically.<br />
<br />
<i>int x = add(1,2,3); // Calls Chainable<Add>::operator(X,Y,Z); returns (1+2)+3.</i><br />
<i> int y = add(1,2,3,4,5,6); // Calls Chainable<Add>::operator(X,Y,Z,H,J...). </i><br />
<i> auto inc = add(1); // Still calls Binary<Add>::operator().</i><br />
<i><br />
</i> The three arg version of <i>Chainable<Add></i> calls <i>add( add(x,y), z )</i>, while the variadic version calls <i>itself( add(x,y), z, h, j... )</i>. It reduces the number of arguments to process by one, being supplied at least four. It is therefore never the base case and always ends by calling its three-arg version.<br />
<br />
<br />
<h4>
<i>ConstructT</i>.</h4>
<br />
<pre class="brush: cpp">template< template<class...> class T > struct ConstructT {
template< class ...X, class R = T< typename std::decay<X>::type... > >
constexpr R operator () ( X&& ...x ) {
return R( forward<X>(x)... );
}
};</pre>
<br />
The first template argument may be confusing to those who have not seen them before. We could write <i>template<class> class T</i>, and that would expect a <i>T</i> that took one type parameter. It may actually take one, zero, or several. <i>template<class...> class T</i> is the generic way to say "we know <i>T</i> takes type parameters, but we don't know how many (and it doesn't yet matter)". It might be <i>std::vector</i>, which can take two; the value type and allocator. It might be <i>std::set</i>, which can take three. (Though, neither <i>vector</i> nor <i>set</i> would work for <i>ConstructT</i>.)<br />
<br />
We deduce the return type using the <a href="http://publib.boulder.ibm.com/infocenter/lnxpcomp/v8v101/index.jsp?topic=%2Fcom.ibm.xlcpp8l.doc%2Flanguage%2Fref%2Fdefault_args_for_templ_params.htm">default template parameter</a>, <i>R</i>. <a href="http://en.cppreference.com/w/cpp/types/decay"><i>std::decay</i></a> transforms the type such that if we pass in an <i>int&</i>, we get an <i>int</i> back.<br />
<br />
<i>typename std::decay<const int&>::type = int</i><br />
<i>typename std::decay<int&&>::type = int</i><br />
<i>typename std::decay<int>::type = int</i><br />
<i></i><br />
So, if <i>T=std::pair</i> and we pass in an <i>int&</i> and <i>std::string&&</i>, <i>R = std::pair<int,std::string></i>.<br />
<br />
<i>ConstructT</i> perfectly forwards the arguments to <i>T</i>'s constructor, but decays the types to ensure that it holds the actual values.<br />
<br />
Just the same as with <i>add</i> and <i>Add</i>, we must create an instance of <i>ConstructT</i> to call it.<br />
<br />
<i>constexpr auto make_pair = ConstructT<std::pair>();</i><br />
<br />
This version of <i>make_pair</i> behaves just like <a href="http://en.cppreference.com/w/cpp/utility/pair/make_pair"><i>std::make_pair</i></a>. Its type is equivalent to this:<br />
<br />
<pre class="brush: cpp">template< struct ConstructPair {
template< class ...X, class R = std::pair< typename std::decay<X>::type... > >
constexpr R operator () ( X&& ...x ) {
return R( forward<X>(x)... );
}
};</pre>
<br />
<h4>
Conclusion.</h4>
C++11 is very big and no one feature is incredibly useful, but in conjunction they build up a powerful synergy. If these features had been a part of the language to begin with, there is no question that they would be considered required reading, just as much as <i>std::vector</i> and the <i><algorithm></i> library are today. They solve problems that have led to years frustration in developers. Perhaps one of the reasons many hate C++ is because of the lacking of many of these features. Becoming fluent in them makes C++ a much nicer language to speak and communicate with.Splinter of Chaoshttp://www.blogger.com/profile/14715348728512729776noreply@blogger.com2tag:blogger.com,1999:blog-8638474063464489651.post-9843574436814287072012-11-13T09:48:00.003-08:002012-11-13T12:11:01.296-08:00The Object-Function Paradox.Previously, I discussed using inheritance to simplify the writing of generic functions. However, it occurred to me that I can't find anyone else advocating the same thing. No one has yet told me it's a bad practice, but I can't find any other examples of it on the web. (If anyone knows of one, please relieve me of my ignorance.) It's a very short amount of code. Obvious and intuitive. So what impedes people from thinking of it? For that matter, why do some have difficulties comprehending it?<br />
<br />
C++ programmers seem to think of functions as something very distinct from objects and types, so obviously the OOP principles of code reuse don't apply. Indeed, the standard says in the first sentence under "The C++ Object Model" (1.8)<br />
<br />
<blockquote class="tr_bq">
The constructs in a C++ program create, destroy, refer to, access, and manipulate objects. An object is a region of storage. [ Note: A function is not an object, regardless of whether or not it occupies storage in the way that objects do. — end note ]</blockquote>
This is technically correct. Functions cannot be objects, they're more like a lexical representation of a code section. The problem is that objects can <i>look</i> like functions, but if we think of this object as a function, then we contradict ourselves because functions cannot be objects.<br />
<br />
The distinction is not so transparent. We can make objects that look, talk, and feel like functions, but they aren't.<br />
<br />
So, when working out a problem, if a programmer notices a repetitious pattern, s/he might say "I'll create another function that executes the common code!" However, the repetition moves from the in-lined pattern to the calling convention. Because this programmer believes functions are not objects, the solution of using inheritance is impossible to cogitate. Because of this fallacy, this programmer may find the solution hard to comprehend.<br />
<br />
We end up with this train of thought: Objects can be functions, but functions cannot be objects, and therefore objects are not functions.<br />
<br />
What does a function do? It executes some arbitrary code, taking in arbitrary parameters and returning some value. Function-objects do just that! So objects can be functions! But functions aren't objects...<br />
<br />
Paradoxes stunt innovation by preventing the problem from being worked out in a logical way. One cannot conceive of how to use OOP-features to implement functions if they believe the two are fundamentally different and incompatible.<br />
<br />
Take <a href="http://en.wikipedia.org/wiki/Zeno%27s_paradoxes">Zeno's famous paradox</a>. At one time, we all believe "all is one". We thought of things as wholes. Zeno seemed to think that odd and pointed out that one can go half way from A to B, and then half way to B from there, but he would never get to B. This is obviously true, but with the assumption that "all is one", we have to think about how, then, can we never reach B since we should eventually be one away? The invention of calculus gave way for mathematicians to properly reason about infinitesimally small ones, and a whole new field of science opened up. Our first physicist was born.<br />
<br />
<a href="http://en.wikipedia.org/wiki/Russell%27s_paradox">Russle's Paradox</a> led to several attempts at improving set theory to remove the paradox, such as the invention of <a href="http://en.wikipedia.org/wiki/Type_theory">Type Theory</a>, which is obviously very important to computer science. <br />
<br />
In quantum mechanics, <a href="http://en.wikipedia.org/wiki/Schr%C3%B6dinger%27s_cat_paradox">Schrodinger's Cat</a> and <a href="http://en.wikipedia.org/wiki/Bell%27s_theorem">Bell's Theorem</a> led to a greater understanding of the field. Even today, many first learn of quantum mechanics through the tale of an alive and dead cat.<br />
<br />
If I was more knowledgeable of the histories of math and physics, I might know more examples, but I hope my point has been persuasive: Paradoxical thinking holds us back, as individuals, as a community.<br />
<br />
I submit to the reader this: Functions are objects. Objects are functions. A function is not a procedure to call, but the pointer to that procedure, and pointers are objects, or it can be an instantiation of a user-defined type (like <i>ConstructChainable</i>). We should think of C-functions as the fundamentally different and incompatible things. They are low-level. Function objects are first-class citizens.<br />
<br />
The hope is that by removing this logical inconsistency in our minds, we can explore areas of possibility that we never imagined. We can invent solutions to problems that before would have caused a cognitive dissonance. We can progress in directions we hadn't realized before. We can really discover something really interesting, but only if we keep an open mind.Splinter of Chaoshttp://www.blogger.com/profile/14715348728512729776noreply@blogger.com1tag:blogger.com,1999:blog-8638474063464489651.post-64828471455584083232012-11-11T07:00:00.002-08:002012-12-14T18:04:59.739-08:00Generic Function Objects, or Generalizing Associativity, Transitivity, and such abstractions over functions and types.So, I posted "<a href="http://yapb-soc.blogspot.com/2012/11/rethinking-stdbinaryfunction.html">Rethinking <i>std::binary_function</i></a>", which was this epiphany that instead of writing all these variadic operator types, I could actually define a base class and inherit its operator overloads. That lead to another realization: <a href="https://gist.github.com/4049946">https://gist.github.com/4049946</a>. What I am about to elaborate on below feels like a discovery rather than an invention. It's also only a few days old. I normally try to present something much more polished, but this is exciting enough to myself that I want to share it sooner rather than later.<br />
<br />
<h4>
Starting from the top.</h4>
In "Rethinking <i>std::binary_function</i>", I thought that all binary functions might benefit from being chained, but actually, not all can be. We can write <i>add(1,2,3)</i> and think of it as <i>(1+2)+3 = 6</i>, but what about <i><</i>, or <i>less</i>? <i>less(1,2,3) </i>would be equivalent to <i>(true < 3)</i>, which is only incidentally correct. And so chaining might not be good for all binary functions, but we can at <i>least</i> say that <a href="http://yapb-soc.blogspot.com/2012/09/partial-application-in-c.html">partially applying</a> the front and back might be useful.<br />
<br />
<pre class="brush: cpp">template< class F, class X >
struct Part {
F f;
X x;
template< class _F, class _X >
constexpr Part( _F&& f, _X&& x )
: f(forward<_F>(f)), x(forward<_X>(x))
{
}
template< class ... Xs >
constexpr auto operator () ( Xs&& ...xs )
-> decltype( f(x,declval<Xs>()...) )
{
return f( x, forward<Xs>(xs)... );
}
};
template< class F, class X > struct RPart {
F f;
X x;
template< class _F, class _X >
constexpr RPart( _F&& f, _X&& x )
: f( forward<_F>(f) ), x( forward<_X>(x) ) { }
template< class ...Y >
constexpr decltype( f(declval<Y>()..., x) )
operator() ( Y&& ...y ) {
return f( forward<Y>(y)..., x );
}
};
template< class F > struct Binary {
template< class X >
constexpr Part<F,X> operator () ( X x ) {
return Part<F,X>( F(), move(x) );
}
template< class X >
constexpr RPart<F,X> with( X x ) {
return RPart<F,X>( F(), move(x) );
}
};</pre>
<br />
We might define a subtraction type like so:<br />
<br />
<br />
<pre class="brush: cpp">constexpr struct Sub : public Binary<Sub> {
using Binary<Sub>::operator();
template< class X, class Y >
constexpr auto operator () ( X&& x, Y&& y )
-> decltype( std::declval<X>() - std::declval<Y>() )
{
return std::forward<X>(x) - std::forward<Y>(y);
}
} sub{}; </pre>
<div>
<br /></div>
<br />
<i>sub</i> is a function object of type <i>Sub</i> and inherits the partial- and reverse partial-application from <i>Binary<Sub>.</i><br />
<i><br /></i>
<i> constepxr auto twoMinus = sub(2); // Calls Binary<Sub>::operator(int); returns Part<Sub,int>.</i><br />
<i> constexptr int zero = twoMinus(2); </i><br />
<i> </i><br />
<i> constexpr auto minusTwo = sub.with(2); // Calls Binary<Sub>::with(int); returns RPart<Sub,int>;</i><br />
<i> constexpr int two = minusTwo(4);</i><br />
<i><br /></i>
<i> constexpr int one = sub(2,1); // Calls Sub::operator(int,int).</i><br />
<br />
<h4>
Associativity and Transitivity.</h4>
Addition is an associative operation, by which I mean<br />
<br />
<i>x + y + z = (x+y) + z</i><br />
<br />
In particular, this demonstrates left (left-to-right) associativity. We can apply this same principle of associativity to functions!<br />
<br />
<br />
<pre class="brush: cpp">template< class F > struct Chainable : Binary<F> {
using Binary<F>::operator();
template< class X, class Y >
using R = typename std::result_of< F(X,Y) >::type;
// Three arguments: unroll.
template< class X, class Y, class Z >
constexpr auto operator () ( X&& x, Y&& y, Z&& z )
-> R< R<X,Y>, Z >
{
return F()(
F()( std::forward<X>(x), std::forward<Y>(y) ),
std::forward<Z>(z)
);
}
template< class X, class Y, class ...Z >
using Unroll = typename std::result_of <
Chainable<F>( typename std::result_of<F(X,Y)>, Z... )
>::type;
// Any more? recurse.
template< class X, class Y, class Z, class H, class ...J >
constexpr auto operator () ( X&& x, Y&& y, Z&& z, H&& h, J&& ...j )
-> Unroll<X,Y,Z,H,J...>
{
// Notice how (*this) always gets applied at LEAST three arguments.
return (*this)(
F()( std::forward<X>(x), std::forward<Y>(y) ),
std::forward<Z>(z), std::forward<H>(h), std::forward<J>(j)...
);
}
};</pre>
<br />
One might define an addition type like this:<br />
<br />
<br />
<pre class="brush: cpp">constexpr struct Add : public Chainable<Add> {
using Chainable<Add>::operator();
template< class X, class Y >
constexpr auto operator () ( X&& x, Y&& y )
-> decltype( std::declval<X>() + std::declval<Y>() )
{
return std::forward<X>(x) + std::forward<Y>(y);
}
} add{}; </pre>
<div>
<br /></div>
<div>
<i>Add</i> inherits <i>Binary<Add>'s</i> ::<i>operator()</i> and ::<i>with</i>, and <i>Chainable<Add></i>'s overloads as well. The three argument overload in <i>Chainable</i> will return <i>add(add(x,y),z)</i>. If given four or more arguments, it will call <i>add</i> on the first two, and call itself on the rest, being at least three. The base case for <i>Chainable</i> will always be the three-argument overload.</div>
<br />
<br />
<i>consexpr auto plusTwo = add(2); // Calls Binary<Add>::operator(int); returns Part<Add,int>.</i><br />
<i> constexpr int fifteen = add(1,2,3,4,5) // Calls Chainable<Add>::operator(int,int,int,int...).</i><br />
<br />
Operations like <i>less</i>, as mentioned above, are not associative. They are, however, transitive, by which I mean:<br />
<br />
<i>x < y < z = (x<y) and (y<z)</i><br />
<br />
In writing a class for transitivity, we can have it just apply each argument to the one right of it, but we need this <i>and</i> to glue the results together. My <i>Transitive</i> class will require both defining the operation (<i><</i>) and the function that folds the results together (<i>and</i>).<br />
<br />
<pre class="brush: cpp">template< class F, class Fold > struct Transitive : Binary<F> {
using Binary<F>::operator();
template< class X, class Y, class Z >
constexpr auto operator () ( X&& x, Y&& y, Z&& z )
-> typename std::result_of<F(X,Y)>::type
{
return Fold() (
F()( forward<X>(x), forward<Y>(y) ),
F()( forward<Y>(y), forward<Z>(z) )
);
}
template< class X, class Y, class Z, class A, class ...B >
constexpr auto operator () ( X&& x, Y&& y, Z&& z, A&& a, B&& ...b )
-> typename std::result_of<F(X,Y)>::type
{
return Fold() ( F()( forward<X>(x), forward<Y>(y) ),
F()( forward<Y>(y), forward<Z>(z),
forward<A>(a), forward<B>(b)... ) );
}
}; </pre>
<br />
We can define <i>less</i> like so:<br />
<br />
<pre class="brush: cpp">struct And : Chainable<And> {
using Chainable<And>::operator();
template< class X, class Y >
constexpr auto operator () ( X&& x, Y&& y )
-> decltype( declval<X>() && declval<Y>() )
{
return forward<X>(x) && forward<Y>(y);
}
};
constexpr struct Less : Transitive<Less,And> {
using Transitive<Less,And>::operator();
template< class X, class Y >
constexpr bool operator() ( X&& x, Y&& y ) {
return forward<X>(x) < forward<Y>(y);
}
} less{};</pre>
<br />
Now, writing <i>less(1,2,3)</i> would be equivalent to writing <i>1<2 and 2<3</i>.<i> less.with(3)</i> would be a function returning whether <i>x</i> is "less than three".<br />
<br />
Here's where things get really interesting. What if I want to use these properties of associativity and transitivity to construct types?<br />
<br />
<br />
<h4>
From the top, again.</h4>
The biggest problem is that types are not functions. Sure, we can write <i>std::pair<X,Y>(x,y)</i> and you might say "Look! A constructor! It's a function!", but we don't like needing to specify our types, so we prefer <i>std::make_pair(x,y)</i>. Even though we have this type with a construction function, it's not as useful as a general function. This is a surprisingly common pattern. We make a type, but we want the result's type to be argument dependent, so we make a function that constructs the type. And we do this for every type.<br />
<br />
There's <i>make_pair</i>, <i>make_tuple</i>, <i>make_shared</i>, <i>make_unique </i>(eventually), and many, many others. Instead of writing any of these, let's abstract the pattern away using our OOP principles!<br />
<br />
<pre class="brush: cpp">template< template<class...> class T > struct ConstructT {
template< class ...X, class R = T< typename std::decay<X>::type... > >
constexpr R operator () ( X&& ...x ) {
return R( forward<X>(x)... );
}
};</pre>
<br />
<i>std::decay</i> is a handy type that removes any reference or const until we get the type itself.<br />
<br />
<i>typename std::decay<const int>::type = int</i><br />
<i> typename std::decay<int&&>::type = int</i><br />
<i> typename std::decay<int>::type = int;</i><br />
<br />
When we call an instance of <i>ConstructT</i>, we may pass in references, r-values, or const values, but the return, <i>R</i>, will hold values.<br />
<br />
Now, we can write<br />
<br />
<i>constexpr auto make_pair = ConstructT<std::pair>();</i><br />
<i> std::pair<int,int> p = make_pair(1,2); </i><br />
<br />
<i> constexpr auto make_tuple = ConstructT<std::tuple>();</i><br />
<br />
Each of what used to be an entire function, a paragraph, is now a line. Now, for associativity!<br />
<i></i><br />
<pre class="brush: cpp">template< template<class...> class T >
struct ConstructChainable : Chainable<ConstructT<T>> {
using Self = ConstructChainable<T>;
using Chainable<ConstructT<T>>::operator();
template< class X >
using D = typename std::decay<X>::type;
template< class X, class Y, class R = T< D<X>, D<Y> > >
constexpr R operator () ( X&& x, Y&& y ) {
return R( forward<X>(x), forward<Y>(y) );
}
};</pre>
<br />
<i>ConstructChainable</i> inherits from <i>Chainable</i> which inherits from <i>Binary</i>, so now we can rewrite the above:<br />
<br />
<i>constexpr auto make_pair = ConstructChainable<std::pair>();</i><br />
<i> constexpr auto pairWith = make_pair.with(y); // Calls Binary<ConstructT<std::pair>>::operator(int).</i><br />
<i> constexpr auto pairPair = make_pair(1,2,3); // Returns an std::pair< std::pair<int,int>, int >.</i><br />
<br />
All this without having ever explicitly<i> </i>defining <i>make_pair</i>!<br />
<br />
Remember the definition of <i>Part</i> from above?<br />
<br />
<i>constexpr auto closet = ConstructChainable<Part>();</i><br />
<br />
This one line is very powerful. Let me demonstrate by showing all the code it replaces!<br />
<br />
<pre class="brush: cpp">// An extra, variadic definition of the class.
template< class F, class X1, class ...Xs >
struct Part< F, X1, Xs... >
: public Part< Part<F,X1>, Xs... >
{
template< class _F, class _X1, class ..._Xs >
constexpr Part( _F&& f, _X1&& x1, _Xs&& ...xs )
: Part< Part<F,X1>, Xs... > (
Part<F,X1>( forward<_F>(f), forward<_X1>(x1) ),
forward<_Xs>(xs)...
)
{
}
};
template< class F, class ...X >
constexpr Part<F,X...> closet( F f, X ...x ) {
return Part<F,X...>( move(f), move(x)... );
}</pre>
<br />
So what does<i> ConstructChainable</i> do? <b>It makes mole hills out of mountains!</b> What's more; it does so optimally. It perfect forwards the arguments all the way to the constructor, whereas I often would write moving functions in order to simplify type deduction.<br />
<br />
So, one might have noticed that <i>closet</i> creates a <i>Part</i>, but it's also derived from <i>Binary</i>, so we can write things like<br />
<br />
<i>constexpr auto cc = closet(closet);</i><br />
<br />
and all sorts of unexpected things!<br />
<i> </i><br />
<i> </i><br />
<h4>
Conclusion.</h4>
I guess one way to describe this phenomenon is perhaps as an update to the factory pattern. I'm tempted to call it a <i>type constructor</i>, but I'm not sure that's technically correct.<br />
<br />
This post has been more off-the-cuff than I normally try to be, and I haven't prepared any test source to ease in the process of learning. Still, I hope you find this as fascinating as I do.<br />
<br />
<b>Gist</b>: <a href="https://gist.github.com/4055136">https://gist.github.com/4055136</a><br />
<br />
I use this trick extensively in my library, Pure. <a href="https://github.com/splinterofchaos/Pure/blob/functions/Functional.h">(Functional.h)</a>Splinter of Chaoshttp://www.blogger.com/profile/14715348728512729776noreply@blogger.com0tag:blogger.com,1999:blog-8638474063464489651.post-1831077460669595832012-11-10T14:21:00.001-08:002012-12-14T18:06:21.436-08:00Highlights from the current ISO proposals.I thought I'd take some time to highlight some of the interesting proposals I read today after having stumbled on a link to them:<br />
<br />
<a href="http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2012/#mailing2012-11">http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2012/#mailing2012-11</a><br />
<br />
I've been waiting over four years for Concepts to become standard. It started to feel like Stroustrup had given up on the idea, but luckily <a href="http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2012/n3351.pdf">N3351 (pdf)</a> by him and Sutton shows there has been progress. It's not honestly the most fascinating read, but when concepts are finally approved, they will improve on <a href="http://www.boost.org/community/generic_programming.html#tag_dispatching">tag dispatch</a>. Since a lot of my code currently uses this technique, I look forward to being able to replace it. <br />
<br />
Really, really looking forward to it. We are forced to invent such witty ways of expressing generic functions with <i>enable_if</i>s and partial template specialization, delayed instantiation, but we <i>could</i> just define some abstract principles about types and use those, abondoning all of it. (I do want to point out that <a href="http://www.generic-programming.org/software/ConceptGCC/">ConceptGCC</a> and <a href="http://www.generic-programming.org/software/ConceptClang/">ConceptClang</a> do exist.)<br />
<br />
<a href="http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2012/n3418.pdf">N3418 (pdf)</a> talks about polymorphic lambdas and <a href="http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2012/n3386.html">N3386</a> talks about using <i>auto</i> as the return type for regular functions. The latter contains this line, which I think says it all:<br />
<br />
<i> "template <class T> auto g(T t) { return t; } // return type is deduced at instantiation time"</i><br />
<br />
<a href="http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2012/n3328.pdf">N3328</a> talks about "Resumable Functions" (continuations) and introduces a few new keywords to facilitate the tricky syntax. <br />
<br />
The Boost::option library is attempting to get in via <a href="http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2012/n3406.html">N3406</a>.<i> </i>The idea of Boost's option is very like that of <i>Maybe</i> in Haskell.<br />
<br />
<a href="http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2012/n3449.pdf">N3449</a> proposes a "Open and Efficient Type-Switch". Since this is a really foreign concept to C++, it'll be interesting to see what comes of it.<br />
<br />
<a href="http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2012/n3404.html">N3404</a> and <a href="http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2012/n3405.html">N3505</a>, both by a Mike Spertus, show some impressive "tidbits" with tuples and templates. The latter especially spoke to me; he shows an example of a templateted type, <i>Foo</i>, holding a function, <i>F</i>, and the example includes this:<br />
<br />
<i>Foo<???>( []{ ... } )</i><br />
<br />
What's the type of <i>F</i>? Lambdas have implementation-defined types, so we just don't know. He argues that the compiler can deduce the type here. We should be able to write <i>Foo([]{...})</i>!<br />
<br />
I'd like to extend this idea. If I wrote a function <i>g(f,x)=f(x,x)</i>, then I'd like <i>g(Foo,x)</i> to equal <i>Foo(x,x) </i>and <i>g(std::pair,x)=std::pair(x,x)</i>. As it is, I end up writing a function that constructs a <i>Foo</i> and call it <i>foo</i>.<br />
<br />
Unpacking tuples into functions (i.e. <i>apply(f,tuple(1,2,3))=f(1,2,3)</i>) has had some interesting discussion over the past few months. Solutions currently include constructing a variadic type of indexes and <i>std::get</i>. <a href="http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2012/n3326.pdf">N3326</a> asks why not do something much more general? I really don't want to do it injustice by trying to explain. It's a great read!<br />
<br />
<br />
There is plenty more I'm not mentioning: parallel programming, filesystem library, reflection, URI's, <i>std::range</i>, and modules. The proposals of today seem very different than the proposals of when I started programming. C++ feels like a language with a much stronger foundation, which lends itself to expansion, inspiration, and innovation. What will this round of standardization bring?Splinter of Chaoshttp://www.blogger.com/profile/14715348728512729776noreply@blogger.com1