Excerpts from the following content was used from the official press release by the leading New Jersey interactive agency Rosetta. This is yet another unique way that interactive agencies are using augmented reality to engage consumer experience. To be honest, it felt strange trying to align my tongue to the screen but it was fun trying to then capture the snowflakes for a good cause.

Snow Day by Rosetta Agency

Rosetta announced a breakthrough in augmented reality technology that debuted in the holiday card sent to Rosetta’s clients and friends. The card, which combines motion detection and facial recognition elements, challenges recipients to collect a million digital snowflakes with their tongues, which will in turn, benefit New Orleans youth.

For Rosetta’s holiday card, the agency developed an algorithm that is able to detect, as accurately as possible, when a person’s mouth is open and when his or her tongue is stuck out. According to Rosetta, this is the first time augmented reality technology has been able to distinguish a feature as intricate as a person’s tongue against the background of a human face. Rosetta created the motion detection and facial recognition technologies using flash and combined the elements in a way that allows them to work in sync and form a game.

When a recipient opens the card, he or she can play an interactive game that involves catching falling snowflakes. The game is designed to automatically launch a viewer’s webcam when the player hits “play.” The player is asked to stick out his or her tongue, and the new augmented reality technology tracks the player’s tongue movements with his or her head and shoulder movements. Each time a player’s tongue catches a snowflake on the screen, a point will be added to the automatically updated tally that shows how many snowflakes have been caught by all card recipients.

Good Cause

When one million snowflakes are captured, Rosetta will sponsor a “snow day” for New Orleans children who have never experienced a snow day. The event will take place in a New Orleans’ City Park on February 20th, 2010 and will feature games and activities, food and beverages and an appearance by “Frosty the Snowman.” The festivities will be capped by six inches of “real” snow.

Rosetta emailed the holiday card on December 15th to thousands of clients and friends who are encouraged to pass it along.

Try Out The Holiday Card

The card can also be viewed / downloaded at http://holiday.rosetta.com/snowday/.

Rosetta’s Creative and Customer Experience Team came up with the idea as a fun way to connect the holidays with this interactive game and the charity effort. Also, they worked with teams from each of Rosetta’s offices across the country to develop the game and “snow day” event. Overcoming the technical challenge of mapping and tracking an object as small as a human tongue was achieved through a collaborative effort by Rosetta’s technology team.

“This card represents some of the best of what Rosetta is all about – great creative ideas, cutting-edge technology, a deep understanding of social media, and a commitment to the larger community,” said Chief Creative Officer, Gary Scheiner.

“We came up with this idea, and then our tech guys figured out how to make it work,” added Toni Hess, Partner and Group Creative Director, who spearheaded the initiative. “It uses facial recognition software, which I don’t believe has ever been used like this before. The technology is invisible, which lets the fun of the idea take center stage.”