Overview.

Partnering with the Cannes Lions festival organizers, I led a team that designed and built a 12’ by 5’ multi-user, multi-touch wall to serve as the information and connection hub for the festival. The touchwall allowed festival delegates to explore 3D maps of Cannes and an expandable map of the festival hall. People could also send themselves emails about restaurants and bars in the surrounding area as well as directions from the venue. Along the top of the screen was a visualization of the entire week’s schedule. Each event was expandable into an event detail view and sharable via email. Additionally, a real-time indicator communicated which events were currently taking place. The touchwall also gave users the ability to connect socially. Users could search for other attendees by name and then send an email request to exchange information. Or, if they were standing side-by-side, they could exchange contact details on the spot. Perhaps the most striking and innovative element of the touchwall was the use of RFID technology. By embedding RFID tags in the delegate badges, the wall was able to identify and authenticate users as they approached and offer them a personalized and unique experience alongside others. The overall specs for the wall were quite impressive, with six projectors edge-blended for an overall resolution of 3840 x 1600. Three IR cameras were also edge-blended for the FTIR interface, 24 IR lasers, and four RFID antennas. The interface was built in Flex 4 and utilized purpose-built .NET applications to triangulate RFID tags across the four antennas, provide real-time updates using Flash Remoting, and provide a CMS platform.

Requirements.

The initial project brief was to produce an interactive experience that would showcase our agency’s interactive capabilities while also providing utility for festival participants. After kicking around a few ideas, the team largely settled on producing a large-format interactive wall to demonstrate nascent multitouch technologies. Following that, a brainstorming session was conducted around understanding the needs of participants. With the underlying focus of Cannes Lions being a conference with speaking engagements, the primary design goals would focus on providing users with temporal and spatial information related to events and fulfilling ancillary needs for attendees that may not be familiar with Cannes.

Challenge 1: Accessibility for Large Format Displays.

After developing the basic requirements, the initial challenge was figuring out how to support the requirements on such a large scale with multiple users interacting simultaneously. Part of this required developing an understanding of physiology and the psychology of personal space. How close would users feel comfortable interacting next to each other? What is the highest that a user feels comfortable reaching to touch an interactive element? What about the lowest? How far away can a user read text? As we began to answer these questions, it became clear that dividing the experience into two zones would provide the best user experience. The upper zone would be devoted to providing information and be considered “public,” while the lower zone below 6 1/2’ would be where direct interaction would take place, in a zone of personal space.

The interaction model was structured so that each user would have their own personal space to address their individualized needs. Thus, if a user were looking for the speaking schedule, they would walk up to the touchwall and tap on Schedule. While this individualized interaction model seemed solid, testing of the initial sketches via projection before the enclosure was fabricated revealed some early flaws. With this model, the utility of the public zone information consisted of a generalized message and the current and upcoming speaking sessions. Additionally, the experience lacked representation of the user within the interface, leaving the touchwall feeling more like a giant web interface instead of something unique.

The revised model revisited the notion of shared space. The schedule moved to a horizontal element and would display the entire week’s sessions, straddling the public and personal zones, allowing users to interact with the lower portion. This change opened up the opportunity for a second type of casual user to treat the touchwall as digital signage. This would allow casual users to view the week’s schedule and remove the requirement for any direct interaction to get key conference information. With this change, a layer of spatial data was introduced on the lower portion of the personal space, providing an interactive map of Cannes displaying hotels, restaurants, bars, and other services.

Challenge 2: Private Authentication in Public Spaces.

With the question of how to solve for casual and interactive users largely solved, unanswered questions around personal interactions began to arise. The interaction model relied on user trust to select their name before conducting any social interaction within the application. This is problematic as it opens up the application to potentially malicious behaviors, including spamming and false appointments. Solving the issue of authentication rose to the top of our priorities. Password authentication seemed the obvious answer, but exploring that led to the typical issues of user onboarding, which were further exacerbated by performing that in a public space. If users were to be onboarded directly via the interface, they would be tying up the touchwall performing the rote tasks of creating a user account. If the account creation process took place outside of the context of the touchwall, that felt very disjointed from creating an immersive experience. Furthermore, if user accounts were pre-generated, the user would still be required to authenticate by typing their password in full view of other users, raising security concerns.

This was when I proposed solving the issues around authentication with an additional layer of technology: RFID tags. A quick hardware prototype was developed using Arduino, a SparkFun RFID reader, and cards to demonstrate the viability of RFID. The team quickly bought into this approach, as it added a level of security and another nascent technology to highlight commercial and marketing opportunities for innovative digital out-of-home experiences. However, this also introduced significant complexity to the project. First, the RFID reader used in the prototype would need to be replaced with robust hardware. Then came the logistical and technological challenges of creating user accounts, associating them with the RFID cards, and distributing the cards themselves to the attendees. To accommodate the logistical aspects, the Cannes Lions staff agreed to allow our team access to the registration database to serialize the tags and insert them into the conference badges. This allowed for a completely frictionless onboarding experience for all attendees: every user would already be setup with an account by simply picking up their badge during check-in.

Challenge 3: Refining Spatial Interactions.

After researching industrial RFID readers, a visit to a hardware vendor was conducted to evaluate different hardware solutions to inform the next iteration of interaction design. After assessing the performance of various passive RFID tags at different distances, orientations, and rates of movement, the findings were that RFID would be a viable solution. However, a configuration that utilized a single RFID antenna would not be accurate enough to locate a user in three-dimensional space and enable a seamless experience. To address this, additional technology was introduced through the selection of the Alien line of RFID readers that feature the capability to connect multiple antennas.

The evolved interaction model would present a cloud of user avatars from detected RFID tags at the maximum distance they could be sensed and then display in higher fidelity as a user moved within 3’ of the touchwall to a distance at which they could tangibly interact. After receiving the RFID hardware and numerous tags with varying capabilities, we proceeded to test the tags on their own. The goal was to determine the distance at which a tag could be sensed to confirm if the interaction model was feasible. As we revisited the spatial implications, a real-time.NET application was created to triangulate the position of a given tag in 3D space according to signal strength received. Through extensive testing and prototyping, we discovered a number of factors that negatively impacted reliable tag detection. Those included angle to reader, vector of movement, oscillation during movement, and surprisingly, signal attenuation due to body mass. The data gathered forced us to abandon the “tease” mode of displaying user avatars at a distance.

The interaction model was refined a final time based on the prototyping results. This led to creation of four distinct areas within the interface that functioned as targets for the user to hold their RFID conference badge up to to start interacting with the touchwall. This had the added benefit of spacing out active users into a comfortable physical distance, while also inviting social interaction.

Results.

The touchwall exceeded all expectations in terms of successfully resolving multi-domain challenges across UX and technology to create a compelling product through innovative spatial interactions. The attendees featured some early-adopters that stopped by to interact with the touchwall as soon as the doors opened. As the day progressed, the buzz built as those users evangelized the touchwall. By the end of the first day, and throughout the subsequent week of events, it was thrilling to watch repeat users stop by to see what was coming up next, interact with others, set up social events, create their event schedules, and discuss commercial applications of the technologies showcased in the touchwall. In terms of objective measurement, 50 million media impressions were made worldwide. 6,000 total RFID authentications were made by 1,800 unique users. 1,200 person-to-person connections were made along with 1,400 contact information exchanges.