Reflx [AR] Video Mirror
UX design in the age of the selfie
2017
Designed and built interactive mirror photo booth for creating branded social media experiences. 3D Skeletal tracking is used to navigate through the interface which is written in NodeJS and packaged as a cross-platform desktop app using ElectronJS. The large ring light comprised of 400 RGBW LEDs is programmed in C an embedded chip. SMS and MMS delivery accomplished using Twilio. Amazon AWS is used for Image and video hosting while the APIs and web galleries are hosted using NodeJS on Heroku. Reflx has spun off in to a new company that offers branded social media experiences for luxury clubs, fashion & beauty industry events and parties.
Project Background
I’d been experimenting interactive window displays using rear projection and Microsoft Kinect. Around that time, I'd been in discussions with Alfredo Leija from DRTY SMMR, a fashion and art community in Brooklyn. He wanted a photo studio to start cataloging the clothing and art being created in the community. A similar idea being tossed around was a “photo booth", for visitors to take studio quality pictures of themselves in the outfits they were trying on. When these ideas came together, the interactive magical photo mirror was created.
This idea was simple in conception: To create a truly magical experience for the visitors of DRTY SMMR; To build and install an interactive installation disguised as a common mirror, that, when approached, is anything but common. When approached, the mirror turns into an augmented reality interactive experience that shoots photos, videos, brands them with DRTY SMMR graphics, automatically posts them to Facebook, and delivers them directly to the visitor via SMS and MMS.
Business Challenge
DRTY SMMR is a very unique place. Most people have never seen a place like it before. It was key to stay on-brand in that respect. I wanted to create something that people hadn't seen before, but more importantly, the goals were to create more social media content and begin email & SMS marketing. So, our business goals were clear:
- Stay on-brand
- Create social media content & encourage sharing
- Lay groundwork for email & SMS marketing
Design Challenge
When designing an installation, surprise, delight, and engagement are key factors. The most difficult factor to achieve in this project however, is that of magic. That sense of disbelief. That feeling of "How is this happening to me right now?". This is particularly difficult to maintain considering the requirement of gathering a users phone number or email address. How does the visitor communicate, talk to, ask and answer questions without ever touching a mouse, keyboard, remote control, or touch screen?
"Any sufficiently advanced technology is indistinguishable from magic."
- Winston Churchill (supposedly but prolly not)
Software Challenge
Data Input
I tested a variety of unique materials, sensors, chips, software, and other technologies in combination to capture and make use of users gestures. Ranging from Microsoft Kinect, to Intel RealSense 3D cameras.
To prevent interactions from becoming stale from visitor to visitor, a complex question/answer structure had to be created. A structure that would allow a technically finite number of contextual interactions to take place, while not following a specific pattern. This was done using a combination of hard-coded question/answer paths, in addition to artificial intelligence.
In order to achieve the business challenge of laying marketing groundwork for Facebook, Instagram, email and SMS, the mirror is wired (okay, technically wireless) in to a variety of networks including Facebook Graph API, email services, and cell phone networks.
The Final Product
After being installed in DRTY SMMR for several months, the reception was incredible. The Reflx project turned in to its own startup company offering photo mirror rentals for experiential marketing at events. A good marketing tool facilitates specific goals. A great marketing tool is one that's interchangeable between campaigns and even brands. In retrospect, the real key to the success of Reflx is to offer all of the listed features, without ever exposing a single piece of tech to the visitor.