Digital Media Designer: Virtual Reality, Mixed Reality, Artificial Intelligence and AAA Games
Prototyping AR Avatars in Unity and ARCore
Avatar R&D
Screen capture from Pixel 2 XL
Early Avatar prototyping with AR Core and Unity showing both facial and full body reactions to users proximity to avatar's personal space. The testing shown is in real-world conditions to observe interaction within dynamic spaces.
Enhanced Character Lighting
Screen capture from Pixel 2 XL
Here the goal was to improve on the quality of ARCore’s global light estimation. The script as is, produced a slightly dull tone on the model. I used reflection spheres and calculated general color and tone to apply additional scene based, dynamic lighting effects. However, this only address lighting across the AR content itself. The issue of AR objects casting real world shadows is a whole other problem.
Artificial Sunlight and Shadows
Screen capture from Pixel 2 XL
The goal was to create realistic daytime shadows to ground the character to the environment.
To achieve this I accessed the GPS data from the device. That provided me with the info to hack together a basic calculation for the suns angle and altitude. This is applied to an environmental lights rotation. The shadow is being rendered on an otherwise invisible plane loaded at the base of the avatar model. Additional values like the shadow strength are being set using ARCore's Light Estimation output.
Real World Object Occlusion Hack using Tango material
Screen capture from Pixel 2 XL
The goal was to occlude AR content behind real world objects.
In my opinion, occlusion of real-world objects is one of the biggest problems currently facing AR. Occlusion is often misunderstood in media covering Augmented Reality. An AR character or image looks convincing until something passes in front of the users’ camera. The problem is that AR content is drawn last in the render cycle thus placing it on top of (or in front of) the camera’s rendering of user and the world.
This example is showing a crude working version of occlusion that I created using a shader borrowed from my old Project Tango days. The Tango occlusion shader was applied to ARCore’s Detected Plane Visualizer and I set the script for both horizontal and vertical plane detection (ARCore 1.2 +). The effect is limited. The planes (walls in this case) must first be scanned by the device forcing the user to prepare the space ahead of the content placement.
Scaled Content in Real World Setting
Screen capture from Pixel 2 XL
The goal was to test the usability limitations of the device in real-world situations.
Visually, AR works well at small scale. Barring the occlusion issues and the possibility of objects being re-arranged and such (more on this topic at a later date).
This is an early prototype of a table top boxing game. Not only was I testing the UI/input and game space scaling. I also had to face the realities of waving a smartphone around in a public place! The smaller the AR content scale, the more discrete the user’s physical involvement.
The proof of concept is on-going. I plan to add cloud anchors to allow 2 human players on different devices to play against each other.
World Scale Street Art AR (early concept)
Screen capture from Pixel 2 XL
Rough Proof of Concept for Persistent World Scale AR social app that provides users with easy to use tools to create personalized art for real-world AR placement and social media sharing.
.png)
Screen captures from Pixel 2 XL
Images above show AR stencils created off-line and placed in ARCore on various real-world surfaces. I created the stencil art in Photoshop using a small set of standard blending techniques. The goal was to determine that if the app could create the image to those standards, how well will the blends hold up when overlaid in real-world conditions? The current version (not shown, examples coming soon) is using an asset called Blend Modes (Elringus/Unity Asset Store) to create the blends within the Unity app with user input.
Below are some of the original panels from a High-Concept flow for graffiti/paint tools.
