The Silk Road
Visual, Interaction and Motion Graphics Design | Interactive Scroll Painting | AR Development
Individual Work Year: May 2017
INTRO
How to combine traditional Chinese scroll painting with AR technology to tell a compelling story about the Silk Road?
<The Silk Road> is my undergraduate thesis project.
It’s a combination of traditional scroll painting and Augmented Reality. The interactive scroll painting is based on the history of the Silk Road. It is divided into two volumes and each of them is three meters long, telling 8 stories along the Silk Road from the beginning of Chang An to the west of Central Asia.
This work has been awarded as the best thesis project of Zhejiang University and as the first prize of China National Human Computer Interaction Design 2017, and has been exhibited in china and NYC Media.
「丝绸之路」是一个传统画卷和增强现实 (Augmented Reality) 结合的作品。该画卷取材于丝绸之路的历史，分为上下两卷，卷长3米，从浪漫和传奇的角度，讲述了丝绸之路东始长安，西至中亚的8个故事。该作品结合了传统画卷和AR技术，诗词曲赋与动画结合，用户扫描画卷中的图片和文字，就会看到画卷中的人和景仿佛从历史中走出，讲述一个个波澜壮阔的故事。AR和画卷的完美融合，让用户感受不到虚拟和现实区别，仿若画卷本身动了起来。该作品曾经拿过浙江大学最佳毕设奖和2017年全国计算机人机交互大赛一等奖，曾在国内和纽约Media Lab展出。Awards:
Outstanding Bachelor Thesis Project of Zhejiang University, China
First Prize of 2017 China Computer Interaction Art and Design (CIAD) Competition, Human Computer Interaction Track
Showcase:
2017 NYC Media Lab
2017 Zhejiang University Thesis Show
Year
May 2017
Tool:
#AR Dev. @Vuforia
#Interaction Design
#Mobile App. @Unity @Xcode
#Illustration @Ps @Ai
#Motion Graphics @Ae @Pr
#storytelling
DEMO
IDEATION
At present, the study of The Silk Road is based on archaeological discoveries and historical civilization. The media forms of The Silk Road or “The Belt and Road”, are mainly traditional media interaction forms including documentaries, books or museum exhibitions. This work reconstructs The Silk Road by new media approaches so that the public can rediscover those stories happened on the Silk Road from a more relaxed and romantic perspective. At the same time the public can enjoy the unique interactions and experiences originating from new media art.
This work combines the traditional scroll painting and augmented reality (AR) technology. This work invents a brand new method to enable natural fusion of animation with background pictures by creating a shader. Users scan static images and texts using AR and enjoy corresponding dynamic pictures and text explanation on their mobile phone. Users can interact with the traditional scroll painting using AR, which makes the insipid historical pictures come to life and tell the moving and legendary stories.
DESIGN
Storytelling Creation
As for the content of the APP, I did a lot of research on Silk Road in the early days hoping to find out and present intriguing and touching stories. Besides visiting museums exhibiting histories of Silk Road, I read several relevant books and watched impressive documentaries. Throughout these processes I collected relevant pictures, texts, and other materials and sorted out the creative and important elements that must be included in my APP. Finally, I selected 8 important events took placed at 8 different locations along the Silk Road as a representative nodes and organized them into 8 stories from east to west: Changan, Mogao Grottoes, Kroraina(Lop Nor), Wusun,Qiuci, Tamara, Kashi, Central Asia
Scroll Painting Design
Setup
Animation Design and Production
I completed my storyboard drawing after I finished my script writing.
While making the animation, I infused Chinese poetry and songs into every story. I tried to make each story's art style to be unique and related to the local culture, so that the audience can have a fresh feeling when they start the story. Besides, I combined traditional Chinese art, for example Chinese paper-cut, ink painting, murals and other artistic elements, with modern design aesthetics and techniques, hoping that traditional chinese art can be recreated by new way of media art. I hope every story's art is somewhat different but the whole work can be unified.
IMPLEMENTATION
APP Development & AR Interaction
I used the unity development platform and the Vuforia open source development kit to implement the AR APP that can be used on both IOS and Android devices.
The interaction of this APP is very intuitive and smooth. As users open the APP and align their phone cameras with images or texts to be identified, corresponding interactions, animations for images and detailed essays for text, will be activated. Users can either pause/play the videos, and watch them in full-screen mode by touching the images or read the content description of each story by navigating the text box.
Shader Creation
In addition, I created a shader that supports IOS and Android devices through GPU based rendering to achieve the perfect fusion of animations and recognition images, creating the impression that the images themselves are moving.
TAKE AWAYS
After I made this project, I realized that It would be great if I could create a string to connect each story together and use this string to guide audiences go through the scroll painting.