Project Overview

Project Directors: Jeffrey Shaw and Dennis Del Favero
Project Programmers: Ardrian Hardjorno, Volker Kuchelmeister, Matthew McGinity
(Additional Software and Hardware Engineering:Jared Berghold, Marc Chee, Robin Chow, Alex Kuptsov, Alex Ong, Xin Guan)
Project Funding: ARC DP0209550 ; ARC DP0345547; ARC LE0453517
2004 - 2018

AVIE: exterior view showing Nebula

AVIE is the world’s first 360-degree stereoscopic immersive interactive visualisation system that includes a motion tracking system and a multi-channel audio system. Driven by an experimental framework, it uses a suite of cutting-edge hardware and software resources enabling the development of applications in Virtual Reality, Artificial Intelligence, Motion Capture and Motion Tracking.

tvispg1 AVIE: interior view showing T_Visionarium II

AVIE has been installed as core visualisation infrastructure at: City University of Hong Kong, Hong Kong; City University of Hong Kong Chengdu Research Institute; École Polytechnique Fédérale de Lausanne, Laussane, Switzerland; Museum Victoria, Melbourne, Australiae; Rensselaer Polytechnic University, New York, USA; University of Technology Sydney, Australia; ZKM, Center for Art and Media, Karlsruhe, Germany.

It has been commercialised for the mine safety training industry by: Mines Rescue Pty Ltd at its Newcastle, Wollongong, Lithgow and Singleton facilities, Australia; Shenyang Research Institute of China Coal Technology & Engineering Group (CCTEG), Fushun, China.

AVIE has been commissioned as a curated visualisation platform by numerous premier international venues for biennales, exhibitions and festivals, including:

  1. Art of Immersion: Virtual Frontiers, ZKM, Karlsruhe, 2017
  2. Digitalife, Palazzo delle Esposizione, Rome, 2017
  3. Sydney Film Festival, Sydney, 2016
  4. Bonlieu Scène Nationale, Annecy, 2016
  5. International Symposium of Electronic Art 2016 (ISEA16), Hong Kong, 2016
  6. GLOBALE, ZKM, Karlsruhe, 2016.
  7. Seoul Mediacity Biennale, Seoul Museum of Art, Seoul, 2016
  8. Montreal Museum of Fine Art, Montreal, 2016
  9. Chronus Art Center, Shanghai, 2014-15
  10. Smithsonian Institution, Washington D.C., 2014
  11. Child, Nation and World Cinema Symposium, Sydney, 2014
  12. 20th International Conference on Virtual Systems and Multimedia, Hong Kong, 2014
  13. International Symposium of Electronic Art 2013 (ISEA13), Sydney, 2013
  14. 10th Shanghai Biennale, Power Station of Art, Shanghai, 2013
  15. SIGGRAPH Asia, Hong Kong, 2013
  16. Hong Kong Book Fair, Hong Kong, 2013
  17. Limited Knowledge, Hong Kong, 2013
  18. Festival d’Avignon, Avignon, 2013
  19. Kaladham Museum, Karnataka, (permanent installation) 2012-
  20. Business of Design Week, Hong Kong, 2012
  21. NODEM, Hong Kong, 2012
  22. Grand Opening of Run Run Shaw Creative Media Centre, Hong Kong, 2012
  23. Smithsonian Institution, Washington D.C., 2012
  24. Matter-Light, Bethune, 2011
  25. Sydney Film Festival, Sydney, 2011
  26. ‘La dispersion du Fils’ in Festival of Avignon, Gymnase Paul Giera, Avignon, 2011
  27. National Institute of Experimental Arts double conference, Sydney, 2011
  28. ‘Inaugural Exhibition’, ALiVE, CityU, Hong Kong, 2011
  29. Digitalife_, La Pelanda, Rome, 2010
  30. STRP Art and Technology Festival_, The Klokgebouw, Eindhoven, 2010
  31. ‘Rio Tinto Volcanic 3D’ in Dynamic Earth, Museum Victoria, Melbourne, 2010
  32. Cinema, Modernity and Modernism, FHAANZ conference, UNSW, 2010
  33. Voyage to the Planet’s TV series, ABC Television, 2010
  34. Inaugural Exhibition, ALiVE Applied Laboratory for Interactive Visualization and Embodiment, Hong Kong, 2010
  35. STRP Festival’, Strijp R Klokgebouw, Eindhoven; ALiVE, CityU, Hong Kong, 2010;
  36. APPEA Conference & Exhibition’, Convention & Exhibition Centre, Brisbane, 2010
  37. ‘Energy Field Days’, Showground, Sydney, 2010
  38. Un Volcan Numérique, Le Volcan Scène Nationale du Havre, Le Harve, 2009
  39. Seconde Nature, Espace Vasarély, Aix-en-Provence, 2009
  40. OPEN CITY‘ in ‘International Documentary Film Festival Amsterdam’ and ‘Architecture Biennale Rotterdam’, Zuiderkerk, Amsterdam, 2009
  41. Imagining Media @ ZKM, Karlsruhe, 2009
  42. Un Volcan Numérique, Le Havre, 2009
  43. Sydney Festival, Sydney, 2008
  44. Biennale of Seville (BIACS3), Carlos V Alambra Palace, Granada, 2008
  45. eARTS Festival: eLandscapes, Zendai Museum of Modern Art, Shanghai, 2008
  46. YOUser: the Century of the Consumer & Panorama Festival, ZKM Media Museum, Karlsruhe, 2007-2009
  47. There is still time… Brother by the Wooster Group, EMPAC, Rensselaer University, New York, 2007
  48. i.Future Festival, Singapore Science Centre, 2007
  49. Inhabiting the Cultural Imaginary, VSMM conference, UNSW, 2007
  50. Panorama Festival, ZKM, Karlsruhe, 2007
  51. ‘Australian Council of University Art & Design Schools Conference’, UNSW, Sydney, 2007
  52. AIMEX Australian Mining Industry Conference’, Sydney, 2007

Video AVIE

AVIE has been developed though three iterations, with further development on-going. AVIE1 (2004-07) facilitated audio-visual data to respond kinaesthetically to the user's body across the full breadth of the projected space; AVIE2 (2008-2010) has augmented this environment by delivering the ability to recompose digital files in real-time; AVIE3 (2011-2017) has delivered a base-line AI Database system that enables digital agents to interact with users,based on their ability to interpret and respond to the user’s behaviour in space. AVIE provides a platform in which 3D audio-visual experiences are created by a combination of multiple projectors and multiple sound sources, driven by a cluster of computers, running a highly versatile real-time 3D VR engine. It comprises custom designed technologies: a Motion Tracking system, a Mixed Reality system and an AI Reasoning system. The AI system coordinates the actions of the visualisation system by allowing it to autonomously reason about its response to user behaviour. The Motion Tracking system tracks the physical location of users in AVIE, while the Mixed Reality system allows the creation of 'scenarios' that exhibit a feedback loop between the world of the user and of the audio-visual data.

AVIE visualisation forms:

  • High-resolution panoramic stereo images with augmented 3D graphics overlays
  • High-resolution panoramic videos, including stereo videos
  • Immersive multimedia data visualisation
  • Real-time interactive 3D environments supported by artificial intelligence systems

AVIE interaction platforms

AVIE utilises two types of interaction:

  1. a) remote device: a wireless tablet device allowing real-time selection and navigation within interactive applications
  1. b) motion tracking: real time software that detects, captures, tracks and interprets human movement and gestures, allowing capture and audience participation in the direct cinematic experience, interplay between users and digital characters and analysis of user behaviour

mining_in_AVIE_compiled AVIE: interior views showing Mining VR

AVIE Components

avie_top AVIE: external view

Panoramic Immersive Stereoscopic 3D Visualisation System

AVIE consists of a cylindrical silvered screen 4 metres high and 10 metres in diameter on which 3D panoramic images sequences and videos are projected. The 6 projectors use active shuttering technology to deliver stereoscopic 3D to end users, and the images from the individual projectors are seamlessly blended by custom software developed in-house at iCinema. 6 high-performance image generators, capable of processing images and videos as well as rendering large virtual environments in real time, are linked to the projectors to
deliver the full immersive 3D experience. It accommodates up to 30 people at any one time.