Project Overview

Project Directors: Jeffrey Shaw and Dennis Del Favero
Project Programmers: Ardrian Hardjorno, Volker Kuchelmeister, Matthew McGinity
AVIE Platform conceived by Jeffrey Shaw and developed by the iCinema Centre
Additional Software and Hardware Engineering: Jared Berghold, Marc Chee, Robin Chow, Alex Kuptsov, Alex Ong, Xin Guan
This project was financially supported under the Australian Research Council’s Discovery and Linkage Infrastructure funding schemes
ARC Grant IDs: DP0209550 ; DP0345547; DP0556659; LE0453517

AVIE: exterior view showing Nebula

AVIE is the world’s first 360-degree stereoscopic immersive interactive visualisation system that includes a motion tracking system and a multi-channel audio system. Driven by an experimental framework, it uses a suite of cutting-edge hardware and software resources enabling the development of applications in Virtual Reality, Artificial Intelligence, Motion Capture and Motion Tracking.

tvispg1 AVIE: interior view showing T_Visionarium II

AVIE has been installed as core visualisation infrastructure at: City University of Hong Kong, Hong Kong; City University of Hong Kong Chengdu Research Institute; École Polytechnique Fédérale de Lausanne, Laussane, Switzerland; Museum Victoria, Melbourne, Australiae; Rensselaer Polytechnic University, New York, USA; University of Technology Sydney, Australia; ZKM, Center for Art and Media, Karlsruhe, Germany.

It has been commercialised for the mine safety training industry by: Mines Rescue Pty Ltd at its Newcastle, Wollongong, Lithgow and Singleton facilities, Australia; Shenyang Research Institute of China Coal Technology & Engineering Group (CCTEG), Fushun, China.

AVIE has been commissioned as a curated visualisation platform by 51 premier international venues for biennales, exhibitions and festivals, including:

  1. Art of Immersion: Virtual Frontiers, ZKM, Karlsruhe, 2017
  2. Chengdu Research Institute, Chengdu, 2017
  3. Sydney Film Festival, Sydney, 2016
  4. International Symposium of Electronic Art 2016 (ISEA16), Hong Kong, 2016
  5. GLOBALE, ZKM, Karlsruhe, 2016.
  6. Chronus Art Center, Shanghai, 2014-15
  7. Smithsonian Institution, Washington D.C., 2014
  8. Child, Nation and World Cinema Symposium, Sydney, 2014
  9. 20th International Conference on Virtual Systems and Multimedia, Hong Kong, 2014
  10. Data Arena, UTS, Sydney, 2014
  11. International Symposium of Electronic Art 2013 (ISEA13), Sydney, 2013
  12. 10th Shanghai Biennale, Power Station of Art, Shanghai, 2013
  13. SIGGRAPH Asia, Hong Kong, 2013
  14. Hong Kong Book Fair, Hong Kong, 2013
  15. Limited Knowledge, Hong Kong, 2013
  16. Festival d’Avignon, Avignon, 2013
  17. Kaladham Museum, Karnataka, (permanent installation) 2012-
  18. Business of Design Week, Hong Kong, 2012
  19. NODEM, Hong Kong, 2012
  20. Grand Opening of Run Run Shaw Creative Media Centre, Hong Kong, 2012
  21. Smithsonian Institution, Washington D.C., 2012
  22. Matter-Light, Bethune, 2011
  23. Sydney Film Festival, Sydney, 2011
  24. ‘La dispersion du Fils’ in Festival of Avignon, Gymnase Paul Giera, Avignon, 2011
  25. National Institute of Experimental Arts double conference, Sydney, 2011
  26. ‘Inaugural Exhibition’, ALiVE, CityU, Hong Kong, 2011
  27. Digitalife_, La Pelanda, Rome, 2010
  28. STRP Art and Technology Festival_, The Klokgebouw, Eindhoven, 2010
  29. ‘Rio Tinto Volcanic 3D’ in Dynamic Earth, Museum Victoria, Melbourne, 2010
  30. Cinema, Modernity and Modernism, FHAANZ conference, UNSW, 2010
  31. Voyage to the Planet’s TV series, ABC Television, 2010
  32. Inaugural Exhibition, ALiVE Applied Laboratory for Interactive Visualization and Embodiment, Hong Kong, 2010
  33. STRP Festival’, Strijp R Klokgebouw, Eindhoven; ALiVE, CityU, Hong Kong, 2010;
  34. APPEA Conference & Exhibition’, Convention & Exhibition Centre, Brisbane, 2010
  35. ‘Energy Field Days’, Showground, Sydney, 2010
  36. eArts Festival: eLANDSCAPES, Shanghai Science & Technology Museum, Shanghai, 2009
  37. Un Volcan Numérique, Le Volcan Scène Nationale du Havre, Le Harve, 2009
  38. Seconde Nature, Espace Vasarély, Aix-en-Provence, 2009
  39. OPEN CITY‘ in ‘International Documentary Film Festival Amsterdam’ and ‘Architecture Biennale Rotterdam’, Zuiderkerk, Amsterdam, 2009
  40. Imagining Media @ ZKM, Karlsruhe, 2009
  41. Un Volcan Numérique, Le Havre, 2009
  42. Sydney Festival, Sydney, 2008
  43. Biennale of Seville (BIACS3), Carlos V Alambra Palace, Granada, 2008
  44. Mines Rescue Pty Ltd Australian Facilities in Woonona, Argenton, Lithgow, Singleton – all Australia, 2008
  45. eARTS Festival: eLandscapes, Zendai Museum of Modern Art, Shanghai, 2008
  46. YOUser: the Century of the Consumer & Panorama Festival, ZKM Media Museum, Karlsruhe, 2007-2009
  47. From Spark to Pixel, Martin-Gropius-Bau, Berlin, 2007
  48. There is still time… Brother by the Wooster Group, EMPAC, Rensselaer University, New York, 2007
  49. i.Future Festival, Singapore Science Centre, 2007
  50. Inhabiting the Cultural Imaginary, VSMM conference, UNSW, 2007
  51. Panorama Festival, ZKM, Karlsruhe, 2007
  52. ‘Australian Council of University Art & Design Schools Conference’, UNSW, Sydney, 2007
  53. AIMEX Australian Mining Industry Conference’, Sydney, 2007
  54. La Rivoluzione Algoritmica, Artescienza: Spazio Deformato, Casa dell’Architettura, Rome, 2006
  55. Festival d’Avignon, Avignon, 2005
  56. Cinémas du Futur, Centre Euralille, Lille, 2004
  57. ZKM, Karlsruhe, 2004

Video AVIE

AVIE has been developed though three iterations, with further development on-going. AVIE1 (2004-07) has delivered an immersive 360-degree 3D environment controlled by a wand interface; AVIE2 (2008-2010) has augmented this environment by delivering the ability to recompose digital files in real-time; AVIE3 (2011-2017) has delivered a base-line AI Database system that enables digital agents to interact with users,based on their ability to interpret and respond to the user’s behaviour in space. AVIE provides a platform in which 3D audio-visual experiences are created by a combination of multiple projectors and multiple sound sources, driven by a cluster of computers, running a highly versatile real-time 3D VR engine. It comprises custom designed technologies: a Motion Tracking system, a Mixed Reality system and an AI Reasoning system. The AI system coordinates the actions of the visualisation system by allowing it to autonomously reason about its response to user behaviour. The Motion Tracking system tracks the physical location of users in AVIE, while the Mixed Reality system allows the creation of 'scenarios' that exhibit a feedback loop between the world of the user and of the audio-visual data.

AVIE visualisation forms:

  • High-resolution panoramic stereo images with augmented 3D graphics overlays
  • High-resolution panoramic videos, including stereo videos
  • Immersive multimedia data visualisation
  • Real-time interactive 3D environments supported by artificial intelligence systems

AVIE interaction platforms

AVIE utilises two types of interaction:

  1. a) remote device: a wireless tablet device allowing real-time selection and navigation within interactive applications
  1. b) motion tracking: real time software that detects, captures, tracks and interprets human movement and gestures, allowing capture and audience participation in the direct cinematic experience, interplay between users and digital characters and analysis of user behaviour

mining_in_AVIE_compiled AVIE: interior views showing Mining VR

AVIE Components

avie_top AVIE: external view

Panoramic Immersive Stereoscopic 3D Visualisation System

AVIE consists of a cylindrical silvered screen 4 metres high and 10 metres in diameter on which 3D panoramic images sequences and videos are projected. The 6 projectors use active shuttering technology to deliver stereoscopic 3D to end users, and the images from the individual projectors are seamlessly blended by custom software developed in-house at iCinema. 6 high-performance image generators, capable of processing images and videos as well as rendering large virtual environments in real time, are linked to the projectors to
deliver the full immersive 3D experience. It accommodates up to 30 people at any one time.