The Virtual Production Studio Concept – An Emerging Game Changer in Filmmaking

October 10, 2022
Authors: and

Abstract

This paper analyses methodological and conceptual shifts in filmmaking brought about by the integrated use of VR technology, game engines, and LED walls in a Virtual Production Studio (VPS). The paper focuses not only on the components of an integrated VPS system, but also on what makes it qualitatively different from traditional filmmaking technologies. The VPS concept transforms the relationships among display technology, live action, image production, and postproduction (VFX) by causing a qualitative and performative shift in filmmaking. The paper analyses the current state of VP research from both conceptual and technological perspectives, global VP facilities, and significant trends in Australia by presenting two business case studies. We have observed three major trends: Globalisation, Collaboration and Industrialisation. Case Study 1 shows how the convergence of technologies from VR gaming and digital filmmaking have made VPS technology possible. Case Study 2demonstrates how adopting strategies of pairing with industry partners and joining a large production hub helps maintain a company’s position in the film industry. The paper concludes by stating that there is a need for an integrated VPS network through shared infrastructure and resources to foster high quality research and to solve industry-identified problems through industry-led collaborative research partnerships.

Cited by

  1. Kavakli, M., & Cremona, C. (2022). The virtual production studio concept – an emerging game changer in filmmaking. In (no ed.), 2022 IEEE on Conference Virtual Reality and 3D User Interfaces (VR)​​​​​​​ (pp. 29–37). IEEE Computer Society. https://doi.ieeecomputersociety.org/10.1109/VR51125.2022.00020.

Keywords

/ / / / / /

References

  1. N. Kadner, “The Virtual Production Field Guide. Volume 1,” Epic Games, https://cdn2.unrealengine.com/Unreal+Engine%2Fvpfieldguide%2F VP-Field-Guide-V1.2.02-5d28ccec9909ff626e42c619bcbe8ed2bf83138d.pdf. 2019.
  2. Weta Digital, Virtual Production, https://www.wetafx.co.nz/research-and-tech/technology/virtualproduction. 2021.
  3. S. Dalkian, “nDisplay Technology Whitepaper,” Epic Games, https://cdn2.unrealengine.com/Unreal+Engine%2Fndisplaywhitepaper-final-updates%2FnDisplay_Whitepaper_FINALf87f7ae569861e42d965e4bffd1ee412ab49b238.pdf. 2019.
  4. L. Kipnis, “Film and changing technologies,” The Oxford Guide to Film Studies, J. Hill and P. Church Gibson, eds., Oxford: Oxford University Press, pp. 595-604, 1998.
  5. S. Srivastav and R. Singh, “Film Viewing Experience: Changes with Technology,” Proc. 28th National Conf. Cultural Representation and Power of Media, available at https://www.academia.edu/35899511/Film_Viewing_Experience_Ch anges_with_Technology. 2017.
  6. B. Latour, Reassembling the social: an introduction to ActorNetwork-Theory. Oxford: Oxford University Press, 2007.
  7. I. Failes. “A Virtual Production Explainer: What It Is, and What It Could Mean for Your Project,” Cartoon Brew, https://www.cartoonbrew.com/vfx/a-virtual-production-explainerwhat-it-is-and-what-it-could-mean-for-your-project-166554.html. 2018.
  8. Technicolor Technicolor Genesis – Real Time Virtual Production, https://www.youtube.com/watch?v=x5O2EjvUbE&t=48s&ab_channel=technicolor. 2019.
  9. DL. Cade, “Incredible Real-Time Visual Effects Tech Replaces Green Screens with LED Walls,” Peta Pixel, https://petapixel.com/2019/08/22/demo-of-incredible-real-timevisual-effects-using-massive-led-walls/. 2019.
  10. Advanced Imaging Society, “Virtual Production Workflows & Insights,”https://www.youtube.com/watch?v=81f6FPnWMCI&ab_channel=A dvancedImagingSociety. 2020.
  11. D. Coldewey, “How Oculus squeezed sophisticated tracking into pipsqueak hardware,” Techcrunch, https://techcrunch.com/2019/08/22/how-oculus-squeezedsophisticated-tracking-into-pipsqueak-hardware. 2019.
  12. “Virtual Production – SimulCam,” Vicon, https://www.vicon.com/cms/wp-content/uploads/2020/09/ViconVirtual-Production.pdf. 2020.
  13. D. Viehmann, “Motion Tracking for Virtual Production,” Drew Viehmann, https://drewviehmann.medium.com/motion-tracking-forvirtual-production-f88e38fe2203. 2020.
  14. Wright, M. Chambers, and J. Israel. “Large Scale VFX Pipelines,” ACM SIGGRAPH 2017, Talks, Article 59, pp. 1–2. available at https://dl.acm.org/doi/10.1145/3084363.3085021, June 2021.
  15. OpenCL. NVIDIA Developer https://developer.nvidia.com/opencl. ND.
  16. OpenGL. OpenGL Overview. https://www.opengl.org/about. 2021.
  17. Unreal Build: Virtual Production Studio. Executive Panel Unreal Engine. https://www.unrealengine.com/en-US/events/unreal-buildvirtual-production-2020. 2020.
  18. KH Martin. “Atomic Fiction Answers the Call for Innovation in Welcome to Marwen,” VFXV, https://www.vfxvoice.com/atomicfiction-answers-the-call-for-innovation-in-welcome-to-marwen. 2019.
  19. M. Seymour, “Art of (LED Wall) Virtual Production Sets, Part Two: How you make one,” VFX Guide, https://www.fxguide.com/fxfeatured/art-of-led-wall-virtualproduction-sets-part-two-how-you-make-one/. 2020.
  20. J. Montgomery, “Who are you: FXGuide in focus.” FXGuide, https://www.fxguide.com/fxfeatured/who_are_you_fxguide_in_focus/?highlight=MIKE%20SEYMOUR. 2002.
  21. A. Schulz, A. Eder, V. Tiberius, S. Casas Solorio, F. Fabro and N. Brehmer, The Digitalization of Motion Picture Production and Its Value Chain Implications. Journalism and Media, 2:397–416, September 2021. https://www.mdpi.com/2673-5172/2/3/24/pdf. 2021.
  22. J. Bennett, “Immersive Performance Environment: A framework for facilitating an actor in Virtual Production,” PhD dissertation, Creative Industries Faculty, Queensland Univ. of Technology, Brisbane, Queensland, available at https://eprints.qut.edu.au/203911/. 2020.
  23. K. Ilmaranta, Cinematic Space in Virtual Production. https://www.academia.edu/44002310/Cinematic_Space_in_Virtual_Production. 2020.
  24. T. Hogg, “Virtual Production Takes a Big Step Forward,” VFX Voice https://www.vfxvoice.com/virtual-production-takes-a-big-stepforward. 2021.
  25. A. Marrs, P. Shirley and I. Wald, Ray Tracing Gems II: Next Generation Real-Time Rendering with DXR, Vulkan, and OptiX. Berkeley, Calif.: Apress, 2021.
  26. P. Debevec, and J. Malik, “Recovering high dynamic range radiance maps from photographs,” Computer graphics and interactive techniques (SIGGRAPH ’97). ACM Press/Addison-Wesley Publishing Co., USA, pp. 369-378, available at https://www.academia.edu/download/29352455/recovering_high_dynamic_range_radiance_maps_from_photographs.pdf. Aug. 1997.
  27. P. Debevec, A. Wenger, C. Tchou, A. Gardner and J. Waese, “A lighting reproduction approach to live-action compositing,” ACM Trans. Graph., (SIGGRAPH 2002), vol. 21, no. 3, p. 547-556, https://dl.acm.org/doi/abs/10.1145/566570.566614. Jul. 2002.
  28. P. Hedman, PP. Srinivasan, B. Mildenhall, JT. Barron and P. Debevec, “Baking Neural Radiance Fields for Real-Time View Synthesis,” Arxiv, abs/2103.14645, 2021 https://arxiv.org/pdf/2103.14645. 2021.
  29. Deloitte, “The Future of Content Creation: Virtual Production – Unlocking Creative Vision and Business Value,” Technology, Media & Telecommunications, https://www2.deloitte.com/content/dam/Deloitte/us/Documents/technology-media-telecommunications/us-tmt-the-future-of-contentcreation-virtual-production.pdf. 2020.
  30. Netflix, “Netflix 2D LED In-Camera VFX Field Guide,” Virtual Production, https://partnerhelp.netflixstudios.com/hc/enus/categories/360005104874-Virtual-Production. 2021.
  31. S. Sylwan, “The application of vision algorithms to visual effects production,” ACCV’10: Proceedings of the 10th Asian conference on Computer vision, vol. part I. Springer-Verlag, pp. 189–199, 2010.
  32. J. Pantaleoni, L. Fascione, M. Hill, and T. Aila, “PantaRay: fast raytraced occlusion caching of massive scenes.,” ACM Trans. Graph., vol. 29, no. 4, pp. 37:1-37:10, available at https://dl.acm.org/doi/10.1145/1778765.1778774, July 2010.
  33. MB. Hullin, J. Hanika and W. Heidric “Polynomial Optics: A Construction Kit for Efficient Ray-Tracing of Lens Systems,” Computer Graphics Forum, vol. 31, no. 4, pp. 1375–1383, available at https://www.cs.ubc.ca/labs/imager/tr/2012/PolynomialOptics/dl.php?fn=PolynomialOptics-Hullin2012.pdf. June 2012.
  34. J. Hanika, P. Hillman, M. Hill and L. Fascione, “Camera Space Volumetric Shadows,” DigiPro ’12, pp.7-14, available at https://dl.acm.org/doi/10.1145/2370919.2370921. Aug. 2012.
  35. Hive Division, LED Wall Virtual Production – Bye Bye Green Screen https://www.youtube.com/watch?v=ysIOi_MP_cs&ab_channel=HiveDivision. 2020.
  36. Australian Cinematographers Society, “Virtual Production – A Cinematographers’ Conversation,” https://www.youtube.com/watch?v=3g_GhufsQ7E&ab_channel=AustralianCinematographersSociety. 2020.
  37. J. Roettgers, “How Video-Game Engines Help Create Visual Effects on Movie Sets in Real Time,” Variety, https://variety.com/2019/biz/features/video-game-engines-visualeffects-real-time-1203214992. 2019.
  38. K. Maher, “Virtual cameras meld digital effects and live action,” New Ideas from the Intersections of Science, Art & Engineering, https://gfxspeak.com/2019/07/11/virtual-cameras-digital/. 2019.
  39. C. Giardina, “NEP Acquires Lux Machina, Halon, Prysm Collective to Launch Virtual Production Business,” Behind the Screen, https://www.hollywoodreporter.com/business/digital/nep-acquireslux-machina-halon-prysm-collective-virtual-production-business1234991516/. 2021.
  40. L. Niklaus, “Unity Vs Unreal: Which Fits You Best?,” Mount CG, https://mountcg.com/unity-vs-unreal/. ND.
  41. The Studio. “The benefits of Virtual Production with Spectre Studio’s,” Speaker series, https://www.facebook.com/thestudioLTD/videos/spectre-studiosspeaker-series/423775158181725. 2019.
  42. “ICVFX Says “Yes” to Brompton Technology Processing,” Light and Sound America, http://www.lightingandsoundamerica.com/news/story.asp?ID=-6Y7J8G. 2020.
  43. “Cutting Edge Creates Buzz in Brisbane With Edge Lab VP Facility,” CX Digital Network, https://www.cxnetwork.com.au/cutting-edge-creates-buzz-inbrisbane-with-edge-lab-vp-facility/. 2021.
  44. S. Dent, “Film’s cinema comeback is driven by nostalgia, not logic,” Engadget, https://www.engadget.com/2016-02-15-films-cinemacomeback-is-driven-by-nostalgia-not-logic.html. 2016
  45. K. Hu, “The Effects of Digital Video Technology on Modern Film,” Master of Science Thesis, Dept. of Television Management, Drexel Univ., UK, 2016.
  46. D. Russo, “Why Hollywood movie-making may become more virtual in a post-coronavirus world,” Technology Executive Council, https://www.cnbc.com/2020/05/23/how-hollywood-movie-makingbecomes-virtual-after-coronavirus.html. 2020.
  47. JD. Witmer and A. Fish, “ASC members offer their perspectives on what the next century has in store for the art and craft of cinematography,” Future of Cinematography: The Next 100 Years, https://ascmag.com/articles/the-next-100-years. 2019