View source for Virtual camera system

{{short description|System to display a view of a 3D virtual world}}
{{pp-pc1}}
{{Use dmy dates|date=July 2021}}
[[File:Virtual-camera-system.png|thumb|Virtual camera system [[Software prototyping|demo]] showing parameters of the camera that can be adjusted]]
{{VG Graphics}}
In [[3D computer graphics|3D video games]], a '''virtual camera system''' aims at controlling a camera or a set of cameras to display a view of a 3D [[virtual world]]. Camera systems are used in video games where their purpose is to show the action at the best possible angle; more generally, they are used in 3D virtual worlds when a third-person view is required.

As opposed to filmmakers, virtual camera system creators have to deal with a world that is interactive and unpredictable. It is not possible to know where the [[player character]] is going to be in the next few seconds; therefore, it is not possible to plan the [[shot (filming)|shots]] as a filmmaker would do. To solve this issue, the system relies on certain rules or [[artificial intelligence]] to select the most appropriate shots.

There are mainly three types of camera systems. In ''fixed camera systems'', the camera does not move at all, and the system displays the player's character in a succession of still shots. ''Tracking cameras'', on the other hand, follow the character's movements. Finally, ''interactive camera systems'' are partially automated and allow the player to directly change the view. To implement camera systems, video game developers use techniques such as [[Constraint programming|constraint solvers]], [[Scripts (artificial intelligence)|artificial intelligence scripts]], or [[autonomous agent]]s.

=={{Anchor|Third person view games}} Third-person view==
In video games, "third-person" refers to a [[perspective (visual)|graphical perspective]] rendered from a fixed distance behind and slightly above the player character. This viewpoint allows players to see a more strongly characterized [[Avatar (computing)|avatar]] and is most common in [[action game]]s and [[action adventure game]]s. Games with this perspective often make use of positional audio, where the volume of ambient sounds varies depending on the position of the avatar.<ref name="fundamentals">{{cite book|last=Rollings|first=Andrew|author2=Ernest Adams|title=Fundamentals of Game Design|isbn=9780131687479|publisher=Prentice Hall|year=2006|url=http://wps.prenhall.com/bp_gamedev_1/54/14053/3597646.cw/index.html|access-date=24 March 2009|archive-date=17 February 2009|archive-url=https://web.archive.org/web/20090217232801/http://wps.prenhall.com/bp_gamedev_1/54/14053/3597646.cw/index.html|url-status=dead}}</ref>

There are primarily three types of third-person camera systems: the "fixed camera systems" in which the camera positions are set during the game creation; the "tracking camera systems" in which the camera simply follows the player's character; and the "interactive camera systems" that are under the player's control.

===Fixed===
[[File:Resident-evil-2-camerawork.jpg|thumb|right| Selection of shots in ''[[Resident Evil 2]]'' that aim at creating tension]]

With a fixed camera system, the developers set the properties of the camera, such as its position, orientation or [[field of view]], during the game creation. The camera views will not change dynamically, so the same place will always be shown under the same set of views. Games that use fixed cameras include ''[[Grim Fandango]]'' (1998) and the early ''[[Resident Evil]]'' and ''[[God of War (franchise)|God of War]]'' games.<ref>{{cite web|last=Casamassina |first=Matt |url=http://www.giantbomb.com/fixed-camera/3015-1715/ |title=fixed-camera |publisher=giantbomb}}</ref>

One advantage of this camera system is that it allows the game designers to use the [[film theory|language of film]], creating mood through camerawork and selection of shots. Games that use this kind of technique are often praised for their cinematic qualities.<ref>{{cite web |last=Casamassina |first=Matt |url=http://uk.cube.ign.com/articles/358/358101p3.html |title=Resident Evil Review |publisher=IGN |access-date=22 March 2009 |archive-url=https://web.archive.org/web/20090325153306/http://uk.cube.ign.com/articles/358/358101p3.html |archive-date=25 March 2009 |url-status=dead}}</ref> Many games with fixed cameras use [[tank controls]], whereby players control character movement relative to the position of the [[player character]] rather than the camera position;<ref name=":0">{{Cite news|date=20 February 2015|title=A eulogy for tank controls|language=en|work=[[PC Gamer]]|url=https://www.pcgamer.com/a-eulogy-for-tank-controls/|access-date=5 March 2018}}</ref> this allows the player to maintain direction when the camera angle changes.<ref name=":2">{{Cite web|last=Matulef|first=Jeffrey|date=26 January 2015|title=Bringing out the Dead: Tim Schafer reflects back on Grim Fandango|url=http://www.eurogamer.net/articles/2015-01-24-bringing-out-the-dead-tim-schafer-reflects-back-on-grim-fandango|access-date=5 March 2018|website=Eurogamer|language=en-UK}}</ref>

===Tracking===
[[File:Shooter person views.png|thumb|An illustration of a protagonist whom a player controls and a tracking camera just behind, slightly above, and slightly facing down towards that character]]
Tracking cameras follows the characters from behind. The player does not control the camera in any way – they cannot for example rotate it or move it to a different position. This type of camera system was very common in early 3D games such as ''[[Crash Bandicoot (video game)|Crash Bandicoot]]'' or ''[[Tomb Raider (1996 video game)|Tomb Raider]]'' since it is very simple to implement. However, there are a number of issues with it. In particular, if the current view is not suitable (either because it is occluded by an object, or because it is not showing what the player is interested in), it cannot be changed since the player does not control the camera.<ref>{{cite web |url=http://uk.dreamcast.ign.com/articles/160/160140p1.html |archive-url=https://web.archive.org/web/20080211014816/http://uk.dreamcast.ign.com/articles/160/160140p1.html |url-status=dead |archive-date=11 February 2008 |title=Sonic Adventure Review |publisher=IGN |access-date=22 March 2009}}</ref><ref>{{cite web|url=http://uk.pc.ign.com/articles/162/162059p1.html |title=Tomb Raider: The Last Revelation Review |date=11 December 1999 |publisher=IGN |access-date=22 March 2009}}</ref><ref>{{cite web |last=Carle |first=Chris |url=http://uk.cube.ign.com/articles/403/403746p3.html |title=Enter the Matrix Review |publisher=IGN |access-date=22 March 2009 |archive-url=https://web.archive.org/web/20090325214145/http://uk.cube.ign.com/articles/403/403746p3.html |archive-date=25 March 2009 |url-status=dead}}</ref> Sometimes this viewpoint causes difficulty when a character turns or stands face out against a wall. The camera may jerk or end up in awkward positions.<ref name="fundamentals"/>

===Interactive===
[[Image:super-mario-64-camera-system-ai.jpg|thumb|right| upright|Instead of staying behind Mario, the camera intelligently rotates to show the path (''[[Super Mario 64]]'').]]
This type of camera system is an improvement over the tracking camera system. While the camera is still tracking the character, some of its parameters, such as its orientation or distance to the character, can be changed. On [[video game console]]s, the camera is often controlled by an [[analog stick]] to provide good accuracy, whereas on PC games it is usually controlled by the [[Mouse (computing)|mouse]]. This is the case in games such as ''[[Super Mario Sunshine]]'' or ''[[The Legend of Zelda: The Wind Waker]]''. Fully interactive camera systems are often difficult to implement in the right way. Thus [[GameSpot]] argues that much of the ''Super Mario Sunshine''' difficulty comes from having to control the camera.<ref>{{cite web |last=Gerstmann |first=Jeff |url=http://uk.gamespot.com/gamecube/action/supermariosunshine/review.html?om_act=convert&om_clk=gssummary&tag=summary%3Bread-review&page=2 |title=Super Mario Sunshine Review for GameCube |publisher=GameSpot |date=4 October 2002 |access-date=22 March 2009 |archive-url=https://web.archive.org/web/20090326003410/http://uk.gamespot.com/gamecube/action/supermariosunshine/review.html?om_act=convert&om_clk=gssummary&tag=summary%3Bread-review&page=2 |archive-date=26 March 2009 |url-status=dead}}</ref> ''The Legend of Zelda: The Wind Waker'' was more successful at it - [[IGN]] called the camera system "so smart that it rarely needs manual correction".<ref>{{cite web |last=Casamassina |first=Matt |url=http://uk.cube.ign.com/articles/390/390314p2.html |title=The Legend of Zelda: The Wind Waker Review |publisher=IGN |date=25 March 2003 |access-date=22 March 2009 |archive-url=https://web.archive.org/web/20090326003127/http://uk.cube.ign.com/articles/390/390314p2.html |archive-date=26 March 2009 |url-status=dead}}</ref>

One of the first games to offer an interactive camera system was ''[[Super Mario 64]]''. The game had two types of camera systems between which the player could switch at any time. The first one was a standard tracking camera system except that it was partly driven by [[artificial intelligence]]. Indeed, the system was "aware" of the structure of the level and therefore could anticipate certain shots. For example, in the first level, when the path to the hill is about to turn left, the camera automatically starts looking towards the left too, thus anticipating the player's movements. The second type allows the player to control the camera relatively to [[Mario]]'s position. By pressing the left or right buttons, the camera rotates around Mario, while pressing up or down moves the camera closer or away from Mario.<ref>{{cite web |url=http://uk.gamespot.com/gamespot/features/video/15influential/p15_01.html |title=15 Most Influential Video Games of All Time: Super Mario 64 |publisher=GameSpot |access-date=22 March 2009 |archive-url=https://web.archive.org/web/20090326041036/http://uk.gamespot.com/gamespot/features/video/15influential/p15_01.html |archive-date=26 March 2009 |url-status=dead}}</ref><ref>{{cite web|url=http://www.1up.com/do/feature?cId=3135350 |title=The Essential 50 Part 36: Super Mario 64 from |publisher=1UP.com |access-date=22 March 2009}}</ref>

==Implementation==
There is a large body of research on how to implement a camera system.<ref>{{cite web|title=Cameracontrol.org: The virtual camera control bibliography|url=http://cameracontrol.org/blog/bibliography/|access-date=6 May 2011|archive-date=13 August 2011|archive-url=https://web.archive.org/web/20110813023237/http://cameracontrol.org/blog/bibliography/|url-status=dead}}</ref> The role of a [[Constraint programming|constraint solver software]] is to generate the best possible shot given a set of visual constraints. In other words, the constraint solver is given a requested shot composition such as "show this character and ensure that he covers at least 30 percent of the screen space". The solver will then use various methods to try to create a shot that would satisfy this request. Once a suitable shot is found, the solver outputs the coordinates and rotation of the camera, which can then be used by the graphic engine renderer to display the view.<ref>{{cite journal|last=Bares|first=William|author2=Scott McDermott|author3=Christina Boudreaux|author4=Somying Thainimit|year=2000|title=Virtual 3D camera composition from frame constraints|journal=International Multimedia Conference|publisher=Marina del Rey|location=California, United States|pages=177–186|url=http://www.cacs.louisiana.edu/~sdm1718/papers/Virtual_3D_Camera_Composition_from_Frame_Constraints.pdf|access-date=22 March 2009|archive-url=https://web.archive.org/web/20100710145126/http://www.cacs.louisiana.edu/~sdm1718/papers/Virtual_3D_Camera_Composition_from_Frame_Constraints.pdf|archive-date=10 July 2010|url-status=dead}}</ref>

In some camera systems, if no solution can be found, constraints are relaxed. For example, if the solver cannot generate a shot where the character occupies 30 percent of the screen space, it might ignore the screen space constraint and simply ensure that the character is visible at all.<ref>{{cite book|last=Drucker|first=Steven M.|author1-link=Steven M. Drucker|author2=David Zeltzer  |year=1995|title= CamDroid: A System for Implementing Intelligent Camera Control|journal=Symposium on Interactive 3D Graphics |isbn=978-0-89791-736-0|url=http://research.microsoft.com/en-us/um/people/sdrucker/papers/sig95symp.pdf|access-date=22 March 2009|archive-url=https://web.archive.org/web/20110605131715/http://research.microsoft.com/en-us/um/people/sdrucker/papers/sig95symp.pdf|archive-date=5 June 2011|url-status=dead}}</ref> Such methods include zooming out.

Some camera systems use predefined scripts to decide how to select the current shot for commonly seen shot scenarios called film idioms. Typically, the script is going to be triggered as a result of an action. For instance, when the player's character initiates a conversation with another character, the "conversation" script is going to be triggered. This script will contain instructions on how to "shoot" a two-character conversation. Thus the shots will be a combination of, for instance, [[over the shoulder shot]]s and [[close-up]] shots. Such script-based approaches may switch the camera between a set of predefined cameras or rely on a constraint solver to generate the camera coordinates to account for variability in scene layout. This scripted approach and the use of a constraint solver to compute virtual cameras was first proposed by Drucker.<ref>{{cite book|last=Drucker|first=Steven M.|author2=David Zeltzer |year=1995|title= CamDroid: A System for Implementing Intelligent Camera Control|journal=Symposium on Interactive 3D Graphics |isbn=978-0-89791-736-0|url=http://research.microsoft.com/en-us/um/people/sdrucker/papers/sig95symp.pdf|access-date=15 March 2015|archive-url=https://web.archive.org/web/20110605131715/http://research.microsoft.com/en-us/um/people/sdrucker/papers/sig95symp.pdf|archive-date=5 June 2011|url-status=dead}}</ref>  Subsequent research demonstrated how a script-based system could automatically switch cameras to view conversations between avatars in a realtime chat application.<ref>{{cite journal|last=He|first=Li-wei|author2=Michael F. Cohen|author2-link=Michael F. Cohen|author3=David H. Salesin|year=1996|title=The Virtual Cinematographer: A Paradigm for Automatic Real-Time Camera Control and Directing|journal=International Conference on Computer Graphics and Interactive Techniques|location=New York|volume=23rd|pages=217–224|url=http://grail.cs.washington.edu/pub/papers/virtcine.pdf|access-date=22 March 2009|archive-url=https://web.archive.org/web/20080828081453/http://grail.cs.washington.edu/pub/papers/virtcine.pdf|archive-date=28 August 2008|url-status=dead}}</ref>

Bill Tomlinson used a more original approach to the problem. He devised a system in which the camera is an [[autonomous agent]] with its own personality. The style of the shots and their rhythm will be affected by their mood. Thus a happy camera will "cut more frequently, spend more time in close-up shots, move with a bouncy, swooping motion, and brightly illuminate the scene".<ref>{{cite book|last=Tomlinson|first=Bill|author2=Bruce Blumberg |author3=Delphine Nain |title=Proceedings of the fourth international conference on Autonomous agents |chapter=Expressive autonomous cinematography for interactive virtual environments |date=2000 |isbn=978-1-58113-230-4|location=Barcelona, Spain|volume=4th|pages=317–324 |url=http://www.ics.uci.edu/~wmt/pubs/autonomousAgents00.pdf|access-date=22 March 2009|doi=10.1145/336595.337513|citeseerx=10.1.1.19.7502|s2cid=5532829|archive-url=https://web.archive.org/web/20050329122053/http://www.ics.uci.edu/~wmt/pubs/autonomousAgents00.pdf|archive-date=29 March 2005|url-status=live}}</ref>

While much of the prior work in automated virtual camera control systems has been directed towards reducing the need for a human to manually control the camera, the Director's Lens solution computes and proposes a palette of suggested virtual camera shots leaving the human operator to make the creative shot selection. In computing subsequent suggested virtual camera shots, the system analyzes the visual compositions and editing patterns of prior recorded shots to compute suggested camera shots that conform to continuity conventions such as not crossing the line of action, matching placement of virtual characters so they appear to look at one another across cuts, and favors those shots which the human operator had previously used in sequence.<ref>{{cite book |last=Lino|first=Christophe|author2=Marc Christie |author3=Roberto Ranon |author4=William Bares |title=Proceedings of the 19th ACM international conference on Multimedia |chapter=The director's lens |date=1 December 2011 |publisher=ACM |pages=323–332 |doi=10.1145/2072298.2072341|isbn=9781450306164|s2cid=14079689}}</ref>

==In mixed-reality applications==
In 2010, the [[Kinect]] was released by [[Microsoft]] as a [[3D scanner]]/[[webcam]] hybrid peripheral device which provides full-body detection of [[Xbox 360]] players and hands-free control of the user interfaces of video games and other software on the console. This was later modified by Oliver Kreylos<ref>{{cite web|url = http://idav.ucdavis.edu/~okreylos/index.html|title = Oliver Krelos' Homepage|access-date = 8 March 2011|archive-date = 23 December 2010|archive-url = https://web.archive.org/web/20101223213150/http://idav.ucdavis.edu/~okreylos/index.html|url-status = dead}}</ref> of [[University of California, Davis]] in a series of YouTube videos which showed him combining the Kinect with a PC-based virtual camera.<ref>{{cite web|url = http://www.tomshardware.com/news/Kinect-Oliver-Kreylos-Hack-3D-Video-Capture-Hector-Martin,11640.html|title = Kinect Used As 3D Video Capture Tool|author = Kevin Parrish|publisher = Tom's Hardware|date = 17 November 2010}}</ref> Because the Kinect is capable of detecting a full range of depth (through [[computer stereo vision]] and [[Structured light]]) within a captured scene, Kreylos demonstrated the capacity of the Kinect and the virtual camera to allow free-viewpoint navigation of the range of depth, although the camera could only allow video capture of the scene as shown to the front of the Kinect, resulting in fields of black, empty space where the camera was unable to capture video within the field of depth. Later, Kreylos demonstrated a further elaboration on the modification by combining the video streams of two Kinects in order to further enhance the video capture within the view of the virtual camera.<ref>{{cite web|url = https://www.engadget.com/2010/11/29/two-kinects-join-forces-to-create-better-3d-video-blow-our-mind/|title = Two Kinects join forces to create better 3D video, blow our minds (video)|author = Tim Stevens|publisher = Engadget|date = 29 November 2010<!-- 1:21PM-->}}</ref> Kreylos' developments using the Kinect were covered among the works of others in the Kinect hacking and [[homebrew (video games)|homebrew]] community in a [[New York Times]] article.<ref>{{cite web|url = https://www.nytimes.com/2010/11/22/technology/22hack.html|title = With Kinect Controller, Hackers Take Liberties|author = Jenna Wortham|work = [[The New York Times]]|date = 21 November 2010}}</ref>

==Real-time recording and motion tracking==
Virtual cameras have been developed which allow a director to film [[motion capture]] and view the digital character's movements in real time<ref>{{cite web |url=http://www.popsci.com/entertainment-amp-gaming/article/2009-02/virtual-camera-captures-actors’-movements-iresident-evil-5i |archive-url=https://web.archive.org/web/20090302073338/http://www.popsci.com/entertainment-amp-gaming/article/2009-02/virtual-camera-captures-actors%E2%80%99-movements-iresident-evil-5i |title="Virtual Camera" Captures Actors' Movements for Resident Evil 5 |last=Hsu |first=Jeremy |date=27 February 2009 |work=[[Popular Science]] |archive-date=2 March 2009 |url-status=dead}}</ref> in a pre-constructed digital environment, such as a house or spaceship.<ref name=wired>{{cite magazine |url=https://www.wired.com/2009/02/resident-evil-5-6/ |title=Resident Evil 5 Offers Sneak Peek at Avatar's 'Virtual Camera' |last=Lewinski |first=John Scott |date=27 February 2009 |magazine=[[Wired (magazine)|Wired]] |access-date=25 February 2015}}</ref> ''[[Resident Evil 5]]'' was the first video game to use the technology,<ref>{{cite web |url=http://www.ign.com/articles/2009/02/28/the-tech-behind-re5 |title=The Tech Behind RE5 |last=Lowe |first=Scott |date=27 February 2009 |work=[[IGN]] |access-date=24 February 2015}}</ref> which was developed for the 2009 film ''[[Avatar (2009 film)|Avatar]]''.<ref name=wired/><ref>{{cite web |url=http://www.popularmechanics.com/culture/movies/a5067/4339455/ |title=How James Cameron's Innovative New 3D Tech Created Avatar |last=Thompson |first=Anne |date=1 January 2010 |work=[[Popular Mechanics]] |access-date=25 February 2015}}</ref>
The use of [[motion capture]] to control the position and orientation of a virtual camera enables the operator to intuitively move and aim the virtual camera by simply walking about and turning the virtual camera rig. A virtual camera rig consists of a portable monitor or tablet device, motion sensors, an optional support framework, and optional joystick or button controls that are commonly used to start or stop recording and adjust lens properties.<ref>{{cite web |url=https://www.optitrack.com/products/insight-vcs/|title=Optitrack InsightVCS|access-date=15 March 2015}}</ref> In 1992, Michael McKenna of MIT's Media Lab demonstrated the earliest documented virtual camera rig when he fixed a Polhemus magnetic motion sensor and a 3.2 inch portable LCD TV to a wooden ruler.<ref>{{cite book |author=Michael McKenna |title=Proceedings of the 1992 symposium on Interactive 3D graphics - SI3D '92 |date=March 1992 |chapter=Interactive viewpoint control and three-dimensional operations |publisher=ACM |pages=53–56 |doi=10.1145/147156.147163|isbn=978-0897914673 |citeseerx=10.1.1.132.8599 |s2cid=17308648}}</ref> The Walkthrough Project at the University of North Carolina at Chapel Hill produced a number of physical input devices for virtual camera view control including dual three-axis joysticks and a billiard-ball shaped prop known as the UNC Eyeball that featured an embedded six-degree of freedom motion tracker and a digital button.<ref>{{cite journal |author=Frederick Brooks Jr. |date=June 1992 |title=Final Technical Report – Walkthrough Project |url=http://www.cs.unc.edu/techreports/92-026/92-026.pdf |journal=Tr92-026 |publisher=University of North Carolina at Chapel Hill |access-date=23 March 2015 |archive-url=https://web.archive.org/web/20150923211411/http://www.cs.unc.edu/techreports/92-026/92-026.pdf |archive-date=23 September 2015 |url-status=live}}</ref>

==See also==
* [[Camera matrix]]
* [[Game engine]]
* [[Virtual cinematography]]
* [[First-person (video games)]]

==References==
{{reflist}}

{{good article}}

[[Category:Video game development]]
[[Category:Cameras]]
[[Category:Video game graphics]]