Warning: That file was not part of the compilation database. It may have many parsing errors.

1/****************************************************************************
2**
3** Copyright (C) 2015 Klaralvdalens Datakonsult AB (KDAB).
4** Contact: https://www.qt.io/licensing/
5**
6** This file is part of the documentation of the Qt Toolkit.
7**
8** $QT_BEGIN_LICENSE:FDL$
9** Commercial License Usage
10** Licensees holding valid commercial Qt licenses may use this file in
11** accordance with the commercial license agreement provided with the
12** Software or, alternatively, in accordance with the terms contained in
13** a written agreement between you and The Qt Company. For licensing terms
14** and conditions see https://www.qt.io/terms-conditions. For further
15** information use the contact form at https://www.qt.io/contact-us.
16**
17** GNU Free Documentation License Usage
18** Alternatively, this file may be used under the terms of the GNU Free
19** Documentation License version 1.3 as published by the Free Software
20** Foundation and appearing in the file included in the packaging of
21** this file. Please review the following information to ensure
22** the GNU Free Documentation License version 1.3 requirements
23** will be met: https://www.gnu.org/licenses/fdl-1.3.html.
24** $QT_END_LICENSE$
25**
26****************************************************************************/
27
28/*!
29 \page qt3drender-framegraph.html
30 \title Qt 3D Render Framegraph
31
32 \brief A framegraph is the data structure that controls how a scene is
33 rendered.
34
35 The Qt 3D Render aspect allows for the rendering algorithm to be entirely
36 data-driven. The controlling data structure is known as the \e framegraph.
37 Similar to how the Qt 3D ECS (entity component system) allows you to define
38 a so-called Scenegraph by building a scene from a tree of Entities and
39 Components, the framegraph is also a tree structure but one used for a
40 different purpose. Namely, controlling \e how the scene is rendered.
41
42 Over the course of rendering a single frame, a 3D renderer will likely
43 change state many times. The number and nature of these state changes
44 depends upon not only which materials (shaders, mesh geometry, textures and
45 uniform variables) are found within the scene, but also upon which high
46 level rendering scheme you are using.
47
48 For example, using a traditional simple \e{forward rendering} scheme is
49 very different to using a \e{deferred rendering} approach. Other features
50 such as reflections, shadows, multiple viewports, and early z-fill passes
51 all change which states a renderer needs to set over the course of a frame
52 and when those state changes need to occur.
53
54 As a comparison, the \l {Qt Quick Scene Graph}{Qt Quick 2
55 scenegraph renderer} responsible for drawing Qt Quick 2 scenes is
56 hard-wired in C++ to do things like batching of primitives and rendering
57 opaque items followed by rendering of transparent items. In the case of Qt
58 Quick 2 that is perfectly fine as that covers all of the requirements. As
59 you can see from some of the examples listed above, such a hard-wired
60 renderer is not likely to be flexible enough for generic 3D scenes given
61 the multitude of rendering methods available. Or if a renderer could be
62 made flexible enough to cover all such cases, its performance would likely
63 suffer from being too general. To make matters worse, more rendering
64 methods are being researched all of the time. We therefore needed an
65 approach that is \e {both flexible and extensible} whilst being simple to
66 use and maintain. Enter the framegraph!
67
68 Each node in the framegraph defines a part of the configuration the
69 renderer will use to render the scene. The position of a node in the
70 framegraph tree determines when and where the subtree rooted at that node
71 will be the active configuration in the rendering pipeline. As we will see
72 later, the renderer traverses this tree in order to build up the state
73 needed for your rendering algorithm at each point in the frame.
74
75 Obviously if you just want to render a simple cube onscreen you may think
76 this is overkill. However, as soon as you want to start doing slightly more
77 complex scenes this comes in handy. For the common cases, Qt 3D provides
78 some example framegraphs that are ready to use out of the box.
79
80 We will demonstrate the flexibility of the framegraph concept by presenting a few
81 examples and the resulting framegraphs.
82
83 Please note that unlike the Scenegraph which is composed of Entities and
84 Components, the framegraph is only composed of nested nodes which are all
85 subclasses of Qt3DRender::QFrameGraphNode. This is because the framegraph nodes
86 are not simulated objects in our virtual world, but rather supporting
87 information.
88
89 We will soon see how to
90 construct our first simple framegraph but before that we will introduce
91 the framegraph nodes available to you. Also as with the Scenegraph tree,
92 the QML and C++ APIs are a 1 to 1 match so you can favor the one you like
93 best. For the sake of readability and conciseness, the QML API was chosen
94 for this article.
95
96 \omit
97 TODO: Add list of framegraph node types
98 \endomit
99
100 The beauty of the framegraph is that combining these simple node types, it
101 is possible to configure the renderer to suit your specific needs without
102 touching any hairy, low-level C/C++ rendering code at all.
103
104 \section1 FrameGraph Rules
105
106 In order to construct a correctly functioning framegraph tree,
107 you should know a few rules about how it is traversed and how to feed it to
108 the Qt 3D renderer.
109
110 \section2 Setting the Framegraph
111
112 The FrameGraph tree should be assigned to the activeFrameGraph property of
113 a QRenderSettings component, itself being a component of the root entity in
114 the Qt 3D scene. This is what makes it the active framegraph for the
115 renderer. Of course, since this is a QML property binding, the active
116 framegraph (or parts of it) can be changed on the fly at runtime. For
117 example, if you want to use different rendering approaches for indoor and
118 outdoor scenes or to enable or disable some special effect.
119
120 \badcode
121 Entity {
122 id: sceneRoot
123 components: RenderSettings {
124 activeFrameGraph: ... // FrameGraph tree
125 }
126 }
127 \endcode
128
129 \note activeFrameGraph is the default property of the FrameGraph component
130 in QML.
131
132 \badcode
133 Entity {
134 id: sceneRoot
135 components: RenderSettings {
136 ... // FrameGraph tree
137 }
138 }
139 \endcode
140
141 \section2 How the Framegraph Is Used
142
143 \list
144 \li The Qt 3D renderer performs a \e{depth first traversal} of the
145 framegraph tree. Note that, because the traversal is depth first,
146 the \e {order in which you define nodes is important}.
147 \li When the renderer reaches a leaf node of the framegraph, it
148 collects together all of the state specified by the path from the
149 leaf node to the root node. This defines the state used to render
150 a section of the frame. If you are interested in the internals of
151 Qt 3D, this collection of state is called a \e RenderView.
152 \li Given the configuration contained in a RenderView, the renderer
153 collects together all of the Entities in the Scenegraph to be
154 rendered, and from them builds a set of \e RenderCommands and
155 associates them with the RenderView.
156 \li The combination of RenderView and set of RenderCommands is passed
157 over for submission to OpenGL.
158 \li When this is repeated for each leaf node in the framegraph, the
159 frame is complete and the renderer calls
160 QOpenGLContext::swapBuffers() to display the frame.
161 \endlist
162
163 At its heart, the framegraph is a data-driven method for configuring the
164 Qt 3D renderer. Due to its data-driven nature, we can change configuration
165 at runtime, allow non-C++ developers or designers to change the structure
166 of a frame, and try out new rendering approaches without having to write
167 thousands of lines of boiler plate code.
168
169
170 \section1 Framegraph Examples
171
172 Now that you know the rules to abide by when writing a framegraph tree, we
173 will go over a few examples and break them down.
174
175 \section2 A Simple Forward Renderer
176
177 Forward rendering is when you use OpenGL in its traditional manner and
178 render directly to the backbuffer one object at a time shading each one as
179 we go. This is opposed to \l {Deferred Renderer}{deferred rendering} where
180 we render to an intermediate \e G-buffer. Here is a simple FrameGraph that
181 can be used for forward rendering:
182
183 \badcode
184 Viewport {
185 normalizedRect: Qt.rect(0.0, 0.0, 1.0, 1.0)
186 property alias camera: cameraSelector.camera
187
188 ClearBuffers {
189 buffers: ClearBuffers.ColorDepthBuffer
190
191 CameraSelector {
192 id: cameraSelector
193 }
194 }
195 }
196 \endcode
197
198 As you can see, this tree has a single leaf and is composed of 3 nodes in
199 total as shown in the following diagram.
200
201 \image simple-framegraph.png
202
203 Using the rules defined \l {Framegraph Rules}{above}, this framegraph tree yields a single
204 RenderView with the following configuration:
205
206 \list
207 \li Leaf Node -> RenderView
208 \list
209 \li Viewport that fills the entire screen (uses normalized
210 coordinates to make it easy to support nested viewports)
211 \li Color and Depth buffers are set to be cleared
212 \li Camera specified in the exposed camera property
213 \endlist
214 \endlist
215
216 Several different FrameGraph trees can produce the same rendering result.
217 As long as the state collected from leaf to root is the same, the result
218 will also be the same. It is best to put state that remains constant longest
219 nearer to the root of the framegraph as this will result in fewer leaf
220 nodes, and hence, fewer RenderViews overall.
221
222 \badcode
223 Viewport {
224 normalizedRect: Qt.rect(0.0, 0.0, 1.0, 1.0)
225 property alias camera: cameraSelector.camera
226
227 CameraSelector {
228 id: cameraSelector
229
230 ClearBuffers {
231 buffers: ClearBuffers.ColorDepthBuffer
232 }
233 }
234 }
235 \endcode
236
237 \badcode
238 CameraSelector {
239 Viewport {
240 normalizedRect: Qt.rect(0.0, 0.0, 1.0, 1.0)
241
242 ClearBuffers {
243 buffers: ClearBuffers.ColorDepthBuffer
244 }
245 }
246 }
247 \endcode
248
249 \section2 A Multi Viewport FrameGraph
250
251 Let us move on to a slightly more complex example that renders a Scenegraph
252 from the point of view of 4 virtual cameras into the 4 quadrants of the
253 window. This is a common configuration for 3D CAD or modelling tools or
254 could be adjusted to help with rendering a rear-view mirror in a car racing
255 game or a CCTV camera display.
256
257 \image multiviewport.png
258
259 \badcode
260 Viewport {
261 id: mainViewport
262 normalizedRect: Qt.rect(0, 0, 1, 1)
263 property alias Camera: cameraSelectorTopLeftViewport.camera
264 property alias Camera: cameraSelectorTopRightViewport.camera
265 property alias Camera: cameraSelectorBottomLeftViewport.camera
266 property alias Camera: cameraSelectorBottomRightViewport.camera
267
268 ClearBuffers {
269 buffers: ClearBuffers.ColorDepthBuffer
270 }
271
272 Viewport {
273 id: topLeftViewport
274 normalizedRect: Qt.rect(0, 0, 0.5, 0.5)
275 CameraSelector { id: cameraSelectorTopLeftViewport }
276 }
277
278 Viewport {
279 id: topRightViewport
280 normalizedRect: Qt.rect(0.5, 0, 0.5, 0.5)
281 CameraSelector { id: cameraSelectorTopRightViewport }
282 }
283
284 Viewport {
285 id: bottomLeftViewport
286 normalizedRect: Qt.rect(0, 0.5, 0.5, 0.5)
287 CameraSelector { id: cameraSelectorBottomLeftViewport }
288 }
289
290 Viewport {
291 id: bottomRightViewport
292 normalizedRect: Qt.rect(0.5, 0.5, 0.5, 0.5)
293 CameraSelector { id: cameraSelectorBottomRightViewport }
294 }
295 }
296 \endcode
297
298 This tree is a bit more complex with 5 leaves. Following the same rules as
299 before we construct 5 RenderView objects from the FrameGraph. The following
300 diagrams show the construction for the first two RenderViews. The remaining
301 RenderViews are very similar to the second diagram just with the other
302 sub-trees.
303
304 \image multiviewport-1.png
305
306 \image multiviewport-2.png
307
308 In full, the RenderViews created are:
309
310 \list
311 \li RenderView (1)
312 \list
313 \li Fullscreen viewport defined
314 \li Color and Depth buffers are set to be cleared
315 \endlist
316
317 \li RenderView (2)
318 \list
319 \li Fullscreen viewport defined
320 \li Sub viewport defined (rendering viewport will be scaled relative to its parent)
321 \li CameraSelector specified
322 \endlist
323
324 \li RenderView (3)
325 \list
326 \li Fullscreen viewport defined
327 \li Sub viewport defined (rendering viewport will be scaled relative to its parent)
328 \li CameraSelector specified
329 \endlist
330
331 \li RenderView (4)
332 \list
333 \li Fullscreen viewport defined
334 \li Sub viewport defined (rendering viewport will be scaled relative to its parent)
335 \li CameraSelector specified
336 \endlist
337
338 \li RenderView (5)
339 \list
340 \li Fullscreen viewport defined
341 \li Sub viewport defined (rendering viewport will be scaled relative to its parent)
342 \li CameraSelector specified
343 \endlist
344 \endlist
345
346 However, in this case the \e {order is important}. If the ClearBuffers node
347 were to be the last instead of the first, this would result in a black
348 screen for the simple reason that everything would be cleared right after
349 having been so carefully rendered. For a similar reason, it could not be
350 used as the root of the FrameGraph as that would result in a call to clear
351 the whole screen for each of our viewports.
352
353 Although the declaration order of the FrameGraph is important, Qt 3D is able
354 to process each RenderView in parallel as each RenderView is independent of
355 the others for the purposes of generating a set of RenderCommands to be
356 submitted whilst the RenderView's state is in effect.
357
358 Qt 3D uses a task-based approach to parallelism which naturally scales up
359 with the number of available cores. This is shown in the following diagram
360 for the previous example.
361
362 \image framegraph-parallel-build.png
363
364 The RenderCommands for the RenderViews can be generated in parallel across
365 many cores, and as long as we take care to submit the RenderViews in the
366 correct order on the dedicated OpenGL submission thread, the resulting
367 scene will be rendered correctly.
368
369 \section2 Deferred Renderer
370
371 When it comes to rendering, deferred rendering is a different beast in
372 terms of renderer configuration compared to forward rendering. Instead of
373 drawing each mesh and applying a shader effect to shade it, deferred
374 rendering adopts a \e {two render pass} method.
375
376 First all the meshes in the scene are drawn using the same shader that will
377 output, usually for each fragment, at least four values:
378
379 \list
380 \li World normal vector
381 \li Color (or some other material properties)
382 \li Depth
383 \li World position vector
384 \endlist
385
386 Each of these values will be stored in a texture. The normal, color, depth,
387 and position textures form what is called the G-Buffer. Nothing is drawn
388 onscreen during the first pass, but rather drawn into the G-Buffer ready
389 for later use.
390
391 Once all the meshes have been drawn, the G-Buffer is filled with all the
392 meshes that can currently be seen by the camera. The second render pass is
393 then used to render the scene to the back buffer with the final color
394 shading by reading the normal, color, and position values from the G-buffer
395 textures and outputting a color onto a full screen quad.
396
397 The advantage of that technique is that the heavy computing power required
398 for complex effects is only used during the second pass only on the
399 elements that are actually being seen by the camera. The first pass does
400 not cost much processing power as every mesh is being drawn with a simple
401 shader. Deferred rendering, therefore, decouples shading and lighting from
402 the number of objects in a scene and instead couples it to the resolution
403 of the screen (and G-Buffer). This is a technique that has been used in
404 many games due to the ability to use large numbers of dynamic lights at
405 the expense of additional GPU memory usage.
406
407\qml
408Viewport {
409 id: root
410 normalizedRect: Qt.rect(0.0, 0.0, 1.0, 1.0)
411
412 property GBuffer gBuffer
413 property alias camera: sceneCameraSelector.camera
414 property alias sceneLayer: sceneLayerFilter.layers
415 property alias screenQuadLayer: screenQuadLayerFilter.layers
416
417 RenderSurfaceSelector {
418
419 CameraSelector {
420 id: sceneCameraSelector
421
422 // Fill G-Buffer
423 LayerFilter {
424 id: sceneLayerFilter
425 RenderTargetSelector {
426 id: gBufferTargetSelector
427 target: gBuffer
428
429 ClearBuffers {
430 buffers: ClearBuffers.ColorDepthBuffer
431
432 RenderPassFilter {
433 id: geometryPass
434 matchAny: FilterKey {
435 name: "pass"
436 value: "geometry"
437 }
438 }
439 }
440 }
441 }
442
443 TechniqueFilter {
444 parameters: [
445 Parameter { name: "color"; value: gBuffer.color },
446 Parameter { name: "position"; value: gBuffer.position },
447 Parameter { name: "normal"; value: gBuffer.normal },
448 Parameter { name: "depth"; value: gBuffer.depth }
449 ]
450
451 RenderStateSet {
452 // Render FullScreen Quad
453 renderStates: [
454 BlendEquation { blendFunction: BlendEquation.Add },
455 BlendEquationArguments {
456 sourceRgb: BlendEquationArguments.SourceAlpha
457 destinationRgb: BlendEquationArguments.DestinationColor
458 }
459 ]
460
461 LayerFilter {
462 id: screenQuadLayerFilter
463 ClearBuffers {
464 buffers: ClearBuffers.ColorDepthBuffer
465 RenderPassFilter {
466 matchAny: FilterKey {
467 name: "pass"
468 value: "final"
469 }
470 parameters: Parameter {
471 name: "winSize"
472 value: Qt.size(1024, 768)
473 }
474 }
475 }
476 }
477 }
478 }
479 }
480 }
481}
482\endqml
483
484 (Above code is adapted from \e {qt3d/tests/manual/deferred-renderer-qml}.)
485
486 Graphically, the resulting framegraph looks like:
487
488 \image deferred-framegraph.png
489
490 And the resulting RenderViews are:
491
492 \list
493 \li RenderView (1)
494 \list
495 \li Specify which camera should be used
496 \li Define a viewport that fills the whole screen
497 \li Select all Entities for layer component sceneLayer
498 \li Set the \c gBuffer as the active render target
499 \li Clear the color and depth on the currently bound render target
500 (the \c gBuffer)
501 \li Select only Entities in the scene that have a Material and
502 Technique matching the annotations in the RenderPassFilter
503 \endlist
504
505 \li RenderView (2)
506 \list
507 \li Define a viewport that fills the whole screen
508 \li Select all Entities for layer component screenQuadLayer
509 \li Clear the color and depth buffers on the currently bound
510 framebuffer (the screen)
511 \li Select only Entities in the scene that have a Material and
512 Technique matching the annotations in the RenderPassFilter
513 \endlist
514 \endlist
515
516 \section1 Other Benefits of the framegraph
517
518 Since the FrameGraph tree is entirely data-driven and can be modified dynamically at runtime, you can:
519
520 \list
521 \li Have different framegraph trees for different platforms and
522 hardware and select the most appropriate at runtime
523 \li Easily add and enable visual debugging in a scene
524 \li Use different FrameGraph trees depending on the nature of what
525 you need to render for a particular region of the scene
526 \li Implement a new rendering technique without having to
527 modify Qt 3D's internals
528 \endlist
529
530 \section1 Conclusion
531
532 We have introduced the FrameGraph and the node types that compose it. We
533 then went on to discuss a few examples to illustrate the framegraph
534 building rules and how the Qt 3D engine uses the framegraph behind the
535 scenes. By now you should have a pretty good overview of the FrameGraph and
536 how it can be used (perhaps to add an \l {early z-fill pass} to a
537 forward renderer). Also you should always keep in mind that the FrameGraph
538 is a tool for you to use so that you are not tied down to the provided
539 renderer and materials that Qt 3D provides out of the box.
540*/
541

Warning: That file was not part of the compilation database. It may have many parsing errors.