Hardware accelerated rendering

From Rust Community Wiki
Jump to navigation Jump to search

Note: This page currently only handles 3D rendering. It will eventually handle 2D rendering.

Hardware accelerated rendering makes use of specialized hardware, usually a graphics processing unit (GPU), to speed up the process of rendering an image. Many real-time graphics applications make use of hardware acceleration.

A little bit of history[edit | edit source]

Note: This section is incomplete, and is in need of some fleshing out.

See also: History of computer graphics, History of graphics processing units

Early computer graphics applications like Spacewar! (1962), Sketchpad (1963) and Pong (1972) used vector graphics and rendered them to CRT displays. The computers driving the displays would continuously execute a set of drawing instructions which would steer the beam of the cathode-ray tube.[citation needed][is it?] While vector graphics allow for "infinite" resolution, there are limits to how fast you can steer the beam of a cathode-ray tube and still get a reasonable image.[citation needed] Shapes which need an interior color other than black take an inordinate amount of time to draw compared to just drawing the outline.[citation needed]

Raster scanning had at that point already been in use by television systems for some time, yet this approach to displaying computer-generated imagery remained unfeasible until computer memory which could store an entire image became available. This was realized through semiconductor-based memory, which enabled the development of SuperPaint (1973), a system capable of video editing and computer animation. It could store an entire frame's worth of color information in its framebuffer.[1][2][3]

[Mention early fixed-pipeline hardware in consoles, arcade machines and computer expansion cards].

[Mention early re-programmable pipelines in graphics hardware].

[Beginnings of OpenGL].

[DirectX and OpenGL almost kind-of-sort-of merging?].

[Problems with OpenGL]. Fixed Function Pipeline

[OpenGL recieves various extensions which expose more and more lower-level bits of functionality].

AMD announces in September of 2013 that they're working with DICE to develop Mantle, a console-like API which lies closer to the metal.[4] In doing so, AMD eventually sparked a revolution in the graphics API space.

Microsoft announces DirectX 12 at GDC 2014.[5]

Apple announces Metal on June 2, 2014.[6]

The Khronos Group announces that they're working on a replacement for OpenGL[7] later refered to as "glNext".[8]

AMD shifts gears in response to DirectX12 and the "glNext" initiative and donates the Mantle API to the Khronos Group to help kick-start the development of "glNext". During GDC 2015, the Khronos Group announces Vulkan, the result of the "glNext" initiative.[9][10]

[Maybe also WebGPU].

A Brief Overview of the Various 3D Graphics APIs[edit | edit source]

OpenGL

The oldest of the bunch. Deprecated by Apple on their platforms in 2018 in favor of Metal.

OpenGL ES

A stripped down version of OpenGL for use on resource-constrained systems.

WebGL

A JavaScript API based on OpenGL ES for use in web browsers.

DirectX < 12

Microsoft's OpenGL. Supported on Windows and Xbox. Uses COM, which can be frustrating to work with in Rust.

WebGPU

The result of the "WebGL Next" initiative. A simplified version of Vulkan intended for use on the web.

In-browser implementations are currently in progress and you can now run examples in both Firefox Nightly and Chrome Canary with their experimental WebGPU feature flags enabled. Note that Chrome Canary does not implement GPU sandboxing at the moment, so sites using the WebGPU API can read the GPU memory of other processes.

Native implementations, Cargo vec.svgwgpu-core and dawn, also exist.

Metal

Apple's low-level 3D graphics API. Object-oriented. Supported on macOS and iOS. Similar to Vulkan.

DirectX 12

Microsoft's latest and greatest 3D graphics API. Supported on Windows 10 and the Xbox One and later. DirectX 12 Ultimate will make the API more portable between Windows and the Xbox. Like previous versions of DirectX, DirectX 12 uses COM. Similar to Vulkan.

Vulkan

Derived from AMDs Mantle. Exposes a C-based interface. Practically every structure defined by the specification is prefixed with both a VkStructureType sType and const void* pNext field. It helps make the API extensible, but it requires the use of unsafe on the Rust side of things.

Not supported on Apple's platforms, but can be run on top of Metal through projects like gfx-portability and MoltenVK.

General Concepts[edit | edit source]

While the concepts outlined below should be mostly relevant for all of the modern 3D graphics APIs, the technical terms and explanations will mostly be drawing from how Vulkan does things.

Vertex

An application-defined primitive which is fed into and processed by the graphics pipeline.

A vertex often contains a position, but it does not have to do so.

Fragments

Shaders

Shaders are small programs which run directly on the GPU as a part of the graphics pipeline.

OpenGL expects shaders to be fed into the API as GLSL text.

Vulkan expects shaders to be compiled down to a binary representation called SPIR-V.

Metal expects shaders in "Metal Shading Language"

DirectX expect shaders in HLSL.

Vertex Shader

Runs on a per-vertex basis. This is where you transform, rotate and scale your 3D models in world space. This is also where you perform perspective projection.

Fragment Shader

Runs on a per-fragment basis. This is where the color of the fragments are decided. This is misleadingly called the pixel shader in Direct3D.

Surface[edit | edit source]

The surface is what the graphics pipeline renders to. This is usually a window.

OpenGL-Specific Concepts[edit | edit source]

The OpenGL context...

Graphics Rendering In Rust[edit | edit source]

A Brief history of graphics rendering in Rust[edit | edit source]

glium[edit | edit source]

The earliest commit available in Glium's repository is was created on October 3, 2014, predating the announcement of Vulkan. Given the size of the commit, it is safe to assume that work on Glium started some time before the work was commited.

On August 27, 2016, Glium's author posted a "post-mortem" on URLO detailing things which he believes "went wrong" with Glium. The author had already reduced his activity on Glium to mostly merging pull requests and eventually seems to have handed the reins over to someone else.

gfx-rs[edit | edit source]

The gfx-rs project is perhaps the most notable effort within graphics rendering in Rust.

Initially, the gfx crate could only run on OpenGL, but gained support for DirectX 11 with the release of 0.10 on March 21, 2016.[11] It would later gain support for both Metal and Vulkan with the release of 0.13.

It became apparent as time went on that they needed target a lower level of abstraction to achieve the performance they were after. This led to the birth of CoreLL and later gfx-hal.[12] gfx-hal was eventually adopted by WebRender[13], a GPU-based 2D rendering engine intended for use in Firefox and Servo.[14]

The gfx repository's main branch is today home to the gfx-hal crate and its various backends.

wgpu[edit | edit source]

The gfx project eventually gave rise to Cargo vec.svgwgpu (home) an implementation of the experimental WebGPU API.

Rust Ecosystem[edit | edit source]

The graphics ecosystem for Rust is divided in 3 main categories:

  • Rust wrappers, which make it easier to interact in safe Rust with a given C API.
  • Raw FFI bindings, which only output the raw methods a C library will have.
  • Multi-API crates, which provide a single interface to use whatever backend they support.

In addition to these main categories, there are a couple of auxiliary categories of crates which you may or may not need depending on what you plan to do.

  • Shader compilers. When working with Vulkan or the gfx-hal stack, you're going to need a shader compiler of some sort. While it is possible to pre-compile shaders using external command-line tools, these crates let you do it in your own Rust code.
  • Windowing crates. All of the above will let you draw content, but you usually need something to draw to: a window. These crates will allow you to create one or multiple windows for your program. These crates aren't strictly necessary if you're only leveraging the compute-capabilities of some of these APIs or if you're only generating some output file (e.g. a .gif or a PNG).

An important crate to note is Cargo vec.svgraw-window-handle . This crate provides a single trait: HasRawWindowHandle. This trait makes it easier for graphics libraries and windowing libraries to decouple from one another. For example, instead of hard-coding every window library that might exist, wgpu instead can be used with any windowing library which uses the raw-window-handle crate.

Rust wrappers[edit | edit source]

Crate Backend raw-window-handle Maintenance Status Description
Cargo vec.svgglium OpenGL no Abandoned by original author. Maintained by community TODO
Cargo vec.svgglow OpenGL no unknown TODO
Cargo vec.svgash Vulkan Through Cargo vec.svgash-window Actively Maintained A very low-level and lightweight wrapper around the Vulkan API. It is automatically generated from vk.xml. It is used indirectly by gfx-hal through gfx-backend-vulkan.
Cargo vec.svgerupt Vulkan yes Actively Maintained Lightweight and low-level bindings to the Vulkan API, fully automatically generated from the Vulkan Headers, with a focus on extensions, documentation and utils. It was newly released and may be unstable. Additionally erupt has a low userbase (~156 crates.io downloads as of May 2nd 2020).
Cargo vec.svgvulkano Vulkan no unknown A Rust wrapper for Vulkan that tries to be convenient to use and prevent invalid usage of the API. Vulkano is by no means complete, but its author believes that the library won't undergo any major breaking changes at this point.

Raw FFI bindings[edit | edit source]

Crate API Description
Cargo vec.svggl OpenGL
Cargo vec.svgmetal Metal
Cargo vec.svgvk-sys Vulkan Used by Vulkano.

Multi-API crates[edit | edit source]

Crate WebGPU Vulkan Direct3D Metal OpenGL raw-window-handle Maintenance Status Description
Cargo vec.svgluminance no no no no yes no Actively Maintained A high level graphics api that uses closures to control pipline state. Built for opengl 3.3 only at present, but further backends are planned.
Cargo vec.svggfx-hal Planned yes 11/12 yes In the works yes Actively maintained A fairly low-level library which tries to not abstract away the details of the underlying APIs too much. It's developed around the idea of the Backend trait, which lets it run on top of various native 3D graphics APIs. All major types in the gfx-hal library have a generic parameter with Backend as a trait bound. Implementations of the `Backend` trait are provided by the various gfx-backend-* crates.
Cargo vec.svgrendy Planned yes 11/12 yes In the works yes Stalled A library created in combination with the Amethyst game engine. The library is not tied to the game engine and can as such be used by anyone. It is based on gfx-hal and seems to aim to offer a safer API without sacrificing a great deal of features and performance. Much like gfx-hal, rendy makes its types generic over gfx-hals Backend trait.
Cargo vec.svgwgpu (home) yes yes 11/12 yes In the works yes Actively maintained A Rust wrapper around Cargo vec.svgwgpu-core when compiled for native platforms. When compiled for the web, it uses wasm-bindgen and web-sys to interact with the WegGPU API. It's shares a lot of similarities with Vulkan in how the API is structured, but it's a lot simpler and exposes no unsafe functionality.

Most people looking for something performant and easy to use should pick wgpu.

Cargo vec.svggfx (home) yes 11 yes yes no Deprecated gfx of the pre-ll variety provides a higher-level API which is more suitable for the average user. The spiritual successor to gfx-pre-ll is wgpu-rs.

Shader Compilers[edit | edit source]

You'll probably need one of these if you're going to use a library which wants to recieve shaders as SPIR-V.

Crate GLSL HLSL MSL Underlying Library Maintenance Status Description
Cargo vec.svgshaderc (home) yes yes yes C/C++ Actively maintained Rust wrapper around shaderc, Google's collection of tools for Vulkan shader compilation. Takes a while to compile if you don't have a pre-compiled version.
Cargo vec.svgglsl-to-spirv yes unknown no Pure Rust Deprecated While the library is written in Rust, all it does is shell out to glslangValidator. Not to be confused with the other glsl-to-spirv.
Cargo vec.svgnaga (home) 🚧 no no Pure Rust Experimental Describes itself as a shader translation library.

Shader reflection[edit | edit source]

It is sometimes useful to determine the type, number and position of the inputs and outputs of a shader at runtime. The OpenGL specification includes functions for this purpose, but such functionality is intentionally left out of the Vulkan specification to lessen the work hardware vendors have to do. The following libraries aim to offer the functionality you're "missing out on" by using Vulkan.

Crate Implementation Language Description
Cargo vec.svgrspirv-reflect Rust
Cargo vec.svgspirv-reflect C++ Wrapper around KhronosGroup/SPIRV-Reflect.
Cargo vec.svgspirq Rust

Shader embedding[edit | edit source]

These crates offer macros which compile shaders defined in other files and then embed SPIR-V inside your executable (or library) in a manner similar to the include_strThis links to official Rust documentation and include_bytesThis links to official Rust documentation macros in the standard library.

Crate Notes
Cargo vec.svginclude-glsl Poorly documented. Infers shader stage from file extension.
Cargo vec.svginline-spirv Also allows inline shader source code. Lets you specify the shader stage, entry point, shader language and additional #DEFINE and #include directives.

Surface creation[edit | edit source]

Displaying the result of your rendering code on a window requires some level of interaction with your windowing library. This could either manifest as the windowing library providing such directly, or as the windowing library offering Cargo vec.svgraw-window-handle support, which offloads the work of creating the "surface" to another library.

Crate Made For raw-window-handle Implemented in Description
Cargo vec.svgglutin OpenGL no Rust Offers surface management for OpenGL as well as re-exporting all of winit. This has been a cause of confusion for some and have caused them to report issues with winit on Glutin's issue tracker.
Cargo vec.svgsurfman OpenGL With feature sm-raw-window-handle Rust Offers surface management for OpenGL. It is narrower in scope than Glutin, which offers window-management, input handling and more.
Cargo vec.svgash-tray Vulkan no Rust Re-exports ash, imgui and winit as well as providing some helpful modules, macros and functions.

Other Crates[edit | edit source]

ash-molten[edit | edit source]

Cargo vec.svgash-molten lets you statically link MoltenVK. This may not be desirable since ash-molten doesn't have access to validation layers. ash on its own already supports linking to MoltenVK at runtime. Static linking also prevents updates to MoltenVK.

vk-mem[edit | edit source]

Cargo vec.svgvk-mem provides FFI bindings and a Rust wrapper around AMD's Vulkan Memory Allocator (VMA).

glsl-layout[edit | edit source]

Cargo vec.svgglsl-layout provides types which conform to the std140 memory layout for interface blocks. It also provides a couple of traits and a macro for defining custom types which conform to std140 layout rules. Has optional support for conversion to and from Cargo vec.svgcgmath and Cargo vec.svgnalgebra types.

shader-types[edit | edit source]

Cargo vec.svgshader-types provides types which try conforming to the std140 memory layout for interface blocks. Deprecated in favor of Cargo vec.svgglsl-layout due to "fundamental flaws" in how padding was achieved.

Tutorials[edit | edit source]

Rust Tutorials[edit | edit source]

OpenGL[edit | edit source]

WebGPU[edit | edit source]

Non-Rust Tutorials[edit | edit source]

OpenGL[edit | edit source]

  • https://learnopengl.com, a really top-notch tutorial. This tutorial also covers a couple of rendering techniques which are transferable to any graphics API.

Vulkan[edit | edit source]

External links[edit | edit source]

References[edit | edit source]

  1. ALTO: A Personal Computer System Hardware Manual, section 1.0 INTRODUCTION
  2. Hormby, T. (2013, August 14). The Pixar Story: Dick Shoup, Alex Schure, George Lucas, Steve Jobs, and Disney. Low End Mac.
  3. Shoup, R. (2001). SuperPaint: An Early Frame Buffer Graphics System. IEEE Annals of the History of Computing
  4. Altavilla D. (2013, September 30). AMD and DICE To Co-Develop Console Style API For Radeon Graphics. Forbes.
  5. Smith, R. (2014, March 24). Microsoft Announces DirectX 12: Low Level Graphics Programming Comes To DirectX. AnandTech.
  6. Machkovech, S. (2014, June 2). Apple gets heavy with gaming, announces Metal development platform. Ars Technica.
  7. Smith R. (2014, August 11). Khronos Announces Next Generation OpenGL Initiative. AnandTech.
  8. Paras, A. (2015, March 1). “Vulkan” Could be glNext’ Name – GDC Reveal Next Week. WCCFTech.
  9. Smith R. (2015, March 2). AMD Lays Out Future of Mantle: Changing Direction In Face of DX12 and glNext. AnandTech.
  10. Smith, R. (2015, March 3). Next Generation OpenGL Becomes Vulkan: Additional Details Released. AnandTech.
  11. gfx-rs/gfx#861 DX11 Backend
  12. gfx-rs. (2017, July 24). We need to go lower
  13. gfx-rs/gfx#1102 Lower level GPU abstraction
  14. servo/webrender#407 Consider using gfx-rs for rendering