Hardware accelerated rendering

Note: This page currently only handles 3D rendering. It will eventually handle 2D rendering.

Hardware accelerated rendering makes use of specialized hardware, usually a graphics processing unit (GPU), to speed up the process of rendering an image. Many real-time graphics applications make use of hardware acceleration.

A little bit of history

Note: This section is incomplete, and is in need of some fleshing out.

See also: History of computer graphics, History of graphics processing units

Early computer graphics applications like Spacewar! (1962), Sketchpad (1963) and Pong (1972) used vector graphics and rendered them to CRT displays. The computers driving the displays would continuously execute a set of drawing instructions which would steer the beam of the cathode-ray tube.^{[citation needed][is it?]} While vector graphics allow for "infinite" resolution, there are limits to how fast you can steer the beam of a cathode-ray tube and still get a reasonable image.^{[citation needed]} Shapes which need an interior color other than black take an inordinate amount of time to draw compared to just drawing the outline.^{[citation needed]}

Raster scanning had at that point already been in use by television systems for some time, yet this approach to displaying computer-generated imagery remained unfeasible until computer memory which could store an entire image became available. This was realized through semiconductor-based memory, which enabled the development of SuperPaint (1973), a system capable of video editing and computer animation. It could store an entire frame's worth of color information in its framebuffer.

[Mention early fixed-pipeline hardware in consoles, arcade machines and computer expansion cards].

[Mention early re-programmable pipelines in graphics hardware].

[Beginnings of OpenGL].

[DirectX and OpenGL almost kind-of-sort-of merging?].

[Problems with OpenGL]. Fixed Function Pipeline

[OpenGL recieves various extensions which expose more and more lower-level bits of functionality].

AMD announces in September of 2013 that they're working with DICE to develop Mantle, a console-like API which lies closer to the metal. In doing so, AMD eventually sparked a revolution in the graphics API space.

Microsoft announces DirectX 12 at GDC 2014.

Apple announces Metal on June 2, 2014.

The Khronos Group announces that they're working on a replacement for OpenGL later refered to as "glNext".

AMD shifts gears in response to DirectX12 and the "glNext" initiative and donates the Mantle API to the Khronos Group to help kick-start the development of "glNext". During GDC 2015, the Khronos Group announces Vulkan, the result of the "glNext" initiative.

[Maybe also WebGPU].

A Brief Overview of the Various 3D Graphics APIs

OpenGL

The oldest of the bunch. Deprecated by Apple on their platforms in 2018 in favor of Metal.

OpenGL ES

A stripped down version of OpenGL for use on resource-constrained systems.

WebGL

A JavaScript API based on OpenGL ES for use in web browsers.

DirectX < 12

Microsoft's OpenGL. Supported on Windows and Xbox. Uses COM, which can be frustrating to work with in Rust.

WebGPU

The result of the "WebGL Next" initiative. A simplified version of Vulkan intended for use on the web.

In-browser implementations are currently in progress and you can now run examples in both Firefox Nightly and Chrome Canary with their experimental WebGPU feature flags enabled. Note that Chrome Canary does not implement GPU sandboxing at the moment, so sites using the WebGPU API can read the GPU memory of other processes.

Native implementations, wgpu-core 0.17.1 and dawn, also exist.

Metal

Apple's low-level 3D graphics API. Object-oriented. Supported on macOS and iOS. Similar to Vulkan.

DirectX 12

Microsoft's latest and greatest 3D graphics API. Supported on Windows 10 and the Xbox One and later. DirectX 12 Ultimate will make the API more portable between Windows and the Xbox. Like previous versions of DirectX, DirectX 12 uses COM. Similar to Vulkan.

Vulkan

Derived from AMDs Mantle. Exposes a C-based interface. Practically every structure defined by the specification is prefixed with both a VkStructureType sType and const void* pNext field. It helps make the API extensible, but it requires the use of unsafe on the Rust side of things.

Not supported on Apple's platforms, but can be run on top of Metal through projects like gfx-portability and MoltenVK.

General Concepts

While the concepts outlined below should be mostly relevant for all of the modern 3D graphics APIs, the technical terms and explanations will mostly be drawing from how Vulkan does things.

Vertex

An application-defined primitive which is fed into and processed by the graphics pipeline.

A vertex often contains a position, but it does not have to do so.

Fragments

Shaders

Shaders are small programs which run directly on the GPU as a part of the graphics pipeline.

OpenGL expects shaders to be fed into the API as GLSL text.

Vulkan expects shaders to be compiled down to a binary representation called SPIR-V.

Metal expects shaders in "Metal Shading Language"

DirectX expect shaders in HLSL.

Vertex Shader

Runs on a per-vertex basis. This is where you transform, rotate and scale your 3D models in world space. This is also where you perform perspective projection.

Fragment Shader

Runs on a per-fragment basis. This is where the color of the fragments are decided. This is misleadingly called the pixel shader in Direct3D.

Surface

The surface is what the graphics pipeline renders to. This is usually a window.

OpenGL-Specific Concepts

The OpenGL context...

Graphics Rendering In Rust

A Brief history of graphics rendering in Rust

glium

The earliest commit available in Glium's repository is was created on October 3, 2014, predating the announcement of Vulkan. Given the size of the commit, it is safe to assume that work on Glium started some time before the work was commited.

On August 27, 2016, Glium's author posted a "post-mortem" on URLO detailing things which he believes "went wrong" with Glium. The author had already reduced his activity on Glium to mostly merging pull requests and eventually seems to have handed the reins over to someone else.

gfx-rs

The gfx-rs project is perhaps the most notable effort within graphics rendering in Rust.

Initially, the gfx crate could only run on OpenGL, but gained support for DirectX 11 with the release of 0.10 on March 21, 2016. It would later gain support for both Metal and Vulkan with the release of 0.13.

It became apparent as time went on that they needed target a lower level of abstraction to achieve the performance they were after. This led to the birth of CoreLL and later gfx-hal. gfx-hal was eventually adopted by WebRender, a GPU-based 2D rendering engine intended for use in Firefox and Servo.

The gfx repository's main branch is today home to the gfx-hal crate and its various backends.

wgpu

The gfx project eventually gave rise to wgpu 0.17.1 (home) an implementation of the experimental WebGPU API.

Rust Ecosystem

The graphics ecosystem for Rust is divided in 3 main categories:

• Rust wrappers, which make it easier to interact in safe Rust with a given C API.
• Raw FFI bindings, which only output the raw methods a C library will have.
• Multi-API crates, which provide a single interface to use whatever backend they support.

In addition to these main categories, there are a couple of auxiliary categories of crates which you may or may not need depending on what you plan to do.

• Shader compilers. When working with Vulkan or the gfx-hal stack, you're going to need a shader compiler of some sort. While it is possible to pre-compile shaders using external command-line tools, these crates let you do it in your own Rust code.
• Windowing crates. All of the above will let you draw content, but you usually need something to draw to: a window. These crates will allow you to create one or multiple windows for your program. These crates aren't strictly necessary if you're only leveraging the compute-capabilities of some of these APIs or if you're only generating some output file (e.g. a .gif or a PNG).

An important crate to note is raw-window-handle 0.5.2 . This crate provides a single trait: HasRawWindowHandle. This trait makes it easier for graphics libraries and windowing libraries to decouple from one another. For example, instead of hard-coding every window library that might exist, wgpu instead can be used with any windowing library which uses the raw-window-handle crate.

Rust wrappers

Crate	Backend	raw-window-handle	Maintenance Status	Description
glium 0.32.1	OpenGL		Abandoned by original author. Maintained by community	TODO
glow 0.12.3	OpenGL			TODO
ash 0.37.3+1.3.251	Vulkan	Through ash-window 0.12.0	Actively Maintained	A very low-level and lightweight wrapper around the Vulkan API. It is automatically generated from vk.xml. It is used indirectly by gfx-hal through gfx-backend-vulkan.
erupt 0.23.0+213	Vulkan		Actively Maintained	Lightweight and low-level bindings to the Vulkan API, fully automatically generated from the Vulkan Headers, with a focus on extensions, documentation and utils. It was newly released and may be unstable. Additionally erupt has a low userbase (~156 crates.io downloads as of May 2nd 2020).
vulkano 0.33.0	Vulkan			A Rust wrapper for Vulkan that tries to be convenient to use and prevent invalid usage of the API. Vulkano is by no means complete, but its author believes that the library won't undergo any major breaking changes at this point.

Raw FFI bindings

Crate	API	Description
gl 0.14.0	OpenGL
metal 0.26.0	Metal
vk-sys 0.7.0	Vulkan	Used by Vulkano.

Multi-API crates

Crate	WebGPU	Vulkan	Direct3D	Metal	OpenGL	raw-window-handle	Maintenance Status	Description
luminance 0.47.0							Actively Maintained	A high level graphics api that uses closures to control pipline state. Built for opengl 3.3 only at present, but further backends are planned.
gfx-hal 0.9.0	Planned		11/12		In the works		Actively maintained	A fairly low-level library which tries to not abstract away the details of the underlying APIs too much. It's developed around the idea of the Backend trait, which lets it run on top of various native 3D graphics APIs. All major types in the gfx-hal library have a generic parameter with Backend as a trait bound. Implementations of the `Backend` trait are provided by the various gfx-backend-* crates.
rendy 0.5.1	Planned		11/12		In the works		Stalled	A library created in combination with the Amethyst game engine. The library is not tied to the game engine and can as such be used by anyone. It is based on gfx-hal and seems to aim to offer a safer API without sacrificing a great deal of features and performance. Much like gfx-hal, rendy makes its types generic over gfx-hals Backend trait.
wgpu 0.17.1 (home)			11/12		In the works		Actively maintained	A Rust wrapper around wgpu-core 0.17.1 when compiled for native platforms. When compiled for the web, it uses wasm-bindgen and web-sys to interact with the WegGPU API. It's shares a lot of similarities with Vulkan in how the API is structured, but it's a lot simpler and exposes no unsafe functionality. Most people looking for something performant and easy to use should pick wgpu.
gfx 0.18.3 (home)			11				Deprecated	gfx of the pre-ll variety provides a higher-level API which is more suitable for the average user. The spiritual successor to gfx-pre-ll is wgpu-rs.

Shader Compilers

You'll probably need one of these if you're going to use a library which wants to recieve shaders as SPIR-V.

Crate	GLSL	HLSL	MSL	Underlying Library	Maintenance Status	Description
shaderc 0.8.2 (home)				C/C++	Actively maintained	Rust wrapper around shaderc, Google's collection of tools for Vulkan shader compilation. Takes a while to compile if you don't have a pre-compiled version.
glsl-to-spirv 0.1.7				Pure Rust	Deprecated	While the library is written in Rust, all it does is shell out to glslangValidator. Not to be confused with the other glsl-to-spirv.
naga 0.13.0 (home)	🚧			Pure Rust	Experimental	Describes itself as a shader translation library.

Shader reflection

It is sometimes useful to determine the type, number and position of the inputs and outputs of a shader at runtime. The OpenGL specification includes functions for this purpose, but such functionality is intentionally left out of the Vulkan specification to lessen the work hardware vendors have to do. The following libraries aim to offer the functionality you're "missing out on" by using Vulkan.

Crate	Implementation Language	Description
rspirv-reflect 0.8.0	Rust
spirv-reflect 0.2.3	C++	Wrapper around KhronosGroup/SPIRV-Reflect.
spirq 0.6.4	Rust

Shader embedding

These crates offer macros which compile shaders defined in other files and then embed SPIR-V inside your executable (or library) in a manner similar to the include_str and include_bytes macros in the standard library.

Crate	Notes
include-glsl 0.2.0	Poorly documented. Infers shader stage from file extension.
inline-spirv 0.1.6	Also allows inline shader source code. Lets you specify the shader stage, entry point, shader language and additional #DEFINE and #include directives.

Surface creation

Displaying the result of your rendering code on a window requires some level of interaction with your windowing library. This could either manifest as the windowing library providing such directly, or as the windowing library offering raw-window-handle 0.5.2 support, which offloads the work of creating the "surface" to another library.

Crate	Made For	raw-window-handle	Implemented in	Description
glutin 0.30.10	OpenGL		Rust	Offers surface management for OpenGL as well as re-exporting all of winit. This has been a cause of confusion for some and have caused them to report issues with winit on Glutin's issue tracker.
surfman 0.8.0	OpenGL	With feature sm-raw-window-handle	Rust	Offers surface management for OpenGL. It is narrower in scope than Glutin, which offers window-management, input handling and more.
ash-tray 0.19.0	Vulkan		Rust	Re-exports ash, imgui and winit as well as providing some helpful modules, macros and functions.

Other Crates

ash-molten

ash-molten 0.15.0+1.2.2 lets you statically link MoltenVK. This may not be desirable since ash-molten doesn't have access to validation layers. ash on its own already supports linking to MoltenVK at runtime. Static linking also prevents updates to MoltenVK.

vk-mem

vk-mem 0.2.2 provides FFI bindings and a Rust wrapper around AMD's Vulkan Memory Allocator (VMA).

glsl-layout

glsl-layout 0.5.0 provides types which conform to the std140 memory layout for interface blocks. It also provides a couple of traits and a macro for defining custom types which conform to std140 layout rules. Has optional support for conversion to and from cgmath 0.18.0 and nalgebra 0.32.3 types.

shader-types

shader-types 0.2.2 provides types which try conforming to the std140 memory layout for interface blocks. Deprecated in favor of glsl-layout 0.5.0 due to "fundamental flaws" in how padding was achieved.

Tutorials

Rust Tutorials

OpenGL

• Glium's Tutorial
• bwasty/learn-opengl-rs, a Rust port of https://learnopengl.com
• rust-tutorials/learn-opengl, based on https://learnopengl.com
• unknownue/vulkan-tutorial-rust, a Rust port of https://vulkan-tutorial.com/
• Rust and OpenGL from scratch. The link to the next tutorial is always somewhere at the bottom of the blog post.

WebGPU

• sotrh/learn-wgpu

Non-Rust Tutorials

OpenGL

• https://learnopengl.com, a really top-notch tutorial. This tutorial also covers a couple of rendering techniques which are transferable to any graphics API.

Vulkan

• https://vulkan-tutorial.com/
• https://vkguide.dev/

External links

• The wgpu-rs website
• Gfx-rs nuts and bolts
• Glium post-mortem
• AGuideToRustGraphicsLibraries2019
• 3D Rendering | Are we game yet?
• Dolphin Emulator and OpenGL drivers - Hall of Fame/Shame
• The Metal Documentation
• The Direct3D 12 Documentation
• Vulkan^® 1.2.139 - A Specification (with all registered Vulkan extensions)