Repository: baldurk/renderdoc

Issue #1933: Cannot replay Unreal Engine DX12 capture
Created: 2020-06-08T12:14:20Z
Description
Created a new empty project in Unreal Engine. Set API to DX12, and captured a frame. I couldn't open capture because an error happened.
Core     PID  17776: [13:04:06] d3d12_device_wrap.cpp(1525) - Error   - Failed on resource serialise-creation, HRESULT: E_INVALIDARG
Steps to reproduce

Open Unreal Engine Editor 4.25
Click File - New Project...
Create a new blank project. (Blueprint, Maximum Quality, Raytracing Disabled, Desktop / Console)
Click Edit - Plugins
Search for RenderDoc Plugin, and enable it.
Click Edit - Project Settings...
Under Platforms - Windows set Default RHI to DirectX12
Click on RenderDoc icon at the top right corner. RenderDoc opens with a capture.
Double click on the capture and a pop up window will appear with the error message.

Environment

RenderDoc version: 1.8 (built from 8b491e1)
Operating System: Windows 10 Pro, version 1909
Graphics API: DirectX 12
Graphics card: GeForce GTX 1080
Graphics driver version: 441.66

Duplicate issue #1984: Failed to replay D3D12 captures
Created: 2020-07-20T08:49:10Z
Description
Failed to open d3d12 captures due to the creation of some heap was failed.
Tried to debug the issue,  here is the log:

RDOC 007216: [15:51:36]          rdcfile.cpp( 244) - Log     - Opening RDCFile F:\SponzaTest\Saved\RenderDocCaptures\2020.07.20-15.11.55_frame6046.rdc
RDOC 007216: [15:51:36]          rdcfile.cpp( 307) - Debug   - Opened capture file for read
RDOC 007216: [15:51:36]          rdcfile.cpp( 244) - Log     - Opening RDCFile F:\SponzaTest\Saved\RenderDocCaptures\2020.07.20-15.11.55_frame6046.rdc
RDOC 007216: [15:51:36]          rdcfile.cpp( 307) - Debug   - Opened capture file for read
RDOC 007216: [15:51:36]             core.cpp(  49) - Log     - Not enabling API validation during replay
RDOC 007216: [15:51:36]             core.cpp(  54) - Log     - Using default GPU replay selection algorithm
RDOC 007216: [15:51:36]             core.cpp(  63) - Log     - Replay optimisation level: Balanced
RDOC 007216: [15:51:36]     d3d12_replay.cpp(3739) - Debug   - Creating a D3D12 replay device
RDOC 007216: [15:51:36]     d3d12_replay.cpp(3853) - Log     - Capture was created on nVidia / NVIDIA GeForce GTX 1060 6GB
RDOC 007216: [15:51:36]          amd_rgp.cpp(  76) - Log     - AMD RGP Interop is not enabled
RDOC 007216: [15:51:36]      dxgi_common.cpp(1926) - Log     - Selected nVidia / NVIDIA GeForce GTX 1060 6GB adapter for replay
RDOC 007216: [15:51:36]     d3d12_replay.cpp(3892) - Log     - Creating D3D12 replay device, minimum feature level D3D_FEATURE_LEVEL_12_1
RDOC 007216: [15:51:36]     d3d12_common.cpp( 127) - Debug   - Enabling debug layer
RDOC 007216: [15:51:36]     d3d12_replay.cpp(3941) - Log     - Created device.
RDOC 007216: [15:51:36]     d3d12_replay.cpp( 101) - Log     - Running replay on nVidia / NVIDIA GeForce GTX 1060 6GB 27.21.14.5167
RDOC 007216: [15:51:36] replay_controller.cpp(1974) - Log     - Created replay driver.
RDOC 007216: [15:51:37]     d3d12_replay.cpp( 134) - Log     - nVidia GPU detected - no counters available
RDOC 007216: [15:51:37] d3d12_device_wrap.cpp(1325) - Log     - Remapping committed resource ResourceId::54 from upload to default for efficient replay
D3D12 ERROR: ID3D12Device::CreateHeap: D3D12_HEAP_FLAGS has invalid flag combinations set. The following flags may not all be set simultaneously. Exactly one must be left unset, or all may be left unset when the adapter supports D3D12_RESOURCE_HEAP_TIER_2 or creating a heap in conjunction with D3D12_HEAP_FLAG_SHARED_CROSS_ADAPTER: D3D12_FEATURE_DATA_D3D12_OPTIONS::ResourceHeapTier = D3D12_RESOURCE_HEAP_TIER_1, D3D12_HEAP_FLAG_SHARED_CROSS_ADAPTER = 0, D3D12_HEAP_FLAG_DENY_NON_RT_DS_TEXTURES = 0, D3D12_HEAP_FLAG_DENY_RT_DS_TEXTURES = 1, and D3D12_HEAP_FLAG_DENY_BUFFERS = 0. [ STATE_CREATION ERROR #631: CREATEHEAP_INVALIDMISCFLAGS]
RDOC 007216: [15:51:37] d3d12_device_wrap.cpp(1525) - Error   - Failed on resource serialise-creation, HRESULT: E_INVALIDARG

According to D3D12 Spec,  at least two of these three flags must be set if your gpu only supports  "heap tier 1 ":

D3D12_HEAP_FLAG_DENY_BUFFERS
D3D12_HEAP_FLAG_DENY_NON_RT_DS_TEXTURES
D3D12_HEAP_FLAG_DENY_RT_DS_TEXTURES

Apparently here only 1 of them was set. This should be the root cause.
Steps to reproduce
Capture on D3D12 and open it on any GPU only supports heap tier 1 should reproduce it,  for NVIDIA it's Kepler to Pascal
Environment

RenderDoc version: 1.8
Operating System: windows 10
Graphics API: D3D12

Closing comment (2020-07-20T09:49:56Z): I believe this is the same as #1933 caused by the D3D12 runtime returning a nonsense and impossible heap descriptor for heaps created via OpenExistingHeapFromAddress. A workaround for that has been added since v1.8 was released, so capturing on a recent nightly build should work.