What we have not really concerned ourselves with up to this point, is memory management. Memory management is fairly important to get right, as the programmer is responsible for respecting the maximum bound of available memory on the hardware.
Available memory on the hardware can be obtained using vkGetPhysicalDeviceMemoryProperties on Vulkan and IDXGIAdapter3::QueryVideoMemoryInfo on Direct3D 12. Any allocations which go outside of bounds of remaining memory will fail, so memory use will need to be kept in mind at all times to prevent critical components being unable to load.
One method in which memory can be cleared for use is through Direct3D's Evict function. This removes the resource data from GPU memory, storing it either in system memory or on disk instead. This is ideal for data that is currently not required on the GPU, but may need to be returned in memory at some point. Returning the memory is done through MakeResident.
Another method is deallocating the memory, but it is important to note memory should not be deallocated until the last command buffer that referenced the resource on the GPU is no longer being executed. GPU behaviour when dealing with deallocated memory is undefined.
Alternatively, other allocation schemes can be created in buffers or other memory objects. The ID3D12Heap and VkDeviceMemory objects allow for memory to be subdivided for resources that need to have memory for multiple instances of itself around, for instance for data that is updated once per frame.
The difference between the type of memory used in order to transfer data from and to the GPU should also be considered. UPLOAD / HOST_VISIBLE memory may be convenient, but can potentially carry associated costs of multiple transfers. For more info, refer to the GPUOpen article on Streaming Memory Management.