GDC'15: Multithreading the Entire Destiny Engine (Vid 59:34) (Destiny)

by Ragashingo , Official DBO Cryptarch, Tuesday, January 24, 2017, 04:01 (2643 days ago) @ Pyromancy

Ok, again, very cool stuff!

This video has a lot about how Bungie split up the process of drawing all the stuff in each frame in order to make the most of the CPU and GPU resources each console had.

A few notes:

• For Reach, the 6 virtual CPUs (2 per real, physical CPU) of the Xbox 360 tended to not be utilized super well. One virtual CPU would constantly be busy with simulating the next frame's AI and physics and unit movement logic while the next virtual CPU would be mostly doing very little. Then you'd have one doing most of the work lining everything up for the GPU to do its graphical magic while virtual CPU 4 might be busy only half the time because all it was doing was processing sounds.
  
• This approach would not work for Destiny for a number of reasons. Destiny does a lot more (like matchmaking at all times). Destiny's worlds and graphics are a lot more complex. Etc. And, of course, the PS3 was even less powerful than the 360 that Reach ran on. And its power is split up differently (just two main CPUs and six really fast but really tiny SPUs that don't do a lot but do specific small tasks really quickly).
  
• Balancing the load of all of Destiny's systems across all the CPUs and GPUs of each console was absolutely essential. The PS3 and even the 360 would not be able to run Destiny at 30fps if it had been built in the same fashion as Reach was. Just looking at this one chart early on that shows roughly half the 360's CPU power being underutilized I'd guess that Destiny would not have run at 10fps if Bungie hadn't put in all the work it did to spilt up the total workload across all the CPU and GPU cores.
  
• They showed a graph later on where the PS3's two main CPUs and six SPUs were basically busy at all times during each game tick of Destiny. There were little gaps all over the place as one task ended and another began but there were no significant pockets of time (and remember, time here is measured in units at least as small as milliseconds if not smaller) that any given resource of the PS3 wasn't busy.
  
• One neat thing Bungie did was to spit up the game state from the information needed to draw each new frame. In Reach, they had to basically duplicate the state of the world (which includes everything like where you and all the other characters were, what each NPC's AI was doing, what the physics system was doing to each object and character, etc) and use one copy to render the current upcoming frame while another copy was used to start simulating how everything moved and changed in preparation to draw the next frame. But this meant that a lot of data that the GPU and the graphics system didn't care about (like what target an Elite was focused on) came a long for the ride.
  
• In Destiny, they split things up so that the graphics rendering system only had access to and only needed access to the information necessary to draw the world. So, it needs which direction you are looking and which enemies are in your field of view, but the graphic system does not have to worry about if a Vandal's AI is currently telling it to duck because a grenade landed nearby. By making this split they reduced the amount of information they had to duplicate from roughly 10MB to 1MB. That might not seem like much, but when every CPU cycle and every bit of memory is important, a 10x reduction in the memory use of a costly part of the rendering system is a very big deal.

Once again, like the other video in this thread, while it seems like Bungie got too clever with their in house development tools and that might have stolen years of development time away from Destiny, it appears that they made some big, deep architectural improvements to the way Destiny's engine worked that were very impressive and that allowed it to perform extremely well even on the last gen consoles.