Super Facinating! (Destiny)

by Ragashingo , Official DBO Cryptarch, Thursday, January 05, 2017, 03:31 (2879 days ago) @ Cody Miller
edited by Ragashingo, Thursday, January 05, 2017, 03:44

Here's some notes I sorta wrote while watching. They explain the problems in a lot more detail and its very much worth watching if you have a bit of programming knowledge. My version below is greatly simiplified and probably a bit wrong but it gets the gist enought right... I hope...

In Halo, all the different assets (sounds, shaders, models, the concept of an enemy AI unit, etc) were all fairly separate. This meant you could swap out the sound an exploding box made with a different sound and quickly rebuild the game and test your result. Maybe that new sound file had to be compressed down into the right console specific format, but no big deal. Same with models, same with everything.

One of the problems with Halo though was you couldn't really do fast small patches. They basically had to ship the entire game as one big unit and the only time we really ever saw major patches was with the big DLC releases where the player was already expecting to download a significant amount of data (or load it off the multiplayer DLC disk they bought.)

For Destiny, Bungie intended to update things much more frequently than with Halo. It no longer made sense to make the player redownload the entire game just to add a mission or whatever. I'm hitting the limit of my understand a bit, but they at the same time were also trying to make it easier to develop Destiny and make rapid changes.

Through some bad assumptions and trying to be too clever they shot themselves in the foot... with a nuke. They didn't go super in depth, but it sounds like a part of this was caused by having to support four very different platforms all at the same time. For instance, the PS3's Cell processor had a main CPU and the smaller simpler high performance SPU sub processors that individually didn't have a lot of memory or resources. So the systems that translated Destiny from artwork and sounds and models into PS3 code had to be very clever about chopping things up so the pieces could each fit into a single SPU.

In Destiny development, unlike in Halo's, it ended up that objects became all linked and tangled together. The example they gave was a box might glow brighter as it took damage. So the box might reference a glow shader and might reference the sound it made when you punched it and so on those sounds and shaders had to know the box's hp value.

In Halo development you could swap out the box or the shader or the sound and since none of those pieces knew anything about the other it all kinda worked.

In Destiny making a change to any of those pieces meant the system that took all the assets and translated them into the proper format to run on the different consoles not only had to recompress the sound file, it had to load up the box's model, and shaders and and everything else that made the box work.

Worse, because of the way the levels are optimized as a whole to save space and memory, changing the sound a box makes when punched might mean that the tool that translated everything to the way it needed to be for the consoles would have to load in and rebuild an entire level area.

So the smallest of changes, like swapping out a sound, would mean the audio guy might need to wait two hours while an entire area like the Mothyards and every object and shader and sound and unit was loaded in from disk and reoptimized back to code the consoles could run.

To make matters worse, some of the systems and tools that were supposed to let them make changes without having to load in everything simply weren't ready in time.

Bungie has since rebuilt their systems significantly to improve build times. They basically found a way to split up the way objects and assets reference each other. Everything still has to reference everything in the end because that's how the game engine works, but their new system lets them isolate each piece so making a little change doesn't kick off a huge overnight rebuild.

So it sounds like swapping out that sound file only triggers a much much smaller rebuild. The tool that builds the game can now often dig down and just mess around with the sound file or the dimensions of a crate instead of having to load in and rebuild an entire level from scratch.

There were other big problems too. The tool that did the job of building the game... it would run out of memory part way through and die because the process it had to do, following the references from object to object in an ever expanding tree of complexity, got so big that the application was literally running out of memory space in Windows.

Ultimately, it sounds like a lot of the big problems have been fixed, or at least significantly worked around. They didn't say exactly, but it would seem that merely swapping out a sound or graphic will no longer trigger an multi-hour or overnight rebuild. What took five hours might now be accomplished in minutes.

We've all been wondering what happened to Destiny. Why it didn't live up to what we hoped it would be. It sound like this major development tool blunder sucked in a lot of programming talent to fix and was a extremely major issue for all of Destiny's development slowing down the workflow of everybody from the mission planners to the individual artists and sound designers. I can't even imagine how they managed to ship such a polished game as they did with these massive problems clogging their development pipeline.

Hearing that things have been made better gives me Hope for the Future that Destiny 2 will be much easier for Bungie to develop and they'll be able to get a lot more work done without super long rebuild times popping up for even the smallest change.