"We're able to show the difference from what's possible today with what's possible tomorrow": Inside Epic's Unreal Engine 5

Epic has always been in the business of seeing the future. Nowadays, however, the situation is a little more complicated than that. "It's kind of a scam," smiles CEO and founder Tim Sweeney, "because while we're predicting the future, we're actually secretly building it in the background so that our predictions are right."

It's a comment that succinctly describes just how influential a force in the videogame industry Epic has become, especially over the last few years. Epic has always been a big player, particularly in terms of tech: the cover of E251 featured Unreal Engine 4, whose graphical capabilities and accessible tools – not to mention its competitive pricing scheme – quickly made it one of the cornerstones of the creation of a new generation of games. Now, its maker has positioned itself at the forefront of the next one. 

Still, back in 2012, Epic was really going through it. In the very same year Unreal Engine 4 was announced, the company revealed that it was to be part-acquired by Chinese multinational conglomerate Tencent. Epic had revealed a trailer for a small side-project called Fortnite the previous year, and the team was starting to realise that the planned series of updates to evolve the game might be best suited to a free-to-play model – and that it was this model that was the future of success in the video game industry. 

If anyone knew how to make that kind of thing work, it was Tencent, the world's number one operator of live games. Epic had its partner, and a vision. But not everybody saw things quite the same way. Soon after the Tencent acquisition, many of Epic's most high-profile staff walked, many unwilling to be part of this new direction for the company and its games. Over the next few years, Unreal Engine 4 went from strength to strength, its reach greatly expanded by going free for all to use in 2016 – but Epic's game projects seemed increasingly destined to melt into obscurity. And then, the prophecy suddenly came true

 Start your engines 

Epic, Sweeney has told us before, has always moved on instinct, taking advantage of opportunities as they come. 2017's battle royale craze was one such opportunity: pivoting survival-slash-building game Fortnite to the hot new genre almost immediately proved itself one of the best business decisions in the history of video games. And from those billions of dollars sprang everything else that Sweeney had been hoping to achieve – a vastly improved, better-optimised Unreal Engine 4, a platform that focused on fairer deals for devs in the form of the Epic Game Store, and some truly forward-thinking moments in gaming, most recently Travis Scott's electrifying live concert and the world premiere of the trailer for forthcoming Christopher Nolan film Tenet in the metaverse that is Fortnite. 

Game maker, engine builder, store proprietor, industry investor: as Sweeney points out, these days, Epic is operating across so many sectors that it looks set to actively shape the future. Enter Unreal Engine 5 – via a tech demo that we're told is running on PS5, and that gives us a long look at easily the most detailed and dramatically well-lit environments we've ever seen grace the interactive format. After a rather surface-level Xbox Series X showcase from Microsoft that left many scratching their heads about what next-gen really meant for games, here it finally was: our first proper look at what games coming to both PS5 and Xbox Series X could look like. Emphasis on the "could". 

Tech demos, as ever, are pure potentiality: the result of what you get out of an engine is largely down to what you're able to put in. But the guiding idea behind Unreal Engine 5 is that developers will need to use less resources, manpower and time than ever to create top-of-the-line next-gen visuals. The engine has been designed to produce the maximum amount of quality for the minimum amount of cost, both financial and otherwise. Nanite, Epic's new micropolygon geometry tool that aims to make polygon budgets and manual LOD (level of detail) asset creation a thing of the past, is a key part of this. It's an idea that's been rattling around heads at Epic for the best part of a decade – chiefly, the head of Brian Karis, technical director of graphics. In around 2008, he came across an interview in which John Carmack was airing some thoughts he'd had about virtualised geometry with voxel ray casting. 

"It was kind of like a spark in the imagination," Karis recalls. "I had had a bunch of really good experiences with virtual texturing – I'd written my own virtual texturing system at a past company before coming to Epic. I'd seen how transformative that was for the team to not have to worry about texture budgets. I thought it would be great if we could apply the same sort of concept and approach – not the implementation exactly – to let the artists be free and not have to deal with budgets, not just for textures, but for geometry, for anything else."

Over several years, Karis would mull over solutions to this problem in his free time – there were other priorities at work, and so his research remained confined to his personal blog. Until, about three years ago now, he hit upon an idea for a specific implementation that he wanted to try out on a full-time basis, and decided to take it to his peers. 

"Brian had suggested 'Hey, I had this idea about how to do environments at a detail level that you'd only ever see in movies,'" chief technical officer Kim Libreri says. "We felt that the current paradigm for rendering real-time computer graphics, and the traditional way of rendering pixels, was going to be a dead end. You'd need, like, three or four console generation clicks to be able to achieve that level of realism. We had a good gut feeling of how the hardware was going to evolve, and that it would enable this new type of technique. So we're like, 'Okay, Brian – start experimenting.'" 

It was right around this time in 2017 that Fortnite was starting to blow up: Karis was effectively "sheltered", he says, from what was happening throughout the rest of Epic, able to work on his idea without being dragged off to help babysit the phenomenon the company suddenly found itself cradling. "We got a little bit nervous, because it was quite a lot of experimenting," Libreri laughs. "But eventually, things started to click with Nanite, and we were like, 'Oh my god, this is really going to work. This is going to change everything for game developers.'" 

Sony, you see, had listened. PS5 was going to make Nanite possible. "It was three or four years ago at least when we started to talk with Mark Cerny about possibilities for the next generation," Sweeney says. Their discussion wasn't just about graphics, but about the growing realisation that storage architecture in game hardware – having to load data from a hard drive, the huge amounts of latency between mass storage and a processor – was a limiting factor in Epic's and all developers' future plans for game-making. The team at Epic received very early hardware access to the next-gen console, and the Sony collaboration has been far longer-running than the Microsoft one, Sweeney says, something which naturally influenced Epic's decision to reveal Unreal Engine 5 using PS5 instead of Xbox Series X. 

"And Sony really did a fantastic job of implementing a new platform around that realisation that storage could be revolutionised," he continues. "PlayStation 5 is built not only on a huge body of flash memory, but also a very high bandwidth and low latency framework for accessing it, and for getting it to wherever you need for any type of work." 

He describes how PS5 renders a texture highly efficiently, fetching it from the high-speed SSD decompressed, into video memory in the exact place it's needed. This is thanks to PS5's IO (or input-output) system, which according to Epic's VP of engineering Nick Penwarden, is "the major innovation with the next-generation console hardware. They have faster CPUs, they have faster GPUs, and that was really important to be able to achieve the visuals that we showed – but the biggest change across console generations is absolutely going to be the IO bandwidth that we're able to achieve with the SSDs that are in next-generation consoles." 

"It's a key unblocker for what Brian and team have built here," Sweeney confirms. "Rendering micropolygons resulting from a 20 billion-polygon scene is hard enough. But actually being able to get that data into memory is a critical challenge. And as a result of the years of discussions and efforts leading up to that, it was a perfect opportunity to partner [with Sony] to show that effort finally coming to fruition with pixels on the screen." 

Pixels – and triangles. The buzzword of that tech demo is key to understanding how Nanite works, why PS5 has made it possible, and why it's such a huge leap forward. Import a film-quality asset such as a photogrammetry scan or ZBrush sculpt made up of millions of polygons – say, the statue in the tech demo – into Nanite, and the engine will automatically render it on a scalable triangle-per-pixel basis. The system memory preserves the infinite amount of detail from the model, but Nanite uses the speed of PS5's SSD to near-instantly stream in on demand only the data – and the triangles, be they millions or even billions – needed to display what the camera (and your eyes) are capable of seeing. 

"It's an analogue to a computer science sort of approach of virtual memory," Karis says. "The concept is that not all of the data that you want to access needs to actually be in RAM, it can be in a much larger space where some of it is off on disk. And only when you access things does it need to bring it into memory and have that stuff be actually resident."

In other words, Unreal Engine 5 should eliminate the need for LODs (or multiple 'level of detail' versions of assets, 3D models manually made to varying degrees of complexity depending on how close the player is to the asset). Not only should it ensure that assets are always seen at maximum possible fidelity in every situation, it could save countless hours of developers' time. The tech goes far beyond backface culling (which detects which polygons are facing away from a viewer, and doesn't draw them, saving on processing power). 

"It's in the form of textures," Karis explains. "It's actually like, what are the texels of that texture that are actually landing on pixels in your view? So it's in the frustum. It's stuff that you're actually seeing that's front-facing, it's stuff that's not occluded by something else. It's a very accurate algorithm, because when you're asking for it, it's requesting it. But because it's in that sort of paradigm, that means that as soon as you request it, we need to get that data in very quickly – there needs to be a very low latency between the ask and the receive or otherwise, you're going to see really bad pop-in." 

Libreri chimes in. "And that's the key. The general technique's not that different from the way movies have been rendered over the years: you only pull the data that you need to generate the frame. The problem with that is that you need to be able to get that stuff very quickly, and be able to dump it very quickly when you don't need it." 

There's one crucial difference in the next generation of consoles that has helped unlock Epic's approach with Nanite: "The previous generation of consoles required you to support spinning hard disks, which means you have variable latency time, and it's not totally predictable how long it's going to take to get a piece of data. With the flash storage, you have way, way more predictability and much, much lower latency to be able to get those reads into memory."

The console manufacturers, Libreri says, were surprisingly open with listening to Epic's requests for flash storage. "Having SSDs in a console does add to the cost, so it's a big decision for them to make. But it definitely makes a massive difference to the ability to, you know, stream an open world of super-complicated geometry into a game so that the user doesn't even notice." 

Lumen, Epic's new fully dynamic global illumination system, plays its own role in helping Unreal Engine 5 developers to create naturalistic and believable visuals with far less resources than usually required. Lumen automatically calculates changes in environments on the fly – whether that be a torch casting shadows across the walls of a cavern, or a ceiling caving in to reveal bright light streaming down from above – thereby eliminating the need for developers to bake individual lightmaps for every environmental change in their game. And then there's the Quixel Megascans library. Epic's acquisition of photogrammetry startup Quixel in 2019 means Unreal Engine 5 developers have free unlimited access to this vast array of photorealistic assets.

"Developers can just drag them right into the engine and start using them," Penwarden says, "and you very quickly get to a point where you have a photoreal scene running in realtime in Unreal. You're not having to have every developer in the world model yet another rock, yet another cliff face, yet another tree."

All of it can be pulled directly from the Quixel Megascans library and used instantly at the highest fidelity possible, giving devs the option of using the hours they might spend crafting a hyper-detailed boulder on a more bespoke, challenging and potentially innovative portion of their game. It all sounds faintly miraculous. Knowing game devs as we do, we are sure that the promise of the sky now being the limit in terms of polycount will encourage some to push data sizes to eye-watering limits; the fact that we have to keep uninstalling other games every time the 200GB-gobbling Call Of Duty: Modern Warfare releases a new update is a constant reminder that we've come face-to-face with the practical limits of this generation. 

"We'll have state-of-the-art compression algorithms," Libreri assures us, explaining that Epic's started working on those already and plans to do a lot more over the coming year. "We'll also start enabling developers to have higher-level descriptions of geometry that aren't just the micropolygon level detail, where it's a displacement – like the way that movie assets are made." 

"But also, I think we can take a wider view of what games could be in the future," Sweeney adds. "Now we have this idea of installing a game, which means downloading every byte of the game from the Internet and putting it on your machine. You know, those boundaries can change over time. Perhaps only part of the game can actually be on your particular machine at any one time." 

He points to the nascent rise of cloud gaming, where the idea is that the entire game lives in the server, thereby invalidating concerns about data size and footprint. "We're just going to have to watch the way the games evolve over time, and they may be more flexible in the future than they are now. Every previous console generation was limited by the expectation that some people don't have Internet, and have to be able to play the game off of a physical disc, right? If that expectation fades away, I think you have entirely new possibilities." 

Future proof

Fortnite's mainstream success in spite of being entirely dependent on a decent Internet connection has only confirmed for Sweeney that this will be the future of video games. "I think that more and more games will go in that direction in the future, and that even games that we look at as singleplayer might in the future have so much content and variety that parts of the experience are streamed in dynamically in the background as you're playing, and you won't even notice." 

Unreal Engine 5, then, could be the start of a paradigm shift for the videogame industry as we know it. Epic is certainly hoping it will be, and is laying the groundwork well in advance – it's recently released for free the multiplatform SDK it's been honing through the last several years of running Fortnite, and has updated its terms to waive the first million dollars of revenue made by games created in Unreal Engine. Yet more encouraging signs, then, that while Epic's influence on the industry grows exponentially, Sweeney and co are prepared to wield it responsibly. It's also future-proofing at its finest. Indeed, Unreal Engine 5 not slated to arrive until late 2021 (although version 4.25, which is out now, is already compatible with PS5) we can't help but study the tech demo and wonder aloud: if next-generation games won't look like this until Unreal Engine 5 arrives in 2021, what are they going to look like in the meantime? There is a long pause.

"Well, you've seen all the stuff that we've done with the engine over the last few years in Unreal Engine 4," Libreri says. "You can make awesome- looking titles. And I think that if you look at gen-four console games today, compared to what you can do on a high-end PC with Unreal Engine 4, there's still a decent gap. I don't think consumers are going to be disappointed with playing games or anything on Unreal Engine 4, I think they're going to look absolutely incredible. What we've done is just taken an extra leap with the Nanite and Lumen technology that takes stuff to movie-quality levels." 

With this tech demo, we're witnessing something far beyond what is likely to appear on the new consoles this winter, although what's inside the machines is more than capable of making it happen, particularly in the dev- friendly circumstances Unreal Engine 5 is designed to nurture. "I honestly think that what we've done is made it so that when Unreal Engine 5 is usable, especially for detail – if that's important to your game, detail and photorealism, I think it sort of makes it look like another console generation click ahead," Libreri says. "But I don't think there's going to be, like, this desert of not-great- looking stuff in the first phase of the console." 

As Penwarden puts it: "What we were able to show in the demo, in terms of visual fidelity as well as the size of game worlds that will be built with UE5, is going to represent the difference from what's possible today with what's possible tomorrow." 

"From a developer engineer point of view," Karis says, "the question is, 'When are you going to really see a hardware fully exploited?' You don't really see what something is truly capable of until more towards the end of the generation. Are you going to see the PS5 fully tapped out on the very first thing that people have tried to use it for? You're never going to see that. I don't think you won't see any difference – you'll see some very cool stuff from UE4 on next-gen [consoles] from other studios. And then with Unreal Engine 5, there's a bit of a step function there: once people get their hands on that, instead of a gradual rise, you'll see a big jump."

Epic is working to make that jump less intimidating to creators. It will allow projects to transition from UE4 to UE5, Penwarden tells us, so that if you're not planning to ship your game in 2021, you can begin development in UE4 and then continue in UE5 to leverage all its additional features. 

"We're hoping to make the transition as smooth as possible," he says, pointing to the fact that many of Epic's new tools, such as its VFX system, Niagara, are now out of beta and ready to be used in full production in UE4 so that developers can figure out how they might best augment their next-gen games. The idea is that moving your project from UE4 to UE5 will take about the same time and effort as a couple of version upgrades in UE4 – 4.20 to 4.24, perhaps. "We're working on making any code API changes that we made a manageable change, so you won't have to completely rewrite your game code or game logic. Most of that is just going to work with UE5's APIs as well." 

As they move to UE5, he is hoping that developers "spend an awful lot less time optimising their game environments around a lot of these technical limitations. And I almost want to say, that they don't realise it – like, they're just building their games and don't realise that they don't have to go back and start merging meshes, and creating LODs and hand-authoring all this stuff." Talk about asking devs to break the habit of a lifetime. Penwarden smiles: "That's right. Developers who are used to sticking within budgets and are thinking, 'No, I need to work this way to make sure it runs in realtime' – not having to do that is probably going to take a little bit of time for them to get used to."

Indeed, the same goes for us: everyone, from developers to players, should prepare for how this kind of next-gen technology – Unreal Engine 5, PS5, Xbox Series X – will change the face of the industry. "As we get into these new generations," Libreri says, "I think there's potential to have new types of gameplay, new types of interactions beyond just talking about what the environment looks like with the Nanite technology. I think that games will evolve. It wasn't until server power in the cloud was powerful enough to host a 100-player game that you could have a battle royale game. So I do think you'll find developers using that power for all sorts of interesting things you just couldn't do before. And it's not just about graphics. There is still a fair way of separation between what we experience in the real world and what you can experience in a modern game. And I think we're going to get a lot closer with this generation, because of the added power across the board – not just GPU, not just SSD, CPUs, the whole works." 

And with Epic's tech now such a critical underpinning of the next generation in games reaching its full potential, we should be prepared to see the company continue to challenge our industry, too. There are those who will naturally have major reservations about the idea of one company having such a broad hold over so many parts of the industry: the money with which games are funded, the engines on which they're made, the stores on which they're sold. But the more we talk to Sweeney about his vision for an industry in which the walls come down and the competitors push each other to be better, the more we start to feel as though it could be in far worse hands than Epic's. 

"We've always been a game developer," Sweeney says, "and we want to build the kind of industry that we want ourselves as a game developer. There's been some awesome examples of this throughout time. You look at all the projects that released their source code, and became much more widely used because of it." He recalls starting his programming career on the Apple II, which came with a manual displaying a circuit diagram of the hardware and a list of all the code in the machine's ROM. "They were opening up the entire thing so developers could do as much as possible. And wherever you see parts of the world backtracking on it, and trying to lock consumers in and impose taxes on them and prevent platforms from talking to each other so that players are isolated into groups of the same hardware – we've always felt these things are holding the industry back." 

Now one of the key tools makers and suppliers, Epic's focus, we're told, is on giving developers all the advantages it's had to help build an industry that's "highly competitive, and healthy, and independent," Sweeney says. "The industry is best when every developer can choose each component to use and building their software, free from tying or coercion, or any other weird tactics. You can choose the best engine for your needs, you can choose the best online services, you can ship on all the platforms you want. And then you can build one game with a community that everybody can connect to. It's really been awesome to see the industry increasingly moving towards this view that this is right.

"If you look at the consoles," Sweeney continues, "Sony, Microsoft and Nintendo, and iOS, Android, PC and Mac are now fully connected together. There are no crossplatform barriers – we still have some walled gardens on iOS and Android, which we're unhappy with, but I think their days are numbered. And everybody's realising this, and they're just wondering about the exact timing and mechanism by which those walls will come down. It's a great time for the industry and I'm happy we can help. Everybody should keep us honest, you know – you should hold us to the same standard that we hold the industry to." 

We decide to immediately take him up on his suggestion. Is Sweeney saying that the ultimate goal for Epic, and for our industry, is to put an end to the decades-old concept of the console war – a future in which it doesn't matter what you play your games on, so long as you can play them? "Oops, sorry! Out of time," he jokes. "I'm not touching that." And, just like that, his face suddenly disappears from our screen. As the rest of us on the call laugh, we suppose it is sort of comforting that there are some things, at least, even Epic can't predict. 

This feature first appeared in Edge Magazine. For more excellent features, like the one you've just read, don't forget to subscribe to the print or digital edition at Magazines Direct.