Skip directly to search

Skip directly to content

 

AI Art in Game Production – an XDS 2022 Table Discussion

 
 

AI | Thomas Bedenk |
27 September 2022

In September 2022, we were invited to host a discussion about “The Future of Art & AI in Game Production” at the XDS conference in Vancouver. XDS stands for External Development Summit and has been hosted in Vancouver since its inception. It was started 10 years ago because there was nothing like it for the games industry. The main purpose is to connect big game studios and publishers with external service providers for engineering, art, QA, UX and localization. Watch this video looking back at the past 10 years of XDS to learn more about the event.

With this article, we would like to give those who could not attend the discussion the opportunity to still benefit. The participants agreed to have their names and the outcome publicized.

Together with leading industry experts, we set out to explore how the currently stunning and rapid progress in AI-assisted art might change the future of how games are being made. I lead the conversation for 90 minutes and was joined for a very engaging discussion by:



Photo of table discussion – photo credit: Chris Wren / XDS
Photo credit: Chris Wren / XDS

We started by all of us talking about our experience with AI-assisted art, and it became very apparent that everyone – either personally or with their team – had done some experiments with MidJourney, Stable Diffusion or similar text-to-2D-art generators. The common opinion was to see them as a tool for inspiration or even creating mood boards. There was an expectation that this technology will evolve extremely fast and also reach other aspects of the art pipeline. Interestingly, Declan mentioned approaches to create a whole website, game screenshots and UI rather than ‘just’ concept art with those tools.

At Endava, we inspired a few people to create a #midjourneypoem by feeding the AI only a poem or lullaby rather than elaborate prompts. Chris touched on other examples, like how AI already disrupted stock trading and the latest powerful AI additions to photoshop. We agreed that even now AI assets are already used for the illustration of articles and as a basis for creating video games.

Radu highlighted the differences between Narrow AI and General AI and the importance of context. Especially Narrow AI, with its small and specific tasks or pattern-finding, is making rapid progress. He specifically pointed out that art and 3D generation are some of the most promising areas for research and future developments in the field. Earlier in the year, Radu already presented some findings of our research work on how to create 3D assets for games with the help of neural networks.

After these initial thoughts, I had the group vote on a few claims by show of hands.

AI will be used as part of the art pipeline in more than 50% of development studios in 3 years: 6 yes, 0 no  AI generated art is already useful right now for concept art: 6 yes, 0 no  AI in game production will be a tool like any other rather than replacing any specific role entirely: 6 yes, 0 no  AI art should be able to be copyrighted: 5 yes, 1 no

It seems that those statements weren’t very controversial in our group, but they sparked further discussion. I also introduced the idea that towards the end of the session, every participant should write down a similar claim and we would all vote on those again.

From there, we headed right into the nature of how we work as artists and if AI will change what creativity or our human role in it is. AI art has already reached the point where it can surprise us. In a lot of its output, we can recognize the style of famous artists or even how the AI-generated result was probably influenced by certain pictures or artifacts of known IP. But is this so different to how human artists work and base their work on experience and influence of existing art? AI can for sure result in happy accidents, but can it be controlled and iterated into a final result? As an artist, after all, you want and need control over the output, so this technology won’t be stuck in ideation.

We discussed how difficult it is to reproduce the same asset or character given the wide range of data used for training the text-to-image AI models. Everyone agreed upon the necessity to build narrower models, fine-tuned to a given style. One application could be to transfer realistic assets into a stylized version matching your game’s art direction and trained using the work of your studio’s artists. The ability to create personalized models, focused on a specific look and feel, is the next step in the evolution of art-generating AI. There already are solutions for text inversion, such as Lexica.art, that can produce the prompt used for the generation of an AI image.

Let’s take a brief technical excursion into the magic behind AI art with our VP Cognitive Computing Radu:

Image-generation models combine the breakthroughs of the research work on zero-data learning, natural language understanding and multimodal learning.

The zero-shot learning approach refers to the ability of a model to generalize its training space to unseen object categories. These models combine the observed and non-observed categories through auxiliary attributes which encodes some distinguishing properties of the objects. The encoding of these properties is a representation of compressed data that defines a new dimension, known as latent space. The latent space, also known as embedding space, is useful for clustering similar representations or for interpolating data to generate new samples. The idea of combining word and image embeddings with the attention mechanism of transformer models, weighting the significance of different parts of the input data, lead to the creation of CLIP (Contrastive Language-Image Pre-training).

CLIP is able to solve bi-directionally the correspondence between text and images, and its architecture is a fundamental part of image generation models. However, given the subjectiveness of the human language, this approach can lead to an abundance of false positives. Thus, using CLIP for stylization needs additional regularization that can be obtained through tools such as generative adversarial networks (GANs).


An important question arising out of this is how it can play into a product or IP. Can artists opt out of being included in the training? Does a human need to do further editing in order to be able to secure rights as part of a product, or do we need to create custom, trained versions that only rely on your own data pool? Do we need a persistent metadata trail of how specific pieces of art were created? If you asked the AI to create a 3D car model that looks like a Ferrari, will you be able to legally protect that piece of art or would you have to restrain yourself to ask the AI to create a sportscar with certain features? Of course, this whole discussion relates to how we currently regulate these questions and where we draw boundaries. What is inspiration and what is piracy? Surely, we won’t have global regulations for this before AI truly manifests itself as a common tool.

Rather than diving into the various phases of an art production pipeline, our discussion focused on these foundational questions. The notion of how to train your AI and what data to use also directly leads into the topic of diversification, especially considering six middle-aged men sitting around the table. On the one hand, an AI should not be biased and be based on a well-balanced dataset; on the other hand, AI needs to also take the context of the user into account to offer its best results. Think about it like entering a search query into Google – often, it wouldn’t be as helpful if it didn’t also consider your current location and language.

So, what does it mean if I only ask the AI for beautiful architecture without any additional specifications? Should it take into account what I prompted earlier and what game I am making and that I might be a European located in Amsterdam? Or should it rather help me as a person to overcome the limitations of my own experiences and surroundings? While this becomes much more of an issue the more general the AI is, it should always be considered.

Even the best discussion has to come to an end eventually, so we entered the final phase. Trying to be a bit more concrete again, we each wrote down a claim on the table in front of us in order to vote on it. During the vote, we still had some debate and contextualization going on. Thus, similar to the prompts of AI art tools, we quickly realized that with such a complicated topic, statements need quite a bit of extra elaboration to be judged. You can clearly see this if you compare the last two statements. Nevertheless, the following image shows that we had very clear votes across the board.

AGI will be a reality in the next 20 years: 5 yes, 1 no  In the next 3 years we will have reliable 3D AI solutions (any part of the pipeline): 6 yes, 0 no  In the next 5 years an AI will be able to completely render a game (of unspecified complexity): 6 yes, 0 no  In 5 years, anyone can art direct (meaning with an accessible assisted approach): 5 yes, 1 no  In 5 years' time there will be an AI that can generate production ready 3D characters: 6 yes, 0 no  In 3 years, it will be possible to create AAA production ready characters by a natural language interface prompt through AI: 0 yes, 6 no

Only time will tell how accurate these predictions are, but if the expected time frame of 3 to 5 years for a massive impact on the games production pipeline turns out to be true, it shows how pressing it is to invest in this technology and approach. In the final minutes of our discussion, we addressed the elephant in the room: where will this investment come from? Will it be big publishers creating proprietary tools across their studios? You can see efforts like EA Seed already pushing the envelope.

Will it be service providers aiming to get a competitive advantage or start-ups coming out with amazing services? While big software companies, like Unity or Adobe, also heavily invest in this field, they aren’t usually focused on very isolated issues or a specific production pipeline. So, there is a great opportunity for smaller game studios to embrace AI as an integral part of their games, like AI-generated biomes and environments or individually crafted items.

Undoubtedly, we are looking at a huge impact and a massive potential to change how games of the future will be made!

Thomas Bedenk

VP Extended Reality

Thomas has over 25 years of experience in digital productions, focusing on Strategy, XR, and UX to support digital acceleration across various industries. Past XR projects he led span from product design and training to marketing applications for internationally known brands. Thomas is a frequent speaker at international conferences, an advisor for start-ups, and a fixture at industry associations and events. His love for digital creation started when he was playing Pac-Man against his twin brother on the Atari 800XL, leading Thomas to develop his first own video game in 1996.

 

Related Articles

  • 27 September 2022

    AI Art in Game Production – an XDS 2022 Table Discussion

  • 07 December 2021

    Hand in Hand with Artificial Intelligence in the Energy Sector

  • 05 October 2021

    How to Improve Intelligent Energy Storage Systems Using AI

  • 11 February 2021

    Mapping the Future Applications of Artificial Intelligence

  • 27 August 2019

    Taming AI in a Cognitive Driven Business World

 

From This Author

  • 30 March 2022

    The Metaverse Evolution and Learning from the Games Industry

  • 23 June 2021

    hi! I’m Thomas Bedenk

Most Popular Articles

Current Challenges in the Transportation & Logistics Industry
 

Transportation & Logistics Insights | Brian Estep | 22 March 2022

Current Challenges in the Transportation & Logistics Industry

4 Buy Now Pay Later Trends Set to Disrupt the Industry
 

Payments | Annmarie Mahabir | 22 February 2022

4 Buy Now Pay Later Trends Set to Disrupt the Industry

How Tech is Changing Sports Betting for the Better
 

Innovation | Andy Davies | 18 November 2022

How Tech is Changing Sports Betting for the Better

5 Things We Learned at World Aviation Festival 2022
 

Mobility | Vojin Rakonjac | 23 November 2022

5 Things We Learned at World Aviation Festival 2022

Cyber Security Incidents in Australia Highlight the Need for a Balance Between Risk and Innovation
 

Payments | David Marsh | 23 November 2022

Cyber Security Incidents in Australia Highlight the Need for a Balance Between Risk and Innovation

The Era of Ecosystems and the Rise of Open Insurance
 

Insurance Insights | Robert Anderson | 22 November 2022

The Era of Ecosystems and the Rise of Open Insurance

Top Challenges in Warehouse and Distribution Centers
 

Transportation & Logistics Insights | Brian Estep | 01 March 2022

Top Challenges in Warehouse and Distribution Centers

Staying relevant in the buoyant cross-border payments market
 

Payments | Peter Theunis | 15 November 2022

Staying relevant in the buoyant cross-border payments market

How the Board Game Catan Conquered the Digital World
 

Innovation | Moritz Hampel | 27 July 2021

How the Board Game Catan Conquered the Digital World

 

Archive

  • 23 November 2022

    5 Things We Learned at World Aviation Festival 2022

  • 23 November 2022

    Cyber Security Incidents in Australia Highlight the Need for a Balance Between Risk and Innovation

  • 22 November 2022

    The Era of Ecosystems and the Rise of Open Insurance

  • 18 November 2022

    How Tech is Changing Sports Betting for the Better

  • 16 November 2022

    4 Ways Insurers Can Leverage Technology to Differentiate Themselves

  • 16 November 2022

    The future of banking in the Nordics – being digital and personal

  • 15 November 2022

    Staying relevant in the buoyant cross-border payments market

  • 15 November 2022

    3 Experts’ Insights on the Complicated Relationship Between Fintechs and Banks

  • 09 November 2022

    How Can Banks Create a Secure, Optimised Cloud-Enabled Architecture?

  • 08 November 2022

    Tech is Good for You: How Wearable Edge Devices Changed Healthcare

  • 01 November 2022

    How Microservices Can Upgrade the Customer Experience

  • 25 October 2022

    How Technology Can Help Monitor the Circular Economy

  • 18 October 2022

    Why it’s time for banks to let go of legacy IT

  • 11 October 2022

    Buy vs. Build in Banking: Which Option is Right for You?

  • 04 October 2022

    The Rise of Super Apps: How Banks Can Compete

  • 27 September 2022

    AI Art in Game Production – an XDS 2022 Table Discussion

  • 20 September 2022

    Payments Data Monetisation is Key to Driving Sustainable Growth

  • 13 September 2022

    Navigating the Healthcare Ecosystem

  • 30 August 2022

    hey y’all! I’m Ashley Grant

  • 23 August 2022

    5 Ways to Fix Your Data Spine in Banking

  • 16 August 2022

    De-risking Digitalisation

  • 09 August 2022

    hi, I’m Brian Estep

  • 02 August 2022

    hey! I’m Lia Rollman

  • 19 July 2022

    The New Ways of Issuing Cards

  • 12 July 2022

    Scores on the Door: Rating Autonomous Vehicles

  • 06 July 2022

    Data-Driven Impact: Don’t Settle for Less

  • 06 July 2022

    We’re in Nottingham – a Q&A on Endava’s New Delivery Centre in the UK

  • 05 July 2022

    hey, I’m Chris Hart

  • 28 June 2022

    Platforms: a Blessing or a Curse?

  • 23 June 2022

    A Payments View on Marketplaces – How to Be(come) Successful

  • 21 June 2022

    Intelligent Commercial Underwriting

  • 14 June 2022

    The Future of Supply Chain: What’s Next?

  • 31 May 2022

    The Future of Autonomous Vehicles in T&L

  • 26 May 2022

    hello! I’m Hannah McCarthy

  • 24 May 2022

    Going Native: Why Cloud-Native Services are Essential

  • 19 May 2022

    How to Tackle Legacy – Breaking Down Walls Between Change and Run

  • 17 May 2022

    Advantages of a Yard Management System

  • 12 May 2022

    Are Phones About to Become the New POS Terminals?

  • 10 May 2022

    The Digital Economy is an Upgrade of Smart Cities and Communities

  • 05 May 2022

    hello! I’m Sumita Davé

  • 03 May 2022

    Physical Automation in the T&L Industry

  • 28 April 2022

    zdravo! I’m Andrej Kotar

  • 26 April 2022

    Open Banking in the US

  • 20 April 2022

    hello! I’m Paul Maguire

  • 19 April 2022

    Digital Automation in the T&L Industry

  • 12 April 2022

    How Do Banks Embrace Embedded Finance – Have the Fintechs Already Won?

  • 06 April 2022

    ESG Data Architecture is a Business Imperative – How to Get Started

  • 05 April 2022

    hi! I am Roy Murphy

  • 05 April 2022

    Modernizing the Shipping and Cargo Process

  • 30 March 2022

    The Metaverse Evolution and Learning from the Games Industry

  • 29 March 2022

    Do Androids Dream of Trading Electric Sheep for Digital Wood? An Introduction to Automated Game Design

  • 23 March 2022

    Real-Time Payments in Australia – Why Corporates Should Get on Board

  • 22 March 2022

    Current Challenges in the Transportation & Logistics Industry

  • 16 March 2022

    bok! I’m Sanja Cvetkovic

  • 15 March 2022

    Rapidly Transforming: Healthtech Trends in 2022

  • 08 March 2022

    How to Digitize Warehouses and Distribution Centers

  • 01 March 2022

    Top Challenges in Warehouse and Distribution Centers

  • 28 February 2022

    Tackling CIB Legacy at its Core

  • 23 February 2022

    salut! I am Isabela Buhai

  • 22 February 2022

    4 Buy Now Pay Later Trends Set to Disrupt the Industry

  • 15 February 2022

    salut! I’m Natalia Ciobanu

  • 14 February 2022

    Product-Led Innovation – a Q&A with Joe Dunleavy

  • 01 February 2022

    Buy Now Pay Later: Will Regulation Burst the Bubble?

  • 31 January 2022

    Innovation Will Spur Ireland’s Race to the Top

  • 28 January 2022

    The Value of Digital and Automation in the Product Returns Process

  • 26 January 2022

    Virtually Disrupted? Keeping Pace with Accelerating Customer Expectations

  • 19 January 2022

    The 3 Big Ps in Modern Insurance: Personalisation, Prediction and Prevention

  • 18 January 2022

    An Introduction to Mobility as a Service in the US

  • 12 January 2022

    Buy or Build? A Game-Changing Question in Insurance

  • 12 January 2022

    hello! I’m Paul Willoughby

  • 11 January 2022

    Payment Service Providers 2.0

  • $name

We are listening

How would you rate your experience with Endava so far?

We would appreciate talking to you about your feedback. Could you share with us your contact details?