<img height="1" width="1" style="display:none;" alt="" src="https://px.ads.linkedin.com/collect/?pid=4958233&amp;fmt=gif">
RSS Feed

Board Games | Sebastian Denzer |
19 October 2021

An essential aspect of software development is assuring the quality of the code one has developed. Testing is an important feedback element for developers, but testing a larger piece of software can be very time-consuming. As developers, we are basically looking for the most efficient approaches – and when it comes to testing, automation helps!


The consequences of poorly tested software range from blemishes and malfunctions to a non-operable or non-functioning application. And users are merciless in their feedback on poorly tested software – and rightly so!

If software is already in live operation and you add even a small new feature, you still have to ensure that the entire application functions smoothly and as expected after the update. That means you need to test your app – again and again. Manual and automated testing are both common software testing mechanisms.

The traditional approach is to have everything tested by the QA team, who are manually testing each part of the application. They will not only find bugs but also expose other problems that automated testing cannot cover, like bad or misleading UX, missing dialogue or localisation issues. The downside of manual testing is that it takes individual testers hours and hours of work.

In many cases, automated testing can be a very efficient alternative and make your developers’ lives a lot easier – be it API testing or integration testing, where you write code to verify the functionality of certain functions, code parts, or entire software modules.


We have seen how essential testing is in general. Of course, it becomes even more important for a live product such as the multiplayer online board game Catan Universe, which we developed for our client United Soft Media GmbH in collaboration with Catan GmbH.

The “Universe” part of the name indicates a whole new level of complexity: the digital version of a whole universe of game variants and modes, including two games in one! Both come with different scenarios or maps, combinations of different game rules, settings for each game, etc.

Mass Testing
Two games in one: Catan – a board game for up to 6 players (left), Rivals for Catan – a card game for 2 players (right).

And on top of that, Catan Universe offers both a single-player and a multiplayer experience, with the latter one using two different matchmaking algorithms to connect multiple clients, i.e. “players”. The multiplayer part itself currently features games for 2-6 players. Behind the scenes, we are also looking into ways to increase this number even further. One day we might be able to allow 100 (in words: one hundred) and more players to play one game of Catan simultaneously.


Catan Universe is a long-running product, being first released in 2016 and enjoying continuous growth over the years. The overall variety of different game setups with all the different game situations – especially in the “endgame” phase which can last up to 1 hour – could not be covered manually by even the largest QA team. We needed a method to test all game permutations with their different modes and scenarios automatically. So, we built something that we call “Automated Play Testing”.

What helped us from the start is that we already had an AI (Artificial Intelligence) implementation for both single-player and multiplayer games: in the game, you can play against well-known Catan characters like Marianne or Louis who bring in their own distinct personalities and game approaches. And while the players enjoy competing against these characters, we also use them as our very own QA testers who can play the game the whole night – without needing pizza and caffeine. Of course, they can never be as clever or creative as human players, but they can cover each and every game situation, and they can do it repeatedly without ever getting bored!

Having our testers in place, what else do we need?

First, the game must start in some kind of “autopilot mode” in which it will simply log into a user account based on a config file containing the autopilot settings, e.g. described as JSON data.


"autoPilot": true,

"email": "autopilot@automail.com",

"password": "123456",

"autoPlay": false,

"turboMode": false


See more ˅
Example config file for auto pilot mode; this is just a subset of all the possible parameters that could be used to set up Automated Play Testing.

Secondly, we want to enter the autopilot mode only in certain situations, so we use command line parameters to control which special functions should be enabled. For example, we differentiate between “active” and “passive” game instances where one active player will invite other, passive players to a game.

CatanUniverse.exe /autopilot /config user1.json

See more ˅
Starting Catan Universe in autopilot mode as “user1” with command line parameters.

Once we can automatically log in, we should start giving the autopilot input on what it must react to or even start its own logic to communicate with other clients. These could be

  • friend or guild membership requests to build a player base we can interact with,
  • lobby invites for “custom matches” (inviting other players that we have in our friend / guild list),
  • or the game client could start the “auto-match” matchmaking (searching for any players that match your favourite game settings).

Autopilot mode: automatically logging in and starting a single-player game – without any user interaction.

We keep the autopilot code separate from our game logic code, so that none of our developers can accidentally break the game logic. Of course, we also want to strip all autopilot-related code from the final release build of the game since we do not want players to use the autopilot AI – they should play on their own! And finally, we definitely do not want to open any security issues by allowing automatic control of multiple game instances – think of DDOS (Distributed Denial of Service) attacks, for example.

Which brings us to the final puzzle piece: to be able to start as many game clients as we want, we need some external tool to allow so-called “multi-boxing”. This is a technique that some players of online games use to play with multiple characters simultaneously, e.g. for getting an increased amount of resources or experience – or only as an “additional challenge”, like playing chess against multiple opponents at once.

Multi-boxing software allows us to start an application multiple times on one single computer, giving each running instance a separate set of files to work with – in our case the user config file – so that they do not interfere with each other.

Now that we have the basic pipeline running with multiple AI players testing the game, we can consider some more advanced settings: it could help to implement a “turbo mode”. As the video below shows, the game is running at an incredibly fast speed. In addition, this also functions as stress test for the servers’ multiplayer session handling because the AI will send game actions to the server faster than any human being could ever play the game.

Autopilot mode: mass-testing games in “turbo mode.”

You may also want to add logging and tracking systems, so that you don’t have to visually check the hundreds of instances running. Actually, this system should run day and night in order to cover all game modes and situations, find any crashes or blockers and even identify situations where the AI stops playing the game because it could not find any valuable actions to perform, thus helping to improve the overall game experience.

Finally, that also means that you should integrate Automated Play Testing into your CI pipeline, so that it runs like a regular automated test on every new release you are planning.


We developed our Automated Play Testing solution in order to increase the quality and lower the costs for the extensive manual testing efforts required to validate the application. Although the initial implementation was cost-intensive, and the maintenance and adding of new features moderately increases the cost, the solution is more economical than the manual effort that would have been required for testing as well as ensuring and improving the quality of the product.

Sebastian Denzer

Senior Game Developer

Sebastian is a Game Developer with more than 15 years of experience in all areas of game development, from gameplay programming with a specialisation in game AI to graphics and sound programming to UI/UX and multiplayer/networking. Since 2019, he has been the Lead Developer for “Catan Universe”, the digital version of the famous board game. Within that role, he ensures that the vision of the product is kept in UX and UI and is always keen to bring the physical board game feeling – the ‘haptics’ – to its digital counterpart. Sebastian, also known as “Denzi”, basically spends his whole life with game development, as he uses pretty much all his spare time to create his own games.


From This Author



  • 13 November 2023

    Delving Deeper Into Generative AI: Unlocking Benefits and Opportunities

  • 07 November 2023

    Retrieval Augmented Generation: Combining LLMs, Task-chaining and Vector Databases

  • 19 September 2023

    The Rise of Vector Databases

  • 27 July 2023

    Large Language Models Automating the Enterprise – Part 2

  • 20 July 2023

    Large Language Models Automating the Enterprise – Part 1

  • 11 July 2023

    Boost Your Game’s Success with Tools – Part 2

  • 04 July 2023

    Boost Your Game’s Success with Tools – Part 1

  • 01 June 2023

    Challenges for Adopting AI Systems in Software Development

  • 07 March 2023

    Will AI Transform Even The Most Creative Professions?

  • 14 February 2023

    Generative AI: Technology of Tomorrow, Today

  • 25 January 2023

    The Joy and Challenge of being a Video Game Tester

  • 14 November 2022

    Can Software Really Be Green

  • 26 July 2022

    Is Data Mesh Going to Replace Centralised Repositories?

  • 09 June 2022

    A Spatial Analysis of the Covid-19 Infection and Its Determinants

  • 17 May 2022

    An R&D Project on AI in 3D Asset Creation for Games

  • 07 February 2022

    Using Two Cloud Vendors Side by Side – a Survey of Cost and Effort

  • 25 January 2022

    Scalable Microservices Architecture with .NET Made Easy – a Tutorial

  • 04 January 2022

    Create Production-Ready, Automated Deliverables Using a Build Pipeline for Games – Part 2

  • 23 November 2021

    How User Experience Design is Increasing ROI

  • 16 November 2021

    Create Production-Ready, Automated Deliverables Using a Build Pipeline for Games – Part 1

  • 19 October 2021

    A Basic Setup for Mass-Testing a Multiplayer Online Board Game

  • 24 August 2021

    EHR to HL7 FHIR Integration: The Software Developer’s Guide – Part 3

  • 20 July 2021

    EHR to HL7 FHIR Integration: The Software Developer’s Guide – Part 2

  • 29 June 2021

    EHR to HL7 FHIR Integration: The Software Developer’s Guide – Part 1

  • 08 June 2021

    Elasticsearch and Apache Lucene: Fundamentals Behind the Relevance Score

  • 27 May 2021

    Endava at NASA’s 2020 Space Apps Challenge

  • 27 January 2021

    Following the Patterns – The Rise of Neo4j and Graph Databases

  • 12 January 2021

    Data is Everything

  • 05 January 2021

    Distributed Agile – Closing the Gap Between the Product Owner and the Team – Part 3

  • 02 December 2020

    8 Tips for Sharing Technical Knowledge – Part 2

  • 12 November 2020

    8 Tips for Sharing Technical Knowledge – Part 1

  • 30 October 2020

    API Management

  • 22 September 2020

    Distributed Agile – Closing the Gap Between the Product Owner and the Team – Part 2

  • 25 August 2020

    Cloud Maturity Level: IaaS vs PaaS and SaaS – Part 2

  • 18 August 2020

    Cloud Maturity Level: IaaS vs PaaS and SaaS – Part 1

  • 08 July 2020

    A Virtual Hackathon Together with Microsoft

  • 30 June 2020

    Distributed safe PI planning

  • 09 June 2020

    The Twisted Concept of Securing Kubernetes Clusters – Part 2

  • 15 May 2020

    Performance and security testing shifting left

  • 30 April 2020

    AR & ML deployment in the wild – a story about friendly animals

  • 16 April 2020

    Cucumber: Automation Framework or Collaboration Tool?

  • 25 February 2020

    Challenges in creating relevant test data without using personally identifiable information

  • 04 January 2020

    Service Meshes – from Kubernetes service management to universal compute fabric

  • 10 December 2019

    AWS Serverless with Terraform – Best Practices

  • 05 November 2019

    The Twisted Concept of Securing Kubernetes Clusters

  • 01 October 2019

    Cognitive Computing Using Cloud-Based Resources II

  • 17 September 2019

    Cognitive Computing Using Cloud-Based Resources

  • 03 September 2019

    Creating A Visual Culture

  • 20 August 2019

    Extracting Data from Images in Presentations

  • 06 August 2019

    Evaluating the current testing trends

  • 23 July 2019

    11 Things I wish I knew before working with Terraform – part 2

  • 12 July 2019

    The Rising Cost of Poor Software Security

  • 09 July 2019

    Developing your Product Owner mindset

  • 25 June 2019

    11 Things I wish I knew before working with Terraform – part 1

  • 30 May 2019

    Microservices and Serverless Computing

  • 14 May 2019

    Edge Services

  • 30 April 2019

    Kubernetes Design Principles Part 1

  • 09 April 2019

    Keeping Up With The Norm In An Era Of Software Defined Everything

  • 25 February 2019

    Infrastructure as Code with Terraform

  • 11 February 2019

    Distributed Agile – Closing the Gap Between the Product Owner and the Team

  • 28 January 2019

    Internet Scale Architecture