Artificial intelligence at the gaming table

ALTKEY found itself at the heart of another Mattel™ flagship project: Pictionary Vs. AI™. A unique way to rediscover the classic 1985 tabletop game Pictionary™.

‍

Using the camera on a mobile device (or a webcam on a PC), our web application's artificial intelligence (AI) will try to correctly guess what users are drawing on their game boards. Users are challenged to identify the strengths and limitations of our AI. First by drawing so that their drawings are correctly identified, and then by evaluating whether or not other users' designs will score points.

‍

The idea was to take technological tools that are in step with the times, such as artificial intelligence and computer vision, and turn them into a highly accessible interactive experience. Once you have the game box, simply scan a QR code to open the web application and the game can begin. The in-app narrative (also created using artificial intelligence - Azure TTS) takes complete charge of users, who need only to follow simple instructions to have fun. The entire application weighs less than 10Mb and no internet connection is required once downloaded. With the privacy of our users' data at heart, our AI requires no external services and runs inside the browser itself.

‍

The biggest challenge of this project was to build an artificial intelligence capable of adequately identifying drawings made by a variety of users, using an equally diverse range of cameras. To achieve this, our technical team leveraged the immense Quick, Draw! database from Google™. This massive database contains over 50 million user sketches, categorized into 345 clues. Next, we trained our own artificial intelligence on our in-house computers with TensorFlow in Python. Currently, our AI can successfully identify 224 clues, covering a wide range, of which "snowman" and "campfire" are examples.

‍

Our team also used different image processing algorithms to process the photos taken by users, in order to standardize the images supplied to our AI. Following our recommendations, the Mattel™ team added markers to the erasable boards so that our algorithms would properly capture their positioning and orientation in space.

‍

Looking ahead, we'd like to continue our collaboration on this product with Mattel™ and plan to add additional game modes to the app and refine our AI to incorporate new clues. We're thinking of splitting our database into different categories to make it easier to integrate new clues into our AI. For example, if you play in the Nature category, the AI will know that the drawing provided cannot be a telephone, which greatly reduces interference and leaves more room for the addition of different clues.

‍