Playing Google’s Hidden Dino Game Using Brainwaves

BCI Aug 5, 2020

Recently, I’ve been researching BCIs (brain-computer interfaces), and I decided to start developing with one myself! BCIs can have a huge impact on people with disabilities. For example, BCIs have been used to increase motor functions for paralysis, provide additional methods of communication, and even regulate emotions. In contrast to all of these incredibly useful purposes, I used a BCI to control Google’s dino game. (You can get to it by typing chrome://dino/ in Chrome)

Blinks produce about 100 times more electrical activity than the brain itself, which makes detecting them quite easy. The process of connecting my blinks to the spacebar included finding a software development kit (SDK), setting up UDP senders and receivers, and sending keys. After doing this project, I have gained a deeper understanding of both BCIs and UDP datagrams. Plus, I now have the added benefit of being able to play Google’s dino game without touching my keyboard!

The SDK

Although it was hard to find, the SDK provides a variety of useful features. LibMuse (The SDK) is provided by Muse and supports Muse headsets designed until 2016. LibMuse provides samples that have a user-friendly GUI. These samples show head position, blink status, jaw clenching status, and much more! This is very useful since it would take a lot of time to get training data to detect these accurately. Under artifacts, the SDK records both blinks and jaw clenches. These are both stored as booleans, or true/false variables. LibMuse is based on the Universal Windows Platform.

Setting Up the UDP Sender

Since LibMuse is based on the Universal Windows Platform, there were some small things that needed to be bypassed. The Universal Windows Platform doesn’t support key sending for security reasons, but I can use Python to accomplish the same goal. A simple way to bypass this is to use UDP as an inter-process communication method. UDP stands for User Datagram Protocol and is a communications protocol. By using this, I can communicate between the two different programs running at the same time. For the most part, I was able to copy and paste this code from the internet, but there were some slight changes that needed to be made.

The sender is written in C++/CLI. Basically, it sends a message to the Python process. The content of the message is not important but it could be in the future (depending on what you are doing).

I placed the sender in one of the callbacks of the SDK. The code basically says, "If you're blinking now, but not before, then the sender sends a message".

The receiver, written in Python, waits for the message from the sender. When this program receives the message, it sends a space key.

Now that I can send a spacebar whenever I blink, I can go ahead and play the Dino game! (Or read newspapers, that’s fine too)

The Final Product

After I got everything set up, I was able to connect the muse and play the T-Rex game. The goal was to have the dino jump whenever I blinked. Unfortunately, my head is a little too small for the headset. However, it did work for my dad! Oh well, maybe next year. After all, I’m still growing?

The perfect cranium, shaved in the name of science

There was a slight lag, which is to be expected. Because of this, the dino can only jump so fast, which means we weren’t able to get very far into the game. No big loss! The overall process of getting the dino to jump whenever a blink was detected was still a lot of fun!

This same program could also be used to scroll articles and start/stop videos on Youtube. I would suggest exchanging the blink variable with the jaw clench variable since people tend to blink a lot.

Summary

Using the Muse to send spacebars whenever I blinked was difficult, but it was a fun and educational process. The SDK was very useful and could detect blinks, jaw clenches, head position and more. Since LibMuse was written using the Windows Universal Platform, I was unable to send keys directly in the program. However, I was able to set up a UDP receiver that sent keys using Python. Finally, I was able to play the T-Rex game, scroll news articles, and start/stop Youtube videos.

There are other projects that can be done by using the same concept. For example, perhaps the Muse could be used to control a 2D platform game. The player could change their direction by turning their head left or right, jump by clenching their jaw, and attack by blinking.

Now, how about you blink at that clap button? :-)

You can see my prior research on BCIs here.

This article was originally published on Medium.