What Research Methodology Did We Use to Study Poker Players

The story of how we arrived at our current methodology starts after we completed the first Beyond Tells recordings in 2014. After I got all the footage from the first study, I sat down with a giant cup of coffee and I dove in. No methodology, no scientific method, I just pulled up when a player had a strong hand and a weak hand, and compared. The issue with this approach is that I was essentially embedding the confirmation bias that typically occurs when looking for poker tells or when studying human behavior at any level. I had an idea of what tells SHOULD look like at the table, and I would try to prove it. This was SUCH a humbling experience because I was frequently wrong. 

I would find a situation where a player looked nervous and reach the conclusion that this player would display 2-3 signs of anxiety when they were bluffing. I would then pull up another instance of them bluffing and they wouldn’t be doing that, or sometimes I would see that same player executing the same signs of anxiety with the strongest hand. 

At this point I wasn’t really trying to conduct research. I was just trying to get video examples of players doing things to make a simple training program on tells. There is obviously a TON of bias when you are observing behavior for the sole purpose of making a product. The unique aspect of this product was that we were giving everyone access to every single hand a player played. Because of that I had to ultimately defend every single thing that I was saying because everybody was going to get access to all the footage. There is no data set that is hidden away in some office. If I said a player did X when they were bluffing and Y when they were strong, people would be able to see if it wasn’t the case. Beyond that, the poker community contains some of the most intelligent, persistent, and sometimes venomous individuals on the planet, so I knew what I was saying needed to be valid. 

Eventually I stopped jumping around and focused on a single player. I noticed that they had bet slightly differently in the 3 hands I was watching. I decided to approach this is a little bit more empirically and I came up with a basic coding methodology. Then I had someone code all these hand movements, and that's when things started to really change and we truly started to STUDY behavior. 

I am telling you this because I want to be honest with how we arrived at our methodology. I believe research, specifically social science research, often contains a systemic flaw because of EGO, POLITICS, GRANT LIMITATIONS and a bunch of other factors. The reality is that you make mistakes, you are going to have biases, and all of this work on behavior has allowed us to create a methodology that I believe is one of the only ways to effectively study behavior in any domain, NOT just poker. Beyond Tells was really supposed to take 3 months and so far it’s been 7 years and our methodology is the reason why it has taken so long.  

For those of you who are not familiar with research, you need to understand that traditionally when you say the word “research” most people are referring to hypothesis driven research. In this type of research the goal is to address a specific, measurable, and answerable question. You are stating a central hypothesis and then designing an experiment to test that hypothesis. However, there is another approach to research that isn’t hypothesis driven and is known as descriptive research, which is often used to describe situations. Both of these methods have strengths and weaknesses, biases in their design, funding, etc. 

What really matters is what you want to accomplish. Through trial and error we built our approach on a research methodology known as Grounding Theory. 

I have always thought that the simplest way to think about research is through questions. You start with a central question which will evolve into sub questions. 

The key question at the heart of Beyond Tells was: “Can a poker player’s behavior at the table be connected to usable information?” That question is broken into dozens of categories. Which information does it makes sense to pay attention to? How do you know you can trust the information? Where on the body do you look for the information? 

The goal was to use both qualitative and quantitative research methods to help us discover just how valuable behavior is at the table. 

This is why we used Grounding Theory. Basically, we record behavior and create a large data set, come up with methods for coding that data set, code everything, and then look for relationships between the data. The useful thing about studying behavior at the poker table is that you have all this data in the form of game actions (player cards, bet sizes, board texture, etc) in addition to non-stop movement that occurs at the table. This allows us to answer questions about the relationship between the actions of the game and behavior. We can do this across the entire data set or just for a specific player. 

When research is hypothesis driven it starts with more specific questions. For example, Do players experience an elevated blink rate when they are in a spot where they perceive to be bluffing?

If this was the goal of Beyond Tells, we would have set up games, recorded blink rates, and published a study on that relationship. However, by focusing on a specific behavior we would be missing out on the value of the hands, or the speed and style of a bet, and so many other different areas. Studying blink rates is not really a deep dive into human behavior at the poker table, it’s just focusing on one element. 

By recording behavior, coding it, and tagging it, we can essentially look at relationships on multiple levels. For example, we manually coded every single time someone blinked in our sample. We marked every single time a player checked their cards and exactly how long they took to check them. We created specific coding methodologies for certain players to describe the different ways they would behave at the table. We did all this without knowing what we would find. We were just turning behavior into qualitative and quantitative data sets and THEN looking for relationships. 

We are not just doing academic research. Our goal is to teach people how to effectively navigate human behavior. By coming up with methods of coding and labeling behavior, it makes it much easier to teach that navigation because we can use data to drive the systems of approach. For example, we can see which areas of the body, which moments, and what type of players are more likely to give off information than others and we can back test the systems we build against our data set to more effectively teach people how to read behavior.  

We are always going to be conducting ongoing research. We are still coding our database using different coding methodologies and trying to refine how we study behavior at the table.

If you are interested in the more nitty gritty aspects of our research, check out some of the other articles on our blog. 

If you are doing a thesis or dissertation and want to do it on the behavior of poker players please reach out. 

If you are a poker player who wants to add a massive edge to their game make sure you check out The Behavioral Edge, which is the first week of the Beyond Tells Training.