Predicting the future isn't a superpower, it's the primary skill we use when we decide what to build. Like any skill, you can get better with practice.
That's where I come in. My name is Danny, and I have a dream.


start
 
Where our leaders reliably predict the future
because they have predicted the past

Where we learn from our mistakes

because our predictions protect us from hindsight bias

Where expectations are communicated clearly 
with the reliable predictions referenced above



 
First an example prediction.

I'm 80% sure you would recommend these exercises to a friend.
I would actually take a 4 to 1 bet on it...
Because 97% of Twitch staff would recommend my previous trainings to a colleague.

This is a broader audience, but I'm still pretty confident.

This is how I talk. Much much clearer than statements like: you're likely to enjoy this, VR is going to be huge, and we've got a real chance at moving the needle on this one.

It's not that hard to talk this way, and be right ( your 80% predictions come true 80% of the time).
Lets keep going.
 
Now let's do one together :D

How big is the internet?
Specifically what percentage of people in the world use the internet (in 12/2016)?

Let's put together a range, we're pretty sure contains the true answer. Just so it's clear what we mean by "pretty sure", lets say we are 80% sure the answer will be within the range (making it an 80% confidence interval).

If we knew literally nothing (1%, 50% and 99% are equally likely):
The right 80% confidence interval would be 10-90%. 

But we know some stuff. I'm pretty sure 80%+ of Europe and North America are online. That's at least 1B out of 7.5B. And I think it's less than half in Africa and parts of Asia.

So let's say we're 80% sure between 35% and 65% of people are online....

Ground Truth 49% of people are online. This Internet thing is kind of a big deal.
http://www.internetworldstats.com/stats.htm
 
The training wheels are off.
It's all you now.

What percent of U.S. adults have Netflix in their household in January 2017?

***Your 80% confidence interval***

Should look something like 10-90%
That'd be the right 80% confidcence interval if you don't know anything about Netflix at all and think 1%, 50% and 99% are equally likely to be the true answer
 
No peeking! Enter your 80% confidence interval first. 









The answer to the previous question is right here.
(to give you a fast feedback loop)

54% of adults report they have Netflix in their household.

Netflix has 93.9 million subscribers globally at the end of 2016. and their global streaming revenue grew 41% year over year.

And only 9% of their subscribers canceled their subscription in 2015
(including people that signed up for a free month)

Contrast this with Hulu, which saw a full half of its existing subscriber base cancel over the course of 2015.

http://variety.com/2017/digital/news/netflix-q4-2016-earnings-1201962823/

https://www.leichtmanresearch.com/press/030617release.html

http://www.einnews.com/pr_news/321428028/parks-associates-announces-update-to-ott-subscriber-churn-rates-for-netflix-hulu-and-amazon-users
 
How many daily active users does Facebook* have?

***Your 80% confidence interval***

Let's get you started in case you don't know much about Facebook. 
We know from earlier there are 3.7B internet users out there.
If we knew literally nothing (1%, 50% and 99% are equally likely):
The right estimate here would be 370M-3,330M. 

*includes Facebook, Messenger, and Instagram
 
No peaking rule stands!









The answer to the previous question is below.

1.2B People use Facebook on a given day.

 
What percent of Internet advertising spend is on Google or Facebook?

***Your 80% confidence interval***

Should look something like 10-90% 

I'm anchoring you with these broad ranges, because the average person's 80% confidence interval only contains the true answers 50% of the time...
 
We're on the honor system... 









The answer to the previous question is below.

76% of advertising spend is on Facebook or Google.

$60B / year
Google is about 50% and Facebook is about 25%.
About 1/3 of the overall spend is on mobile

Mary Meeker 2016
https://www.washingtonpost.com/news/innovations/wp/2016/06/02/the-15-most-important-slides-in-mary-meekers-internet-trends-report/?utm_term=.3cfa284a201a
 
What percent of participants responded that the primary reason to use voice (Alexa, Siri, etc) was "useful when hands/vision occupied" (USA)

***Your 80% confidence interval***

Should look something like 10-90% 
Is that the primary reason you use voice?
Is it the primary reason you see others using voice?
Do you think it's less than half or more than half of the general population?
 
Hidden until the correct moment... 









The answer to the previous question is below.

61% of US respondents say the primary reason to use voice is that it's useful when hands/vision occupied.

Well that problem is not going away anytime soon.

Disclosure I work at Twitch, a subsidiary of Amazon.

Mary Meeker 2016
https://www.washingtonpost.com/news/innovations/wp/2016/06/02/the-15-most-important-slides-in-mary-meekers-internet-trends-report/?utm_term=.3cfa284a201a
 
By what percent did Amazon Web Services (AWS) revenue grow in 2016?

***Your 80% confidence interval***
I'll send you your score at the end, but that's only going to work if you gave intervals and not individual numbers

Hint: Amazon's overall revenue was up 27% year over ear. And AWS grew more...
 
Hidden until the correct moment... 









The answer to the previous question is below.

AWS revenue grew 55% in 2016.

Disclosure I work at Twitch a subsidiary of Amazon. These stats are pulled from news publications based on quarterly reports, not through any internal access I have.

In the first nine months of 2016, AWS had an operating margin of over 25% compared to Amazon’s company-wide operating margin of just 3.2%. So even though it was only expected to be 8.6^ of it's value, Forbes estimated it to contribute about 35% of Amazon's valuation.

http://finance.yahoo.com/news/amazon-enormous-growth-plans-could-200108477.html

https://www.forbes.com/sites/greatspeculations/2016/12/29/amazon-web-services-2016-in-review/#48252e8e2c28
 
Half Time

Why shouldn't you just make your intervals stupidly large, like 0-100%?

Well. You'd believe you a 100% chance of the true answer lying within that interval. So it wouldn't be your 80% confidence interval.

When I was young, my sister would split the cookie and I got to choose which piece I wanted. Here you are setting the odds, and you should be ok being on either side of the 4 to 1 bet that the true answer is in the confidence interval you gave.

Imagining you are making a monetary bet is super helpful here. Do it.
 
What percentage of Apple's Q1 2017 revenue came from iPhone Sales?

***Your 80% confidence interval***

Should look something like 10-90% 
That'd be the right 80% confidcence interval if you don't know anything about Apple at all and think 1%, 50% and 99% are equally likely to be the true answer

Their other main revenue categories: 
iPads
Macs's
Services (digital content, applecare, etc)
Other (Apple TV, Watch, Beats, accessories)


 
You will receive the knowledge below when you are ready... 









The answer to the previous question is below.

69% of Apple's Q1 revenue came from iPhone Sales

9% came from Mac sales
5% came from iPad sales

http://images.apple.com/newsroom/pdfs/Q1FY17DataSummary.pdf
 
What percentage of desktop search engine market share does Google have?

***Your 80% confidence interval**

Clue: Google is the market leader, and market leaders often own 25-90% of a market.
 
You will receive the knowledge below when you are ready... 









The answer to the previous question is below.

Google has 80.5% of desktop search engine market share

Bing is 7.0%
Baidu is 5.8%

and Google has 94% of mobile and tablet search market share
https://www.netmarketshare.com/search-engine-market-share.aspx?qprid=4&qpcustomd=0&qpsp=2017&qpnp=1&qptimeframe=Y

In January and February 2017
 
On average, how much more productive do admins say their teams are after adopting Slack?

***Your 80% confidence interval**

Clues: On average admins attribute a 25% reduction in meetings to Slack.

And 87% of admins agree Slack is succeeding in it's mission to make their working life simpler, more pleasant, and more productive.
 
The answer will be here when you are ready to receive it... 









The answer to the previous question is below.

Admins attribute a 32% increased productivity to Slack (weighted avg)

https://slack.com/results
80% agree Slack has increased transparency in their team.

Slack is my canonical example of how polish adds up.
77% of Fortune 100 companies use Slack.
http://venturebeat.com/2016/05/25/slack-passes-3-million-daily-active-users-930k-paid-seats/
 
What's the ratio between Airbnb's market cap and Hilton's?

***Your 80% confidence interval**

Your answer should like like .1 to 10
.1 would mean Airbnb is 1/10 the size of Hilton
10 would indicate Airbnb is 10x the size of Hilton
 
Clues: Hilton is the largest Hotel Chain.
 
The answer will be here when you are ready to receive it... 









The answer to the previous question is below.

Airbnb has 1.28 x the market cap of Hilton

Airbnb $30 billion
Hilton Worldwide $23.3 billion
AccorHotels $11.3 billion
Marriott International $17.8 billion
InterContinental Hotels Group $8.3 billion
Wyndham Worldwide $7.6 billion
Hyatt Hotels Corp. $6.9 billion
Choice Hotels International $2.6 billion

https://skift.com/2016/09/23/airbnbs-latest-investment-values-it-as-much-as-hilton-and-hyatt-combined/
 
What was the market cap of Bitcoin on 3/23/2017?

***Your 80% confidence interval**

Clues: The market cap for the second largest crytpo currency, Ethereum was 3.9 B that day.
 
... 









The answer to the previous question is below.

Bitcoin has a market cap of 16.7B

It's up by a factor of 2.6 YoY
https://coinmarketcap.com/currencies/bitcoin/#charts

Are you block chain ready?
 
Would you recommend this training to a colleague?


 
Will you recommend it to someone right now?

It's easy, could really change their future, and if you don't do it now you might forget...
     
 
Anything you'd like to tell me?

confusion, suggestions, feedback, etc