x

Confirm Download

Simultaneous file downloading works best in FireFox, Chrome, and Safari browsers. Please keep this window open until all downloads are complete. Some customers prefer using file download managers for Chrome or Firefox to better manage large and multiple file downloads.

Cancel Download

Data-Driven Products

    • Introduction  (Free)

      If you work in a product organization, deciding what to spend your time on consumes a great deal of your life. Many shops rely on a charismatic product leader to dictate the schedule. Without taking a position on the existence of divine guidance, this segment explores the possibility of using data to make systematically better decisions about what to build.
    • 00:10:30

    • Data Collection  (Free)

      Before we can use data in our decision-making, we need to accumulate some data. And in order to do this, we need to understand the kinds of questions we're trying to answer.
    • 00:24:21

    • Prioritizing Products

      If you work in a product organization, deciding what to spend your time on no doubt consumes a great deal of your life. Many shops rely on a charismatic product leader to dictate the schedule. Without taking a position on the existence of divine guidance, this segment explores the possibility of using data to make systematically better decisions about what to build.
    • 00:18:48

    • Evaluating Products - Part 1

      So you've released something to production. Congratulations! Now how do you know if it's doing what you intended it to do? Measurement is the answer, and A/B testing is a highly effective form of measurement.
    • 00:22:34

    • Evaluating Products - Part 2

      This segment begins a series of the common practical issues involved in running product experiments. The most basic problem is that counting things is hard.
    • 00:19:00

    • Making Big Changes

      In some circles, A/B testing has a reputation for being used to make trivial changes. But we also know from lived experience that big bets can go wrong in very big ways. This segment introduces the rationales for why we might want to retain the objective measurement of A/B testing in the context of making large, strategic moves.
    • 00:21:57

    • Winning Trust

      Data-driven decisions don't always have the best PR. In the public imagination, the word "experiment" can sometimes have negative connotations. In this segment we'll discuss how we can act in quantitatively grounded ways while maintaining an active user community.
    • 00:23:10

    • Conclusion

      This segment ties everything we've learned together into a unified methodology for approaching product problems.
    • 00:04:43

Data-Driven Products

  • Publisher: O'Reilly Media
  • Released: September 2015
  • Run time: 2 hours 25 minutes

If you work for a company that develops products of one kind or another, you may have worked on a new feature you thought was truly awesome, only to discover later that it was a dismal failure. In this video course, Dan McKinley shares hard lessons that he learned at Etsy about the value of using data to drive product decisions.

Using several real-world case studies on proposed changes to Etsy’s website, Dan demonstrates why using appropriate metrics to evaluate any product idea is crucial in the early stages—before you spend too much time and money on something that may not pan out.

  • Determine the type of data you need for analyzing specific product questions
  • Prioritize product ideas based on several metrics—do the numbers add up?
  • Examine case studies on two product ideas: one discarded and one pursued
  • Understand the benefit of A/B testing for evaluating product prototypes
  • Explore the novelty effect: products that only achieve short-term effects
  • Look at three methods for attempting large strategic moves, based on case studies
  • Learn how to win the trust of your community when introducing product changes

Dan McKinley is a web developer who spent seven years at Etsy, beginning in 2007 when it was still a fledgling company. During his tenure, he worked on activity feeds, search, recommendations, experimentation, and analytics. Dan currently works for Stripe, a company that provides a suite of APIs to help businesses accept online payments.