In today’s increasingly digitized world, we are creating data in unprecedented ways. Ubiquitous and conspicuous social and mobile connections have empowered social consumers to broadcast their locations, opinions, thoughts and emotions to the world in real-time with nothing more than a few clicks, many times a day, on multiple platforms.

The omnipresence of this data is changing the relationships between individuals, as well as between businesses and their customers. However, most companies are not well positioned to take advantage of this Social Data Revolution (wikipedia). While new market research tools allow us to observe differently how consumers make their decisions, the fundamental shift is that consumers now make their decisions differently. Today’s social consumers are weaved together in the social fabric of social data for their purchasing and lifestyle choice. How prepared is your company to participate and take advantage of this shift?

Traditional enterprises face numerous challenges: views on privacy focused on the legal department rather than on the customer, traditional approaches to decision making, legacy operations and metrics, and anachronistic data strategies, if at all.

They are threatened by companies that understand the power of social data, building a measurement-focused culture and customer-centric data strategy. Amazon started this approach in the 90′s – product reviews have helped customers with billions of decisions. Netflix forecasts demand and optimizes operations based on the data its customers create.

For incumbents open to change, the data their existing customers socialize are both a threat and an opportunity. What do large firms need to do to navigate this rapidly evolving field of social data?

In the heart of Silicon Valley, next to Facebook, the world’s largest social data factory, the Social Data Lab aims to address this question. Directed by Dr. Andreas Weigend, former Chief Scientist of Amazon.com, it helps companies understand the irreversible impact the Social Data Revolution has on individuals, business, and society.

The Lab’s Aldo Briano and Tim Holley, graduate students in Management Science and Engineering at Stanford University, developed the Social Data Intelligence Test. The test assesses how an organization tracks, analyzes, and acts on social data, information that consumers knowingly and willingly share. Completing it takes about 20 minutes. It is designed to encourage executives to think critically about their current practices and to reevaluate existing assumptions.

Scores are represented across four dimensions: Product, Organization, Customer-Centricity, and Company Reputation. They benchmark a company’s competencies and highlight opportunities for improvement, creating a starting point for a strategic realignment in this new era of social consumers.

The Social Data Intelligence Test is available online at socialdatalab.com/intelligence. Please take it and let us know what you think!

Aldo Briano, Tim Holley, Asha Gupta, and Léo Grimaldi are graduate students in Management Science and Engineering at Stanford. Ron Chung. Gabriela Maford and Hin Leung and contributed to this article.
Dr. Andreas Weigend (@aweigend) teaches the course The Social Data Revolution: Data mining and e-commerce at Stanford, directs the Social Data Lab, and was Amazon’s Chief Scientist.

In 1993, The New Yorker published a cartoon by Peter Steiner of two dogs at a computer that became an instant classic, tacked up on bulletin boards everywhere.  The caption: “On the internet, no one knows you’re a dog.”

A recent survey of a group of 98 tech-savvy Stanford students shows that the world Steiner depicted may be behind us.  Asked about how they behave—or would behave—in a variety of situations online, students painted a picture that looked surprisingly like real life.

“Talking online is like talking in public,” summarized one student in the class.

That’s a huge shift from the thinking of a generation ago, when the internet was largely seen as an anonymous frontier—a place where alter egos thrived and people could escape the constraining shackles of real-world social pressures and norms.

Why the change?  Certainly, the increasing ubiquity of online social networks and their role in bringing people’s “true” identities to the internet has been instrumental.  As services such as Facebook and LinkedIn bring real life social networks to the digital realm, it appears that reputation, social capital, and many of the other constructs that govern our real life behavior are coming with it.  As one student readily admitted, “I am extremely conscious of my online social image.”

Indeed, in a world where anything that you put online can be tied back to your identity and seen by others—in particular those you know—you might think twice before chiming in.  In fact, many students felt that these traditional social pressures were even stronger online than in real life.

“Everything online can be tracked and measured,” noted one Management Science & Engineering graduate student.  “If you make unfair, disparaging remarks anywhere, this history could be attached to your persistent online identity.  In some ways, you can be more anonymous offline than online.”

“For me, spoken words are taken away with air,” said another student, “but written words stay forever.”

So why not just remain anonymous?  Increasingly in today’s online world, it seems that in order to be heard, one must also be seen.  As one student pointed out, if you remain anonymous online, you won’t be taken seriously.

“Having the anonymity of the computer is a nice luxury, but I think my opinion won’t hold as much credibility,” she said.

Many companies have recognized the self-enforcing power of these social dynamics.  Amazon.com—which has been at the vanguard of the social data revolution—long ago started publishing the real names of its customers alongside their reviews.  Quora, a leading social question-and-answer service, relies even more heavily on identity and reputation to keep the quality of its users’ contributions high.  And with an ever-growing number of online services leveraging Facebook Connect, the notion of a single, persistent identity that travels with you across the web is not far off.  Those who choose to remain anonymous will likely be pushed increasingly to the fringes of online society.

Changing norms and expectations about privacy in the digital age have only reinforced these pressures.  While online privacy was a core concern for many students, most surveyed also recognized the reality that any information they chose to share online was essentially public, as it could be relatively easily found by anyone who wanted it badly enough.

“I think there’s a general misconception that what you put on the internet won’t be found,” said one student.  “What people need to realize is that no matter what you blog, upload, tweet, etc., you’re responsible for it and it should be something you would be happy to stand behind.”

Watching what you say (and share) is not the only real world phenomenon beginning to permeate the online world.  If these Stanford students have anything to say about it, a fundamental shift will also be taking place in people’s purchase decision-making—toward something that looks a lot more like how people shopped before the internet came around.

It was the internet that spawned crowd-sourced reviews and enabled the wisdom of the masses to inform which products and services we should buy.  New restaurant you’ve been thinking about trying?  Let’s see how many stars it has on Yelp first.  Less than four, you say?  In that case I might only go if a Groupon comes out for it.

Prior to having this data at your fingertips, you had to rely much more on friends’ recommendations to make buying decisions.  And while this fundamental change in behavior isn’t going away, the irony is that online social networks are starting to actually bring us “back to the future”: the clear consensus among the students is that the social data they will leverage in the future will be sourced less from generic public opinion, and much more from their trusted networks of friends.

As one student summarized it, purchasing decisions will be based on “information from people I trust who own the products I want to buy.”

“I will take advantage of social data by taking into account which of my peers have purchased from which retailers or brands,” said another student.  “I could also post my own questions on social network sites to seek more targeted and personal responses.”

There is a clear expectation that with this social overlay will come customization in every facet of the shopping experience, drawn not just from data on an individual’s habits (as Amazon does today), but also from those closest to them.

“There will be an app that recognizes my tastes (based on my profile and behavior) and will match it with those of people who I trust (friends who have recommended things in the past and on which I acted positively),” predicted one student.  “The app will know what stage of my life I am at, and will be able to predict what I have and do not have (based on my online consumption behavior and general online queries).  Therefore the app will suggest the products that I will want to (and be able to) purchase, cross-referencing them with suggestions/reviews from my friends (or other people who have a similar profile to mine).”

What can a survey of a group of computer science, engineering, and business school students at the epicenter of online innovation tell you about online behavior?

It won’t tell you what a grandmother in Urumqi is doing right now.  But it may very well tell you what her grandkids will be doing next year.

Jason Lee and Sameh El Amawy are currently MBA students at the Stanford Graduate School of Business. Evelyn Larrubia is a national award winning investigative reporter and editor from Los Angeles who is spending a year as a Knight Journalism Fellow at Stanford.  Michael Marcotte is also a Knight Journalism Fellow.

“What are other online services [besides Facebook and Twitter] where you frequently share things?”

“What is the most useful app/site/software/etc that you discovered in the last year that you think most people don’t know about but should?”

“What is the coolest startup you know that you think [Prof. Weigend doesn't] know yet?”

1. Play or download the mp3 of the speech,

2. Leave your comments on the slides and see the annotations of others,

3. Leave your comments on the transcript and see the annotations of others, and

It was exciting to be part of the World Innovation Forum, an event packed with insights and a turnout of more than 800 thought leaders and a fantastic line-up of speakers. I had great company on stage, speaking between Chip Heath (who I went to grad school with) and Biz Stone (who co-founded Twitter).

I have put up the audio of my talk [mp3, 35 min, 32MB], the transcript [pdf | docx], and the slides [pdf | pptx]. And in terms of press commentary, check out what The Huffington Post, FastCompany, HSM, OnInnovation, and Steve Todd write about it, and please add your own thoughts via the comment box at the bottom of this post.

I am fortunate to present the insights on WIF2010 and the Social Data Revolution by two guest writers: Noah Burbank, a student in Stanford’s Social Data Revolution class this Spring, and Ted Shelton, the CEO of Open-First. And, as always, please do tell us what you think by leaving a comment below. Thanks!

WTF is WIF??

by Noah Burbank

This last week I won an essay contest. The first prize was a trip to New York and pass to the World Innovation Forum (WIF). I had only one question when I got the congratulations email: WTF is WIF? It would be easy to be cynical about WIF. Why do people come to these things? Does each employee that you send to the conference really come back $1,699 more innovative, not to mention travel costs and lost work? But after two full days in my ill-fitting suit with ill-matching and equally ill-fitting shoes, I think I figured it out. Don’t expect your insurance company to transform into a new and shiny machine when the innovation delegation returns to the office. But if you take a broader view about shaping corporate culture and increasing mindfulness, then it’s hard to say this wasn’t a bargain. Those looking for answers probably went home disappointed, those looking for questions came returned much richer.

There were three kinds of information offered at WIF: histories of innovations, broad paradigms of thought, and new and actionable frontiers for future innovation. The histories inspire hope for future innovations, like Michael Porter’s story about the Western German Migraine Center, which coupled integrated, life-cycle migraine treatment with high-volume specialization to dramatically increase outcomes at a reduced cost. The broad paradigms of thought, like Chip Heath’s elephant and rider metaphor about how motivation is both emotional and rational, provided a common language with which to frame problems. While this is all good and well, without giving some hint about where future innovations can come from, it’s somewhat like giving a captain a compass and sextant but no maps: he can head north and keep a steady course, but he doesn’t really have any reason to go in any one direction rather than another. It was in this third area that Andreas excelled. Blazing very quickly through a number of exciting and complicated themes, Andreas gave the audience something to do.

Having taken a class with Andreas, I already knew a number of the tricks up his sleeve, but more importantly, I knew about what his goal when he speaks. He wants to discuss his thoughts and opinions about culture and technology (the zeitgeist, to use his native German), but he also wants to give people the first actionable steps to participate in the social data revolution. We all understand that the Internet is full of information, but until I actually started writing Python scripts and trying to use that information, I didn’t really understand. Similarly, while everybody can nod their heads along with Andreas about the transition from e-business to me-business to we-business, until you actually start seeing the data, you’re not seeing the whole picture. Not unlike in his class, Andreas gave the WIF audience four concrete homework that anybody could implement:

1. Put a plus sign (+) after a bit.ly link
If you put a plus sign (+) after a bit.ly link, suddenly you’re seeing all of this information about how many times the link was clicked and when. Just seeing the information makes you start asking questions, for example with the link Andreas used, http://bit.ly/16Zidx+, you notice that all 5,630 clicks occurred on the same day, the day the email was sent out. This tells you about the distribution channels. How about Twitter embeddings – who decided to repost this link? The right questions only appear when you’ve gotten your hands dirty and seen some of the data.

2. Ask for the last 1,000 queries entered on your website
Andreas talks about the voyeuristic pleasure of having everybody tell you their darkest secrets – but this is precisely what people enter into their search queries. Go to your IT person and ask for the logs of the last 1,000 queries on your website. Do you notice trends? Are there keywords that you’re not using which people tend toward naturally? Does something odd stand out? Are your competitors being searched on your site?

3. Engage with an individual on twitter who has been following your company or a competitor company
Is someone lamenting about the bad service of your company over twitter? Or, even better, your competitor? Reach out to people who are complaining about your company and assist them to make their experience better. And if they are complaining about your competitors, offer them solutions right when they need it!

4. Reverse mentoring
Does your company know what the latest social trends among the younger generation are? How do the twenty-five year olds spend their time, how do they communicate and think? A great way to find out is by hanging out with them, and learn from them! Get to know your youthful interns, learn about their friends and social lives to gain insights.

I’m glad I had the opportunity to participate in the WIF, and I think that I’ve come away smarter for it. Not smarter in the concrete, now-I-can-do-linear-algebra-but-before-I-couldn’t sense, but smarter for having spent two days thinking about how people have asked fruitful questions in the past, how to frame my questions for the present, and where to find the new questions for the future.

Results of the Open-First Survey at WIF 2010

by Ted Shelton

Speaking at last week’s World Innovation Forum in New York, Dr. Andreas Weigend outlined what the social data revolution is about and how these tools, methods, and data can contribute to innovation. In a survey completed by over 100 of the attendees after the talk, we had the opportunity to sample a cross-section of current thinking on data and innovation, with some expected and some surprising results.

The survey asked attendees three questions, what they felt was most interesting in the material that Dr. Weigend presented, how these ideas might be applied in their own companies, and what barriers they anticipated encountering in trying to implement these ideas.

Most attendees focused on the core tenant of the social data revolution: that people are willing to share information, indeed that Internet services we use every day are creating enormous pools of data. And that they are easily and inexpensively accessible and, if used properly, that they may be of enormous value to the enterprise. As one attendee put it, “you have opened my eyes widely to data and our digital air!”

While innovation was the primary topic of the conference and the talk, the importance of engagement with customers was another strong theme of the talk and one that was emphasized by many survey respondents. People’s willingness to share data is in part a function of the relationship that the company forges with them. And the balance of power in what Dr. Weigend calls “WE-business” is clearly shifting in favor of the consumer. One attendee put it this way, “…consumers have more power and control over companies than the companies themselves.”

A few clear action items came out of this talk for attendees. Armed with knowledge that vast amounts of valuable information are just a few keystrokes away, the majority of those surveyed stated that they intended to immediately look at how their companies collect, share, and use data in their innovation practices. One particular recommendation frequently cited was the suggestion to examine a company’s web logs in order to determine what search terms visitors are using both to arrive at the company’s site and, once there, what they are hoping to find.

One of Dr. Weigend’s messages that clearly stood out for attendees was that small steps could be taken quickly and inexpensively, and could be used to demonstrate value to the organization before larger investments had to be made. Showing the http://bit.ly site and examples of social networking and viewpoint services gave attendees a set of clear starting points for their own investigation (a full list of sites shown during Dr. Weigend’s talk is listed at the bottom of this article).

Nonetheless, one of the most frequently cited barriers to getting started was a lack of resources, particularly money. This was overshadowed though by the enormous number of comments on how difficult it can be to get an organization to do something new, summed up by one attendee with the simple comment, “change is hard.” For innovators this is, of course, the recurring deep challenge we have in every organization and the perceived barrier of limited resources or perception of exposure to risk are often protective coloring for an organization that is resistant to change. Numerous comments to this point were made:

“Culturally we still have people who protect data rather than share.”

“We are still operating in the old cathedral-like style of consumer management”

“The ideas can be seen as scare and out of the realm of control.”

Overall, however, World Innovation Forum attendees participating in this survey were optimistic that small scale experiments can be done that help the organization recognize the benefits of the social data revolution, and begin to accept the changes that it brings. As one participant noted “We need to make the case of why sharing opens us up more as a company and provides us with greater opportunities.”

Ideas abounded on how to start these experiments. A number of people noted that a great way to start is with one local branch or group that may be more forward thinking. Another noted that listening in the hallways and the lunchrooms to the company’s own employees could provide a simple small scale example of the power of social data. And utilizing online tools to gather customer feedback that is already available is another example given of how an organization can inexpensively show how social data can change how we think and work.

List of tools, social networking, and viewpoint sites discussed in Dr. Weigend’s speech:

Foursquare.com – geolocation check-in
bit.ly – URL shortener
Oakland Crimespotting – geolocation crime mapping
DataSF – open data for San Francisco
Bigapps – open data for New York
Fitbit – feedback data for behavioral change
Levi’s – sharing opinions via Facebook’s “Like” button
Groupon.com – collective buying power
Nike running – feedback data for behavioral change
Nokia Betalabs – concurrent engineering with consumers
My Starbucks Idea – product design with consumer input
Threadless.com – product design with consumer input
Stack Overflow – QnA with feedback and review system
Flatseats.com – airlines’ seats reviews
SeatGuru.com – airlines’ seats reviews
Tripkick.com – Hotel rooms reviews

Curious about the best response to the question from IDEAS PROJECT last week?

1. Ron Chung won the prize for the “best answer”.

Here is what Ron said about a “friend” in the era of the social data revolution:

There are two types of ‘friends’, (i) real ‘close-to-heart’ personal friendships, and (ii) online social friendships.

(i) In real personal friendships you more carefully screen and maintain that relationships.  In these situations, you provide more physical and emotional attention compared to online relationships.

(ii) Online social friendships form to maintain touchpoints with people we interact with (sort of like a large addressbook). In the context of consumer internet and social networks/media, an online ‘friend’ is someone you form a weak connection through some form of engagement. This engagement can occur through real world meeting or simply an online exchange (e.g. blog comments, Twitter message, etc).

Also, in these online friendships, there is ambiguity around bilateral versus unilateral ‘friendships’.  For example, Twitter uses ‘followers’ & Facebook uses ‘fans’ to represent unidirection relationships and Facebook uses ‘friends’ to denote bilateral friendships.  However, some Facebook ‘friendships’ are not truly bilateral. They are simply ways for one side to collect ‘friends’ for the sake of amassing a large audience. All of this points to a desire for people maintain touchpoints with people through online medium should they ever want to re-engage them.

In the end, online social friendships give us ambient awareness of what is going on with people, giving us a type of “reality-TV news” channel.

2. A few thoughts by Andreas Weigend:

Daphna Oyserman suggested:

• Someone whose happiness makes me happy and with whom I feel eager to share my own happiness (knowing that the feeling is mutual).

My favorite one-liner came from Jason Wei in my Stanford class:

• Someone I’m comfortable being myself with.

My own points (to the degree anyone can have their own points after reading through hundreds of responses) would be, that a friend to me is:

• Someone whose eyes I want to see the world through.
• Someone who can make me laugh until tears run down my cheeks.
• Someone who brings the best out of me, accepting me the way I am (or want to be).
• Someone who manages to pull me out of a (real or imagined) bad situation.

Please use the comment box below for your comments. Thanks!

3. Finally, Chuanyang Chee shares his insights on the longer SDR survey.

This survey on the Social Data Revolution was developed by Chuanyang and Andreas and taken by Spring 2010 students at Stanford’s The Social Data Revolution, and Tsinghua’s The Digital Networked Economy.

Finding that long-lost best friend from elementary school has become trivial ever since Facebook hit a total subscription of 400 million active users. But having not kept in touch for a couple of years or decades, what is the point of connecting now? Does he even still consider me a friend? Remember me?

The meaning of friendship, as most respondents to a survey revealed, often include descriptions such as caring for each other, trust, sharing personal information and thoughts, and mutual helping. Many of us have more friends than our attention can handle. On average, a survey respondent has 474 friends on Facebook. After a certain society catches the “Facebook fever”, a large proportion of each user’s social circle can be found on it. In the survey, the respondents were asked to come up with an analogy for Facebook and almost 15% of the responses compared Facebook to high school or a year book. Many felt like they were being constantly watched and judged for the things they do on Facebook. One interesting response even compared Facebook to an “online version of society”. To a certain extent, Facebook IS like a society because of the flexibility and scalability that it offers. Fluid like a society, Facebook allows members to join or leave altogether, create and share, hide in a corner and watch, and form groups and networks among willing parties. The versatility of how Facebook can be used also explains why the respondents find Facebook to be less personal than Instant Messaging and email but less public than twitter and blogs.

Facebook has made real-time connectivity (through notifications) possible for the users and it has the key features of IM, email, twitter and blogs. Users can choose to give third-party applications permission to access their information in return for some benefits such as access to games or using the application to find friends near them (geo-location). All these contribute to the surface reason for why people are attracted to Facebook. Going beneath the technological capabilities’ appeal, we need to understand why friends play such a big role in the success of Facebook.

In the survey, respondents were asked what they would do if Facebook was going to shut down in two days and all data would be destroyed. Most of them chose to download photos shared with and shared by others and record down contact information of their friends. Generally, we don’t like to lose the feeling of being connected with others and shared experiences that we have with our friends, regardless of whether we are close to them or not.

Friends are like social mirrors, we use them to know our inner self. We like to hang out with people who are similar to us so that we can know our “self” better. Without our friends, we can’t share things, we can’t form collective memories of what we experienced together, and we certainly can’t receive help when we need it. As Facebook gradually transforms into an online society, the friendship network becomes the glue to keep users from leaving; especially for users with a long friends list. Therefore an efficient way to manage these friendships gradually becomes more important. Unfortunately, Facebook does not seem to prioritize our 1047 friends the same way we would in real-life by collecting and analyzing relevant data.

The process which can be used to categorize our friends is generally observable through our behavior and the explicit information which we share. For close relationships, we tend to have more memorable shared experiences, allow them to see more personal information and would be more willing to help them. We simply form more connections by talking about them, posting and taking photos of them.

The earlier versions of Facebook allowed users to fill in information on how they are related to friends that they add. That was interesting data which could be used to analyze the closeness of a relationship. By collecting such data, Facebook can create different web-like networks to represent and understand each user’s relationship with his or her friends. Friends can be represented with simple nodes and the strength of relationship represented by the length of the links between the nodes. Relevant data collected can be used to adjust the length of the links between the each user’s node and the friend’s node. The physically closer two nodes are to each other, the stronger their relationship is. This is akin to the cognitive associative model in organizing semantic memory where strong associations are formed when two nodes are often activated simultaneously. When two friends update relationship details about themselves, tag photos with both of them in it, chat daily on IM, regularly messages each other and click on each others’ shared items, they generate clues to the strength of their relationship.
When such data are amassed, clusters of friendships would appear and being able to identify such clusters might be valuable for businesses. Instead of advertising indiscriminately to the different friendship clusters, businesses can target the “ring leader” of each cluster and focus their resources on getting the leader to participate in try-outs for certain idea or product. By the block-leader approach, the clusters may then be subsequently influenced through word of mouth.

Presently, Facebook allows us to manage our friends by manually grouping them into certain categories. But if Facebook does not want to be replaced by a new social media network, then the improved version will need to take data collection and analysis into the next level by tracking individuals’ online social behavior. By doing so, Facebook can play a part in managing the relationships by recognising relevant updates, notifications and recommendations coming from each individual user using some algorithm. However, we should not solely focus on strong relationships and forsake the weak ones. This is because Granovetter’s notion of the strength of weak ties suggests that we benefit from having not-so-close friends from different social circles. So what we need is to use technology to manage our network of friendships more efficiently such that we can readily distinguish the friends who will be recommending us our next job from the friends who we get emotional and social support from.

• Play (publicradio.org)
• Download (weigend.com, mp3, 2MB)

And please share what you think… Comment (via Facebook Connect) below!

Transcript (from http://bit.ly/dataNPR)

KAI RYSSDAL: The data trail that we create every day is only growing. Every time we go online, every time we use our cell phones, companies log our preferences. They make suggestions, and they remember what we do. Even though a lot of consumers have gotten used to that, a lot of businesses are still trying to figure out how to use our data to the best effect. One of the first companies to realize the social potential of consumer data was Amazon.com. And Andreas Weigend used to be the chief scientist there. Welcome to the program.

ANDREAS WEIGEND: Good to be here.

RYSSDAL: This thing called the social data revolution, what does that mean really?

WEIGEND: Social data for me means sharing data. Sharing data means, when you think about Facebook, sharing data with your friends. When you think about Twitter, sharing data with the world.

RYSSDAL: We all share data, and that’s fine, and we’re good with that, but your thesis here is that what’s happening is companies are getting smarter about how they use it, aren’t they?

WEIGEND: Yes, I think that we see a split in companies. That those companies, which genuinely embrace what the users can do for them — Amazon.com as an example, wish lists where you can actually recommend stuff to your friends. Facebook is a clear example, all these platforms. Versus those companies which try to define themselves as traditional companies. Unfortunately many airlines fall into that category, many banks fall into that category, and as a new generation in the social-data revolution draws up, the expectation towards the airline and towards the bank is precisely the same as the expectation towards Amazon or other good companies.

RYSSDAL: Isn’t supposed to be about, though, not what users can do for the company but what the company can do for the users?

WEIGEND: No, I think it’s what users can do for the company because ultimately what users do for the company, they do for themselves. So if I tell Virgin America what kind of people I like to chat with — maybe implicitly by chatting them up on the flight between here and San Francisco — then they can do something for me by chance having that nice person sit next to me next time I take that flight.

RYSSDAL: You actually, there’s a quote of yours that I read as I was preparing for this talk where you talk about this thing called explicit data creation. I mean that’s what these companies are trying to do, right?

WEIGEND: Yes, so implicit means that you sniff the digital exhaust, that you go behind them…traces. Explicit means that you create incentives for users to share stuff with you. And why do they do this? Well, on the surface because they want to get some value out of it. Maybe they want to get a camera which fits their needs better, or credit card that suits them more. Deep down, however, I think it is the need for belonging, for self-expression.

RYSSDAL: Is there an element of this that is vaguely creepy, vaguely Big Brotherish?

WEIGEND: It is your choice what data about yourself you’re revealing. So as we move from the implicit data collection to a much larger percentage of what you create being explicit data, you are more in control than you’ve ever been of the bits that you generate. I asked my students at my Stanford course last quarter what was the most interesting experience you had on any social-networking site, and the answer was the fact that we can connect with anybody who we ever met in our life. So those were grad students at Stanford. If you think about people 10 years younger, in their early teens, they would say, what are you talking about, has it ever been different?

RYSSDAL: Andreas Weigend is the former chief scientist at Amazon.com. He teaches data mining at Stanford University. Thanks so much for coming in.

WEIGEND: Thank you.

Transcript:

Ladies and gentlemen, it is an honor for me to be here and to talk to you about what I think it the most interesting, the most exciting thing I can talk to you about. I’m actually going to talk to you about a revolution. I’m going to talk to you about the Social Data Revolution.
I’m going to start by giving you a little bit about my background. I want to start by asking you what do you think those terms have in common: physics, finance, e-commerce, and marketing? In the 1980’s the most exciting data sources in the world were in physics. I did my undergrad education at CERN in Geneva, where I felt like the king of the universe, being logged on one computer with other computers on different continents.
Then I figured I needed to learn some computer science, specifically some machine learning; how could I learn what the underlying patters are from collecting data. I did my PhD at Stanford and in the 1990s I was an Assistant and Associate Professor at NYU, where I looked at the patterns people leave in finance on Wall Street. I looked at the traces of traders.
Then of course, the web came along. I started a company called MoodLogic that allowed people to discover songs they didn’t know to look for. The key idea there was that we got people to give us data, metadata about those songs.
I then went to Amazon.com where I worked directly with Jeff Bezos as his Chief Scientist. Again, I was thinking about how we could make sense out of those traces people leave on Amazon.com. Why, you might ask. Ultimately, it was to sell them stuff but the way we did it was slightly different. We wanted to get people to share data with us so we could do a better job in helping them with their decision-making processes. What all of those things have in common is that data is the deep, underlying element.
Let’s talk a little bit about data. There are more than a billion connected Flash Players in the world. Of course, as I was preparing this talk, my computer asked me, “Should we allow se.amazonaws.com to access your camera and microphone?” I said, sure. Remember it’s okay. Think about what that does for marketing. If the computer knows whether I’m sitting there by myself or maybe with a friend of mine and we’re looking at the same stuff, it can do a much better job in showing me marketing messages, by understanding my situation.
Now, while my background of course is the online world, I have a couple of examples from the offline world. I worked with a European company called METRO Group, one of the world’s largest retailers, on understanding how we can measure the behavior of people in the physical store, and each item of that store has an RFID (Radio Frequency Identifier). As you’re walking through that store, you consider that cream cheese, but you’re worrying about losing some weight so you put it back and you go for the low fat version. METRO Group observed that, just like Amazon.com observes what you are doing online.
Here is another example, a car company. What we see here is that you put a device in your car, another physical object, and that device may make life more fair for you because if you’re not driving – like right now; my car is in San Francisco at my garage; I don’t have to pay any insurance. What are you willing to give up for the fairness of only paying when you drive? Also, if you are returning home at 3:00 in the afternoon, versus at 3:00 in the morning, should you be paying more, or should you be paying less? On the one hand, you might be more awake at 3:00 in the afternoon. On the other hand, there might be less traffic at 3:00 in the morning.
Once again, it is about understanding the patterns of people through the underlying data they produce. The examples we have seen so far were collection devices of implicit data. I now want to tell you how marketing can use the explicitly shared data that people willingly and knowingly share with you.
Here are two examples. The first example is YouTube. The movie I just showed you was from YouTube. Every month, 100 years of video get uploaded to YouTube. Another example which was mentioned this morning is Facebook. Every month, 5 billion pieces of content get shared on Facebook, about 1 piece of content per every person on Earth.
Marketing in this Web 2.0 era, people often ask me what is Web 2.0 and I say, “Web 2.0 means People 2.0.” People have shared. People have changed how they see the idea of what sharing data about themselves and about their friends is.
For us as marketers, when we talk “people” what we really mean is customers. After this introduction, I’m going to talk about C2B, which means customers sharing with business. Then C2C, which is customers sharing with other customers. We’ll do a little exercise and then I will talk about C2W, which is customers sharing with the world. I’ll give you some insights, and then close with a couple minutes about a domain I’m very interested in, namely travel.
Let’s start with C2B, customers sharing with business. Imagine that for all the people in the room, you knew all the things they have bought. Imagine that you also knew all of their friends, both within the room and in the outside world. Finally, imagine that you knew all of their secret desires. What could you do with that?
Is that some dream far in the future, or is it already reality? Let me give you some examples from Amazon.com. One of the ways in which people share data is they share reviews with Amazon.com. First, it’s about that hard drive. They had 116 reviews and you know it’s a pretty good hard drive. People willingly and knowingly share those data.
What do you think is the impact of recommendations? Is it 1%, 2%, 5%, 10%, or 20%? If it was 1%, HSM would not have invited me to come here. If it was more than 50%, then I wouldn’t come here either. It must be something in the middle and indeed, it depends on product group, the price point, and also on the gender of the person who is looking for it. It’s something between 10% and 25% of increase.
Deep down in recommendations is Amazon’s C2B data strategy, purchases, clicks, reviews, wish lists; everything you can get, recommendations based on view data, on click data and on purchase or buy data.
In the simplest case, “Customers who viewed this item also viewed…” which means you collect the clicks as you go along and you leverage the collective intelligence of the people. You collect what they do and you combine it in a way that new people can have their decision-making process supported by what other people have thought about, prior to them. “That’s customers who viewed … eventually viewed …”.
Now, you can do the same for buying data. “Customers who bought this item eventually bought…” What is the difference? Viewing means that you see alternatives you can buy. Buying means you see products that you buy in addition to what you are buying, cross selling, up selling.
I think the best of all is “Customers who viewed this item ultimately bought that item.” Going back to our hard drive here, Amazon.com helping people make decisions based on collective intelligence, it turns out that 42% of people who looked at that item eventually bought that item. That makes you feel you can’t be all that stupid by buying that item.
I promised you something about the secret desires. How does Amazon know about the desires of people, about their intentions? The answer is through search. Who here trusts their husband or wife more than you trust Google? Who here shares more with Google than with your spouse? [Laughter] Okay, so for instance, I shared with Amazon that I was looking for video on demand, specifically for the HSM Management TV. Now, Amazon knows what I’m interested in. That’s what I mean by C2B, the customer sharing with the business. I ask you, if you have a website, do me the favor, and spend one hour tomorrow, or next week, looking at the search terms people enter. You will understand what peoples’ desires are, desires they might not even share with their spouses.
Here is an example of how people desperately want to cancel AOL, all the variations from a search log. At Amazon.com, we tried to implement whatever we can to make it very easy, as lightweight as possible for people to share with us in a C2B way.
For instance, at the bottom of every single page, there is this feedback box. You can say if something is broken, or if an image is not right, or if the language is inappropriate. Per day, Amazon gets about 1,000 such comments, out of a million-plus people visiting. You could say that’s not that much, on the other hand, you have your entire customer base debugging the site for you. If you capture the context as well, such as the page where people make this comment, then you have a very powerful way of collecting the intelligence of people to help you do a better job in the context in which people are.
Let’s summarize what we have learned so far. We have talked about a few data sources. We’ve talked about intention data through search. We have talked about attention data, such as transactions and clicks. We have not really talked yet about situation data, which would be the device you have. Are you searching from your iPhone, your BlackBerry, or your mobile phone? Are you searching from the web? By the way, where are you? Are you in Mexico, are you in Buenos Aires, are you in Germany? Those are all data sources which are very important for the marketer, which in traditional marketing you have almost no idea about.
I figured I would share some ideas for you about connection data, about data between people, and specifically this is what AT&T did in the United States, where they tried to market a new phone product. We compared traditional segmentation such as demographics, psychographics, loyalty data with simply looking at the connection data; who calls whom, in other words, the calling network.
The set up is that AT&T has a new product. They want to compare how much they get with traditional segmentation and it turned out they got a rate of .28%. They threw away all the traditional segmentation and only looked at the calling data. What is your feeling? Do you think it is better if you only use one data source than this 2.8%, or do you think it is worse? The recommendation I might make right now is looking at the data, such as Eduardo makes a phone call to me. Eduardo has bought the product, and then AT&T says, “Andreas, would you like to buy that product,” as opposed to saying, “Andreas is 49 years old, he was born in Germany, lives in San Francisco, has a house in Shanghai; he might be a good candidate for that product.” What is your feeling? Who thinks it’s better if we just look at the data between Eduardo and I? No hands up. Who thinks it’s worse if we just look at the one data source?
This reminds me; people say that 17 minutes into a talk, half of the audience will have fallen asleep and the other half will be having sexual fantasies. [Laughter] Let’s try again. Who here thinks that we’re doing better? Okay, because otherwise, why would I show the example? How much better – it’s actually by a factor of 4.8, not 4.8%, but 4.8X, 380% better, which is a pretty amazing lift, just using one new data source. That would be my advice for the gentleman from the phone company beforehand; look at data other people don’t have. Don’t be stuck in the old way of how things have always been done. This century is a century of data.
By the way, data double about every other year. That means this year, mankind will produce about as much data and share about as much data as the entire history of mankind has produced to the end of last year.
In the Bay area, a lot of new companies emerge that try to give us these data. One company I invite you to play with is called Skydeck. With Skydeck, if you have a BlackBerry you can download a client. If you have an iPhone, you can download a client. Otherwise you can do the website. It analyzes your calling behavior and it’s one of the most powerful tools for a sales force, by reminding you who you should be calling. It’s also very interesting. For example, my friend Go, I called him much more often than he called me. What does that mean?
To summarize this part, businesses try to reach consumers. That is how the conversations use to go. As Phil Kotler said in the morning, “Marcom trying to hire people who get the message out,” but what we are seeing is that is not where the conversations are. The conversations are primarily between customers, C2C conversations.
That is why we now move to the second part of the talk, after we talked about C2B data, we are now going to talk to the C2C aspect of data and the Social Data Revolution. The Social Data Revolution means data that is shared knowingly and willingly.
I am not interested in going through the digital trash or sniffing the digital exhaust. First of all, it can be bad for your health and secondly, it’s a lot of work to find maybe a few nuggets somewhere. Instead, listen to people; listen to what they say to each other. C2C means consumer-to-consumer, and C2W means consumer sharing with the world.
At Amazon.com, when you buy a book, after you have checked out, Amazon asks you, “Do you have some friend who might be interested in that book?” “Yeah, I can think about somebody.” If that person buys that book within a week, he gets a 10% discount and we don’t want you to go empty handed. You will get the same dollar amount credited toward your next purchase. Ah – repeat customers.
The conversion rates were absolutely amazing. Why? Because you just bought the book which means you determined the context. You actually determined the item, that very book, the content, and you also help Amazon do marketing. You determine the connection because you tell Amazon.com to “Please, mail my friend and tell him that I bought that book.” In this case, people know that I’m smart and maybe I’ll make some money, as well.
That has been brought to perfection with this one button, with a company that is about 5 years old – “share” – that is the most important button of this decade. Each month, 5 billion items get shared on Facebook. Let’s spend a couple of minutes on how that works and then we’ll discuss it together. I’ll give you a few minutes for an exercise on how you can use this in your company. I want to show you; first of all, that Facebook is highly relevant, growing faster in Latin America than anywhere else. It’s doubling every four months. The current numbers in Brazil is about a million people, but in four months it will be 2 million. In eight months, it will be 4 million, and in twelve months, it will be about 8 million people. That’s growing quickly.
Worldwide, every given day, about 100 million people come to Facebook. Here is an example of sharing. In my course at Stanford this year, I had Reid Hoffman who is an old friend of mine from Stanford come, he started PayPal, and then he started LinkedIn. He talked to my students about what it really means to be part of that Consumer Data Revolution, or that Social Data Revolution.
With a quick video loaded on YouTube, you can watch it. I shared that with my friends on Facebook. With a push of one button – that is what is new – I can reach everybody who has a confirmed relationship with me. I come back a few minutes later and it turns out that already three people said they like what I just shared. People start tagging stuff, so people interact. Somebody took a photo of me. Somebody else says, “That’s Andreas standing in the room giving a seminar to executives,” and tags me with “Andreas Weigend.” Those are all those lightweight interactions that are happening on Facebook.
Distribution is key there. As an experiment for my course at Stanford, I made a page called “Social Data Revolution” at www.facebook.com/socialdatarevolution where people share what they find interesting. It has been an extremely rich source of information from hundreds of people contributing. What the metrics are is not unique users, not the number of people who have subscribed to that page, but what they do there. Do they comment, post, like things, and so on? Those are the new metrics of engagement.
The difference is they do things knowingly and willingly. Here, “Wow!!! Golden Earrings! Thank you Darling! That’s so cute! But who has got the second pair you’ve bought?” You have to be very careful with what information you grab from people’s behavior which they might not be all that happy to have shared with their 500 best friends.
I want to give you an example from a company in Mexico, called Burger King. You know the term “viral marketing.” In viral marketing, typically the goal is to bring new people to your site. Burger King decided the opposite. Burger King said, “Dump 10 friends; get rid of ten friends, and we’ll give you one free burger.” They executed that on the Facebook platform, similar to the iPhone App Store, an ecosystem for third party applications. It’s about a million developers, a million programmers working and trying to write apps, doing this for little money for companies like Burger King. There are a lot of games there, as well. Here is what happened.
Friendship is strong, but the Whopper is stronger. Dump ten friends and get a free whopper. In the end, your love for the Whopper sandwich proved to be stronger than 230,906 friendships. What happened? After ¼ million people were dumped, Facebook shut it down and said, “We don’t like that app anymore.”
Here are a couple of other networks. LinkedIn one week ago has 236,000 people in Mexico and has grown by more than a factor of 2 in the last year. LinkedIn is a professional network. The idea is that you tease people with some information.
For instance, as I last logged in it said, “Your profile has been viewed by 115 people in the last 15 days, including…” – and then there is some generic description. “If you want to know who these people are, you need to subscribe to the service.” The lowest level is $25 a month and you can also subscribe for the $250 a month to actually get access to a lot of information about people. That’s not cheap, but if you think about your sales or marketing it is cheap. If you just do one good sale, it’s nothing compared to that sale.
One of the features of LinkedIn is that you might want to be introduced to somebody, which is very important for sales. For instance, a German VC, Kolja Hebenstreit, asked me whether I could introduce him to a friend of mine, Amy Jo Kim]. Of course. Short endorsement to Amy: “Kolja is great!”. Done.
For those of you who are interested in buying aggregate data, maybe for sales leads, for risk reduction; here are a couple of examples of what is popular in the U.S., right now. Unbound Technologies in Palo Alto, California; RapLeaf in San Francisco, California; 33 Across in Mountain View, California.
Here are two examples of what these companies provide you with; they look at all these social networks. I just gave you two examples of Facebook and LinkedIn. High 5, Orchid, or whatever ones you think about, they try to understand who your friends are, and produce a list of prospects for a high end hotel chain. If I’m staying in high end hotels, chances are my friends also like to have a decent hotel. They look at my friends, they try and relate with their friends, and then they deliver a prospect list. That’s a very different segmentation from what you’re used to. It’s much similar to the AT&T example than to traditional segmentation examples.
It’s not only about making money; it’s also about not losing money. The example I want to give you for that is fraud reduction. They get claims, all the time, and they need to decide “Should we spend a lot of resources to investigate this claim, or should we just pay it and be done?” It turns out that not only “birds of feather shop together,” but also “birds of a feather steal together.” If my friends are short of shady in the sense that there are a lot of claims coming in that we’re not sure about, they better spend a lot of resources investigating my claim. On the other hand, if my friends are all clean, then no worries; no need to spend any money on me.
What we have seen in this part of the talk is a spectrum from very private data, on the one extreme; to very public data on the other extreme. Consumers have become quite good at asking “What do we get in return for sharing data?” What is relatively new is that they’re willing to share data that we never expected them to share.
In order to give you a little break, what I want to do in the next 8 minutes is I want you to talk to your neighbor. Have a conversation with them, and from the plethora of examples I have given to you about C2C data, figure out one marketing idea, based on C2C data. Think about Facebook, LinkedIn, and try to be specific; what would be your first step, and what would be your measure of success? Write your key idea on a piece of paper. We will have some of the assistants here run through, collect them, and then I will pick a few of them and we’ll discuss them. Talk to your neighbor, figure out one idea; how can you take what I have talked about using a lot of examples and make it concrete? Write it on a piece of paper; get it to me in 6 minutes. I will have 2 minutes to look through them and we’ll discuss 3 to 5 of them.
Thank you for your comments. Somebody says, “Make a section on a webpage where the customers can share opinions.” That is interesting but I have bad news for you. Most people don’t come to your webpage.
I did some work for Nokia. It turned out that for the top ten search results at Google for Nokia Map Activation, none of them were Nokia. The power of what we have here on Facebook, as an example, is that people distribute what they find interesting to their friends. What about influencers here on Facebook?
First of all, there are traditional demographics you can get if you want to target people. Traditional ads allow you to get very rich targeting data because people on Facebook are honest about their gender. Think about it; if they were lying about their gender, their friends would immediately say, “What’s up with that? You always said you’re a man and now you’re suddenly a woman? No way.”
Here is a question about influencer marketing. “What’s the difference between real life and Facebook?” You have all heard about influencer marketing which means indentifying those people who are influential and marketing to them. By marketing to them, you then reach all of their friends for free.
In real life, the chain length between people, for 38% of people, is 4 or longer. With Facebook, 86% of all chain length – work of mouth and mouth-to-mouth communication is 4 or more people. Why is that the case? Pushing a button – “share” is easier than talking to somebody. Besides, by pushing a button you reach all of your friends, whereas by talking, very few people are as lucky as me having hundreds of people who actually listen to them.
That key difference of 38% versus 86% means what you know from the real life world is not true in the virtual world. In real life, what matters is to have influencers tell their friends stuff. What matters on Facebook is how good your message is. Don’t try to really massage the message. Try to use the feedback you get in making the product better, or said very simply, “Don’t focus on the influencers; focus on the product.”
I have many more answers by you but in the interest of time, I want to move onto the next part, our third part which is C2W. I started a search on that other computer for “moon food.” Do you know what moon food is? I didn’t know either, but I just looked at what is a popular search term on Twitter right now, and it turned out that between starting my talk and right now, 10,000 people sent out tweets about moon food. We can refresh this here; the point is while we don’t really know what it means, some people, namely 10,000 people in the last hour found it was worth talking about. That’s an interesting buzz, isn’t it? It’s free. People talk about it and nobody pays for it. You have peoples’ attention if you talk about moon food. At the end of today, we’ll try to figure out together what moon food means.
New media tend to start off as better old media, but then do something very different. Just like television in the early days was people standing around a microphone; TV is not just a better radio but is very different. The web is not just better television, but Facebook with interaction is very different. Twitter is not just better short messages, but it’s very different.
Here is my friend Go Kasai again. Here is his wish list. These are the books he tells the world he is interested in. “This is what I want. This is who I am. This is what I’m interested in.” They share their desires, and they share their intentions.
Here is a video. Nike Plus had the following idea. You get people to share information about themselves with the world, C2W. In this case, it was about running together. You buy a device that you put in your shoe and as you go running that device records where you’re running by GPS, how fast you’re running etc. Then you upload what you have just done, you upload your run to a website called www.NikePlus.com and the world can see where you’re running.
I know people always have security concerns. The world could track you down and do funny things with you, but they can do that anyway. The key thing here is that you have now found, as a marketer, a very different way to connect with your consumers. People buy something from you, shove it in their shoe, connect it to their iPod, and now people come, on average three times a week, to your website. Isn’t that a marketer’s dream? How often would you go to Nike’s website beforehand? Probably never, but once you got that device, you’ve starting going to the website three times a week to compare what you’re doing, to compare yourself to the others, to run with others, to hook up with people and say, “Let’s go running together.”
Trevor Edwards, who is Nike’s corporate Vice President of Global Brand and Management says, “We’re not in the business of keeping the media companies alive. We’re in the business of connecting with consumers and also of consumers connecting with consumers.”
I deliberately chose that example as a very physical example, something you shove into your shoe. Something where the consumer shares with the world what data he is creating. Here are some other examples.
In the virtual world, there is something called Delicious, which was bought by Yahoo a couple of years ago. I used to work with Joshua Schachter, who created Delicious. What people share with the world through Delicious are bookmarks, their bookmarks, the things they find interesting, URLs. Delicious now allows you to explore. If you will, there is a web on top of the web.
HSM controls the links that it puts elsewhere, but this is what users find useful. You can of course follow HSM’s links, but you could also follow the links or connections that people put on top of the web. That is Delicious.
The example that is in everybody’s mouth right now is Twitter. I just want to make the point that whereas with Delicious you share URLs with the world, Twitter is even easier. We were actually considering for the conference to allow you to tweet in and to have, behind me, displayed what your questions are and what your comments are. I tried it out in my last class at Stanford last week, and all the students did were they wrote jokes behind my back. That’s one use of Twitter, to have real time commenting on what’s going on in real life.
I looked up marketing and that was when I made the slides for this talk, a couple of weeks ago. There was a social marketing conference in Chicago, and it was interesting how all these things were less than 20 seconds old. It really is real time. If I want to know what food is good in a certain restaurant, I tweet, I search for that restaurant and it will tell me that tonight at the Taco Al Pastor, this is what you should really get. That is real time search.
Here is John Batelle who started the Web 2.0 conference. He tweets saying, “Just landed in Atlanta – very long trip. Yargh.” Somebody says, “Hey, have you followed what I have been doing with …” and then his random website. We can push this ever further with this little cartoon – what companies actually do.
BestBuy monitors Twitter for what people are saying about BestBuy. BestBuy has people who constantly watch for the BestBuy tag on Twitter, as well as related tags. Some people are happy and say, “BestBuy so totally rocks. I just bought this game station and it works so well. I had awesome service. The Geek Squad came and they fixed all my problems,” and other people are not happy. They share it in a C2W way with the world.
If somebody is unhappy there, the customer service agent immediately gets online and tries to have a conversation with that person by saying, “Hey, I work for BestBuy. I saw your tweet. I saw you’re unhappy. Here is what I can do for you. Do you want me to call you right now? How can I help you?”
Think about the difference from traditional marketing. You have people, in a moment, who think about your product and who think about your company and you can reach them right there, right at that moment, on Twitter. You need basically no infrastructure. You need some person who has access to the computer and knows how to type. Not only about people who might be unhappy with your own product, but think about people talking about your competitors.
Let’s say somebody is not happy with a washing machine they bought at Sears. There is nothing wrong with BestBuy talking to that person and saying, “I saw your tweet and you’re not happy with your washing machine. Let’s have a conversation. Let’s see how we can help you.” Or somebody has computer problems. “Don’t worry about it. Let’s send the Geek Squad over to your house.” For marketing or customer service, these are truly unprecedented ways.
Slightly more traditionally, Dell using Twitter for promotions. In the first eighteen months which ended in December of last year, Twitter had a total of 7,000 people who were following Dell. They sold stuff for a million dollars. It has exploded in the last six months, in the sense that it’s ten times more people than there were six months prior to this. That is a factor of 10 over a half year. Ten times ten with the same growth is a factor of 100 over the year and that’s an amazing growth.
The revenues are small. They just sold stuff worth $3 million right now. The fixed cost for doing this is essentially zero. You can start tomorrow, coming up with some promotion you want to run on Twitter, and create your own experiences there.
If we think back, ladies and gentlemen, the task used to be connecting computers. Then we moved from connecting computers to connecting pages, about fifteen years ago. Then, the last couple of years, we moved to connecting people. Think about Facebook as an example here. What I’ve been trying to get across to you in the last hour was that what the web is really about is connecting data.
I showed you that the car is basically a chip with wheels. The shoe is essentially a chip with heels. You thought you owned the customer because the costs of the customer going somewhere else were quite high. Unfortunately ladies and gentlemen, I have bad news for you as marketers. You don’t own the customer anymore. The customer is quite able to check out other places where they might get products that suit them better. Remember, I told you one of the key things about Amazon.com was that Amazon was supporting the decision making process of the customer. You don’t own the customer.
You would say, “But the product, I know about my product.” I have bad news there, too. Google knows more about your product than you do. Maybe the brand? “At least we own the brand,” but no, if you do a search the brand is owned by the people who talk about your brand, and no longer about you.
What’s left? What is left are sites that are platforms, like Get Satisfaction, where people are going in order to get customer service. They go to a neutral site and there is a little button that says, “I have this problem too” and people enter what they are interested in.
Another example here is how we moved from controlled production for the masses to uncontrolled production by the masses; that’s why you’ve lost your brand. Starbucks, My Starbucks Idea – whereas Get Satisfaction is a neutral platform, this site is owned by Starbucks and it is about sharing, voting, discussing and seeing what other ideas other people have. 60,000 contributions were there when I checked a few weeks ago.
What the Social Data Revolution is really about is how the mindset of consumers has shifted. People trust reviews. People trust their friends more than they trust official specs. People use their friends’ attention as a filter for information and as a way of discovering things.
The main insight for marketing is that you have to come up with ways for how you use that social filter to have people discover your products and services. In closing, I have a few examples from travel.
One website is called Flatseats.com where people discuss in gory detail, first and business class seats in much more detail than any airline would share with you. For instance here, early this year, United came out with a new business class and within days – January 2, January 3, January 6, we had detailed reviews and the question was, “I just don’t understand why United Airlines could not think outside of the box.” Did United listen to them? No.
The second example is SeatGuru. Do you want to know which seat you want to sit in? SeatGuru has seats labeled in each aircraft by tens of thousands of people. You know whether that one has that back which doesn’t recline fully, or that missing arm rest on the lovely exit seat.
What about hotel rooms? TripKick is an example where every single hotel room gets rated. It’s not enough to know that you’re staying at the Hilton Hotel in Mexico City. It turns out that rooms with 04 at the end have an oversized room, a nice quiet corner room. On the other hand, rooms with 61 are possibly next to the ice machine, with a lot of noise and next to the elevator. You probably don’t want to stay in that one. Would the hotel tell you that information? Of course not. If you call the central Hilton reservation line, would they know about it? Probably not. Does the web know about it? Yes.
Booking.com, which was purchased by Priceline, is now four times larger than Priceline itself. It gives you reviews based on your own status. Let’s say if you’re a single traveler and you really loved that hotel, chances are if you are a family with seven children you might not love that hotel. By conditioning on your specific purpose of traveling and by who you are, the reviews they extract from you are more honest and help other people more.
Booking.com, Agoda.com – the Asian counterpart, PriceLine.com are of course in the business of helping you as a customer to make better decisions. Sometimes, we want to get closer to the intentions. We want to get closer to the future. Here is a company called VirtualTourist, which allows you to talk about not only where you have been, but also (in green) to about where you want to visit. You create your own map and then say where you want to go. Suddenly, in this case Lufthansa says, “We can help you out. If that’s where you want to go, these are the special deals you can get.” By you sharing the airlines know your intent, your friends know your intent, and they can make you special deals.
Dopplr is a company that allows you to share your trips and it’s very powerful. If you happen to go to the same cities as some of your friends, a couple of times a year, you can be pretty sure they’re going to the same conference you’re going to.
Jet Blue airways uses the C2W world of Twitter, of giving people special deals. I was told that they even added flights based on peoples’ intent shown on Twitter.
My last slide here is a few questions for you. Who talks to whom? It’s ultimately consumers talking to consumers. Who trusts whom? We have seen the shift in trust from institutions to individuals. Who is in control? Business Week quotes me as having invented the word “me-business” a while ago. We’ve moved from e-business, where the company is in control to me-business or in other words, from CRM (Customer Relationship Management) to CMR (Customer Managed Relationships). Customers want to manage the relationships, not to be managed by the companies. Finally, who pays whom? It’s not that obvious. If you have a GPS device, and you leave it on so certain companies can improve their maps on it, shouldn’t those companies be paying you for helping them make their product better?
I want you to remember one thing from this talk. It is that this pyramid, what used to be on the top is actually turning upside down. It’s now the customer on top. You’re not helpless, because they’re sharing with you as a business, C2B. They’re sharing on platforms with their friends, C2C, and they’re sharing with the world, C2W. Gracias, thank you very much.

Audio: http://weigend.com/files/audio/Weigend_MEX_2009.07.01.mp3
Transcript: http://weigend.com/files/audio/Weigend_MEX_2009.07.01.doc

Check out this article at http://blogs.harvardbusiness.org/now-new-next/2009/05/the-social-data-revolution.html

You’re working on that big project when momentum stalls at 9:06 PM and you find yourself on Facebook staring at the news feed.  You are confronted by a stream of updates from that melodramatic train wreck of a former high school classmate, whose friend request you accepted out of guilt last week.  You couldn’t care less about Jenny or her life events, and yet you can’t stop yourself from mindlessly clicking on the “Britney Spears Circus Tour! Backstage – Rawk it, Diva!” photo album update, wasting 13 minutes of your precious time observing Jenny’s inelegant fall into adulthood.

The Problem

Sounds familiar?  Who hasn’t clicked on social networking content, even while fully expecting it to be irrelevant?  The problem is more than simple procrastination or self-control (although both may be there as well).

The bigger problem is that today’s preeminent communication and social networking technologies (Email, Facebook, Twitter) have declared defeat in the battle to deliver relevant information.  Instead of progress, they resorted to sorting on the easiest variable possible – time – leaving the challenge of determining relevance apparently to the reader, but actually to chance.

While flawed, this approach used to make sense.  Time is a respectable proxy for relevance when using a communication channel that delivers a mere trickle of content. But when the content volume burgeons, time becomes a glaringly insufficient stand-in for relevance.  Drawing parallels to other contexts makes this truth abundantly clear.  Imagine if Amazon’s recommendations simply showed you the items that had most recently arrived in the warehouse, and left it up to you to determine which were relevant to you.  Or, if Google showed you the most recently modified webpage as the first hits in search results.  When it comes to lots of options in shopping or search, time does not solve the relevancy problem.  Yet we seem surprisingly willing to accept it in the context of technologies that we deeply rely on to stay in touch with the world — despite being inundated by an ever-increasing, unmanageable volume of emails and status updates.

From the perspective of evolutionary psychology (listen to the conversation with Geoffrey Miller about his new book “Spent: Sex, Evolution and Consumer Behavior”), the willingness to accept this lack of relevance almost makes sense.  It is well known that we like more options than are actually good for us, i.e., than our feeble minds can handle.  It made a lot of sense when we (or our ancestors) still lived in caves and needed every nugget of information. But, alas, our minds haven’t evolved at a speed comparable to the growth of communication content, and we are reluctant to declare bankruptcy and give up control over our information flow. Most people seem unready to have a machine-learning relevance engine make the “mistake” of ranking an email so low they miss it. They are far less upset when insufficient attention and information management capabilities cause the same crucial piece of communication to go accidentally overlooked. We just like that illusion of control.

Increasing relevance maximizes the return on the recipient’s time and bounded attention by cutting down on the fully-loaded cost of communication, which includes both sender’s costs and the recipient’s costs.  The marginal costs for the sender are often reduced to the time it takes to reach an embryonic thought and the milli-joules of energy it takes to press the “Enter” key.  But other costs of communication don’t disappear from the system — especially those for the recipient, including direct reading costs, search costs, interruption and annoyance costs (which also harm the social capital or brand of the sender, though they are generally not priced in by the sender), self-control costs (try not to click on those aforementioned Facebook photos!), and last but definitely not least, the opportunity costs of more relevant content missed.

Toward Solutions

So… are we willing to relinquish control over our communication and outsource the work of determining relevance?  Assuming the answer is yes, this post will now proceed from boisterous damnation of the state of relevance towards solution ingredients.   Keeping in mind that any proposed solution must prove superior only to the status quo (and our biased appraisal of its efficacy), here are three ideas that might perform better than randomness (aka time):

1. Artificial intelligence on the receiver side.  This approach relies on what we can learn from past behavior and inferred preferences of users.  For example, Facebook would show you in your stream more posts from those friends whose prior posts you clicked on, whose profiles you view the most, who are most connected to you in the social graph.  The problem with this mythical panacea of machine learning is that it is much harder to achieve than it is to imagine.  Unlike Google’s PageRank or Amazon’s recommendations, the number of similar data points is often too small to allow for reliable conclusions about relevance.  For instance, computers struggle to ascertain the difference between the first email from that salesperson you definitely don’t want to hear from and that first email from your new date you are so excited about. Or, as another example, if you had the misfortune of clicking on Jenny’s Britney Spears concert photos, you will certainly be forced to see baby shower photos and lyrics from a Taylor Swift song tomorrow in your Facebook feed.
2. Artificial scarcity on the sender side.  A system that introduces scarcity can take many forms.  For instance, senders could be forced to pay for their messages (a draconian re-implementation of postage).   The Palo Alto based company Seriosity has attempted a more innovative spin on “paying” for messages.  Users who work at a company that uses Seriosity spend coins to make important emails appear as higher priority in recipients’ inboxes.  In order to create scarcity, senders are only given a limited number of coins per month (but how should those be distributed?).  Reputation systems can also introduce artificial scarcity.  Senders possess explicit reputations that are dynamically adjusted based on the quality and relevance of their messages.
3. Metadata.  One of the most powerful forms of metadata a sender can attach to a message is a prediction of the relevance of the message for the recipient. In the simplest case, senders attach their predictions of how relevant the recipient will find a given message. Unfortunately, this invites a world in which all senders attempt to shout the loudest to have their messages read, or at the very least over-estimate their own importance.  For it to work efficiently, the incentives of the senders need to be aligned with the interests of the recipients.  One way to align incentives is to make senders explicitly aware of the bounded attention spans of recipients and create a feedback loop.  Senders can get feedback on how relevant the recipient actually finds their message. How much time did they spend reading it, if any?  How important was it to the recipient, and how did this compare to what the sender predicted? This feedback gives them an incentive to adjust their behavior and internalize the costs they are pushing to recipients.

Creating this feedback loop is challenging for several reasons, including the violation of social norms it can involve (people may prefer willful ignorance of whether others find their content relevant and resent recipients who tell them the truth), but the potential benefits of an explicit attention economy warrant further experimentation with solutions.

Invest in Metadata

Even if creating a full feedback loop is challenging, much can be done with sender-generated metadata to help manage the chaos of communication.   When will we have a standard for metadata to communicate “If you look at only a single post (status update, picture, etc.) from me, this is the one!”?  Tags (like #socialdata on twitter), and other ways for senders to create metadata along with their messages to help recipients manage their information overload are hopefully forthcoming.   The effort involved in thoughtfully attaching metadata to one’s messages is a small investment up front to reap a larger benefit in the future: access to the recipient’s attention.

What do you think?

How does this post fit with you and your use of communication channels, both as sender and recipient?  As a sender, would you be willing to create more metadata with the messages you send?  What additional metadata would you like to create to help your recipients navigate their information overload?  As a recipient, how confident are you in your current ability to identify the relevant communication in your various inboxes and feeds? Which of the ideas mentioned do you like, which do you dislike for improving relevance?

And finally, are we missing any crucial ingredient to seriously improve relevance?  Do you understand the criteria that make communication content relevant to you, or is it still on the same level as pornography was for U.S. Supreme Court Justice Potter Stewart when he declared: “I can’t define it, but I know it when I see it”?

All Facebook links are created equal. But links can differ in strength—for example, a close friend versus a casual acquaintance. Links can be in different categories, like your boss versus a random hookup. And links can be asymmetric—Amy may think that Bob is a good friend, yet Bob may not trust Amy at all! The world is not a binary place.

Discovering Discovery: Don’t ask, Do tell

How can we use data to investigate these different properties of links? Today’s social networks do a lousy job of leveraging our existing data. Why do you need to manually confirm my friend request if we’re already calling, IM-ing, and emailing each other all the time? These data sources should be able to make a good guess about the strength and type of our relationship. Why not use existing data sources to propose better default responses?

If we give our networks a richer structure for our links and relationships, we will also be able to discover interesting facts about ourselves. Why is this important? By investigating implicit relations, we can gain insight into our relationships and how they work. For example, I might be surprised to find out that whenever I email my friend John, he always writes me back promptly whereas I always take 10 times longer to respond to him! Armed with this knowledge, I would ask my system to tell me to get my act together and crank out that response if I’m getting too delinquent.

Facebook 1.0 has helped us create an intimate network of our 17,000 friends. Will Facebook 2.0 help us manage them?

Mind the Explicit, Mine the Implicit

What else can data tell us about the quality of our relationships? One way to use data is to figure out differential interest in budding relationships. It’s easy to do this by looking at communications patterns in email, for example—does one person spend hours crafting that perfect email, only to get a reply that took only a few minutes to write? Or has he suddenly acquired a brand new set of favorite books, movies, and music that just happens to match his new love interest? People leave rich traces on the web—we can discover much more about them than the data they explicitly give.

This is only possible if we can look at the user’s history. After all, we can only make inferences about our behavior if we have a past to compare it against. But this introduces new questions: how much would you pay to know how long Monty spent writing you that email? How much would you pay to keep your data private?

Trust Networks

Social networks are also great for learning about trust. Let’s say that I’m thinking of entering in a business deal with you, but I don’t know you too well. Should I trust you?

There’s an easy way to use the power of networks to answer this question. Let’s just look at all of your other connections: do they trust you? We can give people reputation scores by allowing users to rate their interactions with friends. To make the system even more powerful, we could allow users to link their reputations. To illustrate: let’s say I trust my friend Mike so much that I am willing to attach a trust coefficient of 0.9. This implies that if Mike’s rating goes up by 1, I should get a rating boost of 0.9. Conversely, if someone has a bad experience with Mike and downgrades his rating by 1, my rating will also go down by 0.9. Through the power of the community, reputation ratings would spread quickly. (What trust coefficient would you attach to the author of this post?)

Reward Content Generation

One of the best ways to engage users is to get them to understand how every bit of data they contribute will end up benefiting them. In the example of trust networks, people can improve their own reputations by linking themselves with others. In my previous post on communication, I talked about a system where providing feedback on an email’s relevance would directly benefit you in the future. Online social networks need to reward people to provide explicit data, too.

The Facebook Feed was a brilliant idea for surfacing relevant content created by friends. Ideally, the Feed would create a positive feedback loop: good content provided by friends would get high ratings, which would motivate them to post even more good content. However, an early system of allowing users to rate the submissions of their friends was poorly designed—only 21% of users used the feature. On a rainy day, April 15, 2008, Facebook turned off the feedback system. What a step backward! I wish Facebook instead had created a better machine learning system to reward its users to generate and surface good content.

Social networks based on mutually confirmed binary relations was Day One in evolution of social networks. Introducing, richer semantics, more expressive structures including trust coefficients are the beginning of Day Two. What will the second week bring?