How you can use social machinery to power personalized news delivery

People have spent the past five years in Social Networking 101. We figured out what social networks are, created our profiles and connected with friends. We rethought fundamental aspects of human relationships and adopted new ways of informing ourselves (“If the news is that important, it will find me.”) We learned new ways to stage revolutions and to follow them from afar.

In the process of adding friends, following people and retweeting things, we have created a mosaic of what we like, which can be used to train Web services. We don’t think about it, but it’s there. And it’s all out there.

If you haven’t already, take a couple of minutes to try out Intel’s Museum of Me. When you log in with Facebook, it creates a stunning video tour of a futuristic museum about your life and friends.

Then visit LinkedIn’s Connection Timeline to visualize your career and the people you’ve met over time.

Bitly.tv shows the Web’s top videos, based on clicks through bit.ly links on social networks.

Now take a peek at Tweetmeme or Bitly.tv to see what Web pages and videos are trending, or log in to The Tweeted Times to build a personalized news product from links shared by your friends or their friends.

You could find any of those services via Google, which tells you which search results have been shared by your friends, or Bing, where pages your friends have “liked” rank higher.

What you’re seeing in these services and many more are early stages of a new layer spreading across the Web — the social layer. It’s becoming key to how online content companies deliver information that increasingly flows through Twitter and Facebook.

The social layer of the Web is the next phase. It uses our data and social graphs as machinery to power new services that have nothing to do with updating your status, “liking” or retweeting. It’s just the Web, transformed into your Web.

Betaworks CEO John Borthwick

“Social is really changing the nature of the Web,” said John Borthwick, CEO of realtime Web company Betaworks. “We believed it would change the way people found things, but … the social exhaust of data is actually changing websites.”

For the news business and any other that aims to provide relevant information to online communities, the consequences are profound. Readers will become increasingly dissatisfied with news sites and apps that show them the same stories everyone else sees.

How the social infrastructure affects news delivery

The social layer doesn’t replace social networks, of course. We’ll still continue to use Facebook, Twitter, and the things that come after them to keep up with friends, find new ones and discover information. The social layer is built on top of the networks, made possible by the fact that the networks themselves are no longer a novelty.

In 2010 about 6 of 10 Internet-connected Americans used a social network, almost double the number in 2008. This social layer will grow deeper as social networks become more universal.

So far, services tapping into the social layer take a few different approaches. Some transform your social network activity into something quite different (Museum of Me). Others use broadly aggregated social sharing data to identify trends, not specific to an individual (Tweetmeme).

But the biggest advances are coming from services that use data about your friends to power personalized curation.

The News.me iPad app, a Betaworks-funded social news service built on top of Twitter, creates a stream of news being read by people you follow. Betaworks is funding another new product called Findings that will create a social layer around Kindle ebook reading, Borthwick said.

Another news aggregation iPad app, Zite, takes personalization a step further.

Zite knows the topics I’m interested in based on my Twitter data.

When you log in with Twitter, Zite analyzes your friends and activity to recognize the topics you care about so it can deliver top news in those subjects. But it doesn’t stop there. Zite goes beyond your own social graph, recommending items noted by users whom you don’t follow but who seem to share your tastes.

“It’s really about expanding from your social graph into a more-expansive social graph,” Zite CEO Mark Johnson told me.

How to break into the social layer

So what can a news organization do to use the social machinery to power news delivery?

“There are no really good turnkey solutions for this stuff out there right now,” Johnson observed. Content-management systems don’t include social personalization technology yet. News organizations are going to have to build their own tools for now. That’s challenging, but there are some entry-level options.

First, the Facebook Activity Feed plugin can be added to your site in minutes. It will show your readers what stories their Facebook friends have shared on your site.

Twitter, unfortunately, does not offer such a plugin. But if you ask users to sign in with their Twitter credentials, you could use the Twitter API to comb their timeline for any tweets linking to pages on your site, and highlight them in a widget.

Also possible, though more complex, would be to track what a user reads and shares on your site over time, learn the subjects that interest them, and begin to recommend other content based on that. The New York Times has a system like this that recommends stories based on what you have read on the site in the past 30 days.

The Huffington Post tells me what is popular among my friends and suggests stories I might like based on my social data.

These types of systems require a robust registration engine capable of connecting to network APIs and storing a history for each user, and it may require adding more metadata about topics and categories to each page. But those are good, long-term investments that will prove more useful as the social layer spreads.

A more advanced possibility is to analyze a user’s Twitter or Facebook data — not for specific links to your site, but to determine their general interests (like Zite does) — and show them news in those categories.

The Huffington Post is taking a shot at this with its “Stories You Might Like” feature, which Social Media Editor Rob Fishman introduced earlier this year:

“Using the articles and interests that you’ve made public on your Facebook profile — both from within Facebook and across the Web — we’ve developed a recommendation engine to help point you towards HuffPost stories tailored to your personal tastes.

“If, like me, you watch Jon Stewart every night, we’ll serve you up the newest story about the comedian. A reader of The New York Times? Here’s the latest. Where it gets interesting, though, is when the system starts to anticipate your interests. Hence, I may not be a fan of Madonna on Facebook, but because I’m a fan of similar artists, I get the latest on the Queen of Pop.”

To brainstorm ways to tap into the social layer for your news organization, start thinking about how you could mash up your news with data available on social networks about your readers:

• Who are they? (Birthday/age, gender, hometown, school, workplace)
• What do they like? (Interests, specific pages or people)
• Where have they been? (Check-ins on FB places or Foursquare)
• What links have they shared?

With access to that data, imagine what you can do to offer new news products.

Challenges in building the social layer

These personalization efforts face at least a few big challenges. One of them is that you don’t share all the interests of all your friends. Some people are in your network primarily because they are family, high school classmates or coworkers.

To really succeed, a personalization service will have to figure out which friends are the strongest signal of your interests.

Another problem: Social networking companies could resist if they don’t like how their services are being used. All these services depend upon companies like Twitter and Facebook providing access through an API, which they can (and do) change.

Partly for that reason, the magazine-style app Flipboard has kept the social networks themselves “front and center,” CEO Mike McCue told me in a recent phone interview.

Flipboard reformats a user’s Facebook or Twitter streams into a visual, touchable interface, but it maintains the chronological order of network activity. It doesn’t use your friends’ status updates and tweets to try to deliver you something else.

Flipboard CEO Mike McCue

“Facebook and Twitter won’t be too psyched about using them as signals. You have to think about the whole ecosystem here and who’s going to benefit and how are you helping them,” McCue said. “Ultimately, those social networks will not be happy about that model, and I don’t think that model is really durable.”

It’s also possible for a news product to go too far in personalizing the news.

“If you make these social signals too strong and make these feedback loops too strong, you just get everyone reading the same stuff,” Borthwick told me. “You’ve got to find a balance there.”

For that reason, News.me has tried to tune its recommendation algorithm to show you what’s popular now, but also make room for serendipity, he said.

Finally, privacy will certainly be a concern. The better any service gets at understanding people’s interests and habits, the more likely it is that some people will find it invasive.

But most people tend to be pretty open online about their interests, Zite CEO Johnson told me. And if you are clear upfront about how you will and won’t use their data to personalize the news, you may be able to avoid trouble.

Privacy is something to worry about when developing a personalized information product, Johnson said, but “this is probably one of the least creepy ways you can use the social Web data right now.”

Correction: The original version of this post said The New York Times bases its story recommendations on content that a user has read in the past three days, but it actually uses the past 30 days.