On the Future of MetaFilter

Metahawks by Tau Zero

MetaFilter is the little weblog that could, established in 1999 as one of the first community blogs. Over its fifteen year history it has expanded from a place to discuss interesting things on the web to include Ask MetaFilter as a community question and answer (Q&A) site, along with more subsections for things like music by members, completed projects by members, meetups among members, and most recently TV and movies.

While MetaFilter is relatively small (only about 62,000 have paid the one-time $5 for an account to date and 12,000-15,000 of those members come back to interact with the site every day), we have a great group of members, and I think we consistently have some of the best discussion on the web, with the sites attracting over 80 million readers last year. Our commenters are literate and thoughtful, and our site is watched around the clock by a staff of moderators. Despite the site’s modest stature its influence makes waves in the larger world (like mentions on popular TV shows: Tremé and Mythbusters).

Unfortunately in the last couple years we have seen our Google ranking fall precipitously for unexplained reasons, and the corresponding drop in ad revenue means that the future of the site has come into question.

Peak Ads?

The sum total of MetaFilter’s current financial predicament can be seen in this image of MetaFilter’s total ad revenue for the last 9 years:

There’s a study going around called Peak Ads, which describes a hypothesis that perhaps advertising already reached its peak and people are harder to reach these days as they are prone to ignoring ads. MetaFilter definitely saw a peak in revenue in late 2011/early 2012 and has seen a swift decline since. Today in May of 2014, we’re seeing revenue at levels we haven’t seen since 2007. There is a staff of eight people at MetaFilter these days, and back in 2007 it was just three. The situation is bad enough that we’re now having to staff down just to stay solvent. This is the story of how that came to be.

Live by the Google sword, die by the Google sword

The economics of MetaFilter are strange to say the least. For the first five or six years of the site, most user activity was at www.metafilter.com, the main blog and discussion part of the site that acts as a sort of water cooler for the web. Sometime around 2006, Ask MetaFilter began to eclipse the main site’s traffic, as Google search results started to increasingly lead people searching for help to that section of the site. In terms of audience, every month 6 million people look at 10 million question pages at Ask MetaFilter. Most of those people only visit one page, find their answer, then leave. The main MetaFilter blog on the other hand only sees about half a million people but they look at nearly 6 million pages themselves.

In terms of ad revenue, somewhere around 90% of the revenue comes from Google Adsense, on Ask MetaFilter, with the rest of the sites getting a smattering of smaller amounts. The big spikes in the graph relate to when advertising sizes and positions were changed, when MetaFilter was added to an Adsense beta that offered increased customization, and whenever Google updated its own index.

For over 13 years, traffic steadily grew across all MetaFilter sites, with our Q&A site serving as the main driver of traffic thanks to it being a solid resource for advice and information, available to all. As revenue grew, I was able to hire people to help keep the site spam-free and the quality of information high. During the biggest boom we got up to a staff of eight, five of whom were full-time employees receiving full health, dental, and eyecare paid for by the company along with a 401k retirement program with company matching funds as well as the occasional holiday bonus. Thanks to the excellent scheduling work of Jessamyn, we work sane hours with rotating shifts to prevent burn-out. The quality of discussion and advice on our site is high because we have a great community and thoughtful moderation.

The money situation changed one day in November 2012, when I saw a drastic reduction in traffic and revenue to Ask MetaFilter. I double-checked to make sure the initial estimates were correct, and it appeared that Ask MetaFilter lost 40% of its traffic overnight. I read up on all the various (somewhat dicey) SEO forums and it seems MetaFilter crossed a line in one of the many “Panda” updates Google was doing to its index. Advice online was to scale back the “top heavy” ads and to scale back whatever SEO things were were doing (we were doing nothing in terms of SEO, as I find the whole business kind of gross). I removed most ads to the absolute minimum of just 1 or 2 per page (that’s what the really low trough in the graph above represents) while running the business off of savings in the hopes Google would see fit to roll back our penalties in a future index update.

As Winter 2012 became Spring 2013, traffic remained flat and we all took big pay cuts to make ends meet. Google sunsetted their beta program MetaFilter was in and we went back to the standard Google Adsense ads which did pretty well and revenue improved a bit. Over the course of 2013, a series of messages from the Adsense team hit me with varying degrees of severity. We were temporarily banned from the system due to some text questions talking about sexual health (questions from users that include terms for body parts etc., but Google interprets that as the site being “adult”) and had to greatly beef up our ad display blocking by subject matter. Late last year, I was told that despite the past decade of Google’s Adsense pages suggesting ads should match your site, different background colors were now required to better discern ads from content, resulting in another large decrease.

For the last year and a half, MetaFilter’s revenues have continued to decrease and traffic has slipped a bit as well. Additionally, mobile web traffic has grown substantially (especially at certain times: nights and weekends we see 60-70% of all traffic on mobile/tablet) and ad performance on mobile is much less effective, where mobile pages only make about 1/3 to 1/10th as much as a desktop page. On average, every 3-6 months for the past year and a half we’ve seen additional ~20% drop-offs in traffic and revenue, and that’s been a challenge to deal with.

This is a Google Analytics graph of traffic on Ask MetaFilter from 2011-2014 showing the late 2012 reduction (that one big spike was a recent thread going viral for a day). We never really recovered and have been slipping since.

MetaFilter as SEO spamfarm? Huh?

Most recently, we’ve been getting several emails every morning asking us to remove links to sites that were mentioned on MetaFilter, every big web publisher gets them now. The text of these emails is slightly distressing, but here’s an actual message Google sent to someone, who in turn contacted us pleading for removal:

We’ve reviewed the links to your site and we still believe that some of them are outside our quality guidelines.

Sample URLs:

http://ask.metafilter.com/194610/get-me-and-my-stuff-from-point-a-to-point-b-possibly-via-point-c

Please correct or remove all inorganic links, not limited to the samples provided above. This may involve contacting webmasters of the sites with the inorganic links on them.

Google calling links found on MetaFilter “inorganic” is troubling. We have a staff of six full-time moderators in five timezones throughout the world (two are in Europe) to make sure zero spam ends up on the site. We have a variety of internal tools that help us track all spam down: views of all activity by new users that contain links and lists of comments added to old questions that had a link in them (two patterns we found comment spammers trying in the past). Moderators scan every new user of the site looking for telltale signs of a possible spammer and we have ways of marking potentially problematic accounts behind-the-scenes that give us additional view of their activity on the site later on. Those views are checked every few hours and when we occasionally find people posting comment spam, we remove it and ban the accounts immediately. We also have a robust flagging system used by members of the site so they can alert us instantly when spotted and we take the same actions on spammy additions. We have a total of over ten million comments across on all our sites combined and we spend so much time and energy tracking the few problem comments down that I would be hard-pressed to find even a single public comment that could be considered comment spam.

Every time I investigate these “unnatural link” claims, I find a comment by a longtime member of MetaFilter in good standing trying to help someone out, usually trying to identify something on Ask MetaFilter. In the course of explaining things, they’ll often do a search for examples of what they’re describing and include those for people asking a question. Whatever was #1 in Google for “crawlspace vent covers” in a question of “How to reduce heating costs in the Winter?” might show up, and now years later, the owners of sites that actively gamed Google to get that #1 spot at the time are trying to clean up their act but unfortunately I have a feeling MetaFilter is suffering as collateral damage in the process.

Since we’ve never seen a return to our pre-Fall 2012 traffic levels, I have to assume whatever hidden law we broke we’re still breaking, or that Google sees us as a home for comment spam even though we boot every single one we can find though a series of sophisticated methods, and the whole experience has been frustrating to say the least. At this point, I’m at wits end trying to figure out why our high-quality site, featuring good advice from a dedicated community of real people with a best-in-industry 24-hour moderation staff has seen such big decreases.

On the flip side, I’ll accept that MetaFilter is from “two or three Internets ago”, and perhaps this is Google’s way of saying they’re changing with the times and we’re not. I’m ok with that too, but since Google is a giant black box to outsiders, we’ll never really know.

The future of MetaFilter and beyond

We’ve cut back as much as we can over the last 18 months, we’ve changed web hosts, going from hardware in racks to cloud hosting (cutting our bills by 80%). We’ve all taken pay cuts.

But at this point with recent shifts in search traffic and ad revenue MetaFilter is now losing several thousand dollars per month and if nothing were to change, MetaFilter would be sunk by the middle of summer when our already depleted savings run completely dry.

What other options?

I started looking to move away from reliance on advertising revenue and Google in particular, but unfortunately Google has a very good product in Adsense (and DoubleClick which they bought, and Adwords which they also do) and other ad networks could only offer 1/2 to 1/4 of the rates we find with Google’s Adsense.

We even explored selling MetaFilter with the goal being someone else running the difficult financial/advertising aspects of the site while the staff remained to run the day-to-day operations of the community. It’s a lot to ask of a potential acquirer that we sell but change things as little as possible, and after months of negotiations with potential partners none proved successful.

MLKSHK and App.net

My favorite image sharing site MLKSHK recently announced a planned shutdown (done in a nice way with exports and lots of explanation and lead time) and as soon as I saw the news I knew I didn’t want to find myself writing the same thing anytime soon. Despite facing impending financial doom on this fifteen year old site, I’m not quite ready to throw in the towel.

App.net, the API-platform-but-everyone-considers-a-Twitter-competitor announced it was scaling back all full-time employees from the project but hoping to run it “normally for an indefinite period.” While that sounds like a difficult spot they’re in, I think that strikes a better chord for how I’m thinking about MetaFilter.

Tightening the belt further

MetaFilter is frequently mentioned as one of the best places to find high quality conversations on the Internet, and it got that way through having a large dedicated moderation staff. MetaFilter is the most highly moderated site of its size on the Internet. I know this because I’ve talked to people working at all the larger, more famous sites and they most often have just a single or small number of people working part or full time as moderators, with the bulk of moderation work handled either by volunteers and users themselves, or through software automation of one form or another.

In order to cut costs further, I announced our first ever lay-offs. At the end of May 2014, I’m cutting half our moderation staff to make up for revenue shortfalls. Starting June 1st, we’ll continue operating as normal, but with less staff with more of my time will be devoted to day to day moderation on the site.

Overall, moderation conforms to an 80-20 rule where with the right approach and moderate effort you can get a very good community site, but going the extra mile takes multiples of effort to make small gains. I feel in the past we were very close to getting the best possible community site we could because we had so many resources available to us, and as we scale back the goal is to maintain that high quality as much as possible with the hope being the overall decrease won’t be so much as to drastically change the community.

Finally, a bit of barn raising

When we announced the layoffs, I downplayed the idea of Metafilter members giving us money, because I had doubts about whether that would have any practical effect on our situation. But then an amazing thing happened: people independently dug up a largely-forgotten donation link buried on the site’s about page, and even figured out how to set up monthly subscription payments to MetaFilter using the PayPal’s crude tools. I previously estimated a small population of maybe a couple hundred people might pony up a buck or two each month, which wasn’t really enough to change our situation. Much to my surprise, several hundred people have already given money as one-time contributions and/or set up monthly subscriptions, with the average contribution at nearly $10. The dream of maybe replacing some of our ad revenue with member support is looking like it could actually happen. So, grateful for the spontaneous outpouring of support, we’ve built up a formal fund-the-site page here:

Click here if you’d like to help fund MetaFilter

Conclusion

I’m sad to have to cut staff positions, but I believe the members are what make a community great and I am looking forward to MetaFilter’s sustained presence online. We may have to make hard choices in the future as the revenue landscape continues to change, but the goal will always be towards having the best place online to have conversations, share interesting things on the web, and offer advice to one another.

Finally, I thank everyone that used the site over the last 15 years, the great dedicated staff, as well as those helping fund it today. MetaFilter couldn’t be what it is without all of you.