Introducing Our New Awesome Search! (And The Story Behind It)

Posted on Jul 27, 2012 - 1:18 PM PST

I’m Bob Hiler. I’ve been helping Andrew with Mixergy for over a year. And search has been a problem for that entire time.

Nobody can find anything. And we have the email complaints to prove it.

Site Search: The Problem That’s Surprisingly Hard to Solve

The worst part is that everybody thinks improving Mixergy’s search is so easy. It’s frustrating when you’re struggling with a problem, and most people don’t even think it’s that hard of a problem.

We get a fair amount of emails like this:

Can you make search work on your site!? I just want a search box and a button. Why complicate things?

Happily, with the help of Swiftype, we finally think we’ve licked the search problem on Mixergy! Please kick the tires on our new search box in the sidebar to the right.

Start typing a search… Do you see the auto-suggestions? :-)

Our Journey To Fix Search On Mixergy

I was originally planning to end this post here. I also thought about writing a long list of reasons why fixing search on Mixergy has been such a hard problem, and then explaining how our new solution fixes those issues.

But that’s not what Mixergy’s about. Mixergy is about folks telling the world the real story of what happened.

So here’s the real story. I’ll start at the beginning.

Attempted Solution #1: WordPress Search

When Mixergy first launched, the site used WordPress’ native search function.

It’s not exactly WordPress’ fault, but WordPress’ native search functionality falls far short of the gold standard set by Google. In particular, Google lists search results by relevance. So the most relevant posts appear first on Google.

In contrast, WordPress’ native search lists search results from most recent to oldest. So if the most recent post mentions “facebook” once in passing, that post will appear first when a user searches for “facebook”.

Sadly, Google has spoiled users to expect the most relevant posts first… So Andrew got a lot of email complaints about how Mixergy’s WordPress search sucked.

Attempted Solution #2: Site Search by Lijit

Then Mixergy added Lijit’s site search in July 2010.

One cool extra is that Lijit automagically creates a demographic profile of your users, similar to how Quantcast. Apparently, 54% of people reading this are male, 52% are 18-34, 98.55% are Caucasian (?!), 77% have no kids, 35% make more than $100k, and 54% haven't finished college.

Lijit is very similar to a Google Custom Search Engine. You set up an account, add the search box to your site, and voila! People can search your site! Both sites make money by adding ads to your search results.

Unfortunately, Lijit only searches public posts. And on February 15, 2011, Andrew started publishing Mixergy Premium Courses that were only available to logged-in Mixergy Premium members. Andrew didn’t want these new Courses to be indexed by Lijit (or Google). So the love affair with Lijit was about to end.

Attempted Solution #3: “Search Everything” Search for Premium Courses

Right around this time, I got to know Andrew and started helping out. Once Andrew started publishing an unsearchable Course every week, Mixergy faced an ever-growing search problem. We started getting an increasing number of complaints from frustrated searchers.

In January 2012, Andrew forwarded me an email from a reader suggesting we use a plugin to add search for Premium courses (thanks Mo Alkady!). Andrew asked me:

Bob, can we try testing this plugin for searching premium courses only?

I was thinking we’d use it to search the courses only.

I ended up using this WordPress plugin, which let us search on the posts in the “Course” category.

Me versus Our Hydra Problem

This solution helped, but like fighting a mythical Hydra, chopping off the head of one of our problems resulted in two new problems.

First, users just couldn’t understand that this search box only searched Premium Courses. We tried every UX hack we could think of. We only put the “premium course search” text box on the Mixergy Premium page. We also made the text box say “Search Premium Courses Only.”

But nothing we did worked. Google has apparently trained the world to see a search box and think “oh, that’s a magic box that will help me find anything I want.” And no UX tweaks we did could overcome that expectation in the user’s mind.

The second problem is that WordPress and its MySQL database suck at searches. Specifically the MySQL database doesn’t have an “index” for WordPress blog posts. So every time somebody searched on Mixergy’s premium courses, the server would have to scan the entire database. The server couldn’t just look in an index.

So even occasional searches of Mixergy’s premium courses would tax the server. And searches were very slow.

I’m sure somebody smarter than me could solve this problem by adding the right “index” to our database. But it’s not easy to get a database to return Google-quality results when people search for keywords.

So we just limped along. We endured the occasional server slowdowns caused by these searches. And we just hoped that any frustrated users would email us to ask for help.

Attempted Solution #4: Indexing Mixergy in OUR BRAINS!

Believe me, plenty of people have emailed us about their search problems!

Sometimes, the search is pretty oblique, like this one:

[Can you please help me find an] interview with a UK inventor who licenses his products ideas. He was the guy which I think created his own currency (something like that) and licensed gadget/gimmick products out.

(By the way, the answer is Who is Sheridan Simove?. And it’s the #1 result for “own currency”. :-))

Our solution to answer these questions was to leverage the amazing powers of the human brain! And not just one brain: Andrew, Arie and Andrea here at Mixergy have essentially memorized all the interviews and courses, and use their brain as an index. To their credit, they have promptly responded to each and every one of those “please help me find this” questions.

Looking back at our sent emails makes me wince at how much time this took. Here’s Andrew responding to one question: “It took me 30 minutes, but I found it!”

So painful. We had to do something…

Attempted Solution #5: Search by Interviewee Name and Company

The only problem was that improving search seemed impossible. We had tried a bunch of hacks for a year and hadn’t achieved a breakthrough.

At this point, I had no hope that we could ever have a magical search box That Just Works. It just seemed too hard and beyond my paltry skills. So I thought about how we might lower our standards to achieve something less than a breakthrough. I am not a proud man.

Andrew had mentioned that some users just wanted to see a searchable directory of interviews and courses that showed (1) the name of the interviewee and (2) the interviewee’s company.

Maybe just doing that would be enough? So I tried to figure out how to achieve that lowered goal. I added two new Custom Fields to a bunch of posts that specified who was interviewed, and what company they worked at. I then wrote code that searched on these personal and company names. It was pretty slick!

You can see the "Try your search on Google" link in this screenshot (click to enlarge).

But Andrew then came back at me with a valid point: what if users searched on a topic? So I tried another UX hack. At the bottom of each search results page, I added a link “Retry this search on Google”. I figured if there were no results, people would see the link and try the search again.

But like all the other UX hacks, it wasn’t enough. Andrew and I both thought this search experience sucked, because it didn’t search Premium Courses. And this solution would require somebody to manually add guest and company names to over 900 Mixergy Interviews, Courses, and Cheat Sheets.

Who wants to work hard to achieve mediocrity?

Searching For A Breakthrough

I realized that all our dirty UX hacks weren’t cutting it.

But that implied that the only way we could achieve a breakthrough was by actually creating a single search box that automagically searched all Interviews, Courses, and Cheat Sheets on Mixergy on any arbitrary keyword, all without overloading our servers. That seemed impossible!

But I got to thinking. I’d never actually tried to achieve the impossible. I had only tried to hack my way to mediocrity.

It was then that I remembered a post I’d read on Hacker News linking to a TechCrunch article about a new search startup. The name of the post was Swiftype (YC W12) Builds Site Search That Doesn’t Suck.

That sounded promising!

Attempted Solution #6: Swiftype Search

It was Friday night and I was tired. But I was desperate enough to sign up for Swiftype before I stopped working (it’s free during their public beta).

I found Swiftype’s WordPress plugin. I installed it and pressed a button that “synced” all our posts and pages into Swiftype.

I had a basic search up and running within five minutes! And even out of the box, Swiftype (like Google) ordered search results in order of “relevance”. And because the heavy lifting for the search occured on Swiftype’s servers, our server didn’t overheat.

I was also happily surprised to get a personal email from a Swiftype co-founder that very night:

Hi Bob,

Just wanted to drop you a line as I noticed you had checked out Swiftype for Mixergy. We are big fans of the content you guys produce. In fact, I just listened to the interview with Emmett earlier today. We would be really excited to have you on board, so please let us know if there is anything we can do to help!

Best,
Quin Hoxie
Founder – Swiftype

I played around Swiftype over that weekend. I had always been frustrated that I couldn’t “theme” Lijit or a Google Custom Search Engine search results. In particular, I wanted to add an icon to each search listing to indicate whether it was a free Interview or Cheat Sheet, or a premium Course. Swiftype lets you use WordPress to style the results however you want, which blew me away.

Over the last two weeks, I also worked with Quin to add a few features. In particular, I wanted to be able to filter search results by category, so users could see only Interviews, only Courses or only Cheat Sheets. This would help them find things like “that course about ad buying”. You can also click on the “sort by date” button to see the most recent results instead of the most relevant results.

Finally, because of Swiftype’s “sync” feature, the private content of Mixergy’s Premium Courses was searchable, without us having to expose that content for the world to see.

Success!

You can check out some sample searches here: facebook, buying ads, lean startup, and seo. Or just search from the sidebar.

I’m excited! We have actually created a magic box that will help people find anything they want!

And that’s the story!

If this post was a Cheat Sheet, the first “tactic” would be that if you want search that doesn’t suck, you’d be a fool not to sign up for Swiftype.

But hopefully, you also enjoyed hearing about our long journey as we tried to cure the pain that we and our customers felt. :-)

  • http://www.JiansNet.com/ Jian

    Interesting…So definitely agree that search is one area I also find frustrating all the time. One way is to just search using Google by doing a “site:” command.

    Because search is a frustrating experience, I am also passionate about search and have been always thinking how to make search better on http://www.JiansNet.com

    On drop down suggestions, I think it is a great improvement. People sometimes don’t know what to search for, so with suggestions, it is easier to give people hint on what to use for keywords. I’ve implemented that also on JiansNet and find it super helpful. Although I need to make it more friendly for both English and Chinese text too.

  • http://mixergy.com Andrew Warner

    That’s been the standard way to search the site, but there’ve been a bunch of issues with that, including:

    - posts announcing upcoming guests, show up higher than those for the actual interview
    - there’s no way to say, “I only want interviews,” in Google’s results – there are no useful auto-suggestions

  • http://www.JiansNet.com/ Jian

    I see the problem here. Typically I just use Google to search mixergy past interviews and could find it after looking through the results. But, you are right, with more content on the site, it could be hard to go through with a vanilla search  like Google does.

  • Anonymous

    This looks like a great search. However, the basic problem stays: if I’m on this website, most likely I’m a new entrepreneur or I’ll soon be or I have a business but not time to randomly search. Now what happens when you “don’t know what you want”…. but if I see keywords, summaries, subjects like “How to raise funds” THEN I’ll be interested. 

    Right now, the script seems to be mostly looking in the headline, so if I search “money” it returns me stuff with “money” in the headline… this is not useful at all. A headline that doesn’t have money in the headline could be talking the whole time about it but be named “how to raise capital”. You may also find a talk or someone you know has come here, be then you can just go in google and type “Nick Oneil mixergy” and you’ll find him anyway.

    The point is I think the best way to find what users want is to show them what exist in the database, using tags/keywords manually created + adding summaries about the talk with all the different subjects (4 to 10 items in most talks). 

    Add on top of that the ability to take personal notes of flag to remember a certain part of an interview and this would be an amazing tool for entrepreneurs.

  • Anonymous

    Actually I really like the search because I only use the search when I want to see an interview AGAIN. Since I am a member I do watch all the interviews anyways but when I want to use some of the stuff I learned I want to be able to find it again and I usually do remember some important detail about the course or interview, so it works for me.

    However, maybe there is a way to integrate the transcripts into the search so that you have access directly to the whole interview and its content. I am not sure if the search already does that and if not, whether it is possible.

    In any case I enjoyed you describing the process of searching for a solution and I now understand why I had filled out those surveys.

    Keep it up.

  • Bob Hiler

    Thanks for filling out those surveys! That was definitely a huge source of insights… :-)

    And yes, transcripts are definitely indexed and searchable.

    Actually, the last step of this search project was for Arie and Andrea to add a transcript to every single course! It was a lot of work, but this dramatically improve the searchability of courses!

    Thanks Arie and Andrea!

  • Bob Hiler

    FYI, the search algorithm currently weighs keywords in titles twice as much as keywords in the body. This is an adjustable parameter, but 2x seemed to be a good balance in our tests. Do you think the weight should be lower?

    Also, it sounds like we’re on the same page about the importance of some way to browse interviews/courses about important topics like “raising money”. We actually looked at using tags/categories for this, but there are already over 900 posts! So that would mean we’d have to go back and categorize 900+ posts in up to 50 subcategories. That would be a huge endeavor.

    So now we’re currently trying to create a browsable directory *without* using tags/categories. We have another idea we’re exploring to do this. Hopefully, we’ll have an update about that soon! :-)

  • Anonymous

    Hi Bob, thanks for the reply and the precision. 2x should be great for what it does, the results I saw were good for my keywords.

    Amazingly, tagging could be much much faster and easier than you think. Refer to this long text if you have time (http://www.fastcompany.com/magazine/134/boy-wonder.html). Basically, you could >>ask the mixergy fans<< to do it for you, and you double check after. In other words, you tags all the new one added each week, but send a request for help to the fans to help tags the interviews. You must tag 2-3 interviews first as example and make a small checklist to help them in how you want this done. Then you give them a list from most popular to least popular (like the one I made here : http://drinkableknowledge.com/casestudies/mixergy/mixergy-interviews-by-social-popularity.php) and you let them do that for you. Amazingly, you have a very strong community of people that are eager to help mixergy, so my guess is that there are people that would gladly help for this. Afraid of quality? So were the people opposed to Wikipedia when the idea was first introduced. Crowdsourcing an encyclopedia? This is crazy! It’s gonna suck! But know, amazingly, someone who has the time and will to do something will do it right because they care, and then the more people add, modify, correct stuff, the higher the quality it becomes.

    Good luck!

  • Bob Hiler

     Thanks!

    And I agree with you. I’m not worried about the *quality* of crowdsourcing tags/categories. I just don’t want to write the code that does that. :-)

  • Anonymous

    …maybe… ask 20 mixergy users to join to help. Then you create a quick Google Doc that you share to those 20 people and ask them to do the first 50 most popular interviews. (again, with guides to help them). As a reward, you can give a little something, or just thanks. 

  • http://sherlockdocs.com/ Paul

    Thanks – a huge improvement. I remember Arie spending a week to find an interview for me one time and with the new search I found it right away

  • Bob Hiler

    I’m glad it’s working for you, Paul. :-)

  • SesSean

    OK, I have to say it works. I was looking for a wordpress hosting provider but could not remember the name. Before this when I typed, I was getting zippykid but what I was looking for was WP engine. Cool, good job by swiftype. So, when is Andrew going to bring them on.

  • Bob Hiler

    Awesome, I’m glad you found what you were looking for. :-)

  • -IH

    Thanks Bob. This is super-helpful for anyone looking to implement search on their own sites. Now we can skip the headaches and know potential pitfalls of other solutions. 

    And btw, seems to be working well for me!

  • Bob Hiler

    Sweet, glad to hear it’s working well for you! :-)

    To clarify something, you could probably use a Google Custom Search Engine if all your URLs are public, you don’t have any structured metadata “attached” to your pages, and you don’t mind ads showing on your search pages that might link to competitors.

    Where Swiftype shines is in a complex search scenario like the one Mixergy faces. Not all of our URLs are public. We have structured metadata attached to our posts (e.g. whether it’s an interview, course, or cheat sheet). And we’d prefer not to have ads on our search pages (although that’s not a dealbreaker).

  • -IH

    Yep, I was wondering about exactly that actually — Google Site Search, which seems to work well for most people. In light of needing to search private URLs though, makes sense why it wouldn’t work.

    Thank you Swiftype for solving a real customer problem! Alot of us entrepreneurs here have (or are building) membership sites, so this is a golden nugget.

  • http://www.facebook.com/Vitiell0 Daniel Vitiello

    Just wanted to say thanks to you guys for truley being commited to making a good product. I can imagine how frustrating that must have been to try and find a solution that most people think is so simple for over a year. 

    You guys are true legends for actually replying and helping all of those people find the interviews that they were looking for in the mean time. 

    Im about to try it out right now! Let’s see if I can find some of those incredible copywriting tips, by Dane Maxwell right?

  • http://mixergy.com Andrew Warner

    Thanks.

    Let me know if the search helped you find the copywriting program.

    I just tested it and it seems to work.

New Here? Start With These