Skip to content

The Industry Review

One Guy's Thoughts On Technology, Social Media, Internet Marketing, Artificial Intelligence, and more


Category: Artificial Intelligence

The wisdom of the crowds


This post is based on the excellent session, “Crowdsource Your Success”, that was given in Affiliate Summit East 2010 though I expanded it and added my own perspective.

Crowdsourcing is becoming increasingly popular these days. According to Wikipedia, Crowdsourcing “is the act of outsourcing tasks, traditionally performed by an employee or contractor, to a large group of people or community (a crowd), through an open call“.


Using Crowdsourcing, you can submit a job description and get multiple bids each already satisfying the specifications you desire. Since you get multiple people trying to create what you want, the results are potentially diverse and can be surprisingly creative. Of course, this is usually more expensive than just using regular outsourcing sites such as oDesk or Elance – but the disadvantage of those is that no matter how good your contractor, you are ’stuck’ with a single design.

Crowdsourcing works because of the ‘wisdom of the crowds’ principle: the idea that a crowd – a collection of individuals – is much more likely to get the right answer than a single individual.

A good example is the game show “Who wants to be a millionaire?”. Asking the audience for the answer is more likely to result in the right answer than asking your friend or Regis Philbin.

This principle has been adopted by computer science as well (and probably other fields). In my academic career I used to create multiple artificial neural – instead of a single one – networks that solved a problem. The right solution was determined by taking the solution that the largest number of networks ‘voted’ would work best.

Here are some suggestions given at the talk to get the maximum from crowdsourcing:

  1. You reap what you sow: define your project properly or you may get something very different from what you had in mind.
  2. Tight deadlines are very effective as people like discovering quickly whether they’ve won a bid.
  3. Don’t be a jerk: Designers thrive on feedback, give feedback and recognition.
  4. The project has to require your involvement: a crowdsourcing project is not ’set and forget’.
  5. Keep it simple: be realistic in your expectations and ask for what is reasonable.
  6. Don’t be too cheap: most people aren’t going to be paid, so keep this in mind.
  7. Announce there will be multiple winners to boost designer participation (assuming that is the case!)

Until the talk in Affiliate Summit, I (naively) thought crowdsourcing is limited to graphic design/web design and 1-2 other types of applications. I was mainly familiar with 99design.

The following is a list of crowdsourcing resources given at the talk. I had no idea there were so many! When I’ve done a Google search I found even more though it’s hard to tell which are good. If you are familiar with anything that is not included and is a good resource, please let me know and I’ll add it.

Note that I’m still looking for a place to crowdsource copywriting (sadly, not my strength!) – so if you are familiar with a good site for that purpose, suggestions would be welcome.

Banner, landing page and graphic design
99designs: the most well known resource for Crowdsourcing.

Landing page optimization:
FiveSecondTest: allows you to get quick feedback on landing page designs.

  • Are your calls to action standing out? Get people clicking on hot spots
  • Can visitors understand what the site is about?
  • Give viewers a memory test: what can they remember about the landing page?

PPC management
Trada: Allows you to turn over PPC management to a group of AdWords qualified professionals.

Ad copy
BoostCTR: allows you to outsource your ad copy so that your CTR is boosted. Guaranteed improvement!

GeniusRocket: professional videos and animations.

Product development
Quirky: submit new ideas for products or influence products currently in production (and earn cash)

Feature Requests
UserVoice: a giant suggestion box. You get a lot of comments which are prioritized. Best ideas are voted to the top.


Software development
TopCoder: an excellent resource for software developers.

Find JV partners
Jigsaw: a massive crowsourced database of contact information

Content writing
Spudaroo: useful for web content as well as resumes, leases, etc.

Beta Testing
UserTesting: usability testing for your website.


Amazon Mturk: Although the Amazon Mechanical Turk is not exactly a crowdsourcing resource, by offering to pay a small amount for ideas, you can effectively crowdsource names. An example was given of a person who paid $27.50 to get name ideas for his iPhone app (the result was iReadFast). Note that there used to be a site (which I vaguely remember) that was used for this purpose but has been apparently closed.


 Mail this post

Popularity: 12% [?]

Post to Twitter Tweet This Post

Technorati Tags: , , , , , , , , , , , , , , , , , , , , , , , , , , ,

  • Share/Bookmark


An illustration of my evolved virtual hierarchical multicellular agent

This continues my previous post that linked to the first part of an interview about my Ph.D. work.

This part is much shorter: Ian and I just generally discuss related aspects of my work.

As I mentioned in my previous post, I’m hesitating whether to publish my own version of the interview. It has parts that don’t appear in Ian’s version (primarily explanations in layman’s terms of what I did as well as some elementary concepts that I mention), less technical stuff, and other things which simply did not belong in his interview. I think I will publish it, but do it at the same time I’m publishing another post, so readers will have something else to escape to in case it’s proving to be boring or too long. Based on the responses I got from my last post, I actually don’t think my readers found the post boring at all (somewhat surprising to me), but my version is long. So I won’t take any chances ;) .

Ian’s interview with me (Part 2)

Edit: Oh, forgot to mention. My post will have links to movies, some mine, so not, that could be interesting to watch and will help illustrate the points made. Not sure how I’ll link to mine and whether they’re worthy of viewing, but the other stuff certainly is.


 Mail this post

Popularity: 1% [?]

Post to Twitter Tweet This Post

Technorati Tags: , , , , ,

  • Share/Bookmark

Mosaic World

Mosaic World

A while ago I was approached by Ian Ma, a guy who found me through my blog, and was interested in interviewing me for a Machine Learning community about my Ph.D. work – a project called Mosaic World. I warned Ian: ask an Academic – even an ex-academic – about his work, and don’t be surprised if you’ll get a really long answer.

Since I’ve been asked by several people “really, what is it that you did during your studies?” I thought of combining both. Therefore, I wrote a (very) long post about what I’ve done, in layman’s terms, took bits of it for Ian’s interview and added some bits that were not in my original post (mainly technical aspects). I planned to publish it together with another post, preferably a humorous one, in case I scare some readers away.

After reading my answers, Ian decided to break up my interview to two parts and politely asked me if I could not to publish my post until he publishes both parts of the interview. I’m actually wondering whether I’ll post my version at all, since much of it is the same… perhaps I’ll just do it “for the record” so if anyone ever asks me what I’ve done, I’ll have somewhere to point him at. If I do that, I’ll also post some links to some online movies (some mine, some not) which I think are cool and will help convey the points I make.

Either way, if you’re interested in what I’ve done, my answers to Ian are not very technical (only slightly more than my original version). If there’s any chance that you won’t find this interesting, feel free to skip ahead.

Ian’s interview with me (Part I)

And now, I need to work on that humorous post ;) (which I’ve actually been planning to do for 2 weeks now!)


 Mail this post

Popularity: 1% [?]

Post to Twitter Tweet This Post

Technorati Tags: , , , , ,

  • Share/Bookmark
Get Adobe Flash playerPlugin by wordpress themes