My preliminary findings: Google AppEngine is really cool (obviously), and using Python again after almost OD-ing on PHP was very pleasant. I’m still learning to properly use the datastore though, setting a multi-column primary key to guarantee unique (date, currency) records wasn’t very straightforward. Also the import of the historical data was a bit of a hassle with the import script timing out, until I found how the BulkLoader can automatically do this in multiple HTTP requests. Finally, getting this to run on my own domain, currencies.apps.grandtrunk.net, took some time until I found out the right DNS magic to set in DreamHost‘s panel (if you’re interested: I’m now fully hosting apps.grandtrunk.net, which allows me to set the domain validation code using a normal file uploaded to DreamHost; apps.grandtrunk.net is also the domain I told Google Apps to use, while at DreamHost I needed to set a CNAME (alias) record for currencies.apps.grandtrunk.net that points to ghs.google.com). The cron job is also humming along nicely now downloading daily updates, so convert away while I watch the dashboard seeing my quota trickle down…

Using the StackOverflow public data set (the October 2009 one, which I still had lying around from last time), I set about plotting the number of questions versus the number of answers contributed by each user. This graph shows the raw results:

Clearly, several very active users tend to ‘specialize’ towards either the Questions or the Answers axis. One user is even up to almost 800 questions, but barely gave any answers. Can you earn reputation points that way? Sure, if they are good questions, other people will pass by that have hit the same dead-end and express their eternal gratitude of seeing their problem already solved by voting the question up. Here’s the same graph from before (except for the log-log axes) but including colors to show reputation classes:

The next graph plots each user’s reputation versus their “Q-to-A ratio” (their number of questions divided by their number of answers). For users without any answers I just plotted their number of questions (red dots).

The vast majority of users give much more answers than they have asked questions (so they have a Q/A-ratio of less than one) and inhabit the region near the horizontal axis. Most high-reputation users also tend to answer more than they ask. In the lower reputation regions (<1000) much higher Q/A-ratios can be seen (including those with an ‘infinite’ ratio, i.e. with no answers).

(As I found out later, someone else had already computed the top Q/A-ratios on an older SO dump, although I always prefer looking at things with some graphs…)

Finally, here are some Q/A-ratio distributions: the first one shows the number of users in each Q/A-ratio class. (Never mind the fact that the 1/1 class seems extraordinarily large in comparison to it’s neighbours, I’ve used a pretty strange class grouping function (an arctangens, so it can show both 0 and infinity symmetrically around 0) to make it fit properly on the graph.) The distribution is clearly weighted towards the right, again showing that most users have contributed more answers than questions.

We can also weigh users by their reputation, so we get the fraction of total reputation that is represented by the users with a certain Q/A-ratio. The already large group on the left side of the Q/A scale becomes even more important now, meaning that most reputation is owned by users with a low (<1/10) Q-to-A ratio. This is confirmed by the next graph which shows the average reputation of all users in a given Q/A class.

It’s easy to do this using tags: some sort of clustering should be applied according to how often each pair of tags shows up at the same question (a user that knows both ASP and ASP.net shouldn’t be considered a ‘diverse’ person, so this should be factored out first), next we can count in how many different clusters that this user has contributed a good answer.

I’ve had a stab at trying this on the last (October 2009) StackOverflow public data set. I’ve ignored the SU and SF parts of the dump. The idea is to count how many of SO’s questions you could conceivably answer, given your proficiency in each of the tags of that question.

First I’ve scored all answers each user has given. 20 points go to it being the accepted answer, another 80 points are distributed over all answers to a question in relation to the answer’s votes. This is not exactly the same as the reputation earned for that answer, since popular questions get a lot more up-votes to their answers, but in this analysis this isn’t really worth much more than a good answer to an unpopular question.

The points for all your answers are distributed over the tags with which the question is tagged. This gives each user a number of points for all tags. These points are converted into a proficiency that user has for this tag: proficiency = 1 - exp(-points / 500). So 1000 points (10 very good answers) gets you to 86%, more points will asymptotically get you to 100%.

At this point we can compute the average proficiency over all tags for each user, which is plotted in this graph (average tag proficiency versus reputation, for users with reputation > 1000):

The next step is to compute the question proficiency. This is done, for each question and each user, by taking the geometric average of this user’s tag proficiencies over all tags that this question is tagged with (I’ve choosen the geometric average rather than the arithmetic one, since you’re supposed to be knowledgable on all components of a question in order to answer it). These per-question proficiencies can again be averaged (arithmetically), yielding the user’s average question proficiency which is a measure for how many of the site’s questions he/she could answer. This graph plots average question proficiency versus reputation:

Note that this last one is a fairly heavy query (~1 second per user), so it could more feasible to base an actual implementation on tag proficiency only (see also this graph of tag vs. question proficiency), although this would overvalue knowledge of topics SO doesn’t care much about.

Analysing the question proficiency vs. reputation graph, we see that both are clearly related (if you answer enough questions, you’re bound to have covered most of the tags). Still, some users reach a possible threshold value of 20% question proficiency at a reputation of only 10,000; while others couldn’t get there even at 60,000 reps.