Mean, Median, Mode and Histograph (Statistics in Erlang Part 2)

Erlang, Math, Programming, Statistics, Tools and Libraries 1 Comment

[digg-reddit-me]Mean is a complex topic, and is covered in Statistics in Erlang part 1.

Median and mode are less complex.  It’s worth noting that mode reports its results as a list, because it’s possible for there to be several modes for a list.  Also, mode is not strictly numeric – it works for mixed-type lists too (you can take the mode of a list of atoms, for example.)

Mode is really just a reduction of the results of histograph, which is similarly open to arbitrary type list contents.  Mode also uses a toy function even_or_odd which is provided here.

As usual, this code is part of the ScUtil library.  ScUtil is free and MIT license, because the GPL is evil.

This closes issue 100.  This closes issue 105.  This closes issue 134.  This closes issue 135.

histograph(List) when is_list(List) ->
    [Head|Tail] = lists:sort(List),
    histo_count(Tail, Head, 1, []).

histo_count([], Current, Count, Work) ->
    lists:reverse([{Current,Count}]++Work);

histo_count([Current|Tail], Current, Count, Work) ->
    histo_count(Tail, Current, Count+1, Work);

histo_count([New|Tail], Current, Count, Work) ->
    histo_count(Tail, New, 1, [{Current,Count}]++Work).

even_or_odd(Num) when is_integer(Num) ->
    if
        Num band 1 == 0 -> even;
        true            -> odd
    end.

median(List) when is_list(List) ->
    SList = lists:sort(List),
    Length = length(SList),
    case even_or_odd(Length) of
        even -> [A,B] = lists:sublist(SList, round(Length/2), 2), (A+B)/2;
        odd  -> lists:nth( round((Length+1)/2), SList )
    end.

mode([]) -> [];

mode(List) when is_list(List) ->
    mode_front(lists:reverse(lists:keysort(2, scutil:histograph(List)))).

mode_front([{Item,Freq}|Tail]) ->
    mode_front(Tail, Freq, [Item]).

mode_front([ {Item, Freq} | Tail], Freq, Results) ->
    mode_front(Tail, Freq, [Item]++Results);

mode_front([{_Item,_Freq} |_Tail],_Better, Results) ->
    Results;

mode_front([], _Freq, Results) -> Results.

Arithmetic Mean, Geometric Mean, Harmonic Mean and Weighted Arithmetic Mean (Statistics in Erlang Part 1)

Erlang, Math, Programming, Statistics, Tools and Libraries 4 Comments

[digg-reddit-me]I’ll be putting up some statistical functions I’ve had to write recently.  This is the first batch.  I confess, I find the erlang implementations far more readable than the pure math definitions one finds around; I’ve been thinking about writing tutorials, but with this code here, I’m not entirely sure it’s necessary.

At any rate, the code follows.  As with so much of my erlang code, this code is part of the ScUtil library.  ScUtil is free and MIT license, because the GPL is evil.

There’re more statistics coming, I just don’t want to make any one post too huge, and I don’t want keyword saturation.

This partially closes issue 119.  This closes issue 133.  This closes issue 136.  This closes issue 137.

list_product(List) when is_list(List) ->
    list_product(List, 1).

list_product([], Counter) ->
    Counter;

list_product([Head|Tail], Counter) ->
    list_product(Tail, Counter*Head).

arithmetic_mean(List) when is_list(List) ->
    lists:sum(List) / length(List).

geometric_mean(List) when is_list(List) ->
    math:pow(scutil:list_product(List), 1/length(List)).

harmonic_mean(List) when is_list(List) ->
    length(List) / lists:sum([ 1/X || X<-List ]).

weighted_arithmetic_mean(List) when is_list(List) ->
    weighted_arithmetic_mean(List, 0, 0).

weighted_arithmetic_mean([], Num, Denom) ->
    Num/Denom;

weighted_arithmetic_mean([{W,V}|Tail], Num, Denom) ->
    weighted_arithmetic_mean(Tail, Num+(W*V), Denom+W).

Dear Mister Harris: Because They Can

Gaming, General Interest, Miscellaneous, Rants 2 Comments

One Cliff Harris has recently asked why people pirate his games.  Unfortunately, I wrote my entire response to a mischaracterization by the slashdot article that pointed me to his question; they suggested that Cliff was trying to convert these warezers to sales, which on reading the actual article I’ve realized he actually is not.  Nonetheless, I coincidentally answered his question during my rant, and then towards the end I take it on more directly.  Still, there are a lot of people who want to know the answer to the question Cliff didn’t ask, so I’m leaving it in place.

Assuming that developers are missing out on potential sales from disgruntled pirates

… is a flawed assumption. Mister Harris appears to fail to understand the mindset of the pirate, who is a person who has confused what they want with what is ethical.

I’ve been running and co-running a number of small communities about game development for more than a decade now. Several of them have a real problem with pirates who show up looking for help with piracy. It’s sometimes hard to tell the difference between a pirate and a kid using the wrong terms for things (“how do I build my ROM”, etc); as such there’s sort of an ongoing competition among the people who run these groups to see who can get these goons to uncover themselves the fastest, usually by feigning sympathy.

As a result, I’ve seen about three times as many warezers as the human population of Earth. Every single one tries to tell me, after they’re removed, how it’s not their fault they stole – the game is too expensive, or they don’t want to feed EA, or they’ll pay for it if they like it. Many of them have already forgotten that during the sympathy phase, they gave us lists of the games they had. Particularly galling are the people who brag that they have ROMs of every single DS game, or what have you, then turn around and pretend that it’s just due to cost.

With respect, Mr. Harris, you’re asking the wrong question. You could be selling your game for a quarter with a change accepting machine in their rooms; they wouldn’t buy your game. They’re out there getting every game they can find, often just for the bragging rights of having stolen more than their peers. Many of the people stealing your game haven’t even heard of it and will never play it. These people cannot be converted into customers; they are too used to theft to recognize it as such, invariably vomiting up the same tripe about a false and meaningless distinction between copyright violation and theft, because they don’t think of themselves as thieves and cannot face the honest nature of what they’re doing. These people will never voluntarily give up money for your hard work, and you cannot get them to stop taking your work.

There are two somewhat more legitimate questions you might ask, however.

The first is “how can I profit from these people.” That’s not the same thing as turning them into customers. For example, though I do pay for my games, I play a lot of free games on the web which I wouldn’t pay for (I’d just play more Civ instead.) DesktopTD is a great example: when it was news to me I would not have bought it because it looks poor, and by now I’ve played it so much that I don’t even play it for free anymore. During my addiction I might have paid a couple of bucks for it, but probably not, and the market doesn’t offer a sales mechanism that hits that phase.

However, DesktopTD has probably made about $3.50 from me by now. I’m not pulling that number out of thin air; I made an honest estimate of plays based on my best guess about when I found the game and how often I play, and ran it through the numbers for MochiAds. Admittedly, I’m not a warezer, so my example applicability is limited, and indeed I do know a few people who brag that they’re running ad blockers so they’re not inconvenienced with ten seconds of advertisement to put money in the developer’s hands, even though the developer is giving their game away. Most of these people, unsurprisingly, are warezers.

The other question is a bit more direct. Say you’re an interior designer. You’re brand new, the ad agency is several weeks from having your commercial on TV, but you have your cards and your flyers and business hasn’t picked up yet, so you decide to go drive around and make some people aware of your services. You have a choice: drive around the lower middle class neighborhood, where your services are needed more commonly, or around the rich neighborhood, where one uptake is worth twenty from the lower middle class neighborhood.

When it comes down to it, there are a huge number of people willing to pay $50 for a game. Those people expect games to have huge production values, grand sweeping storylines, volumes of beautiful artwork, a custom soundtrack and hundreds of cheat codes.

A one-man game designer can aim at the $5 or even $10 niche without problems; witness XBLA, WiiWare, the Apple store, et cetera. Even so, the one-man game designer is clutching at threads to get those sales; they’re just one person, and there isn’t enough time in the day to make what one of EA’s hundred fifty person teams can make.

What you’re going to find is that if you can just barely get the people who pay $50 to pay $5, then getting the people who won’t buy things at $50 to even spend $1 is damned near impossible.

So, lemme ask you a question in return: why are you driving around the lower middle class neighborhood?

I want to know why people pirate my games. I honestly do.

The answer to your question is simple: they do it for different reasons.  Some just want to play your game and don’t want to pay for it.  Some are collectors.  Some pirated your game because their pirate buddy said “you should try this game”; same word of mouth that you’re used to thinking of as driving sales to you.  Some do it because in some crowds it is a status symbol to out-pirate other people.  Some caught it in a torrent collection with a different item they want.  If your game features top of the line theft protection, some will pirate it because they’re offended you should want to protect your work, and see defying that as a way to stick it to the proverbial man.

More germanely, though, none of them face the reason they do it, so asking why they do it is going to get you a stack of excuses and hollow justifications.  One thing you’ll find out if you ask a psychologist is that even among the badguys, basically nobody thinks of themselves as a badguy.  There are books where psychologists of mafia organizers talk about how their clients have talked themselves into believing that their process of arranging protection and hush money, murdering people and running contraband is somehow necessary or vital to society, how they’re just “giving people what they want.”  It’s exceptionally entertaining in moral gray areas, such as when talked about by pimps, who are doing something that large parts of the world, including some parts of our country, see as acceptable.  The point, however, is better made with obvious scumbags, such as the people who arrange serial murder.

You can’t ask a person who won’t face who they are what made them who they are.  All you’ll get are their self serving fantasies.

They don’t know the truth any better than you do, sir.

Interesting stuff at “In Search of Concise Software”

Erlang, General Interest, Miscellaneous, Programming No Comments

Someone made a relatively intelligent comment on one of my blog posts, leading me to believe he’s not from the internet.  Nonetheless, I took a look at his site, In Search of Concise Software, and there’s some interesting junk posted … leading me to believe he’s not from the internet.

Pity about Blogspot’s interface.  Even so, it’s worth a look.

Free Microsoft Products for Functional Programmers

General Interest, Miscellaneous, Programming 1 Comment

If you want some free legal Microsoft products, let them learn from you about functional programming.

Reading Module Attributes in Erlang

Erlang, Programming, Tools and Libraries 12 Comments

[digg-reddit-me]Oddly, the Erlang standard library does not have a function for reading out module attributes.  One is relatively straightforwardly hacked together with beam_lib, if you know what you’re doing, but most people wouldn’t know to look there, and it’s kind of a hassle to write.

So, I wrote it for you.

Usage is straightforward: scutil:get_module_attribute(lists, export).

get_module_attribute(Module,Attribute) ->

    case beam_lib:chunks(Module, [attributes]) of

        { ok, { _, [ {attributes,Attributes} ] } } ->
            case lists:keysearch(Attribute, 1, Attributes) of
                { value, {Attribute,[Value]} } -> Value;
                false                          -> { error, no_such_attribute }
            end;

        { error, beam_lib, { file_error, _, enoent} } ->
            { error, no_such_module }

    end.

type_of(): Variable Type Detection in Erlang

Erlang, Programming, Tools and Libraries 6 Comments

[digg-reddit-me]Amusingly, there isn’t actually a way to ask Erlang the type of a variable, and new programmers often struggle with this on basis of not realizing they’re expected to use is_foo() guards.

This function, type_of(), resolves the issue.  As such, none of the testing stuff is included; that’s all in the library, rather than here on the blog.

If you’re running Erlang 11 or earlier, the bitstring clause will fail.  That is a new type.  Just comment it out and move on with your day; it’s a synonym for “binary” and on your system the word “binary” will come out instead.

type_of(X) when is_integer(X)   -> integer;
type_of(X) when is_float(X)     -> float;
type_of(X) when is_list(X)      -> list;
type_of(X) when is_tuple(X)     -> tuple;
type_of(X) when is_bitstring(X) -> bitstring;  % will fail before e12
type_of(X) when is_binary(X)    -> binary;
type_of(X) when is_boolean(X)   -> boolean;
type_of(X) when is_function(X)  -> function;
type_of(X) when is_pid(X)       -> pid;
type_of(X) when is_port(X)      -> port;
type_of(X) when is_reference(X) -> reference;
type_of(X) when is_atom(X)      -> atom;

type_of(_X)                     -> unknown.

Gearing Up for Public SVN

C/C++, ECMA / Javascript, ECMAScript, Erlang, Game Algorithms, PHP, Programming, Tools and Libraries, Web and Web Standards No Comments

I’m going to be releasing a few new libraries in the next several days, both by archive and public subversion.  I’ve already bought the domains and built a forum for them.  I even wasted a couple hours subversion automating everything down to the line of having little library websites made automatically, with custom per-library color palettes.

I was a little bored.

So we’re going to have three libraries progressing in the immediate future, with quite a few more over time:

  1. HtStub – An embeddable, zero-config, zero-behavior secure Erlang webserver
  2. SC A-Star – An efficient, modular ECMAscript (flash/actionscript, javascript, jscript etc) A* implementation with support for custom grid geometries (includes algorithm tutorial)
  3. TestErl – unit, regression and stochastic testing for Erlang
  4. ScUtil – a fairly large list of gap filling functionality for Erlang

In the near future, I will add C++ and PHP libraries, as well as many some libraries for more obscure languages like FormulaONE, Mozart-Oz, Factor and maybe (sadly) Delphi.  I have more than 30 libraries ready for release.

All those repos are pretty empty at the moment.  That will change in coming days, and I’m sure I’ll post lots of boring little snippets here about whatever minor new thing my crap does.

Yay!

Better Spam Every Day

Humor 1 Comment

This time the subject was “Jesus Christ To Star In Next Series Of Big Brother”, and the link it tried to use to get me to look was “John McCain Sex Video Yanked from Youtube”.

Laughing to tears.  At work.  Not good.  :D

Spam getting funnier rapidly

Bunk News, General Interest, Humor 4 Comments

The most recent one I got said “Bush down to 8 friends on MySpace”.  Immediately, I thought “yeah, and they’re all Karl Rove”.

Can’t stop laughing.  I think I actually enjoy SPAM now.  How creepy is that?

« Previous Entries Next Entries »