[digg-reddit-me]Mean is a complex topic, and is covered in Statistics in Erlang part 1.

Median and mode are less complex.  It’s worth noting that mode reports its results as a list, because it’s possible for there to be several modes for a list.  Also, mode is not strictly numeric – it works for mixed-type lists too (you can take the mode of a list of atoms, for example.)

Mode is really just a reduction of the results of histograph, which is similarly open to arbitrary type list contents.  Mode also uses a toy function even_or_odd which is provided here.

As usual, this code is part of the ScUtil library.  ScUtil is free and MIT license, because the GPL is evil.

This closes issue 100.  This closes issue 105.  This closes issue 134.  This closes issue 135.

histograph(List) when is_list(List) ->
    [Head|Tail] = lists:sort(List),
    histo_count(Tail, Head, 1, []).

histo_count([], Current, Count, Work) ->
    lists:reverse([{Current,Count}]++Work);

histo_count([Current|Tail], Current, Count, Work) ->
    histo_count(Tail, Current, Count+1, Work);

histo_count([New|Tail], Current, Count, Work) ->
    histo_count(Tail, New, 1, [{Current,Count}]++Work).

even_or_odd(Num) when is_integer(Num) ->
    if
        Num band 1 == 0 -> even;
        true            -> odd
    end.

median(List) when is_list(List) ->
    SList = lists:sort(List),
    Length = length(SList),
    case even_or_odd(Length) of
        even -> [A,B] = lists:sublist(SList, round(Length/2), 2), (A+B)/2;
        odd  -> lists:nth( round((Length+1)/2), SList )
    end.

mode([]) -> [];

mode(List) when is_list(List) ->
    mode_front(lists:reverse(lists:keysort(2, scutil:histograph(List)))).

mode_front([{Item,Freq}|Tail]) ->
    mode_front(Tail, Freq, [Item]).

mode_front([ {Item, Freq} | Tail], Freq, Results) ->
    mode_front(Tail, Freq, [Item]++Results);

mode_front([{_Item,_Freq} |_Tail],_Better, Results) ->
    Results;

mode_front([], _Freq, Results) -> Results.

[digg-reddit-me]I’ll be putting up some statistical functions I’ve had to write recently.  This is the first batch.  I confess, I find the erlang implementations far more readable than the pure math definitions one finds around; I’ve been thinking about writing tutorials, but with this code here, I’m not entirely sure it’s necessary.

At any rate, the code follows.  As with so much of my erlang code, this code is part of the ScUtil library.  ScUtil is free and MIT license, because the GPL is evil.

There’re more statistics coming, I just don’t want to make any one post too huge, and I don’t want keyword saturation.

This partially closes issue 119.  This closes issue 133.  This closes issue 136.  This closes issue 137.

list_product(List) when is_list(List) ->
    list_product(List, 1).

list_product([], Counter) ->
    Counter;

list_product([Head|Tail], Counter) ->
    list_product(Tail, Counter*Head).

arithmetic_mean(List) when is_list(List) ->
    lists:sum(List) / length(List).

geometric_mean(List) when is_list(List) ->
    math:pow(scutil:list_product(List), 1/length(List)).

harmonic_mean(List) when is_list(List) ->
    length(List) / lists:sum([ 1/X || X<-List ]).

weighted_arithmetic_mean(List) when is_list(List) ->
    weighted_arithmetic_mean(List, 0, 0).

weighted_arithmetic_mean([], Num, Denom) ->
    Num/Denom;

weighted_arithmetic_mean([{W,V}|Tail], Num, Denom) ->
    weighted_arithmetic_mean(Tail, Num+(W*V), Denom+W).

[digg-reddit-me]Oddly, the Erlang standard library does not have a function for reading out module attributes.  One is relatively straightforwardly hacked together with beam_lib, if you know what you’re doing, but most people wouldn’t know to look there, and it’s kind of a hassle to write.

So, I wrote it for you.

• get_module_attribute is a member of the ScUtil library.
• This closes issue 117.

Usage is straightforward: scutil:get_module_attribute(lists, export).

get_module_attribute(Module,Attribute) ->

    case beam_lib:chunks(Module, [attributes]) of

        { ok, { _, [ {attributes,Attributes} ] } } ->
            case lists:keysearch(Attribute, 1, Attributes) of
                { value, {Attribute,[Value]} } -> Value;
                false                          -> { error, no_such_attribute }
            end;

        { error, beam_lib, { file_error, _, enoent} } ->
            { error, no_such_module }

    end.

[digg-reddit-me]Amusingly, there isn’t actually a way to ask Erlang the type of a variable, and new programmers often struggle with this on basis of not realizing they’re expected to use is_foo() guards.

This function, type_of(), resolves the issue.  As such, none of the testing stuff is included; that’s all in the library, rather than here on the blog.

• type_of() is a member of the free ScUtil library.
• This closes issue 132.

If you’re running Erlang 11 or earlier, the bitstring clause will fail.  That is a new type.  Just comment it out and move on with your day; it’s a synonym for “binary” and on your system the word “binary” will come out instead.

type_of(X) when is_integer(X)   -> integer;
type_of(X) when is_float(X)     -> float;
type_of(X) when is_list(X)      -> list;
type_of(X) when is_tuple(X)     -> tuple;
type_of(X) when is_bitstring(X) -> bitstring;  % will fail before e12
type_of(X) when is_binary(X)    -> binary;
type_of(X) when is_boolean(X)   -> boolean;
type_of(X) when is_function(X)  -> function;
type_of(X) when is_pid(X)       -> pid;
type_of(X) when is_port(X)      -> port;
type_of(X) when is_reference(X) -> reference;
type_of(X) when is_atom(X)      -> atom;

type_of(_X)                     -> unknown.

I’m going to be releasing a few new libraries in the next several days, both by archive and public subversion.  I’ve already bought the domains and built a forum for them.  I even wasted a couple hours subversion automating everything down to the line of having little library websites made automatically, with custom per-library color palettes.

I was a little bored.

So we’re going to have three libraries progressing in the immediate future, with quite a few more over time:

1. HtStub – An embeddable, zero-config, zero-behavior secure Erlang webserver
2. SC A-Star – An efficient, modular ECMAscript (flash/actionscript, javascript, jscript etc) A* implementation with support for custom grid geometries (includes algorithm tutorial)
3. TestErl – unit, regression and stochastic testing for Erlang
4. ScUtil – a fairly large list of gap filling functionality for Erlang

In the near future, I will add C++ and PHP libraries, as well as many some libraries for more obscure languages like FormulaONE, Mozart-Oz, Factor and maybe (sadly) Delphi.  I have more than 30 libraries ready for release.

All those repos are pretty empty at the moment.  That will change in coming days, and I’m sure I’ll post lots of boring little snippets here about whatever minor new thing my crap does.

Yay!

I’ve actually built a bunch of small Erlang libraries which I find tremendously useful.  Over the last weekend, I’ve set up public SVN for a lot of them, built a small forum and started a project manager.  I’m actually restarting each and every project in the light of better adherence to best practices, since the code’s going to be distributed now, such as test suites from ground zero, but the point is, I’m making my libraries available and easy to find now.

The first three I’m going to release are scutil, testerl and htstub.

• scutil is my personal utility library.  It’s a hodgepodge of semi-random stuff.  there’s a whole lot of crap down there.
• testerl is a unit/regression and stochastic testing library.  One of the main goals of testerl is to have low impact on the code it tests, which it achieves through a single module attribute and a list (syntax identical to export) of the non-exported functions that the test rig needs access to for testing.
• htstub is a socket manager and HTTP protocol implementation.  It’s not a webserver – it has no filesystem access and doesn’t know how to respond to requests – it’s more of a transparent way to handle HTTP.  htstub is useful for adding a web front end to applications.

I actually intend to release some c++, actionscript, php, javascript and delphi stuff soon too, If I Get Around To It ™; that said, now that I have SVN largely automated, it’s quite likely.

I’ve been thinking about making an image plugin for WordPress.  I want to restart my image of the day process, but the import process has been dreadful, and there’s no programmatic access to the image list, meaning things like random images and images from subgroups aren’t particularly reasonable.  To that end I need to write my own, and since I’d love the rank that comes from having a high-usage plugin, I need a clear idea of what things go into an image plugin, what features are missing from existing plugins, et cetera.

I’m already doing complex efficient randomization, API access, bulk posting, timed bulk posting, base autotagging, auto-categorization, a catalog widget, and I’m going to make sure that my stuff is compatible with the All In One SEO pack.  I’m going to provide integration points, and I’m going to provide an example integration with LightBox, or one of its relatives.  I’m going to provide voting, moderated tag suggestion and home-post permalinking.

I’ll also be writing strict, browser/version portable code without hacks.  Yay!

Please let me know what you’d want to see if there were a new image plugin coming out.

I had to write these for a colleague some months ago, and promptly forgot in classic fashion. Here’s something to mock my inability to write coherent install docs for posterity: Setting Up eJabberD From Scratch. A how to, of sorts, I guess. This was written for a Centos server, but is probably accurate for most Unices (don’t really know for sure.) Read the rest…

When wandering around Das Intarweb one sees a lot of sad, sad code. In fact, people who should know better get busted on weak crypto all the time. Indeed, even the PHP manual examples have unacceptable security flaws. When one is writing encryption code for one’s own site, that turns into a problem. Here’s a library to wrap and a little primer on using the standard encryption facilities in PHP safely and correctly.

Read the rest…

MSHTML is an awesome user interface tool, but it has a whole lot of standard behaviors, many of which aren’t what one wants for an application (since it’s designed for the web.) This is a list of stuff you need to do to embed IE COM and have it behave like a normal application. There’s more than a person might expect.

Read the rest…