I am starting to play (pun intended) with jMusic. I am just learning the basics of music composition. jMusic is quite cool, but having the usual Java overhead makes things oh so boring! Therefore I am starting developing a DSL in Groovy to write some scores. Score is exactly the first class that was DSLed. Something of a trivial nature. Just replacing this

score = new Score("My new Score")

with:

score = score(name: "My new Score")

and the same to Note and Phrase. Furthermore I would like to avoid all the usual “import everything”.

Solution? Create a class with a static method that accepts an environment to which I add the necessary functions. So, in an external file I have class Music to do just this:

1
2
3
4
5
6
7
8
9
10
11
12
13
class Music implements JMC {
  static init(env){
 
    env.score = { Map args ->
      Score score = new Score(args["name"])
      return score
    }
 
    ProgramChanges.fields.each {env."$it.name"=ProgramChanges."$it.name"}
    Durations.fields.each {env."$it.name"=Durations."$it.name"}
    Pitches.fields.each {env."$it.name"=Pitches."$it.name"}
 
    ...

So, lines 4-7 I am creating a new property with a closure. The property accepts a map of parameters and creates a new Score object that is returned. Similar things exist with Note and Phrase.

Now look at lines 9-11. I am importing all fields of those classes into the environment namespace. “That is namespace polution”, I hear you say. Well, maybe, but it happens to be a jMusic design philosophy (you will find that in many classes of jMusic), and, at least for now it is pretty manageable. If it becomes problematic, this can always be changed. Writing CLARINET sounds better than writing ProgramChanges.CLARINET or even creating an INSTRUMENT property/class in the environment to hold all instruments. This is particularly useful with Pitches and Durations (because we tend to write A LOT of these). A simple script looks like this for now:

Music.init(this)
score = score(name:"Row Your Boat")
flute = part(title: "Flute", instrument: FLUTE, channel: 0)
trumpet = part(title: "Trumpet", instrument: TRUMPET, channel: 1)
clarinet = part(title: "Clarinet", instrument: CLARINET, channel: 2)
int[] pitchArray = [C4,C4,C4,D4,E4,E4,D4,E4,F4,G4,
		    C5,C5,C5,G4,G4,G4,E4,E4,E4,
		    C4,C4,C4,G4,F4,E4,D4,C4]
double[] rhythmArray = [ C, C,CT,QT, C,CT,QT,CT,QT, M,
			QT,QT,QT,QT,QT,QT,QT,QT,QT,QT,
			QT,QT,CT,QT,CT,QT, M]
 
phrase1 = phrase(startTime: 0.0)
phrase1.addNoteList pitchArray, rhythmArray
...

Nothing particularly fantastic, but somewhat less clutter.
This is version 0.0.0.0.0.1 pre-pre-pre-alpha. ;)
Watch this space for newer versions (more useful).
For now this serves to show two very basic DSL techniques with Groovy: adding methods to the environment and inspecting classes to copy fields.

Share and Enjoy:
  • Digg
  • del.icio.us
  • Facebook
  • Google Bookmarks
  • DZone
  • Reddit
  • Technorati
  • LinkedIn
  • connotea
  • FriendFeed
  • Twitter

Not long ago I published an article on the scientific journal Evolutionary Applications. This is a fairly new journal, so new that when I submitted my article it even did not have an impact factor. But it was clearly a great journal: well managed, good editors, sensible publicity, interesting articles and, in my case, the chosen reviewers of my paper were clearly knowledgeable and insightful. The first IF came out a 4.7 and that was to be expected. Furthermore it seemed a sustainable IF due to the obvious high quality of the journal. While access was free before the IF attribution it was pretty clear that the journal was “old-fashioned” (i.e. closed access after the initial “promotion period”). It was fair game that it would someday become closed-access. So far, so good. Nothing unexpected.

It became closed in 2011, too closed: It is not offered with the standard Willey-Blackwell package to libraries. This means that libraries have to explicitly buy it. From totally free, it became, to most researchers completely inaccessible. In practice 99% of the potential readership cannot access it (I am getting persistent requests for a copy of my article due to this).

In this time of economic crisis hitting university libraries, and where access is becoming increasingly easy (think open-access journals), this is the surest way to kill a journal. Well, there are more than enough garbage out there that deserves to disappear, but that is not the case of Evolutionary Applications. This is a real loss of a great, promising journal. I do not think the journal will maintain a shade of its current impact if this situation is not solved. Hopefully Willey-Blackwell will understand that they are killing one of their best recent journals and include EVA in the standard library offering. If not, I dare write this prehumous recognition to a great journal.

Share and Enjoy:
  • Digg
  • del.icio.us
  • Facebook
  • Google Bookmarks
  • DZone
  • Reddit
  • Technorati
  • LinkedIn
  • connotea
  • FriendFeed
  • Twitter

xtranormal is a company/service which allows you to produce movies just from text. Essentially you choose one or two characters, get a scenario and some music and then you create your script. The script is mostly the test spoken by the charater(s), but also camera changes, sounds, and character behaviour. All this wrapped up in a clever and easy web interface (they also have a desktop application by I did not use that). From a techy perspective the technology seems uber cool: text-to-speech (very good quality), 3D character animation and a nice interface on top. Adding to this their business model seems quite fair.

So that you can see what you can do with just one hour of work, here is an example that now sits in my home page (feel free to laugh at my expense, I do not mind ;) )

Go there, and just try to make a video of your own to get the feel for the technology.

PS – no relationship with the company, just a delighted user here.

Share and Enjoy:
  • Digg
  • del.icio.us
  • Facebook
  • Google Bookmarks
  • DZone
  • Reddit
  • Technorati
  • LinkedIn
  • connotea
  • FriendFeed
  • Twitter

I am currently in the process of assuring that two of my bioinformatics applications are multi-platform. This would be a simulator of age structured populations newAge and a library to access the HapMap project, interPopula. I am also responsible for a small part of the Biopython project. I’ve been only concerned with Linux and Windows (I do not have a Mac, but Linux stuff seems to work there as Mac has a *nix base). I would like to share here my experiences, maybe for the benefits of others.

The overall experience has been quite positive. I do have a strong Java/JVM background and it seems to me that Python is almost as much write-once, run-anywhere. At least if “anywhere” is old fashioned computer platforms. I would split the issues as follows:

  1. Python code – I have not had a single problem to report. My code includes sub-process management and file system access. The only thing where some care was needed is the use of os.sep (so that directory + os.sep + file yields either directory/file or directory\file. I also maintain Java/JVM applications and I do remember having more problems than this when assuring cross-platform work (see more below), unfortunately I just forgot precisely the problem to document it.
  2. GUI (wxPython) – Here there are indeed some minor problems. The semantics of the API seems sligthly different (e.g. Skip methods in events), or at least the Windows implementation might be buggy as the same event is called twice if there is a .Skip call. There are also some minor layout issues, but those were to be expected.
  3. External expectations – matplotlib can rely on LaTeX to pretty-print text (formulas and such), one of my scripts did exactly that. Well, in most *nixes LaTeX is around, not so much on Windows, there was a subtle, slightly hidden dependency on LaTeX. With most other libraries there was no big problems to be found (e.g NumPy)
  4. The database API – This officially sucks! The problem is that parametrized SQL is not standard. For instance, with SQLite one writes “select column1 from table where column2=?” to be able to parametrize the value for column2, but with psycopg (PostgreSQL) you have “select column1 from table where column2=%s”. There is still, at least a positional version. Even if you write standardized SQL (and it is possible to write SQL that works in many different flavours of servers) you will end up writing different versions for different drivers because of the non-standardization of parameters.

I happen to be also deploying a Java Web Start based application (Groovy+JVM), namely ogaraK, a simulator of malaria population genetics. Multi-platform is not that easy if you have a Swing GUI. The semantics of windows sizing operations differ slightly on the Mac. Also some components (like HTML rendering widgets) are buggy in the Linux OpenJDK. Generating Java 6 .classes and then having recent Macs failing because they just go to 1.5 is irritating. While some newer versions of Mac OS X do indeed support 6, it does not seem realistic for now, if Mac support is desired to go with anything above 1.5 :( .

All in all, Python fares pretty well as long as database stuff is not involved. I would dare to say that multi-platform GUI development with wxPython is slightly easier than with Java Swing.

PS – Bias disclaimer: I am a strong supporter of the JVM platform (as long as the word Oracle is not included), much more than of Python (in fact a big part of my Python usage is on top of the JVM via Jython). So my “hidden agenda”, if there was one, would be pro-JVM.

Share and Enjoy:
  • Digg
  • del.icio.us
  • Facebook
  • Google Bookmarks
  • DZone
  • Reddit
  • Technorati
  • LinkedIn
  • connotea
  • FriendFeed
  • Twitter

In Southern Europe, many of us always looked to the North (Scandinavia) as an example. Not with envy, but as an example on how to do things well. One of these countries has a unfortunate tradition to bend-over to the whatever bully that happens to be around. The most infamous case was Transit of Nazi troops (I refuse to call them German, because I refuse to equate Germany with Nazism — Nazism was an outlier of an otherwise civilized nation — but I digress).

It should come as no surprise the behaviour of the Swedish government in the whole Assange affair. Now as in WWII, Sweden as a tradition of bending over to whatever big power exists. Swedish pragmatism at work. Decency and civilization be damned!

I am not comparing the US with the Nazis. Indeed I happen to have the utmost respect for the only state in the world that was founded with Freedom in its genetic code. But, it is clearly the only superpower left and much of its contemporary behaviour leaves much to be desired and is in direct contradiction with the values of the USA founding fathers. But my point here is not about the US of A.

Imagine if Sweden had some backbone. I am Portuguese, and I am used to my government not having a moral backbone of any kind (see, e.g. the rendition cases and related Wikileaks cables). But I have to confess that I was expecting for Sweden to have improved and learned from its shameful past.

No cigar.

Share and Enjoy:
  • Digg
  • del.icio.us
  • Facebook
  • Google Bookmarks
  • DZone
  • Reddit
  • Technorati
  • LinkedIn
  • connotea
  • FriendFeed
  • Twitter

ogaraK

I would like to announce ogaraK, a simulator of malaria population genetics.

ogaraK is a Java Web Start application developed in Groovy and Java which allows to simulate parasite population genetics. It can be used, for instance, to compare the effects of different drug deployment policies.

A set of Python scripts are made available to analyze the results (mainly frequencies of resistance loci over time).

ogaraK is free software (GPL v3).

ogaraK can also be used to simulate some of common theories about sex: Epistasis, Red-Queen and spatial heterogeneity (but not based on size, as the underlying model has no concept of population numbers).

I would like to ask the reader for some basic help: If you can, could you please test the application (A single click to run, if you have Java 1.5+)? I know most people will not understand the results or the parameters, but just a simple run would help (and reporting back if something goes wrong!). The objective is to make the application available to epidemiologists, and due to their lack of IT knowledge, a robust application is needed. Any comments would really be appreciated (I make no financial profit from this free application). If you know Java Web Start, then if you could activate the console and return any error detected (even from a blind execution), that would be most appreciated.

Share and Enjoy:
  • Digg
  • del.icio.us
  • Facebook
  • Google Bookmarks
  • DZone
  • Reddit
  • Technorati
  • LinkedIn
  • connotea
  • FriendFeed
  • Twitter

Preamble: The problem of writing a defense of a certain thing X is that, most people interpret that as an attack to potential alternatives. This is not how this post should be read. There is no One Single Solution. My defense of Groovy is based on a set of assumptions that do not hold true for many people. In fact, they do not even hold true for myself. Different people and different development problems entail different solutions.

My main assumption in defense of Groovy is that you are, a Java person. Java is your day to day programming language and you are comfortable with that. Though you are comfortable with that you want to try something different: maybe you want to try a scripting language, maybe you do not like too much boilerplate code.

Why Groovy? Because it gives you a lot of goodies with essentially no learning curve. An illustrative example: I’ve spent the whole morning thinking that I was editing a Java source file, but I was indeed working on a Groovy file. You see, Groovy not being a superset of Java ends up being almost that: Code in Java is, in most cases already code in Groovy. So, if you know Java, you can write Groovy. You will not write the best idiomatic Groovy, you will not gain any of Groovy’s goodies (and you will pay a performance penalty, BTW). But this is an amazing head-start if you want to go in the direction of higher-level languages (I would argue that it is even smoother than C from C++ as the paradigm does not change from Java to Groovy, it is OO to OO). If you go the Jython way then you have to learn a new language. If you go Scala or Clojure then you have to learn a whole new paradigm (and deal with the impedance of imperative semantics in typical Java libs against standard functional semantics).

For little to no cost you have now many goodies associated with scripting languages (dynamic, low boiler-plate coding, DSLs, better meta-programming, …). In some cases Groovy even out-competes supposedly more elegant languages (are Scala meta-programming facilities still as bad as in the past?).

Another interesting advantage of Groovy is that, if you want to revert back to Java then it is much easier. Why would you want to do that? Well, for performance reasons. In a Groovy application that I have, a small part of the code is extremely intensive, so I had to rewrite it in Java. This revealed to be a trivial exercise (similar syntax, similar semantics).

Again, let me stress out this: your requirements and your personal path are fundamental in any decision you take. There is no true language (OK, Prolog…. ;) ). Different people, different approaches. All I am saying is: if your background is strongly grounded on Java and you feel comfortable with that, then Groovy is probably the way to go.

Disclaimer: While I have a couple of applications made in Groovy, most of my scripting efforts in the JVM world involve Jython (another fine language implementation, appropriate in a different set of circumstances) and I also believe, that from a declarative and highly expressive language point of view, what is being done with Clojure certainly deserves mention.

Share and Enjoy:
  • Digg
  • del.icio.us
  • Facebook
  • Google Bookmarks
  • DZone
  • Reddit
  • Technorati
  • LinkedIn
  • connotea
  • FriendFeed
  • Twitter

First, let me tell you a little story: A few years ago I worked for an university and we decided to replace a closed, proprietary backend infrastructure with an open one. The strange thing is that we replaced IBM with… IBM. You see, a long time ago IBM had a self-centered, closed mentality but since Gerstner’s time as IBM CEO things changed a lot. So, the old unpalatable IBM was replaced with the new, desirable IBM. A similar argument can be made for Sun… you probably still remember the closed Sun, the closed Java. Interestingly when we replaced IBM with IBM we also replaced the database backend: from Oracle to IBM.

Oracle.

Let me continue my little story: Fast-forward a few years and I accepted a job at the same university as data centre manager. The 3 biggest universities in the city had bought the same accounting/payroll system. This system had a Oracle backend: database, application server and financial extensions. It so happened that each component had completely different licensing terms: one was on concurrent connections another on concurrent users, yet another on the total number of users. The notion of user also varied: The financial application had thousands of users, but it connected with a single user to the application server. The usage of multiple CPUs was also subjected to licensing. I hate to think on the countless hours spent by the data centre managers just parsing all this. It still gives me a headache.

But it does not stop here: even as a manager I was still very technically inclined (and had previous experience as a database manager): I would subscribe to the view that Oracle databases are a too-much complex database system that seem to be highly profitable to Oracle and the contractors that are payed to maintain such a complex system. Between Oracle DB and IBM DB/2 I would take DB/2 any time (at least 5 years ago where my professional path changed). Of course, for most applications PostgreSQL is more than enough…

The point is, as you probably have noticed by now: I am not comfortable with Oracle’s corporate culture. I am not comfortable with Oracle taking the lead on Java. In fact I can think of no worse corporation to lead Java.

I would also like to draw your attention to size and power bias. My point is this: people tend to dislike Microsoft, but part of that dislike does not come from corporate culture but from the influence and power that Microsoft has (still) in computing. I dread to think of the consequences of people like Larry Ellison or Steve Jobs having the same influence as Bill Gates. I am not defending Bill, just suggesting that it could be much worse. We do not feel the pain of Oracle (or Apple for that matter) because they do not have the massive OS/Office market share that MS has.

I always had a love relationship with Java (though I have a background in declarative languages). It was amazing to see the appearance of new languages on top of the JVM (I use Groovy, Clojure and Jython). The idea of a shared VM (and shared libraries) which a set of cooperating and competing languages on top is wonderful…

But I am starting to doubt that the JVM world will stay an open community where the best ideas can be incepted and flourish.

If IBM had bought Sun…

I do not trust you, Larry!


Recommending readings (from James Gosling’s blog):
Quite the firestorm
The shit finally its the fan
Cynical chuckes

Share and Enjoy:
  • Digg
  • del.icio.us
  • Facebook
  • Google Bookmarks
  • DZone
  • Reddit
  • Technorati
  • LinkedIn
  • connotea
  • FriendFeed
  • Twitter

Nice reads mostly older:

  • [jvm] YouDebug – A debug with a scriptable UI (not a GUI). Allowing for automated debugging. The main justification for this is e.g. on costumer sites where you cannot fire a graphical debugger. But an automated debugging framework can be the ground for some cool debugging tools. Furthermore it is scriptable in Groovy!
  • [general]Are we there yet? – “Rich Hickey advocated for the reexamination of basic principles like state, identity, value, time, types, genericity, complexity, as they are used by OOP today, to be able to create the new constructs and languages to deal with the massive parallelism and concurrency of the future”. This is a 1 hour video (with synchronized slides). While I disagree with the basic premise that concurrency is fundamental in the future, this presentation is one of the most invaluable videos about the design of programming languages.
  • [lisp] A Lisp User-group meetings calendar (Google based)
  • [clojure] Be mindful of Clojure’s binding – Pitfalls of lazy sequences and thread bindings. The lazy sequence stuff seems more of an example of global variables are bad thing, as the example is dependent on an external symbol evaluation, but clearly one has to be mindful of lazy versus non-lazy evaluation.
Share and Enjoy:
  • Digg
  • del.icio.us
  • Facebook
  • Google Bookmarks
  • DZone
  • Reddit
  • Technorati
  • LinkedIn
  • connotea
  • FriendFeed
  • Twitter

I would like to add just a few more arguments regarding the issue of notation.

There is a certain austerity tradition coming from the CS community (at least from the BEST part of it). Most people see syntactic sugar as a waste of time and energy. Furthermore everything is just trees and a linear representation with the root at the head is just fine to represent trees. Lisp embodies this tradition to the full. And for the most part is a great tradition.

This is all fine, and I would understand the defensiveness considered that we live in a world polluted with overly complex languages which just bring problems and no advantages to compensate.

But some level of syntactic sugar is important for humans. And while meta-programming is nice, I suppose most programming languages are still used by human programmers ;) .

It would suggest to have a look at the literature on human cognition, namely the strategies used to read text: it is messy. But I prefer to go in a different direction. I just want to expose how some subtle representation issues might have a massive impact: Have you ever tried to do basic arithmetic with roman numerals? Have a look here, namely “How can I use Roman numerals to do arithmetic problems?”. While this is a different problem, it serves to emphasize that representation matters in the most strangest ways.

Even if it doesn’t, humans are a conservative species, they are not ready to change everything at the same time. Having a to jump to imperative+heteroiconic+infix to functional+homoiconic+prefix is too much. Lisp derived languages loose potential to attract more people. And in this case is is really worth it? Is it really THAT IMPORTANT to be prefixed? Is that the main conceptual selling point of Lisp?

But most importantly there are solutions that are not dirty:.

First nobody is forcing infix notation, one can have both at the same time (check Prolog :- op – you can do BOTH +(1 2) and 1+2).

Second hard-coding operators is not necessary at all: again check the Prolog suggestion. You can have all infix operators that you want (including none)

It is easy to conceptualize a superset of Lisp with infix notation and ZERO INITIAL infix operators.

If I can shout something is this: Check Prolog’s :- op ! Simple, elegant. And fully coherent with the Lisp philosophy at its core.

Share and Enjoy:
  • Digg
  • del.icio.us
  • Facebook
  • Google Bookmarks
  • DZone
  • Reddit
  • Technorati
  • LinkedIn
  • connotea
  • FriendFeed
  • Twitter