The Scala vs Erlang whirlwind

11 Replies

Over the last week or two there’s been a bit of commotion with various parties in the blogosphere making the case for Scala against Erlang or for Erlang against Scala. Here’s a see spot run summary of the main writers and their positions / content:

Ted Neward (1, 2) – Ted (how confusing) is in the Scala camp, and thinks that the library approach of Scala’s actor library is preferable to Erlang’s VM (BEAM). He cites managability as a major concern. He also thinks that adapting a process style model into the JVM would be easier than adding SNMP monitoring to BEAM. The length of the Barcelona project bibliography suggests otherwise, but we’ll never know unless some brave soul goes and tries to do this. Fortunately, the JDK is open source now. One has to wonder whether such a change could make its way through the JCP, though. Unfortunately for Ted, I found that many of his arguments were weakened by his lack of knowledge about Erlang.

Steve Vinoski (1, 2, 3) – Steve’s articles are more about the reliability aspects of Erlang, and he’s mostly trying to correct Ted’s facts on Erlang. He thinks that Erlang had proven its reliability chops by running for years non-stop. Given the frequency with which Java app servers need to be (or are) bounced, this doesn’t seem that incredible to me.

Patrick Logan (1, 2, 3) – Patrick piled on after Steve and has spent most of his writing trying to correct/challenge Ted’s assertions about Erlang. Patrick thinks that the conventional (i.e. JVM and CLR) runtimes will have problems implementing an Erlang style shared-nothing model, since the pre-existing libraries for those runtimes are not engineered in a shared-nothing manner.

Barry Kelly was an observer of the Neward-Vinosk-Logan discussion, and added some commentary on the impact of VM primitives on things like lift. This is a point which resonates with me, because it seems to me that both languages and language runtimes will need some work to meet the challenges of large scale multicore computing.

Yariv Sadan has done a pile of stuff in Erlang, and supplied his own summary of the differences between Scala and Erlang. There is a very informative exchange between Yariv and lift author David Pollak in the comments of this one.

That’s the short rundown. This is a very interesting problem space — before I turned into database programming language guy in graduate school, I was angling to be a concurrent programming language guy. Along the way to that I got pretty good doses of functional and logic programming, as well as actor programming. That work was in the context of people planning to build (for the day) highly concurrent computers, on the order of 1000’s of processors. Today, multicore hardware is not quite up to that level, but it is approaching it pretty quickly. If there is any force in computing that is likely to precipitate the need for a new programming ecosystem (language, runtime, libraries), I think concurrent programming is it. I also think there is just not enough experience with this problem to have a real sense of what is really going to work. Cliff Click and Brian Goetz were right when they said that we just don’t have a good programming model for this stuff. Absent a model, I don’t know how we can think that we really understand what the runtime needs to deliver.

Jython Users at Europython

5 Replies

I’ve been asked to moderate a Jython panel at Europython this year, and we are looking for Jython users to be represented on the panel. So if you are a Jython user and you are gong to be at Europython, please drop me a note or leave a comment.

Book Review: The Photographer’s Eye

2 Replies

Learning to lighting has done a lot for my photography. It’s not just gaining a new appreciation for light of all kinds, but also the fact that lit photographs have cause me to think about photographs differently. I used to be a much more reactive photographer – I would just be waiting for moments to happen in order to capture them. Now, I’ve become a little more thoughtful about what I want to the end picture to look like, even in situations that are fairly fast moving. Improving my grasp of composition is definitely something that I am working on.

My favorite book on photographic composition was Photographic Composition by Tom Grill and Mark Scanlon. At least it was until I read Michael Freeman’s The Photographer’s Eye: Composition and Design for Better Digital Photos.The entire first chapter is all about framing, which is the most extensive treatment that I’ve seen so far. In addition to a treatment of the usual compositional elements, there’s also a chapter on design basics, which I definitely needed some help with (and probably still do). Perhaps the best chapter of all is the last one on process, where Freeman walks through a case study or two, showing how he prepared and then composed a picture in a photojournalistic setting. Learning someone else’s thought process always seems to yield a bump up for me, much more so than just learning the straight mechanics.

Of course, with all things photographic, it’s not enough to read the book and understand it. The trick is putting it all into practice.

Scala liftoff

1 Reply

I stayed around in San Francisco for one more day after JavaOne, in order to attend the Scala liftoff. The liftoff was an open space style conference (which has a more specific meaning than “unconference”, at least to me). My friend Kaliya Hamlin did a great job of facilitating the day.

Scala has steadily been gaining attention, and hasn’t yet hit (at least in my eyes) the hype part of the classic Gartner hype cycle. I’ve been poking about with Scala, mostly because of the type inferencing, the Actor library, and lift. I have great respect for the work that Martin Odersky has done over the years, which also has me interested. Couple that with what I learned about closures in Java at JavaOne, and the list of reasons to look more deeply at Scala is getting long, especially if you are determined to have a statically typed languages.

I wasn’t able to make it to any of sessions on lift. It just worked out that other sessions overlapped them in a pathological way. While this is unfortunate, I am sure that I’ll be able to pick up anything that I need from the mailing lists and other documentation. I was able to attend two sessions on actors. One of the sessions had people with questions about actors, but no Scala actor experts were in that group. There was some discussion of Pi-calculus and the join calculus, but no discussion of the actual actor theory.

Steve Yen’s session on actor-d was pretty useful. Steve set out to build a version of memcached using Scala’s actors. He spent most of his slot talking about Scala/Java isms that he ran into – this was important since he was comparing to the C memcached. By the time he got to the actor related stuff, he was almost out of time. Steve found that he had to remove actors from the main loop of his server in order to get sufficient performance. He wanted to get statistics from the server in the background and discovered that he main loop actor was always processing messages and was never idle long enough to report statistics. He ended up replacing the actor with plain old Java Threads (POJT?). This was in addition to all the fact that he ran into many of the standard Java problems as well. I’m not sure what to conclude from this. I don’t recall what kind of hardware he was on, and I am not convinced that he had the right architecture for an actor based system. Some of his experience also seemed contrary to what the lift folks have been claiming. I think that we are in for a decent amount of investigation here. One of Martin’s statements about Scala is that it is possible (and better) to extend the language via libraries than via actual language constructs. For the most part, I agree with this, but there are certain extensions which have interactions with the runtime – like concurrency. In those cases, I don’t see how the library approach allows taking advantage of runtime features. The current version of Scala actors is implemented as a library.

One of the things that I am currently working on is support for Python in NetBeans, so I dropped into the session on IDE support for Scala. With the exception of IntelliJ, none of the IDE plugin principals were present, so it was hard to have a really productive discussion. Martin did attend the session and we talked about the possibiliy of getting hooks into the existing Scala compiler, particularly the parser and the type inferencer. That could yield some big dividends for people working on IDE support. One IDE feature that I would like to see is the ability to hit a key, and have the IDE “light up” all the inferred types, overlaid on the existing program code. This would allow developers to see if their intuition about the types actually matched that of the type inferencer. I’d like a feature like this for Python/Ruby/Groovy/Javascript code as well. Further discussion was deferred to the scala-tools mailing list.

The other session that I participated in was the session on Scala community and governance. Several people wondered about this during Kaliya’s “What questions do you have about Scala” portion of the schedule building. When nobody else put up a session in this area, I grabbed a slot, hoping to spur some conversation – if for no other reason than my own education. Fortunately, Martin had already been thinking about the problem. He is going to adopt a Python style governance, with him (and EPFL) having the final say on language design matters. There will be Scala Enhancement Proposals (SEPs), like the Python PEPs. I’m very happy with this. I think that Python has done very well at maintaining the balance between (lots) of community input on the language design, while still retaining that “quality without a name”. One of the things that I said during the CommunityOne general session panel was that particular individuals in the right place, at the right time, matter at great deal. After watching Martin for the day, and seeing his interactions on the mailing list over the last few months, I think that the design of Scala is in very good hands.

We also talked about the evolution of the Scala libraries. The Scalax project is working to build a set of utility libraries for Scala. Martin views scalax as a place where anyone can submit a library, have it tested, vetted, reworked, etc. Eventually some code in scalax would be candidates for addition to the Scala standard libraries. This also seems like a sane approach to me. I like the idea of having a place for libraries to shakeout before going into the standard libraries. Martin also mentioned a LINQ in Scala project. I need to track that one down too.

It is good to be in a multi-language world again. There’s room for Scala, Python, Ruby, and others. Another language that I am keeping my eye on is Newspeak.

JavaOne 2008: Part 2

3 Replies

I’ve been to so many conferences and seen so many talks that it’s hard for me to really get excited about conference presentations. I went to talks here and there, but nothing at JavaOne was really reaching out at grabbing me (in fairness, this happens at other conferences also, so it’s not just JavaOne). Or at least that was true until the last day.

Friday opened with a keynote by James Gosling, who served as the MC for a train of presenters on various cool projects.

Cool stuff

First up was Tor Norbye, who has done a lot of good work on support for editing different languages in NetBeans. Tor has been working on JavaScript support for NetBeans 6.1, and he showed off some cool features, like detecting all the exits from a function, semantic highlighting of variables, and integrated debugging between NetBeans and Firefox. All of which was cool. When I was managing the Cosmo group at OSAF, I tried a bunch of Javascript IDE’s and never really liked any of them. I haven’t done a lot with NetBeans 6.1 yet, but I will. Tor showed one feature, which was the killer one for me. NetBeans knows what Javascript will work in which browser. You can configure the IDE for the browsers that you want to support, and this affects code completion, quick fix checking and so on. Definitely useful. Here are several more references on the Javascript support in NetBeans 6.1.

The Java Platform

It’s easy for me (and others, I’d bet) to think mostly of JavaEE or perhaps JavaME when thinking about Java. That’s understandable given the worlds fixation on web applications, and looking ahead to mobile. But the majority of the talks in Gosling’s keynote session had nothing to do with Java SE, EE, or ME (at least in the phone sense).

Probably the hit (applause meter wise) of the keynote was LiveScribe‘s demonstration of their Pulse Smart Pen. This is an interesting pen that records the ink strokes that it makes, and any ambient audio that it records while the writing is happening. The ink and audio can be uploaded to a computer, as long as that computer runs Windows (apparently a Mac version is in the works). Unfortunately, the pen works by sensing marks on a special paper (that would be the razor blades), so there’s a limitation on how useful this can be. The presenter said that a future version of the software would allow people to print their own special paper, but that’s still a future item for now. By reading special marks on the special paper, you get a pretty cool user interface. The pen itself can run Java programs, and there is a developer kit available for it. If they can get by the limitation of special paper, I think that this is going to be pretty interesting.

Sentilla showed off their Mote hardware, which seem like RFID chips that can run Java programs. except that these RFID chips can form mesh networks amongst themselves and can have various kinds of sensors attached. There are lots of applications for these things, going well beyond inventory tracking and such.

Sun Distinguished Engineer Greg Bollella demonstrated Blue Wonder, which is a replacement for the computers used to control factories. Blue Wonder combines off the shelf x86 hardware, Solaris, and real time Java to provide a commodity solution for factory control applications. This is far afield of Web 2.0 applications, but just as cool, in my mind.

By the end of the keynote I was reminded of the long reach of the JVM platform, something that I’d lost sight of. The latest craze in the Web 2.0 space is location data — O’Reilly has an entire conference devoted to the topic. I think that sensor fusion of various kinds (not just location sensors) is going to play a big role in the next generation of really interesting applications. The JVM looks like it’s going to be a part of that. I don’t think than any other virtual machine technology is close in this regard.

Java’s future

I also went to a talk on Maxine, a meta-circular JVM. By the twitter reactions of the JRuby and Jython committers, I’d say that Maxine is going to get some well deserved attention when it is open sourced in June. I’m particularly interested because the PI’s for Maxine worked on PJava, and MVM. Given the differences between the Erlang VM and the JVM, I think that the ability to experiment with MVM is going to be pretty interesting. Apparently, there’s already some form of MVM support in Maxine – we’ll find out for sure in June.

During the conference I had a meeting with Cay Horstmann, and at the end of the meeting Josh Bloch saw Cay and wanted to talk to him about the BGGA closures proposal for Java. Turns out that Josh has an entire slide deck which consists of a stream of examples where BGGA does the wrong thing, generates really cryptic error messages, or requires an unbelievable amount of code. The fact that BGGA depends on generics, which are already really hard, doesn’t give me much confidence about closures in Java. If you are a statically typed language fan, I think that you ought to be worried about whether Java, the language, has any headroom left.

The last session that I went to was Cliff Click and Brian Goetz‘s session on concurrency. Unsurprisingly, the summary of the talk is “abandon all hope, ye who enter here”. I was glad to see a section in the talk about hardware support/changes for concurrency. The problem is that concurrency is going to introduce end-to-end problems, from the hardware all the way up to the application level, and I think that every stop along the way is going to be affected. Unlike sequential programming, where we are still largely reinventing the wheels of the past, there is no real previous history of research results to be mined for concurrency. Hotspot and other VM’s are close to implementing most of the tricks learned from Smalltalk and Lisp, but those systems were mostly used in a sequential fashion, and while there were experiments with concurrency, there was much less experience with the concurrent systems than the sequential ones. Big challenges ahead.

JavaOne 2008: Part 1

1 Reply

JavaOne is a pretty intense experience, simply by virtue of the size. If CommunityOne was twice the size of OSCON, then JavaOne is three times the size of OSCON, and it shows . There was an immediate change in feel and atmosphere once JavaOne got into full swing. You could barely move sometimes, and there were a bunch of people whose job was to corral the crowds into some semblance of order.

JavaOne 2008

As a Sun employee, I was on a restricted badge, which made it hard to get into sessions (you are basically flying standby). On the other hand, I had plenty to do. I participated in a dynamic languages panel for press and analysts (who have their own track), which was pretty fun. The discussion was lively enough that we could have gone for another hour. There was one persistent fellow who really wanted there to be just one language, or wanted us to declare language X better for task Y. When I got started in computing, people learned and worked in several languages. Its only been recently that a language (Java) was popular enough that people could just learn one language, and the growth of web applications pretty much guarantees a multi-language future because of server side and client side differences. In the end, we’re back to finding and using the best tool for the job, or at least the most comfortable tool for the job. This is probably going to cause heartburn for big IT shops, but developers seem to be happy about it.

I took a walk through the Java Pavilion with Tim Bray one afternoon. He got into the AMD booth’s aromatherapy display (and yes, he has a similar shot of me doing the same thing). One of the highlights of that excursion was Tim introducing me to Dan Ingalls, who made a number of very substantial contributions to Smalltalk, including its original VM and the BitBlt graphics operation. I am a great admirer of the work that was done in Smalltalk, and it was an honor to meet Dan and have him explain the Lively Kernel to me. A short (and probably not quite fair) description of the Lively Kernel is to take the lessons learned from Smalltalk/Squeak and implement them in the browser using Javascript, AJAX, and SVG.

Unsurprisingly, I got the most value at JavaOne from the networking. And that means dinners, hallway conversations, and yes, the parties. Usually when I go to conferences, I am just a party attender. This time, I also worked at some of the parties. It was a little different to walk around the SDN party wearing a t-shirt with “SDN Event Staff” painted large on the back. I still had a good time. Between the T-shirt and the camera, I definitely had some good conversations.

Another benefit of being at a huge is company is that they can really throw a big party. Like hiring Smash Mouth to play for a private concert:

I’ve uploaded the rest of my photos from the conference to this Flickr set.

I actually do have some technical commentary, but I am going to put that into another post.

CommunityOne

3 Replies

Live or semi liveblogging conferences has been getting more and more difficult for me to do. The combination of meetings, networking/parties, and photographs means that it takes longer to assemble the requisite material. Here’s a bit on CommunityOne, which took place on Monday.

Many people (mostly Sun folks) have been asking me if this is my first JavaOne. My answer is, “it’s not, but it is my first one in ten years”. It’s been quite some time since I’ve been to a conference run by a big company like Sun (as opposed to an O’Reilly or open-source community conference). Even though the basics are the same, I definitely feel a kind of culture shock. I was asked to be on a panel during the general session, first thing in the morning, in order to get miked up and to run though the flow. Production values are much higher than I am used to. I keep thinking of CommunityOne as a small event, but in reality it is huge. I am told that registration was around 5000 people, which is twice the size of OSCON, which is the largest conference that I’ve been to in the last 4 or 5 years. Some pictures might help with the scale and production values:

The panel was on community models, although the content was closer to the edge where companies and open source communities meet/collaborate/fight. I think that I had two or three chances to speak, including the final set of remarks before the close of the panel. I have some more thoughts on that topic, but they are deserving of their own post, so that will be showing up after JavaOne is over.

Probably my favorite thing that happened at CommunityOne was the demonstration of ZFS’s reliability in the face of hardware failures. Sun Fellow Jim Hughes has demonstrated this a few times at Sun Tech days, and I’ve been meaning to write about that. I got to meet Jim before the keynote, and I had a very good seat to observe the hardware failure.

Jim usually destroys 2 of the drives in the ZFS pool, and it looked like Rich Green (EVP of Software) was going to get to smash the other one, until Jeff Bonwick, the inventor of ZFS, showed up to do the honors himself.

Smashing things makes for cool demos – you can watch the video replay if you like.. I’ve been paying more attention to ZFS ever since Theo Schlossnagle sat with me and a few other people in a bar at ApacheCon in Atlanta last year. We were talking about the voracious storage needs of photographers, and Theo was really singing the praises of ZFS. There were so important things that happened to ZFS for OpenSolaris 00805 (which was launched at CommunityOne). The most important is that you can now boot off of a ZFS volume. I hope (but don’t know for sure) that the work that made this possible will make it possible for Macs to boot off of a ZFS volume. My photo storage is getting all fragmented, and I could really put ZFS to good use. I suppose that I could build a ZFS storage appliance based on OpenStorage, but at the moment that is more work that I want to do.

I spent much of the rest of CommunityOne at the Redmonk unconference. I was drafted for an impromptu discussion on dynamic and other programming languages, which included a drop in from David Pollak, developer of the very cool lift framework for Scala, and organizer of the Scala liftoff which is happening on Saturday, right after JavaOne. There was also a very active session on Twitter – probably the biggest of the unconference. ~~Jim~~ Jay ~~Evans~~ Edwards from Twitter came along to participate in that one

I have a bunch more photos from CommunityOne. At the rate that things are going, I will probably just do a single post on JavaOne. There are plenty of other people doing liveblogging, for those who need a bigger information flow.

Update: corrected Jay Edwards’ name. Thanks to @monkchips

Next Stop: CommunityOne/JavaOne

The Open Screen project

5 Replies

Around this time last year, Adobe open sourced its Flex framework for rich internet applications. Today Adobe announced the Open Screen project, which encompasses a number of things, probably most importantly, the removal of the license restrictions on the SWF file format used by Flash. The other aspects of the announcement relate to Adobe’s Flash Player, and while they are steps towards openness, Adobe’s player will remain closed. The importance of opening Adobe’s player has decreased because dropping the file format licensing should make things easier for the Gnash folks. The worry then is that we’ll end up with incompatible versions of Flash, which is in almost nobody’s interest. That’s probably the next problem that needs addressing.

(Finally) upgraded to Leopard

6 Replies

At the beginning of the month I got my work laptop, one of the early 2008 MacBook Pro’s. It came wit h 10.5.2 preinstalled, so I took the opportunity to upgrade to Leopard. It’s been an interesting experience sitting on the sidelines watching everyone else explore the new features, but I don’t regret it. I had a few hiccups moving things over, mostly related to installing Python packages. But I had fewer migration problems than I expected, especially given how customized a setup I am running.

Spotlight is actually fast now. It’s not absolutely lightning fast, but it is now fast enough to be usable, at least for me. This might also be partially related to the 7200RPM disk in the new laptop, but either way, I am happy, because this is the thing I wanted the most from Leopard.. Well, except for the fact that mds still sometimes shows up as the top CPU consumer on the machine.

Mail.app improved a lot. There was a bug in the reply all command that made moderating Apache mailing lists a pain. Gone. Account setup, and more importantly, self-signed certificate support is hugely improved. No more trips to Keychain Access to make things work. If only it could remember the position of multiple viewer windows across restarts.

I used one of the virtual desktop managers on my Powerbook and then stopped, so I didn’t expect much out of Spaces. It turns out that spaces is actually pretty good. It works with multiple displays and plays well with Expose. I’m using it all the time I just wish there were shortcuts to go to a particular Space.

Against my better judgment, I’m using Time Machine for primary backups at the moment. This is due to an unfortunate fragmentation of external Firewire disks. After JavaOne I need to spend some time rearranging stuff on various hard drives, so that I can put a SuperDuper! backup into the rotation again. I don’t yet have complete faith in Time Machine’s reliability. Just yesterday, I successfully used Time Machine to save my bacon, and then later in the day, a Time Machine backup just failed with an error. Running Disk Utility over the backup disk didn’t find any problems, and after a reboot, backups seemed to work again. Doesn’t exactly inspire confidence, though. I also tried to use a USB disk that is attached an Airport Extreme, and Time Machine could never finish its initial backup. I like to work with a large display, so wireless Time Machine is not a huge deal to me, but it really ought to work. It’s likely to be several more point releases of 10.5 before I really feel like I can rely on Time Machine.

Quicklook is useful when I remember to use it. I’m getting tons of PDF slideshows in e-mail, and Quicklook is good for those. I really wish there was a Quicklook plugin that work on OpenOffice documents.

The next one is dumb but true. I really like having pictures for my desktop backgrounds. The translucent menu bar isn’t a factor in this. Apple provided a nice set of pictures for the backgrounds, and I have it set to rotate the background fairly often. All I need to do now is find some time to pull some of my own pictures into the rotation. The new animations for the picture folder screensaver is also cool – it drops the new pictures on top of the older ones, so that over time your screen looks like a pile of photographs.

On the infrastructure front, things seem a mixed bag. Things are snappy, but the machine also has 4GB of RAM in it. I have yet to see a kernel panic, but it seems less stable than 10.4. I’ve had a bunch of little problems: the issue with the Time Machine disk, loss of sounds after using Front Row to play a DVD, a bug that makes it impossible for file moves to work correctly in PathFinder, and syslog running amokfor no good reason. I had to turn off Growl because it was causing NetNewsWire refreshes to clog up. At the moment, anytime the machine makes a noise, the screen flashes (and yes, I did check the preferences for Universal Access). I’ve also been having a problem with my Firewire 800 disks just disappearing, sometimes during operations. Turning off one of the drives seems to be helping the problem, but that’s 500GB of storage that is offline. It makes me wish (again) that ZFS was going to be in 10.5.3, but alas, it will not. In any event, 10.5.3 will certainly be welcome. All these little problems are really starting to annoy me.