Category Archives: java

CommunityOne / JavaOne 2009

This was my second year attending these events as a Sun employee. Everything in software at Sun seems to revolve around these two events. For quite some time before the show, people are working away furiously getting things ready to be unveiled, myself included.

CommunityOne

For me, the big theme at CommunityOne was cloud computing. Sun itself was emphasizing cloud stuff and the latest release of OpenSolaris, 2009.06, which were the main topics of the CommunityOne general session. The Sun Cloud is due out sometime this summer, so much of the cloud part of the session was having partners come up and tell about their experiences working with our cloud. The OpenSolaris team has done a huge amount of work in 2009.06. The feature that stuck out to me the most is “Crossbow“, which is a completely rewritten networking stack. Solaris already had CPU virtualization technology built into it via the zones feature. Crossbow makes it possible to virtualize networking configurations. This means that you could run an instance of OpenSolaris on your laptop (either natively or via VirtualBox, VMWare or whatever) and actually have a virtualized data center configuration running right there. That’s pretty interesting stuff.

I went to several cloud sessions, and I’d have to say that the current state of cloud computing is pretty rough. At least that’s true at the Infrastructure as a Service level where Amazon and the Sun Cloud are. As an example, I went to a good presentation by fellow Sun employees on cloud computing patterns. I happened to be sitting with James Governor and Stephen O’Grady of Redmonk, and I turned to Stephen and said “these patterns are all at a level that I never want to have to worry about”. The patterns themselves were fine, but I personally don’t want to have to deal with things at that level in a cloud platform. There is lots of room for improvement and innovation in this space.

My CommunityOne talk was called “Programming Languages for the Cloud”.

Programming Languages For The Cloud

The talk is based on my experience as a language guy who has been asked to work on cloud computing stuff. As such, I’m really trying to raise questions (for which I don’t yet have answers) about places where work on programming languages might usefully intersect with cloud computing. I figured that this would be a niche kind of talk, so I was very surprised to find myself in one of the larger rooms at Moscone, complete with a live video feed. I was even more surprised to see that the room was pretty full. After the presentation, one of the Salesforce.com engineers working on Apex (their domain specific language for the cloud) came up to the front. We ended up having lunch and I learned a bunch interesting stuff about their experience withe Apex. This sort of thing is what makes conferences worthwhile.

JavaOne

I spent the first day of JavaOne prepping for my presentation, “Seeding the Cloud”, which was about some ways that tools could help developers who choose to build applications in the cloud.

Seeding The Cloud

Ashwin Rao and I had some pretty interesting demos lined up, but we had problems with the internet connection in the room so a number of the demos failed. I learned later that the internet connection for all of Moscone Center had gone out, which made me feel slightly better. As someone commented to me, it was a good illustration of some of the weak points of the cloud (web, really) model.

The demonstration that I really wanted to show was an extension of some work that the Kenai team has done. Kenai is going to have support for doing continuous integration via Hudson, and the machines for doing that can be allocated as cloud instances. This is great if you have a project in Java or some other language that has major build steps. Another use for a dynamically allocated farm of machines is to do web UI testing on browser combinations. Back at PyCon, I put a bug into Adam Christian and Mikeal Roger‘s ears about this. Adam and Mikeal are the primary guys behind the Windmill web UI testing framework. Adam has been working with Hudson author Kohsuke Kawaguchi, and between the two of them they came up with a way for Hudson to start up a bunch of different browsers on different operating systems. If my demo had worked, people would have seen me kick off a Hudson build from inside of Netbeans 6.7, and then we would have watched (via RDC) the various browsers running though some UI tests on a web application. Oh well.

I spent the rest of JavaOne ducking into various language and concurrency talks. Jonas Boner gave a very nice talk comparing some of the concurrency mechanisms that are available on the JVM. Alex Miller gave a talk on Java concurrency gotchas. The net effect of Alex’s talk was to reinforce the fact that we need one or more of the mechanisms that Jonas covered in his talk. Also in the concurrency vein, I stopped in on Philipp Haller and Frank Sommer’s ‘s talk on Scala Actors. Probably the most fun concurrency thing was a random conversation with Clojure author Rich Hickey and Jonas Boner in the speaker room.

JavaOne is big on the keynote / general sessions. I only went to two, the opening session, and Bob Brewin’s technical keynote. The big news (to me) in Bob’s keynote was Mark Reinhold’s demonstration of a modularized JDK. This is cool for a variety of reasons, like reducing the footprint of the downloads, ability to build distribution packages trivially, and so forth. But the thing that made me happies was news that the CLASSPATH is finally going way, to be replaced by a module-info.java file.

The opening general session was very subdued. There were a variety of partner / sponsor segments, but things were really running at a low energy level until the end when Scott McNealy took the stage and then introduced Oracle CEO Larry Ellison. Despite Ellison’s reassurances to the Java community, it was a sad moment. I’ve only been at Sun for a little over a year, but my history with the company is pretty long. When I was in grad school, Brown was on of the first large installations of Solaris (replacing SunOS). Like many developers, I’ve used Java over the years. Sun has made a number of very important contributions to the computer industry, and it’s sad to me that a company so full of innovation was unable to remain independent.

Photography

This year things were so busy and frenetic that I really didn’t have much time to pull out the camera. Between presentations and meeting up with Sun people from all over the world, there just wasn’t time. Here are a few from the few times that my camera escaped its bag:

Bob Brewin’s Technical Keynote

The Extra Action Marching Band on the CommunityOne Expo Floor

The CommunityOne Party

DSLDevCon 2009

2 Replies

I’ve been having trouble coming up with a good summary of the (Domain Specific Language) DSL DevCon. That’s partly because there was a lot of information to absorb between Lang.NET and the DevCon. Even more so, I’m finding it hard to distill what I saw, what I didn’t see, what I wanted to see, and what I think we need to see next. That’s odd because I’ve accepted the notion of DSL’s should be a part of the programmer’s toolbox ever since I sat through the “metalinguistic abstraction” section of Sussman and Abelson’s MIT class in 1984.

Reporting

I’m going to call out four talks that really stood out for me. There were more than just these four, but it was either these four or all of them, and all of them is too much work.

Guillaume LaForge’s talk on Groovy DSL’s was important because he not only showed how to build DSL’s using Groovy, but he’s actually working with real customers, like Mutual of Omaha, who are using those DSL’s in production.
I was happy to hear Markus Voelter’s talk Textual DSL’s and Code Generation with Eclipse Tools because a lot of the noise that I’ve heard on the DSL front has been coming from the Ruby and .NET side of the world. One thing that got my attention at the DevCon was the importance of tooling, so it was good to see that there are some tooling efforts in the Java space. It’s too bad that no one from JetBrains was there to present on MPS.
Brad Cross and Ted Neward did a talk entitled “Functional vs. Dynamic DSLs: The Smackdown”. I came away from this talk wanting more, and not in a good way. Brad and Ted really needed about 2 hours in order to give all the relevant background a chance to settle in. During the talk they presented a set of things which differentiated the functional programming and dynamic language styles of creating “Internal” (I really dislike the Internal/External terms) DSLs. Unfortunately, there wasn’t enough time to really dig in and explore the meat of what they said. I think that a deep addressing of the points that they made would be a very important contribution to the DSL topic. Maybe we’ll get to see a series of blog posts, developerWorks articles, or even an academic paper of some kind.
I view Intentional Software as one of those grand computer science projects. Having worked on Chandler, I have an appreciation for the perils of large, grand efforts. This is the first time that I had a chance to see a presentation by anyone from Intentional Software, and it is just as well that it was a demo of their just shipped product. I took note when the Intentional Software project was started back in 2002, but I’ve not heard a lot about their progress since then. What we saw was a demonstration of a production version of their “Domain Workbench” which is a system for allowing domain experts and programmers to work together to build a system which domain experts can then use to write software. Instead of writing programs, the programmers write the generator which takes the domain language (which can be visual) and then generates code. The system represents the domain information in a way that allows multiple, editable, “projections” (views). The demonstrations that we saw included an actuarial workbench, complete with mathematical notation, and an electronics workbench, expressed as circuit diagrams. If you are interested, your best bet is to watch the video when the videos are posted.

I am pretty impressed with what I saw, but there are lots of questions. How many domains can this actually work for? How hard is it to write generators? What’s the business model for domain workbenches? It seems pretty clear to me that for the domains and organizations where this can work, this approach is going to have a pretty sizable impact. Perhaps not this year, but within the next 5 years. I have to hand it to the Intentional Software guys. Their presentation was pretty low key, and they are going out of their way to not hype their stuff. They plan to work with a small number of customers to gradually prove out their approach. In an area which is highly susceptible to hype, it was refreshing to see people trying to keep expectations to a reasonable level.

The DevCon (and Lang.NET) were also my chance to meet two people who I’ve followed for sometime from afar: Ted Neward, and Larry O’Brien. Ted is well known and I’ve been following his blog for some time. He’s local to the Puget Sound area, and it’s probably just bad timing that we never met before this week. Larry O’Brien has been a commenter on my blog, as well as a responder on Twitter. I’ve appreciated his blog as well as the columns that he’s written over the years. It was great fun to run to the back of the room after each talk and see what the Twitter cabal (which included Larry) had to say about the material we had just seen.

Analysis

I think that DSL’s are inevitable. It’s remarkable to me how prescient Abelson and Sussman were when they defined three categories of abstraction: control abstraction, data abstraction, and metalinguistic abstraction. If you look at some of the recent frenzies in languages, you’ll see that we are mostly improving the ability of various languages to perform various kinds of abstraction. These concepts are not new, but they are appearing in languages which are approachable by today’s practitioners. Object oriented programming? Data abstraction. Closures? Control Abstraction. Pattern Matching/Algebraic datatypes? Data and control abstraction. DSLs and the capabilities needed to enable them? Metalinguistic abstraction.

Language as an abstraction is very powerful, and requires support from the underlying language as well as the tools. These two topics (as well as specific examples of domain specific languages) were the focus of the DevCon. The audience makeup appeared to be mostly language and compiler geeks. There were a few people (mostly consultants as far as I could tell) who write business applications, but this group was pretty small. This is important because most of the DSL’s presented were aimed at very computer science kinds of domains. If DSL’s are to have a broader impact, then it would be great to see more business people at events like this.

One thing which was not addressed at all was the process end of this. In order to build DSL’s for non computer domains, there has to be a collaboration between developers and domain experts. The Intentional Software guys recognize this via some “groupware” to facilitate this process. However, tooling alone is not enough to bridge this gap. I hope that we’ll be hearing reports on the process of collaboration between developers and domain experts as more and more people build DSL’s.

This is an interesting space, from a technical point of view. There is lots of cool language design and compiler stuff, some of my favorite topics. On the business end, it seems like there are some decent sized opportunities here, and that tooling is going to play a very large role — language support for DSL’s will be important, but may be overshadowed by the importance of good tools.

Update: the videos are now avaiable

Lang.NET 2009

5 Replies

Back in September, I attended the JVM Language Summit. This week I’ve been doing the Bainbridge Island to Redmond ferry commute in order to attend the Lang.NET symposium. Here’s some of the stuff that stood out to me.

Hopscotch

Gilad Bracha talked about Hopscotch, the IDE for Newspeak. Newspeak is one of those things that’s on my very long list of things to look at. For a Smalltalk or Lisp developer, much of the talk was familiar — an illustration of the power of writing the IDE for a dynamic language in a dynamic language. In fact Bracha asserted that the true power of Smalltalk is the dynamism in the IDE itself. I think that’s probably true today, since almost all popular IDE’s are written in a static language. The other interesting feature of Hopscotch is the pervasive use of a web browser style UI in the IDE. This looks promising, and gives me hope that a web browser based IDE might be feasible.

Newspeak is implemented atop the Squeak VM, and when questioned about this choice, Bracha said that it would be much harder to implement Newspeak on either the CLR or JVM (and he’d be a position to know about the JVM in particular). He was hopeful that John Rose’s work on dynamic extensions for the JVM would take out some (but not all) of the pain.

Powershell

At the JVM Language Summit, Rich Hickey gave a 15 minute talk that left me wishing he had been given 30 minutes. Jeffrey Snover’s 15 minute talk on Powershell left me feeling the same way. Powershell may not be a monad toting concurrency friendly uber functional language on the CLR, but it was compelling to me because of it’s sheer practicality and usefulness. I’ve been following Powershell via blog posts since it was announced, but seeing it demoed by its designer is another thing altogether. The Powershell folks have done some nice work to present shell users with a very consistent user interface. A side effect of this work makes it easy to make GUI’s that can output Powershell commands, and apparently in future versions of Windows, the GUI admin tools will do this. In addition to rationalizing the user visible behavior of the shell, Powershell is able to do some very impressive stuff in terms of remote execution, limited/secure execution, single machine transactions, and more.

I spent some time talking with Snover later in the day, and he seemed to think that UNIX shells could gain a fair amount of PowerShell’s capabilities by recognizing that pipes ship bytestreams, adopting a data format (like JSON or XML) for those byte streams, and proceeding from there. That might be true technically, but that would be a huge cultural change for that community.

Lars Bak on V8

Lars Bak talked about his work on V8. V8 is the nth VM that Bak has worked on, and my main takeaway from his talk was that the V8 team has tried to do a lot of careful measurement of real (and anticipated future) Javascript programs. They’ve then turned around and used those measurements to guide them in selecting from the techniques developed for Self, StrongTalk, and Hotspot. None of those techniques would be a mystery to people familiar with the literature. But the insight into this all applies to Javascript was pretty interesting.

Since this is all about performance, it’s interesting to note that V8 has just about doubled its performance during its one year life. Bak predicted that by 3-4 years in perrformance will have quadupled. There was a funny moment where someone asked about comparison’s with Firefox’s TraceMonkey tracing JIT. Bak’s reply was “let’s run the (Google V8) benchmarks“, which he then proceeded to do (V8 won). Bak is clearly competitive about this, which can only mean good things for Javascript users.

F# is something that’s been at the edge of my radar because of asynchronous workflows. Lang.NET was a chance for me to get a quick education on what that’s all about. It turns out that async workflows are an application of computation expressions, which are a way of doing monadically inspired language extension. Async workflows end up being a way of using CPS style tasking but with a nice syntax. Most of this became clear during Tomas Petricek’s talk on Reactive pattern matching for F#. The code that Petricek showed was very evocative of actor style code and the use of F# pattern matching was evocative of Erlang’s use of the same. One interesting point was the use of join patterns, something that doesn’t have a direct analog in either Erlang or typical actor implementations.

Amanda Laucher‘s talk Concurrency for F# was a discussion of a client engagement that she did using F# and the concurrency features of the language. The application in question was an insurance application, and the speedup numbers looked pretty interesting. This is notable because most of the concurrency examples that you can find are outside the domain of “business programming”. Having this kind of data is really useful for people interested in the concurrency space. Amanda’s work was done as a Microsoft case study, and published version will be available at some point.

Erik Meijer

Erik pretty much gets his own section because he’s that entertaining/interesting. This time he was talking about the Livelabs Reactive Framework, where he described a way to use LINQ for distributed computations, using the specific example of an AJAX dictionary based suggestion function. Erik started by using (throwing) coins to illustrate the IEnumerable and IObservable interfaces. He then used mindless symbol pushing to demonstrate that these two interfaces were duals of each other, and that they obeyed the rules for monads. That being true, he then showed how you could define observable collections which could be used with LINQ.

Tidbits

Herman Venter talked about a Common Compiler Infrastructure for the CLR. There’s some nice stuff in here if you are implementing a language on the CLR. One user of this infrastructure is the Code Canvas, a “spatial development environment”. It’s good to see that people are stretching the notion of what a development environment should be. The thing that stuck out to me most from this talk was not the technology (which is good), but Venter’s call to action at the end. The CCI is being open sourced, and Venter’s call for participation and contribution was indistinguishable from the same end of talk call at any open source conference. A sign of changes underway.

Joshua Goodman did a product rundown on What’s new in CLR V4 for Languages. There’s several years worth of changes to the CLR queued up for V4, which will be a big update. Most interesting to me was the inclusion of a Hill Climbing algorithm for allocating / managing threads. This strikes me as a kind of policy decision, something which you might want to plug into the CLR as opposed to having it installed under the hood. It will be interesting to see how this ends up working out in production settings.

Deja Vu

In a previous life, I did some work on database programming languages, and I experienced deja vu twice during the conference. The first was during Philip Wadler’s Links: Web programming without tiers. The goal of Links is enable front end and back end web programming using a single language. Wadler didn’t discuss the front end part during his talk. He focused on the back end, particularly on the ability to translate programming language code into SQL, in a fashion slightly beyond LINQ. This is done using a combination of the monadic techniques used for LINQ, as well as a type/effect system which is used to help determine whether a piece of code is SQLizable or not. The work was eerily familiar, and when Wadler quoted Limsoon Wong and Leonid Libkin’s work on Kleisi, I knew why. We looked at their work on structural recursion (and comprehension notation) for our work on list and tree queries.

The last talk(s) of Lang.NET was a double header talk on Oslo/M by Paul Vick and David Langworthy. David and I were officemates while we were graduate students at Brown. Even more deja vu was induced when I saw David demoing how the M toolchain could generate SQL schemas and queries. As he put it to me afterwards – “this is all that stuff we were talking about 10 years ago”. Well, closer to 20 now, but who’s counting. M is in its very early days – I think it will be more interesting when it can generate CLR code to match the SQL.

Last bits

Thanks to Harry Pierson for making sure that I got an invitation to Lang.Net. People were very welcoming, even though I had to ask questions about various bits of Microsoft alphabet soup. I suspect that this is the only conference I will go to all year where Macs are the minority. It’s probably good for me to get out of the bubble every once in a while…

Overall, I found the topics to be a little more broad ranging than the JVM Summit. There were quite a few talks on tools, while there were comparatively fewer at the JVM summit, and those tools discussed at the JVM summit were really aimed at language implementors. One thing that worked well with the JVM Summit was allowing for some open space sessions so that attendees could get together and discuss topics of mutual interest, often based on what had been presented during the sessions. I think that some similar time slots would have enhanced the experience.

I definitely came away impressed by some of the work going on in the CLR ecosystem, and I hope that the exchange between the JVM and CLR communities will continue and expand.

Update: Videos of the talks are now available.

The First Annual JVM Language Summit

10 Replies

You know that a conference is good when you go home with a list of stuff that you never heard of but now need to go follow up on. The JVM Language Summit was exceptional in this regard. Sun provided a location and a few of the speakers, but most of the speakers at the Summit were not Sun employees, although there were a few Sun alumni amongst the speaker ranks. The topics that were discusssed went all the way from type theory (including the usual greekified type proofs), typical language design stuff, VM design, all the way down to discussions of how high allocation rates can cause hot data to get flushed out of caches on the bare metal. Slides for all the talks are available on the wiki for the conference, and some of the talks will have video at either InfoQ or YouTube. Here are some of my favorites from the three days.

Clojure

I’d been aware of Clojure prior to the summit and had looked at the page on Clojure’s use of persistent data structures, so I thought that I had some idea of what was interesting about Clojure. I was wrong. Rich Hickey’s 30 minute presentation on Clojure had a really large amount of information per unit time. By the end of the time I was really interested in Clojure, and I was able to find out a bit more about it by going to an open space session and by being at the same table as Rich during dinner one night. As and old Lisp guy, my usual reaction to Lisps on the JVM or CLR is why? They don’t typically fit in that well with the host VM, and there are great implementations of Common Lisp that can compile to very efficient machine code. I was looking forward to Arc, but that has turned out to be very disappointing. Clojure has taken a very practical approach to the Lisp parts of the language. It fits in very nicely with the JVM, is able to call Java code easily and has the potential to achieve very good performance on the JVM. Also, Rich has made a number of design decisions which improve the syntax (he showed a short program in both Python and Clojure, and they occupy the same amount of vertical space and have roughly the same visual density) of Clojure. He’s also generalized many operations that would have worked on lists to work on sequences, which really means any Java sequence type. Like many Lisps, you can supply data type hints, and the compiler will use those to make the program more efficient. There is a nice library of collection operations, which look very comparable (or better) than Python or Ruby’s facilities for collection types. There are some other really interesting data structures in the libraries, like bit partitioned hash tries and zippers.

Beyond the Lispish stuff in Clojure, there are a several interesting features in Clojure related to the problem of concurrency. In Clojure things are immutable by default, which is a huge benefit – a benefit shared by functional languages, and quasi functional languages like Erlang. Beyond that, Clojure supports persistent data structures as a way of managing state in a concurrency friendly manner. The idea is that “updating” a persistent data structure yields both the version before and after the update. This means that updates don’t impede readers of the old version, and are not blocked by readers of the old version.

Lastly, Clojure provides an interesting mechanism for utilizing Software Transactional Memory. Normally STM systems make all accesses to memory transactional. This makes the STM a bottleneck, and makes it much more likely that the performance of the STM system will be the limiting factor in a concurrent system. Clojure requires you to make uses of the STM explicit via its Ref structure. This yields a potentially much more controlled usage of the STM, which could help preventing the STM from being a bottleneck.

My original impression of Clojure was that it was still in the very early stages, but it seems to be bit further along that that. I was surprised by the size of the community, and by other parts of the ecosystem, like the tool support. There are several Emacs modes, integration with SLIME, and even a Netbeans plugin for Clojure.

I will definitely be giving Clojure a closer look, and I am not alone. There was a lot of energy in the room during and after Rich’s talk, and there was a burst of Twitter traffic during the talk. It’s pretty interesting to see the number of language geeks on Twitter.

Davinci Machine

If you’ve been following John Rose’s blog and Charlie Nutter’s recent writings on invokedynamic, you wouldn’t be very surprised by the content of John’s presentation on the DaVinci Machine Project. This is a highly important piece of work for non-Java languages on the JVM, so it was good to hear John tell a more complete version of what he’s been up to. It was also my chance to meet John in person. We somehow missed each other at JavaOne, so it was good to put a face to the name, and have some in person contact. John and Brian Goetz did a great job of organizing the summit, and John was always trying to find out what kind of features would be useful to JVM implementors. JSR-292 can’t happen soon enough.

Fortress

David Chase talked about the work that folks at Sun labs have been doing on Fortress. I never really paid much attention to Fortress, since they are really aiming at the scientific, high performance computing space, and that’s kind of outside of my interests. The Fortress guys are doing some interesting explorations as far as concurrency is concerned. In fact, David referred to Fortress as “infested with parallelism”. My todo item from the Fortress talk has to do with the work-stealing model that they have for concurrency. Apparentl this work is based on a data structure known as an ABP queue, so I’ll be tracking down the paper on that one.

JRuby

I’ve heard Charlie Nutter talk about JRuby several times, and have talked with him a little about JRuby. Even so, his talk on JRuby was really interesting, because he was able to go full out for an hour to a very sophisticated audience. I know from talking to some of the Jython guys, that there were a few aha moments for them, even though they’ve been to the same talks that I’ve been to.

Dynalang MOP

Attila Szegedi described his proposal for a MOP for dynamic languages. Once you start hosting a bunch of languages on the JVM (or any VM), then people start to ask if they can call code written in language A (say Clojure) from language B (say Python). The tough part is that the code in language A may have compiled to Java bytecodes in a way that doesn’t really resemble Java code, and you can end up in a situation where B can call A but does it by grabbing things which are really artifacts of A’s implementation. Of course, A’s implementors will continue to improve A, and in the process of doing so, might change the details. You can see what the problem is going to be. Attilla’s MOP would go a long way towards helping here. I hope that people will give it a serious look.

Gradual Typing

Jython committer Jim Baker has been after me about the work that Jeremy Siek (UC Boulder) has been doing on adding types to Python. His system is called gradual typing and allows a programmer to selectively add type annotations to a program. It’s a cool idea in principle, and I hope that it will end up being cool when it finally gets implemented all the way. I have to admit that the first time that I saw an annotated program, I had a violent reaction. There were a ton of angle brackets due to the type annotations. Jeremy and his students are working on ways the reduced the amount of notation that is needed. I hope that they’ll be successful — in Python at least, it’s going to be key to whether people will adopt it or not.

Fundamentalist FP

I’ve been an admirer of Erik Meijer‘s work for some time, so I was glad to be able to hear him speak in person. There was another talk on LINQ, so Erik didn’t talk about that. Instead he talked about what he called “Fundamentalist Functional Programming”, which is really just the functional programming that the old school functional programming people have always talked about. I think that Erik is concerned about the amount of lifting of functional programming ideas and idioms, without a full understanding of the essence of functional programming. His presentation style is very entertaining. The major thrust of his argument was that for the past 50 years of computing, we have been abstracting, but abstracting over the wrong things. He asserted that the thing that we really need to do is to abstract over evaluation order. Given the coming many/multicore world, this is understandable, but don’t think that I agree that all the other lessons that we’ve learned about abstraction are invalid. He provided the simplest explanation of monads that I have ever heard or read, as well as showing how to handle things like object creation and process creation monadically. In the end, though, his talk reduced to the essentials of lazy pure functional programming.

Bytecodes for fast JVM’s

Cliff Click asked that JVM language implementors send him an implementation of a particular program written in their language. Cliff then ran those programs (in their respective languages) on Hotspot and on the Azul JVM. His talk was a report of his findings as to what was keeping various languages from getting the best results on the JVM. He said that he wasn’t trying to compare the merits of one language versus another, but more to give the implementors insight into what was up with their code. I found this talk to be tremendously interesting because Cliff really knows the guts of HotSpot and because he was able to be very specific about what was causing problems for the various languages.

Parrot

I’ve known Allison Randal for several years now, mostly via her organizing of the FLOSSFoundations meetings that happen every year at OSCON. In all that time, we’ve never really sat down and talked about her work on Parrot, and it’s been several years since I heard a talk on Parrot. I give John and Brian a lot of credit for inviting Allison to come and talk about Parrot. The architecture of Parrot is very unlike either the JVM or the CLR. They started with very different assumptions and goals, which unsurprisingly lead to a different design. As far as I can tell, Parrot is looking reasonable on the performance front, will be able to use the C libraries of Python, Ruby, PHP, etc, without much hassle, and will have a good story for interoperability between hosted languages. Control flow is modeled using continuations which means that continuations are really cheap to create. Allison also talked about what a different method of doing call site caching – Parrot does the caching in the Parrot class object, not in little caches strewn all over the call sites. This makes it easy to invalidations the cache when the class hierarchy changes, for example. I’m still trying to digest all of what I heard, as well as the conversation that several of us had with Allison after her talk.

The Parrot team has been lying low and working away on Parrot, and they are definitely making progress. Allison showed some very preliminary benchmarks of the incomplete Python, Ruby, and PHP implementations on top of Parrot versus the C based versions. She told me afterwards that the project has reached the point where they are working to time based milestones, and that they are hoping to do a 1.0 release early in 2009. Chalk up another to-do.

Random Thoughts

There aren’t any pictures to go with this report because I was not motivated to take any. There were several people snapping away quite frequently during the conference, and I didn’t want to add the slaps from the D3 to the cacophany and the flash light show.

It seems clear to me that many folks share some of the same problems, and I hope that on result of the summit will be that people will start to work together when it makes sense. I know that the Jython and JRuby folks are working in that direction, and it seems likely to me that there will be some collaboration around the dynalang MOP as well. There was a lot of good energy in the room: people were very respectful and curious about other people’s work.

I think that the only regret that I had was that this was the first annual JVM Language Summit. Imagine where we’d be if this had been the fifth…

Update: finished the sentences about persistent data structures in Clojure

Jython 2.5 Alpha 1

1 Reply

The Jython development team has released the first alpha of Jython 2.5. The guys were hoping to have this done in time for EuroPython, but it wasn’t to be. Still, they are ahead of where they planned to be, so this is good news. If you are interested in an up to date version of Python on the JVM, go kick the tires and report bugs. Getting more testing done should help speed the process of shaking out all the bugs.

Jython Users at Europython

5 Replies

I’ve been asked to moderate a Jython panel at Europython this year, and we are looking for Jython users to be represented on the panel. So if you are a Jython user and you are gong to be at Europython, please drop me a note or leave a comment.

Scala liftoff

1 Reply

I stayed around in San Francisco for one more day after JavaOne, in order to attend the Scala liftoff. The liftoff was an open space style conference (which has a more specific meaning than “unconference”, at least to me). My friend Kaliya Hamlin did a great job of facilitating the day.

Scala has steadily been gaining attention, and hasn’t yet hit (at least in my eyes) the hype part of the classic Gartner hype cycle. I’ve been poking about with Scala, mostly because of the type inferencing, the Actor library, and lift. I have great respect for the work that Martin Odersky has done over the years, which also has me interested. Couple that with what I learned about closures in Java at JavaOne, and the list of reasons to look more deeply at Scala is getting long, especially if you are determined to have a statically typed languages.

I wasn’t able to make it to any of sessions on lift. It just worked out that other sessions overlapped them in a pathological way. While this is unfortunate, I am sure that I’ll be able to pick up anything that I need from the mailing lists and other documentation. I was able to attend two sessions on actors. One of the sessions had people with questions about actors, but no Scala actor experts were in that group. There was some discussion of Pi-calculus and the join calculus, but no discussion of the actual actor theory.

Steve Yen’s session on actor-d was pretty useful. Steve set out to build a version of memcached using Scala’s actors. He spent most of his slot talking about Scala/Java isms that he ran into – this was important since he was comparing to the C memcached. By the time he got to the actor related stuff, he was almost out of time. Steve found that he had to remove actors from the main loop of his server in order to get sufficient performance. He wanted to get statistics from the server in the background and discovered that he main loop actor was always processing messages and was never idle long enough to report statistics. He ended up replacing the actor with plain old Java Threads (POJT?). This was in addition to all the fact that he ran into many of the standard Java problems as well. I’m not sure what to conclude from this. I don’t recall what kind of hardware he was on, and I am not convinced that he had the right architecture for an actor based system. Some of his experience also seemed contrary to what the lift folks have been claiming. I think that we are in for a decent amount of investigation here. One of Martin’s statements about Scala is that it is possible (and better) to extend the language via libraries than via actual language constructs. For the most part, I agree with this, but there are certain extensions which have interactions with the runtime – like concurrency. In those cases, I don’t see how the library approach allows taking advantage of runtime features. The current version of Scala actors is implemented as a library.

One of the things that I am currently working on is support for Python in NetBeans, so I dropped into the session on IDE support for Scala. With the exception of IntelliJ, none of the IDE plugin principals were present, so it was hard to have a really productive discussion. Martin did attend the session and we talked about the possibiliy of getting hooks into the existing Scala compiler, particularly the parser and the type inferencer. That could yield some big dividends for people working on IDE support. One IDE feature that I would like to see is the ability to hit a key, and have the IDE “light up” all the inferred types, overlaid on the existing program code. This would allow developers to see if their intuition about the types actually matched that of the type inferencer. I’d like a feature like this for Python/Ruby/Groovy/Javascript code as well. Further discussion was deferred to the scala-tools mailing list.

The other session that I participated in was the session on Scala community and governance. Several people wondered about this during Kaliya’s “What questions do you have about Scala” portion of the schedule building. When nobody else put up a session in this area, I grabbed a slot, hoping to spur some conversation – if for no other reason than my own education. Fortunately, Martin had already been thinking about the problem. He is going to adopt a Python style governance, with him (and EPFL) having the final say on language design matters. There will be Scala Enhancement Proposals (SEPs), like the Python PEPs. I’m very happy with this. I think that Python has done very well at maintaining the balance between (lots) of community input on the language design, while still retaining that “quality without a name”. One of the things that I said during the CommunityOne general session panel was that particular individuals in the right place, at the right time, matter at great deal. After watching Martin for the day, and seeing his interactions on the mailing list over the last few months, I think that the design of Scala is in very good hands.

We also talked about the evolution of the Scala libraries. The Scalax project is working to build a set of utility libraries for Scala. Martin views scalax as a place where anyone can submit a library, have it tested, vetted, reworked, etc. Eventually some code in scalax would be candidates for addition to the Scala standard libraries. This also seems like a sane approach to me. I like the idea of having a place for libraries to shakeout before going into the standard libraries. Martin also mentioned a LINQ in Scala project. I need to track that one down too.

It is good to be in a multi-language world again. There’s room for Scala, Python, Ruby, and others. Another language that I am keeping my eye on is Newspeak.

JavaOne 2008: Part 2

3 Replies

I’ve been to so many conferences and seen so many talks that it’s hard for me to really get excited about conference presentations. I went to talks here and there, but nothing at JavaOne was really reaching out at grabbing me (in fairness, this happens at other conferences also, so it’s not just JavaOne). Or at least that was true until the last day.

Friday opened with a keynote by James Gosling, who served as the MC for a train of presenters on various cool projects.

Cool stuff

First up was Tor Norbye, who has done a lot of good work on support for editing different languages in NetBeans. Tor has been working on JavaScript support for NetBeans 6.1, and he showed off some cool features, like detecting all the exits from a function, semantic highlighting of variables, and integrated debugging between NetBeans and Firefox. All of which was cool. When I was managing the Cosmo group at OSAF, I tried a bunch of Javascript IDE’s and never really liked any of them. I haven’t done a lot with NetBeans 6.1 yet, but I will. Tor showed one feature, which was the killer one for me. NetBeans knows what Javascript will work in which browser. You can configure the IDE for the browsers that you want to support, and this affects code completion, quick fix checking and so on. Definitely useful. Here are several more references on the Javascript support in NetBeans 6.1.

The Java Platform

It’s easy for me (and others, I’d bet) to think mostly of JavaEE or perhaps JavaME when thinking about Java. That’s understandable given the worlds fixation on web applications, and looking ahead to mobile. But the majority of the talks in Gosling’s keynote session had nothing to do with Java SE, EE, or ME (at least in the phone sense).

Probably the hit (applause meter wise) of the keynote was LiveScribe‘s demonstration of their Pulse Smart Pen. This is an interesting pen that records the ink strokes that it makes, and any ambient audio that it records while the writing is happening. The ink and audio can be uploaded to a computer, as long as that computer runs Windows (apparently a Mac version is in the works). Unfortunately, the pen works by sensing marks on a special paper (that would be the razor blades), so there’s a limitation on how useful this can be. The presenter said that a future version of the software would allow people to print their own special paper, but that’s still a future item for now. By reading special marks on the special paper, you get a pretty cool user interface. The pen itself can run Java programs, and there is a developer kit available for it. If they can get by the limitation of special paper, I think that this is going to be pretty interesting.

Sentilla showed off their Mote hardware, which seem like RFID chips that can run Java programs. except that these RFID chips can form mesh networks amongst themselves and can have various kinds of sensors attached. There are lots of applications for these things, going well beyond inventory tracking and such.

Sun Distinguished Engineer Greg Bollella demonstrated Blue Wonder, which is a replacement for the computers used to control factories. Blue Wonder combines off the shelf x86 hardware, Solaris, and real time Java to provide a commodity solution for factory control applications. This is far afield of Web 2.0 applications, but just as cool, in my mind.

By the end of the keynote I was reminded of the long reach of the JVM platform, something that I’d lost sight of. The latest craze in the Web 2.0 space is location data — O’Reilly has an entire conference devoted to the topic. I think that sensor fusion of various kinds (not just location sensors) is going to play a big role in the next generation of really interesting applications. The JVM looks like it’s going to be a part of that. I don’t think than any other virtual machine technology is close in this regard.

Java’s future

I also went to a talk on Maxine, a meta-circular JVM. By the twitter reactions of the JRuby and Jython committers, I’d say that Maxine is going to get some well deserved attention when it is open sourced in June. I’m particularly interested because the PI’s for Maxine worked on PJava, and MVM. Given the differences between the Erlang VM and the JVM, I think that the ability to experiment with MVM is going to be pretty interesting. Apparently, there’s already some form of MVM support in Maxine – we’ll find out for sure in June.

During the conference I had a meeting with Cay Horstmann, and at the end of the meeting Josh Bloch saw Cay and wanted to talk to him about the BGGA closures proposal for Java. Turns out that Josh has an entire slide deck which consists of a stream of examples where BGGA does the wrong thing, generates really cryptic error messages, or requires an unbelievable amount of code. The fact that BGGA depends on generics, which are already really hard, doesn’t give me much confidence about closures in Java. If you are a statically typed language fan, I think that you ought to be worried about whether Java, the language, has any headroom left.

The last session that I went to was Cliff Click and Brian Goetz‘s session on concurrency. Unsurprisingly, the summary of the talk is “abandon all hope, ye who enter here”. I was glad to see a section in the talk about hardware support/changes for concurrency. The problem is that concurrency is going to introduce end-to-end problems, from the hardware all the way up to the application level, and I think that every stop along the way is going to be affected. Unlike sequential programming, where we are still largely reinventing the wheels of the past, there is no real previous history of research results to be mined for concurrency. Hotspot and other VM’s are close to implementing most of the tricks learned from Smalltalk and Lisp, but those systems were mostly used in a sequential fashion, and while there were experiments with concurrency, there was much less experience with the concurrent systems than the sequential ones. Big challenges ahead.

JavaOne 2008: Part 1

1 Reply

JavaOne is a pretty intense experience, simply by virtue of the size. If CommunityOne was twice the size of OSCON, then JavaOne is three times the size of OSCON, and it shows . There was an immediate change in feel and atmosphere once JavaOne got into full swing. You could barely move sometimes, and there were a bunch of people whose job was to corral the crowds into some semblance of order.

JavaOne 2008

As a Sun employee, I was on a restricted badge, which made it hard to get into sessions (you are basically flying standby). On the other hand, I had plenty to do. I participated in a dynamic languages panel for press and analysts (who have their own track), which was pretty fun. The discussion was lively enough that we could have gone for another hour. There was one persistent fellow who really wanted there to be just one language, or wanted us to declare language X better for task Y. When I got started in computing, people learned and worked in several languages. Its only been recently that a language (Java) was popular enough that people could just learn one language, and the growth of web applications pretty much guarantees a multi-language future because of server side and client side differences. In the end, we’re back to finding and using the best tool for the job, or at least the most comfortable tool for the job. This is probably going to cause heartburn for big IT shops, but developers seem to be happy about it.

I took a walk through the Java Pavilion with Tim Bray one afternoon. He got into the AMD booth’s aromatherapy display (and yes, he has a similar shot of me doing the same thing). One of the highlights of that excursion was Tim introducing me to Dan Ingalls, who made a number of very substantial contributions to Smalltalk, including its original VM and the BitBlt graphics operation. I am a great admirer of the work that was done in Smalltalk, and it was an honor to meet Dan and have him explain the Lively Kernel to me. A short (and probably not quite fair) description of the Lively Kernel is to take the lessons learned from Smalltalk/Squeak and implement them in the browser using Javascript, AJAX, and SVG.

Unsurprisingly, I got the most value at JavaOne from the networking. And that means dinners, hallway conversations, and yes, the parties. Usually when I go to conferences, I am just a party attender. This time, I also worked at some of the parties. It was a little different to walk around the SDN party wearing a t-shirt with “SDN Event Staff” painted large on the back. I still had a good time. Between the T-shirt and the camera, I definitely had some good conversations.

Another benefit of being at a huge is company is that they can really throw a big party. Like hiring Smash Mouth to play for a private concert:

I’ve uploaded the rest of my photos from the conference to this Flickr set.

I actually do have some technical commentary, but I am going to put that into another post.

Leopard, Java, and Open Source

38 Replies

I haven’t gotten around to upgrading to Leopard yet for several reasons, probably the most prominent of which is that Lightroom doesn’t work correctly, and I’m starting to use it a lot (more on that in a later post, perhaps). But it’s not for lack of Java 6. I’ve been following the Java on Leopard thing with bemusement, but John Gruber’s most recent post sparked a few thoughts.

Since I haven’t posted in a while, let me remind you of the context. For a while I was a Java developer, but that was another life ago, and since then, I’ve been a Python developer, and am now a manager of Java (and Javascript) developers. I’d agree with John that Java is not directly important to the Mac. No important piece of Mac software that I am aware of is written in Java, and the only important (unless you count Azureus, which Mac folks would not) client side Java apps are Java IDE’s or custom corporate applications. So it is hard to make a compelling argument that a late Java is directly bad for Macintosh sales, which Apple is surely focused on.

Nonetheless, I do think that Java, and all those Java developers (who many in the Mac community look down their nose at) are important. Their pushing for Titanium Powerbooks and MacBook Pros helped (in a lot of situations that I am directly familiar with) to improve Apple’s credibility in development shops, which helped Apple get to where it is today. I might still be using Windows if I hadn’t gone to ApacheCon 4-5 years ago and started to see the Mac’s, which were being used by my Java developing friends.

Gruber says that Java is not made to “just build” on any Unix-like OS:

Several irritated Java developers suggested that I’d feel differently if it were a developer runtime that I personally cared about — that I’d be irate if, say, Perl or Ruby or Python were dropped or degraded in Leopard. But that’s not a good comparison; Perl, Python, and Ruby pretty much compile out of the box on Mac OS X. Apple doesn’t have to do much at all — at least relative to Java — to include them on Mac OS X. Why? Because that’s how these tools are designed and engineered — they’re made to “just build” on any Unix-like OS. It’s not Apple’s responsibility that Java isn’t like that — it’s Sun’s.

Actually, I don’t think that he is correct here. When I worked at Apple, one of the projects that I worked on was a port of Java 2 to run atop the Newton operating system. I personally wrote the driver code for networking and the file system, and I can tell you from first hand experience, that Java definitely builds fine on Unix like operating systems. That’s not the problem. The problem is where OS X is not a Unix like operating system.
The places where there seem to be problems are the places where Java needs to talk to Carbon to do all that client side GUI Java stuff. I don’t think that you can claim that Carbon is part of “any Unix-like OS”.

I do think that there is something important buried in the quote from Gruber’s post. Look at the difference between the runtimes that got “kept” in Leopard. Perl, Python, and Ruby (Let’s leave aside for a moment the sad truth that hardcore Python and Ruby developers end up installing their own local runtimes). Not only were these runtimes bundled, but 2 of the 3 were actually improved – things like bridges to Cocoa, DTrace probes, and so on. What’s a critical difference between these runtimes and Java? How did all these improvements happen? Many of them were done by people outside of Apple, on a schedule that was not Apple’s, but which coincided with Apple’s. The Ruby DTrace probes were done by Joyent, the Python Objective-C bridge was done by people outside Apple. Apple pretty much just had to pick up the changes that were made. How did this happen? Those runtimes are open source, as were all the improvements that I just mentioned.

A few years ago at JavaOne, Sun took a poll of Java developers to see if open sourcing Java was important to them. If I remember right, about half those developers said no. From where I sit, it looks like an open source Java would have contributed substantially to having Java 6 ready to go for Leopard. Today, Sun has opened up the source code for Java, but a version of Java based on that opened codebase has yet to arrive. Maybe open source Java really is important after all. I guess we’ll have to wait for OS 10.6 and Java 7 to find out.